JP2004280697A - Image processor and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor and program capable of efficiently detecting only a necessary change. <P>SOLUTION: This image processor 10 comprises means 40 and 50 for dividing first image data and second image data into a plurality of areas with the same dividing pattern, and tabulating pixel values related to the color or luminance of pixels belonging to each area, with respect to the first image data and second image data, to compute an area pixel value; a threshold storage means 80 for storing a threshold used for differential determination of the area pixel value of each area in association with each area divided by the dividing pattern; and a means 60 for performing an image change detection processing. The means 60 computes the difference between the area pixel values of the corresponding areas of the first image data and second image data, compares the difference with the threshold stored in the threshold storage means in association with the area to determine whether or not the difference is within the threshold range, and detects the presence of a change when it is out of the range. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image processing device and a program.
[0002]
[Prior art]
In an image processing device used in a security system, an image of a place to be monitored by a surveillance camera is captured, a change between frames is detected by image processing, and an image is saved when a change occurs at a certain level. .
[0003]
[Patent Document 1]
JP-A-5-56432
JP-A-6-176153
[0004]
[Problems to be solved by the invention]
For example, Japanese Patent Application Laid-Open No. 5-56432 discloses a method of making a judgment based on a change in the luminance of a pixel and its image area. However, according to this method, when the background is like a flower on one side, there is a drawback that the area where the luminance has changed is largely judged just because the wind is good.
[0005]
Further, for example, in Japanese Patent Laid-Open No. 6-176153, a binary image obtained by performing a certain background process is projected on the X-axis and the Y-axis to determine a difference based on a difference in luminance of a pixel and an image area thereof. An approach is disclosed. However, according to this method, in the case of an image for which no peak is obtained, a change in the image cannot be captured.
[0006]
The present invention has been made in view of the above technical problems, and an object of the present invention is to provide an image processing apparatus and a program capable of efficiently detecting only necessary changes. .
[0007]
[Means for Solving the Problems]
(1) The present invention relates to an image processing apparatus,
The first image data and the second image data are divided into a plurality of areas by the same division pattern, and the pixel values relating to the color or luminance of the pixels belonging to each area are determined for the first image data and the second image data. Means for counting and calculating an area pixel value;
Means for calculating the difference between the area pixel values of the corresponding areas of the first image data and the second image data;
Threshold value storage means for storing a threshold value used for determining a difference between area pixel values of each area in association with each area divided by the division pattern;
The difference is compared with a threshold value stored in a threshold value storage unit in association with the area, and it is determined whether or not the difference is within a threshold value range. Means for performing image change detection processing for detecting presence;
It is characterized by including.
[0008]
The pixel value relating to the color or luminance of a pixel belonging to each area is, for example, color information or luminance information of the value of each pixel of an image (digital image or digitally converted image) from a camera. For example, when the image data is in the YUV format, it may be a value of either the YUV value or the YUV component, or may be a value derived from these values by calculation. For example, when the image data is in the RGB format, it may be the RGB value or the value of any one of the RGB components, or a value derived from these values by calculation.
[0009]
Further, for example, when luminance information and color difference information come as information on each pixel, those values may be used, or values derived by calculation from these values may be used.
[0010]
The area pixel value obtained by summing up the pixel values may be a total value of the pixel values, an average value of the pixel values, or a value obtained by summing up the pixel values and obtaining a value from the value by a given function. .
[0011]
According to the present invention, since the comparison is made based on the area pixel value not in the pixel unit but in the area unit, for example, a change such as a subtle fluctuation caused by wind or the like is absorbed (without detecting such a minute change), and the true state is detected. (For example, a change caused by invasion of a person or an object) can be efficiently detected.
[0012]
In addition, since the comparison is performed using the area pixel value in units of area, the calculation load can be reduced.
[0013]
(2) The image processing device of the present invention
The image processing apparatus further includes means for changing a threshold value used in the image change detection processing.
[0014]
For example, the threshold value used in the image change detection process may be changed by changing the threshold value stored in the threshold value storage means based on an external input.
[0015]
Further, for example, a plurality of threshold tables are provided, and a threshold table used in the image change detection process is changed by switching the threshold table to be used based on an external input. It may be.
[0016]
According to the present invention, an optimum threshold value can be set according to the content of an image.
[0017]
The input information may be, for example, an external input. In this case, an optimal threshold value can be set according to the content of the image by the user inputting from the outside.
[0018]
The input information may be, for example, input information generated internally by the system. In this case, the threshold value can be automatically changed by the system detecting a predetermined condition.
[0019]
(3) The present invention relates to an image processing device,
The first image data and the second image data are divided into a plurality of areas by the same division pattern, and the pixel values relating to the color or luminance of the pixels belonging to each area are determined for the first image data and the second image data. Means for counting and calculating an area pixel value;
Means for calculating the difference between the area pixel values of the corresponding areas of the first image data and the second image data;
Means for performing image change detection processing for determining the presence or absence of an image change based on the difference between the area pixel values;
It is characterized by including.
[0020]
The pixel value relating to the color or luminance of a pixel belonging to each area is, for example, color information or luminance information of the value of each pixel of an image (digital image or digitally converted image) from a camera. For example, when the image data is in the YUV format, it may be a value of either the YUV value or the YUV component, or may be a value derived from these values by calculation. Further, for example, when the image data is in the RGB format, it may be an RGB value or a value of one of the RGB components, or a value derived from these values by calculation.
[0021]
Further, for example, when luminance information and color difference information come as information on each pixel, those values may be used, or values derived by calculation from these values may be used.
[0022]
The area pixel value obtained by summing up the pixel values may be a total value of the pixel values, an average value of the pixel values, or a value obtained by summing up the pixel values and obtaining a value from the value by a given function. .
[0023]
According to the present invention, since the comparison is made based on the area pixel value in the area unit instead of the pixel unit, for example, a change such as a subtle fluctuation caused by wind or the like is absorbed (without detecting such a minute change), and the true state is detected. (For example, a change caused by invasion of a person or an object) can be efficiently detected.
[0024]
In addition, since the comparison is performed using the area pixel value in units of area, the calculation load can be reduced.
[0025]
(4) The image processing apparatus of the present invention
The image processing apparatus further includes means for changing a division pattern for dividing the first image data and the second image data into a plurality of areas.
[0026]
For example, a plurality of division patterns are prepared, and a division pattern for dividing the first image data and the second image data into a plurality of areas is changed by switching a division pattern to be used based on input information. It may be.
[0027]
According to the present invention, an optimal division pattern can be set according to the content of an image.
[0028]
The input information may be, for example, an external input. In such a case, the user can input an external input to set an optimal division pattern according to the content of the image.
[0029]
The input information may be, for example, input information generated internally by the system. In this case, the system can automatically change the division pattern by detecting a predetermined condition.
[0030]
For example, including means for changing the division pattern according to the change detection situation,
When a change is detected, the division pattern may be changed to a division pattern having a higher division number. In this manner, it is possible to perform control so that detection with a low calculation load (low detection accuracy) is performed in normal times, and detection with a high calculation load (high detection accuracy) is performed only when there is a suspected abnormality.
[0031]
In addition, for example, in accordance with a change detection state, a means for changing a division pattern is included,
When a change is detected, the division pattern may be changed to a division pattern having a lower division number. In this way, once a change is detected and the image is stored, it is possible to perform control so as to detect the change again and save the image when there is a larger change thereafter. For this reason, it is possible to prevent a situation where many similar images are stored at the time of change.
[0032]
(5) The image processing apparatus of the present invention
The image change detection processing further includes means for specifying an area to be used as a determination target when determining the presence or absence of a change,
It is characterized in that the presence or absence of an image change is determined based on the difference between the area pixel values of the designated area.
[0033]
(6) The image processing apparatus of the present invention
It is characterized in that the area pixel value is calculated by summing up the luminance values or luminance components of the pixels belonging to each area.
[0034]
By using the luminance value or luminance component of a pixel, a change can be detected with high accuracy.
[0035]
(7) The image processing apparatus of the present invention
The image processing apparatus further includes image capturing means for periodically capturing an image,
It is characterized in that reference image data is used as the first image data, and image data periodically taken in is used as the second image data.
[0036]
By doing so, it is possible to detect a change in the case where the acquired image (the image captured periodically) changes with respect to the reference image. Therefore, it is effective when detecting a change in an image in a place where the brightness or the like does not change with time, such as in a room using artificial lighting.
[0037]
(8) The image processing apparatus of the present invention
The image processing apparatus further includes image capturing means for periodically capturing an image,
It is characterized in that image data periodically taken in is used as the first image data and the second image data.
[0038]
Since the first image and the second image are both temporally changed in this way, even when detecting a change in an image in a place that temporally changes, such as natural light, a change in the surrounding brightness is detected. And the image change can be detected with high accuracy.
[0039]
(9) The image processing apparatus of the present invention
The image processing apparatus further includes means for picking up an image to be subjected to change detection at a specified interval with respect to an image captured periodically.
[0040]
The specification of the pick-up interval may be performed by, for example, the number of frames (how many frames should be picked up once) or by the time interval. Also, the comparison images may be selected by storing only the picked-up images in the comparison buffer, or all images may be stored in the comparison buffer, but some images may not be used for the comparison process. You may make it. For example, when control is performed so that the comparison process is performed at a predetermined time interval, an image acquired at an interval shorter than the predetermined time interval may be overwritten with a next acquired image without being used for the comparison process. become.
[0041]
If the difference is obtained for each frame and the comparison is performed, the calculation load increases despite the small change. According to the present invention, a change can be detected efficiently by setting an interval at which a change is not overlooked.
[0042]
(10) The image processing apparatus of the present invention
When the first image and the second image are compared by the image change detection processing and it is determined that there is a change, means for storing the latter image in time-series storage means for storage is included.
[0043]
(11) The image processing apparatus of the present invention
When the first image and the second image are compared with each other in the image change detecting process and it is determined that there is a change, a means for printing and outputting the latter image from a printing unit in a time series is included.
[0044]
(12) The present invention is a computer-executable program,
The first image data and the second image data are divided into a plurality of areas by the same division pattern, and the pixel values relating to the color or luminance of the pixels belonging to each area are determined for the first image data and the second image data. Means for counting and calculating an area pixel value;
Means for calculating the difference between the area pixel values of the corresponding areas of the first image data and the second image data;
Threshold value storage means for storing a threshold value used for determining a difference between area pixel values of each area in association with each area divided by the division pattern;
The difference is compared with a threshold value stored in a threshold value storage unit in association with the area, and it is determined whether or not the difference is within a threshold value range. Means for performing image change detection processing for detecting presence;
It is realized by a computer.
[0045]
(13) The program of the present invention
Means for changing the value of the threshold used for the image change detection process,
It is realized by a computer.
[0046]
(14) The present invention is a computer-executable program,
The first image data and the second image data are divided into a plurality of areas by the same division pattern, and the pixel values relating to the color or luminance of the pixels belonging to each area are determined for the first image data and the second image data. Means for counting and calculating an area pixel value;
Means for calculating the difference between the area pixel values of the corresponding areas of the first image data and the second image data;
Means for performing image change detection processing for determining the presence or absence of an image change based on the difference between the area pixel values;
It is realized by a computer.
[0047]
(15) The program of the present invention comprises:
A computer is provided with means for changing a division pattern for dividing the first image data and the second image data into a plurality of areas.
[0048]
(16) The program of the present invention comprises:
The image change detection process further includes a program that causes a computer to implement means for designating an area to be used as a determination target when determining whether there is a change,
It is characterized in that the presence or absence of an image change is determined based on the difference between the area pixel values of the designated area.
[0049]
(17) The program of the present invention comprises:
It is characterized in that the area pixel value is calculated by summing up the luminance values or luminance components of the pixels belonging to each area.
[0050]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. The embodiments described below do not unduly limit the contents of the invention described in the claims. In addition, all of the configurations described below are not necessarily essential components of the invention.
[0051]
FIG. 1 shows a block diagram of an image processing system according to the present embodiment.
[0052]
The image processing system 10 includes a first image data storage unit 20 in which first image data is stored. The function of the first storage unit 20 is realized by a volatile memory such as a DRAM (Dynamic Random Access Memory), a nonvolatile memory such as an SRAM (Static Random Access Memory), a flash memory, or a ROM (Read Only Memory). .
[0053]
The image processing system 10 includes a second image data storage unit 30 in which second image data is stored. The function of the second storage unit 30 is realized by a volatile memory such as a DRAM (Dynamic Random Access Memory), a nonvolatile memory such as an SRAM (Static Random Access Memory), a flash memory, or a ROM (Read Only Memory). .
[0054]
The image processing system 10 includes a first image data area luminance (or color) calculation unit 40. The first image data area luminance (or color) calculation unit 40 divides the first image data into a plurality of areas according to the same division pattern as that of the second image data. Then, pixel values relating to the luminance (or color) of the pixels belonging to each area are totaled to calculate the area luminance (or color) value. Here, the area luminance (or color) value may be a total value or an average value of pixel values relating to luminance (or color) of pixels belonging to each area.
[0055]
The image processing system 10 includes a second image data area luminance (or color) calculation unit 50. The second image data area luminance (or color) calculation unit 50 divides the second image data into a plurality of areas according to the same division pattern as that of the first image data. Then, pixel values relating to the luminance (or color) of the pixels belonging to each area are totaled to calculate the area luminance (or color) value. Here, the area luminance (or color) value may be a total value or an average value of pixel values relating to luminance (or color) of pixels belonging to each area.
[0056]
The image processing system 10 includes a division pattern setting unit 70. The division pattern setting unit 70 sets information relating to area division, which is a unit for calculating area luminance (or color) of the first image data and the second image data. For example, only one type of division pattern may be set, or a plurality of division patterns may be set to be switchable.
[0057]
The image processing system 10 includes an area threshold information setting unit 80. The area threshold information storage unit 80 sets a threshold value used for determining a difference between area pixel values of each area in association with each area divided by the division pattern.
[0058]
The image processing system 10 includes an image change detection unit 60. The image change detection unit 60 calculates the difference between the first image data and the area luminance (or color) value of the second image data for each area, and calculates the first image data and the second image data for each area. By comparing the difference of the area luminance (or color) value of the image data with the threshold value corresponding to the area, it is determined whether or not the difference is within the range of the threshold value. And a change detection signal 62 is output.
[0059]
Here, as the threshold, a corresponding threshold may be read from the threshold information setting unit 80 and used.
[0060]
The functions of the first image data area luminance (or color) calculation unit 40, the second image data area luminance (or color) calculation unit 50, and the image change detection unit 60 are performed by an application-specific integrated circuit (IC) (Application Specific Integrated Circuit: It is realized by controlling hardware such as an ASIC) or a CPU (Central Processing Unit) with firmware or software.
[0061]
The functions of the division pattern storage unit 70 and the area threshold information storage unit 80 are as follows: volatile memory such as DRAM (Dynamic Random Access Memory), SRAM (Static Random Access Memory), flash memory or ROM (Read Only Memory). Or a hardware such as an application specific integrated circuit (ASIC) or a CPU (Central Processing Unit) such as a CPU or a Central Processing Unit (IC). May be done.
[0062]
FIG. 2 is a diagram for describing the area luminance according to the present embodiment.
[0063]
110 is the first image stored in the first image data storage unit, and 210 is the second image stored in the second image data storage unit. In the present embodiment, the first image 110 and the second image are divided into a plurality of areas by the same division pattern.
[0064]
A given area 120 of the first image is composed of, for example, m1 × m2 = n pixels P1, P2,... Pn, and pixel values relating to the luminance (or color) of each pixel P1, P2,. Are a1, a2,... An. In this case, assuming that the area pixel value of a given area of the first image is A1, A1 can be represented by the following equation, for example.
[0065]
A1 = (a1 + a2 +... + An) / n
Similarly, a given area 220 of the second image is composed of, for example, m1 × m2 = n pixels P1, P2,... Pn, and the luminance (or color) of each pixel P1, P2,. It is assumed that the pixel values of b1, b2,. In this case, assuming that the area pixel value of a given area of the second image is B1, B1 can be represented by the following equation, for example.
[0066]
B1 = (b1 + b2 +... + Bn) / n
In FIG. 2, the area is divided into a plurality of rectangular areas having the same size, but the invention is not limited to this. For example, it may be divided into rectangular areas of different sizes, or may be divided into polygons other than rectangles.
[0067]
FIG. 3 is a flowchart for explaining an example of a process of detecting a change in an image according to the present embodiment.
[0068]
The first image data and the second image data are divided into a plurality of areas by the same division pattern, and for each area of the first image data and the second image data, the color or luminance of a pixel belonging to each area Pixel luminance values are calculated by calculating the area luminance (or color) (step S10).
[0069]
Next, a variable i is initialized (i = 1) (step S20).
[0070]
Next, the variable i is incremented (i = i + 1) (step S30).
Next, a difference between the area luminance values of the i-th area of the first image data and the second image data is calculated (step S40).
[0071]
Next, the threshold information of the i-th area is read and compared with the difference between the area luminance values of the i-th area (step S50).
[0072]
Then, the difference is compared with the corresponding threshold value to determine whether the difference falls within the range of the threshold value. If not, it is detected that there is a change (steps S60, S70).
[0073]
If the difference is within the range of the threshold value, it is determined whether or not the comparison process has been completed for all the areas (steps S60 and S80).
[0074]
If the comparison processing has not been completed for all the areas, the processing of steps S30 to S80 is repeated again (step S80, steps S30 to S80). If the comparison processing has been completed for all the areas, there is no change. A judgment is made (steps S80, S90).
[0075]
FIG. 4 is a diagram for explaining an example of a circuit configuration when the image processing system of the present embodiment is realized by dedicated hardware.
[0076]
The image processing system according to the present embodiment includes a control circuit 310, an average luminance calculation circuit 320, a difference detection circuit 330, a threshold value storage register 340, an odd frame area luminance storage register 350, an even frame area luminance storage register 360, It includes a first selector 370, a second selector 380, a third selector 390, and the like.
[0077]
The threshold value storage register 340 stores the threshold value of each area in association with each area.
[0078]
In the odd frame area luminance storage register 350, a total value of each area of the odd frame image data is stored in association with each area.
[0079]
The even-number frame area luminance storage register 360 stores the total value of each area of the image data of the even-numbered frame in association with each area.
[0080]
The control circuit 310 outputs various control signals 312, 314, 316 to the average luminance calculation circuit 320, the first selector 370, the second selector 380, the third selector 390, and the difference detection circuit 330. Control. Control circuit 310 includes area size register 311. The area size register 312 stores information on area division patterns (for example, information on area size, number of areas, area coordinates, and the like).
[0081]
The average luminance calculation circuit 320 counts the input pixel values (the pixel values of the first image data or the second image data) based on the luminance calculation control signal 312 of the control circuit 310 to calculate the area luminance value to which the pixel belongs. Value (stored in the area of the corresponding area of the odd-numbered frame area brightness storage register or the even-numbered frame area brightness storage register), and the summed value after the addition is added to the odd-numbered frame area brightness storage register or the even-numbered frame. The processing for storing the data in the corresponding area of the area brightness storage register is performed.
[0082]
The first selector 370 reads the threshold stored in the threshold storage register 340 based on the read control signal 314 from the control circuit 310, and outputs the read threshold 372 to the difference detection circuit 330. I do.
[0083]
The second selector 380 reads the total area luminance value stored in the odd-numbered area luminance storage register 350 based on the read control signal 314 from the control circuit 310, and uses the read area luminance total value 382 as a difference detection circuit. Output to 330 or average luminance calculation circuit 320.
[0084]
The third selector 390 reads the total area luminance value stored in the area luminance storage register 360 for even-numbered frames based on the read control signal 314 from the control circuit 310, and converts the read total area luminance value 392 into a difference detection circuit. Output to 330 or average luminance calculation circuit 320.
[0085]
Based on the read control signal 316 from the control circuit 310, the difference detection circuit 330 calculates the area brightness total value stored in the odd frame area brightness storage register 350 and the area stored in the even frame area brightness storage register 360. The luminance sum value and the threshold value stored in the threshold value storage register 340 are input and stored in the area luminance sum value and the even frame area luminance storage register 360 stored in the odd frame area luminance storage register 350. Is calculated, and the difference is compared with the input threshold value. If the difference is not within the range of the threshold value (difference> threshold or difference ≧ threshold) Value), a change detection signal is output.
[0086]
FIG. 5 and FIG. 6 are flowcharts for explaining an example of processing when the image processing system of the present embodiment is realized by dedicated hardware.
[0087]
First, the initial setting process will be described with reference to FIG.
[0088]
The shape and size of an area for specifying the division pattern are set (step S110). For example, information on the shape and size of the area is set in the area size register. For example, the setting may be made from the outside or the system may make the setting.
[0089]
Next, the odd-numbered frame area luminance register and the even-numbered frame area luminance register are cleared (step S120).
[0090]
Next, the threshold of each area is stored in the threshold storage register (step S110). For example, the setting may be made from the outside or the system may make the setting.
[0091]
Next, the change detection processing will be described with reference to FIG.
[0092]
First, the initial setting process described with reference to FIG. 5 is performed (step S210).
[0093]
Next, the area luminance value of each area of the first frame is calculated and stored in the odd-numbered frame area luminance setting register (step S220).
[0094]
Next, the area brightness value of each area of the second frame is calculated and stored in the even frame area brightness setting register (step S230).
[0095]
Next, 2 is set to the variable k (step S230).
[0096]
Next, the luminance value of each area of the odd-numbered frame area luminance value storage register and the even-numbered frame area luminance value storage register are sequentially compared. A detection signal is generated and output (step S250).
[0097]
When the change detection signal is output, it is determined that there is a change (steps S260 and S270).
[0098]
If no change detection signal is output, it is determined whether the comparison process has been completed for all areas (steps S260 and S280).
[0099]
When the comparison process has been completed for all the areas, it is determined that there is no change (steps S280 and S290).
[0100]
If the comparison process has not been completed for all the areas, the variable k is incremented (steps S280 and S300).
[0101]
Next, it is determined whether k is an odd number or an even number. If k is an odd number, it is stored in an odd frame area luminance value storage register. If k is even, it is stored in an even frame area luminance value storage register. The process is repeated (steps S310, S320, S330).
[0102]
FIGS. 7A to 7C are diagrams for explaining an area division pattern of an image.
[0103]
FIG. 7A is a first division pattern for dividing an image into 14 × 11 rectangular (small) areas, and FIG. 7B is a second division pattern for dividing an image into 7 × 5 rectangular (medium) areas. FIG. 7C shows a third division pattern for dividing an image into 4 × 3 rectangular (large) areas.
[0104]
In the present embodiment, the division pattern may be set and changed by an external input.
[0105]
When detecting whether or not an image has changed by detecting a change in area luminance, it can be said that the change can be detected more accurately by dividing the image into more detailed areas. That is, a change can be detected with high accuracy in the order of the first divided pattern> the second divided pattern> the third divided pattern.
[0106]
However, if the area is divided into more detailed areas, the amount of computation increases when implemented by software, and the circuit scale increases when implemented by hardware.
[0107]
Therefore, it is preferable to set an optimal division pattern according to the content of the image to be detected.
[0108]
Further, for example, a plurality of division patterns may be prepared so that the division patterns can be switched. For example, the switching may be performed by an external setting, or the system may automatically switch.
[0109]
For example, including means for changing the division pattern according to the change detection situation,
When a change is detected, the division pattern may be changed to a division pattern having a higher division number (for example, a third division pattern → a second division pattern → a first division pattern), or vice versa.
[0110]
FIGS. 8A and 8B are diagrams for describing designation of an area to be determined whether there is a change. A hatched portion 410 in FIG. 8A represents an image change detection area. That is, the image 400 is divided into 36 rectangular areas of 6 × 6, and the change detection target based on the comparison of the area pixel values is the 12 areas belonging to the change detection area 410.
[0111]
For example, in the case of a surveillance camera fixedly set to monitor the vicinity of an entrance, the same scene is always photographed, especially when no people or objects enter or exit. In addition, an area where a change occurs due to the entry or exit of a person or an object is limited to some extent. Therefore, change detection may be performed only in an area necessary for detecting a change in an image.
[0112]
In this way, a change can be detected with a smaller processing load as compared with a case where all areas are subjected to the change detection processing.
[0113]
FIG. 8B is an example of a change detection area in the case of an image different from FIG. 8A. An image 420 in FIG. 8B is an image of a monitoring camera monitoring the vicinity of the sliding door. In the case of the same image, it is efficient to detect a change in a portion where the sliding door opens. Therefore, as shown in FIG. 8 (B), it is preferable to set an opening portion (shaded area) of the sliding door as a change detection area.
[0114]
The change setting area may be set by an external input, and the change detection may be performed on the set area by comparing a difference between area luminance values and a threshold value.
[0115]
FIG. 9 shows a functional block diagram of an image processing apparatus 500 configured to use the image data of the reference image as the first image data.
[0116]
The image processing device 500 includes an image capture unit 522, a first storage unit 524, an image change detection unit 526, a second storage unit 528, and a transmission unit 530.
[0117]
The image capturing unit 522 periodically captures an image. The function of the image capturing unit 522 is realized by a CCD camera or the like.
[0118]
The first storage image data unit 524 stores reference image data that is image data of the reference image. The function of the first storage unit 524 is realized by a volatile memory such as a DRAM (Dynamic Random Access Memory), a nonvolatile memory such as an SRAM (Static Random Access Memory), a flash memory, or a ROM (Read Only Memory). .
[0119]
Note that as the image data of the reference image, an image captured from the image capturing unit 522 at the time of initial setting may be stored in the first image data storage unit 524.
[0120]
The image change detection unit 526 causes the second image data storage unit 528, which is a temporary storage memory (work memory), to store the acquired image data, which is the image data of the acquired image acquired by the image acquisition unit 522, Then, the second image data (acquired image data) stored in the image data storage unit 528 is compared with the first image data (reference image data) stored in the first image data storage unit 524.
[0121]
Then, the image change detection unit 526 detects whether or not the acquired image (second image) has changed based on the reference image (first image) based on the comparison result. When detecting that the acquired image has changed, the image change detection unit 526 outputs (changes) an image change detection signal 530. The image change detection unit 526 does not output (does not change) the image change detection signal 530 when detecting that the acquired image has changed. The function of the image change detection unit 526 is realized by controlling hardware such as an application specific integrated circuit (ASIC) and a CPU (Central Processing Unit) by firmware or software.
[0122]
The function of the second image data storage unit 528 is realized by a volatile memory such as a DRAM or a nonvolatile memory such as an SRAM or a flash memory.
[0123]
The storage image data storage unit 532 stores the obtained image data on condition that the image change detection unit 526 detects that the obtained image has changed. The function of the storage image data storage unit 532 is realized by a volatile memory such as a DRAM or a nonvolatile memory such as an SRAM or a flash memory.
[0124]
The image stored in the storage image data storage unit 532 may be automatically output (printed out) from a printing unit (for example, a printer) at a predetermined timing.
[0125]
Such an image processing device 500 has a CPU (not shown) and a memory (not shown). In the image processing apparatus 20500, the following control is performed by the CPU that has performed the processing according to the program stored in the memory.
[0126]
FIG. 10 shows an example of an operation flow of the image processing apparatus of FIG.
[0127]
In the image processing apparatus 500, first, the reference image is captured by the image capturing unit 522, and is stored in the first image data storage unit (step S410).
[0128]
Thereafter, in the image processing device 500, the image capturing unit 522 periodically captures the image.
[0129]
First, an image (acquired image) that is periodically photographed is captured by the image capturing unit 522 and stored in the second image data storage unit (step S420).
[0130]
Next, a change detection process is performed on the acquired image (the second image stored in the second image data storage unit) and the reference image (the first image stored in the first image data storage unit) (step S430). ). The change detection process can be performed by the method described with reference to FIGS.
[0131]
If there is a change, a change detection signal is output, and the obtained image is stored in the storage image data storage unit (steps S440, S450, S460).
[0132]
When the process is not completed (step S470), the process returns to step S420, and the processes of steps S420 to S470 are repeated again.
[0133]
By doing so, it is possible to detect a change in the case where the acquired image (the image captured periodically) changes with respect to the reference image. Therefore, it is effective when detecting a change in an image in a place where the brightness or the like does not change with time, such as in a room using artificial lighting.
[0134]
The reference image data stored in the first storage unit may be updated after a given period. For example, the first image and the second image may be time-series images that are periodically captured.
[0135]
FIG. 11 shows a functional block diagram of an image processing apparatus having a configuration in which both the first image and the second image are time-series images that are periodically captured. Here, the same portions as those of the image processing apparatus 500 shown in FIG. 9 are denoted by the same reference numerals, and the description will be appropriately omitted.
[0136]
The image processing apparatus 600 differs from the image processing apparatus 500 shown in FIG. 9 in that acquired image data is stored in the first image data storage unit 524 at a given cycle. Here, the image processing device 600 includes an image change detection unit 552 that detects a change in the acquired image, similarly to the image change detection unit 526 illustrated in FIG.
[0137]
In the image processing apparatus 600, the acquired images periodically acquired may be alternately stored in the first image data storage unit 524 and the second image data storage unit 528. The acquired image data captured by the image capturing unit 522 is stored in the first storage unit 524 as reference image data, on condition that the image change detecting unit 552 detects that the acquired image has not changed. You may make it so. The acquired image data may be stored in the first storage unit 524 every time an image is captured, or the acquired image data may be stored in the first storage unit 524 after a given period. It may be.
[0138]
Then, the image change detection unit 552 detects a change by performing a change detection process on the first image stored in the first image data storage unit and the second image stored in the second image data storage unit. In this case, an image change detection signal 530 is output.
[0139]
When a change is detected, the image after the change (the image captured later in time) stored in the first image data storage unit 524 or the second image data storage unit 528 is stored in the storage image data storage. Stored in the unit 532.
[0140]
The image stored in the storage image data storage unit 532 may be automatically output (printed out) from a printing unit (for example, a printer) at a predetermined timing.
[0141]
Since the first image and the second image are both temporally changed in this way, even when detecting a change in an image in a place that temporally changes, such as natural light, a change in the surrounding brightness is detected. And the image change can be detected with high accuracy.
[0142]
Next, luminance (or color) information of a pixel used in the present embodiment will be described.
[0143]
When YUV data is used as the pixel information, for example, the Y component data of each pixel can be used as the pixel luminance information.
[0144]
Here, a description will be given assuming that the image data acquired from the image capturing unit is in the RGB format, but the present invention is not limited to this. In image data in the RGB format, each pixel is represented by an RGB primary color signal.
[0145]
The acquired image data in the RGB format is converted into image data including a Y component (luminance component), a Cb component, and a Cr component. Then, the image change detection unit performs a change detection process by calculating an area luminance value using the Y component of the acquired image data and the Y component of the reference image data.
[0146]
In this way, by extracting only the luminance component and performing the change detection processing based on the luminance component, it is possible to reduce the data amount and the processing load.
[0147]
Next, an example of a hardware configuration that can realize the present embodiment will be described with reference to FIG.
[0148]
The image processing apparatus according to the present embodiment includes a CPU 910, a DMAC 920, a camera module 930, a memory (frame memory or the like) 940, an image change detection unit 950, an interrupt controller (INTC) 960, and peripheral modules (peripheral modules required as a system). 970, etc., which are connected by a bus 980.
[0149]
Here, the image change detection unit 950 may be realized by software by a CPU executing a program stored in a storage medium (not shown), or may be realized by hardware by a dedicated circuit. Good.
[0150]
Image data of an image periodically captured by the camera module is stored in a frame memory 940 using a bus 980 by a DMAC (Direct Memory Access Controller) 920 or the like.
[0151]
The image change detection unit 950 may read out the image stored in the frame memory 940 by a different path (942) from the bus 980. In this way, bus traffic can be reduced.
[0152]
When a change in the image is detected, an interrupt signal 952 for detecting the difference is output to an interrupt controller (INTC) 960. May be notified.
[0153]
Each unit may be executed only by hardware, or may be executed only by a program stored in an information storage medium or a program distributed through a communication interface. Alternatively, it may be executed by both hardware and a program.
[0154]
Next, a method of setting a threshold used for detecting a change in the image of the monitoring camera installed at the entrance will be described as an example.
[0155]
FIGS. 14A to 14C are diagrams for explaining the division pattern and the area luminance value of the image of the monitoring camera installed at the entrance.
[0156]
FIG. 14A shows a state in which the image of the surveillance camera installed at the entrance is divided into a plurality of areas (areas composed of 8 × 8 pixels) in a predetermined division pattern.
[0157]
Here, reference numeral 710 is an area at the upper right end of the image, and 720 is an area of the Kamoi portion.
[0158]
FIG. 14B is a table showing the luminance values of the pixels belonging to the upper right end area (area composed of 8 × 8 pixels). The average luminance value (area luminance value) of the pixels belonging to the area is 138.86.
[0159]
FIG. 14C is a table showing the luminance value of each pixel belonging to the area of the Kamoi part (area composed of 8 × 8 pixels). The average luminance value (area luminance value) of the pixels belonging to the area is 193.75.
[0160]
In general, the area at the upper right end and the area of the Kamoi section are areas not to be detected. The reason is that the two areas are not a part deviating from the passage route of the intruder or a route passing in the early stage of the intrusion.
[0161]
Therefore, for such an area, by setting a large threshold value (for example, about 100 in this case), it is possible to prevent the detection of a change in the luminance difference of the area luminance value of this area.
[0162]
FIGS. 15A and 15B and FIGS. 16A and 16B are diagrams for explaining an example of threshold setting of the detection target area.
[0163]
Reference numeral 730 in FIG. 15A and 740 in FIG. 16A are areas at the same position, which are a portion of the entrance door that is closed and opened, and a portion that corresponds to an intrusion route.
[0164]
The image in FIG. 15A is an image when a cat appears in a portion where the sliding door is open, and FIG. 5 is a table showing luminance values of pixels. The average luminance value (area luminance value) of the pixels belonging to the area is 231.94.
[0165]
The image in FIG. 16A is an image when there is no cat in the part where the sliding door is open, and FIG. 16B belongs to the area 740 (area composed of 8 × 8 pixels) at this time. It is a table showing the luminance value of each pixel. The average luminance value (area luminance value) of the pixels belonging to the area is 177.69.
[0166]
The difference between the average luminance value (area luminance value) of the area when the image changes in this way is 54.25. Therefore, by setting about 25, which is about half of this value, as the threshold value of the area, it is possible to detect a change somewhat more sensitively.
[0167]
FIGS. 17A and 17B and FIGS. 18A and 18B are diagrams for explaining an example of change detection.
[0168]
Reference numeral 750 in FIG. 17A and 760 in FIG. 18A are areas at the same position, which are a part of the entrance door that is closed and opened, and also a part corresponding to an intrusion route.
[0169]
The image in FIG. 17A is an image when the sliding door is closed, and FIG. 17B is a diagram illustrating the luminance value of each pixel belonging to the area 750 (area composed of 8 × 8 pixels) at this time. FIG. The average luminance value (area luminance value) of the pixels belonging to the area is 110.
[0170]
The image of FIG. 18A is an image when the sliding door is slightly opened, and FIG. 18B is the image of each pixel belonging to the area 760 (area composed of 8 × 8 pixels) at this time. It is a table showing values. The average luminance value (area luminance value) of the pixels belonging to the area is 157.
[0171]
In such a case, for example, since there is a 47 difference in area luminance between the areas, if the threshold value is set to 25, a change is detected.
[0172]
Note that the present invention is not limited to the above-described embodiment, and various modifications can be made within the scope of the present invention.
[0173]
Further, in the invention according to the dependent claims of the present invention, a configuration in which some of the constituent elements of the dependent claims are omitted may be adopted. In addition, a main part of the invention according to one independent claim of the present invention can be made dependent on another independent claim.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating an image processing system according to an embodiment.
FIG. 2 is a diagram for describing area luminance according to the present embodiment.
FIG. 3 is a flowchart illustrating an example of a process of detecting a change in an image according to the embodiment;
FIG. 4 is a diagram illustrating an example of a circuit configuration when the image processing system according to the present embodiment is realized by dedicated hardware;
FIG. 5 is a flowchart illustrating an example of a process when the image processing system according to the present embodiment is realized by dedicated hardware;
FIG. 6 is a flowchart illustrating an example of a process when the image processing system according to the present embodiment is realized by dedicated hardware;
FIGS. 7A to 7C are diagrams for explaining an area division pattern of an image; FIGS.
FIGS. 8A and 8B are diagrams for explaining designation of an area to be determined whether or not there is a change; FIG.
FIG. 9 is a functional block diagram of an image processing apparatus 500 configured to use image data of a reference image as first image data.
FIG. 10 illustrates an example of an operation flow of the image processing apparatus in FIG. 9;
FIG. 11 is a functional block diagram of an image processing apparatus having a configuration in which both a first image and a second image are time-series images periodically captured.
FIG. 12 is a diagram illustrating a comparison method when RGB data is used as pixel information.
FIG. 13 is an example of a hardware configuration according to the embodiment;
FIGS. 14A to 14C are diagrams for explaining a division pattern and an area luminance value of an image of a surveillance camera installed at the entrance.
FIGS. 15A and 15B are diagrams for explaining an example of threshold setting of a detection target area.
FIGS. 16A and 16B are diagrams for explaining an example of threshold setting of a detection target area.
FIGS. 17A and 17B are diagrams for explaining an example of change detection; FIG.
FIGS. 18A and 18B are diagrams for explaining an example of change detection.
[Explanation of symbols]
Reference Signs List 10 image processing system, 20 first image data storage unit, 30 second image data storage unit, 40 first image data area luminance (or color) calculation unit, 50 second image data area luminance (or color) Calculation section, 60 image change detection section, 62 change detection signal, 70 division pattern setting section, 80 area threshold information setting section, 300 image processing system, 310 control section, 320 average luminance calculation section, 330 difference detection section, 340 Threshold register, 350 odd-frame area brightness value storage register, 360 even-frame area brightness value storage register, 370 first selector, 380 second selector, 390 third selector,

Claims (17)

An image processing device,
The first image data and the second image data are divided into a plurality of areas by the same division pattern, and the pixel values relating to the color or luminance of the pixels belonging to each area are determined for the first image data and the second image data. Means for counting and calculating an area pixel value;
Means for calculating the difference between the area pixel values of the corresponding areas of the first image data and the second image data;
Threshold value storage means for storing a threshold value used for determining a difference between area pixel values of each area in association with each area divided by the division pattern;
The difference is compared with a threshold value stored in a threshold value storage unit in association with the area, and it is determined whether or not the difference is within a threshold value range. Means for performing image change detection processing for detecting presence;
An image processing apparatus comprising: In claim 1,
An image processing apparatus comprising means for changing a threshold value used in the image change detection processing. An image processing device,
The first image data and the second image data are divided into a plurality of areas by the same division pattern, and the pixel values relating to the color or luminance of the pixels belonging to each area are determined for the first image data and the second image data. Means for counting and calculating an area pixel value;
Means for calculating the difference between the area pixel values of the corresponding areas of the first image data and the second image data;
Means for performing an image change detection process of determining the presence or absence of an image change based on a difference between area pixel values. In any one of claims 1 to 3,
An image processing apparatus comprising: means for changing a division pattern for dividing first image data and second image data into a plurality of areas. In any one of claims 1 to 4,
The image change detection processing further includes means for specifying an area to be used as a determination target when determining the presence or absence of a change,
An image processing apparatus for determining the presence or absence of an image change based on a difference between area pixel values of a designated area. In any one of claims 1 to 5,
An image processing apparatus for calculating an area pixel value by summing luminance values or luminance components of pixels belonging to each area. In any one of claims 1 to 6,
The image processing apparatus further includes image capturing means for periodically capturing an image,
An image processing apparatus, wherein reference image data is used as first image data, and image data periodically taken in is used as second image data. In any one of claims 1 to 6,
The image processing apparatus further includes image capturing means for periodically capturing an image,
An image processing apparatus using image data periodically captured as first image data and second image data. In any one of claims 7 and 8,
An image processing apparatus further comprising means for picking up an image to be subjected to change detection at a specified interval from an image captured periodically. In any one of claims 1 to 9,
An image storing means for storing the latter image in a storage means in a time series when it is determined that there is a change by comparing the first image and the second image by the image change detection processing; Processing equipment. In any one of claims 1 to 10,
An image processing apparatus, comprising: means for comparing the first image and the second image by the image change detection processing to print out the latter image from a printing unit in a time series when it is determined that there is a change. . A computer-executable program,
The first image data and the second image data are divided into a plurality of areas by the same division pattern, and the pixel values relating to the color or luminance of the pixels belonging to each area are determined for the first image data and the second image data. Means for counting and calculating an area pixel value;
Means for calculating the difference between the area pixel values of the corresponding areas of the first image data and the second image data;
Threshold value storage means for storing a threshold value used for determining a difference between area pixel values of each area in association with each area divided by the division pattern;
The difference is compared with a threshold value stored in a threshold value storage unit in association with the area, and it is determined whether or not the difference is within a threshold value range. Means for performing image change detection processing for detecting presence;
A program that is realized by a computer. In claim 12,
Means for changing the value of the threshold used for the image change detection process,
A program that is realized by a computer. A computer-executable program,
The first image data and the second image data are divided into a plurality of areas by the same division pattern, and the pixel values relating to the color or luminance of the pixels belonging to each area are determined for the first image data and the second image data. Means for counting and calculating an area pixel value;
Means for calculating the difference between the area pixel values of the corresponding areas of the first image data and the second image data;
A program for causing a computer to execute image change detection processing for determining whether or not an image has changed based on a difference between area pixel values, and a program to be realized by a computer. In any one of claims 12 to 14,
A program for causing a computer to implement means for changing a division pattern for dividing first image data and second image data into a plurality of areas. In any one of claims 12 to 15,
The image change detection process further includes a program that causes a computer to implement means for designating an area to be used as a determination target when determining whether there is a change,
A program for determining whether an image has changed based on a difference value between area pixel values of a designated area. In any one of claims 12 to 16,
A program for calculating an area pixel value by summing luminance values or luminance components of pixels belonging to each area.

Images