WO2004079681A1

WO2004079681A1 - Monitor unit

Info

Publication number: WO2004079681A1
Application number: PCT/JP2004/001969
Authority: WO
Inventors: Takeshi Hamada
Original assignee: Quality Labs. Corporation
Priority date: 2003-03-07
Filing date: 2004-02-20
Publication date: 2004-09-16
Also published as: JPWO2004079681A1

Abstract

When the energy amount of a macro-block i at time t is Pi(l, t), the energy amount of the macro-block i at time (t+T) is represented as Pi(l, t+T). When energy amounts Pi(l, t) and Pi(l, t+T) are compared with each other, a change in energy amount during time T elapsed after time t is found. The condition of a subject can be monitored from this changed condition.

Description

Specification

TECHNICAL FIELD The present invention relates to a monitoring device, and more particularly to a monitoring device having a sensor close to the function of the human eye. BACKGROUND ART In recent years, monitoring devices have been placed on factory lines to check for defective products, and have been placed in stores and dwellings, and have been used in a wide range of fields such as security applications. Here, in the conventional monitoring apparatus, an object is usually imaged using an imaging element such as a CCD, and image recognition is performed based on the obtained image signal to determine whether or not there is an abnormality. However, image recognition in a conventional monitoring apparatus based on an image signal obtained from an image sensor captures a time-series change of a subject to be monitored, for example, a past image and a current image are matched. It is to judge whether or not it is, and if it does not match, it takes measures such as issuing an alarm. However, aside from an unmanned warehouse, if an image sensor is installed in a city or the like, the past image and the current image rarely coincide completely, so it is difficult to determine whether there is an abnormality from the obtained image signal. It was difficult to judge. To cope with such a problem, Japanese Patent Application Laid-Open No. 2000-3141679 discloses that a background image is obtained by capturing a monitoring image at a plurality of timings. By registering the image and comparing it with the current image, it is possible to avoid determining that an abnormality has occurred even if, for example, the illumination of the background flickers. However, since the illumination is fixed and blinks periodically, it is easy to register it as a background image. However, in a situation such as a busy street where the subject changes every moment and the same scene is never encountered again, even if the background image can be registered using the above-mentioned technology, It is difficult to determine whether there is an abnormality. On the other hand, human eyes can overlook the crowd, for example, but can not miss it as background noise if there is no major change, but for example, when a strangely dressed person passes by, it immediately recognizes it. It has the characteristic of being noticeable. In addition, a technique is known in which a watermark such as a trademark is included in a video signal in order to prevent illegal copying. However, adding a watermark means replacing the original video signal with another signal, and the larger the amount, the more the original video may be damaged. Also, by applying a process such as repetitive compression, analog conversion, or format conversion to a video signal containing a watermark, the watermark may be damaged and may not function as an identification signal. In particular, when watermarks are added to a video signal, there is a problem that the watermark cannot be differentiated from the still image watermark, and that the watermark technology is not always effective. Disclosure of the invention The present invention has been made to solve such a conventional problem, and an object of the present invention is to provide a monitoring device having a sensor that more closely resembles the characteristics of human eyes. Another object of the present invention is to provide a monitoring device that can determine whether another image or video has been copied without damaging the original image or video signal. The monitoring device of the present invention includes a visual sensor for recognizing a subject and converting it into an energy amount;

Analyzing means for monitoring the amount of energy from the visual sensor and issuing an alarm when a temporal change in the amount of energy exceeds a threshold value. According to the monitoring device of the present invention, a visual sensor that recognizes a subject and converts the energy into an amount of energy, monitors the amount of energy from the visual sensor, and issues an alarm based on a temporal change in the amount of energy and a threshold. Since it has an analysis means for issuing a warning, a small change in the subject can be overlooked, and an alarm can be issued by capturing only a large change. Further, it is preferable that the visual sensor obtains an energy amount for each divided region (for example, macro block) obtained by dividing the subject image. The monitoring device of the present invention is a computing unit that computes an energy amount of a predetermined region in digital data corresponding to an original image, and computes an energy amount of a predetermined region in digital data corresponding to an image to be compared. When,

By comparing the two energy amounts calculated by the calculating means, it is characterized by having a judging means for judging whether or not the comparison target image is formed based on the original image. And According to the monitoring device of the present invention, by comparing the two energy amounts calculated by the calculation means, it is determined whether or not the comparison target image is formed based on the original image. Since it is characterized by having means, there is no need to put a watermark or the like into the digital data constituting the original image, and the original image is not damaged. In addition, since the characteristic of the amount of energy does not change much by processing such as compression of digital data, the correlation with the original image can be determined even if the comparative image is highly processed. I can clarify. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a schematic configuration diagram of a monitoring device according to a first embodiment. FIG. 2 is a diagram schematically showing a screen imaged by the image sensor 2. As shown in FIG. FIG. 3 is a diagram schematically showing an energy amount of each block. FIG. 4 is a diagram showing a temporal change of the energy amount. FIG. 5 is a schematic configuration diagram of a monitoring device according to the second embodiment. FIG. 6 is a diagram showing the change over time in the amount of energy. BEST MODE FOR CARRYING OUT THE INVENTION A first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a schematic configuration diagram of the monitoring device according to the first embodiment. In FIG. 1, a subject image is formed on a light receiving surface of an image sensor (CCD or the like) 2 by an optical system 1 and converted into an electric signal. The electric signal is input to the CPU 3 and analyzed. The CPU 3, which is an analysis means, drives an alarm device (monitor, speaker, etc.) 4 under a predetermined condition to issue an alarm. Note that a visual sensor is configured by the image sensor 2 and the CPU 3.

Specific analysis contents of the CPU 3 will be described. FIG. 2 is a diagram schematically showing a screen imaged by the image sensor 2. As shown in FIG. In the screen shown in FIG. 2, Mo pixels are arranged in the horizontal direction and No pixels are arranged in the vertical direction. This screen is divided into I macroblocks. As a method of dividing by macroblocks, the macroblocks may have different sizes from each other or may overlap each other. However, it is equally divided into small blocks of vertical X horizontal = n Xm pixel size. That is, when a i and b i are integers, M i = a i m, N i = b i n (i = 0, ···, I). At this time, it is necessary that the original screen is equally divided into small blocks. Here, the energy amount P of the small block SB is expressed by the following equation 1.

[Equation 1]

Here, S (x, y) is each sample value (pixel value) in the small block SB. The energy amount P i of the macro block i is an average value of the small blocks included in the macro block i. . The energy amount P0 of the screen is the average value of the small blocks included in the screen P (see Fig. 3). Next, the change with time of each energy amount P i is monitored. More specifically, the CPU 3 obtains each energy amount Pi for an image obtained from the image pickup device 2 by performing image pickup at regular time intervals, and stores it in the built-in memory. Here, for example, assuming that the energy amount of the macroblock i at the time t is P i (1, t), the energy amount of the macroblock i at the time (t + T) is P i (1, t + T) Can be expressed as Comparing the energy amount P i (1, t) with P i (1, t + T) shows the change in the energy amount when the time T has elapsed from the time t. From the state of this change, the state of the subject can be monitored. Here, it is assumed that the monitoring targets are classified into four states: "stationary", "gentle change", "severe change", and "periodic change". Time on the horizontal axis, Figure 4 shows an example of plotting the energy amount P i of the macroblock i on the vertical axis. Here, in the graph of Fig. 4, the range of ① corresponds to “stationary”, the range of ② corresponds to “gentle change”, the range of ③ corresponds to “severe change”, and the range of ④ corresponds to “period”. Change ". This means that, for example, when the present invention is applied to a monitoring device of a merchandise store, “stationary” corresponds to an unmanned state before the store is opened, and “gentle change” is a state in which the customer is doing normal shopping after the store is opened. A “violent change” corresponds to a situation in which a burglary or the like has invaded and the people in the store have rapidly moved. Illumination II and hazard lamps of vehicles correspond to the “periodic change” in ②. Therefore, the change amount of the energy amount P i corresponding to ③ in FIG. 4 is stored as a threshold value, and the CPU 3 analyzes the electric signal output from the image sensor 2 based on the electric signal. If the alarm is exceeded automatically, an unmanned surveillance system with the same function as human eyes will be constructed. Since the change of the energy amount P i is performed in a plurality of macroblocks i, even if one macroblock i outputs a change pattern of the energy amount Pi corresponding to ③, the other macroblocks i If the change pattern of the energy amount P i corresponding to ② is shown, it is possible not to issue an alarm. At this time, if the image around the cash register is included in a specific macroblock i, the energy amount Pi of that macroblock is weighted, and if it is comprehensively determined from the energy amounts Pi of other macroblocks i, A more precise alarm can be issued. The manner in which the monitoring targets are classified is described below.

① Stationary

At a certain time T, the following conditions are satisfied, so that the macroblock i It is determined that the monitoring target included in is stopped. More specifically, the change in the energy amount of the included small blocks is

△ P (l, t) = | P (l, t) — P (l, t-1) I

, I Δ P (K t) I ≤Th 1, for all t = 1, ···· τ

And

[Equation 2] ΔΡ (l, t) ≤ Th2

t = t

Is satisfied, it is determined that the monitoring target included in the macroblock i is stationary. The thresholds Th 1 and Th 2 can be determined by experiments and the like.

② gentle change

If the stationary condition is not satisfied during the fixed time T, it is determined that the monitoring target is changing gently. Here, the energy amount of macroblock i is used. More specifically, the change in the energy amount P i of the macroblock i is

△ P i (1, t) = I P i (1, t) one P i (1, t— 1) I

When asked for

(Equation 3) ΔΡ1 (l, t)> Th3

t = t

Is satisfied, it is determined that the monitoring target is changing gently. The threshold value Th3 can be determined by experiments or the like.

Rapid change

If the following conditions are satisfied at the fixed time T, the monitoring target changes drastically Judge that you are. Here, the energy of the macroblock i. Energy density dP i of macroblock i at time t

Is defined by the following equation.

[Equation 4]

1

IPiilt = Pi (l _? T) »Pi (l ₉ tl)

Td t = t-td Here, when dP i (1, t) ≥ Th d, it is determined that the monitoring target is changing drastically. The threshold value Th d can be determined by experiments or the like.

④ Periodic change

Fourier transform processing is performed on the energy amount P i of the macroblock i. If a specific frequency has a bias in the energy on the pulse, it is determined that the change is a periodic change. The present invention should not be construed as being limited to the above embodiment, and it is needless to say that the present invention can be appropriately modified and improved. For example, the monitoring device of the present invention can be used to monitor the state of a patient in a hospital in addition to a monitoring device for crime prevention, or can be mounted on a vehicle to monitor the outside traffic conditions to prevent traffic accidents. You can avoid it. A second embodiment of the present invention will be described with reference to the drawings. FIG. 5 is a schematic configuration diagram of a monitoring device according to the second embodiment. In FIG. 5, a first arithmetic unit 11 receives an original video signal via an interface (not shown). On the other hand, the second arithmetic unit 12 receives the comparison target video signal via an interface (not shown). Arithmetic units (arithmetic means) 11 and 12 extract specific blocks at the same position (the whole screen may be used), and repeatedly calculate the energy amount in them at the same time interval according to the above equation (1). After that, the arithmetic devices 11 and 12 input the obtained operation results to the determination device (determination means) 13. The determination device 13 compares the energy amount PA (i, t) of the original video signal with the energy amount PB (i, t) of the comparison target video signal. When the energy amount is plotted on the vertical axis and the time axis is plotted on the horizontal axis, the calculated energy amount is plotted with respect to time to obtain a graph as shown in FIG. 6, for example. Here, the determination device 13 calculates the difference between the energy amount PA (i, t) of the original video signal and the energy amount PB (i, t) of the video signal to be compared at each time, and calculates the difference. The sum Δ is obtained by the following equation (5).

[Equation 5]

1 τ

Δ = -TjT ∑ l PA (i, t)-P _B (i, t) | / W (PA, P _B )

1 t = l Note that W is a weighting function, and for example, W (PA, PB) = 10 g 10 (PA + PB) can be used. Here, if Δ = 0, the determination device 13 determines that the original video signal and the video signal to be compared completely match (Perfect matc), that is, determines that they are the same video signal. On the other hand, if 0 <Δ <ΤΗ (threshold), the video signal to be compared with the original video signal (eg, graph の in FIG. 6) (Eg, graph B in FIG. 6) substantially match (neighly match), and it is determined that some processing has been performed on the original video signal. Further, if TH (threshold value) <△, the comparison target video signal (eg, graph C in FIG. 6) does not match the original video signal (eg, graph A in FIG. 6) (No t mate 1Ί), It can be determined that it is unrelated to the original video signal. Since such a determination result is input from the determination device 13 to the monitor 14, the user can recognize the determination result of the determination 13 by looking at the monitor 14. According to the monitoring device of the present invention, for example, even if the video signal recorded on a DVD or the like has been processed with respect to the original video signal, the difference in the amount of energy is compared with the threshold Δ, The relevance to the original video signal can be objectively derived, thereby providing one criterion of whether or not it was illegally copied.

Claims

The scope of the claims

1. A visual sensor for recognizing a subject and converting the energy into an amount of energy, and analyzing means for monitoring the amount of energy from the visual sensor and issuing an alarm based on a temporal change of the amount of energy and a threshold value. Monitoring device.

2. The monitoring device according to claim 1, wherein the visual sensor calculates an energy amount for each divided region obtained by dividing the subject image.

3. computing means for calculating the amount of energy in a predetermined area in the digital data corresponding to the original image and calculating the amount of energy in a predetermined area in the digital data corresponding to the image to be compared;

A monitoring unit that determines whether the comparison target image is formed based on the original image by comparing the two energy amounts calculated by the calculation unit. Equipment.