JP3504245B2 - Image monitoring device - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a store, an ATM
For monitoring intruders in storage facilities, offices, etc.
The present invention relates to an image monitoring device. [0002] Stores, ATM installation facilities and offices
In fields such as monitoring intruders, image transmission
Method, monitoring by monitoring image transmission, VTR recording of monitoring image
Recording, monitoring by image processing recognition, and hands that combine them
The law has been used. The following describes these methods.
You. Surveillance by surveillance image transmission requires analog I
A TV camera is installed, and a monitoring
Analog images transmitted to the data center and monitoring deployed at the center
A worker constantly monitors the monitoring image. In this case, images from a plurality of cameras are generally
The monitor sends the images consciously or constantly.
Monitor by switching every hour. When the screen is switched
Intrusion sensors such as infrared and ultrasonic waves installed in the monitoring environment
Some use intruder detection signals from sensors. Again
It is not the same as the log image, but the pressure by Jpeg or Mpeg.
Some data is converted and then transmitted. [0004] VTR recording of surveillance images is analyzed in a surveillance environment.
Install a log ITV camera in a remote location or surveillance environment
VTR equipment installed in a place that cannot be seen from
And an image is recorded. Supervision by image processing recognition
The vision is the change of the image in the surveillance environment and the person from the image
Notification and VTR activation are performed by detecting candidate areas.
It is. [0005] The prior art as described above
Surgery has the following problems. First, the surveillance image transmission
The problem with monitoring is that while the processing mechanism is simple,
Laying and transmitting a dedicated transmission line to perform log image transmission
Installation of booster etc. to guarantee signal deterioration due to
Belt equipment is required. Therefore the cost for these facilities
Is very large and the center
Need to be kept. On the other hand, a system for transmitting compressed data of an image
System has less data than the above,
General purpose instead of analog dedicated line
Digital line can be used,
Since many pieces of information other than the
The problem of equipment is not solved. [0007] Describe the problem in VTR recording of surveillance images
And this system basically intrudes separately from emergency calls
Record the intruder's video for later situation estimation and intruder identification
But for recording at regular intervals.
Cannot actually record important video as an intrusion event
There are cases. The problem with monitoring by image processing is that
If the detection rate is reasonable, this system only
Data transmission and storage,
Product costs can be reduced overall.
You. Also, only when an event occurs,
The need for constant deployment is reduced by monitoring images
Is reduced. However, after detecting the event, the transmission or
The nature of the stored image is good for the observer to confirm
Often not. This is, for example,
There are no characteristic parts such as faces in the person area, and the image is small
Image quality is deteriorated due to compression,
You. The present invention provides an image monitoring device with excellent processing efficiency.
Aim. [0010] An image monitoring apparatus according to the present invention.
Is imaging means for capturing an image of the monitoring area, and
Processing means for performing a predetermined process on the video imaged by the step
Receiving an image on which predetermined processing has been performed by this processing means.
To obtain the image difference amount between two consecutive frames,
If the sum is below a certain value, any of the
Removal means for removing other images while leaving only the
Only the image left by the step is converted to http protocol
And transmission means for transmitting. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described with reference to the drawings.
The embodiment will be described in detail. In the following description
Uses this system to detect intruders in a surveillance area
The following describes an apparatus for monitoring the above. First, the overall configuration and processing of this embodiment will be described.
An outline will be described. Person monitoring system (image
The overall configuration of the monitoring device is shown in FIG. This system is
Person extraction means 1, intermediate server means 2, server means 3, etc.
The person extracting means 1
The candidate area is extracted, and the intermediate server
Select the image with the best appearance and in the server means 3
Generating a key for image search and http (hyperte
xt transfer protocol)
Provide the interface. And each means 1, 2, 3
Are connected by wired or wireless digital communication
You. Therefore, one or more person extracting means
1 for the person candidate area in the input image detected by
The person extracting means 1 or the intermediate server means 2
Check the features and select the frame that looks best
And camera (this corresponds to the person extraction means 1)
And the server means 3 obtains the information obtained from the intermediate server means 2.
Temporarily stores the optimal human image sequence, and then
These image sequences are transmitted. At this time, for example, the detected person
For the object series, its moving direction, color, way of movement, and reference person
Create an image search key such as similarity with the image,
Only the desired image is transmitted according to the request from the user
Processing is also performed. As described above, the person extracting means 1 and the intermediate server 2,
Transmission between the intermediate server means 2 and the server means 3
Depending on the route conditions and the number of
The processing range of each process is appropriately changed. For example
In the person extracting means 1, extraction and confirmation of a person candidate area are performed.
Yes, but if there are many events per unit time
Performs only a relatively rough
Processing, such as confirmation by features such as the face described later.
Those with the highest strikes are assigned to the intermediate server means 2. The person extracting means 1 and the intermediate server will be described below.
The configuration and processing of the stage 2 and the server means 3 will be described. Figure
FIG. 2 is a diagram showing the configuration of the person extracting means 1. Color, also
Is an image composed of a monochrome TV camera and an A / D converter.
Digit input every several frames by image input means 4
Once stored in the human detection image storage means 5
Then, as described later, the person candidate area extracting means 6
The approximate area of the intruder is detected by
In the part, characteristic region patterns (for example, face,
The presence or absence of hands, feet, etc.)
Therefore, the person candidate area image and the confirmation result are
The communication means 8 uses Ethernet 13 or IEEE 13
Intermediate server using protocol such as Tcp / IP on 94
Transmit. If the input image was color above,
Only the luminance component is extracted and sent to the person detection image storage means 5.
accumulate. In the person extracting means 1, a certain period of time
The number of extracted target person regions described above, that is, the detection frequency
Measured by the detection frequency measurement means 9 inside the object extraction means 1
If this is greater than a preset threshold,
Should not take the time to process this measure.
Therefore, the person area confirmation means 7 which generally has a processing cost
Processing is not performed, but an intermediate server described below is requested to share the processing.
I do. The following is a feature of the person extracting means 1.
As a function, a person candidate area extracting means 6 and a person area confirming means
7 and the person candidate communication means 8 will be described. Person candidate
The processing of the area extracting means 6 is as follows.
Any of several methods can be used. For example, a continuous sampling image (frame
Is calculated for each pixel, and a certain threshold
Extract the binarized area in the above, and the meaning of 4-connection or 8-connection
The area and circumscribed rectangle of the area connected spatially by
This is a method to detect a person whose size is reasonable as a person candidate.
You. The flow of this processing is shown in the flowchart of FIG. This
The binarization threshold at the time is the frequency distribution for each luminance of the difference image
To determine the value that maximizes the variance ratio
Method can be used (Otsu, discrimination and least squares criterion
Automatic threshold selection method based on IEICE, IEICE, Vol. J63
-D, No. 4, pp. 349-356, 1980
See). Here, before performing the analysis of the connection area,
Perform several expansion and contraction operations on the area
Can reduce the effects of over-extraction and "blurring"
No. In this case, the primary reliability C1 of the extraction area is set to, for example,
It is calculated as follows. C1 = (Vf, Vft) / (| Vf || Vft |) (1) where V = (W, H, S) Vt = ((Twu + Twl) / 2, (Thu + Thl) / 2, (Tsu + Tsl) / 2) (2) where W and H are circumscribed rectangles of the extraction area, respectively.
And S is the area of the extraction area. Also Tw
u and Twl are vertical sizes of the circumscribed rectangle, and Thu and Thl are circumscribed rectangles.
Tsu and Tsl are the upper limits of the extraction area.
This is a threshold indicating a value and a lower limit, and 0 ≦ Cf ≦ 1. The vectors V and Vt for calculating the reliability are
Alternatively, the following may be performed by adding the circularity R. V = (W, H, S, R), Vt = ((Twu + Twl) / 2, (Thu + Thl) / 2, (Tsu + Tsl) / 2), Tr ... (2 ') R = (perimeter of extraction region) / S, where Tr is the standard value of R as a person. As another method, a method based on posterior probability conversion is used.
(Nakai, moving object detection method using posterior probability
Law, Information Processing Research Report, 94-CV-90, pp. 1-8, 19
94). Process flow of this posterior probability conversion method
Is shown in the flowchart of FIG. Background of event θ0, event
Let θ1 be an intruder. Observe the brightness of pixels in a partial image
Prior probabilities of θ0 and θ1 before measurement are w0 and w1, respectively.
(Where w0 + w1 = 1), the posterior probability w1 ′
Is the conditional probability of pixel luminance I under event θk
When (I | θk), Bayes' theorem gives w1 ′ = {w1p (I | θ1)} / {p (I | θ0) (1-w1) + p (I | θ1) w1} (3) This w1 'penetrates something larger than a certain threshold T.
It is extracted as a pixel belonging to an object. Here,
From the frequency distribution h (x, y, I) of the image And p (I | θ1) generally has no data sample
Therefore, it is estimated that p (I | θ0) = 1 / (Imax−Imin) (5). Also, the above applies to w1 every time the input image is updated.
= W1 'and the initial value of w1 = 1.0 / 2.0
May be. In order to extract a person candidate area, an input image
For each pixel i in the image, w'i in equation (2) is obtained and w '
Pixels with i> T, connected in the sense of 8-connection or 4-connection
Of the objects, the area and circumscribed rectangle size
What is necessary is just to select what is appropriate. According to this method the background
Even if there is a steady fluctuation in the area, the fluctuation can be statistically absorbed.
There are advantages. In the above, the background (that is, the object)
The posterior probability for each pixel, but compare it with the threshold
Before the extraction (binarization) by
The side may be smoothed. This is near the object.
Assuming the continuity of all random fields,
This is to reduce wear. Also for further noise removal
For this reason, the expansion and contraction process is repeated several times after the above binarization.
You may return. Also in this case, the reliability C1 of the extraction area is
It is calculated based on the equation (1).
Here, Vt is calculated as follows, for example. V = (W, H, S, P) Vt = ((Twu + Twl) / 2, (Thu + Thl) / 2, (Tsu + Tsl) / 2, Tp) (2 ″) = (W, H, S, R, P) Vt = ((Twu + Twl) / 2, (Thu + Thl) / 2, (Tsu + Tsl) / 2), Tr, Tp) (2 "') where P is an extraction rectangle Is the normalized sum of posterior probabilities in
It is determined as follows. [0026] Where Tp is a standard value of the above-mentioned P value for a person,
Rect indicates an extracted circumscribed rectangular area. Next, in the person extracting means 1 shown in FIG.
The person area checking means 7 will be described. Person area confirmation hand
Step 7 sets the detection frequency described above in the detection frequency measurement means 9.
Inquires whether the threshold is exceeded. If detected
If the frequency is less than or equal to the threshold Th, the body of various persons
Images are collected and the information is statistically compressed
Apply pattern matching using the dictionary
Pixels inside the person candidate area extracted as
Detects whether a characteristic part exists. This state
As shown in FIG. As such a method, a certain size is used.
A plurality of normalized feature partial images are classified into a plurality of
And canonical discrimination based on it
is there. The flow of processing by this method is shown in the flowchart of FIG.
Shown in First, the following steps are taken. (I) For a plurality of classes of normalized images
Calculation of intra-class variance B and inter-class variance W (ii) (B + βSw) Φ = {(1−β) W + βI {Φ}
Where 0 <β <1, Φ: eigenvector with eigenvector Φi as the column vector
Matrix, Λ: Eigenvalue matrix having eigenvalue λi as a diagonal element. (Iii) Each vector in the cluster i is calculated as described above.
Φ, and the mean Mi, Φ, and possibly the variance
Let the variance Vi be a dictionary. (Vi) At the time of matching, the screen is divided into small areas j and
Is a vector that projects the image vector for each small area of
Euclidean distance and Mahalanobi between Le Vj and each class i
Find the distance. In the above, at the time of class creation, the desired object characteristic image
Outside, other images similar to it as a non-object class
Is used, similar regions other than the desired region are not extracted.
It becomes easier to extract only the region. For example, desired target feature image
Faces of multiple persons facing front, non-object class
To use a face similar pattern on the background. Here, the object class is defined as the face regions of a plurality of persons.
Area, and adapt them to the direction of the face.
Divided into several different classes. And non-object classes
Beforehand, manually pick the face-like pattern in the background manually.
And use them. The matching result of step (vi)
Of which the distance is small to Nth, the object class
If the sum of the distances belonging to
It is assumed that the person area has been confirmed. In this case the second
The next reliability C2 is calculated, for example, as follows. [0034] After the above processing, the above-mentioned person candidate area extracting means 6
As described above, outside the extracted candidate area
The above matching is performed, and the distance is reduced to M
Save the data up to the second as data of the non-object class.
You. Next, in the person extracting means 1 shown in FIG.
The person candidate communication means 8 will be described. As previously mentioned
The detection frequency of the person candidate is reduced by the detection frequency measurement means 9.
The result compared with the fixed interval Th is transmitted to the intermediate server.
And from the intermediate server, depending on the result,
The processing sharing instruction message is sent by the object extracting means 1.
Therefore, the processing in the person extracting means 1 follows that. (I) When the detection frequency ≧ Th, the processing by the person extracting means 1 is performed by the above-described person candidate area extracting means.
6 only. (Ii) When the detection frequency <Th, the processing by the person extracting means 1 is performed by the person candidate area extracting means 6 and
This is the processing of the person area confirmation unit 7. Here, in the case of (i), this person candidate
The output result of the communication means 8 is, for example, the above-mentioned person candidate area extraction
The extraction area (surrounding rectangle) coordinates extracted by the means 6
Outgoing time and the reliability C1 of the extraction area. In the case of (ii),
Is the output result of the person candidate communication means 8, for example,
(I) In addition to the coordinates and time of the extraction area (surrounding rectangle)
Then, the reliability C of the extraction area is calculated and transmitted as follows.
May be. C = min (C1, C2) (8) In the case of (ii), the following is obtained from the result of step (vi).
Extract and transmit information. This is the best scene camera
The information to select, the above match in the person candidate area
Of the result ranges from the smallest distance to the Nth
Matching distances and dictionaries that matched each of them
Category name, position on each screen, and candidate area
The above matching is performed outside the area, and the distance becomes smaller.
To the M-th non-object class data
Matching distances, and the words that matched each of them
The document category name and the position in each screen are also transmitted. Further, in the above-mentioned transmitted target extraction result,
In response, the image transmission message sent from the intermediate server means 2
Desired image transmission operation as described below
Do. Next, the intermediate server means 2 of FIG. 1 will be described.
FIG. 7 is a diagram showing the configuration of the intermediate server means 2. In this
The inter-server means 2 is a person extraction communication means 10, a person extraction assistance
Means 11, optimal imaging selection means 12, server communication means 13
And an extracted person image storage means 14. Below,
The respective operations will be described. The person extraction communication means 10 is provided for each person extraction means.
Load sharing of the person extraction function according to the detection frequency in step 1, and
And the recognition certainty from the person extracting means 1 and the intermediate server
Controlling the transmission of the extracted image according to the communication load of the means 2;
As a result, the person sent from the person extracting means 1
Receive the object extraction image. The received person extraction image is the extraction person
The position information, the reliability, and the extraction
Stored with the departure time. This communication processing is performed in the following procedure, for example.
Be executed. The processing flow of this person extraction communication means 10
This is shown in the flowchart of FIG. (I) Acquisition of internal state of person extraction means and finger of person extraction method
The number of temporarily saved images from each person extracting means 1 as an internal state
It is determined whether Ns and the detection frequency H exceed the threshold Th.
And the detection frequency described by the person candidate communication means 8
Processing in the person extracting means 1 is performed according to the
Whether to use only the means 6 or the person area confirmation means 7
Indicate whether to do this. By doing this,
Need to confirm the candidate,
Prevents missing detection even in places where many people intrude frequently
To make things possible. Also, it is not necessary to confirm
In such an environment, the person candidate area extracting means 6
It is also possible to perform only the processing. Note that the processing of the person area confirmation means 7 is also described above.
If the process is to be performed inside the intermediate server means 2, the above-mentioned person
As described in the area confirmation means 7, the saved person
Matching result outside the complement area (non-object class data
(3) also transmits the surrounding coordinates. (Ii) An image corresponding to the recognition certainty factor and the communication load
Image transmission control Next, each person extraction means 1 to the above-mentioned person candidate communication means 8
As described above, the extraction area, the extraction time, and the reliability Ckl of the extraction area
Is sent, and the detection frequency H by each person extracting means 1 is
k, from communication traffic to the following method, for example
Therefore, the control of image transmission, that is, the transmission image area Rk
Ask. Here, k is the person extraction means number, and l is one person
It is assumed that 0 ≦ l ≦ L by the extraction area number in the object extraction means. [0044] However, the extraction area is Rkl, and when Tkl> the whole screen, Tkl
= The entire screen. U indicates the sum of the sets. If SH ≧ Tk3 Rk = temporarily save without transmitting the current extracted image,... (9 ″ ′) where SH is an amount representing communication traffic.
For example, using the sum of the detection frequencies in all person extraction means,
You may do so. Ckl is the reliability of each extraction area, and Hk is each person's extraction procedure.
And K is the number of accesses of each person extraction means.
You. Further, the image of the non-target class in (i) is transmitted.
In this case, the same
Treat it. Next, the person extraction assisting means 11 shown in FIG.
explain. FIG. 9 shows the flow of processing performed by the person extraction assisting means 11.
It is a flowchart shown. The person extraction assisting means 11
In the above description of the person candidate communication means 8 in FIG.
When frequency ≧ Th, exactly the same as the person area confirmation means 7
One process is performed. For non-target classes in person confirmation,
The total number of dictionary update data since the last dictionary update
If a certain number (Ns × α) or more could be collected from the transmitted image
In this case, the dictionary for personal identification according to the present invention is reconstructed.
U. Here, Ns is the conventional number of non-target samples,
It is a constant between α0 and 1, which is used as a sample for reconstruction.
Is the new sample Ns × α and the conventional sample
Ns × (1-α) are randomly selected from
(2) (b) (i) to (iii) of the person area confirmation means 7
A new dictionary is created by the processing in. Next, the optimum image pickup selecting means 12 shown in FIG.
explain. FIG. 10 shows an optimum imaging selecting means 12 according to the present invention.
6 is a flowchart showing the flow of the processing of FIG. Figure 2
As described in (ii) in the description of the candidate communication means 8,
As the matching result of step (vi) of the area checking means 7
Of the matching result areas in the person candidate area,
Matching distance from N to N, and it
Dictionary category name that matches each of them, in each screen
Is transmitted to the optimal imaging selection means 12. Here, as a definition of the optimality,
The distribution of locations among the results is unified and matches
The distance from the front-end dictionary category is small.
Shall be. A feature amount F (optimum person imaging)
For example, the following can be used
You. [0050] Where σ is the standard deviation of the coordinate distribution of each matching result area
And Di is the i-th matching distance.
This is the distance between the split window and the dictionary. A, B, Cj are heavy
Is the only constant, especially Cj matches the i-th window described above.
Weight according to the direction of the face of the determined dictionary j, which is close to the front
Set as large as possible. Then, it is connected to the intermediate server means 2
From a plurality of (k = 0 to K) person extraction means 1
In the object extraction area, the physical position of each other (in the real world)
Each area where the (coordinates) overlap is paired with the same object.
And the largest of the above F values for them
The video from the person extraction means k having Fk is optimized
select. Here, the physical position is measured in advance
Imaging environment (direction, depression angle,
And the coordinates of the person candidate area detected during the person extraction
Ask from. The above determination of A, B, Cj
Can be various methods, for example,
Teach the best one in multiple sample images
Using σx, σy, Di in those samples
If the F value of the one taught as optimal is larger than the others,
And decide. That is, what is taught as optimal
When a pull k ′ is set, ′ (Fk | k = k′−Fk | k ≠ k ′) (12) Learning is performed so as to maximize all samples.
A, B, and Cj are determined. Server communication means 13 of FIG.
Means that the optimal person image sequence is
When prepared, this sequence image is transmitted to the server.
You. Finally, the server means 3 of FIG. 1 will be described.
You. The server means 3 is transmitted from the server communication means 13.
Received the optimal image sequence, and the http protocol
It is converted to a format conforming to and transmitted. At this time, the above series
Contains redundant images, and the server means 3
If the communication traffic volume of the
Server means 3 leaves only one of them and removes the others.
To transmit. Here, the above-mentioned redundancy is the
It is said that the similarity between the frames of the appropriate image series is high
In a sense, for example, the sum of image difference amounts between two frames is constant
If the value is less than the threshold value or the correlation value of the image between two frames is different
A general one can be used, for example, when the threshold value is equal to or more than a certain threshold value. Here, the amount of communication traffic to the server and
At a certain time during http access to this server
The number of packets in the interval Ph, the image with the intermediate server 2, and the extraction
The number of packets Pm in a fixed time used for transmitting the output attribute
Can be defined as the sum of In the above, the intermediate server
The optimal image sequence obtained from the bar 2 includes the position and the extraction time.
These attributes are included in the server means 3,
In addition to the frequency distribution of pixel values in the color space within the region,
The average moving vector obtained from the
It is added and added to the search tag information. Furthermore the above optimal person
For the face area in the object image sequence,
The projection value vector with the face dictionary described in (vi) of step 7 also belongs to
By assigning it as a gender value, the image of the search target person
Given, the person's face is
It is possible to search and present from a series image. According to the present invention, functions are assigned to each processing.
Responsiveness and cost reduction. That is, a person
Monitoring is the most basic function and requires real-time processing
Built-in TV camera with person extraction function
An intermediate server that evaluates the optimality of the video extracted from camera,
Processing time is required for search and tagging, but real-time
Server that performs processing that does not require
Transmit digitalized and summarized information between each
This makes it possible to reduce transmission and storage responsiveness and reduce costs.
You. Further, according to the present invention, the detection of the characteristic portion of the person
This makes it possible to reduce the possibility of erroneous extraction and to detect an optimal image.
That is, in the person extraction function, a person candidate area is extracted.
After that, the characteristic part (face, etc.) of the person appears in that area.
In order to evaluate whether a person has
Event detection frequency can be reduced.
You. In addition, the character features extracted at that time are displayed on the screen.
Using the current method, the image frame is later
To see if it is good for checking
An optimal image sequence can be selected from a plurality of frames. Further, according to the present invention, the elimination of the redundant image and the negative
Further transmission and storage efficiency by dynamic processing sharing according to load
Is improved. In other words, the appearance selected above
To evaluate the redundancy between frames of the optimal image
Eliminating frames reduces the amount of transmission and storage,
It is possible to improve efficiency. Intruder detection frequency
And each processing layer (camera, intermediate server) according to the transmission load.
Server and server) by automatically relocating
To improve the transmission and storage efficiency of the entire monitoring system
It becomes possible. Therefore, the cost can be reduced compared to the prior art,
The present invention enables person-free person monitoring.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is an overall configuration diagram according to an embodiment of the present invention. FIG. 2 is a configuration diagram of a person extracting unit according to the embodiment of the present invention. FIG. 3 is a flowchart showing a first method of a person candidate area extracting unit. FIG. 4 is a flowchart showing a second method of the person candidate area extracting means. FIG. 5 is a diagram for explaining a person area confirmation unit according to the present invention. FIG. 6 is a flowchart showing processing in a person area confirmation unit according to the present invention. FIG. 7 is a diagram showing a configuration of an intermediate server unit according to the present invention. FIG. 8 is a flowchart showing processing of a person extraction communication unit in the intermediate server unit according to the present invention. FIG. 9 is a flowchart showing processing of a person extraction assisting means in the intermediate server means according to the present invention. FIG. 10 is a flowchart showing a process of an optimum imaging selecting means in the intermediate server means according to the present invention. [Description of Signs] 1 ... person extraction means, 2 ... intermediate server means, 3 ... server means

Claims (1)

(57) [Claims 1] An image pickup means for picking up an image of a monitoring area, a processing means for performing predetermined processing on an image picked up by the image pickup means, and a predetermined processing by this processing means Receiving the image to which the image processing has been applied, calculating an image difference amount between two consecutive frames, and, when the sum of the images is equal to or smaller than a predetermined value, removing one or more of the remaining images and removing the other image. An image monitoring apparatus, comprising: a transmission unit that converts only an image left by the removal unit into an http protocol and transmits the HTTP protocol.