JP5758165B2 - Article detection device and stationary person detection device - Google Patents

Description

  The present invention relates to an apparatus for detecting an object existing in a monitoring space by performing image analysis on a captured monitoring image, and more particularly to an apparatus for detecting a stationary article and a stationary person in the monitoring space.

  Conventionally, there is a demand for detecting various crimes from surveillance camera images. For example, when a burglar breaks into a manned store and tries to seize money stored in a safe, he or she may attack the employee because of fear of reporting to the outside or resistance. In some cases, restraining the limbs with a rope or adhesive tape may deprive the body of freedom.

  Victims such as employees who have been deprived of physical freedom in this way will be left in the store after the crime, so victims who were delayed in the detection of robbery incidents due to delayed reporting or who were injured There are concerns about delays in rescue.

  For this reason, the proposal of the abnormality detection apparatus which detects automatically that a victim is in the state which cannot perform the above notification operations, and reports is desired.

  When the presence of a victim as described above is detected by behavior analysis using image processing, the behavior characteristics of the victim are such that they remain stationary and do not move. However, when an article brought in by a person is left in the monitoring area, the same behavioral characteristics are exhibited, so that it is difficult to distinguish the person from the brought-in article.

  Here, as a technique for distinguishing between “the article brought in by a person is stationary” and “the person who has been moving is stationary” in Patent Document 1, even if the person stops, the person is in a completely stationary state. A detection device using what has not been disclosed is disclosed.

Japanese Patent No. 4218368

  However, there is a case where it is difficult to distinguish “stationary of an article” and “stationary of a person” by the technique described in Patent Document 1. For example, if a victim is fainted by an attack from an intrusion robbery, even a person will be in a completely stationary state, so it is difficult to distinguish it from an article.

  An object of the present invention is to accurately determine an article that has been brought in, and to identify a left person and accurately detect a left person.

The present invention provides an image acquisition unit that sequentially acquires a monitoring image obtained by photographing a monitoring region, a background image obtained by photographing the background of the monitoring region, a storage unit that stores a person model imitating the outline of a person, and the monitoring image A control unit that detects a stationary article by processing the image, and the control unit includes a change area extraction unit that extracts a change area from the monitoring image and the background image; Article feature storage means for superimposing the person model on the change area extracted from the monitoring image , and storing in the storage section a feature in the monitoring image of an article area inside the change area and outside the person model; A stationary object region detecting means for tracking the variation region in the sequentially acquired images, determining whether the variation region is stationary, and detecting a stationary object region; and detecting the stationary object region. Once, the still object region is determined whether having the characteristics of the stored goods area in the storage unit, the still object region determines the article determination means that the article if it has a characteristic of the article area It is characterized by having.

  Moreover, it is preferable that the feature of the article region is a color histogram indicating an appearance frequency for each color in the article region.

  In addition, a stationary person detection apparatus according to the present invention includes the above-described article detection apparatus, and the control unit determines that the stationary object area is stationary when the detected stationary object area does not have the feature of the article area. It further has a still person area determining means for determining that it is a person area.

  According to the present invention, an article brought into the monitoring area can be detected effectively, and a stationary person can also be detected by identifying the article brought in. Accordingly, it is possible to prevent an erroneous report by distinguishing between an abnormal state in which the victim is in a state where the victim cannot move due to a pushing burglary or the like and a normal state in which the brought-in article is stationary.

It is the schematic diagram which showed the structure of the image of a monitoring space, and an article | item detection apparatus (stationary person detection apparatus). It is a block diagram which shows the structure of an article | item detection apparatus (stationary person detection apparatus). It is a flowchart which shows the operation | movement of abnormality detection.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

"overall structure"
A report system 1 that includes an article detection apparatus (stationary person detection apparatus) according to the present embodiment, detects an abnormal scene in which a stationary person is present instead of a stationary carry-in article, and notifies a security center or the like. To do.

  FIG. 1 is a schematic diagram showing an image of the configuration and arrangement of the notification system 1.

  The notification system 1 includes an article detection device (stationary person detection device) 2, a controller 3, and a center device 5.

  The article detection device (stationary person detection device) 2 is installed on the ceiling of a room in which a room in which an important object such as a safe 6 is installed is used as a monitoring space. The article detection device (stationary person detection device) 2 outputs an abnormal signal when detecting an abnormal scene occurring in the monitoring space.

  The article detection device (stationary person detection device) 2 is connected to a controller 3 via a communication line, and the controller 3 is a security center or the like installed at a remote place via a wide-area communication network 4 such as a telephone line or an Internet line. Connected to the center device 5. The abnormality signal output from the article detection device (stationary person detection device) 2 is transmitted to the center device 5 via the controller 3.

"Configuration of article detection device (stationary person detection device)"
FIG. 2 is a block diagram showing the configuration of the article detection device (stationary person detection device) 2. The article detection device (stationary person detection device) 2 includes an imaging unit 20, a storage unit 21, an output unit 23, and a control unit 22.

  In FIG. 1, the article detection device (stationary person detection device) 2 is provided on the ceiling of the monitoring area, but only the imaging unit 20 may be provided on the ceiling, and other devices may be provided in another place.

(Imaging unit 20)
The imaging unit 20 is a monitoring camera using, for example, a CCD imaging device. The imaging unit 20 is usually attached to the ceiling of the monitoring space (monitoring area). The shooting may be continuous shooting such as video shooting, but it is preferable that the monitoring area is sequentially shot at predetermined time intervals. The captured monitoring images of the monitoring space are sequentially output to the control unit 22. The time interval at which the monitoring image is captured is 1/5 second, for example. Hereinafter, the unit of time recorded at the time interval of imaging is referred to as time. In the present embodiment, the imaging unit 20 includes a fisheye lens and is installed with its optical axis directed vertically downward, and images the entire room as a monitoring space.

(Storage unit 21)
The storage unit 21 is a storage device such as a ROM (Read Only Memory) or a RAM (Random Access Memory). A flash memory or a hard disk may be used. The storage unit 21 stores various programs and various data, and inputs / outputs such information to / from the control unit 22. Various data stored in the storage unit 21 includes a background image 210 necessary for background difference processing and a person model 211 necessary for extracting an article region. Furthermore, the storage unit 21 also includes a tracking feature storage unit 212 that stores features of the object region necessary for tracking the object region, and an article feature storage unit 213 that stores the extracted feature of the article region.

(Control unit 22)
The control unit 22 is configured using an arithmetic device such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an MCU (Micro Control Unit), and reads out and executes a program from the storage unit 21 to extract an object region. Functions as means 220, object tracking means 221, article area extraction means 222, stationary object area detection means 223, and abnormality determination means 224.

(Output part)
The output unit 23 is a communication unit that outputs an abnormal signal to the outside of the article detection device (stationary person detection device) 2. When an abnormality signal is input from the abnormality determination unit 224 of the control unit 22, the output unit 23 outputs the abnormality signal to the controller 3.

"Processing flow in article detection device (stationary person detection device) 2"
FIG. 3 is a flowchart showing an abnormality detection process in the article detection apparatus (stationary person detection apparatus) 2. Various data processing is basically performed in the control unit 22.

(S1: Initialization)
When the power is turned on, each unit is initialized and starts operating. The initialization includes a process of storing the monitoring image immediately after startup in the storage unit 21 as the background image 210. Here, it is preferable to acquire the background image every time when a person does not enter, but various means such as storing a fixed screen to some extent can be employed.

(S2: Monitor image acquisition)
When the initialization is completed, a monitoring image is acquired. That is, the imaging unit 20 outputs a monitoring image every time the monitoring space is imaged, and the control unit 22 acquires the monitoring image. In S2, a monitoring image at the current time is acquired and sent to the control unit.

(S3: Background difference / labeling)
The control unit 22 removes the background from the monitoring image at the current time sent from the imaging unit, extracts the object region, and labels the obtained object region. The object area extraction unit 220 compares the monitoring image obtained from the imaging unit 20 with the background image 210 stored in the storage unit 21, and extracts a pixel whose difference from the background image 210 is equal to or greater than a threshold as a change pixel. To do. Note that brightness, color, edge strength / direction, and the like can be used as the difference feature amount. As the threshold value, a predetermined value determined in advance may be used, or may be changed dynamically.

  Further, it is desirable to remove light / shadow regions by normalized correlation after the difference calculation, and to remove sesame salt-like noise by expansion / contraction processing.

  The object region extraction unit 220 performs labeling after extracting the changed pixels. Labeling is a process of assigning the same number to spatially connected pixels. With this process, individual numbers are assigned to the blocks (sets) of changed pixels. Hereinafter, this lump is referred to as an object region. Note that the extracted object area includes a person area and a brought-in object area including an article or the like.

(S4: Object tracking)
Data about the object region extracted by the object region extraction unit 220 is supplied to the object tracking unit 221. The object tracking means 221 is extracted by the object area extraction means 220 by the previous time and stored in the tracking feature storage section 212 of the storage section 21 and the object area extraction means 220 at the current time. The tracking characteristics of each object area extracted by the above are compared, and the object areas are tracked by associating the object areas.

  For this purpose, the object tracking unit 221 extracts an image feature of the object region as a tracking feature every time when the appearance of the object region in the monitoring region is confirmed, and stores it in the tracking feature storage unit 212 of the storage unit 21. Then, the tracking feature of the object region extracted at the previous time stored in the tracking feature storage unit 212 of the storage unit 21 is compared with the tracking feature of the object region extracted at the current time. The object area determined to be the most similar among the objects is associated. As the tracking feature, a color or edge histogram, a gravity center position of an object region, a circumscribed rectangle, or the like is used.

  If the object area extracted at the current time is not associated with any of the tracking features stored in the storage unit, it is determined that the person image is an object area newly appearing at the current time. The tracking feature is stored in the tracking feature storage unit 212.

  If there is a tracking feature stored in the tracking feature storage unit 212 that is not associated with the object region extracted at the current time, the tracking information is moved out of the field of view of the input image. That is, it is determined that it belongs to a person who has gone out of the office, and is deleted from the tracking feature storage unit 212. Or it is good also as what hold | maintains a fixed time and tries said correlation process in the meantime.

  The above-described processing in the object tracking unit 221 is general as tracking processing in the field of image processing technology, and thus the details are omitted.

(S5: Human model fitting)
The article area extraction unit 222 reads the person model from the storage unit 21 and applies it to the object area.

  In this embodiment, the human model is an ellipse. The size and inclination of the ellipse are determined for each image coordinate according to the standard person size and camera installation conditions. For example, in a fish-eye camera installed vertically downward on the ceiling surface, the human model is an ellipse having a long axis that overlaps with a straight line extending radially from the center of the image to the center of the ellipse. The major axis length L = L ′ + α, the minor axis length S = S ′ + β is L ′, S ′ is a value determined from a standard person size, and α, β are fluctuations for absorbing individual differences. Width. The maximum fluctuation range (αmax, βmax) of α and β is defined in advance. In addition, since a person is not always upright, a major axis deviation angle (shift angle from the major axis radial direction) θ is also considered. The maximum fluctuation range (φmax) is defined for the same deflection angle.

  From the above, the parameters of the human model are the six parameters of the central coordinate (x, y), the major axis length (L), the minor axis length (S), and the major axis deviation angle (θ).

  An ellipse that most closely matches the object region is searched while these are varied stochastically. In this embodiment, the goodness of fit of the human model to the object region is expressed by the following evaluation function.

(Evaluation function: E) = − {(pixels that are object regions but not human model regions)
+ Γ × (pixel model region but not object region)}

  The first term is the number of pixels that are not covered by the human model in spite of the object area, and the second term is the number of pixels that are not covered by the human model even though it is not the object area. The degree of match is evaluated by the number of protruding pixels. The evaluation function takes a negative value, and the larger the value (closer to 0), the higher the matching degree between the object region and the person model.

  γ is a parameter for balancing the first term and the second term. For example, when a threshold value for difference processing is set so that a changed pixel is easily extracted, the value of γ may be increased.

As an initial value of the person model parameter, for example,
-The center coordinates (x, y) are the center of gravity coordinates of the object region-The long axis length (L) is a random number with [-αmax, αmax] in the range of L 'and the value is added. The short axis length (S) is a random number with [-βmax, βmax] in the range of [−βmax, βmax] for S ′, and the value is added. The deviation angle (θ) is [−φmax, φmax]. Select a random value for the range.

  From this state, the parameters of the ellipse are updated as follows using, for example, a probabilistic search method.

(I) An ellipse is drawn with initial parameters, and the evaluation function value Ecur is calculated. Then, the value of the evaluation function for the initial parameter is set to the maximum value of the evaluation function. That is, Emax = Ecur.

(Ii) A new parameter group is generated using random numbers.
Center coordinates: Xnew = X + rand [−δXmax, δXmax]
Ynew = Y + rand [−δYmax, δYmax]
Long axis length: Lnew = L + rand [-αmax, αmax]
Short axis length: Snew = S + rand [−βmax, βmax]
Deviation angle: θnew = θ + rand [−φmax, φmax]
Here, δXmax and δYmax are fluctuation ranges of the center coordinates defined in advance. Rand [] represents random number generation with the value in [] as the range.

(Iii) An ellipse represented by a new parameter group (Xnew, Ynew, Lnew, Snew, θnew) is drawn, and an evaluation function value Enew is calculated.

(Iv) If the value Enew of the evaluation function is equal to or greater than the value Ecur of the previous evaluation function (if the new parameter group is better fit), the parameters are switched. That is, the value of the evaluation function is set to Ecur = Enew, and the values of the parameters X, Y, L, S, and θ are replaced with (Xnew, Ynew, Lnew, Snew, θnew).

(V) Otherwise, a random number of [0, 1) is generated. That is, a random number between 0 and 1 is generated.

(Vi) For Enew obtained in (iii), if Ecur / Enew is larger than the random number, the parameters are switched. Let Ecur = Enew.
(Vii) Compare Ecur and Emax. Here, if Ecur <Emax, Ecur = Emax.

(Viii) The above (ii) to (vii) are repeated a prescribed number of times, and the parameter giving Emax is set as the final solution.

  Although the parameter is basically updated in the direction in which the evaluation value increases by the above operation, the evaluation value is probabilistically decreased by the operations (v) to (vi). This is a method for searching for a better solution by skipping the local solution on the evaluation function because the evaluation function is not monotonous but has a complicated form.

  Also, continuity in the time direction can be introduced into the evaluation function that represents the goodness of fit between the object region and the person model. For example, by calculating the color histogram in the person model at the previous time and adding the degree of coincidence with the color histogram in the person model at the current time to the evaluation function, an oval is used so that the color does not vary greatly between frames. Can be fitted.

(S6: article region extraction)
In this way, after fitting the most suitable person model to the object area in S5, the article area extracting unit 222 sets the remaining area excluding the part where the person model is fitted from the object area as the article area. Extract. This is because the object area that protrudes from the person model is highly likely to be a carry-in article, and the object area that protrudes from the person model is extracted as an article area. If there are a plurality of protruding areas in the object area, the area having the maximum area is selected and extracted as an article area.

(S7: Determination of presence / absence of article area)
The article area extraction unit 222 determines whether or not an article area exists based on whether or not the article area extracted in S6 has a size equal to or larger than a threshold value. The threshold (area threshold) is, for example, about 30% of the area of the fitted elliptic model. That is, among the article regions extracted in S6, those having a small area are deleted from the article region.

(S8: Feature amount registration of article area)
In S7, when the article region exists, the article region extraction unit 222 calculates the feature amount of the part corresponding to the article region in the monitoring image, and the article feature storage unit of the storage unit 21 as the feature amount of the article region. 213 is registered. The feature amount is, for example, a color histogram or an edge histogram. In this way, the article feature amount is sequentially registered for the article area generated in the monitoring area. Moreover, you may delete the item feature-value which does not exist in a monitoring area | region for a fixed period at any time.
As described above, the article region extraction unit 222 functions as an article feature storage unit, and stores the extracted article feature amount in the storage unit 21.

(S9: stationary object determination)
When the registration of the feature amount in S8 is completed, and when it is determined in S7 that the article region does not exist, the stationary object region detection unit 223 includes the object region tracked by the object tracking unit 221. It is determined whether there is a stationary object region. The determination of the stationary state is based on, for example, the following conditions (i) to (iii).

(I) There is no inter-frame difference in the object area. For example, it is determined that there is no interframe difference when the interframe difference is 5% or less of the area of the object region.

(Ii) The barycentric coordinates of the object area have not moved. For example, when the movement distance is 3 pixels or less, it is determined that the barycentric coordinates are not moving.

(Iii) A circumscribed rectangle of the object area is formed, and the size of the circumscribed rectangle is not changed. For example, when the rectangle change rate is 5% or less, it is determined that there is no change in the size of the circumscribed rectangle.

  Then, an object region that satisfies all the conditions (i) to (iii) for a predetermined time or more, for example, 10 frames or more is determined as a stationary object. Of course, the two conditions (i) to (iii) are satisfied for a predetermined time or more, or the period in which only one condition is not satisfied is a predetermined time or less and the time for satisfying the three conditions is a predetermined time or more. Various conditions can be set.

(S10: Comparison of feature amount between article region and stationary object)
The abnormality determination unit 224 compares the feature of the stationary object determined in S9 with the article feature amount stored in the storage unit 21 in S8. When a color histogram is used as a feature, the similarity between both feature amounts is expressed as follows. Assuming that the similarity is Sim, the color histogram of a completely stationary object is l (i), and the article feature is h (i) (i represents the index of the bin of the histogram), it can be expressed as

    Sim = ΣMIN (l (i), h (i))

l (i) and h (i) are the already stored item feature values and the appearance frequency for the color i of the still object recognized this time, and for each i, l (i) and h (i) By selecting the smaller one and integrating it, an integrated value of the number of pixels common to both of each color can be obtained. Since the sum of bins of histograms l (i) and h (i) is normalized to 1, this similarity Sim is an index indicating how much the two histograms overlap. Returns 1 (maximum similarity), 0 (minimum similarity) if there is no overlap.
When the stationary object is determined in S9, if the article feature quantity is not stored in the storage unit 21, the abnormality determining unit 224 determines that the stationary object is a stationary person, and proceeds to S14.

(S11-S15: Identification of goods and stationary people)
The abnormality determination unit 224 compares the similarity between the feature quantity of the stationary object determined in S10 and the article feature quantity stored in the storage unit 21 with a predetermined threshold, and the similarity is equal to or greater than the threshold. Is determined that the stationary object has an already stored article feature amount, and is determined to be a stationary article (carry-in article) brought in (S12). On the other hand, if the similarity is not greater than or equal to the threshold, it is determined that the person is a stationary person (S14). Note that a part of the abnormality determination unit 224 that determines an article is an article determination unit 224a, and a part of the abnormality determination unit 224 that determines that it is a stationary person is a stationary person region determination unit 224b. Here, the threshold value regarding the similarity of features can be set to 0.75 (75% of the distribution overlaps) when the feature is a color histogram, for example. Further, in S14, it is further determined whether or not a person who has been recognized as a stationary person satisfies other various conditions such as whether it is determined to be a person in the monitoring area or has characteristics of the person. Also good.

  If it is determined that the article is a stationary article, it is desirable that the article is not extracted in the subsequent background difference process, and the corresponding stationary object region is taken into the background image (S13).

  It is also possible to accumulate this determination in the time direction. That is, if the state where the similarity is equal to or greater than the threshold value in S11 continues for a predetermined time or more, it is determined as a stationary article. By accumulating the time, it is possible to make a determination with high accuracy even when the matching of feature amounts is not stable due to noise in the input image, for example.

(S15: Abnormal output)
The abnormality determination unit 224 outputs an abnormality signal to the output unit 23 when it is determined that the person is a stationary person in S14.

"Effect of this embodiment"
According to this embodiment, a human model represented by an ellipse is fitted to an object region extracted from a monitoring image, and the feature amount is registered assuming that the region protruding from the human model is an “article region”. . Then, when an object area that continues to be stationary appears, the feature quantity of the already registered “article area” is compared with the feature quantity of the stationary object area (stationary object area), and a certain degree of similarity can be obtained. In other words, it is determined that it is “stationary of an article brought in” and not “stationary of a person”. Accordingly, it is possible to effectively detect the brought-in article, and it is possible to reduce the possibility that the brought-in article is erroneously determined as a stationary person.

  DESCRIPTION OF SYMBOLS 1 Reporting system, 2 article detection apparatus (stationary person detection apparatus), 3 controller, 4 wide area network, 5 center apparatus, 6 safe, 20 imaging part, 21 memory | storage part, 22 control part, 23 output part, 210 background image, 211 human model, 212 tracking feature storage unit, 213 article feature storage unit, 220 object region extraction unit, 221 object tracking unit, 222 article region extraction unit, 223 stationary object region detection unit, 224 abnormality determination unit

Claims (3)

An image acquisition unit that sequentially acquires a monitoring image obtained by photographing the monitoring area, a background image obtained by photographing the background of the monitoring area, a storage unit that stores a person model imitating the outline of a person, and processing the monitoring image A control unit for detecting a stationary article, and an article detection device comprising:
The controller is
Change area extraction means for extracting a change area from the monitoring image and the background image;
Article feature storage means for superimposing the person model on the change area extracted from the monitoring image , and storing in the storage section a feature in the monitoring image of an article area inside the change area and outside the person model; ,
A stationary object region detecting means for temporally tracking the variation region in the sequentially acquired images, determining whether the variation region is stationary, and detecting a stationary object region;
When the stationary object area is detected, it is determined whether the stationary object area has the characteristics of the article area stored in the storage unit . If the stationary object area has the characteristics of the article area, the stationary object area is an article. Article determination means for determining
An article detection apparatus comprising: The article detection apparatus according to claim 1.
The feature of the article region is a color histogram indicating an appearance frequency for each color in the article region. While having the article detection apparatus according to claim 1 or 2,
The controller is
A still person detection apparatus, further comprising: a still person area determining unit that determines that the still object area is a still person area when the detected still object area does not have the feature of the article area.

Images