WO2017126187A1

WO2017126187A1 - Video monitoring apparatus and video monitoring method

Info

Publication number: WO2017126187A1
Application number: PCT/JP2016/082598
Authority: WO
Inventors: 雅志神谷; 芳知中村; 内藤　正博
Original assignee: 三菱電機株式会社
Priority date: 2016-01-20
Filing date: 2016-11-02
Publication date: 2017-07-27
Also published as: JP2019050438A

Abstract

A video monitoring apparatus characterized by being provided with: a person tracking unit that detects persons from a frame image and tracks each of the detected persons over a plurality of frames; a people-flow information generation unit that, from a tracking result from the person tracking unit, recognizes persons, from among the detected persons, whose moving directions and moving speeds are estimated to be close to one another, to be groups, and that generates people-flow information including a group moving direction and a group moving speed for each of the groups; a people-flow information analysis unit that determines from the people-flow information whether an unusual event has occurred in a people flow, and specifies a position when it is determined that an unusual event has occurred; a suspicious person candidate detection unit that detects, as a suspicious person candidate, a person, from among the persons, who is present near the position specified by the people-flow information analysis unit; and a suspicious person candidate presentation unit that presents the detected suspicious person candidate.

Description

Image monitoring apparatus and image monitoring method

This invention relates to a video monitoring device for monitoring a suspicious person or the like using a camera.

In recent years, surveillance systems have been developed to monitor suspicious individuals, etc., using images taken with cameras installed in various places, such as in stores, facilities, and urban areas. In particular, safety measures are required in places where many people gather, such as event venues, and it is important to identify suspicious persons and to appropriately guide evacuation when problems such as incidents occur.

Conventionally, a security guard or other monitoring person looks at the display on which the camera image is displayed, estimates the suspicious person from the person appearing on the image displayed on the display, and performs the action of the person who is suspected of the estimated suspicious person In general, the suspicious person is identified by checking the camera image or directly in the field. However, this method has a problem that it takes time to identify a suspicious person if the number of people or the number of camera images is large.

Therefore, by using a computer and comparing with a suspicious behavior model learned in advance based on the movement trajectory of the person being monitored from the camera image, a suspicious person who is performing abnormal behavior is identified and identified. There was a method (for example, refer patent document 1).

JP 2008-217602 A (page 4-7, FIG. 3)

However, in the method of Patent Document 1, if there are many people gathering, it is necessary to follow the movement trajectory of all the people shown in the camera image. If the number of people is large, the computer processing is enormous. there were.

The present invention has been made to solve the above-described problems, and an object of the present invention is to provide a video monitoring apparatus that suppresses an increase in processing when the number of persons shown in a camera video increases.

In the video monitoring apparatus according to the present invention, a person is detected from a frame image, a person tracking unit that tracks each detected person over a plurality of frames, and a tracking result of the person tracking unit, Among them, the human flow information generating unit that recognizes the person estimated to be close in moving direction and moving speed as a group, and generates human flow information including the group moving direction and the group moving speed for each group, A human flow information analysis unit for determining whether or not an abnormal event has occurred, and identifying a position when it is determined that an abnormal event has occurred; among the persons, the human flow information analysis unit is present in the vicinity of the position specified by the human flow information analysis unit A suspicious person candidate detecting unit that detects the person to be detected as a suspicious person candidate, and a suspicious person candidate presenting unit that presents the detected suspicious person candidate.

Since this invention recognizes a plurality of detected persons as a group and detects suspicious candidates from the change in the flow of the group, even if the number of persons appearing in the camera video increases, it can be grasped collectively by the group. it can. For this reason, there is an effect that it is possible to suppress an increase in processing even if the number of persons shown in the camera video increases.

1 is a block diagram showing a configuration of a video monitoring system according to a first exemplary embodiment. It is a figure which shows an example of the person tracking information D1 concerning Embodiment 1. FIG. It is a figure which shows an example of the human flow information D2 concerning Embodiment 1. FIG. It is a figure which shows an example of the suspicious person candidate presentation information D5 shown on the display. FIG. 3 is a flowchart showing processing of the video monitoring apparatus 10 according to the first exemplary embodiment. It is a block diagram which shows the structure of the video monitoring system concerning Embodiment 2. FIG. It is a block diagram which shows the structural modification of the video surveillance system concerning Embodiment 2. FIG.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing the configuration of the video monitoring system according to the present embodiment. The video monitoring apparatus 10 presents the suspicious person candidate presentation information D5 on the display 30 with the video from the camera 20 as an input. The video monitoring apparatus 10 includes a person tracking unit 101, a human flow information generation unit 102, a human flow information analysis unit 103, a suspicious person information detection unit 104, and a suspicious person information presentation unit 105.

The camera 20 holds captured images in units of frames. Further, the camera 20 outputs to the video monitoring apparatus 10 in units of frames. Hereinafter, the captured image is also referred to as a frame image. The camera 20 is provided at a position where an area with a person's flow line can be imaged. The “flow line” is a line that represents a path that a person is supposed to take when moving naturally.

The person tracking unit 101 detects a person from the frame image input from the camera 20. Further, when the detected person is recognized as the same person over a plurality of frames, the person tracking unit 101 generates person tracking information D1 indicating the movement information. The person tracking unit 101 outputs the generated person tracking information D1 to the person information generating unit 102. In addition, the person tracking unit 101 outputs the person detection result to the suspicious person candidate detecting unit 104.

As a person detection method by the person tracking unit 101, there is a method in which a large number of person images are collected in advance and a person feature such as brightness gradient information and contour information is learned. In this method, when a person discriminator is created and a frame image is acquired from the camera 20, it is determined whether or not the person discriminator includes an area having a feature similar to the learned human feature in the frame image. Determine. If an area is included, the coordinates of the area (starting point coordinates and vertical and horizontal lengths) are specified. In addition, a person detection method using a feature amount called HOG (Histogram of orientated gradients) is common, but in the present invention, a person is detected using any of the known person detection methods.

As a person tracking method by the person tracking unit 101, the person feature extracted when the person is detected by the person detection method described above is used, and the similarity between the person features detected in a plurality of frames is equal to or higher than a certain threshold. In such a case, there is a method for determining that these persons are the same. However, when there are a plurality of combinations having a similarity exceeding the threshold, the one with the highest similarity is determined as the same person.

Also, even if the degree of similarity is high, it can be determined that they are not the same person if the specified coordinates differ greatly. For example, the time between frames that the person tracking unit 101 continuously acquires from the camera 20 is 33 ms, and the actual distance to one pixel on the frame image is 0.5 m. Also, it is assumed that the start point coordinates of a person specified by the above-described person detection method and whose similarity is equal to or greater than the threshold are 1000 pixels apart between consecutive frames. In this case, if this person is the same person, it will proceed 500 m in 33 ms, which is practically impossible, so it can be determined that they are not the same person. As described above, since the moving speed of a similar person can be estimated from the information on the positional relationship of the camera 20 installed in the monitoring system and the time interval between frame images, the moving speed is unlikely to be considered a human moving speed. It is determined that they are not the same person.

FIG. 2 is a diagram illustrating an example of the person tracking information D1 generated by the person tracking unit 101. As shown in FIG. 2, the person tracking information D1 includes a person number D1a indicating each detected person, a detection frame number D1b, a detection coordinate D1c, and person feature information D1d.

The person number D1a is a number that the person tracking unit 101 assigns separately to enable identification of different persons.

The detected frame number D1b indicates the frame number of the frame image in which the detected person is shown for each detected person. The frame number may be a number assigned to each frame image output from the camera 20. The frame number may be a number assigned by the person tracking unit 101 for each frame image.

The detected coordinates D1c are coordinates at which the person tracking unit 101 detects the person in the frame image. The coordinates are expressed by a two-dimensional axis with the pixel position at the upper left of the frame image as (0, 0), for example, and the position where the center position of the detected person is estimated is detected coordinates D1c. For example, the position of a part of the detected person such as the center position of the head may be used as the detection coordinate D1c.

The person feature information D1d is a person feature extracted by the person tracking unit 101. The person feature is, for example, a HOG feature amount, and indicates a feature amount obtained by histogramating the relationship between the gradient direction (edge direction) and the gradient strength in the image area obtained by dividing the frame image. For example, when the image size of a frame image or an image area in which a person is shown is 30 pixels × 60 pixels, the gradient intensity for each gradient direction in units of cells, with 1 unit cell of the divided image region being 5 pixels × 5 pixels To obtain a histogram. For example, when the histogram is formed, the gradient direction is divided into 9 directions of 20 [deg] from 0 [deg] to 180 [deg].

In FIG. 2, the person feature of the person 1 in the frame number 1625 has a gradient strength of 30 in the gradient direction 0 [deg], a gradient strength of 19 in the gradient direction 20 [deg], and a gradient direction of 40 [deg]. It shows that the gradient strength is 17. Similarly, the person feature of the person 2 in the frame number 1173 has a gradient strength of 10 in the gradient direction 0 [deg], a gradient strength of 2 in the gradient direction 20 [deg], and a gradient strength in the gradient direction of 40 [deg]. Is 57.

From the person tracking information D1, the human flow information generation unit 102 recognizes, as a group, detected people whose movement direction and movement speed are close, and generates human flow information D2. Here, the human flow means that when a person who moves at the same speed in the same direction from the person tracked by the person tracking unit 101 and recognizes a person existing within a certain range (neighborhood) as a group, it is one of each group. The movement information in the above frame (group center coordinates, movement direction, movement speed) is represented. The center coordinates of the group are average coordinates of the detection coordinates D1c of the persons included in the group.

FIG. 3 is a diagram illustrating an example of the human flow information D2 output from the human flow information generation unit 102 to the human flow information analysis unit 103. D2a is a group number, which is a number assigned separately to enable identification for different groups. FIG. 3 shows that persons corresponding to person numbers D1a, 1, 4, and 7 are recognized as one group (group number 1). Further, the center coordinate D2c of the group corresponding to the group number 1 is (275, 196). As described above, the human flow information D2 includes the person number D2b of the included person, the center coordinates D2c of the group, the group movement direction D2d, and the group movement speed D2e for each group. The collective movement direction D2d represents an angle of a direction moved between consecutive frames when a specific coordinate direction is set to 0 [deg]. For example, the unit of the collective movement direction D2d is [deg].

The collective movement speed D2e represents the distance of the central coordinates D2c of the collective between successive frames. For example, the unit of the collective movement speed D2e is a moving pixel distance [pixel / frame] in one frame unit. The collective moving speed D2e may be obtained by adding the square of the number of moving pixels in the horizontal direction and the square of the number of moving pixels in the vertical direction during one frame. The collective moving speed D2e may be a value obtained by calculating the square root from the sum of the square of the number of moving pixels in the horizontal direction and the square of the number of moving pixels in the vertical direction during one frame. The distance per pixel can be converted by setting in advance from the positional relationship in which the camera 20 is provided. For example, the distance conversion setting per pixel can be determined from the number of floor tile pixels in the frame image captured by the camera 20 and the actual floor tile distance.

The human flow information analysis unit 103 analyzes the human flow information D2 generated by the human flow information generation unit 102 and generates anomaly occurrence information D3. Also, the incident occurrence information D <b> 3 is output to the suspicious person candidate detection unit 104.

The anomaly occurrence information D3 is the anomaly occurrence frame number D3a when the anomaly occurs when it is determined that an anomaly has occurred in the person flow as a result of analyzing the person flow information D2 by the person flow information analysis unit 103. The coordinates at the time of occurrence of an anomaly are included as the anomaly occurrence coordinates D3b.

A method will be described in which the human flow information analysis unit 103 determines that an anomaly has occurred in the human flow. An anomaly in the human flow indicates a sudden change in the group movement speed D2e in the group being tracked or a change in which the number of persons belonging to the group being tracked decreases rapidly. The former is a case where the time fluctuation amount of the collective movement speed D2e exceeds a set threshold value. For example, when the collective movement speed D2e is rapidly increased, an abnormality occurs in the vicinity of the collective movement speed D2e. it is conceivable that. As an example of the latter, if three people belong to a group and suddenly start moving in different directions, the conditions of “neighboring” or “same direction” for recognizing the group will not be met. The group that belonged to disappears and each is newly tracked as a different group. However, if the group moves out of the angle of view of the camera 20 due to the movement and no longer appears, it is not determined that an abnormality has occurred. That is, the former is a case of running away in the same direction as a group, and the latter is a case of running away so as to be scattered all at once.

Since the human flow information analysis unit 103 detects that an abnormal change has occurred in the human flow by detecting a sudden change, for example, one of the persons belonging to the group starts moving in different directions without changing the speed. If this is the case, basically, this is not judged as an incident. In this case, since the destinations are simply different, there is a high possibility that they have started moving in different directions. On the other hand, when an abnormality occurs, the collective movement speed D2e is considered to become very large at that time and thereafter.

When the person flow information analysis unit 103 determines that there is an abnormality, it generates the abnormality occurrence information D3 and outputs it to the suspicious person candidate detection unit 104. On the other hand, when it is determined that there is no abnormality, the human flow information analysis unit 103 does not output the abnormal occurrence occurrence information D3 to the suspicious person candidate detection unit 104 and waits until the human flow information D2 is updated.

The suspicious candidate detection unit 104 detects the suspicious candidate candidate by analyzing the person tracking information D1 generated by the person tracking unit 101 and the anomaly occurrence information D3 generated by the human flow information analyzing unit 103, and the suspicious candidate information D4 is detected. Generate. Further, the suspicious person candidate detecting unit 104 outputs the suspicious person candidate information D4 to the suspicious person candidate presenting part 105. Here, when the anomaly occurrence information D3 is input from the human flow information analysis unit 103 to the suspicious person candidate detection unit 104, it can be estimated that there has been a notification of an anomaly. Therefore, the suspicious candidate detection unit 104 operates only when the incident occurrence information D3 is input from the human flow information analysis unit 103 to the suspicious person candidate detection unit 104 or when the incident change information D3 is changed. It is good.

The suspicious person candidate information D4 includes a person number D1a assigned in the person tracking information D1 for a person detected by the suspicious person candidate detecting unit 104 as a suspicious person candidate, a detection frame number D1b that is information related to the person, a detection coordinate D1c, The information extracted from the person feature information D1d is included.

The suspicious person candidate detecting unit 104 compares the anomaly occurrence coordinates D3b with the person detection coordinates D1c, and identifies a person existing at a distance within a preset threshold from the anomaly occurrence coordinates D3b as a suspicious person candidate. Alternatively, the suspicious person candidate detection unit 104 compares the anomaly occurrence coordinates D3b and the person detection coordinates D1c, and identifies a person existing at a distance closest to the anomaly occurrence coordinates D3b as a suspicious person candidate. Or the suspicious person candidate detection part 104 specifies only the preset number of persons as a suspicious person candidate in an order from the person close | similar to the anomaly occurrence coordinate D3b.

The preset threshold for distance and threshold for the number of suspicious candidates can be determined arbitrarily. For example, if you want to detect as many suspicious candidates as possible and ultimately identify them by visual inspection of guards, set a threshold to increase the distance, or set a threshold to increase the number of suspicious candidates be able to. When it is desired to narrow down candidates as much as possible, a threshold value can be set to shorten the distance, or a threshold value can be set to reduce the number of people to one.

The suspicious candidate presentation unit 105 inputs the suspicious candidate information D4 generated by the suspicious candidate detection unit 104 and the frame image output from the camera 20, and generates suspicious candidate presentation information D5 on the display 30. Output.

The display 30 inputs and displays the suspicious candidate presentation information D5 generated by the suspicious candidate presentation unit 105.

FIG. 4 is a diagram showing an example of the suspicious candidate presentation information D5 presented on the display 30. As shown in FIG. In FIG. 4, among the persons shown in the frame image output from the camera 20, a person who is detected as a suspicious candidate is displayed in a rectangle. By presenting the information on the display 30 in this manner, it is easy for the security guard to recognize at a glance whether or not there is a suspicious candidate and its position.

Next, the processing flow of the video monitoring apparatus 10 will be described. FIG. 5 is a flowchart showing processing of the video monitoring apparatus 10.

First, when the camera 20 holds a new frame image that has not been subjected to the group tracking process (S001: Yes), the person tracking unit 101 acquires this frame image (S002). If the camera 20 does not hold a new frame image (S001: No), the person tracking unit 101 waits until the camera 20 holds a new frame image.

The person tracking unit 101 acquires a frame image (S002), and then performs a person detection process on the frame image (S003).

When a person can be detected (S004: Yes), the person tracking unit 101 obtains a similarity between the detected person feature of the person and the feature of the person detected in the previous frame. However, if no person is detected in the previous frame, the person tracking unit 101 does not perform this process (S005). If no person can be detected, the person tracking unit 101 waits until the camera 20 holds a new frame image (S004: No).

When a person with a high degree of similarity is detected by comparison with the previous frame, the person tracking unit 101 determines that the person is the same person and generates person tracking information D1 (S006: Yes). If a person with high similarity cannot be detected, the person tracking unit 101 waits until the camera 20 holds a new frame image (S006: No).

The human flow information generation unit 102 receives the person tracking information D1 from the person tracking unit 101, and calculates the moving direction and moving speed of each person being tracked (S007).

The human flow information generation unit 102 further recognizes, as a group, a person who is within a certain range (near) for each person being tracked and corresponds to the calculated moving direction and moving speed. The human flow information D2 including the person number D2b, the central coordinate D2c of the group, the group movement direction D2d, and the group movement speed D2e is generated and transmitted to the human flow information analysis unit 103 (S008).

The human flow information analysis unit 103 receives the human flow information D2 from the human flow information generation unit 102, and determines whether or not an anomaly has occurred in the human flow (S009).

When the human flow information analysis unit 103 determines that there is an abnormality, the human flow analysis unit 13 transmits the abnormality occurrence information D3 to the suspicious person candidate detection unit 104 (S009: Yes). When it is determined that there is no change, the human flow information analysis unit 103 waits until the camera 20 holds a new frame image (S009: No).

The suspicious person candidate detection unit 104 receives the anomaly occurrence information D3 from the human flow information analysis unit 103. Further, the suspicious person candidate detection unit 104 receives the person tracking information D <b> 1 from the person tracking unit 101. Thereafter, the suspicious person candidate detection unit 104 compares the anomaly occurrence coordinate D3b with the person detection coordinate D1c, and identifies a person existing at a distance within a certain threshold from the anomaly occurrence coordinate D3b as a suspicious person candidate (S010). ). Alternatively, the suspicious person candidate detection unit 104 compares the anomaly occurrence coordinates D3b and the person detection coordinates D1c, and identifies a person existing at a distance closest to the anomaly occurrence coordinates D3b as a suspicious person candidate. Alternatively, the suspicious person candidate detection unit 104 specifies a certain number of suspicious person candidates as a suspicious person candidate in order from a person close to the anomaly occurrence coordinate D3b.

Through the above processing, the suspicious person candidate detecting unit 104 generates suspicious person candidate information D4 and transmits it to the suspicious person candidate presenting part 105.

The suspicious person candidate presentation unit 105 receives the suspicious person candidate information D4 from the suspicious person candidate detection unit 104. Further, the suspicious person candidate presenting unit 105 acquires a frame image from the camera 20. Further, the suspicious person candidate presenting unit 105 generates suspicious person candidate presentation information D5 as illustrated in FIG. 5 and transmits it to the display 30, and the display 30 displays it (S011). In FIG. 5, the frame image received from the camera 20 is surrounded by a rectangle of a person who is determined as a suspicious candidate so that the guard can easily recognize the presence or position of the suspicious candidate at a glance. ing.

Thereby, since the video monitoring apparatus 10 according to the present embodiment recognizes a plurality of detected persons as a group and detects suspicious candidate candidates from the change in the flow of the group, the number of persons reflected in the camera video is large. Even then, it can be grasped collectively by the group. From this, the video monitoring apparatus 10 has an effect that it is possible to suppress an increase in processing even if the number of persons shown in the camera video increases.

In addition, the video monitoring device 10 can detect the disturbance of the human flow caused by the behavior of the suspicious person escaping from the incident occurrence point or the surrounding person escaping when the incident occurs. In addition, the video monitoring apparatus 10 can automatically detect suspicious candidate candidates that exist in the vicinity of the point where the abnormality has occurred, and can track the candidate person and display it on the display. For this reason, the video monitoring apparatus 10 according to the present invention can detect a suspicious person who moves based on a suspicious person behavior pattern learned in advance as a suspicious person candidate. Accordingly, the security guard can quickly grasp the position of the suspicious candidate candidate and the escape route from a large amount of video data in the video surveillance device, and thus can perform appropriate security promptly.

The above-mentioned suspicious candidate detection unit 104 detects suspicious candidates from human-flow anomaly information. However, disguise equipment such as masks and sunglasses and weapons are used by image recognition technology using the frame image of the camera. By detecting, equipment information can be used for detection of a suspicious candidate. For example, a person possessing disguise equipment or a weapon can be determined as having a high degree of suspiciousness. In this case, the color of the rectangle in the suspicious person candidate presentation information D5 generated by the suspicious person candidate presentation unit 105 is made more conspicuous for a person with a high suspicious degree. As a result, the guard can grasp the position of the suspicious person more quickly.

In addition, the suspicious person candidate detection unit 104 performs age estimation by an image recognition technique using a frame image of the camera. Thus, the suspicious person candidate detection unit 104 can use the information on the estimated age for detecting the suspicious person candidate. For example, it is considered unlikely that a 10-year-old child is a suspicious person. For this reason, assuming that such a person has a lower suspicious degree, the suspicious person candidate detection unit 104 changes the rectangular color in the suspicious person candidate presentation information D5 to an inconspicuous color. Alternatively, the suspicious person candidate detection unit 104 does not display such a person as a suspicious person candidate. By doing in this way, since a suspicious candidate can be narrowed down, a guard can grasp a true suspicious person's position more quickly.

In addition, the suspicious person candidate detection unit 104 can narrow down the suspicious person candidates from the positional relationship and the movement state of the person. For example, it is considered that a person who is not a suspicious person in the vicinity takes an action of escaping from a suspicious person after the occurrence of an incident. For this reason, among the persons who move in the same direction after the occurrence of an anomaly, there is a high possibility that the person who is further away from the anomaly occurrence point is not a suspicious person. This is because even if a suspicious person pursues a person who is not a suspicious person, the opposite is not possible. By doing in this way, the suspicious person candidate detection part 104 can narrow down a suspicious person candidate. For this reason, the guard can grasp the position of the true suspicious person more quickly.

Further, when the moving speed of the suspicious candidate candidate is equal to or lower than a specific threshold, the suspicious candidate detection unit 104 may determine that the suspicious candidate candidate is not a suspicious candidate because it is highly likely that the escaping action is not taken. By doing in this way, the suspicious person candidate detection part 104 can narrow down a suspicious person candidate. For this reason, the guard can grasp the position of the true suspicious person more quickly.

Embodiment 2. FIG.
FIG. 6 is a block diagram of the configuration of the video monitoring system according to the second embodiment. The video monitoring apparatus 11 is an apparatus that can perform the video monitoring method according to the second embodiment. In FIG. 6, the same or corresponding components as those shown in FIG. 1 are denoted by the same reference numerals as those in FIG. In the video monitoring apparatus 11 according to the second embodiment, the suspicious person candidate detection unit 117 is connected to the abnormal sound detection unit 116. In the video monitoring apparatus 11 according to the second embodiment, the abnormal sound detection unit 116 is connected to the microphone 40. The video monitoring apparatus 11 according to the second embodiment is different from the video monitoring apparatus 10 according to the first embodiment in that abnormal sound detection is used to detect a suspicious candidate. Except for this point, the second embodiment is the same as the first embodiment.

The microphone 40 transmits audio data to the abnormal sound detection unit 116. The microphone 40 may be a single microphone, or a plurality of microphones 40 may be used so that the direction of the sound source can be extracted with high accuracy.

The abnormal sound detection unit 116 acquires audio data from the microphone 40 and analyzes the audio data. For example, the abnormal sound detection unit 116 classifies the voice data from the voice data into “conversation”, “BGM”, “footstep”, “scream”, “anger”, and the like. The abnormal sound detection unit 116 detects a sound highly relevant to the incident, which is different from the life sound such as “scream” or “anger” from the classified sound data. In the analysis of voice data, the frequency (voice pitch) and amplitude (voice volume) can be compared with model data in each classification, and the classification can be discriminated by taking a correlation.

The abnormal sound detection unit 116 detects “scream” and “anger” by the classification of the voice data, and also estimates the direction of the sound source. The sound source direction estimation technology has been actively researched and developed in recent years, and the same method can be adopted. In order to estimate the sound source direction of a plurality of microphones, there is a method of detecting a time difference between audio data collected by each microphone.

The abnormal sound detection unit 116 detects “scream” and “anger”, and transmits the position to the suspicious candidate detection unit 117 as a result of estimating the direction of the sound source. For this position, the correspondence between the estimated position in the microphone 40 and the coordinate system imaged by the camera 20 is investigated in advance, and the position estimated by the microphone is replaced with the position in the camera coordinate system and transmitted. Since the operation and action of the suspicious candidate detection unit 117 are the same as those of the first embodiment, description thereof is omitted.

As described above, in the video monitoring apparatus 11 according to the present embodiment, the position where the abnormal sound has occurred is specified by detecting the abnormal sound. Therefore, even in a crowded state in which a person moves intricately and the variation in the human flow described in the first embodiment cannot be captured by the camera, the video monitoring device 11 is located at the position where the abnormality has occurred. Can be identified, and suspicious candidates can be detected from the surrounding area. This makes it possible for security guards to identify suspicious individuals from a large amount of video data from video surveillance devices and to track their actions, quickly grasp the situation, and execute appropriate security. I can do it.

FIG. 7 is a block diagram showing a modification (video monitoring device 12) of the video monitoring device according to the first and second embodiments of the present invention. 7, components that are the same as or correspond to the components shown in FIG. 1 are given the same reference numerals as those in FIG. A sensing device 50 in FIG. 7 is a modification of the microphone 40 shown in FIG. An anomaly detection unit 126 in FIG. 7 is a modification of the anomaly detection unit 116 shown in FIG. For example, the sensing device 50 is an odor sensor, and the anomaly detection unit 126 is an odor detector. In addition to the example shown in the first embodiment, as a method for detecting anomalies using a frame image from the camera 20 as the sensing device 50, in addition to the example shown in the first embodiment, an area having a remarkably high brightness is detected. There is a method of recognizing and specifying the position. In addition, as a method for detecting anomalies, there is a method for identifying a position by recognizing a fire or explosion by image processing. Furthermore, there is a method of not only using a single sensor as described above but also using a plurality of sensors, detecting an abnormality by combining the results of analysis of the sensor data, and specifying the position. By doing in this way, the video monitoring apparatus 12 can specify the position where the abnormality has occurred with higher accuracy, and can estimate the suspicious person.

The video monitoring apparatus 11 according to the present embodiment may report the detected suspicious candidate information to the security system. In addition, the video monitoring apparatus 11 may transmit information to a facility that collects security information such as a central information center.

10, 11, 12 Video monitoring apparatus 101 Human tracking unit 102 Human flow information generation unit 103 Human flow

information analysis unit

104, 117, 127 Suspicious person information detection unit 105 Suspicious person information presentation unit 116 Abnormal sound detection unit 126 Abnormality detection unit 20 Camera 30 Display 40 Microphone 50 Sensing device

Claims

A person tracking unit for detecting a person from a frame image and tracking each detected person over a plurality of frames;
Based on the tracking result of the person tracking unit, the person who is estimated to be close in moving direction and moving speed is recognized as a group, and human flow information including the group moving direction and the group moving speed is generated for each group. A human flow information generator,
From the human flow information, it is determined whether or not a human flow anomaly has occurred, and a human flow information analysis unit that identifies a position when it is determined that an anomaly has occurred,
Among the persons, a suspicious person candidate detection unit that detects the person existing near the position specified by the human flow information analysis unit as a suspicious person candidate;
A suspicious candidate presentation section for presenting the detected suspicious candidate,
A video surveillance apparatus comprising:
When the person flow information generation unit estimates that the movement direction of each person belonging to the group has taken a different direction among the recognized group, the group disappears from the person flow information and newly Recognize as a different group and generate new human flow information,
The video monitoring apparatus according to claim 1, wherein the human flow information analysis unit determines that an abnormality has occurred when the group disappears from the human flow information.
The video surveillance according to claim 1, wherein the human flow information analysis unit determines that an anomaly has occurred when a temporal variation amount of the collective movement speed exceeds a preset threshold value. apparatus.
The suspicious individual candidate detection unit detects the person who is closest to the position specified by the human flow information analysis unit among the persons as the suspicious individual candidate. The video monitoring apparatus according to claim 1.
The suspicious person candidate detecting unit detects a predetermined number of persons as the suspicious person candidates in the order in which the person is present near the position specified by the human flow information analyzing unit. The video monitoring apparatus according to any one of claims 1 to 4.
The suspicious person candidate detecting unit detects the person existing at a distance within a specific threshold from the position specified by the human flow information analyzing unit among the persons as the suspicious person candidate. The video monitoring apparatus according to any one of 1 to 5.
The suspicious person candidate detection unit, when the movement direction of the plurality of suspicious person candidates is the same around the position specified by the human flow information analysis unit, among the persons, the human flow information analysis unit specified The video monitoring apparatus according to claim 1, wherein the person close to the position is detected as the suspicious person candidate.
The suspicious person candidate detection unit excludes the person whose moving speed is equal to or less than a preset threshold among the persons from the suspicious person candidate. Video surveillance device.
The said suspicious person candidate detection part estimates the age information of the said person from the said video data, and uses it for narrowing down the detected suspicious person candidate, The one described in any one of Claim 1 to 8 characterized by the above-mentioned. Video surveillance device.
It further includes an abnormal sound detection unit that collects voice data to detect a specific sound including a scream or an anger, and estimates a sound source direction of the detected specific sound,
10. The suspicious person candidate detection unit detects the suspicious person candidate using the specific sound detected by the abnormal sound detection unit and the estimated sound source direction. 11. The video surveillance device described in 1.
An anomaly detection unit that detects an anomaly state set in advance and estimates an occurrence position of the anomaly state, further includes,
10. The video monitoring apparatus according to claim 1, wherein the suspicious person candidate detecting unit detects the suspicious person candidate using the occurrence position of the abnormal state. 11.
The video monitoring apparatus according to any one of claims 1 to 11, wherein a detection result of the suspicious person candidate detection unit is reported to a security system.
A person tracking step of detecting a person from the frame image and tracking each detected person over a plurality of frames;
Based on the tracking result of the person tracking step, the person who is estimated to be close in moving direction and moving speed is recognized as a group, and human flow information including the group moving direction and the group moving speed is generated for each group. Human flow information generation step,
From the human flow information, it is determined whether or not a human flow anomaly has occurred, and a human flow information analysis step for specifying a position when it is determined that an anomaly has occurred,
Among the persons, a suspicious person candidate detecting step for detecting the person existing near the position specified by the human flow information analyzing step as a suspicious person candidate;
A suspicious candidate presentation step for presenting the detected suspicious candidate,
A video surveillance method comprising:
It further includes an abnormal sound detection step of collecting sound data to detect a specific sound including a scream or an anger, and estimating a sound source direction of the detected specific sound,
14. The video monitoring method according to claim 13, wherein the suspicious candidate detection step detects the suspicious candidate using the specific sound detected in the abnormal sound detection step and the estimated sound source direction.