JP2020136855A - Monitoring system, monitor support device, monitoring method, monitor support method, and program - Google Patents

Abstract

To suppress search leak even when person search is difficult in a search target region.SOLUTION: A monitoring system for monitoring a person on the basis of video imaged by a plurality of cameras installed at different positions includes: responsible camera determination means for determining at least one camera out of the plurality of cameras on the basis of installation information on the plurality of cameras and a target region set as a search target, as a responsible camera in charge of imaging a peripheral region determined by taking the target region as a reference; and camera control means for controlling an imaging direction of the responsible camera on the basis of video of the peripheral region imaged by the responsible camera.SELECTED DRAWING: Figure 4

Description

The present invention relates to a monitoring system, a monitoring support device, a monitoring method, a monitoring support method and a program.

With the spread of surveillance camera systems in recent years, the number of cases of shooting people in various places is increasing. The surveillance camera system not only predicts danger and detects the occurrence of an incident in real time, but also comprehensively searches for suspects, accomplices, lost children, and missing persons by viewing the accumulated images. It can be carried out. Face matching technology and person matching technology are being studied as a technology for searching a target at high speed with less leakage from a large amount of accumulated images. In each technique, feature information for collating the same person is extracted and accumulated from the images shot in advance, and compared with the feature information similarly extracted from the images of the specified search target person. Then, it is determined whether or not they are the same person.

In the conventional human search, it is necessary to extract the searchable features from the video. However, in a situation where people are crowded in the area to be searched and it is not possible to shoot an image corresponding to the required feature amount, it is not possible to perform a person search with high accuracy. On the other hand, in Patent Document 1, when the number of persons detected in the captured image captured by the first camera is larger than a predetermined value, the imaging area of the first camera is set to the second camera and the second camera. A technique for sharing and shooting with the three cameras is disclosed. As a result, a plurality of persons can be efficiently monitored.

Japanese Unexamined Patent Publication No. 2005-117542

However, when the degree of congestion in the search target area is high, it may not be possible to extract the required feature amount even if it is shared by a plurality of cameras. In addition, it may be difficult to photograph the search target area because a camera for photographing the search target area is not installed.

The present invention has been made in view of such problems, and an object of the present invention is to reduce omission of search even when it is difficult to search for a person in a search target area.

Therefore, the present invention is a monitoring system that monitors a person based on images taken by a plurality of cameras installed at different positions, and the installation information of the plurality of cameras and a target set as a search target. Based on the area, at least one camera from the plurality of cameras is determined by the responsible camera determining means for determining the camera in charge of shooting the peripheral area determined with reference to the target area, and the responsible camera. It is characterized by having a camera control means for controlling the shooting direction of the camera in charge based on the captured image of the peripheral region.

According to the present invention, it is possible to reduce search omissions even when it is difficult to search for a person in a search target area.

It is an overall block diagram of the monitoring system which concerns on 1st Embodiment. It is a hardware configuration diagram of a monitoring support device. It is a functional block diagram of a camera and a monitoring device. It is a sequence diagram which shows the monitoring support processing. It is a figure which shows an example of a monitoring area. It is a figure which shows an example of a monitoring area. It is a figure which shows an example of a monitoring area. It is a functional block diagram of the monitoring support device which concerns on 4th Embodiment. It is a figure which shows an example of a monitoring area.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First Embodiment)
FIG. 1 is an overall configuration diagram of a monitoring system according to the first embodiment. The monitoring system includes a plurality of cameras 100, a monitoring device 110, and a monitoring support device 120. The monitoring device 110 can communicate with the plurality of cameras 100, and can also communicate with the monitoring support device 120. It is assumed that the monitoring device 110, the camera 100, and the monitoring support device 120 are connected by, for example, a Local Area Network (LAN) or the like.

The plurality of cameras 100 are installed in the monitoring area, and the images obtained by each camera 100 are transmitted to the monitoring device 110. The monitoring device 110 controls each camera 100 and stores an image of each camera 100. The monitoring device 110 also searches for a person in the image of the camera 100. The monitoring support device 120 acquires an image from the monitoring device 110, and performs processing for supporting a person search based on the image.

FIG. 2 is a hardware configuration diagram of the monitoring support device 120. The monitoring support device 120 includes a CPU 201, a ROM 202, a RAM 203, an external storage device 204, an input device interface 205, and an output device interface 206. The CPU 201 controls the entire monitoring support device 120. The ROM 202 stores programs and parameters that do not need to be changed. The RAM 203 temporarily stores programs and data supplied from the external storage device 204 and the like. The external storage device 204 is a hard disk drive (HDD) or flash memory drive (SSD) fixedly installed on the camera 100, an optical disk detachable from the camera 100, a magnetic or optical card, an IC card, a memory card, or the like. .. The external storage device 204 can make the processing result persistent and store a large amount of data.

The input device interface 205 is an interface with an input device 208 such as a pointing device or a keyboard that inputs data in response to a user operation. The output device interface 206 is an interface with an output device 209 such as a monitor for outputting the execution result of the program of the camera 100. Reference numeral 207 is a system bus for communicably connecting each unit of the camera 100. The hardware configuration of the monitoring device 110 is the same as the hardware configuration of the monitoring support device 120 described with reference to FIG.

FIG. 3 is a functional configuration diagram of the camera 100 and the monitoring device 110. First, the monitoring device 110 will be described. The monitoring device 110 includes a communication processing unit 301, a video database (DB) 302, a person search unit 303, and a camera control unit 304.

The communication processing unit 301 controls communication with an external device. The video DB 302 stores the video received from each camera 100. The person search unit 303 searches for a person in the video stored in the video DB 302. For example, the person search unit 303 extracts a person portion from the image captured by the camera 100, and vectorizes characteristic elements such as color components, edge information, direction, and movement from the extracted person image (feature vector). ) Is generated and this vector is accumulated in advance. Then, the person search unit 303 uses the video to be searched and the accumulated vector to perform a person search by a distance neighborhood search after accumulating the images. In recent years, a method using deep learning has been widely used for generating a feature vector from a human image, and such a method may be applied for extraction. The person search unit 303 may perform the person search process, and the specific process in the person search process is not limited to the embodiment.

The camera control unit 304 controls pan / tilt / zoom (hereinafter referred to as PTZ) according to the control performance of each camera 100. It is assumed that the camera control unit 304 can identify a plurality of cameras 100 and can control each camera 100 to a desired state. The camera control unit 304 acquires a control instruction for a predetermined camera 100 from the monitoring support device 120 via the communication processing unit 301, and controls the camera 100 related to the control instruction according to the control instruction.

Further, it is assumed that the setting information 305 is stored in the storage unit such as the external storage device 204 of the monitoring device 110. Here, the setting information 305 includes map information and installation information. The map information is information indicating a map including positions where a plurality of cameras 100 are installed. In the map information, not only the camera 100 but also the positions of roads, buildings, and the like are shown. The installation information is information in which the identification information of each camera 100 is associated with the installation location and control information. Here, the installation location is a coordinate value indicating a position on the map. The control performance information is information indicating an angle or direction in which the shooting direction of the corresponding camera 100 is known, and information indicating performance related to control such as PTZ.

Next, the monitoring support device 120 will be described. The monitoring support device 120 includes a communication processing unit 311, a target area setting unit 312, a situation determination unit 313, a peripheral area setting unit 314, a responsible camera determination unit 315, a human flow analysis unit 316, and a shooting direction determination unit 317. And have. The communication processing unit 311 controls communication with an external device. The target area setting unit 312 sets the target area and time zone for the person search according to the user operation via the input device 208. Hereinafter, the area set by the target area setting unit 312 is referred to as a target area.

The situation determination unit 313 determines whether or not the target area set by the target area setting unit 312 is a search difficult area according to a predetermined condition. Here, the search difficult area is an area in which it is difficult to obtain an image suitable for a person search. In the present embodiment, the situation determination unit 313 determines whether or not the target area is a difficult search area according to the condition that the target area is determined to be a difficult search area when the camera 100 capable of photographing the target area is not installed. It shall be judged. The situation determination unit 313 acquires the setting information 305 from the monitoring device 110 via the communication processing unit 311 and refers to the setting information 305 in determining whether or not the area is difficult to search.

Further, the conditions referred to by the situation determination unit 313 are not limited to the embodiment, and can be freely set by the administrator or the like. For example, although there is a camera 100 capable of photographing the target area, it is conceivable that the camera 100 may not be able to photograph normally due to the influence of an incident, an accident, or a malfunction of the camera 100 itself. Corresponding to this, the situation determination unit 313 determines that the target area is a difficult shooting area even when there is a camera 100 capable of shooting the target area, but the camera 100 cannot normally shoot. May be good.

Further, when the degree of congestion (density) of people in the target area is high, it is difficult to obtain an image of each person from the image of the target area. Correspondingly, the situation determination unit 313 may refer to the condition that the target area is determined to be a difficult search area when the density of the target area is equal to or higher than the threshold value.

The peripheral area setting unit 314 sets the peripheral area to be the target of the person search instead of the target area based on the target area. In the present embodiment, the peripheral region is a region having a radius of a predetermined distance centered on the target region. However, the peripheral area may be an area determined based on the position of the target area, and is not limited to the embodiment. That is, the peripheral area does not necessarily have to include the target area.

When it is determined that the target area is a difficult-to-search area, the responsible camera determination unit 315 determines the responsible camera from among the plurality of cameras 100. Here, the responsible camera is a camera that can be in charge of shooting the peripheral area. Specifically, the responsible camera determination unit 315 refers to the setting information and determines the camera installed in the peripheral area as the responsible camera. As another example, the responsible camera determination unit 315 may determine the camera 100 whose shooting range controlled by the PTZ includes at least a part of the peripheral region as the responsible camera.

The human flow analysis unit 316 acquires the video captured by the camera in charge from the monitoring device 110 via the communication processing unit 311. Then, the person flow analysis unit 316 analyzes the image taken by the camera in charge and specifies the direction and amount (congestion degree) of the person flow. The human flow analysis unit 316 is, for example, a machine that prepares a large number of images in which the image and the direction and amount of the human flow are associated with each other, and trains a recognition model that estimates the direction and amount of the human flow when the image is input. Identify the direction and amount of people flow by learning methods.

The process for specifying the direction and amount of the flow of people is not particularly limited. As another example, the human flow analysis unit 316 may measure the distance of a person by measuring the reflected wave according to the irradiation of the laser, and obtain the direction and amount of the human flow from the change in distance. As another example, when a sensor for detecting a walking state is installed on the floor surface of the monitoring range, the human flow analysis unit 316 acquires the detection result of this sensor, and the direction of the human flow is obtained from the detection result. And the amount may be specified.

The shooting direction determination unit 317 determines the shooting direction of each camera in charge based on the direction and amount of the flow of people. The processing of the shooting direction determination unit 317 will be described in detail later. When the shooting direction of each responsible camera is determined by the shooting direction determination unit 317, the identification information of each responsible camera and the information in which the shooting direction is associated with each other are transmitted to the monitoring device 110 as a shooting direction notification. The camera control unit 304 of the monitoring device 110 receives the shooting direction notification via the communication processing unit 301, and performs PTZ control of each camera in charge according to the shooting direction notification.

FIG. 4 is a sequence diagram showing monitoring support processing by the monitoring device 110 and the monitoring support device 120. FIG. 5 is a diagram showing an example of a monitoring area corresponding to the setting information stored in the monitoring device 110. As shown in FIG. 5, the monitoring area is an area including a road, and a plurality of cameras 100 are installed.

In S400, the target area setting unit 312 of the monitoring support device 120 sets the target area to be the target of the person search. Specifically, the monitoring support device 120 acquires the map information included in the setting information stored in the monitoring device 110 in response to the request from the user, and displays this on the output device 209. Then, the user inputs a point or a range on the map in response to a user operation via the input device 208. The target area setting unit 312 sets the target area according to the information input according to the user operation. At this time, the user may specify the point to be monitored without being aware of the installation status of the camera 100. Further, when it is desired to limit the time zone of the search target, the target area setting unit 312 also sets the target time zone in association with the target area.

Next, in S401, the situation determination unit 313 determines whether or not the target area set in S400 is a search difficult area. This process is an example of the area determination process. For example, in the situation shown in FIG. 5, it is assumed that the area 501 is set as the target area and the area 501 cannot be photographed from any of the cameras 100. In this case, the situation determination unit 313 determines that the target area is a difficult search area. When the status determination unit 313 determines that the target area is a difficult-to-search area (YES in S401), the status determination unit 313 proceeds to the process in S402. When the status determination unit 313 determines that the target area is not a difficult-to-search area (NO in S401), the status determination unit 313 ends the monitoring support process. When the monitoring support process is completed, each camera 100 shoots a video in a preset shooting range, and the obtained video is stored in the video DB 302 of the monitoring device 110.

In S402, the peripheral area setting unit 314 sets a region having a radius of a predetermined distance as the peripheral region centered on the target region set in S400. In the example of FIG. 5, a region 502 having a radius of a predetermined distance centered on the target region 501 is set as a peripheral region. Next, in S403, the responsible camera determination unit 315 determines the camera 100 installed in the peripheral area as the responsible camera. In the example of FIG. 5, six cameras 100 of cameras A to F are determined as responsible cameras. Since the camera G is installed outside the peripheral area 502, it is removed from the camera in charge. Regarding the installation position of each camera 100, the responsible camera determination unit 315 acquires setting information from the monitoring device 110 via the communication processing unit 311 and refers to this.

Next, in S404, the responsible camera determination unit 315 selects one responsible camera whose shooting direction has not been determined as the processing target. Next, in S405, the human flow analysis unit 316 acquires the image captured by the selected camera selected in S303 from the monitoring device 110 via the communication processing unit 311. Next, in S406, the human flow analysis unit 316 specifies the direction and amount of the human flow in the acquired video. For example, suppose a crime occurs in the target area. In this case, it is conceivable that the surrounding persons move in the direction away from the target area. It is also possible that the criminal also moves away from the target area. Therefore, for example, from the image of the camera A, the direction of the flow of people as shown by the arrow X1 is specified. Further, for example, from the image of the camera C, the direction of the flow of people as shown by the arrow X2 is specified.

Next, in S407, the shooting direction determination unit 317 determines the shooting direction of the camera in charge being selected based on the direction of the flow of people, that is, the moving direction of the person. Next, in S408, the communication processing unit 311 transmits a shooting direction notification including the shooting direction determined in S407 and the identification information of the corresponding camera in charge to the monitoring device 110. When the monitoring device 110 receives the shooting direction notification, in S410, the camera control unit 304 performs PZT control so that the camera in charge indicated in the shooting direction notification faces the shooting direction. As a result, the camera in charge captures the image in the shooting direction determined in S407, and the image captured by the camera in charge is stored in the image DB 302 of the monitoring device 110. In S410, the camera control unit 304 further associates the identification information of the camera in charge with the information indicating that the camera is in the control state and stores it in the storage unit. When the camera control is completed, the monitoring device 110 transmits a camera control completion notification to the monitoring support device 120.

Here, the processing of S407, S408, and S410 will be described in detail. It is assumed that only one person flow is detected from the image of the camera in charge. In this case, the shooting direction determination unit 317 sets the camera direction so as to face this human flow. For example, when an incident or an accident occurs in the area 501 of FIG. 5, it is conceivable that people who are densely packed in the area 501 diffuse radially around the area 501 as shown by an arrow 508. In such a case, the direction facing the human flow is determined as the shooting direction of each camera in charge.

In the example of FIG. 5, the camera A installed in the direction shown by the broken line is pan-controlled in S410 so that the shooting direction is changed as shown by the solid line. Similarly, in S410, the camera C is pan-controlled so that the shooting direction is changed from the direction shown by the broken line to the direction shown by the solid line. As a result, even if the criminal who maliciously caused an accident or incident at a crowded point or a related person escapes due to crowds, the image of the camera in charge whose shooting direction is controlled shows the front of the criminal. The possibility of capturing an image increases. Furthermore, since the face image of the search target person such as a criminal may exist as a blacklist, the possibility of detecting such a search target person from the image of the camera in charge increases.

In addition, it is assumed that an incident or accident occurs in the area 501, and while the rubberneckers gather from the surroundings in the area 501, the criminal escapes from the area 501. In the case of dealing with such a case, the shooting direction determination unit 317 determines the shooting direction of each responsible camera so that it can shoot in two directions, the direction of the flow of people and the direction facing the person. For example, paying attention to the direction of the human flow as shown by the arrow X1, the shooting direction of the camera A may be the direction facing the human flow X1, and the shooting direction of the camera F may be the direction X1 of the human flow. As described above, the shooting direction may be determined according to the direction of the flow of people, and the specific conditions for determining the shooting direction are not limited to the embodiment.

In the present embodiment, the photographing direction determination unit 317 determines that the human flow is the main human flow when the amount of the human flow is equal to or more than the threshold value, and when the human flow is the main human flow, as described above. , The shooting direction shall be determined based on the direction of the flow of people. That is, when the amount of the flow of people is less than the threshold value, the process of determining the shooting direction according to the direction of the flow of people is not performed. This makes it possible to prioritize the shooting of important people.

In the monitoring support device 120, after the processing of S408, in S409, the responsible camera determination unit 315 determines whether or not the processing of S405 to S408 has been completed for all the responsible cameras. The responsible camera determination unit 315 completes the monitoring support process when the processes for all the responsible cameras are completed (YES in S409). If there is an unprocessed camera in charge (NO in S409), the camera in charge determination unit 315 proceeds to process in S404. In this case, in S404, the unprocessed camera in charge is selected, and the processing after S405 continues.

By the above processing, the shooting direction of the camera in charge is appropriately changed. Then, shooting is performed by the camera in charge controlled so as to face the front of the person, and the video shot by the camera in charge is stored in the video DB 302. It should be noted that the image taken by the camera in charge is stored in association with the position information of the target area.

If a time zone for the target area is set, the shooting in the shooting direction determined in S407 is continued until this time zone elapses, and ends when the time zone elapses. Then, when the time zone elapses, the camera control unit 304 returns the shooting direction of each responsible camera to the default direction.

Next, a case where a search with the target area set in the monitoring support process as the target area is executed will be described. The user inputs an area (referred to as a target area) desired to be searched and a time zone desired to be searched via the input device 208 of the monitoring device 110. When such information is input, the person search unit 303 of the monitoring device 110 extracts the image by the camera in charge associated with the designated target area from the image DB 302, and performs a person search in the extracted image.

The camera in charge is an image whose shooting direction is in the direction opposite to the direction of the flow of people. Therefore, in the image taken by the camera in charge, the chances of a person facing the front are increasing as compared with before the shooting direction is controlled. Therefore, when a query of a front-facing person is input, the possibility of successful matching of the person can be improved. Further, when the monitoring device 110 of the present embodiment can search for a video that matches the query, the monitoring device 110 presents the search result to the user while indicating that the video is an indirectly detected person video. As a result, the user can obtain good search results in the peripheral area as a substitute for the search results in the target area.

As described above, when it is difficult to photograph the target area of the person search, the monitoring support device 120 according to the present embodiment sets the peripheral area to indirectly search for the person in the target area, and sets the peripheral area in the peripheral area. The camera to be located is determined as the camera in charge of shooting the peripheral area. Then, the image of the peripheral area taken by the camera in charge is accumulated in the image DB 302. This makes it possible to search for the target person by matching with the image of the target person obtained from another camera or eyewitness testimony. That is, the search for the target person in the peripheral area makes it possible to indirectly search for the target person in the target area, and even when it is difficult to search for the target person in the search target area, it is possible to reduce the omission of the search.

Furthermore, since the witness can easily confirm it, the front image of the target person is often used as a search condition. In addition, many of the suspects' blacklists include images including the front. Corresponding to these, in the embodiment, the camera is controlled so that the responsible camera captures a front image of the person. This makes it possible to increase the chances of matching even for general search conditions.

A first modification of the first embodiment will be described. In the present embodiment, the camera 100 and the monitoring device 110 are different devices, but the camera 100 and the monitoring device 110 may be configured as one device. That is, the monitoring device 110 may have the functional configuration of the monitoring support device 120 shown in FIG. Further, as another example, each function of the camera 100 and the monitoring device 110 may be provided by any device constituting the monitoring system, and the device having each function is not limited to the embodiment. Absent. Further, the monitoring system may perform up to the accumulation of the image taken by the camera or the feature amount extracted from the image, and the person search may be performed by another device.

Next, a second modification will be described. The shooting direction determination unit 317 may determine the shooting direction of each camera in charge based on the flow of people, and the specific shooting direction is not limited to the embodiment. For example, there are cases where a plurality of similar people flow appear in the video. In this case, the shooting direction determination unit 317 determines the shooting direction so as to repeat the pan so as to face each of the plurality of people. For example, by repeating panning at regular intervals, it is possible to shoot in a direction facing each of a plurality of people who can be shot by one camera in charge.

For example, suppose that information suggesting the possibility of an incident or accident occurring in the area 501 of FIG. 5 is obtained. In this case, the responsible camera determination unit 315 determines the surrounding cameras as the responsible camera regardless of the presence or absence of the camera corresponding to the area 501. Then, the shooting direction determination unit 317 determines the shooting direction of each camera in charge so as to shoot both the person moving in the direction away from the area 501 and the person moving in the direction toward the area 501. As a result, the monitoring device 110 can monitor the entry and exit of a person from the image of the camera in charge into the area 501. This makes it easier to specify, for example, the time when a person enters the area 501 and the time obtained from the area 501, and in this case, the information is more useful than when only one of the times can be specified. In this case, in order to clarify the entry and exit of a person, the video DB 302 stores each video so that it can be identified whether the flow is toward the area 501 or the flow coming out of the area 501. To do. When the person search unit 303 is instructed to search the area 501, the person search unit 303 may identify the entry and exit of the same person into the area 501 and present the search result.

Further, as a third modification, the monitoring support device 120 may control not only the shooting direction of the camera in charge but also the zoom. As a result, for example, the camera in charge can take an image in which the face of the person has a ratio of the face of the person in the entire image to be a predetermined value or more.

(Second Embodiment)
The difference between the monitoring system according to the second embodiment and the monitoring system according to the first embodiment will be mainly described. In the monitoring support device 120 according to the second embodiment, the target area setting unit 312 analyzes the direction and amount of the human flow based on the image stored in the image DB 302, and analyzes the direction and amount of the human flow, and the abnormality in the direction of the human flow and the human flow. Detect quantity anomalies. Then, the target area setting unit 312 automatically sets the target area according to the detected abnormality.

More specifically, the target area setting unit 312 stores the direction and amount of the flow of people in the video for each unit period, for example, every 5 minutes. In the target area setting unit 312, when the difference between the direction of the human flow in the last obtained unit period and the direction of the human flow in the unit period obtained immediately before that is equal to or more than the threshold value, that is, the direction of the human flow has changed. In some cases, it is detected as an abnormality in the direction of human flow. Then, when the target area setting unit 312 detects an abnormality in the direction of the human flow, the target area setting unit 312 sets the area that is the starting point of the human flow as the target area. The process of detecting a change is not limited to the embodiment. As another example, when the difference between the direction of the flow of people in the last obtained unit period and the average value of the direction of the flow of people in the predetermined period immediately before that is equal to or more than the threshold value, the target area setting unit 312 is used. Anomalies in the direction of human flow may be detected.

Further, the target area setting unit 312 detects as an abnormality in the amount of human flow when the difference between the amount of human flow in the last obtained unit period and the amount of human flow in the unit period obtained immediately before it is equal to or more than the threshold value. To do. Then, when the target area setting unit 312 detects an abnormality in the amount of human flow, the target area setting unit 312 sets the area in which the abnormality in the amount of human flow occurs as the target area.

Large changes in the direction and volume of traffic over a short period of time are likely to be due to some anomaly. Therefore, in the second embodiment, the target area is automatically set according to the change in the direction and amount of the flow of people in this way. This process is the process of S400 in the monitoring support process described with reference to FIG. The processing after S401 is the same processing as described in the first embodiment.

Further, the size of the target area when the target area setting unit 312 automatically sets the size is assumed to be predetermined. As another example, the size of the target area may be determined according to the degree of change in the direction and amount of the flow of people and the certainty of the position of the target area determined from the direction of the flow of people. For example, depending on the structure of the road and the position of the camera, a local area may not be determined as the starting point from the direction of the flow of people. In such a case, the size of the target area is increased as compared with the case where the local area is determined. As a result, it is possible to prevent the location where the abnormality has occurred from being excluded from the monitoring target.

For example, in the example of FIG. 6, it is assumed that there is a change in direction in the human flow 602. In this case, the target area setting unit 312 sets the area 601 that is the starting point of these human flows 602 as the target area. The other configurations and processes of the monitoring system according to the second embodiment are the same as the configurations and processes of the monitoring system according to the first embodiment. As described above, the monitoring support device 120 according to the second embodiment can automatically set the target area based on the image in the monitoring area.

Next, a first modification of the second embodiment will be described. The process for automatically setting the target area is not limited to the embodiment. As another example, for example, the target area setting unit 312 may set the target area according to the number of people (congestion degree) instead of the flow of people in the video. Specifically, the target area setting unit 312 first specifies the number of people. For example, when the camera 100 is installed at a high place, the number of persons may be calculated by counting the heads of persons with relatively few hidden objects. Then, when the change in the number of people exceeds the threshold value, the target area setting unit 312 may set the changed area as the target area. Further, as another example, the number of people may be counted by a number-of-persons approximate number counter in which a large number of images in which a captured crowded image and an approximate number of people are associated with each other are associated with each other by machine learning.

Further, as a second modification, the target area setting unit 312 may sample an area in which a person exists in the surveillance image and determine the overlap of the person in the sampled area. Then, the target area setting unit 312 may set the sampled area as the target area when the degree of overlap is equal to or greater than the threshold value. The target area setting unit 312 detects each person's part feature in the person's image, and determines the overlap based on the amount of the part that could not be detected. As another example, if the camera can obtain depth information, the target area setting unit 312 performs segmentation according to the context of the person in the image based on the distance information for each pixel, and determines the overlap. You may go.

(Third Embodiment)
The difference between the monitoring system according to the third embodiment and the monitoring system according to the other embodiment will be mainly described. It is also conceivable that the image referred to in the person search is an image in a direction other than the front of the person. In such a case, it is desirable not only to increase the chances of shooting the image of the person facing the person, but also to acquire the image of one person captured from various directions. On the other hand, in the monitoring system according to the third embodiment, a first region closer to the target region and a second region having a distance from the target region farther than the first region are set as peripheral regions. Then, the images of the camera in charge corresponding to each area are accumulated. Further, the monitoring system of the third embodiment determines the shooting direction so as to shoot a person image in a different direction for each area.

For example, in the example of FIG. 7, the peripheral area setting unit 314 sets the first area 702 and the second area 703 as the peripheral area of the target area 701. Further, the camera determination unit 315 in charge determines the shooting direction facing the human flow for the cameras H to L located in the first area 702, and determines the shooting direction of the cameras M to O in the second area 703 for the human flow. Determine the shooting direction so that the side and back are shot. The other configurations and processes of the monitoring system according to the third embodiment are the same as the configurations and processes of the monitoring system according to the other embodiments. As described above, according to the monitoring system according to the third embodiment, it is possible to capture a person image in a plurality of directions in the peripheral area. As a result, search omission can be reduced.

A first modification of the third embodiment will be described. When the target area is congested, the number of people who are less closely related to the target area increases as the distance from the target area increases, but the flow of people is considered to be sparse. Therefore, the peripheral area setting unit 314 may perform a process opposite to the process described in the third embodiment. That is, the peripheral area setting unit 314 determines the shooting direction such that the side surface and the back surface of the human flow are photographed with respect to the camera in the first area 702, and the front surface of the human flow is photographed with respect to the camera in the second area 603. The shooting direction may be determined.

For example, when a point that is constantly monitored is gradually congested and it becomes temporarily difficult to monitor, control is performed so as to match the monitoring purpose of the monitoring point. When it is not possible to predict what kind of search condition image will be input later, the monitoring system controls to shoot images from a plurality of directions as described in the third embodiment. Do.

A second modification will be described. In the third embodiment, what kind of shooting direction the monitoring support device 120 determines according to the flow of people is determined in advance. However, it may be determined dynamically according to the flow of people, or it may be arbitrarily set or changed according to the user operation. Further, it may be possible to arbitrarily set and change whether to shoot in only one direction or to shoot in a plurality of directions as described above according to the user's operation. In this way, by making the conditions for camera control variable, it is possible to improve the opportunity for searching for a person depending on the situation.

(Fourth Embodiment)
Next, the difference between the monitoring system according to the fourth embodiment and the monitoring system according to the other embodiment will be mainly described. As described in the first embodiment and the second embodiment, even if the image of the peripheral area is accumulated by performing the camera control, the overlap and movement of the person in the image, leakage in the person detection process, etc. As a result, the information obtained varies from person to person. In such a case, the monitoring support device 120 according to the third embodiment gives priority to obtaining information from a person who has obtained less information than a person who has already obtained much information. Control the shooting direction of the camera.

FIG. 8 is a functional configuration diagram of the monitoring support device 800 according to the fourth embodiment. The monitoring support device 800 according to the third embodiment has a person management unit 801, a person DB 802, and a shooting opportunity determination unit 803, in addition to the functional configuration of the monitoring support device 120 of the first embodiment. ing.

The person management unit 801 acquires the image captured by the responsible camera determined by the responsible camera determination unit 315 from the monitoring device 110 via the communication processing unit 311. Then, the person management unit 801 manages the person existing in the peripheral area by extracting the feature amount of the person from the video, associating it with the identification information of the person and accumulating it in the person DB 802. The information stored in the person DB 802 is characteristic information for determining the identity of the person. The person management unit 801 determines the identity of the person in the image taken by the camera in charge and the person stored in the person DB 802 based on the feature amount, and when the person is not the same person as the person stored in the person DB 802. In addition, the person obtained by the camera in charge is selected as the person to be managed.

Then, the shooting direction determination unit 317 determines the shooting direction so as to give priority to the management target person for shooting. As a result, the camera control unit 304 performs camera control so as to preferentially shoot the managed person. Since the purpose of the camera control in this case is to increase the information of the person whose amount of information is insufficient, for example, the tracking is continued until the whole body image of the managed person can be taken. Further, as another example, the camera control unit 304 may take a zoom image of the periphery of the person to be managed. As a result, it is possible to increase the information that matches the image that is the search condition.

For example, it is assumed that the camera 901 (broken line) shown in FIG. 9 is controlled so as to face the main flow of people 903. At this time, if the person 904 is newly photographed and it is found that the person 904 does not have a sufficient feature amount, the photographing direction may be controlled so as to track the person 904 like the camera 902.

Further, the shooting opportunity determination unit 803 determines whether or not there is a shooting opportunity for the person determined to be the management target person. The shooting direction determination unit 317 determines a shooting direction for tracking the person to be managed when the shooting opportunity determination unit 803 determines that there is no shooting opportunity. As a result, the managed person is tracked only when there is no shooting opportunity. As a result, it is not necessary to control the camera according to the management target person for all the persons determined to be the management target person, so that the processing cost can be reduced.

For example, the person 904 in FIG. 9 is moving in a direction away from the peripheral area 900, and is a person newly photographed by the camera 902. From this, it is considered that there are few shooting opportunities after that. In this way, the shooting opportunity determination unit 803 may determine the presence or absence of a shooting opportunity based on the image shot by the camera in charge. Further, for example, when a person who has moved into the peripheral area 900 from the outside of the peripheral area 900 is detected as in the direction 905, it is considered that this person has little relationship with the target area. Therefore, the monitoring system controls this person not to be tracked. As a result, it is possible to narrow down the target of the person matching process that appears within the range of the related cameras in this embodiment, and it is possible to suppress the processing cost.

It should be noted that the process related to the determination of the shooting opportunity is not essential. When the processing related to the shooting opportunity is not performed, the monitoring support device 800 may determine the shooting direction so as to always shoot the managed person when the managed person is detected. In this way, by controlling the camera so as to make up for the shortage according to the amount and content of the acquired information of the person being photographed, the amount of information per person can be increased. The effect of improving opportunities can be expected.

(Fifth Embodiment)
Next, the monitoring system according to the fifth embodiment will be mainly described as being different from the monitoring system according to the other embodiments. In the present embodiment, the peripheral area setting unit 314 of the monitoring support device 120 adjusts at least one of the position and size of the peripheral area based on the image of each camera selected as the responsible camera.

For example, in the example of FIG. 5, it is assumed that the flow rate of people in the image of the camera A in charge is higher than the flow rate of people in the images of the other cameras B to F by a predetermined ratio or more. In this case, the peripheral area setting unit 314 changes the position of the peripheral area by shifting the center position of the peripheral area in the direction of the camera A in charge. Further, as another example, in this case, the peripheral area setting unit 314 increases the distance from the target area to the outer boundary of the peripheral area only in the direction in which the responsible camera A is located, thereby increasing the size of the peripheral area. May be changed. Further, as another example, when the area without roads on the map is included in the peripheral area, the area without roads may be excluded from the peripheral area. When the peripheral area is changed as described above, the responsible camera determination unit 315 redetermines the responsible camera. As a result, the number of cameras in charge can be increased in areas where the flow rate of people is high. The other configurations and processes of the monitoring system according to the fifth embodiment are the same as the configurations and processes of the monitoring system according to the other embodiments.

When the flow rate of people is large, there is a high possibility that some people will not be able to shoot with just one camera. On the other hand, in the present embodiment, since it is possible to increase the number of cameras that capture an area with a large flow rate of people, it is expected that the number of search features that can be acquired will increase.

(Sixth Embodiment)
Next, the monitoring system according to the sixth embodiment will be mainly described as being different from the monitoring system according to the other embodiments. This monitoring system has a main camera for constantly monitoring a predetermined area of the monitoring area and an auxiliary camera for functioning as a responsible camera when the target area is set as described in the above embodiment. And, respectively. As a result, even when the auxiliary camera starts shooting in the peripheral area corresponding to the target area, the main camera can maintain the preset monitoring state and accumulate the image. The other configurations and processes of the monitoring system of the sixth embodiment are the same as the configurations and processes of the monitoring system according to the other embodiments.

Further, as a modification of the sixth embodiment, for example, the main camera is determined as the responsible camera, and is controlled so as to face the shooting direction determined by the shooting direction determining unit 317, and the auxiliary camera corresponds to this. May be controlled to shoot in a direction different from that of the main camera. This makes it possible to photograph the flow of people from a plurality of directions. In addition, it is possible to photograph the front of a plurality of people. As a result, it can be expected that the number of search features that can be acquired will increase.

An object of the present invention is to supply a recording medium on which a program code of software that realizes the functions of the embodiment is recorded to a device, and the computer, CPU, or MPU of the device reads and executes the program code stored in the recording medium. It is also achieved by that. In this case, the program code itself read from the recording medium realizes the function of the above-described embodiment, and the recording medium on which the program code is recorded constitutes the present invention.

As a recording medium for supplying the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a non-volatile memory card, a ROM, a DVD, or the like can be used. it can.

In addition, it goes without saying that the functions of the examples may be realized by performing actual processing by the Operating System (OS) or the like running on the computer based on the instruction of the program code read by the computer. No.

Further, the program code read from the recording medium may be written in the memory provided in the function expansion board inserted in the computer or the function expansion unit connected to the computer. Based on the instruction of the program code, the function expansion board or the CPU provided in the function expansion unit may perform a part or all of the actual processing, and the processing may realize the function of the above-described embodiment. Needless to say.

Although the preferred embodiment of the present invention has been described in detail above, the present invention is not limited to the specific embodiment, and various modifications are made within the scope of the gist of the present invention described in the claims.・ Can be changed.

100 Camera 110 Monitoring device 120 Monitoring support device

Claims (19)

A surveillance system that monitors people based on images taken by multiple cameras installed at different locations.
Based on the installation information of the plurality of cameras and the target area set as the search target, at least one of the plurality of cameras is in charge of photographing the peripheral area determined with the target area as a reference. The means of determining the camera in charge to be determined as the camera in charge, and
A surveillance system comprising a camera control means for controlling a shooting direction of the camera in charge based on an image of the peripheral area captured by the camera in charge. The monitoring system according to claim 1, wherein the camera control means controls the shooting direction of the camera in charge based on the moving direction of a person in the image of the peripheral region. Further, it has an area determination means for determining whether or not the target area is a difficult-to-search area based on the installation information of the plurality of cameras.
The monitoring system according to claim 1 or 2, wherein the camera control means controls the shooting direction of the camera in charge when the target area is a difficult-to-search area. The surveillance system according to any one of claims 1 to 3, wherein the responsible camera determining means determines a camera capable of photographing the peripheral area as the responsible camera. The surveillance system according to any one of claims 1 to 3, wherein the responsible camera determining means determines a camera set in the peripheral area as the responsible camera. The monitoring system according to any one of claims 1 to 5, further comprising an area setting means for setting the target area according to a user operation. The monitoring system according to any one of claims 1 to 5, further comprising an area setting means for setting the target area based on an image captured by the camera. The area setting means according to claim 7, wherein the target area is set based on at least one of a change in the degree of congestion of a person and a moving direction of the person in the image captured by the camera. Monitoring system. An extraction means for extracting the feature amount of a person from the image taken by the camera in charge, and
Further having a storage means for storing the feature amount in the storage means,
The camera control means has at least one of the pan / tilt / zoom of the camera in charge so as to capture a predetermined image of the person when the feature amount of the person included in the image is not stored in the storage means. The monitoring system according to any one of claims 1 to 8, wherein one is controlled. The camera control means is when the feature amount of the person included in the image is not stored in the storage means and at least one of the position and the moving direction of the person satisfies a predetermined condition. The monitoring system according to claim 9, wherein the camera in charge is controlled so as to capture a predetermined image of the person. The monitoring system according to claim 9 or 10, wherein the camera control means controls the whole body of the person to be photographed. The monitoring system according to claim 9 or 10, wherein the camera control means controls to photograph the face of the person. The monitoring system according to claim 12, wherein the camera control means controls to capture an image such that the ratio of the face of the person in the entire image is equal to or higher than a predetermined value. The surveillance system according to any one of claims 1 to 13, wherein the responsible camera determining means changes the responsible camera based on an image captured by the responsible camera. It is a monitoring support device that supports the monitoring of people based on images taken by multiple cameras installed at different positions.
Based on the installation information of the plurality of cameras and the target area set as the search target, at least one of the plurality of cameras is in charge of photographing the peripheral area determined with the target area as a reference. The means of determining the camera in charge to be determined as the camera in charge, and
A monitoring support device comprising a shooting direction determining means for determining a shooting direction of the camera in charge based on an image of the peripheral region shot by the camera in charge. A monitoring method performed by a monitoring system that monitors a person based on images taken by multiple cameras installed at different positions.
Based on the installation information of the plurality of cameras and the target area set as the search target, at least one of the plurality of cameras is in charge of photographing the peripheral area determined with the target area as a reference. The step of determining the responsible camera to be determined as the responsible camera, and
A monitoring method including a camera control step for controlling a shooting direction of the camera in charge based on an image of the peripheral area taken by the camera in charge. It is a monitoring support method executed by a monitoring support device that supports monitoring of a person based on images taken by multiple cameras installed at different positions.
Based on the installation information of the plurality of cameras and the target area set as the search target, at least one of the plurality of cameras is in charge of photographing the peripheral area determined with the target area as a reference. The step of determining the responsible camera to be determined as the responsible camera, and
A monitoring support method including a shooting direction determination step for determining a shooting direction of the camera in charge based on an image of the peripheral area shot by the camera in charge. A program for causing a computer to function as each means of the monitoring system according to any one of claims 1 to 14. A program for causing a computer to function as each means of the monitoring support device according to claim 15.

Images