JP2019200718A - Monitoring device, monitoring method, and program - Google Patents

Abstract

To provide a monitoring device a monitoring method, and a program that are capable of appropriately notifying a monitoring person of any possibility of danger in person monitoring.SOLUTION: An attribute group detection device as a monitoring device includes: a person detection unit 201 as detection means for detecting a person in a captured image; an attribute estimation unit 202 as attribute estimation means for estimating an attribute of the person detected by the detection means on the basis of the captured image; a group determination unit 203 as classification means for classifying a plurality of persons detected by the detection means attribute groups on the basis of a result of estimation by the attribute estimation means; and an external notification unit 204 as output processing means for performing control to output a warning in accordance with the number of persons belonging to each of the attribute groups.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a monitoring device, a monitoring method, and a program.

  2. Description of the Related Art Conventionally, a monitoring system that monitors a person based on an image captured by a monitoring camera is known. For example, Patent Literature 1 discloses a technique for specifying a monitoring target person from the characteristics of a person included in a captured image and notifying the person when the stay time of the person exceeds a certain time. Patent Document 2 discloses a technique for detecting a person and a person position from a moving image acquired at a place where an unspecified number of persons gather from a viewpoint of marketing or the like, and grouping the persons from the positional relationship. .

JP 2007-74330 A JP 2006-92396 A

  However, the conventional monitoring system has a problem that it is difficult to detect a situation where there is a possibility of danger.

  The present invention has been made in view of such problems, and an object thereof is to appropriately notify a monitoring person or the like when there is a possibility of danger in person monitoring.

  Therefore, the present invention is a monitoring device, a detection unit that detects a person in a captured image, an attribute estimation unit that estimates an attribute of the person detected by the detection unit based on the captured image, and the attribute Classification means for classifying a plurality of persons detected by the detection means into an attribute group based on an estimation result by the estimation means, and output processing for controlling to output a warning according to the number of persons belonging to the attribute group Means.

  According to the present invention, when there is a possibility of danger in person monitoring, it is possible to appropriately notify a monitor or the like.

It is a hardware block diagram of a monitoring apparatus. It is a functional block diagram of a monitoring apparatus. It is a flowchart which shows the monitoring process. It is explanatory drawing of an attribute group. It is a figure which shows the example of a display screen. It is a functional block diagram of the monitoring apparatus which concerns on 2nd Embodiment. It is a flowchart which shows the monitoring process which concerns on 2nd Embodiment. It is explanatory drawing of the distance between groups. It is a flowchart which shows the monitoring process which concerns on 3rd Embodiment. It is explanatory drawing in a modification.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a hardware configuration diagram of the monitoring apparatus 100. The information processing apparatus 100 includes a CPU 101, a ROM 102, a RAM 103, an HDD 104, a display unit 105, an input unit 106, and a communication unit 107. The CPU 101 reads the control program stored in the ROM 102 and executes various processes. The RAM 103 is used as a temporary storage area such as a main memory and a work area for the CPU 101. The HDD 104 stores various data, various programs, and the like. The display unit 105 displays various information. The input unit 106 includes a keyboard and a mouse, and accepts various operations by the user. The communication unit 107 performs communication processing with an external device via a network.

  Note that the functions and processing of the information processing apparatus 100 to be described later are realized by the CPU 101 reading a program stored in the ROM 102 or the HDD 104 and executing this program. As another example, the CPU 101 may read a program stored in a recording medium such as an SD card instead of the ROM 102 or the like.

  FIG. 2 is a functional configuration diagram of the monitoring device 100. The monitoring apparatus 100 includes a human body detection unit 201, an attribute estimation unit 202, a group determination unit 203, an external notification unit 204, and a setting unit 205. The human body detection unit 201 acquires a moving image or a still image as a captured image supplied from the outside. The supply source is not particularly limited, and may be an imaging device or a server device that supplies a captured image via a wired or wireless connection. Further, the supply source is not limited to the outside, and the human body detection unit 201 may acquire a captured image from the memory of its own device. The human body detection unit 201 acquires the setting value of the area where the human body detection process is performed from the setting unit 205, and performs the human body detection process on the acquired setting area of the captured image. A specific processing method of the human body detection process may be any form as long as the human body and the position of the human body are detected every predetermined time with respect to the acquired captured image.

  The attribute estimation unit 202 performs attribute estimation on the human body detected by the human body detection unit 201. The type of attribute to be estimated is set by the setting unit 205. In this embodiment, age and sex are given as attribute information to be estimated. However, the attribute information is not particularly limited as long as it is attribute information obtained from an individual. The group determination unit 203 determines, based on the attribute information transmitted from the attribute estimation unit 202, whether or not there are a small number and a large number of attribute groups for the grouping attribute set by the setting unit 205. For the determination, a grouping threshold (number of persons or ratio) set by the setting unit 205 is used. The determination result by the group determination unit 203 is sent to the external notification unit 204.

  The external notification unit 204 controls to output an alert notification according to the determination result of the group determination unit 203. The alert notification output destination is selected by the setting unit 205. The output destination of the alert notification may be an output device such as the display unit 105 provided in the own device, or may be an output device of another device. The output device as the output destination of the alert notification may be a speaker. When the output destination is a display unit or an external display device, the alert notification to be output may be output as it is, or graphics corresponding to the alert notification may be output.

  Note that at least a part of the functions and processes of the monitoring apparatus 100 may be realized by, for example, cooperating a plurality of CPUs, RAMs, ROMs, and storages. As another example, at least a part of the functions and processing of the monitoring apparatus 100 may be realized using a hardware circuit. Moreover, the monitoring apparatus 100 may be an image processing circuit incorporated in a photographing apparatus such as a monitoring camera.

  FIG. 3 is a flowchart showing monitoring processing by the monitoring apparatus 100. First, in step S301, the setting unit 205 sets various parameters when detecting an attribute group. Next, in S302, the human body detection unit 201 starts human body detection. The human body detection unit 201 acquires a captured image and detects a human body in the acquired captured image. The human body detection unit 201 uses the parameters of the setting area set in S301 by the setting unit 205 in the human body detection process. The parameter set by the setting unit 205 is not particularly limited as long as it is used for human body detection. Next, in S303, the CPU 101 performs end determination. The CPU 101 determines to end when there is no input of a captured image. The CPU 101 also determines that the process has ended in response to an external trigger. When the CPU 101 determines that the process is finished (YES in S303), the monitoring process is finished. If the CPU 101 does not determine the end (NO in S303), the process proceeds to S304.

  In step S304, the attribute estimation unit 202 performs attribute estimation on the human body detected in step S302. The attribute estimation unit 202 uses the parameters set in S301 by the setting unit 205 for attribute estimation processing. The parameter set by the setting unit 205 is not particularly limited as long as it is used for attribute estimation. The attribute information of each individual that has been estimated is sent to the group determination unit 203. The process of S304 is an example of an attribute estimation process.

  Next, in S305, the group determination unit 203 classifies the plurality of persons detected in S302 into a plurality of groups using the attribute information estimated in S304. Note that the conditions for classifying into groups are set in advance. The group determination unit 203 of the present embodiment classifies the attribute groups into adult and child attribute groups. Then, the group determination unit 203 determines that this attribute group is a small number of attribute groups when the number of persons belonging to the attribute group is less than the first number of people threshold. The group determination unit 203 also determines that this attribute group is a large number of attribute groups when the number of persons belonging to the attribute group is equal to or greater than the second number of people threshold. If there are a small number of attribute groups and one or more attribute groups (YES in S305), the group determination unit 203 advances the process to S306. If at least one of the small number of attribute groups and the large number of attribute groups does not exist (NO in S305), the group determination unit 203 advances the process to S303.

  In S306, the external notification unit 204 controls to output a warning. For example, the external notification unit 204 controls the display unit 105 to display warning information indicating a warning. As another example, the external notification unit 204 may be controlled to output a warning to a predetermined notification destination. After the process of S306, the CPU 101 advances the process to S303. Here, the process of S306 is an example of an output process for controlling to output a warning according to the number of persons belonging to the attribute group.

  For example, as shown in FIG. 4, it is assumed that a child attribute group is determined to be a large number of attribute groups, and an adult attribute group is determined to be a small number of attribute groups. In this case, a warning is output in S306. As shown in FIG. 4, when there are adults among many children, there is a risk that the adults may approach the children. The same is true if there are a few women in a large number of men or a small number of elderly people in a large number of adults.

  Next, a UI related to the monitoring process will be described. FIG. 5 is a diagram illustrating an example of a display screen related to the monitoring process. Regarding the human body detection / attribute estimation setting, the user can set the detection area 511 on the monitoring video 510 by pressing the area setting button 501. When human body detection and attribute estimation are performed on the set detection area 511, the result is displayed in the table 502.

  Regarding the grouping setting, the user selects an attribute for grouping from the pull-down menu 503, and sets a threshold for division at 504. In the pull-down menu, it is possible to specify various combinations such as age or sex as described above (child, adult), (male, female), (adult, elderly), and the like. The attribute estimation result is displayed in real time by a pie chart 505. The attribute grouping result is displayed in a colored silhouette state on the detected human body on the monitoring video 510 and is updated in real time. This process is an example of a display process for controlling the person belonging to the first group and the person belonging to the second group to be displayed in a distinguishable manner. By performing such display, the user can confirm at a glance where the minority / majority attribute group exists on the video. Regarding the alert setting, the detection lamp 506 lights up when the grouping determination is made. The alert notification destination can be selected from a pull-down menu 507. As described above, it is possible to effectively grasp the presence of the minority / majority attribute group by performing appropriate settings on the UI.

  Thus, in a situation where there is a possibility of danger, the monitoring apparatus 100 according to the present embodiment can output a warning. In other words, the monitoring device 100 can appropriately notify a monitoring person or the like when there is a possibility of danger in person monitoring.

  As a first modification of the first embodiment, the monitoring apparatus 100 may exclude a specific person when classifying a plurality of persons into a group. It is assumed that the specific person has been registered by the detection setting unit 205, and the group determination unit 203 determines the specific person according to the registered information. Then, the group determination unit 203 excludes the person determined as the specific person from the group classification target. That is, the group determination unit 203 performs classification on a person other than a person registered in advance. Thereby, for example, a person who wants to be excluded such as a security guard can be excluded from the group. Therefore, it is possible to more appropriately determine the possibility of danger.

  As a second modification, the monitoring device 100 may be an image processing circuit incorporated in an imaging device such as a monitoring camera. In this case, when it is determined that there are a small number of attribute groups and a large number of attribute groups, the monitoring apparatus 100 replaces the subsequent processing with a small number of attribute groups by performing PTZ driving. May be tracked. When a small number of attribute groups are composed of a plurality of persons, the monitoring apparatus 100 performs PTZ driving so that the center of gravity of the member is in the center of the imaging range. In this way, after the attribute group is determined, the movement of the group can be tracked.

  As a third modified example, the monitoring apparatus 100 may be controlled to output a warning according to the number of persons belonging to the attribute group, and specific processing for that purpose is not limited to the embodiment. . For example, the monitoring apparatus 100 may limit the output according to the ratio between the number of persons in one attribute group and the number of persons in another attribute group.

(Second Embodiment)
Next, the monitoring device 100 according to the second embodiment will be described mainly with respect to differences from the monitoring device 100 according to the first embodiment. FIG. 6 is a functional configuration diagram of the monitoring apparatus 100 according to the second embodiment. The monitoring apparatus 100 according to the second embodiment further includes a distance determination unit 601 in addition to the functional configuration of the monitoring apparatus 100 according to the first embodiment. The distance determination unit 501 determines whether the distance between the minority group and the majority group is less than a distance threshold. The distance determination unit 601 uses the distance between members of different groups set by the setting unit 205 for the determination.

  FIG. 7 is a flowchart showing monitoring processing by the monitoring apparatus 100 according to the second embodiment. Among the processes of the monitoring process shown in FIG. 6, the same processes as the processes of the monitoring process according to the first embodiment described with reference to FIG. As described in the first embodiment, in S306, the external notification unit 204 controls to output a warning. Here, in order to distinguish from a warning described later, the warning in S306 is referred to as a first warning. After the process of S306, in S701, the distance determination unit 601 obtains an intergroup distance. Here, the inter-group distance is a distance between a person belonging to a small number of attribute groups and a person belonging to a large number of attribute groups, as shown in FIG.

  In the present embodiment, the distance determination unit 601 obtains the minimum value as the inter-group distance among the distances between persons belonging to the minority group and persons belonging to the multiple attribute group. As another example, the group determination unit 203 may obtain the maximum value as the inter-group distance among the distances between persons belonging to a small number of attribute groups and persons belonging to a large number of attribute groups. Further, the group determination unit 203 may obtain an average value of distances between persons belonging to a small number of attribute groups and persons belonging to a large number of attribute groups as an inter-group distance.

  Returning to FIG. 7, in S701, the distance determination unit 601 further determines whether or not the inter-group distance is less than a preset distance threshold. If the distance between groups is less than the distance threshold (YES in S701), the distance determination unit 601 advances the process to S702. If the distance between groups is equal to or greater than the distance threshold (NO in S701), the distance determination unit 601 advances the process to S303.

  In step S702, the external notification unit 204 controls to output a warning. This warning is referred to as a second warning. Note that the first warning and the second warning may be information that allows the user to recognize that they are different warnings. After the process of S702, the CPU 101 advances the process to S303. The remaining configuration and processing of the monitoring device 100 according to the second embodiment are the same as the configuration and processing of the monitoring device 100 according to the first embodiment.

  It is considered that the possibility of danger increases as the inter-group distance between a small number of attribute groups and a large number of attribute groups decreases. Therefore, the monitoring device 100 according to the second embodiment controls to output a warning when the inter-group distance becomes less than the distance threshold. As a result, it is possible to appropriately notify the supervisor or the like.

(Third embodiment)
Next, the monitoring device 100 according to the third embodiment will be described mainly with respect to differences from the monitoring device 100 according to another embodiment. The monitoring apparatus 100 according to the third embodiment controls to output a warning in two stages according to the distance between groups. FIG. 9 is a flowchart illustrating the monitoring process performed by the monitoring apparatus 100 according to the third embodiment. Among the processes of the monitoring process shown in FIG. 6, the same processes as the processes of the monitoring process according to the first embodiment described with reference to FIG.

  As described in the second embodiment, in S306, the external notification unit 204 controls to output the first warning. Then, after the processing of S306, in S901, the group determination unit 203 obtains an intergroup distance and determines whether the intergroup distance is less than the distance threshold. Here, in order to distinguish from the distance threshold described later, the distance threshold used in S901 is referred to as a first distance threshold. If the inter-group distance is less than the first distance threshold (YES in S901), the group determination unit 203 advances the process to S902. When the inter-group distance is equal to or greater than the first distance threshold (NO in S901), the group determination unit 203 advances the process to S303. In step S902, the external notification unit 204 performs control to output a second warning. Note that the processing in S901 and S902 is the same as the processing in S701 and S702 described with reference to FIG.

  After the process of S902, in S903, the group determination unit 203 determines whether the inter-group distance is less than the second distance threshold. Here, the second distance threshold is a distance smaller than the first distance threshold. If the inter-group distance is less than the second distance threshold (YES in S903), the group determination unit 203 advances the process to S904. If the inter-group distance is equal to or greater than the second distance threshold (NO in S903), the group determination unit 203 advances the process to S303. In step S904, the external notification unit 204 controls to output a third warning, and then proceeds to step S303. Other configurations and processes of the monitoring apparatus 100 according to the third embodiment are the same as the configurations and processes of the monitoring apparatus 100 according to the other embodiments.

  For example, as shown in FIG. 8, there are cases where a small number of adults approach a child and follow up. In such a case, according to the monitoring device 100 according to the third embodiment, after the second warning is output, the third warning is issued when the distance between the child and the adult is further reduced by the chasing operation. Can be output.

  As a first modification of the third embodiment, in the present embodiment, the inter-group distance is referred to detect the chasing operation. However, in S903, it is only necessary to detect the chasing operation. The general process is not limited to the embodiment. As another example, the presence / absence of a chasing operation may be detected based on the moving speeds of persons in a small number of attribute groups.

  As a second modified example, the monitoring apparatus 100 may acquire emotion as attribute information and may perform further alerting after attribute group determination. As the acquisition of emotion, a method of acquiring an expression from a face is generally known. This example deals with anger and fear. The monitoring apparatus 100 estimates the emotions of the persons in each group after the group determination. This process is an example of the emotion estimation process. And the monitoring apparatus 100 is controlled to output a warning when the ratio of anger and fear is equal to or greater than a certain threshold value. Note that the threshold value of the ratio is set in advance by the setting unit 205. In this way, by considering not only the presence of a small number and a large number of attribute groups but also the emotions of the persons in the group, the possibility of danger can be determined more appropriately.

  Furthermore, in the above-described embodiment, whether or not to perform the process may be controlled according to the density of the human body detected. It is expected that the presence of minority / majority attribute groups can be more dangerous when the density of people is high. Therefore, the monitoring apparatus 100 according to the present modification calculates the human density based on the size and angle of view of the human body detected by the human body detection unit 201.

  For example, in order to confirm a face, surveillance camera images are often taken from an oblique direction. In this case, the person on the near side with respect to the monitoring camera is large and the person on the far side is small and displayed on the video (for example, FIG. 10A). By analyzing the acquired image, the position and size of the detected person can be obtained. Using the position and size of the person, the monitoring apparatus 100 designates a region for the image so that the actual area of each region is the same (for example, FIG. 10B). Count the number of people in each specified area and calculate the density. In this embodiment, the density is calculated by counting the number of people in the area, but the method of calculating the density using the distance between the detected person and the person, learning a plurality of images, and calculating the density using learning. A calculation method may be used. And the monitoring apparatus 100 performs the process after an attribute estimation process, only when a person's density is more than fixed. Thereby, the group determination can be performed only when the risk level is expected to be higher. This process is an example of an output process that limits the output of a warning according to the density of a person.

  The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to such specific embodiments, and various modifications can be made within the scope of the gist of the present invention described in the claims.・ Change is possible.

(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

DESCRIPTION OF SYMBOLS 100 Monitoring apparatus 201 Human body detection part 202 Attribute estimation part 203 Group determination part 204 External notification part 205 Setting part

Claims (12)

Detecting means for detecting a person in the captured image;
Attribute estimation means for estimating an attribute of a person detected by the detection means based on the captured image;
Classification means for classifying a plurality of persons detected by the detection means into attribute groups based on the estimation result by the attribute estimation means;
An output processing means for controlling to output a warning according to the number of persons belonging to the attribute group.   The output processing means includes a first group in which the number of persons belonging to the attribute group is less than a first number threshold, and a number of persons in the attribute group that is greater than or equal to a second number threshold greater than the first number threshold. The monitoring apparatus according to claim 1, wherein control is performed to output a warning when one or more second groups exist.   The monitoring according to claim 2, wherein the output processing means controls to output a warning according to a distance between a person belonging to the first group and a person belonging to the second group. apparatus.   The output processing means controls to output a warning when a distance between a person belonging to the first group and a person belonging to the second group is less than a distance threshold. The monitoring device described in 1. The output processing means includes
A first group in which the number of persons belonging to the attribute group is less than the first number threshold and a second group in which the number of persons belonging to the attribute group is greater than the first number threshold and greater than or equal to the second number threshold are respectively Control to output the first warning if one or more exist,
Control to output a second warning when the distance between the person belonging to the first group and the person belonging to the second group is less than a first distance threshold;
A third warning is output when the distance between the person belonging to the first group and the person belonging to the second group is less than a second distance threshold smaller than the first distance threshold. The monitoring device according to claim 1, wherein the monitoring device is controlled.   The monitoring apparatus according to claim 2, further comprising display processing means for controlling the persons belonging to the first group and the persons belonging to the second group to be displayed in a distinguishable manner. Further comprising emotion estimation means for estimating the emotion of at least one of the persons belonging to the first group and the persons belonging to the second group based on the captured image;
The monitoring apparatus according to claim 2, wherein the output processing unit further controls to output a warning according to a person's emotion.   The monitoring apparatus according to claim 2, further comprising a tracking unit that tracks at least one of the person belonging to the first group and the person belonging to the second group.   The monitoring device according to claim 1, wherein the classification unit performs classification on a person other than a person registered in advance.   The monitoring apparatus according to claim 1, wherein the output processing unit limits the output of the warning according to the density of the person detected by the detection unit. A monitoring method executed by a monitoring device,
A detection step of detecting a person in the captured image;
An attribute estimation step for estimating an attribute of the person detected in the detection step based on the captured image;
A classification step of classifying a plurality of persons detected in the detection step into attribute groups based on the estimation result of the attribute estimation step;
And an output processing step of controlling to output a warning according to the number of persons belonging to the attribute group.   The program for functioning a computer as each means of the monitoring apparatus of any one of Claims 1 thru | or 10.

Images