JP2021007034A - Control apparatus, control method, and program - Google Patents

Abstract

To notify a user if there is only a single person in an image.SOLUTION: Detection means detects a person from a motion picture. Specifying means specifies whether or not a person in a first area set in the motion picture among persons detected by the detection means is a single person. Notification control means, when the specifying means specifies that the person is a single person, controls a notification device to carry out notification processing of information allowing a user to recognize existence of the single person if the single person stays in the first area for more time as compared with a predetermined time. The specifying means determines the person as a single person if there is any of the persons in the first area having a face direction or an eye line that does not intersect with those of others and staying alone.SELECTED DRAWING: Figure 3

Description

The present invention relates to a control device, a control method, and a program that detect a person from a moving image and cause a notification device to execute a notification process.

Conventionally, a method of detecting a person from an image is known. Patent Document 1 discloses a method of detecting a person who is shielded from an image.

Japanese Unexamined Patent Publication No. 2014-199506

In the technique disclosed in Patent Document 1 described above, it is possible to inform the position of a person in an image. For example, in the above technique, a person existing in an image can be circled and the position of the person can be presented to the user. However, by simply indicating the position of the person based on the above-mentioned technique, the user is whether the detected person is working alone or isolated from the group, or is collaborating with another person. There was a problem that I could not judge at a glance. For example, in the field where data that should be prevented from leaking is handled, a rule may be imposed that prohibits working alone, and a work observer monitors the workers via an imaging device. You may want to call attention to a single worker who is working alone, isolated from the group. However, just by looking at the captured image, the work observer cannot determine at a glance whether each of the workers in the captured image is working independently or in multiples. In addition, it is burdensome for the work observer to constantly monitor the worker.
An object of the present invention is to notify a user when a person in an image is alone.

Therefore, the control device of the present invention is a single person with respect to the detection means for detecting a person from the moving image and the person in the first area set in the moving image among the persons detected by the detecting means. When the time that the person is in the first area exceeds a predetermined time in a state where the specific means for identifying the person is specified as the single person by the specific means, the existence of the single person is detected. It has a notification control means for controlling the notification device to execute a notification process of information recognizable by the user, and the specific means is a person in the first area and has a face with another person. When there is at least one of a person whose direction or line of sight does not intersect and a person who is one person in the first area, the person is identified as a single person. To do.

According to the present invention, when the person in the image is alone, the user can be notified.

It is a figure which shows an example of the system configuration and the hardware configuration of the system. It is a figure which shows an example of the functional structure of an image pickup apparatus and a client apparatus. It is a flowchart which shows an example of the processing of a notification control. It is a figure which shows an example of the captured image. It is a figure which shows an example of the image which shows the result of performing a person detection. It is a figure which shows an example of the horizontal plane image. It is a figure for demonstrating the process of determining whether or not it is a single worker.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<Embodiment 1>
In the present embodiment, in the field where data for which leakage should be prevented is handled, in which two or more workers are obliged to perform the work, each worker is associated with a single worker (associated with another person). If there is a person who does not exist), the process of notifying the work observer to that effect will be described. Such a situation is an example of a situation in which the present embodiment can be used, and can be used in various situations in which it is desired to determine whether a person is a single person or a plurality of people. For example, the present invention can also be used in situations where it is determined whether a passerby, a visitor, etc. is alone or in a group, and used for marketing, solving social problems, and the like.
FIG. 1 shows an example of the system configuration and the hardware configuration of the system in this embodiment. In FIG. 1, the imaging device 110 performs imaging. The client device 120 drives the image pickup device 110 and displays the captured image captured by the image pickup device 110. The input device 130 is composed of a mouse, a keyboard, and the like, and performs user input to the client device 120. The display device 140 is composed of a display or the like, and displays an image output by the client device 120. Although the client device 120 and the display device 140 are shown as independent devices in FIG. 1, the client device 120 and the display device 140 may be integrated and configured. Further, the input device 130 and the display device 140 may be integrated, or the client device 120, the input device 130, and the display device 140 may be integrated. The network 150 connects the image pickup device 110 and the client device 120. The network 150 is composed of a plurality of routers, switches, cables, etc. that meet communication standards such as a local network. In the present embodiment, the communication standard, scale, and configuration are not limited as long as the image pickup device and the client can communicate with each other. For example, the network 150 may be configured by the Internet, a wired LAN (Local Area Network), a wireless LAN (Wireless LAN), a WAN (Wide Area Network), or the like. Further, the number of image pickup devices connected to the client device 120 is not limited to one, and may be a plurality of.

The hardware configuration of the image pickup apparatus 110 will be described. The image pickup apparatus 110 includes an image pickup unit 11, a CPU 12, a memory 13, and a communication I / F 14 as a hardware configuration. The imaging unit 11 has an imaging element and an engineering system for the subject on the imaging element, and under the control of the CPU 12, images are taken on the imaging element with the intersection of the optical axis of the optical system and the imaging element as the imaging center. The image pickup device is an image pickup device such as a CMOS (Complementary Metal-Oxide Semiconductor) or a CCD (Charged Coupled Device). The CPU 12 controls the entire image pickup apparatus 110. The memory 13 stores a program, an image captured by the imaging unit 11, data used when the CPU 12 executes processing, and the like. The communication I / F 14 controls communication with the client device 120 via the network 150 based on the control of the CPU 12. The function of the image pickup apparatus 110 and the like are realized by the CPU 12 executing the process based on the program stored in the memory 13. A processor other than the CPU may be used.
The hardware configuration of the client device 120 will be described. The client device 120 includes a memory 21, a CPU 22, a communication I / F23, an input I / F24, and a display I / F25 as hardware configurations. The CPU 22 controls the entire client device 120. The memory 21 stores a program, a captured image transmitted from the imaging device 110, data used when the CPU 22 executes processing, and the like. The communication I / F 23 controls communication with the image pickup apparatus 110 via the network 150 based on the control of the CPU 22. The input I / F 24 connects the client device 120 and the input device 130, and controls the input of information from the input device 130. The display I / F 25 connects the client device 120 and the display device 140, and controls the output of information to the display device 140. When the CPU 22 executes the process based on the program stored in the memory 21, the function of the client device 120 and the process of the flowchart of FIG. 3 described later are realized. However, the hardware configuration of the client device 120 is not limited to that shown in FIG. For example, the client device 120 may have an audio output device such as a speaker, which is an example of the notification device. When the client device 120 has a voice output device as a hardware configuration, an example of the notification process is a voice output process such as a warning.

FIG. 2 is a diagram showing an example of the functional configuration of the image pickup apparatus 110 and the client apparatus 120. The image pickup apparatus 110 includes a control unit 111, a signal processing unit 112, a drive control unit 113, and a communication control unit 114 as functional configurations. The control unit 111 controls the image pickup device 110. The signal processing unit 112 processes the image signal captured by the image pickup unit 11. For example, the signal processing unit 112 encodes the image captured by the imaging unit 11. As the coding method, the signal processing unit 112 can use, for example, JPEG (Joint Photographic Experts Group). Alternatively, the signal processing unit 112 uses the H.S. 264 / MPEG-4 AVC (hereinafter referred to as H.264) can be used. Alternatively, the signal processing unit 112 can use HEVC (High Efficiency Video Coding coding method) as the coding method. However, the coding method is not limited to these. Further, the signal processing unit 112 may select a coding method from a plurality of coding methods to perform coding.
The drive control unit 113 controls to change the image pickup direction and the angle of view of the image pickup unit 11. In the present embodiment, the case where the imaging unit 11 can change the imaging direction in the pan direction and the tilt direction and can change the imaging angle of view will be described, but the present invention is not limited to this. The image pickup apparatus 110 may not have a function of changing the image pickup direction in the pan direction and the tilt direction, and may not have a function of changing the angle of view. The communication control unit 114 transmits the captured image processed by the signal processing unit 112 to the client device 120. Further, the communication control unit 114 receives a control command from the client device 120 to the image pickup device 110.
The client device 120 includes a control unit 121, a communication control unit 123, an information acquisition unit 124, and a display control unit 125 as functional configurations. The control unit 121 controls the client device 120. The communication control unit 123 receives the captured image from the client device 120. The information acquisition unit 124 receives the user input by the input device 130 and acquires the input information from the input device 130. The display control unit 125 outputs an image to the display device 140, and causes the display device 140 to execute a display process described later. The display device 140 is an example of a notification device. The display process is an example of a notification process.

Next, the flow of the notification control process in the present embodiment will be described with reference to FIG. FIG. 3 is a flowchart showing an example of notification control processing in the client device 120. The client device 120 repeatedly acquires a captured image, that is, a moving image, from the imaging device 110, and notifies a work observer who is a user when there is a single worker in the acquired captured image.
The control unit 121 acquires an captured image from the imaging device 110 (S200). FIG. 4 is a diagram showing an example of the captured image 300 acquired by the client device 120 from the imaging device 110. In FIG. 4, workers 301 and 302 and workers 303 and 304 represent collaborative workers (persons associated with other persons) who are working in pairs. On the other hand, the worker 305 represents a single worker who is working alone. Further, the workbench 306 represents a workbench existing at a position far from the image pickup apparatus 110. The workbench 307 represents a workbench located near the imaging device 110.
At this time, it is preferable that the position, mounting angle, and angle of view of the imaging unit 11 of the imaging device 110 are adjusted in advance so that the captured images acquired by the client device 120 do not overlap as much as possible. In the present embodiment, the client device 120 processes the captured image acquired from the image pickup device 110. This makes it possible to process real-time live video. However, it is not limited to this. For example, the client device 120 may acquire a moving image stored in the imaging device 110 and process each frame, or process each frame of the moving image stored in the client device 120. May be done. Alternatively, the client device 120 may access the recording server or the like and process each frame of the moving image stored in the recording server. Further, in the present embodiment, the collaborators are set as a pair, but the present invention is not limited to this. The client device 120 may have three or more co-workers.

Next, the control unit 121 performs a person detection process on the captured image 300 acquired in S200 (S201). In order to perform the person detection process, the control unit 121 first scales the captured image 300 in various sizes in the present embodiment. By scaling, people of various sizes can be detected. Next, the control unit 121 raster scans the captured image of each scale using a detection window of a specific size. At this time, the control unit 121 calculates the feature amount of the person in advance using the training data, and the error between the feature amount calculated in the detection window at the time of scanning and the feature amount based on the above-mentioned learning data is a threshold. If it is smaller than, it is judged to be a person. Various methods such as pattern matching processing can be used for the person detection processing.
FIG. 5 is a diagram showing an example of an image 400 showing the result of performing person detection on the captured image 300. In FIG. 5A, the ellipses 401 to 405 represent the positions where the workers 301 to 305 are detected, respectively. Further, the arrows 406 to 408 are half straight lines in which the tail end side of the arrow is used as a base point and the head side is extended infinitely to express the orientation of each person's face. Further, as described above, in addition to the detection method using the feature amount of the person, as shown in FIG. 5B, as the rules of the worker, a predetermined uniform, bibs (vest-shaped clothes), and hat You may decide to wear at least one of the above. That is, the feature amount of the object worn by the person to be detected as the worker may be stored in the memory 21 or the like of the client device 120 as a rule setting file in advance. In this way, the control unit 121 can obtain a uniform or the like having a feature amount that matches the rule based on the rule of the setting file and the feature amount such as the color and shape of the object worn by the person to be processed of the captured image. It is also possible to detect the worn worker.

ここで、式（２）中のｐｘ１、ｐｘ２、．．．、ｐｘ４、ｐｙ４、及びｑｘ１、ｑｘ２、．．．、ｑｘ４、ｑｙ４は、ユーザーによって指定された撮像画像上の四点のｘ、ｙ座標、及び水平面画像上の四点のｘ、ｙ座標をそれぞれ表す。
次に制御部１２１は、撮像画像上のｘ座標ｐｘ、Ｙ座標ｐｙとホモグラフィ行列Ｈとに対し、以下の式（３）を適用することにより、ｐｘ、ｐｙに対応する水平面画像上のＸ座標ｑｘと、Ｙ座標ｑｙとを計算することができる。

Next, the control unit 121 converts the position of the person detected in S101 into the coordinates on the horizontal plane image projected from directly above the shooting range of the captured image (S202). In the present embodiment, it is assumed that each point on the captured image corresponds to which point on the horizontal plane image in advance. For example, the user designates four points on the captured image displayed on the display device 140 or the like and four points on the horizontal plane image by operating the input device 130 or the like. The control unit 121 calculates the homography matrix H in advance based on the four points on the captured image input by the user, the four points on the horizontal plane image, and the following equations (1) and (2). ..

Here, px1, px2, ... In the equation (2). .. .. , Px4, py4, and qx1, qx2 ,. .. .. , Qx4, qy4 represent the x and y coordinates of the four points on the captured image specified by the user, and the x and y coordinates of the four points on the horizontal plane image, respectively.
Next, the control unit 121 applies the following equation (3) to the x-coordinate px, Y-coordinate py and the homography matrix H on the captured image, thereby applying X on the horizontal plane image corresponding to px, py. The coordinate qx and the Y coordinate qy can be calculated.

FIG. 6 is a diagram showing an example of a horizontal plane image 500 of a work site. In FIG. 6, the coordinates 501 to 505 represent the results of converting the representative points of the ellipses 401 to 405, which are the person detection positions, into the coordinates on the horizontal plane image. In the present embodiment, the representative point is set as the center of the person detection position as an example. The representative point may be the center of the area corresponding to the head of the person. Further, the areas 506 and 507 represent areas when the workbench 306 and 307 are represented on a horizontal plane image, respectively. In the present embodiment, it is assumed that a sketch of the work site created by CAD or the like is used as the horizontal plane image, but this is not the case. For example, a deformed image of the captured image may be used, or a handwritten image may be used. Further, it is not necessary to have an image, and it is sufficient to have the above-mentioned coordinate information inside the program. That is, the horizontal plane image 500 does not have to be generated, and only the coordinate position on the horizontal plane image 500 may be calculated. Further, if the image is taken with a composition (overhead view or the like) close to that of the horizontal plane image 500, the processing of S203 does not necessarily have to be performed. In this case, it is possible to determine whether the worker is a single worker or a collaborative worker by using the distance on the captured image. Moreover, various other methods may be used as the method of generating the horizontal plane image 500.

Next, the control unit 121 updates the list of detected persons (S203). At the beginning of the iterative process, the control unit 121 lists all the detected persons. At this time, the control unit 121 stores the coordinates of the representative point on the captured image in which the person is detected and the coordinates of the representative point on the horizontal plane image in each node in the list. If it is the second and subsequent times of the iterative process, the control unit 121 performs the following process on all the detected persons.
(Process 1). The control unit 121 calculates the distance between the coordinates on the captured image of the person stored in each node in the list and the coordinates on the captured image of the person, and if it is equal to or less than the threshold value, the target node is set to its own. Replace with data. At this time, the control unit 121 saves the history of the coordinates saved in the node so far.
(Process 2). If there is no node that satisfies the condition in (Process 1), the control unit 121 adds its own data to the end of the list. In addition, the control unit 121 performs the following processing on all the detected persons after completing the above processing.
(Processing 3). The control unit 121 deletes the node that has not been updated.

Next, the control unit 121 determines whether or not there is a single worker in the captured image (S204 to S206). The processing flow is as follows.
(Process 1). The control unit 121 targets the head of the list for processing (S204).
(Process 2). The control unit 121 determines whether or not the person to be processed is a single worker (S205). In the present embodiment, the control unit 121 confirms the following items with the person to be processed in order to determine whether or not it is a single worker.
(Item 1). The control unit 121 determines whether or not the detected person exists within the radius R with the coordinates on the horizontal plane image of the person to be processed as the center.
This radius R may be appropriately set according to the situation in which the present embodiment is applied. For example, when monitoring work indoors, the radius R can be a distance corresponding to 2 m. However, at this time, when the control unit 121 refers to the coordinate history stored in the node of the person to be processed and determines that the person to be processed is moving, the control unit 121 is more than when it is determined that the person is not moving. , Increase the value of R, which is the threshold value. At this time, if the collaborators work side by side in the x-axis direction on the horizontal plane image, the control unit 121 may use an ellipse in which a circle having a radius R is contracted in the y-axis direction. Similarly, the control unit 121 may use an ellipse in which a circle having a radius R is contracted in the x-axis direction, as long as the collaborators work side by side in the y-axis direction on the horizontal plane image. The control unit 121 sets a person who meets the conditions of (item 1) as a candidate for a single worker, and confirms the following (item 2) and (item 3).
(Item 2). When there is a person who satisfies (item 1), the control unit 121 determines whether the half line representing the face orientation intersects with the half straight line representing the face orientation of the person to be processed.
However, at this time, if the control unit 121 cannot detect the orientation of the face, (item 2) is not included in the determination of whether it is a single worker or a joint worker, which will be described later.
(Item 3). When a person satisfying (item 1) exists, the control unit 121 determines whether the person exists outside the non-detection area.
In the present embodiment, the control unit 121 operates the input device 130 and the like to set the area designated by the user as a non-detection area in which the operator cannot exist. This makes it possible to prevent a non-worker moving on the other side of a glass wall and a person appearing on a poster or the like placed on a desk from being mistakenly detected as a collaborator. The setting of the non-detection area is not limited to the input by the user via the input device 130 or the like. The control unit 121 may automatically set the area surrounded by the markers arranged on the captured image as a non-detection area. As a method of detecting using the marker, for example, the two-dimensional marker installed at the four corners of the non-detection area is registered in the client device 120 in advance by raster scanning the captured image. There is a method of pattern matching with the information of.
When all of the above (item 1) to (item 3) are satisfied, the control unit 121 determines that the worker is a collaborator, and proceeds to (process 3-1). That is, the control unit 121 associates the workers satisfying all of the above (item 1) to (item 3) with each other, and determines that they are joint independent workers. In this way, the control unit 121 has a function as an association unit. If the control unit 121 does not satisfy at least one, the control unit 121 determines that the person to be processed is a single worker, and proceeds to (Process 3-2).

Here, the process of determining whether the worker is a single worker or a collaborative worker is not limited to the above-mentioned process. For example, the control unit 121 associates the persons with each other based on at least one of the position of the person, the orientation of the face of the person, and the distance between the persons detected from the captured image, and is a collaborator. It may be determined. For example, the control unit 121 may associate people in a predetermined range from a predetermined object of the captured image, such as a workbench, or estimate the direction (or line of sight) of the person's face. People whose face orientations (or eyes) intersect may be associated with each other. Further, for example, the control unit 121 may associate people whose distances between people are within a predetermined distance. Then, the control unit 121 may determine that the person associated with the other person is a collaborative worker and the person who is not associated with the other person is a single worker. Further, the control unit 121 may further associate the persons with each other based on the color or shape of the person's uniform, bibs (vest-shaped clothes), or hat. For example, the control unit 121 associates persons having the same uniform shape among persons whose distance between persons is within a predetermined distance, and even if the persons have a distance between persons within a predetermined distance, the shape of the uniform. People with different values may not be associated with each other.

式（４）中のｑｘ、ｑｙはそれぞれ、処理対象の人物の、水平面画像上でのｘ座標、及びｙ座標を表す。図７では、ｑｘ、ｑｙはそれぞれ座標６０２のｘ座標、及びｙ座標を表す。式（４）中のａ、ｂは楕円の変換パラメータである。例えば、ａ＝１、ｂ＝１とすると、楕円は半径Ｒ、中心（ｑｘ、ｑｙ）の円となる。また、ａ＝１、ｂ＝１／２とすると、ｘ軸方向の長さがＲ、ｙ軸方向の長さがＲ／２となる楕円となる。本実施形態では共同作業者同士が水平面画像上のｘ軸方向に並んで作業を行うため、ａ＞ｂとなる値を用いることとする。 The determination in the above (item 1) to (item 3) will be described with reference to FIG. In FIG. 7, the coordinates 601 to 605 represent the result of converting the center of the ellipse 401 to 405, which is the person detection position, into the coordinates on the horizontal image. Further, the areas 606 and 607 represent non-detection areas designated by the user. Further, the ellipse 600 represents an ellipse composed of x and y that satisfies the following equation (4).

Qx and qy in the equation (4) represent the x-coordinate and the y-coordinate of the person to be processed on the horizontal plane image, respectively. In FIG. 7, qx and qy represent the x-coordinate and the y-coordinate of the coordinate 602, respectively. A and b in the equation (4) are conversion parameters of the ellipse. For example, if a = 1 and b = 1, the ellipse is a circle with radius R and center (qx, qy). Further, when a = 1 and b = 1/2, the ellipse has a length in the x-axis direction of R and a length in the y-axis direction of R / 2. In the present embodiment, since the collaborators work side by side in the x-axis direction on the horizontal plane image, a value such that a> b is used.

For example, it is assumed that the control unit 121 targets a person with an ellipse 402, which is a person detection position, during scanning of the list. At this time, the control unit 121 determines the above (item 1) to (item 3) with respect to the coordinates 602 on the horizontal plane image corresponding to the ellipse 402 which is the person detection position.
First, in (item 1), the control unit 121 determines whether or not the detected person exists inside the ellipse 600. In the situation shown in FIG. 7, since the coordinates 601 on the horizontal image corresponding to the ellipse 401 of the person detection position are included in the ellipse, the control unit 121 determines that the requirement is satisfied.
Next, in (item 2), the control unit 121 has a half straight line 407 representing the face orientation of the ellipse 402, which is the person detection position, and an ellipse 401, which is the person detection position of the person who satisfies the condition of (item 1). It is determined whether or not the arrow 406 indicating the direction of the face intersects. Under the circumstances described in the example of this embodiment, the arrows 406 and 407 intersect, and the control unit 121 determines that the requirements are satisfied.
Next, in (Item 3), the control unit 121 has the area 606 in which the coordinates 601 on the horizontal plane image corresponding to the ellipse 401, which is the person detection position of the person satisfying the condition of (Item 1), are the non-detection area. It is determined whether or not it is included in 607. Under the situation described in the example of the present embodiment, since the coordinates 601 are not included in the non-detection regions 606 and 607, the control unit 121 determines that the requirements are satisfied. Since all the requirements (item 1) to (item 3) have been satisfied as described above, the control unit 121 determines that the person with the ellipse 402, which is the person detection position, is a collaborator.
Further, in the present embodiment, the control unit 121 selects a person who meets the condition of (item 1) as a candidate for a single worker. However, this is not the case, and the control unit 121 may make a determination based on the designation of the area. For example, the control unit 121 sets a plurality of regions on the captured image or the horizontal plane image on which the pair of each worker will exist, based on the user input. At this time, the control unit 121 may set the condition (item 1) as to whether or not the person exists in the same detection area as the person of interest.

(Processing 3-1). The control unit 121 determines whether or not the end of the list is the processing target (S206).
If it is not at the end of the list, the control unit 121 targets the next person for processing, and returns to (processing 2) (S207). If it is the end of the list, the control unit 121 ends the scan in the list and sets the cumulative number of frames in which a single worker exists to 0 (S208).
(Processing 3-2). The control unit 121 increments the cumulative number of frames in which a single worker exists, that is, increases the value of the variable by 1 (S209).
Next, the control unit 121 determines whether or not to notify the user that a single worker exists, and determines whether or not to notify the user.
If the conditions are satisfied, a notification is given (S210, S211). In the present embodiment, the control unit 121 notifies the user of a warning when the cumulative number of frames (cumulative period) in which a single worker exists exceeds the threshold value. By using this determination condition, for example, it is possible to suppress false notification that a person is a single worker due to a momentary failure in detecting a person even though he / she is a collaborative worker. Note that the control unit 121 may notify the user of a warning when the time in which the single worker exists exceeds the threshold value, instead of determining by the cumulative number of frames. The control unit 121 may notify the user of a warning when the period (cumulative number of frames or time) in which a single worker exists exceeds the threshold value. Further, the control unit 121 may notify the user of a warning when the continuous period (continuous period) in which a single worker exists exceeds a predetermined threshold value. Here, it is assumed that the threshold value is stored in the memory 21, for example. The control unit 121 can change the threshold value according to a user setting operation via the input device 130 or the like. Further, the threshold value may be set to zero. That is, if there is a single worker, the user may be notified immediately.
Further, in the present embodiment, as a method of notifying the user that a single worker exists, a method of displaying a UI indicating that a single worker exists on the display device 140 will be described as an example. For example, the control unit 121 displays a display such as superimposing a double circle on the captured image 300 or the position of a single worker on the horizontal image 500 or superimposing a red circle on the display device 140 via the display control unit 125. Let the process be executed. However, this is not limited to this, and the control unit 121 may send an e-mail to the user, issue an alert from the speaker, or the like. Further, the notification may be given by causing a light emitting means such as an LED to emit light.
Next, the control unit 121 determines whether to repeat the process from the acquisition of the captured image (S200) to the determination (S210, S211) of whether or not to notify the user of the presence or absence of a single vendor (S212). In the present embodiment, the control unit 121 determines that the iterative process is terminated when it becomes impossible to acquire the captured image from the image pickup device 110 or when the user gives an instruction to interrupt the iterative process.
As described above, if there is a period of the moving image that includes a single worker who is not associated with another person in the moving image, the notification process is executed accordingly. This makes it possible to inform the observer that there is a single worker.

In the present embodiment, the control unit 121 has mainly described using an example of setting the detection area. When the control unit 121 detects a single work, the control unit 121 may superimpose the detection area when the single work is detected on the image and display it on the display device 140. Further, the control unit 121 superimposes and displays all the detection areas on the image, and when the independent work is performed, the control unit 121 changes the detection area where the independent work is performed to a color different from other detection areas. It may be displayed explicitly or highlighted, such as by blinking. By doing so, the work observer has good visibility and can grasp whether the person in the image is single or plural.
Further, in the present embodiment, an example in which a person or the like is detected and displayed by one client device is shown, but processing may be performed by a plurality of client devices. For example, a first client device for inputting a video to detect and notify an independent work or the like, and a second client device for receiving the notification and outputting an alarm or performing superimposed display. The processing may be performed. An example of the first client device is a VCA (Video Content Analysis) device. An example of the second client device is a VMS (Video Management System) device. The hardware configuration of the first client device and the second client device is the same as the hardware configuration of the client device 120 of FIG. The function of each client device is realized by the CPU of each client device executing processing based on the program stored in the memory of each client device.

By receiving the warning that there is a single worker in the work site, the work observer can find the worker who violates the rule and can call the worker's attention.

<Other Embodiments>
The present invention supplies a system or device via a network or storage medium with a program that realizes one or more functions of the above-described embodiment. It can also be realized by a process in which one or more processors in the computer of the system or apparatus reads and executes a program. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Although the preferred embodiment of the present invention has been described in detail above, the present invention is not limited to the specific embodiment. The functional configuration of the image pickup apparatus 110 may be implemented in the imaging apparatus as a software configuration realized by the CPU 12 executing processing based on a program stored in the memory 13. Further, a part or all of the functional configurations of the image pickup apparatus 110 may be mounted on the image pickup apparatus 110 as a hardware configuration. Similarly, the functional configuration of the client device 120 may be implemented in the client device 120 as a software configuration realized by the CPU 22 executing processing based on the program stored in the memory 21. Further, a part or all of the functional configurations of the client device 120 may be mounted on the client device 120 as a hardware configuration.
In addition to superimposing the double circle and the red circle on the position of the single worker, the single worker may be superposed with the character string "single worker", or a warning color such as red may be displayed. It may be superimposed on a single operator on the captured image. Further, these may be arbitrarily combined and displayed in an superimposed manner. A solid line ellipse, a double line ellipse, a character string such as "single worker", and a warning color such as red are examples of objects indicating that the person is not associated with another person.
Further, the hardware configuration of the client device and the image pickup device described above is an example, and for example, a plurality of CPUs, memories, communication I / Fs, and the like may be provided.

As described above, according to the processing of each of the above-described embodiments, when the person in the image is a single person, the user can be notified.

21 memory 22 CPU
120 Client device 140 Display device

Claims (9)

A detection means that detects a person from a moving image,
Among the persons detected by the detection means, the specific means for identifying whether the person in the first area set in the moving image is a single person, and
When the time in which the person is in the first area exceeds a predetermined time in a state where the person is identified as the single person by the specific means, notification processing of information that allows the user to recognize the existence of the single person is performed. , Notification control means that controls the notification device to execute,
Have,
The specific means is at least a person who is in the first area and whose face orientation or line of sight does not intersect with another person, and a person who is one person in the first area. A control device characterized in that when any person is present, the person is identified as a single person. The notification device is a display device and
The control device according to claim 1, wherein the notification control means causes the display device to execute the display process to execute the notification process. The control device according to claim 2, wherein the display process is a process of superimposing and displaying an object indicating that the person is a single person. Further having an acquisition means for acquiring a moving image from an imaging device,
The control device according to any one of claims 1 to 3, wherein the detection means detects a person from the moving image acquired by the acquisition means. The control device according to any one of claims 1 to 4, wherein the detection means detects a person from the moving image based on a feature amount of the person. The control device according to claim 5, wherein the detection means further detects a person from the moving image based on a feature amount of an object worn by the person. The control device according to claim 6, wherein the object worn by the person is at least one of a uniform, a bibs, and a hat. It is a control method executed by the control device.
A detection process that detects a person from a moving image,
Among the persons detected in the detection step, a specific step of identifying whether the person in the first area set in the moving image is a single person, and
In the state of being identified as the single person in the specific step, when the time that the person is in the first area exceeds a predetermined time, notification processing of information that allows the user to recognize the existence of the single person is performed. , The notification control process that controls the notification device to execute,
Including
In the specific step, at least a person who is in the first area and whose face orientation or line of sight does not intersect with another person, and a person who is one person in the first area. A control method characterized in that when any person is present, the person is identified as a single person. A program for causing a computer to function as each means of the control device according to any one of claims 1 to 7.

Images