JP7247310B2 - CONTROL DEVICE, CONTROL METHOD, PROGRAM AND SYSTEM - Google Patents

Description

The present invention relates to a control device, a control method, a program, and a system for detecting a person and displaying it on a display device.

Conventionally, a method for detecting a person from an image is known. Japanese Patent Application Laid-Open No. 2002-200001 discloses a method of detecting a person who is hidden from an image.

JP 2014-199506 A

With the technique disclosed in the above-mentioned Patent Document 1, it is possible to notify the position of the person in the image. For example, the technique can circle a person present in the image and present the location of the person to the user. However, only indicating the position of the person based on the above technology does not allow the user to determine whether the detected person is isolated from the group and works alone, or whether the detected person is working in close proximity with other people. There was a problem that could not be judged at a glance. For example, at sites that handle data that should be prevented from leaking, there are rules that prohibit working alone. There are cases where it is desirable to call attention to a lone worker who is isolated from a group and is working alone. However, just by looking at the captured image, the work supervisor cannot determine at a glance whether each of the workers in the captured image is working alone or in groups. In addition, it is a heavy burden for work supervisors to constantly monitor workers.
An object of the present invention is to present whether a person is alone.

Therefore, in the control device of the present invention, the person detected by the detection means for detecting the person in the area where the person works is a person who is working alone. On the condition that the person detected by the detection means in the area is specified as a person working alone, the person working alone in the area where the work must be done by a plurality of people. and display control means for controlling the display means to display information relating to the presence of the person.

According to the present invention, it is possible to present whether a person is alone.

It is a figure which shows an example of the system configuration|structure of a system, and a hardware configuration. 1 is a diagram illustrating an example of functional configurations of an imaging device and a client device; FIG. 6 is a flowchart illustrating an example of notification control processing; It is a figure which shows an example of a captured image. It is a figure which shows an example of the image which shows the result of having performed the person detection. It is a figure which shows an example of a horizontal plane image. FIG. 4 is a diagram showing an example of a work area frame; It is a figure which shows an example of a display screen. It is a figure which shows an example of a display screen. It is a figure which shows an example of a display screen. It is a figure which shows an example linked with an entrance/exit management system. 6 is a flowchart illustrating an example of notification control processing; It is a figure explaining an example of determination.

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the present invention will be described below with reference to the drawings.

<Embodiment 1>
In this embodiment, at a site where data to be leak-prevented is handled, where two or more workers are obliged to work together, the work inspector is a single worker (a person who cannot be associated with other people). ) or a collaborator (a person associated with another person) is presented. Such a situation is an example of a situation in which the present embodiment can be used, and can be used in various situations in which it is desired to determine whether a person is single or plural. For example, the present invention can be used for marketing, solving social problems, etc. by determining whether a passerby or a visitor is alone or in a group.
FIG. 1 shows an example of the system configuration and hardware configuration of the system according to this embodiment. In FIG. 1, an imaging device 110 performs imaging. The client device 120 drives the imaging device 110 and displays a captured image captured by the imaging device 110 . The input device 130 includes a mouse, keyboard, etc., and performs user input to the client device 120 . The display device 140 includes a display or the like, and displays images output by the client device 120 . Although the client device 120 and the display device 140 are shown as independent devices in FIG. 1, the client device 120 and the display device 140 may be integrated. Further, the input device 130 and the display device 140 may be integrated, or the client device 120, the input device 130 and the display device 140 may be integrated. A network 150 connects the imaging device 110 and the client device 120 . The network 150 is composed of a plurality of routers, switches, cables, etc. that satisfy communication standards such as a local network. In this embodiment, the communication standard, scale, and configuration are not limited as long as communication between the imaging device and the client can be performed. For example, the network 150 may be configured by the Internet, a wired LAN (Local Area Network), a wireless LAN (Wireless LAN), a WAN (Wide Area Network), or the like. Also, the number of imaging devices connected to the client device 120 is not limited to one, and may be multiple.

A hardware configuration of the imaging device 110 will be described. The imaging device 110 includes an imaging unit 11, a CPU 12, a memory 13, and a communication I/F 14 as a hardware configuration. The imaging unit 11 has an imaging device and an optical system of a subject on the imaging device, and performs imaging on the imaging device with the intersection of the optical axis of the optical system and the imaging device as the imaging center under the control of the CPU 12 . The imaging element is an imaging element such as a CMOS (Complementary Metal-Oxide Semiconductor) or a CCD (Charged Coupled Device). The CPU 12 controls the imaging device 110 as a whole. The memory 13 stores programs, images captured by the imaging unit 11, data used when the CPU 12 executes processing, and the like. The communication I/F 14 controls communication with the client device 120 via the network 150 under the control of the CPU 12 . The functions of the imaging device 110 and the like are realized by the CPU 12 executing processes based on the programs stored in the memory 13 . Note that a processor other than the CPU may be used.
A hardware configuration of the client device 120 will be described. The client device 120 includes a memory 21, a CPU 22, a communication I/F 23, an input I/F 24, and a display I/F 25 as a hardware configuration. The CPU 22 controls the client device 120 as a whole. The memory 21 stores programs, captured images transmitted from the imaging device 110, data used when the CPU 22 executes processing, and the like. The communication I/F 23 controls communication with the imaging device 110 via the network 150 based on the control of the CPU 22 . The input I/F 24 connects the client device 120 and the input device 130 and controls input of information from the input device 130 . The display I/F 25 connects the client device 120 and the display device 140 and controls output of information to the display device 140 . The CPU 22 executes processing based on the programs stored in the memory 21 to implement the functions of the client device 120 and the processing of flowcharts shown in FIGS. 3 and 12, which will be described later.

FIG. 2 is a diagram showing an example of functional configurations of the imaging device 110 and the client device 120. As shown in FIG. The imaging device 110 includes a control unit 111, a signal processing unit 112, a drive control unit 113, and a communication control unit 114 as functional configurations. The control unit 111 controls the imaging device 110 . The signal processing unit 112 processes image signals captured by the imaging unit 11 . For example, the signal processing unit 112 encodes the image captured by the imaging unit 11 . The signal processing unit 112 uses, for example, JPEG (Joint Photographic
Experts Group) can be used. Alternatively, the signal processing unit 112 uses H.264 as an encoding method. H.264/MPEG-4 AVC (hereinafter H.264) can be used. Alternatively, the signal processing unit 112 can use HEVC (High Efficiency Video Coding encoding scheme) as the encoding scheme. However, the encoding method is not limited to these. Also, the signal processing unit 112 may select an encoding method from among a plurality of encoding methods and perform encoding.
The drive control unit 113 performs control to change the imaging direction and the angle of view of the imaging unit 11 . In the present embodiment, the imaging unit 11 can change the imaging direction in the pan direction and the tilt direction, and change the imaging angle of view. However, the present invention is not limited to this. The imaging device 110 may not have the function of changing the imaging direction in the pan direction or the tilt direction, or the function of changing the angle of view. The communication control unit 114 transmits the captured image processed by the signal processing unit 112 to the client device 120 . Furthermore, the communication control unit 114 receives a control command for the imaging device 110 from the client device 120 .
The client device 120 includes a control unit 121, a communication control unit 123, an information acquisition unit 124, and a display control unit 125 as functional configurations. The control unit 121 controls the client device 120 . The communication control unit 123 receives the captured image from the client device 120 . The information acquisition unit 124 receives user input from the input device 130 and acquires input information from the input device 130 . The display control unit 125 outputs the image to the display device 140 and causes the display device 140 to execute display processing, which will be described later.

Next, the flow of processing in this embodiment will be described with reference to FIG. FIG. 3 is a flow chart showing an example of display control processing in the client device 120 . The client device 120 repeatedly acquires a captured image, that is, a moving image from the imaging device 110, and executes a process of presenting on the display device 140 whether or not the single worker is included in the acquired captured image so as to be identifiable. Let
The control unit 121 acquires a captured image from the imaging device 110 (S200). FIG. 4 is a diagram showing an example of a captured image 300 acquired by the client device 120 from the imaging device 110. As shown in FIG. In FIG. 4, workers 301 and 302 and workers 303 and 304 represent co-workers working in pairs. On the other hand, the worker 305 represents a lone worker who works alone. A workbench 306 represents a workbench located far from the imaging device 110 . A workbench 307 represents a workbench that is located near the imaging device 110 .
At this time, it is preferable to previously adjust the position, mounting angle, and angle of view of the imaging unit 11 of the imaging device 110 so that the captured images acquired by the client device 120 do not overlap with each other as much as possible. In this embodiment, the client device 120 processes the captured image acquired from the imaging device 110 . This makes it possible to process real-time live video. However, it is not limited to this. For example, the client device 120 may acquire a moving image stored in the imaging device 110 and process each frame, or may process each frame of the moving image stored in the client device 120. may be performed. Alternatively, the client device 120 may access a recording server or the like and process each frame of a moving image stored in the recording server. Also, in the present embodiment, the co-workers are in pairs, but the invention is not limited to this. The client device 120 may have three or more collaborators.

Next, the control unit 121 performs person detection processing on the captured image 300 acquired in S200 (S201). In order to perform person detection processing, the control unit 121 first scales the captured image 300 in various sizes in this embodiment. By performing scaling, people of various sizes can be detected. Next, the control unit 121 raster scans the captured image of each scale using a detection window of a specific size. At this time, the control unit 121 calculates the feature amount of the person in advance using learning data, and the error between the feature amount calculated within the detection window during scanning and the feature amount based on the learning data is the threshold value. If it is smaller than , it is judged to be a person. Person detection processing can use various methods such as pattern matching processing. FIG. 5 is a diagram showing an example of an image 400 showing the result of performing person detection on the captured image 300. As shown in FIG. In FIG. 5A, ellipses 401-405 represent the positions at which workers 301-305 are detected, respectively.

Next, the control unit 121 performs person detection processing on the captured image 300 acquired in S200 (S201). In order to perform person detection processing, the control unit 121 first scales the captured image 300 in various sizes in this embodiment. By performing scaling, people of various sizes can be detected. Next, the control unit 121 raster scans the captured image of each scale using a detection window of a specific size. At this time, the control unit 121 calculates the feature amount of the person in advance using learning data, and the error between the feature amount calculated within the detection window during scanning and the feature amount based on the learning data is the threshold value. If it is smaller than , it is judged to be a person.
FIG. 5 is a diagram showing an example of an image 400 showing the result of performing person detection on the captured image 300. As shown in FIG. In FIG. 5A, ellipses 401-405 represent the positions at which workers 301-305 are detected, respectively. Arrows 406 to 408 are half straight lines extending infinitely from the tail side of the arrow to the head side for expressing the orientation of each person's face. In addition to the detection method using the feature amount of a person as described above, at least one of a predetermined uniform, bibs, and hat is used as a worker rule, as shown in FIG. 5B. may decide to wear That is, it is also possible to store in the memory 21 or the like of the client device 120 in advance a rule for the feature amount of an object worn by a person to be detected as a worker as a setting file. In this way, the control unit 121 selects a uniform or the like having a feature amount that matches the rule based on the rule of the setting file and the feature amount such as the color and shape of the object worn by the person whose captured image is to be processed. It is also possible to detect workers wearing them.

ここで、式（２）中のｐｘ１、ｐｘ２、．．．、ｐｘ４、ｐｙ４、及びｑｘ１、ｑｘ２、．．．、ｑｘ４、ｑｙ４は、ユーザーによって指定された撮像画像上の四点のｘ、ｙ座標、及び水平面画像上の四点のｘ、ｙ座標をそれぞれ表す。
次に制御部１２１は、撮像画像上のｘ座標ｐｘ、Ｙ座標ｐｙとホモグラフィ行列Ｈとに対し、以下の式（３）を適用することにより、ｐｘ、ｐｙに対応する水平面画像上のＸ座標ｑｘと、Ｙ座標ｑｙとを計算することができる。

Next, the control unit 121 converts the position of the person detected in S101 into coordinates on a horizontal plane image obtained by projecting the imaging range of the captured image from directly above (S202). In the present embodiment, it is assumed that each point on the captured image corresponds to a point on the horizontal plane image, which is determined in advance. For example, the user operates the input device 130 or the like to designate four points on the captured image displayed on the display device 140 or the like and four points on the horizontal plane image. The control unit 121 calculates the homography matrix H in advance based on the four points on the captured image input by the user, the four points on the horizontal plane image, and the following equations (1) and (2). .

Here, px1, px2, . . . , px4, py4, and qx1, qx2, . . . , qx4, qy4 respectively represent the x, y coordinates of four points on the captured image and the x, y coordinates of four points on the horizontal plane image designated by the user.
Next, the control unit 121 applies the following equation (3) to the x-coordinate px, the Y-coordinate py on the captured image, and the homography matrix H, so that the X on the horizontal plane image corresponding to px, py A coordinate qx and a Y coordinate qy can be calculated.

FIG. 6 is a diagram showing an example of a horizontal plane image 500 of a work site. In FIG. 6, coordinates 501 to 505 represent results of transforming representative points of ellipses 401 to 405, which are human detection positions, into coordinates on the horizontal plane image. In this embodiment, the representative point is the center of the person detection position as an example. The representative point may be the center of the area corresponding to the person's head. Areas 506 and 507 respectively represent areas when the workbenches 306 and 307 are represented on the horizontal plane image. In this embodiment, it is assumed that a floor plan of the work site created by CAD or the like is used as the horizontal plane image, but this is not the only option. For example, an image obtained by deforming a captured image may be used, or a handwritten image may be used. Alternatively, the image may not be included, and the program may simply have the aforementioned coordinate information. That is, the horizontal plane image 500 may not be generated, and only the coordinate position on the horizontal plane image 500 may be calculated. Also, if an image is captured with a composition close to the horizontal plane image 500 (such as a bird's-eye view), the process of S203 does not necessarily have to be performed. In this case, it is possible to determine whether the worker is a single worker or a joint worker by using the distance on the captured image. Also, various other methods may be used for generating the horizontal plane image 500 .

Next, the control unit 121 updates the list of detected persons (S203). If it is the beginning of the repetition process, the control unit 121 lists all the detected persons. At this time, the control unit 121 stores the coordinates of the representative point on the captured image where the person is detected and the coordinates of the representative point on the horizontal plane image in each node in the list. If it is the second or later repetition process, the control unit 121 performs the following process on all detected persons.
(Processing 1). The control unit 121 calculates the distance between the coordinates on the captured image of the person stored in each node in the list and the coordinates on the captured image of itself. Replace with data. At this time, the control unit 121 saves the history of the coordinates saved in the node until now.
(Processing 2). If there is no node that satisfies the conditions in (process 1), the control unit 121 adds its own data to the end of the list.
Further, the control unit 121 performs the following processing after completing the above processing for all the detected persons.
(Processing 3). The control unit 121 deletes nodes that have not been updated.

Next, the control unit 121 determines whether or not the lone worker exists in the captured image (S204 to S206). In this embodiment, as shown in FIG. 7A, the control unit 121 sets work area frames ( 601 to 606 ) within the captured image display based on user operation or the like. The flow of processing is as follows.
(Processing 4). The control unit 121 checks whether the coordinates of the person to be processed on the captured image are within the work area frame, starting from the top of the list. Stores a flag indicating that the At this time, if there are a plurality of work areas, the control unit 121 stores a unique ID number assigned to each work area frame in a list (S204).
(Processing 5). The control unit 121 searches the list and confirms how many workers belong to each work area frame (S205). When the number of workers in the work area frame is 1, the control unit 121 determines that an individual work is being performed in that area, and otherwise determines that an individual work is not being performed (S206). . That is, when there is one worker in the work area frame, the control unit 121 determines that the worker in that area is a single worker who is not associated with another person. When there are two or more workers in the work area frame, the control unit 121 associates the workers in the area with each other and determines that they are joint workers. Thus, the control unit 121 functions as an associating unit.
In this embodiment, the control unit 121 first determines whether or not the person is within the work area frame based on the list of detected persons. It may be determined by checking and only the number of people in the work area.
Also, the description has been given assuming that the work area frame is set within the captured image display, but as shown in FIG. may

Here, the processing for determining whether the worker is a single worker or a joint worker is not limited to the processing described above. For example, the control unit 121 associates the persons with each other based on at least one of the positions of the persons detected from the captured image, the orientation of the faces of the persons, and the distance between the persons, and identifies them as collaborators. You may make it judge. For example, the control unit 121 may associate people in positions within a predetermined range from a predetermined object in the captured image, such as a workbench, etc., or estimate the face direction (or line of sight) of the person, It is also possible to associate persons whose face directions (or line of sight) intersect with each other. Further, for example, the control unit 121 may associate persons whose distance between them is within a predetermined distance. Then, the control unit 121 may determine that persons who are associated with other persons are joint workers, and persons who are not associated with other persons are independent workers.
In addition, the control unit 121 may further associate persons with each other based on the uniforms, bibs, or hat colors or shapes of the persons. For example, the control unit 121 associates persons whose uniform shape is the same among persons whose distance between them is within a predetermined distance, and even if the persons whose distance is within a predetermined distance may not be associated with each other. In this way, the control unit 121 may associate the person based on the feature amount of the object worn by the person.
In addition, even when a plurality of workers are present within the work area frame, the control unit 121 can determine whether the face direction of the plurality of workers within the work area frame or the distance between the workers is different. , may be determined to be a single worker.

Next, the control unit 121 determines whether or not the individual work is being performed, and if the conditions are satisfied, records the log as the independent work and notifies (S207 to S209). The flow of processing is as follows.
(process 6). If there is a work area frame in which independent work is being performed in (process 5), the control unit 121 increments the single work accumulated frame count, that is, increases the value of the variable by 1, and at the same time A person belonging to the work area is extracted from the list (S207). When the independent work is not performed, the control unit 121 sets the number of frames accumulated for the independent work to 0 (S208).
(process 7). The control unit 121 determines that the single work is being performed when the single work accumulated frame count exceeds the threshold (S209). By using this determination condition, it is possible to suppress false notification that the worker is a single worker due to a momentary failure to detect a person despite the fact that the worker is a co-worker.
When the control unit 121 determines that it is an independent work, the control unit 121 displays the occurrence time and duration of the work, the imaging device 110 where the work is occurring, and the display device 140 so that the location of the individual work in the captured image can be known. is displayed on the screen (S210). The screen of FIG. 8(A) is an example of a screen showing a list of moving images in the time zone in which the single work was detected. The text information "Individual work detection list" and the time, duration, and location information shown in FIG. is. It should be noted that, depending on the application, it is also possible to display a list of time-framed moving images that do not include a person who is not associated with another person. In that case, for example, text information such as "list of joint work detections" is displayed.
Next, the control unit 121 determines whether or not to repeat the processing from acquiring the captured image (S200) to determining whether or not to notify the user of the existence of a single trader (S210, S211) (S212). The control unit 121 determines to end the repetitive processing when acquisition of the captured image from the imaging device becomes impossible or when the user gives an instruction to interrupt the repetitive processing.

In the display at S210, the control unit 121 displays the information (list) about the independent work together with the thumbnail image as shown in FIG. In addition, the control unit 121 may display a timeline together with the display of the recorded video as shown in FIG. 8B, and display the time zone of the independent work on the timeline. The portion of the timeline represented as the time zone of independent work in FIG. 8B (blacked out and hatched portions) is an example of information that can be identified as including a person who is not associated with another person. . In addition, for example, the black-painted portion is a time period in which there is one single worker, and the shaded portion is a time period in which a plurality of single workers are present. Also, "A-1" is information indicating an ID or the like for specifying the imaging device. The example of FIG. 8B shows timelines for two imaging devices "A-1" and "B-1".
On the other hand, it is identifiable that a person who is not associated with another person is not included in the portion (white portion) of the timeline represented as the time period that is not the time period for working alone in FIG. 8B. It is an example of information. In addition, the control unit 121 may display the time zones of independent work detected by a plurality of cameras on one timeline. As described above, at least on the timeline, information that can identify that a person who is not associated with other persons is included and information that can be identified that a person who is not associated with other persons is not included At least one of the above may be displayed. It suffices if, in the moving image, it is possible to distinguish between a time period in which a person who is not associated with another person is included and a time period in which a person who is not associated with another person is not included.
Also, for example, the user designates the display of each independent work with a mouse, finger, or the like, as shown in FIG. 9A. Then, the control unit 121 causes the display device 140 via the display control unit 125 to reproduce the video recorded by the corresponding imaging device and display the captured image at that time, as shown in FIG. You may enable it to confirm. By doing so, user convenience is enhanced. At this time, the control unit 121 may reproduce only the recorded video at the time of occurrence of the independent work. Replaying for a longer time than the time is more preferable because the situation before and after can be easily confirmed.
In addition, the control unit 121 may connect the recorded images of the time period detected as the independent work and reproduce them as a digest image. In other words, the moving images in the list shown in FIG. 9A may be reproduced continuously. In this case, the digest video may not include a moving image of a time period during which the worker works alone for several seconds.
In order to prevent erroneous detection and the like, the notification of the single work will issue an alarm if the single work continues for a predetermined time, but the duration of the single work is the time from the first frame detected. Therefore, the control unit 121 may set the section displayed as an independent work on the timeline or the start of recording or playback to be the time when the independent work is detected instead of the time when the alarm is issued.

In addition, the control unit 121 may display the timeline as follows, in addition to the single work time zone when the cumulative number of frames in which the single worker exists exceeds the threshold value and an alarm is issued. That is, even if the control unit 121 determines that the work is performed alone for a short period of time during which the warning is not issued, the control unit 121 confirms both by displaying in a different color, shape, or pattern from the time zone for the single work in which the warning was issued. It may be made possible. In this case, there is a possibility that the lone work detected only for a moment may actually be caused by one of the co-workers temporarily hiding behind a shield. should be provided. At this time, the control unit 121 issues an alarm based on the larger one of the two thresholds. For example, the first threshold is used for information that can be identified as containing a person who is not associated with another person. Then, a second threshold, which is larger than the first threshold, is used to determine whether to issue an alarm.
In addition, the control unit 121 may save a list of imaging devices, areas, and time information in which independent work has been detected as a CSV file, and paste the list with thumbnail images, screen shots of recorded videos, etc. as a report. , may be made printable.
In addition to the captured image for detecting the independent work, if there is an image capturing device that similarly captures an area at a different angle or captures a surrounding area, the control unit 121 performs the following operations. You may make it process. In other words, the control unit 121 may display a video that has been captured and recorded by an imaging device that has been set to be associated in advance.
Furthermore, as shown in FIG. 10(A), in the case of a system that detects independent work in live video and issues a warning in real time if it is determined that it is a single work, the work supervisor can check the current situation and , it is necessary to check the past situation. Therefore, the control unit 121 may control to display both the live video and the recorded video. In FIG. 10A, by changing the index on the upper right in the image of room A, information is displayed that makes it possible to identify that a person who is not associated with another person is included.
As shown in FIG. 10B, the control unit 121 may delete the selected image or the like from the series of displays when the work supervisor determines that there is no false alarm or problem.

In this embodiment, the control unit 121 has mainly been described using an example of setting the work area frame. When the control unit 121 detects the independent work, the control unit 121 may display the work area frame when the independent work is detected on the display device 140 by superimposing it on the image. In addition, the control unit 121 superimposes and displays all the work area frames on the image, and when the independent work is performed, the work area frame where the independent work is performed is displayed in a different color from the other work area frames. It may be displayed explicitly or emphatically, such as by changing to , blinking, or the like. By doing so, the work supervisor can grasp with good visibility whether the person in the video is single or plural.
Also, in the present embodiment, an example in which a person or the like is detected and displayed by one client device has been described, but processing may be performed by a plurality of client devices. For example, a first client device receives video input, detects and notifies independent work, etc., and a second client device receives the notification, outputs an alarm, displays a list, or displays on a timeline. , and the client device may perform the processing. An example of the first client device is a VCA (Video Content Analysis) device. An example of the second client device is a VMS (Video Management System) device. The hardware configurations of the first client device and the second client device are the same as the hardware configuration of the client device 120 in FIG. The functions of each client device are realized by the CPU of each client device executing processing based on a program stored in the memory of each client device.

FIG. 11A shows an example of cooperation between the system of this embodiment and the entry/exit management system 160 . In the entry/exit management system, the times when workers enter and leave the work room are recorded. Therefore, as shown in FIG. 11(B), the control unit 121 acquires information on the entry and exit times of each worker from the entry/exit management system 160, and based on the acquired information, the work entered into the work room is determined. extract the person Then, the control unit 121 may record the list of the extracted worker names as a part of the information of the single work occurrence list or display it on the screen. However, this is not the only option, and the control unit 121 may send an email to the user. This recording and display may be realized by the system of the present embodiment, or may be incorporated into an entrance/exit management system or the like. The list of extracted worker names is an example of room entry/exit information.

<Embodiment 2>
In the present embodiment, the configuration is the same as that of the first embodiment, and the determination of whether or not there is a lone worker is performed based on the distance instead of the detection based on the work area frame. The flow of processing in this embodiment will be described with reference to FIG.
The processing from when the control unit 121 acquires a captured image from the imaging device 110 (S1101) to when the list of detected persons is updated (S1103) is the same as the processing described in the first embodiment.
Next, the control unit 121 determines whether or not the lone worker exists in the captured image (S1104 to S1106). The flow of processing is as follows.
(Processing 1). The control unit 121 treats the person at the top of the list as a processing target (S1104).
(Processing 2). The control unit 121 determines whether the person to be processed is a single worker (S1105). In this embodiment, the control unit 121 confirms the following items for a person to be processed in order to determine whether the person is a single worker.

(Item 1). The control unit 121 determines whether there is a person detected within a radius R centered on the coordinates of the person to be processed on the horizontal plane image.
This radius R may be appropriately set according to the situation to which the present embodiment is applied. For example, when monitoring indoor work, the radius R can be a distance corresponding to 2 m. However, at this time, the control unit 121 refers to the coordinate history stored in the node of the person to be processed, and determines that the person to be processed is moving. The value of R, which is the threshold value, is increased. Further, at this time, the control unit 121 may use an ellipse obtained by contracting a circle with a radius R in the y-axis direction if co-workers work side by side in the x-axis direction on the horizontal plane image. Similarly, the control unit 121 may use an ellipse obtained by contracting a circle with a radius R in the x-axis direction if co-workers work side by side in the y-axis direction on the horizontal plane image.
(Item 2). When a person satisfying (item 1) exists, the control unit 121 determines whether or not the person exists within the detection area (work area frame).
In this embodiment, the control unit 121 sets an area designated by the user by operating the input device 130 or the like as the detection area where the worker exists. As a result, for example, it is possible to prevent a non-worker moving behind a glass wall and a person appearing on a poster placed on a desk from being erroneously detected as a co-worker. However, the setting of the detection area is not limited to input by the user via the input device 130 or the like. The control unit 121 may automatically set an area surrounded by markers placed on the captured image as the detection area. As a method of detecting using a marker, for example, two-dimensional markers placed at the four corners of the detection area are raster-scanned in the captured image, thereby detecting the above-described two-dimensional markers registered in the client device 120 in advance. There are ways to pattern match information.
If both (item 1) and (item 2) above are satisfied, the control unit 121 determines that the person is a collaborator, and proceeds to (process 3-1). That is, the control unit 121 associates workers who satisfy all of the above (item 1) and (item 2) with each other and determines that they are joint workers. If at least one condition is not satisfied, the control unit 121 determines that the person to be processed is a single worker, and proceeds to (processing 3-2).

Here, the processing for determining whether the worker is a single worker or a joint worker is not limited to the processing described above. For example, the control unit 121 may associate persons with each other based on at least one of the positions of the persons detected from the captured image, the orientation of the faces of the persons, and the distance between the persons. For example, the control unit 121 may associate people in positions within a predetermined range from a predetermined object in the captured image, such as a workbench, etc., or estimate the face direction (or line of sight) of the person, It is also possible to associate persons whose face directions (or line of sight) intersect with each other. Further, for example, the control unit 121 may associate persons whose distance between them is within a predetermined distance. Then, the control unit 121 may determine that persons who are associated with other persons are joint workers, and persons who are not associated with other persons are independent workers. In addition, the control unit 121 may further associate persons with each other based on the color or shape of their uniforms, bibs (vest-like clothing), or hats. For example, the control unit 121 associates persons whose uniform shape is the same among persons whose distance between them is within a predetermined distance, and even if the persons whose distance is within a predetermined distance It is also possible to make judgments based on a plurality of conditions such that persons with different distances are not associated with each other, or a combination of different colors of bibs within a predetermined distance is a collaborator, and others are a single worker.

式（４）中のｑｘ、ｑｙはそれぞれ、処理対象としている人物の、水平面画像上でのｘ座標、及びｙ座標を表す。図１３では、ｑｘ、ｑｙはそれぞれ楕円１２０２のｘ座標、及びｙ座標を表す。式（４）中のａ、ｂは楕円の変換パラメータである。例えば、ａ＝１、ｂ＝１とすると、楕円は半径Ｒ、中心（ｑｘ、ｑｙ）の円となる。また、ａ＝１、ｂ＝１／２とすると、ｘ軸方向の長さがＲ、ｙ軸方向の長さがＲ／２となる楕円となる。本実施形態では共同作業者同士が水平面画像上のｘ軸方向に並んで作業を行うため、ａ＞ｂとなる値を用いることとする。 Determinations in the above (item 1) and (item 2) will be described with reference to FIG. In FIG. 13, ellipses 1201 to 1205 represent results of transforming the centers of ellipses 401 to 405, which are human detection positions, into coordinates on the horizontal plane image. Also, area 1206 represents the detection area specified by the user. Also, an ellipse 1200 represents an ellipse composed of x and y that satisfies the following equation (4).

qx and qy in Equation (4) respectively represent the x-coordinate and y-coordinate of the person to be processed on the horizontal plane image. In FIG. 13, qx and qy represent the x- and y-coordinates of the ellipse 1202, respectively. a and b in Equation (4) are ellipse transformation parameters. For example, if a=1 and b=1, the ellipse is a circle with radius R and center (qx, qy). Further, when a=1 and b=1/2, the ellipse has a length of R in the x-axis direction and a length of R/2 in the y-axis direction. In the present embodiment, co-workers work side by side in the x-axis direction on the horizontal plane image, so a value that satisfies a>b is used.

For example, it is assumed that the control unit 121 sets the person in the ellipse 402 of the person detection position as the processing target during scanning of the list. At this time, the control unit 121 makes the above (item 1) to (item 2) determinations with respect to the coordinates of the ellipse 1202 on the horizontal plane image corresponding to the ellipse 402 of the human detection position. First, in (item 1), the control unit 121 determines whether or not the detected person exists inside the ellipse 1200 . In the situation shown in FIG. 13, the coordinates of the ellipse 1201 on the horizontal plane image corresponding to the ellipse 401 of the person detection position are included in the ellipse, so the control unit 121 determines that the determination is satisfied. Next, in (item 2), the control unit 121 determines whether the coordinate 601 on the horizontal plane image corresponding to the ellipse 401 of the detection position of the person who satisfies the condition of (item 1) is included in the area 1206. . Under the circumstances described in the example of this embodiment, the coordinates of the ellipse 1201 are included in the area 1206, so the control unit 121 determines that the determination is satisfied. Since both (item 1) and (item 2) are satisfied, the control unit 121 determines that the person in the ellipse 402 is a co-worker.

(Processing 3-1). The control unit 121 determines whether or not the person at the end of the list is to be processed (S1106). If the person is not at the end of the list, the control unit 121 sets the next person as the processing target, and returns to (processing 2) (S1107). If it is the end of the list, the control unit 121 ends the scanning in the list and sets the cumulative number of frames in which the lone worker exists to 0 (S1108).
(processing 3-2). The control unit 121 increments the cumulative number of frames in which the lone worker exists (S1109).
Next, the control unit 121 determines whether or not to notify the user of the existence of the lone worker, and notifies the user if the conditions are satisfied (S1110, S1111).
The control unit 121 performs the same processing as in the first embodiment regarding the display processing and the like at the time of notification.
Further, in the present embodiment, the control unit 121 displays a UI indicating the existence of the lone worker on the display device 140 as a method of notifying the user of the existence of the lone worker. For example, the control unit 121 superimposes a double circle or a red circle on the position of the lone worker on the captured image 300 or the horizontal plane image 500 . However, this is not the only option, and the control unit 121 may send an e-mail to the user, or output to an external output and issue an alert from a speaker.

<Other embodiments>
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or device via a network or storage medium. It can also be realized by a process in which one or more processors in the computer of the system or device reads and executes the program. It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

Although preferred embodiments of the invention have been described in detail above, the invention is not limited to such specific embodiments. The functional configuration of the image capturing apparatus 110 may be implemented in the image capturing apparatus as a software configuration realized by executing processing based on programs stored in the memory 13 by the CPU 12 . Also, part or all of the functional configuration of the imaging device 110 may be implemented in the imaging device 110 as a hardware configuration. Similarly, the functional configuration of the client device 120 may be implemented in the client device 120 as a software configuration realized by the CPU 22 executing processing based on the programs stored in the memory 21 . Also, part or all of the functional configuration of the client device 120 may be implemented in the client device 120 as a hardware configuration.
In addition to superimposing a double circle or a red circle on the position of the lone worker, it is also possible to superimpose a character string such as "lone worker" on the position of the lone worker, or use a warning color such as red. It may be superimposed on the single worker on the captured image. Alternatively, these may be combined arbitrarily and displayed in a superimposed manner.
Moreover, the hardware configurations of the client device and the imaging device described above are examples, and for example, a plurality of CPUs, memories, communication I/Fs, and the like may be provided.

As described above, according to the processing of each embodiment described above, it is possible to appropriately present whether the person in the image is single or plural. By receiving a notification that there is a lone worker in the work site and easily checking the video, the work inspector can find the worker who is violating the rules and warn the worker. can be encouraged.

21 memory 22 CPU
120 client device 140 display device

Claims (20)

an identifying means for identifying whether a person detected by a detecting means for detecting a person in an area where a person works is a person working alone;
On the condition that the person detected by the detection means in the area is specified by the specifying means as a person who is working alone, in the area where a plurality of people must work a display control means for controlling the display means to display information about the presence of a person working in the
A control device comprising: The identifying means identifies whether the duration of the person detected by the detecting means working alone in the area exceeds a predetermined time,
The display control means, on the condition that the specifying means specifies that the person detected by the detecting means in the area has been working alone for a duration exceeding a predetermined time, 2. The control device according to claim 1, wherein said information relating to the presence of a person working in said room is displayed on said display means. 3. The control device according to claim 1, wherein the detection means detects a person from a moving image of an area in which the person works. The detection means detects a person from a moving image of an area in which the person works,
The display control means detects that the cumulative number of frames in which the identifying means identifies the person detected by the detecting means in the area as a person working alone exceeds a threshold. 3. The control device according to claim 1, wherein, as a condition, said information relating to the presence of a person working alone in said area is controlled to be displayed on said display means. 5. The display control means according to claim 1, wherein the display control means performs control so as to highlight, as the information, the area superimposed and displayed on the captured moving image. Control device. 6. The control device according to claim 5, wherein the display control means performs control so as to highlight the area by changing the color of the frame of the area. 7. The specifying unit according to any one of claims 1 to 6, wherein the specifying unit specifies whether the person detected by the detecting unit is a person working alone for each area. Control device. A setting means for setting the area input by the user on the captured image,
8. The identifying means identifies whether the person detected by the detecting means is working alone in the area set by the setting means. 2. The control device according to item 1. 9. The method according to any one of claims 1 to 8, wherein, when the person detected by the detecting means is one person in the area, the identifying means identifies the person as a person working alone. The control device according to any one of claims 1 to 3. When the detection means detects a plurality of persons and the distance between the plurality of persons is greater than a predetermined distance, the identification means identifies the person as a person working alone. The control device according to any one of claims 1 to 9, characterized by: The identifying means identifies the person as a person working alone when a plurality of persons are detected by the detecting means and the face orientations or lines of sight of the plurality of persons do not intersect. The control device according to any one of claims 1 to 10, characterized in that: The display control means controls not to display the information on condition that the person detected by the detection means in the area is specified by the specifying means as not a person working alone. 12. The control device according to any one of claims 1 to 11, characterized in that: an input means for accepting a specification by a user;
An area specified by the user through the input means in the captured image is set as an area for identifying whether the person detected by the detection means for detecting a person from the captured image is a person working alone. setting means;
a specifying means for specifying whether the person detected by the detecting means is a person working alone in the area set by the setting means;
display control means for controlling to display a timeline related to the captured image;
has
The identifying means further identifies whether or not the duration of the person detected by the detecting means working alone in the area exceeds a predetermined time,
The display control means includes a time zone of the duration specified by the specifying means as exceeding the predetermined time, a time zone of the duration specified by the specifying means as not exceeding the predetermined time, A control device that controls to display information that can identify the on the timeline. an input means for accepting a specification by a user;
An area specified by the user through the input means in the captured image is set as an area for identifying whether the person detected by the detection means for detecting a person from the captured image is a person working alone. setting means;
a specifying means for specifying whether the person detected by the detecting means is a person working alone in the area set by the setting means;
and display control means for controlling to display a timeline related to the captured image,
The setting means can set a plurality of the areas on the captured image,
The display control means controls a time period in which there are a plurality of areas specified by the specifying means that the person detected by the detection means is a person working alone, and is a person working alone, and a time period in which there is a single area specified by the specifying means, and information that can be identified is displayed on the timeline. Control device. A control method executed by a control device,
an identification step of identifying whether the person detected by the detection means for detecting the person in the area where the person works is a person working alone;
In the identification step, on the condition that the person detected by the detection means in the area is identified as a person working alone, a person working alone in the area where a plurality of people must work and a display control step of controlling to display information about the presence of a person working in the . an input step that accepts user specifications;
The area specified by the user in the input step in the captured image is set as an area for identifying whether the person detected by the detection means for detecting a person from the captured image is a person working alone. a configuration step;
a specifying step of specifying whether the person detected by the detecting means is a person working alone in the area set in the setting step;
a display control step of controlling to display a timeline related to the captured image;
including
In the identifying step, it is further identified whether or not the duration of the person detected by the detecting means in the area being a person working alone exceeds a predetermined time,
In the display control step, the time zone of the duration specified in the specifying step as exceeding the predetermined time, and the time zone of the duration specified in the specifying step as not exceeding the predetermined time; A control method characterized in that information capable of identifying a is controlled to be displayed on the timeline. an input step that accepts user specifications;
The area specified by the user in the input step in the captured image is set as an area for identifying whether the person detected by the detection means for detecting a person from the captured image is a person working alone. a configuration step;
a specifying step of specifying whether the person detected by the detecting means is a person working alone in the area set by the setting step;
a display control step of controlling to display a timeline related to the captured image;
In the setting step, a plurality of areas can be set on the captured image,
In the display control step, a time zone in which a plurality of areas are specified in the specifying step as the person detected by the detecting means as a person working alone, and the person detected by the detecting means. is a person who is working alone, and a time period in which there is a single area specified in the specifying step, and information that can be identified is displayed on the timeline. control method. A program for causing a computer to function as the control device according to any one of claims 1 to 14. an identifying means for identifying whether a person detected by a detecting means for detecting a person in an area where a person works is a person working alone;
On the condition that the person detected by the detection means in the area is specified by the specifying means as a person who is working alone, in the area where a plurality of people must work a display control means for controlling the display means to display information about the presence of a person working in the
A system characterized by comprising: A control method implemented by a system comprising:
an identification step of identifying whether the person detected by the detection means for detecting the person in the area where the person works is a person working alone;
In the identification step, on the condition that the person detected by the detection means in the area is identified as a person working alone, a person working alone in the area where a plurality of people must work a display control step for controlling to display information about the presence of a person working in
A control method comprising:

Images