JP2020074506A - Person behavior monitoring apparatus and person behavior monitoring system - Google Patents

Abstract

To provide a person behavior monitoring apparatus capable of efficiently monitoring persons staying and prowling in a facility in real-time even in an environment where many people come and go.SOLUTION: The person behavior monitoring apparatus includes a processor 12 that is configured to acquire pick-up images 31 from each of multiple cameras 1 to obtain tracking information by analyzing the pick-up images 31 and to generate a monitor image including images relevant to a prowling person. The processor 12 is configured to generate a monitor image by overlapping a first-person rectangle 32 representing a candidate of the prowling person and a second person rectangle 32 representing the prowling person with the pick-up image 31 with respect to the person detected from the pick-up images 31 for each camera 1 on the basis of first and second tracking tables.SELECTED DRAWING: Figure 7

Description

  The present invention relates to a person behavior monitoring apparatus and a person behavior monitoring system that generate a monitoring image from captured images of a plurality of cameras installed in a facility and display it on a display device.

  On the premises of a railway station, if there is a person who performs dangerous behavior, it will interfere with safe and smooth train operation. It is necessary to discover it and take appropriate action early.

  As a technique for quickly finding such a person requiring attention, a technique for recognizing a person's action from a captured image of a camera to determine suspicious behavior and finding a suspicious person is known. Reference 1). In addition, a person is detected from the image captured by the camera, the number of appearances that appears in the captured image is acquired for each person, and when the number of appearances is large, it is determined that there is a high possibility of prowling, and a prowler is detected. The technology is known (see Patent Document 2).

JP, 2010-272958, A International Publication No. 2014/132841

  However, in these conventional techniques, in an environment where a large number of people come and go, such as in a station yard, in some cases, a large number of people who need attention are discovered, so it is appropriate to identify which person should be noted. There is a problem in that it is not possible to make a judgment, and the supervisor cannot efficiently perform the monitoring work for monitoring the person staying in the facility in real time.

  Therefore, according to the present invention, even in an environment in which many people come and go, a person who can watch a person who needs attention in a real-time manner and can efficiently perform a monitoring operation of monitoring a person who stays in a facility in real time can be performed. The main purpose is to provide a monitoring device and a person behavior monitoring system.

  The human behavior monitoring device of the present invention is a human behavior monitoring device that generates a monitoring image based on captured images of a plurality of cameras installed in a facility and displays the monitoring image on a display device, and A processor is provided that acquires a captured image, acquires tracking information by analyzing the captured image, and generates a monitoring image that includes an image of a prowling person, and the processor is detected from the captured image for each camera. The tracking information including the long-stay flag corresponding to the person being tracked is registered in the first tracking table, and among the persons registered in the first tracking table, the tracking information The tracking information corresponding to the person who has completed the tracking process is registered in the second tracking table, and the person being tracked registered in the first tracking table is the first long-term stay flag. If the person is a person who has become true, the first fixed image indicating that the person is a prowling person candidate is superimposed on the captured image, and the tracking registered in the first tracking table is performed. The long stay flag of the inside person becomes true, and the person being tracked is compared with the person registered in the second tracking table to which the long stay flag is true, and the same person is checked. If it is determined that the person is a probable person, the first fixed image is changed to a second fixed image indicating that the person is a prowling person, and the second fixed image is superposed on the captured image. Is generated.

  Further, the person behavior monitoring system of the present invention includes a plurality of cameras installed in a facility, a server device that acquires captured images of the plurality of cameras via a network and generates a monitoring image, and outputs from the server device. And a display device that displays the monitoring image, wherein the server device acquires the captured images of each of the plurality of cameras and analyzes the captured images to provide tracking information. And a processor that generates a surveillance image including an image of a prowling person, the processor performing, for each of the cameras, tracking processing of the person detected from the captured image, and corresponding to the person being tracked. The tracking information including the long-term stay flag is registered in the first tracking table, and among the persons registered in the first tracking table, the tracking information of the tracking process is registered. When the tracking information corresponding to the completed person is registered in the second tracking table and the person being tracked registered in the first tracking table is the person whose long-time stay flag becomes true for the first time. , The first fixed image indicating that the person is a prowling person candidate is superimposed on the captured image, and the long stay flag of the person being tracked registered in the first tracking table is If it is true, the person being tracked is compared with the person whose true long-term stay flag registered in the second tracking table is true, and if it is determined that they are the same person, then that person is Then, the first fixed image is changed to a second fixed image indicating a prowling person, and a monitoring image is generated by superimposing the second fixed image on the captured image.

  According to the present invention, based on the tracking information registered in the first and second tracking tables, the first fixed image and the second fixed image capable of identifying the degree of stay of a person are superimposed on the captured image. By doing so, the surveillance person can notice the prowling person after recognizing the high need for attention. As a result, even in an environment in which many people come and go, it is possible to efficiently perform the monitoring work for monitoring the prowling people staying in the facility in real time.

Overall configuration diagram of a person behavior monitoring system according to the present embodiment Block diagram showing a schematic configuration of the server 2 Explanatory drawing which shows the monitoring image displayed on the display apparatus 3. Explanatory drawing which shows an example of the tracking situation of a person Explanatory diagram showing registered contents of a real-time tracking table and a tracking completion table managed by the server 2. Explanatory drawing which shows the registration content of the prowling person table managed by the server 2. Flowchart showing the procedure of processing performed by the processor 12 of the server 2. Flow chart showing the procedure for long-term stay determination (ST105) Flow chart showing the processing procedure of prowl detection (ST108)

  A first invention made to solve the above problems is a human behavior monitoring apparatus that generates a monitoring image based on captured images of a plurality of cameras installed in a facility and displays the monitoring image on a display device. Acquiring the captured image for each of the cameras, obtain tracking information by analysis of the captured image, a processor for generating a monitoring image including an image of a prowling person, the processor, for each camera, Tracking processing is performed on the person detected from the captured image, the tracking information including the long-stay flag corresponding to the person being tracked is registered in the first tracking table, and is registered in the first tracking table. The tracking information corresponding to the person whose tracking processing has been completed is registered in the second tracking table, and the person being tracked registered in the first tracking table is In the case where the long-stay flag is true, a first fixed image indicating that the person is a prowling person candidate is superimposed on the captured image in association with the person, and the first tracking table is used. The long-term stay flag of the person being tracked to be registered becomes true, and person matching is performed between the person being tracked and the person to whom the long-stay flag registered in the second tracking table is true. When it is determined that they are the same person, the first fixed image is changed to a second fixed image indicating that the person is a prowling person, and the second fixed image is changed to the captured image. The configuration is such that a superimposed monitoring image is generated.

  According to this, based on the tracking information registered in the first and second tracking tables, the first fixed image and the second fixed image capable of identifying the degree of stay of a person are superimposed on the captured image. Thereby, the observer can notice the prowling person after recognizing the high degree of attention. As a result, even in an environment in which many people come and go, it is possible to efficiently perform the monitoring work for monitoring the prowling people staying in the facility in real time.

  According to a second aspect, the processor uses the second camera to capture the same person as the person whose true long-term stay flag is true by the tracking process of the captured image by the first camera. When the long-time stay flag becomes true by the image tracking processing, the first fixed image is changed to the second fixed image.

  According to this, when the long-time stay flag of the same person becomes true in the images captured by different cameras, the first fixed image is changed to the second fixed image to call the attention of the observer. can do.

  In the third invention, the first fixed image and the second fixed image are frame images surrounding at least a part of a person region in the photographed image, and the colors, shades, and lines in the frame images. The display attribute of at least one of the seed and the line thickness is changed according to the degree of stay.

  According to this, the degree of stay of a person can be clearly displayed on the monitoring image.

  Further, a fourth invention is configured such that the first fixed image and the second fixed image are at least one of a figure, a character, and a symbol displayed inside or near a person region in the photographed image. And

  According to this, the degree of stay of a person can be clearly displayed on the monitoring image.

  A fifth aspect of the present invention is that the processor, if there is a plurality of pieces of tracking information for which the long-term stay flag is true for the same person, indicates that a warning person exists in the alarm device. It is configured to perform an alarm operation to notify the observer.

  According to this, it is possible to promptly notify the monitor that the person requiring attention exists.

  A sixth aspect of the invention is to provide a plurality of cameras installed in a facility, a server device that acquires captured images of the plurality of cameras via a network to generate a monitoring image, and the server device that outputs the server device. A human behavior monitoring system comprising: a display device that displays a monitoring image, wherein the server device acquires the captured images of each of the plurality of cameras, and acquires tracking information by analyzing the captured images. A processor for generating a surveillance image including an image of a prowling person, the processor, for each of the cameras, performs a tracking process of the person detected from the captured image, and a long time corresponding to the person being tracked. The tracking information including the stay flag is registered in the first tracking table, and corresponds to the person who has completed the tracking processing among the people registered in the first tracking table. If the tracking information is registered in the second tracking table and the person being tracked registered in the first tracking table is the person whose long-time stay flag becomes true for the first time, the person is handled. Then, the first fixed image indicating that the person is a prowling person candidate is superimposed on the captured image, the long stay flag of the person being tracked registered in the first tracking table becomes true, and the tracking is performed. When it is determined that the person in the middle and the person registered in the second tracking table for which the long-term stay flag is true are the same person and the person is the same person, the first person The standard image is changed to a second standard image indicating a prowling person, and the second standard image is superimposed on the captured image to generate a monitoring image.

  According to this, similarly to the first aspect of the invention, it is possible to efficiently perform the monitoring work of monitoring the prowling person staying in the facility in real time even in an environment in which many people come and go.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is an overall configuration diagram of a person behavior monitoring system according to the present embodiment.

  This person behavior monitoring system monitors the behavior of a person who stays in the facility, and includes a camera 1, a server 2 (person behavior monitoring device), a display device 3, and an alarm device 4. ..

  A plurality of cameras 1 are installed in the facility, and each captures a plurality of monitoring areas set in the facility. This camera 1 is communicatively connected to a server 2 via a network.

  The server 2 is installed in an appropriate place inside or outside the facility. The server 2 is composed of an information processing device such as a PC, and a display device 3 is connected thereto. The server 2 acquires a captured image from the camera 1, acquires information about the state of a person in the monitoring area by analyzing the captured image, generates a monitoring image including the information, and then generates the monitoring image. It is displayed on the display device 3 and is viewed by an observer. In the present embodiment, a prowling person (a person requiring attention) is detected by analyzing a captured image, and a monitoring image in which a standard image showing the prowling person is superimposed on the captured image is generated and displayed on the display device 3.

  The alarm device 4 is connected to the server 2 and performs an alarm operation such as lighting a lamp or outputting a sound in response to an instruction from the server 2. In the present embodiment, when a prowling person is detected, the alarm device 4 is caused to perform an alarming operation to notify the monitor that a prowling person exists.

  Next, the schematic configuration of the server 2 will be described. FIG. 2 is a block diagram showing a schematic configuration of the server 2.

  The server 2 includes a communication unit 11, a processor 12, a storage unit 13, and an output unit 14.

  The communication unit 11 communicates with the camera 1 via the network. In the present embodiment, the captured image transmitted from the camera 1 is received. The storage unit 13 stores a captured image of the camera 1, a program executed by the processor 12, and the like. Under the control of the processor 12, the output unit 14 outputs screen information to the display device 3 and an alarm operation instruction to the alarm device 4.

  The processor 12 executes the program stored in the storage unit 13. In the present embodiment, a person is tracked in the image captured by the camera 1, and it is determined whether or not the person is a long-term stayer who continuously stays in the shooting area of one camera 1 for a predetermined time or longer. , A prowling person (a person who needs attention with a high degree of attention) is detected based on the result of this long stay determination. Then, a monitoring image in which a standard image showing the prowling person is superimposed on the captured image is generated and displayed on the display device 3, and an alarm operation for notifying the supervisor of the presence of the prowling person is given to the warning device 4. Let it be done.

  Next, the monitoring image displayed on the display device 3 will be described. FIG. 3 is an explanatory diagram showing a monitoring image displayed on the display device 3.

  There are people who stay for a relatively long time, such as people waiting for the arrival of a train, people taking a break, and people waiting for another person on the premises of the station. Such a person does not wander around much and continues to stay within the shooting area of one camera 1. On the other hand, a person who takes a dangerous action tends to move around a relatively wide range while walking in the shooting areas of a plurality of cameras 1 while staying in the shooting area of one camera 1.

  Therefore, in the present embodiment, a person is tracked in the captured image 31 of the camera 1, and it is determined whether or not the person is a long-term staying person in the shooting area of one camera 1 for a long time. A person who is detected as a prowling person (a person who needs attention with a high degree of caution) and is represented as a prowling person when the same person is judged multiple times as a time resident. A monitor image in which the rectangle 32 (standard image, frame image) is superimposed on the position corresponding to the person in the captured image 31 is generated and displayed on the display device 3.

  If the person who may be determined to be a prowling person is made to pay attention to the surveillance person before the prowling person is determined, it is possible to immediately respond to the prowling person at the determined timing. Therefore, in the present embodiment, when it is first determined that the person is a long-term stayer, that is, once the determination is made that the person is a long-term stayer, a prowling candidate person (a person who needs attention with a low degree of caution). It is determined that the person rectangle 32 (standard image) representing the prowling candidate person is superimposed on the position corresponding to the person in the captured image 31, and the monitor image is generated and displayed on the display device 3.

  Further, in the present embodiment, the prowling candidate person and the prowling person are identified by the difference in the display mode of the person rectangle 32. In the example shown in FIG. 3, the person rectangle 32 can be identified by the line type of the frame line. That is, the person rectangle 32 is displayed by a dotted line for the prowling candidate person, and the person rectangle 32 is displayed by a solid line for the prowling person. In the example shown in FIG. 3, the person ID assigned to each person is displayed near the person rectangle 32.

  When there are a plurality of persons who display the person rectangle 32 in one captured image 31, that is, a plurality of prowling candidate persons or prowling persons, different colors (for example, red, orange, blue, green, etc.) are set for each person. ), The person rectangle 32 may be displayed.

  Here, FIG. 3A is a monitoring image of the first camera 1 (CamA) that photographs the ticket gate. In this monitoring image, the two persons (Id1, Id2) are determined to be long-staying persons because the staying time is long, and the person rectangle 32 is displayed. , A dotted-line person rectangle 32 representing a prowling candidate person is displayed.

  FIG. 3B is a surveillance image of the second camera 1 (CamB) that photographs the home, and is captured after the surveillance image of FIG. 3A. In this monitoring image, the two people (Id11, Id22) are determined to be long-stayers because the staying time is long, and the person rectangle 32 is displayed. However, one person (Id11) is At the timing of the monitoring image of 3 (A), since it is the same as the person determined to be a long-term resident, it becomes a prowling person, and a solid-line person rectangle 32 representing the prowling person is displayed. Since the other person (Id22) has not been determined to be a long-term resident in the past, it becomes a prowl candidate person, and a dotted person rectangle 32 representing the prowl candidate person is displayed.

  In the present embodiment, a person rectangle 32 (frame image) that surrounds the whole body of a person is displayed as a standard image representing the prowling candidate person and the prowling person (degree of stay of the person), and the line type of the frame line is displayed. Although the prowling candidate person and the prowling person are identified by (dotted line and solid line), the display attribute for identifying the prowling candidate person and the prowling person is not limited to the line type of the frame line.

  For example, the prowling candidate person and the prowling person may be identified by the color of the frame line. For example, the prowling candidate person may have a green frame line and the prowling person may have a red frame line. Alternatively, the prowling candidate person and the prowling person may be identified by the shade of the frame line. For example, the prowling candidate person may have a thin frame line and the prowling person may have a dark frame line. Further, the prowling candidate person and the prowling person may be identified by the thickness of the frame line. For example, the prowling candidate person may have a thin frame line and the prowling person may have a thick frame line.

  Further, the person rectangle 32 (frame image) may be one that surrounds the whole or a part of the person area, and may be, for example, one that surrounds the upper half of the person or the face.

  Furthermore, the standard image representing the prowling candidate person and the prowler person's distinction (degree of stay of the person) is not limited to the frame image surrounding the person area, and may be at least one of figures, characters, and symbols. Alternatively, it may be displayed inside or near the person area.

  For example, as a standard image, a figure or symbol of an appropriate shape such as a circle is displayed in the center of the person area, and a figure (a balloon or an arrow) that points to the person area is displayed near the person area. The display mode of the figure may be changed between the prowling person and the prowling candidate person. Further, the characters “prowling candidate” and “prowling” may be displayed near the person area. Further, symbols representing the prowling person and the prowling candidate person may be displayed inside or near the person area.

  Next, the long stay determination and the prowl detection performed by the processor 12 of the server 2 will be described. FIG. 4 is an explanatory diagram showing an example of the tracking situation of a person.

  In the present embodiment, a long-time stay determination is performed to determine whether the person detected from the image captured by the camera 1 is a long-term stayer. In this long-time stay determination, the time during which a person is tracked in the image captured by one camera 1, that is, the elapsed time from the tracking start time to the current time is measured. Then, this measurement time is set as a stay time during which the person stays in the shooting area of one camera 1, and when the stay time exceeds a predetermined threshold value (for example, 20 seconds), the person is kept for a long time. It is determined that the person is a resident, and the person rectangle of the prowling candidate person is displayed.

  Further, in the present embodiment, when it is determined by the long stay determination that the person is a long stay person, the person who is determined to be the long stay person in the past and the person who is determined to be the long stay person this time are different from each other. When matching (person re-identification processing) is performed and a person determined to be the same person is found, the person is determined to be a prowling person, and a person rectangle of the prowling person is displayed.

  In the example shown in FIG. 4, first, the person (Id1) appears in the captured image of the second camera 1 (CamB), and this person is stayed for a long time. The person rectangle of the person is displayed. Next, the person (Id2) appears in the captured image of the first camera 1 (CamA), and since this person is not stayed for a long time because the stay time is short, the person rectangle is not displayed.

  Next, the person (Id11) appears in the captured image of the first camera 1 (CamA), and this person is determined to be a long-term resident because the staying time becomes long. At this time, there is a person (Id1) that was determined to be a long-term resident in the past, so person verification is performed with this person. If the same person is determined by this person matching, the person rectangle of the prowling person is displayed.

  Also, the person (Id44) appears in the captured image of the first camera 1 (CamA), and this person is determined to be a long-term resident because the staying time becomes long. At this time, person matching is performed with the person (Id1) that was determined to be a long-term resident in the past, but since it is not the same person, the person rectangle of the prowling candidate person is displayed.

  In addition, the person (Id222) appears in the captured image of the second camera 1 (CamB), and this person is determined to be a long-term resident because the staying time becomes long. At this time, since there are three persons (Id1, Id11, Id44) that have been determined to be long-term residents in the past, person verification is performed with these persons, but since they are not the same person, prowl The person rectangle of the candidate person is displayed.

  In the present embodiment, at the timing when it is determined that the person is a long-term resident, the person is collated with the person who was previously determined to be the long-term resident, but the person is detected from the captured image. At such timing, person verification may be performed with a person who has been determined to be a long-term resident in the past. In this way, when a prowling candidate person or a person who has been determined to be a prowling person in the past is detected from a captured image, that person is immediately displayed with a person rectangle corresponding to the prowling candidate person and the prowling person. You can

  Further, in this way, the person who has been determined to be a prowling candidate in the past has a longer staying time in the current shooting area of the camera 1, and when it is determined to be a long-term stayer, the prowling candidate person is a prowling person. Is changed to, and the person rectangle of the prowling candidate person is switched to the person rectangle of the prowling person and displayed.

  Next, the real-time tracking table and the tracking completion table managed by the server 2 will be described. FIG. 5 is an explanatory diagram showing registered contents of the real-time tracking table and the tracking completion table.

  The real-time tracking table (real-time tracking information) shown in FIG. 5A is for registering information about the person being tracked, and in this real-time tracking table, the camera ID (Cam), person ID (Id), and tracking Information on each item of a start time (Begin Time), a latest stay time (Exist Time), and a long stay flag (Long Stay Flag) is registered.

  In the present embodiment, the processor 12 of the server 2 performs person detection and person tracking processing on each frame (captured image) sequentially transmitted from the camera 1 and acquires the camera ID (Cam ), Person ID (Id), tracking start time (Begin Time), and latest stay time (Exist Time) are registered in the real-time tracking table.

  Here, the camera ID (Cam) is identification information given to each camera 1. The person ID (Id) is identification information given to each person detected from the captured image. The tracking start time (Begin Time) is the time when a person first appears in a captured image and tracking is started, that is, the capturing time of the frame in which the person is first detected. The latest stay time (Exist Time) is the current time during tracking, that is, the shooting time of the current frame in which a person is detected. The long stay flag (Long Stay Flag) indicates whether or not the person is a long stay (True / False).

  In addition, the processor 12 of the server 2 calculates the difference between the tracking start time (Begin Time) and the latest stay time (Exist Time) in the real-time tracking table as the stay time, and determines whether the detected person is a long-term stayer. A long stay determination is made to determine whether or not the result is registered in the real-time tracking table as a long stay flag. That is, the long stay flag (Long Stay Flag) is rewritten from “False” to “True” for the person who is determined to be the long-term stayer.

  The tracking completion table (confirmed tracking information) shown in FIG. 5B is for registering information about a person whose tracking has been completed. In this tracking completion table, a camera ID (Cam), a person ID (Id), Information on each item of tracking start time (Begin Time), tracking end time (Exist Time), and long stay flag (Long Stay Flag) is registered.

  Here, the item whose latest stay time (Exist Time) in the real-time tracking table shown in FIG. 5A is the target person exits the shooting area of the camera 1 and the tracking ends, It is determined as the tracking end time, that is, the last time detected in the captured image. Others are the same as the real-time tracking table.

  Next, a prowling person table managed by the server 2 will be described. FIG. 6 is an explanatory diagram showing the registered contents of the prowling person table.

  This prowling person table (prowling person information) is for registering information about a person determined to be a prowling person. In this prowling person table, the camera ID (Current Cam) currently being tracked and the person currently being tracked are tracked. Information on each item of ID (Current Id), camera ID when previously tracked (Previous Cam), person ID when previously tracked (Previous Id), and degree of prowling person (Walk Around Score) is registered. There is.

  In the present embodiment, in the processor 12 of the server 2, when the long stay flag (Long Stay Flag) becomes “True” in the real-time tracking table (see FIG. 5A), the person, that is, the long time Prowl detection is performed to determine whether the resident is a prowling person.

  In this prowl detection, a person whose long stay flag (Long Stay Flag) is “True” in the tracking completion table (see FIG. 5B), that is, a person who has been determined to be a long stay person in the past, , A person whose long stay flag (Long Stay Flag) is “True” in the real-time tracking table (see FIG. 5A), that is, the same person as the person who has been determined to be the long stayer this time. Whether or not the person is collated (person re-identification processing) is performed.

  In this person matching, the similarity between two persons is calculated as a prowling person degree (Walk Around Score), and when the probing person degree exceeds a predetermined threshold value (for example, 0.80), it is determined that they are the same person. To do. When a person determined to be the same person is found, the person is determined to be a prowling person, and information regarding the person is registered in the prowling person table.

  Next, a procedure of processing performed by the processor 12 of the server 2 will be described. FIG. 7 is a flowchart showing a procedure of processing performed by the processor 12 of the server 2. FIG. 8 is a flowchart showing a processing procedure of long stay determination (ST105). FIG. 9 is a flowchart showing a processing procedure of prowl detection (ST108).

  The processor 12 of the server 2 first acquires a captured image (frame) transmitted from the camera 1 and received by the communication unit 11, and adds a camera ID and a frame number (ST101). Then, person detection is performed on the captured image (ST102). In the camera 1, the captured image (frame) is added with the capturing time and transmitted.

  In this person detection, a person area in a captured image is specified, a person rectangle is set on the captured image so as to surround the person area, and information about the position and size of the person rectangle, that is, the upper left corner of the person rectangle is set. Get the coordinates (x, y), width (w) and height (h) of the vertex of. A known image recognition technique may be used for this person detection. A person ID is given to the person rectangle.

  Next, it is determined whether or not a person is detected from the captured image (ST103). If no person is detected, the process proceeds to the next captured image (frame). On the other hand, if a person is detected, then person tracking is performed (ST104).

  In this person tracking, the person detected in the captured image (frame) of this time is associated with the person detected in the captured image (frame) of the past. A known image recognition technique may be used for this person tracking.

  At this time, three tracking states of new tracking (new), tracking completion (complete), and tracking (track) are determined. In the case of new tracking (new), the information of the corresponding person is registered in the real-time tracking table (see FIG. 5A), and in the case of tracking completion (complete), the tracking completion table (see FIG. In ()), the information of the corresponding person is registered, and in the case of tracking (track), the latest stay time (Exist Time) of the real-time tracking table is updated.

  Next, a long-term stay determination is performed (ST105). In this long-time stay determination, it is determined whether or not the person tracked in the captured image is a long-term stayer who has continuously stayed in the shooting area of the camera 1 that captures the captured image for a predetermined time or longer. ..

  FIG. 8 shows a detailed processing procedure of this long stay determination. Here, when the tracking state (s) is tracking (track) and the stay time T exceeds the threshold TH (for example, 20 seconds) (Yes in ST203), the person who has been a long-time stay Then, the long stay flag of the corresponding person in the real-time tracking table (see FIG. 5A) is set to “True” (ST204). Here, n is the serial number of the person rectangle, N is the total number of person rectangles, and each person detected in this person detection is sequentially determined.

  The stay time T is the elapsed time from the tracking start time to the current time, and is the difference between the tracking start time (Begin Time) and the latest stay time (Exist Time) in the real-time tracking table (see FIG. 5A). Is calculated as In addition, a person whose tracking state (s) is new tracking (new) and tracking completion (complete) is excluded from the target of the long stay determination.

  When the long-term stay determination (ST105) is performed in this way, next, as shown in FIG. 7, the prowling candidate person is displayed (ST106). Here, a person who is determined to be a long-time stayer, that is, a person whose long-time stay flag (Long Stay Flag) in the real-time tracking table (see FIG. 5A) is “True” is set as a prowling candidate person. indicate. That is, the person rectangle (dotted line) of the prowling candidate person is displayed on the corresponding person in the captured image (see FIG. 3).

  Next, it is determined whether or not there is a person who has been determined to be a long-term visitor in the past, that is, a person whose long-stay flag (Long Stay Flag) is “True” in the tracking completion table (see FIG. 5B). It is determined (ST107). Here, if there is no person who has been determined to be a long-term resident in the past (No in ST107), the process proceeds to the next captured image (frame). On the other hand, when there is a person who has been determined to be a long-term resident in the past (Yes in ST107), prowl detection is performed (ST108).

  In this prowl detection (ST108), person comparison (person re-identification processing) is performed between a person who was previously determined to be a long-term resident and a person who was determined to be a long-term resident this time, and the same person is determined. If the identified person is found, the person is determined to be a prowling person, and the prowling person table (see FIG. 6) is updated.

  FIG. 9 shows the processing procedure of this prowl detection in detail. Here, k and K are persons who have been determined to be long-term stayers in the past, that is, the long stay flag (Long Stay Flag) is “True” in the tracking completion table (see FIG. 5B). The serial number and the total number of persons, and m and M are persons determined to be long-term stayers this time, that is, the long stay flag (Long Stay Flag) in the real-time tracking table (see FIG. 5A) is " It is the serial number and the total number of people who are "True". One-to-one person matching is performed in sequence between the person who was previously determined to be a long-term resident and the person who was determined to be a long-term resident this time. ing.

  Here, in the present embodiment, person verification is performed for a person who has been determined to be a long-term resident within a predetermined verification target period that is traced back from the present to the past. The length of the matching target period may be set to about several tens of minutes, for example. Further, it is preferable that the user can appropriately set the length of the matching target period.

  A known method may be used for person verification. Specifically, for two persons to be collated, a person image obtained by cutting out a person rectangle area from the photographed image is acquired, and a color feature amount is extracted from the two person images. For example, the feature amount vector is acquired by generating a histogram in the HSV color space from each person image. Next, by comparing the color feature amounts, the degree of similarity between the two person images is calculated. Then, a combination having the highest degree of similarity is selected, and when the degree of similarity is higher than a predetermined threshold value, the persons in the two person images are determined to be the same person.

  The person determined to be a long-term resident in the past and the person determined to be a long-term resident this time do not always appear in images captured by different cameras 1. That is, in the case where all the places of the facility are not photographed by the camera 1, when they appear in the photographed image of the same camera 1 after disappearing from the photographed image of the camera 1 without appearing in the photographed image of another camera 1. Therefore, in this case, there is a case where it is determined that the image captured by the same camera 1 is a long-term resident.

  When the prowl detection is performed in this way (ST108), next, as shown in FIG. 7, a prowling person is displayed (ST109). Here, a person rectangle for the prowling person is displayed on the captured image for the person detected as the prowling person by the prowl detection. At this time, for the prowling person, the long stay flag (Long Stay Flag) is “True” in the real-time tracking table (see FIG. 5A), and the same camera ID and person ID as in the real-time tracking table are combined. It is registered in the roaming person table (see FIG. 6). Further, while the long stay flag (Long Stay Flag) is “True” in the real-time tracking table, that is, while the person is detected, the display of the person rectangle for the prowling person is continued.

  The embodiments have been described above as examples of the technology disclosed in the present application. However, the technique of the present disclosure is not limited to this, and can be applied to embodiments in which changes, replacements, additions, omissions, etc. have been made. Further, it is also possible to combine the constituent elements described in the above-described embodiment to form a new embodiment.

  For example, in the above embodiment, an example of a railway station has been described, but the present invention is not limited to such a railway station, and can be widely applied to various facilities.

  Further, in the above-described embodiment, the monitor screen is displayed on the display device connected to the server so that the monitor can view the screen. By installing the server and the display device in the monitor room, the monitor screen of the display device can be displayed. Can be viewed by the monitor, but if a mobile device such as a smartphone or tablet device can be connected to the server via a wireless communication network such as a wireless LAN, the monitoring screen can be displayed anywhere in the facility. Can be viewed. Further, if the user terminal such as a mobile device or a PC can be connected to the server via a wide area communication network such as the Internet, the monitoring screen can be browsed at any place outside the facility.

  Further, in the above-described embodiment, when the determination result that the same person is a long-term stayer is obtained once, a standard image (person rectangle) of the prowling candidate person is displayed, and the same person is displayed for a long time. When the judgment result of a long stayer is obtained twice, the standard image of the prowling person is displayed. However, when the judgment result of a long stayer for the same person is obtained three times or more. The display mode of the standard image may be further changed according to the number of determinations.

  In addition, after displaying the standard image of the prowling candidate person (rectangular person), the next time, the determination result of the long staying person is immediately displayed without displaying the standard image of the prowling person (person rectangular shape). When the results are continuous (twice or more), a standard image of a prowling person may be displayed.

  The person behavior monitoring apparatus and the person behavior monitoring system according to the present invention are monitoring work in which an observer appropriately pays attention to a person who needs attention and monitors the person staying in a facility in real time even in an environment where many people come and go. It is effective as a person action monitoring device and a person action monitoring system which generate a monitor image from the images taken by a plurality of cameras installed in the facility and display it on the display device. .

1 camera 2 server (person behavior monitoring device, server device)
3 display device 4 alarm device 11 communication unit 12 processor 13 storage unit 14 output unit

Claims (6)

A human behavior monitoring device that generates a monitoring image and displays it on a display device based on images captured by a plurality of cameras installed in a facility,
Acquiring the captured image for each of the plurality of cameras, acquiring tracking information by analysis of the captured image, a processor for generating a monitoring image including an image of a prowling person,
The processor is
Tracking processing of a person detected from the photographed image is performed for each camera, and the tracking information including a long stay flag corresponding to the person being tracked is registered in the first tracking table,
Of the persons registered in the first tracking table, the tracking information corresponding to the person who has completed the tracking process is registered in the second tracking table,
When the person being tracked registered in the first tracking table is the person whose long-time stay flag becomes true for the first time, the first person indicating that the person is a prowling person candidate in association with the person. Superimpose a standard image on the captured image,
The long stay flag of the person being tracked registered in the first tracking table is true, and the person being tracked and the long stay flag registered in the second tracking table are true. When it is determined that the person is the same person by collating the person with the person, the first standard image for the person is changed to a second standard image indicating a prowling person, and the second standard image is changed. A person behavior monitoring device, characterized by generating a monitoring image in which a standard image is superimposed on the captured image. The processor is
By the tracking processing of the captured image by the first camera, the same person as the person whose true long-term stay flag becomes true is identified by the tracking processing of the captured image by the second camera by the long-term stay flag. The human behavior monitoring device according to claim 1, wherein the first fixed image is changed to the second fixed image when is true.   The first fixed image and the second fixed image are frame images that surround at least a part of the person region in the captured image, and at least the color, the shade, the line type, and the line thickness in the frame image. The human behavior monitoring device according to claim 1, wherein any one of the display attributes is changed.   The first fixed image and the second fixed image are at least one of a figure, a character, and a symbol displayed inside or near a person region in the captured image. The human behavior monitoring device according to claim 2.   When a plurality of pieces of the tracking information in which the long-term stay flag is true are present for the same person, the processor performs an alarm operation to notify an observer of the presence of a prowling person in the alarm device. The human behavior monitoring device according to any one of claims 1 to 4, wherein Multiple cameras installed in the facility,
A server device that acquires captured images of the plurality of cameras via a network and generates a monitoring image,
A display device for displaying the monitoring image output from the server device;
A human behavior monitoring system comprising:
The server device is
Acquiring the captured image for each of the plurality of cameras, acquiring tracking information by analysis of the captured image, a processor for generating a monitoring image including an image of a prowling person,
The processor is
Tracking processing of a person detected from the photographed image is performed for each camera, and the tracking information including a long stay flag corresponding to the person being tracked is registered in the first tracking table,
Of the persons registered in the first tracking table, the tracking information corresponding to the person who has completed the tracking process is registered in the second tracking table,
When the person being tracked registered in the first tracking table is the person whose long-time stay flag becomes true for the first time, the first person indicating that the person is a prowling person candidate in association with the person. Superimpose a standard image on the captured image,
The long stay flag of the person being tracked registered in the first tracking table is true, and the person being tracked and the long stay flag registered in the second tracking table are true. When it is determined that the person is the same person by collating the person with the person, the first standard image for the person is changed to a second standard image indicating a prowling person, and the second standard image is changed. A person behavior monitoring system characterized by generating a surveillance image in which a standard image is superimposed on the photographed image.

Images