JP2018093347A - Information processing apparatus, information processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technology for reducing unnecessary activation of alerts even when abnormalities occur frequently and activation of alerts is excessively increased.SOLUTION: A software configuration 301 of a video monitoring system comprises: an analysis part that analyzes a video to analyze the state of an object; a determination part that determines whether the object is an exclusion object; and an activation part that activates an alert when the object is in a predetermined state and is not the exclusion object.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an information processing apparatus, an information processing method, and a program.

  In recent years, video surveillance systems have been required to monitor not only conventional stores such as convenience stores and rental video stores, but also wide areas such as shopping streets, shopping malls, parks and the like outdoors.

  In addition, a technique for learning normal behavior such as a person from video information of a network camera and detecting behavior deviating from the learned normal behavior as an abnormal behavior has entered a practical stage. There is an increasing need for a video monitoring system that supports video monitoring work that has been performed visually by a monitor by using a technique for detecting abnormal behavior.

  When a monitoring video is centrally monitored in a management room such as a security center using a video monitoring system, the monitor monitors the video acquired from a plurality of network cameras by displaying a list on the screen. At this time, the monitoring person who performs monitoring must simultaneously confirm a large number of moving monitoring images, and fatigue is accumulated only by watching. Also, because it is a task of constantly checking the video, it requires concentration and may be missed when monitoring for a long time.

  On the other hand, in Patent Document 1 and Patent Document 2, it is difficult to overlook the occurrence of abnormalities, such as when the video monitoring system detects abnormal behavior and performs highlighting, alerting, or linked display of abnormal video. Assistive technologies are disclosed.

Japanese Patent No. 5155279 Japanese Patent No. 3830150

  However, in the conventional technology, if abnormalities occur frequently, alerts are increased too much, and the confirmation work is rather troublesome. In addition, even if the monitor confirms that there is no problem by looking at the screen, the video surveillance system performs highlighting and alert notification unless the conditions for detecting an abnormality are resolved, so that many times of confirmation are required. Resulting in.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a technique for reducing unnecessary alerts.

An information processing apparatus according to the present invention that achieves the above object has the following configuration. That is,
An analysis means for analyzing the image and analyzing the state of the object;
Determining means for determining whether the object is an exclusion object;
A reporting means for issuing an alert when the target is in a predetermined state and not the exclusion target;
It is characterized by providing.

  According to the present invention, it is possible to reduce unnecessary alerts.

1 is a block diagram illustrating an overall configuration of a video monitoring system according to an embodiment of the present invention. (A) And (b) is a figure which shows an example of a video surveillance system, (c) is a figure which shows the hardware structural example of each apparatus of the video surveillance system in one Embodiment of this invention. It is a block diagram which shows the software structural example of the video surveillance system in 1st embodiment. It is a figure which shows an example of the data table regarding the exclusion setting information in one Embodiment of this invention. It is a figure which shows an example of the exclusion object area | region in one Embodiment of this invention. It is a flowchart which shows the procedure of the process which the video surveillance system in 1st embodiment implements. It is a figure which shows the example of a screen which receives the setting of the exclusion target in one Embodiment of this invention. It is a block diagram which shows the software structural example of the video surveillance system in 2nd embodiment. It is a flowchart which shows the procedure of the process which the video surveillance system in 2nd embodiment implements.

  Hereinafter, embodiments will be described with reference to the drawings. The configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations.

(First embodiment)
<Overview>
First, a configuration example of a video surveillance system according to an embodiment of the present invention will be described with reference to FIG. In FIG. 1, a video monitoring system 101 is constructed for a supermarket, a convenience store, a rental video store, a shopping street, a shopping mall, a park, or the like outdoors. The video monitoring system 101 includes a client PC 102, a video analysis server 103, a recording server 104, and one or more network cameras 105.

  The client PC 102 receives input from the user, gives instructions to the video analysis server 103, the recording server 104, and the network camera 105, and displays the results.

  The video analysis server 103 receives an instruction from the client PC 102 and performs video analysis processing based on the instruction. The video analysis process is an analysis process on the video acquired from the recording server 104 or the network camera 105 to determine a target such as a person shown in the video, to track the movement of the target, It means analyzing behavior.

  The recording server 104 receives an instruction from the client PC 102 or the video analysis server 103 and performs video recording processing based on the instruction. The video recording process is a process of instructing video acquisition to one or more network cameras 105 connected to the same network as the recording server 104 and storing the video. The video recording process also includes a process of storing meta information obtained from the network camera 105. The meta information is, for example, time information, position information, camera setting information, sensor information attached to the network camera, and the like.

  The network camera 105 is an imaging device that is provided on the ceiling, wall, or utility pole of a building and acquires surrounding images. The network camera 105 receives instructions from the client PC 102, the video analysis server 103, and the recording server 104, and performs video acquisition, camera rotation control, audio recording, and the like. Various sensors such as a human sensor, a temperature sensor, a voice sensor, and an infrared sensor may be connected to the network camera 105 in order to acquire the above-described meta information. The devices of the client PC 102 to the network camera 105 are connected to a network and can communicate with each other. Also, each function may be integrated into one PC or server and configured by one.

  Next, FIG. 2 is a diagram illustrating an example of a video monitoring system. FIG. 2A shows an example of a UI for viewing and displaying the video acquired by the plurality of network cameras 105 of the video monitoring system to the supervisor. The screen area of the huge display is divided into 16 areas, and images acquired from each network camera 105 are displayed in the areas with numbers 1 to 16. The video analysis server 103 performs video analysis on the acquired video, and when an abnormality is detected from the analysis result, an emphasized display such as changing the color of the screen as in the areas of 4, 10, and 16 is performed. FIG. 2B is a display of the client PC 102 for confirming the confirmed abnormality in detail. As an example, the fourth video in which an abnormality is detected is shown. In this video, a person is shown falling down, and an example in which it is determined to be abnormal is shown. In this case, the area around the person who has been analyzed as abnormal is highlighted with a frame 20 so that the abnormal part can be easily understood.

<Hardware configuration>
Next, a hardware configuration of each information processing apparatus constituting the video monitoring system according to the present embodiment will be described. FIG. 2C is a block diagram illustrating an example of each device configuration of the client PC 102, the video analysis server 103, and the recording server 104 according to the present embodiment. Note that each device does not necessarily have the same configuration, and may not have a part thereof.

  The information processing apparatus includes a CPU 201, ROM 202, RAM 203, external storage device 204, network I / F 205, input device I / F 206, display I / F 207, keyboard 208, mouse 209, display 210, and system bus 211.

  The CPU 201 performs control processing for the entire apparatus. A ROM (Read Only Memory) 202 is a read-only memory. The ROM includes a PROM (Programmable ROM) in which a user can write a program electrically, and a mask ROM in which contents are written at the time of manufacture. In this embodiment, any ROM can be used. Good.

  A RAM (Random Access Memory) 203 is a memory that can be freely written and read. The RAM 203 has functions such as temporarily storing data when performing the processing of the present embodiment. The external storage device 204 is a nonvolatile storage device. Examples of the external storage device 204 include an SSD, HDD, FDD, MO drive, CD drive, DVD drive, and Blu-ray (registered trademark) drive. The external storage device 204 stores program data for operating the device and stores data necessary for the operation of the program data.

  A network I / F 205 processes communication control for connecting to a network such as an intranet. Various communication interfaces are applied to the network I / F 205 according to the user's environment, and the network I / F 205 includes a wired LAN and / or a wireless LAN interface. The input device I / F 206 processes input from the keyboard 208 and the mouse 209, and is used for operating the apparatus and inputting data.

  The display 210 is a display unit connected to the system bus 211 via the display I / F 207. The display 210 is realized using a CRT, a liquid crystal display, a plasma display or a projector and a control circuit thereof. The system bus 211 exchanges various data between the blocks in the apparatus and supplies power. The system bus 211 includes an address line, a data line, a control line, a power / ground line, and the like.

<Software configuration>
FIG. 3 is a block diagram showing a software configuration 301 of the entire video monitoring system 101 in the first embodiment. Reference numeral 302 denotes an analysis control unit that performs overall control. Reference numeral 303 denotes a video management unit that instructs the network server 105 to acquire and store video in the recording server 104. The video management unit 303 gives control instructions such as video acquisition, camera rotation direction control, and audio recording to the plurality of network cameras 105. In response to an instruction from the abnormality analysis unit 304, the video management unit 303 returns the current video acquired from the network camera 105 or the stored past video.

  In the video analysis server 103, the video analysis server 103 performs video analysis processing for analyzing the video and analyzing the target state, and analyzes whether the target is in a predetermined state (for example, an abnormal state). In order to determine whether or not an abnormality has occurred, a method is used in which a video in a normal state is learned in advance from the video management unit 303, and a case in which it is out of the learned normal state is detected as an abnormality. The method of detecting an abnormality is not limited to this, and may be a method of detecting an abnormality when an operation pattern registered in advance is detected. The abnormality analysis unit 304 acquires a video from the video management unit 303, detects a target such as a person in the video, and obtains the position of the target. If multiple objects are found, find multiple positions. The obtained object is compared with the normal state learned in advance, and an abnormal value is calculated. It is assumed that the higher the abnormal value, the higher the possibility that the abnormal value is abnormal. For example, a feature amount such as a symmetric pixel value distribution in a normal state is held, and the difference from the target feature amount in the video is calculated as a dissimilarity, and the dissimilarity is calculated as an abnormal value. Also good.

  Reference numeral 305 denotes a determination unit that determines whether to issue an alert based on the analysis result of the abnormality analysis unit 304 in the video analysis server 103. The determination unit 305 compares the abnormal value obtained by the abnormality analysis unit 304 with a threshold value set in advance, and determines whether or not to issue an alert.

  Reference numeral 306 denotes an exclusion setting management unit that manages exclusion settings in the video analysis server 103. The exclusion setting management unit 306 receives and holds exclusion setting information sent from the exclusion setting input reception unit 307. Also, exclusion setting information is returned in response to an instruction from the determination unit 305. The exclusion setting information is information for setting an abnormality to be excluded from the alert notification target in the system, and includes information on the detected abnormality and the exclusion target.

  Reference numeral 307 denotes an exclusion setting input reception unit that accepts input by the user and performs exclusion setting in the client PC 102. The exclusion setting input reception unit 307 receives input from the user using the screen UI, and transmits the received exclusion setting information to the exclusion setting management unit 306.

  Reference numeral 308 denotes an alert issuing unit that performs alert notification in the client PC 102. Alert notifications can be anything such as popping up a notification screen or blinking the alert screen, emphasizing the screen, sounding an alarm, sounding through an external system such as e-mail, etc. Good. The alert notification unit 308 operates in response to an instruction from the determination unit 305.

  Although an example of the software configuration has been described above, the function of each processing unit is not limited to the above example. The function of a certain processing unit may be provided in another device, or all the functions may be integrated into one device.

<Data structure related to exclusion setting information>
Next, an example of a data table related to exclusion setting information managed by the exclusion setting management unit 306 will be described with reference to FIG. FIG. 4 is a data table representing exclusion setting information and data related to the exclusion setting information.

  The table 401 includes items of exclusion setting ID, exclusion setting time, camera ID, exclusion target ID, and detection abnormality. The exclusion setting ID is an ID given to manage exclusion setting information. The exclusion setting time represents the time when exclusion is set for the target. The camera ID is an ID for identifying a network camera that has acquired a video in which an object for which exclusion has been set is shown. The exclusion target ID is an ID for uniquely identifying a target for which exclusion has been set. The detection abnormality indicates the type of abnormality in which the target for which the exclusion setting is set is determined. By referring to the information in this table, it is possible to grasp the history of exclusion setting or no exclusion setting.

  A table 402 includes items of camera ID and installation position. The camera ID is the same as the camera ID described in the table 401. The installation position is information indicating the position where the camera is installed, such as “1F before the stairs”.

  The table 403 includes items of an exclusion target ID and an exclusion target area. The exclusion target ID is the same as the camera ID described in the table 401. The exclusion target area is information indicating a position in the video where the exclusion target is shown. “Top: 100 px” and “Left: 800 px” indicate positions of 100 pixels vertically and 800 pixels horizontally from the upper left corner of the video. “Width: 100 px” and “Height: 200 px” indicate rectangular areas to be excluded from the above positions, and the size of the rectangle is 100 pixels vertically and 200 pixels horizontally.

  FIG. 5 shows an example of a rectangular area to be excluded. Reference numeral 501 denotes a video in which an exclusion target ID “T1001” is set. Reference numeral 502 denotes a rectangular area to be excluded.

<Processing>
Hereinafter, the procedure of processing performed by the video monitoring system 101 according to the first embodiment will be described with reference to the flowchart of FIG. In step S <b> 601, the video management unit 303 acquires a video from the network camera 105. In S602, the abnormality analysis unit 304 performs video analysis on the video acquired by the video management unit 303 in S601, detects an abnormality in the video, and obtains an abnormal value and a position where the abnormality has occurred. In step S <b> 603, the determination unit 305 compares the abnormal value obtained in step S <b> 602 with a threshold set in advance in the system. If the abnormal value exceeds the threshold, the process proceeds to S604. On the other hand, if the abnormal value does not exceed the threshold value, the process proceeds to S610.

  In step S <b> 604, the determination unit 305 compares the abnormality content detected in step S <b> 602 with the exclusion setting information held by the exclusion setting management unit 306 and determines whether exclusion setting is performed. If it is set to exclude, the process returns to S601. On the other hand, if no exclusion is set, the process proceeds to S605. The comparison between the detected abnormality content and the exclusion setting information is determined based on whether “camera ID”, “exclusion target region”, and “detection abnormality” in the table data described with reference to FIG. As a method of determining a match, a method may be used in which a match is determined if the value difference is within a certain range even if the values do not exactly match.

  In step S <b> 605, the alert notification unit 308 acquires a result determined by the determination unit 305 that the setting is not excluded, and issues an alert according to the result. Note that the alert notification unit 308 acquires an instruction to issue an alert that is output according to the result determined not to be excluded by the determination unit 305, and issues an alert according to the instruction. You may make it do. In step S606, the exclusion setting input receiving unit 307 determines whether there is user input. If there is user input, the process proceeds to S607. On the other hand, if there is no user input, the process proceeds to S609.

  In step S <b> 607, the exclusion setting input reception unit 307 receives user input and sets an exclusion target. Here, the process of accepting user input and setting an exclusion target will be described with reference to FIG. Reference numeral 701 in FIG. 7A denotes a screen UI that receives settings for exclusion. This screen displays a monitoring video and, when an abnormality is detected, displays a frame so that the location where the abnormality has occurred can be seen as shown at 702. Here, as an example, a person in a region surrounded by a frame 702 is squatting, and the squatting is detected as abnormal. There are various types of abnormalities such as thugs and fights. If a frame 702 is selected by user input when an abnormality is detected, the line changes to a solid line so that it can be seen that the frame is selected as shown in FIG. A button 703 is displayed. After that, when the exclusion setting button 703 is pressed by user input, as shown in FIG. 7C, the frame becomes a thick line so that it can be seen that the exclusion setting has been made, and the color in that area changes and the exclusion setting area changes. The display shows that there is. In addition, as shown in FIG. 7D, the excluded areas may be displayed as a list. Reference numeral 704 denotes an example of a screen UI for managing whether or not the exclusion-set area is displayed as a list and confirmed. In the example of 704, detected abnormalities are displayed as a list, and user input is accepted, so that information indicating whether the user has confirmed can be left as a confirmation flag.

  In step S608, the exclusion setting management unit 306 stores “time”, “camera ID”, “exclusion target region”, and “detection abnormality” when the exclusion setting button 703 is pressed in step S607 as exclusion setting information. In step S609, the analysis control unit 302 determines whether to end the analysis. If not, the process returns to S601.

  In step S610, the determination unit 305 compares the abnormality detected in step S602 with the exclusion setting information held by the exclusion setting management unit 306, and determines whether exclusion setting has been performed. If it is set to be excluded, the process proceeds to S611. On the other hand, if no exclusion is set, the process proceeds to S609. The comparison method here is the same as the processing in S604. In S611, the exclusion setting management unit 306 updates the exclusion setting information by deleting the exclusion setting information that matches in S610 from the data table 401. Thus, the series of processes in FIG. 6 is completed.

  In this embodiment, in S607, the user's input is accepted and the exclusion target is set. However, the present invention is not limited to this. For example, there is no input from the user even if a predetermined time elapses after the alert is issued. You may make it set a thing as an exclusion object. This is equivalent to assuming that the system determines that there is no problem if the supervisor takes no action when alerts occur frequently.

  In S602, if an analysis is performed on each video frame, an abnormality may be detected for a moment, or it may be determined that there is no abnormality for a moment. If exclusion setting or cancellation is performed for such temporary outliers, the usefulness of the system may be impaired. For this reason, instead of calculating an abnormal value from one video frame, the abnormal value may be calculated in consideration of preceding and subsequent frames. This makes it possible to calculate an abnormal value that is robust against temporary outliers.

  As described above, according to the present embodiment, it is possible to suppress alert notification of abnormalities that occur continuously by setting the abnormalities that occur as exclusion targets. Thereby, the trouble of confirmation by the supervisor is reduced, and the possibility of missing another abnormality can be reduced.

(Second embodiment)
In the first embodiment, it has been shown that it is possible to suppress alert notifications of continuously occurring abnormalities by setting the abnormalities that occur as exclusion targets. On the other hand, in 2nd embodiment, even if it is a case where the object which becomes the cause of abnormality which has suppressed alert alerting moves, the example which suppresses alert alerting continuously is shown.

  First, the movement of an object that causes an abnormality will be described. For example, when a person who detects an abnormality “squatting” moves in a squatting state, the abnormality state “squatting” continues but the person who caused the abnormality is moving. “A target that causes an abnormality moves” refers to such a situation. Some types of abnormalities such as “fighting” are accompanied by movement. For example, it includes a case where the subject moves while maintaining a state that causes the abnormality detection, such as when a person who dances on the street performance is erroneously detected as an abnormality such as “fight”. The overall outline and data structure of the video surveillance system in the second embodiment are the same as those in the first embodiment, and thus description thereof is omitted.

<Software configuration>
FIG. 8 is a block diagram showing a software configuration 801 of the entire video monitoring system 101 in the second embodiment. Reference numerals 302 to 308 are the same as those in FIG. Reference numeral 809 denotes a target tracking unit that tracks a target detected by the abnormality analysis unit 304 in the video analysis server 103. The target tracking unit 809 performs a tracking process based on the target position information obtained by the abnormality analysis unit 304.

  An example of this tracking process will be described. In the tracking process, the position information of the target is received from the video at the current time, and the target at the periphery of the position is extracted from the video at the next time. Next, the target at the current time is compared with the extracted target to determine whether or not they are the same target. If it is determined that they are the same target, the position information of the target is additionally held as a result of tracking. This is repeated for each frame of the video to track the target. As a result, it is possible to track which position the target is at which time. The tracking process is not limited to the method described here, and any method may be used as long as it can be determined whether or not the object is the same even when the target moves in the video.

<Processing>
Hereinafter, the procedure of processing performed by the video monitoring system 101 according to the second embodiment will be described with reference to the flowchart of FIG. However, since each process of S901 and S904-S912 is the same as each process of S601, S603-S611, description is abbreviate | omitted.

  In step S902, the abnormality analysis unit 304 performs video analysis on the video acquired by the video management unit 303 in step S901, detects an abnormality in the video, and obtains an abnormal value and a position where the abnormality has occurred. At this time, the target tracking unit 809 acquires information on the obtained abnormality occurrence position, performs tracking processing, and holds the tracking processing result.

  In step S903, the analysis control unit 302 acquires the result of the tracking process from the target tracking unit 809, and updates the exclusion setting information based on the tracking result. At this time, the analysis control unit 302 acquires the current position information of the target from the result of the tracking process, and updates the exclusion target area in the table 403, for example, “Top: 200 px, Left: 800 px”. At this time, the size of the rectangular area indicating the exclusion target area may be updated.

  As described above, according to the present embodiment, it is possible to continuously suppress unnecessary alert notification even when an object causing an abnormality that suppresses alert notification moves. Therefore, even when the object causing the abnormality moves, unnecessary alerts can be appropriately suppressed, and the labor of the supervisor can be reduced.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  101: Video monitoring system, 102: Client PC, 103: Video analysis server, 104: Recording server, 105: Network camera, 302: Analysis control unit, 303: Video management unit, 304: Abnormality analysis unit, 305: Determination unit, 306: Exclusion setting management unit, 307: Exclusion setting input receiving unit, 308: Alert issuing unit, 809: Target tracking unit

Claims (9)

An analysis means for analyzing the image and analyzing the state of the object;
Determining means for determining whether the object is an exclusion object;
A reporting means for issuing an alert when the target is in a predetermined state and not the exclusion target;
An information processing apparatus comprising: The analysis means calculates an abnormal value of the target in the video, and if the abnormal value exceeds a threshold, analyzes that the target is in an abnormal state,
The information processing apparatus according to claim 1, wherein the notification unit issues the alert when the target is in the abnormal state and not the exclusion target. A receiving means for receiving user input;
The information processing apparatus according to claim 1, wherein the determination unit determines whether the target is the exclusion target based on the exclusion setting information received by the reception unit.   4. The apparatus according to claim 3, further comprising an updating unit that updates the exclusion setting information by deleting the target from the exclusion target when the target is not in a predetermined state and is the exclusion target. The information processing apparatus described. Tracking means for tracking the target in the predetermined state in the video;
Updating means for updating the exclusion setting information according to the result of tracking by the tracking means;
The information processing apparatus according to claim 3, further comprising:   The information processing apparatus according to claim 3, wherein the exclusion setting information includes information on an exclusion target area indicating a position of the exclusion target in the video.   A setting unit configured to set the target as the exclusion target when the receiving unit has not received the exclusion setting information even after a predetermined time has elapsed since the alert was issued by the reporting unit; The information processing apparatus according to any one of claims 3 to 6. An information processing method,
An analysis process for analyzing the image and analyzing the state of the object;
A determination step of determining whether the target is an exclusion target;
An alerting step for issuing an alert when the subject is in a predetermined state and not the exclusion subject;
An information processing method characterized by comprising:   The program for functioning a computer as each means of the information processing apparatus of any one of Claims 1 thru | or 7.

Images