JP7214449B2 - Information processing device, information processing method and program - Google Patents

Description

The present invention relates to detection of anomalies by surveillance cameras.

2. Description of the Related Art There is a technology for detecting suspicious persons and suspicious behavior using images obtained from surveillance cameras in event venues, commercial facilities, office buildings, and the like. For example, in Japanese Patent Laid-Open No. 2002-100000, the number of times the face direction changes within a certain period of time is used as a criterion for discriminating a suspicious person. In Japanese Patent Laid-Open No. 2002-200010, a behavior pattern defined as being suspicious is prepared in advance, and a person who behaves in accordance with the pattern is determined as a suspicious person. In Japanese Patent Laid-Open No. 2002-200010, whether or not a suspicious person is determined is determined based on the frequency distribution of the behavior feature values, which are the moving speed of the target, the amount of change in the moving direction, the distance to a predetermined object, and the height of the head. There are also techniques for detecting suspicious persons and suspicious actions using information obtained from sources other than video. For example, in Patent Literature 4, a waterborne radar and multiple cameras are used to detect another vessel that is hiding behind an adjacent vessel and is navigating. In Patent Document 5, the weight of a person entering and leaving a room to be monitored is measured, and if the weight differs between when entering and leaving the room, it is determined that the person is acting suspiciously in the room.

Japanese Unexamined Patent Application Publication No. 2011-4050 JP 2013-45344 A JP-A-2013-88870 JP-A-2005-277705 JP-A-2009-239467

However, these techniques have the following problems when monitoring a large area such as an event venue with a plurality of cameras. First of all, the techniques disclosed in Patent Documents 1 to 3 are intended to determine whether or not the object to be detected is acting suspiciously when it is captured in the image. It is not possible to detect that In addition, in Patent Document 4, multiple cameras are used to cover blind spots with a single camera, so suspicious actions performed in places where any camera is used will be blind spots. It is not possible. Finally, in Patent Document 5, suspicious behavior can be detected in locations that cannot be captured by surveillance cameras. It is difficult to detect suspicious behavior in locations that are difficult to narrow down.

The present invention has been devised in view of such problems. It detects the occurrence of behavior.

An information processing apparatus according to the present invention has the following configuration. That is, imaging means for photographing a predetermined area, detection means for detecting a person whose human body is at least partly hidden behind a shield in the area photographed by the imaging means, and a person who is hidden by the shield. before and after, determination means for determining a suspicious person candidate based on changes in characteristics of the detected person or the person's surroundings; Output means for outputting

According to the configuration of the present invention, it is possible to output to the user the place of occurrence of suspicious behavior occurring in a blind spot of a surveillance camera.

FIG. 2 is a block diagram showing an example of functional configuration; 4 is a flowchart for explaining processing in the first embodiment; An example of a place where a blind spot to be processed by the suspicious person determination unit 1004 in the first embodiment occurs. An example of a suspicious person determination process performed by the suspicious person determination unit 1004 in the first embodiment. 9 is a flowchart for explaining processing in the second embodiment; 10 is a flowchart for explaining processing in the third embodiment;

Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. It should be noted that the embodiment described below is an example of a specific implementation of the present invention, and is one of specific examples of the configuration described in the claims.

[First embodiment]
An example of an information processing system configured as follows, which is an information processing system for detecting suspicious behavior in video data acquired by an imaging device, will be described. That is, when a person in a blind spot is detected in video data acquired from an imaging device near the blind spot, and the characteristics of the person change before and after entering the blind spot, suspicious behavior is detected in the vicinity of the blind spot. Detect as a place where is assumed to have occurred.

In this embodiment, a case will be described in which such a system is applied to an information processing apparatus such as a server, which has sufficient specifications to process video data. As shown in FIG. 1, the system according to the present embodiment includes an imaging device 1100 that captures an image of a monitored area, a server device 1000 that holds and processes data acquired from the imaging device 1100, and a network that connects these two devices. consists of 1200. It is assumed that a plurality of imaging devices 1100 are connected to the server device 1000 .

First, the configuration of the server device 1000 according to this embodiment will be described using the block diagram of FIG. An instruction input unit 1001 is a user interface such as a group of buttons, and allows a user to operate the server device 1000 .

A moving image storage unit 1002 is for storing moving images acquired by the imaging device 1100 . Each process in the present embodiment is performed on this saved moving image.

A person detection unit 1003 detects a person appearing in a moving image.

The suspicious person determination unit 1004 is for estimating whether the person detected by the person detection unit 1003 is acting suspiciously. However, depending on the embodiment, a target person who is acting suspiciously may be narrowed down based on the processing results of other components.

The location estimation unit 1005 estimates the location where the person determined by the suspicious person determination unit 1004 to be acting suspiciously actually acted suspiciously. This result is the result that is finally presented to the user in this embodiment.

The suspicious person candidate detection unit 1006 considers that a person who meets a specific condition is highly likely to act or has acted suspiciously, and picks the person up as a suspicious person candidate.

The person search unit 1007 searches for a specific person from the moving images saved in the moving image storage unit 1002 . In this embodiment, it is assumed that the person detected by the suspicious person candidate detection unit 1006 is to be searched.

The person tracking unit 1008 tracks a person specified manually by the user or automatically according to the processing results of other components in the moving image acquired from the imaging device 1100 .

The external factor holding unit 1009 acquires and holds factors other than the features of the person, which are used when the suspicious person determination unit 1004 makes an estimation. It is assumed that the external factor storage unit 1009 receives information for estimating whether or not the behavior is suspicious based on clues other than moving images.

The information display unit 1010 displays information such as the location of the suspicious behavior estimated by the location estimation unit 1005 and the moving image acquired by the imaging device 1100. The information display unit 1010 includes a liquid crystal screen, a speaker, and the like. there is

The suspicious person candidate detection unit 1006, person search unit 1007, and person tracking unit 1008 are used in other embodiments, and are not used in this embodiment. Therefore, it is not an essential component when implementing only this embodiment.

Next, the configuration of the imaging device 1100 according to this embodiment will be described using the block diagram of FIG. The storage unit 1101 stores various information, various data, various tables, and the like.

An imaging unit 1102 is an imaging sensor that converts an optical image of a subject through an optical system such as a lens into an electronic image.

An input unit 1103 receives inputs such as various information and various instructions from the photographer. This input may be performed by directly connecting an input device such as a keyboard to the imaging apparatus 1100 or through the server apparatus 1000 .

The output unit 1104 outputs data converted by the imaging unit 1102, information stored in the storage unit 1101, and the like to the outside. As an output destination, in addition to the network 1200, a USB, a display cable, or the like may be used.

Next, operations of the server device 1000 and the imaging device 1100 will be described using the flowchart of FIG. In this embodiment, each functional unit shown in FIG. 1 is assumed to be implemented by a computer program and stored in a server device. However, in the following description, these functional units may be described as main bodies of processing. Realize the function. Note that all the functional units shown in FIG. 1 are not limited to being implemented by computer programs, and part or all of them may be implemented by hardware.

First, in step S201, a moving image is acquired by the imaging unit 1102 of the imaging device 1100. FIG. The shooting at this time is performed according to the shooting parameters stored in the storage unit 1101 . This moving image is sent to the server device 1000 via the network 1200 and recorded in the moving image storage unit 1002 . At this time, the unit of the moving image to be recorded is, for example, every fixed time period or every time frame set in advance according to the schedule of the event. Alternatively, it may be processed sequentially in a streaming format. Of course, the recording unit may be specified manually at any time. Further, as described above, it is assumed that a plurality of imaging devices 1100 are connected to the server device 1000, so the recording units of the imaging devices 1100 may be the same or different.

Next, in step S<b>202 , the person detection unit 1003 performs person detection for each frame of the moving image recorded in the moving image storage unit 1002 . Objects to be detected here do not have to be moving images captured by all imaging devices. For example, it is conceivable to target a place where hiding occurs constantly, specifically, the vicinity of a place as shown in FIG. A brief description of each example is provided. First, a scene 301 is a place such as a restroom where photography cannot be performed due to privacy concerns. It is difficult to install an imaging device in such a shielded place, and it is relatively easy to hide the whole body of the human body and perform suspicious actions. , should be subject to human detection. Next, a scene 302 is a place that cannot be physically photographed, such as a stairway or behind a wall. In such a shielded place, the whole human body may be hidden like the scene 301, but only a part of the human body may be hidden by the shield. However, since it is possible to perform suspicious actions in the hidden part anyway, the video before entering and after leaving this place should be targeted for person detection. Finally, scene 303 is a place that includes dynamically occurring hiding, such as blending in with a crowd of people. This scene is essentially the same as the scene 302 except that the hidden factor is dynamic or static. Of course, such dynamic hiding does not always occur. you can stay Alternatively, an image of only the background may be prepared in advance, and added as a processing target when the ratio of the background hidden in the moving image increases. By such determination, it is possible to limit the number of imaging devices to be processed and improve the processing efficiency. Further, candidates to be judged as suspicious persons are further narrowed down from among the detected persons. Specifically, a person whose body is partly or wholly hidden in a moving image is detected. This is because it is not possible to directly confirm whether or not a person has acted suspiciously while hiding, so it is necessary to make a judgment target according to the present embodiment.

Next, in step S203, the suspicious person determination unit 1004 compares the person detected in step S202 before and after the occurrence of hiding to see if any suspicious characteristics have changed, and performs the determination in step S204. . This is because if a change in characteristics has occurred, there is a high possibility that the animal is behaving suspiciously at the location where the hiding occurred. Specific criteria for judging suspicious behavior include, for example, the following. First, there is a change in appearance, such as a decrease or increase in baggage, a change in clothing, and the like. This is a standard for suspicious behavior because it may occur due to actions such as setting explosives that were hidden or receiving firearm parts brought in in pieces. For example, for a person whose face is judged to be the same as a reference, local feature values around the person are compared. and so on. It should be noted that the suspicious person determination unit determines whether or not the person is the same person. Next, a specific criterion for judging suspicious behavior is a change in movement speed, such as a decrease in movement speed or an increase in movement speed before and after hiding. This is a standard for suspicious behavior because it can be considered that the number of hidden luggage has decreased and the animal has become able to move faster, or that the movement speed has decreased due to an increase in luggage or the beginning of being cautious of the surroundings. This can be done, for example, by using human tracking technology to measure the speed of the person. If it is necessary to measure more accurately, it is also possible to calibrate between the real world and the camera before starting shooting, calculate the correspondence between the image coordinate system and the world coordinate system, and use that relationship. Finally, changes in temperature, such as body temperature and temperature changes in belongings. In this case, a special imaging device such as a thermo camera is required, but by looking at this feature, it is possible to detect changes that may occur when suspicious behavior such as carrying a heat generating object or changing body temperature due to tension is performed. become able to.

Also, a pattern for determining suspicious behavior will be described. As shown in FIG. 4, two patterns are assumed. The first is a case where the person is clearly not normally doing such as losing a large package as shown in the pattern 401 . In this case, it is assumed that the explosive or the like is being carried as it is. This is an action that is unlikely to occur normally, such as the loss of a package that is large enough to be noticed immediately if it is lost. Therefore, if it occurs once, it is determined as a suspicious action. The second is a pattern 402 in which actions such as a gradual increase in the number of packages are frequently performed but do not occur frequently. In this case, it is assumed that the parts of the firearm brought in after being disassembled and divided are received one by one. Since this is unlikely to be repeated multiple times, the degree of suspicion is gradually increased each time it occurs, and the person is determined to be a suspicious person when a preset threshold value is exceeded.

Furthermore, here, the external environment may be taken into consideration when making the determination. In this case, information acquired and held by the external factor holding unit 1009 is used. Examples of considerations include: For example, temperature information is stored, and if it is considered that the person has taken off his or her jacket due to a sudden rise in temperature, it is determined that the behavior is natural even if the appearance features have changed. Alternatively, if there is validity such as passing by the cloak even if the appearance characteristics have changed, it is judged to be a natural behavior. In addition, when related goods are purchased at the event site, it is determined that this is a natural behavior even if the number of luggage increases. In this case, the purchase history of the venue may be used at the same time. These external information are obtained, for example, from a server used by a department in charge of purchasing at the venue or from a management system for room temperature, etc., via a network. Furthermore, when food such as boxed lunches has been eaten and thrown away, it can be determined that it is natural even if the amount of luggage is reduced. In this case as well, the purchase history, action history, etc. of the venue may be used. Conversely, if there are no trash cans near the place where the hiding occurred, but the amount of luggage is reduced, even if the reduced amount is thought to be food, leave the suspicious item somewhere. judge that it did. Alternatively, if there is no shop near the place where the hiding occurred, and the number of luggage increases, it is determined that the suspicious item has been handed over even if the increased amount is thought to be food.

If it is determined in step S204 that the person is acting suspiciously, the process advances to step S205 to estimate and output the place where the suspicious action occurred. This is estimated by the location estimating unit 1005 based on the location where the hiding occurred and the length of time of the occurrence. Specifically, centering around the place where hiding occurred, a range where it is considered possible to move during the time when hiding occurred is presented as a place where suspicious behavior occurred. At this time, if information such as a map and structure of the monitoring location is held, it is possible to make a more accurate judgment than simply judging whether or not the distance can be moved considering the information. By displaying this location information on the information display unit 1010, it is possible to provide the user such as a security guard with information on the possibility of occurrence of suspicious behavior and its location. On the other hand, if it is determined in step S204 that the character is not acting suspiciously, no particular action is taken and this determination operation ends.

As described above, according to the present embodiment, it is possible to estimate a suspicious behavior that may have occurred in a hidden place and its place from a video image of a hidden place. As a result, it is possible to present to the user even the blind spot of the imaging device as a candidate for the location where the suspicious behavior occurs.

[Second embodiment]
The present embodiment aims at determining whether or not a person who meets a specific condition has taken a suspicious action as a suspicious person candidate. In this embodiment, processing is controlled using the determination result of the suspicious person candidate detection unit 1006 . In the following, differences from the first embodiment will be mainly described, and unless otherwise specified, the same as the first embodiment.

Consider a case where suspicious behavior could not be detected for some reason in the determination of the first embodiment. Even in this case, after some time has passed since the suspicious action, the user may indirectly act suspiciously. Concretely, there are cases in which the person leaves the exit even in the middle of the event, or has a tense expression such as a stiff face for a long time even though the person is neither a performer nor an athlete. In the present embodiment, the behavior that is indirectly thought to be suspicious is used as a key to go back and re-determine the suspicious behavior, or to execute the determination and detection of the suspicious behavior. In other words, the former can prevent overlooking, and the latter can limit the processing time.

There is no change in the system configuration between the first embodiment and this embodiment. Next, operations of the server device 1000 and the imaging device 1100 in this embodiment will be described using the flowchart of FIG. As in the first embodiment, each functional unit shown in FIG. 1 is assumed to be implemented by a computer program and stored in a server device. However, in the following description, these functional units may be described as main bodies of processing. Realize the function. Note that all the functional units shown in FIG. 1 are not limited to being implemented by computer programs, and part or all of them may be implemented by hardware. Also, the steps assigned the same numbers as in FIG. 2 have the same functions as those in the first embodiment, and therefore description thereof will be omitted.

First, in step S501, as in the first embodiment, after a moving image is acquired by the imaging unit 1102 of the imaging device 1100, in step S502, the suspicious person candidate detection unit 1006 satisfies the conditions for indirectly determining that the suspicious person is suspicious. Detect a person who performed a matching action. Examples of specific conditions include leaving in the middle of the event mentioned above and being nervous for a long time. For the former, for example, an event timetable may be registered in advance, and the determination can be made by comparing the timetable with the exit time. The latter can be determined by applying an existing facial expression recognition technology to a person in an image, and determining whether the person is tense for a certain period of time or more.

Next, in step S503, the action history of the person detected in step S502 is searched. Here, the person search unit 1007 searches all moving images stored in the moving image storage unit 1002 to see if the detected person appears. This search is performed using the existing person search technology using the face image of the person detected in step S502 as a key.

Next, in step S504, the suspicious person candidate detection unit 1006 determines whether or not there is a hidden person among the search results in step S503. If there is no hiding, it is concluded that there was no suspicious behavior, and the process ends. If there is hiding, it is determined that the previous determination by the suspicious person determination unit 1004 was strict and there was an omission, and the process moves to step S505.

In step S505, a process of relaxing the judgment criteria for suspicious behavior more than before is performed. Using this newly set determination criterion, the suspicious person determination unit 1004 performs the same suspicious behavior determination as in the first embodiment, specifically, the processes after step S203 in FIG.

As described above, according to the present embodiment, even if suspicious behavior cannot be detected once from a moving image of a place including hiding, reprocessing can be performed using the behavior that is indirectly thought to be suspicious as a key. can. As a result, it is possible to present to the user a candidate for the occurrence of suspicious behavior in the blind spot of the imaging device, with less oversight.

[Third embodiment]
The purpose of this embodiment is to track a person who satisfies a specific condition as a suspicious person candidate, and to determine whether or not the person has acted suspiciously. In this embodiment, processing is controlled using the determination result of the suspicious person candidate detection unit 1006 . In the following, differences from the first and second embodiments will be mainly described, and unless otherwise specified, the same as the first and second embodiments.

In the second embodiment, a behavior indirectly thought to be suspicious that occurred after a suspicious behavior was targeted. Contrary to the second embodiment, however, it is conceivable that the actual suspicious behavior may be performed after indirectly performing the suspicious behavior. Therefore, in the present embodiment, a person who has indirectly acted suspiciously is tracked, and suspicious action determination is performed intensively.

There is no change in the system configuration between the first and second embodiments and this embodiment. Next, operations of the server device 1000 and the imaging device 1100 in this embodiment will be described using the flowchart of FIG. As in the first and second embodiments, each functional unit shown in FIG. 1 is assumed to be implemented by a computer program and stored in a server device. However, in the following description, these functional units may be described as main bodies of processing. Realize the function. Note that all the functional units shown in FIG. 1 are not limited to being implemented by computer programs, and part or all of them may be implemented by hardware. Also, the steps assigned the same numbers as in FIG. 2 have the same functions as those in the first embodiment, and therefore description thereof will be omitted.

In steps S601 and S602, a person to be tracked is detected after a moving image is acquired by the imaging unit 1102 of the imaging device 1100, as in the second embodiment. In this case, the suspicious person candidate detection unit 1006 indirectly detects a person who has acted in accordance with the conditions for being judged as suspicious. However, this embodiment does not include conditions under which it is assumed that the target person will leave the monitoring area, such as leaving during the event. In addition, even if the conditions set in advance are not met, the user may determine that the object should be tracked. For example, a pointing device such as a mouse can be used to click and designate the target person in the image.

Next, in step S603, the person detected in step S602 is tracked using the person tracking unit 1008. FIG. This search is performed using the existing person tracking technology using the face image of the person detected in step S602 as a key.

During tracking, the suspicious person determination unit 1004 always determines whether or not the tracking target is hidden in step S604. If no hiding has occurred, the process moves to step S605 to determine whether or not the tracked person is within the monitored area. As long as it is within the monitored area, the process moves to step S604 and is repeated. If it goes out of the monitoring area, the process is terminated. When hiding occurs, the suspicious person determination unit 1004 performs the same suspicious behavior determination as in the first embodiment, specifically, the processes after step S203 in FIG.

As described above, according to the present embodiment, it is possible to detect suspicious person candidates in advance by using, as a key, actions that are indirectly thought to be suspicious from moving images of locations including hiding, and to perform suspicious behavior determination. As a result, it is possible to present to the user a candidate for the occurrence of suspicious behavior in the blind spot of the imaging device, with less oversight.

[Other embodiments]
Other embodiments include the following.

Up to this point, each embodiment has been described independently, but it is also possible to combine them. For example, according to the third embodiment, it is possible to determine a person to be monitored intensively and perform processing, and to reduce omissions of detection according to the second embodiment.

The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

1000 server device 1001 instruction input unit 1002 moving image storage unit 1003 person detection unit 1004 suspicious person determination unit 1005 location estimation unit 1006 suspicious person candidate detection unit 1007 person search unit 1008 person tracking unit 1009 external factor holding unit 1010 information display unit 1100 imaging Apparatus 1101 Storage unit 1102 Imaging unit 1103 Input unit 1104 Output unit

Claims (9)

detection means for detecting a person whose human body is at least partly hidden behind a shield in an image of a predetermined area;
determination means for determining a suspicious person candidate based on a change in the characteristics of the detected person or the person's surroundings before and after the occurrence of the hiding by the shield;
An information processing apparatus, comprising: output means for outputting a place where part or all of the human body of the suspicious person candidate is hidden by a shield. further comprising a person search means for searching for a moving image of a person who satisfies a predetermined condition within the predetermined area,
2. The information processing apparatus according to claim 1, wherein said determination means performs determination on a person searched by said person search means. further comprising person tracking means for tracking a specific person in the captured moving image;
3. The information processing apparatus according to claim 2, wherein said determination means performs determination on a person being tracked by said person tracking means. 4. The information processing apparatus according to claim 3, wherein said person tracking means tracks a person who meets said predetermined condition or a person designated by a user. 5. Any one of claims 1 to 4, wherein said output means changes the range of the output location according to the length of time during which part or all of said suspicious person candidate is hidden by a shield. The information processing device according to the item. 6. The apparatus according to any one of claims 1 to 5, further comprising an external factor holding means, wherein the judgment means judges the suspicious person candidate based on the change in the characteristics and the information held by the external factor holding means. The information processing apparatus according to any one of items 1 and 2. a detection step of detecting a person whose human body is at least partly hidden behind a shield in an image of a predetermined area;
a determination step of determining a suspicious person candidate based on changes in the characteristics of the detected person or the person's surroundings before and after the occurrence of the hiding by the shield;
and an output step of outputting a location where part or all of the human body of the candidate for suspicious person is hidden by a shield. detection means for detecting a person whose human body is at least partly hidden behind a shield in an image obtained by photographing a predetermined area;
determination means for determining a suspicious person candidate based on a change in characteristics of the detected person or the person's surroundings before and after the occurrence of the hiding by the shield;
A program for functioning as output means for outputting a place where part or all of the human body of the candidate for suspicious person is hidden by a shield. imaging means for imaging a predetermined area;
a detecting means for detecting a person whose human body is at least partly hidden behind a shield in the image captured by the imaging means;
determination means for determining a suspicious person candidate based on a change in the characteristics of the detected person or the person's surroundings before and after the occurrence of the hiding by the shield;
An information processing system, comprising: output means for outputting a place where part or all of the human body of the suspicious person candidate is hidden by a shield.

Images