JP2021135268A - Monitoring method and monitoring device - Google Patents

Abstract

To strengthen measures for persons having a dangerous thing in a place where an unspecified number of people pass through.SOLUTION: A monitoring method of an embodiment includes: a first acquisition step for acquiring reflected wave detection results from a radar device that irradiates a prescribed monitoring area with millimeter waves and receives reflected waves; a second acquisition step for acquiring photographed images from a photographing unit that photographs the prescribed monitoring area; a determination step for determining the possibility of existence of a dangerous thing on the basis of the reflected wave detection result; a specification step for specifying, when it is determined in the determination step that there is possibility of existence of a dangerous thing, a person image that is an image portion of a person estimated to have the dangerous thing on the basis of the photographed image; and a tracking step for tracking the person image that indicates a first person represented by an imaged image having been acquired after the first photographed image, on the basis of the feature quantity of the first person represented by the person image of the first photographed image having been acquired from a photographing unit.SELECTED DRAWING: Figure 1

Description

Embodiments of the present invention relate to monitoring methods and monitoring devices.

Conventionally, for example, at an airport security checkpoint, dangerous objects (firearms, cutlery, bombs, etc.) can be detected with high accuracy by inspection using a metal detector, a body scanner using millimeter waves, or the like.

On the other hand, in places where an unspecified number of people pass, such as general areas where you can enter without an airport ticket or a railway station, such inspections cannot be performed, so for crime prevention, for example, surveillance with a surveillance camera is required. It is done.

Japanese Patent No. 6276736 Japanese Unexamined Patent Publication No. 2004-361365 JP-A-2018-156586 JP-A-2018-146257 JP-A-2018-146250 JP-A-2018-146249 Japanese Patent No. 5560397

However, although a certain degree of crime prevention effect can be obtained by surveillance with a surveillance camera, it is not easy to find a person who has a dangerous substance inside clothes or a bag, and improvement is desired. ing.

Therefore, the present invention has been made in view of the above circumstances, and is a monitoring method capable of strengthening measures against a person possessing a dangerous substance in a place where an unspecified number of people pass. The subject is to provide a monitoring device.

The monitoring method of the embodiment includes a first acquisition step of irradiating a predetermined monitoring area with a millimeter wave (W band) and acquiring a detection result of the reflected wave from a radar device that receives the reflected wave, and the predetermined monitoring area. A second acquisition step of acquiring a captured image from the photographing unit for photographing the image, a determination step of determining the existence possibility of the dangerous substance based on the detection result of the reflected wave, and the existence possibility of the dangerous substance by the determination step. When it is determined that there is, a specific step of identifying a person image which is an image part of a person who is presumed to possess the dangerous object based on the photographed image, and a first acquired from the photographing unit. A person showing the first person represented by the captured image acquired after the first captured image based on the feature amount of the first person represented by the person image in the captured image 1. Includes tracking steps to track images. The tracking step may include, for example, an auxiliary tracking step using the position, moving direction, and moving speed of the person obtained from the radar device.

FIG. 1 is a diagram showing an outline of the overall configuration of the monitoring system of the embodiment. FIG. 2 is a diagram showing an outline of the configuration of the information processing apparatus of the embodiment. FIG. 3 is a diagram showing an example of monitoring-related information stored in the information processing apparatus of the embodiment. FIG. 4 is a flowchart showing processing by the information processing apparatus of the embodiment.

Hereinafter, the monitoring method and the monitoring device of the present embodiment will be described with reference to the accompanying drawings. In the present embodiment, an object containing a large amount of metal components such as a firearm, a knife, and a bomb is assumed as a dangerous substance.

First, with reference to FIG. 1, an outline of the overall configuration of the monitoring system S of the embodiment will be described. FIG. 1 is a diagram showing an outline of the overall configuration of the monitoring system S of the embodiment. The monitoring system S is applied to a place where an unspecified number of people pass, such as a general area where an airport ticket can be entered without an airport ticket or a railway station. The entire monitoring area monitored by the monitoring system S is composed of four monitoring areas Ra, Rb, Rc, and Rd. The monitoring area Ra includes entrances in all monitoring areas. Further, when the monitoring areas Ra, Rb, Rc, and Rd are not particularly distinguished, they are referred to as monitoring areas R.

The surveillance system S includes an information processing device 1, radar devices 2a and 2c, and surveillance cameras 3a, 3b, 3c and 3d (photographing unit). Hereinafter, when the radar devices 2a and 2c are not particularly distinguished, they are referred to as radar devices 2. Further, when the surveillance cameras 3a, 3b, 3c and 3d are not particularly distinguished, they are referred to as surveillance cameras 3.

The radar device 2 irradiates the target monitoring region R with a millimeter wave (W band) to receive the reflected wave, and detects the intensity of the reflected wave (reflection intensity) for each subdivided region. The millimeter wave to be irradiated may be, for example, 78 GHz band, 94 GHz band, or the like, but is not limited thereto.

The irradiated millimeter wave is reflected by the ground, structures, people, etc. in the monitoring area R, but the reflectance greatly differs depending on the type (composition) of the object. The reflectance of dangerous goods is particularly high, and millimeter waves pass through ordinary clothes and bags to some extent. Therefore, when a dangerous substance is present inside the clothes or inside the bag, the radar device 2 detects a large reflection intensity for that portion even if it cannot be seen from the outside.

The surveillance camera 3 is, for example, a digital camera or the like, and photographs the target surveillance area R. When the radar device 2 is installed in the same monitoring area R, the surveillance camera 3 captures the same range (including substantially the same range) as the detection range of the radar device 2. For example, for the surveillance area Ra, the detection range by the radar device 2a and the shooting range by the surveillance camera 3a are set to be the same.

On the other hand, the surveillance area Rb is provided with only the surveillance camera 3b. Further, the surveillance area Rc is provided with a radar device 2c and a surveillance camera 3c. Further, only the surveillance camera 3b is provided in the surveillance area Rd.

It is essential that one or more surveillance cameras 3 are installed in all the surveillance areas R and that the radar device 2a is installed in the surveillance area Ra including the entrance, but the others are optional. For example, a plurality of surveillance cameras 3 may be provided for one surveillance area R, or a plurality of radar devices 2 may be provided. For example, since the monitoring area Ra including the entrance is particularly important, a plurality of radar devices 2 may be installed. Further, the radar device 2 may be provided in the monitoring area Rb, the radar device 2 in the monitoring area Rc may be omitted, or the radar device 2 may be provided in the monitoring area Rd.

Next, the configuration of the information processing device 1 of the embodiment will be described with reference to FIG. FIG. 2 is a diagram showing an outline of the configuration of the information processing device 1 of the embodiment. The information processing device 1 is a computer device such as a server device, and includes a processing unit 11, a storage unit 12, an input unit 13, a display unit 14, and a communication unit 15. Here, for the sake of brevity, the information processing apparatus 1 will be described as a single device, but the present invention is not limited to this. For example, there may be a configuration in which there are a plurality of lower-level servers corresponding to a predetermined number of monitoring areas R, and there is a higher-level server that controls the plurality of lower-level servers. In the following, it will be described mainly assuming that the target is the monitoring area Ra.

The storage unit 12 is composed of, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an HDD (Hard Disk Drive), an SSD (Solid State Drive), and the like, and stores various information. The storage unit 12 stores, for example, the detection result of the reflected wave by the radar device 2, the captured image by the surveillance camera 3, various set values, predetermined monitoring-related information, and the like.

The processing unit 11 is composed of, for example, a CPU (Central Processing Unit), and includes an acquisition unit 111, a determination unit 112, and a specific unit 113 as functional units realized by executing a program stored in the storage unit 12. , A tracking unit 114, a calculation unit 115, a notification unit 116, and a control unit 117.

The acquisition unit 111 functions as, for example, a first acquisition unit that acquires the detection result of the reflected wave from the radar device 2. Further, the acquisition unit 111 functions as, for example, a second acquisition unit that acquires a captured image from the surveillance camera 3 that captures each of the plurality of surveillance areas R.

The determination unit 112 determines the possibility of existence of a dangerous substance based on the detection result of the reflected wave. For example, the determination unit 112 determines that there is a possibility of existence of a dangerous substance when there is a portion of the target monitoring area R whose reflection intensity is equal to or higher than a predetermined reflection intensity threshold value. In that case, for example, a plurality of reflection intensity threshold values may be set, and the possibility of existence of a dangerous substance may be determined in 101 steps of 0 to 100%.

When the determination unit 112 determines that the possibility of existence of a dangerous substance is equal to or higher than a predetermined dangerous substance threshold value (for example, 30%), the identification unit 113 estimates that the dangerous substance is possessed based on the captured image. The person image which is the image part of the person to be performed is specified. Specifically, it is as follows. As described above, for the surveillance area Ra, the detection range by the radar device 2a and the shooting range by the surveillance camera 3a are set to be the same. In addition, the specific unit 113 extracts a person from the captured image based on a known algorithm.

Then, the identification unit 113 collates the detection result data by the radar device 2a with the photographed image data by using the respective two-dimensional coordinates and the like, so that the person who is presumed to have the dangerous object in the photographed image is in possession of the dangerous object. Identify the person image that is the image part.

The tracking unit 114 tracks the person image identified by the specific unit 113. For example, the tracking unit 114 has a feature amount of the first person represented by a person image in the first captured image acquired from the surveillance camera 3a (for example, the color and size of clothes, ancillary items (bags, etc.)). ), The person image showing the first person represented by the captured image acquired after the first captured image is tracked. Further, the tracking unit 114 can also track a person by using the position, the moving direction, and the moving speed of the person obtained from the radar device 2.

Specifically, the tracking unit 114 first tracks a person image in the monitoring area Ra. Then, when the person image exits the monitoring area Ra, the tracking unit 114 uses the position, moving direction, and moving speed of the person obtained from the radar device 2a, for example, to move to the monitoring area R (for example, the monitoring area Rb). ) Is specified.

Further, for example, as shown in FIG. 1, when the person image (reference numeral M) in the monitoring area Rb moves to the right of the paper surface, there are two possibilities of the movement destination, the monitoring area Rc and the monitoring area Rd. The tracking unit 114 identifies the monitoring area R (for example, the monitoring area Rc) of the movement destination by calculating and using the position, the moving direction, and the moving speed of the person image based on the captured image.

The calculation unit 115 (suspiciousness calculation unit) calculates the suspiciousness of the person based on the position, the moving direction, the moving speed, and the detection result of the reflected wave of the person image. For example, a person who goes back and forth in the same place or a person with a higher reflection intensity calculates a higher degree of suspicion.

The notification unit 116 notifies a predetermined information terminal (not shown) of information including the current position, the possibility of existence of a dangerous substance, and the degree of suspicion about the person. Here, the predetermined information terminal is, for example, a smartphone or the like carried by a security guard or the like.

The control unit 117 performs processing other than the processing performed by each unit 111-116.

The input unit 13 is a keyboard, a mouse, a touch panel, a switch, a microphone, and the like, and is a means for allowing the user to input various operation instructions.

The display unit 14 is a means for displaying various information, for example, an LCD (Liquid Crystal Display).

The communication unit 15 is an interface for transmitting and receiving various information to and from an external device connected by wire or wirelessly.

FIG. 3 is a diagram showing an example of monitoring-related information stored in the information processing device 1 of the embodiment. As an example, the monitoring-related information stores each information of a person ID (Identifier), a feature amount, a suspicious degree, a moving area, and a current position. The person ID is an identification number of a person estimated by the specific unit 113 to possess a dangerous object in the photographed image. The feature amount is the feature amount data calculated by the tracking unit 114 for the person.

The suspicious degree is the suspicious degree of the person calculated by the calculation unit 115. The movement area is the data of the monitoring area R to which the person has moved, which is obtained by the tracking unit 114. The current position is the current position of the person obtained by the tracking unit 114. The monitoring-related information is not limited to these, and other information such as the current movement direction and the current movement speed of the person may be stored.

Next, the processing by the information processing apparatus 1 of the embodiment will be described with reference to FIG. FIG. 4 is a flowchart showing processing by the information processing device 1 of the embodiment.

First, in step S1, the acquisition unit 111 acquires the detection result of the reflected wave from the radar device 2.

Next, the determination unit 112 performs a reflection characteristic analysis in step S2, determines whether or not there is a dangerous substance (whether the possibility of existence of the dangerous substance is equal to or higher than a predetermined dangerous substance threshold value) in step S3, and Yes. In the case of, the process proceeds to step S4, and in the case of No, the process returns to step S1.

In step S4, the identification unit 113 identifies a person image which is an image part of a person who is presumed to possess the dangerous goods based on the captured image.

Next, in step S5, the tracking unit 114 tracks the person image identified in step S4.

Next, the notification unit 116 determines whether or not it is the notification timing, and if Yes, the process proceeds to step S7, and if No, the process proceeds to step S8.

In step S7, the calculation unit 115 calculates the suspicious degree of the person, and the notification unit 116 carries information including the current position, the possibility of existence of a dangerous object, and the suspicious degree of the person by a security guard or the like. Notify to.

In step S8, the tracking unit 114 determines whether or not the tracking has been completed, and if Yes, the process ends, and if No, the process returns to step S5.

As described above, according to the monitoring system S of the present embodiment, when it is determined that there is a possibility of existence of a dangerous substance based on the detection result of the reflected wave by the radar device 2, the dangerous substance is determined based on the captured image. By identifying the person who is presumed to possess the dangerous goods, it is possible to strengthen the measures against the person who possesses the dangerous goods in the place where an unspecified number of people pass.

For example, by using the radar reflection intensity, it is possible to detect a person who has a dangerous object inside clothes or a bag with high accuracy. Further, even when a large number of people are shown in the captured image, the suspicious person can be narrowed down by the radar reflection intensity, so that the processing efficiency of the information processing apparatus 1 is improved and the processing speed is increased.

Further, in the monitoring area R other than the monitoring area Ra including the entrance, tracking can be continued based only on the image captured by the surveillance camera 3 even if the radar device 2 is not installed, so that the cost can be reduced. Further, in the monitoring area R where the radar device 2 is installed, the position, moving direction, and moving speed of the person obtained from the radar device 2 may be used for tracking.

Further, since the tracking unit 114 can track a person across a plurality of monitoring areas R based on the position, moving direction, and moving speed of the person image, the person can be more reliably traced beyond the individual monitoring areas R. Can be tracked. In addition to that, by notifying the guards and the like, it is possible to more reliably realize the response such as the security guards calling out to the person concerned. In addition, by notifying not only the current position and the possibility of existence of dangerous goods but also the degree of suspicion of the person concerned, the guards and the like can take more appropriate measures.

The program executed by the information processing apparatus 1 of the present embodiment is a file in an installable format or an executable format, such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD (Digital Versatile Disk). It is recorded and provided on a computer-readable recording medium.

Further, the program may be stored on a computer connected to a network such as the Internet and provided by downloading via the network. Further, the program may be configured to be provided or distributed via a network such as the Internet. Further, the program may be configured to be provided by incorporating it in a ROM or the like in advance.

The program has a module configuration including the above-mentioned parts 111 to 117, and as actual hardware, the above-mentioned parts are loaded on the main storage device by the processor reading the ~ program from the above-mentioned storage medium and executing the program. , Each of the above parts is generated on the main storage device.

Although embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. This novel embodiment can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. Further, this embodiment and its modifications are included in the scope and gist of the invention, and are also included in the scope of the invention described in the claims and the equivalent scope thereof.

Further, the radar device 2 is not limited to the fixed type, and may be a scanning type.

Further, the surveillance camera 3 is not limited to the visible light camera, and may be an infrared camera.

Further, in the above-described embodiment, the number of monitoring areas R is set to four for the sake of brevity, but the number of monitoring areas R is not limited to four, and any number of monitoring areas R may be used, for example, tens or hundreds. And so on.

1 ... Information processing device, 2 ... Radar device, 3 ... Surveillance camera, 11 ... Processing unit, 12 ... Storage unit, 13 ... Input unit, 14 ... Display unit, 15 ... Communication unit, 111 ... Acquisition unit, 112 ... Judgment unit , 113 ... Specific unit, 114 ... Tracking unit, 115 ... Calculation unit, 116 ... Notification unit, 117 ... Control unit, S ... Monitoring system

Claims (5)

The first acquisition step of acquiring the detection result of the reflected wave from the radar device that irradiates the predetermined monitoring area with the millimeter wave and receives the reflected wave, and
A second acquisition step of acquiring a captured image from a photographing unit that captures the predetermined monitoring area, and
A determination step for determining the possibility of existence of a dangerous substance based on the detection result of the reflected wave, and
When it is determined by the determination step that there is a possibility of existence of a dangerous substance, a specific step of identifying a person image which is an image portion of a person who is presumed to possess the dangerous substance based on the photographed image. When,
It is represented by a captured image acquired after the first captured image based on the feature amount of the first person represented by the person image in the first captured image acquired from the photographing unit. A monitoring method comprising a tracking step of tracking a person image indicating the first person. The monitoring area monitored by the monitoring method is configured to divide the area including the predetermined monitoring area into a plurality of monitoring areas.
In the first acquisition step, a photographed image is acquired from a photographing unit that photographs each of the plurality of monitoring areas.
The monitoring method according to claim 1, wherein the tracking step tracks the person image across a plurality of the monitoring areas by using the position, moving direction, and moving speed of the person obtained from the radar device. A suspiciousness calculation step for calculating the suspiciousness of the person based on the position, moving direction, moving speed, and detection result of the reflected wave of the person image, and
The monitoring method according to claim 2, further comprising a notification step of notifying a predetermined information terminal of information including the current position, the possibility of existence of the dangerous substance, and the degree of suspicion of the person. A first acquisition unit that acquires the detection result of the reflected wave from a radar device that irradiates a predetermined monitoring area with a millimeter wave and receives the reflected wave.
A second acquisition unit that acquires a captured image from a photographing unit that captures the predetermined monitoring area, and
A determination unit that determines the possibility of existence of a dangerous substance based on the detection result of the reflected wave,
A specific unit that identifies a person image that is an image portion of a person who is presumed to possess the dangerous material based on the photographed image when the determination unit determines that the dangerous material may exist. When,
It is represented by a captured image acquired after the first captured image based on the feature amount of the first person represented by the person image in the first captured image acquired from the photographing unit. A monitoring device including a tracking unit that tracks a person image indicating the first person. The monitoring area monitored by the monitoring device is configured to divide the area including the predetermined monitoring area into a plurality of monitoring areas.
The first acquisition unit acquires a captured image from a photographing unit that captures each of the plurality of monitoring areas.
The monitoring device according to claim 4, wherein the tracking unit tracks the person image across a plurality of the monitoring areas by using the position, moving direction, and moving speed of the person obtained from the radar device.

Images