JPWO2020152851A1 - Digital search security systems, methods and programs - Google Patents

Abstract

Provided is a digital search security system, a method and a program capable of outputting an alarm before a picking act or when a thief tries to invade by a method different from the picking act. In the digital search security system (1000), a person detection unit (111) that detects a person in the security area and a person in the security area detected by the person detection unit (111) possess tools for intrusion crime. It is provided with a tool detection unit (112) for detecting the presence of a suspicious person and a control unit (110) for notifying the existence of a suspicious person in response to the detection of the tool for an intrusion crime by the tool detection unit (112). ..

Description

The present invention relates to a digital search security system, method and program for performing crime prevention by photographing a security area with a surveillance camera.

Patent Document 1 states, "As an alarm device for detecting a picking action, a picking action detection sensor (vibration sensor) detects a vibration sound due to the picking action, and when such a vibration sound is detected, an alarm is given. Was output to call the attention of those around (paragraph 0002). " The technique of Patent Document 1 relates to an alarm device that outputs an alarm through a built-in speaker when a picking action is detected.

Japanese Unexamined Patent Publication No. 2003-217307

The alarm device described in Patent Document 1 cannot detect until there is a picking action, and cannot output an alarm when a thief tries to invade by a method different from the picking action.
An object of the present invention is to provide a digital search security system, a method and a program capable of outputting an alarm before a picking act or when a thief attempts to invade by a method different from the picking act. ..

In the digital search security system according to the present invention, a person detecting means for detecting a person in a security area and a person in the security area detected by the person detecting means possess a tool for an intrusion crime. It is characterized by comprising a tool detecting means for detecting a suspicious person and a control means for notifying the existence of a suspicious person in response to the detection of the tool for an intrusion crime by the tool detecting means.

With this configuration, when the tool detection unit 113 detects a tool for an intrusion crime, it reports the existence of a suspicious person, so that a thief (intruder) before the picking act or by a method different from the picking act. Can output an alarm when an attempt is made to break in.

By determining the risk level of the intrusion crime based on the type, combination or number of the tools, the control means can respond to the intrusion crime more quickly according to the risk level. In addition, it is possible to send telops according to the degree of risk and report to related organizations.

The control means includes a shooting means for capturing an image in the security area, and the control means saves or limits the number of surveillance cameras by making the shooting direction of the shooting means variable and determining a person at a plurality of positions. An abnormal situation can be grasped by using the surveillance camera provided. It is possible to reduce the cost by reducing the resources for monitoring.

An ID terminal detecting means for detecting an ID (Identification) terminal possessed by a non-intruder is provided, and the control means excludes a person who possesses the ID terminal detected by the ID terminal detecting means from a suspicious person. Therefore, it is possible to reduce unnecessary reports and improve the effectiveness of monitoring. In addition, resources for monitoring can be reduced to reduce costs.

The control means can reduce unnecessary notifications and improve the effectiveness of monitoring by canceling the suspicious person determination at a predetermined time zone in which people come and go in the security area. In addition, resources for monitoring can be reduced to reduce costs.

The recording means includes a recording means for recording a photographed image of the photographing means, and the recording means records a high image quality for a predetermined short time after shooting, and after a predetermined short time elapses, converts the image quality to a low image quality for a predetermined long time. When the control means detects a change in the shooting direction, it saves the high-quality shot image for a predetermined short time until that time, thereby performing digital search security. It is possible to reduce the resources of the recording means by recording the captured high-quality image required for control while recording the captured image with low image quality after a lapse of time.

The tool can be a picking pick, a crowbar, a glass cutter, a knife, a burner, or a thumb-turner to determine which tools or weapons can be used for intrusion.

The control means is provided with a behavior detecting means for detecting the behavior of a person, and the control means further detects the suspicious behavior of the person by notifying the existence of the suspicious person in response to the detection of the suspicious behavior of the person. It can be detected in real time. In addition, suspicious behavior can be detected in real time even with a configuration using an inexpensive camera device.

Further, the digital search security method of the present invention has a step of detecting a person in the security area and that the person in the security area detected by the person detecting means possesses a tool for an intrusion crime. It is characterized by comprising a step of detecting and a step of notifying the existence of a suspicious person in response to the detection of the tool for an intrusion crime by the tool detecting means.

Further, the present invention detects that a computer has a person detecting means for detecting a person in a security area and a person in the security area detected by the person detecting means possesses a tool for an intrusion crime. It functions as a digital search security system including a tool detecting means for detecting a tool and a control means for notifying the existence of a suspicious person in response to the detection of the tool for intrusion crime by the tool detecting means. It is a program for.

According to the present invention, it is possible to output an alarm before the picking action or when a thief tries to invade by a method different from the picking action.

It is a block diagram which shows the structure of the digital search security system which concerns on embodiment of this invention. It is a block diagram of the control part of the monitoring device of the digital search security system which concerns on embodiment of this invention. It is a block diagram which shows the system which used the mobile terminal apparatus of the digital search security system which concerns on embodiment of this invention. It is a figure which shows the security application operation of the smartphone of the digital search security system which concerns on embodiment of this invention. It is a flowchart which shows the registration process of the suspicious person of the control part of the monitoring device of the digital search security system which concerns on embodiment of this invention. It is a flowchart which shows the registration process of the suspicious person of the control part of the monitoring device of the digital search security system which concerns on embodiment of this invention. It is a flowchart which shows the digital search security processing of the control part of the monitoring device of the digital search security system which concerns on embodiment of this invention. It is a flowchart which shows the digital search security processing of the control part of the monitoring device of the digital search security system which concerns on embodiment of this invention. It is a figure explaining the movement of the neck used in the suspicious person determination by the behavior of the digital search security system which concerns on embodiment of this invention. It is a figure which shows the skeleton model in the case of automatically registering a suspicious person in the state of the palm in the normal state of the digital search security system which concerns on embodiment of this invention. It is a figure which shows the skeleton model in the case of automatically registering a suspicious person in the state of the palm at the time of abnormality of the digital search security system which concerns on embodiment of this invention. It is a figure explaining the basic recognition of the behavior suspicious operation by the skeleton extraction of the digital search security system which concerns on embodiment of this invention. It is a figure explaining the example 1 of the behavior suspicious operation by the skeleton extraction of the digital search security system which concerns on embodiment of this invention. It is a figure explaining the example 2 of the behavior suspicious operation by the skeleton extraction of the digital search security system which concerns on embodiment of this invention. It is a figure explaining the example of the behavior suspicious operation by the quick movement of the eyes of the digital search security system which concerns on embodiment of this invention. It is a figure which shows the skeleton model in the case of automatically registering a suspicious person by possessing the key of the digital search security system which concerns on embodiment of this invention. It is a figure which shows the skeleton model in the case of automatically registering a suspicious person by possessing a knife of the digital search security system which concerns on embodiment of this invention. It is a figure which shows the skeleton model in the case of automatically registering a suspicious person by possessing a pistol of the digital search security system which concerns on embodiment of this invention. It is a figure which shows the skeleton model in the case of automatically registering a suspicious person by possessing a club of the digital search security system which concerns on embodiment of this invention. It is a top view which shows the site (security area) to which the digital search security system which concerns on embodiment of this invention is applied. It is a figure which shows the intrusion detection example by the digital search security system which concerns on embodiment of this invention.

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings.
(Embodiment)
FIG. 1 is a block diagram showing a configuration of a digital search security system according to an embodiment of the present invention.
This digital search / security system is suitable for application to business establishments that require a high degree of management, such as residences, corporate offices, factories, research institutes, information processing rooms, and money counting rooms.

As shown in FIG. 1, the digital search security system 1000 includes a plurality of surveillance cameras 11 (photographing means, tool detecting means) installed in a security area, and a motion sensor 20 (human detecting means, tool detecting means). , Wi-Fi (Wireless Fidelity) terminal (hereinafter referred to as "Wi-Fi master unit") 30 installed in the security area, beacon (iBeacon) (registered trademark) master unit 40, and related persons (family members) carry A mobile terminal device 50 (ID (Identification) terminal), a monitoring device 100 that controls the entire system, and an AI (Artificial Intelligence) accelerator (Accelerator) 200 (human detection means) are provided. The security area is a caution area (security target area), and includes, for example, a site, a front door and a veranda in the case of a residence, an elevator hall, a veranda and a window in the case of an office.

Although the monitoring device 100 is installed in the security area, it may be installed outside via a network (not shown). When the monitoring device 100 is installed on a server on the network, a plurality of security areas can be monitored.

<Surveillance camera 11>
The surveillance camera 11 captures an image in the security area.
A part or all of the surveillance camera 11 is a PTZ camera having a PTZ (pan / tilt / zoom) function, and is remotely controlled by the surveillance device 100. The surveillance cameras 11 are installed around the outer wall of the security target building, for example, at the entrance / exit where an intruder can enter / exit, the window opening, the outer periphery of the site boundary, and the corresponding site area. The image taken by the surveillance camera 11 is output to the surveillance device 100 and recorded in the recording unit 160.

<Human sensor 20>
The motion sensor 20 is a thermo camera, an infrared camera, or the like, and detects the temperature of a sensing object in the security area to detect a suspicious person in the security area.

<Wi-Fi master unit 30>
The Wi-Fi master unit 30 exchanges information with the Wi-Fi slave unit 51 of the mobile terminal device 50 using Wi-Fi. Further, the Wi-Fi master unit 30 can acquire position information by Wi-Fi positioning, that is, can acquire position information using a Wi-Fi access point and a predetermined position information service.

<Beacon master unit 40>
iBeacon is a wireless technology that uses Bluetooth Low Energy. The i-beacon is composed of a combination of a beacon master unit 40, which is a beacon device on the transmitting side, and an application of a mobile terminal device 50 (corresponding to the beacon slave unit 52 described later) that supports radio wave reception from the beacon master unit 40. .. The i-beacon transmits unique ID information necessary for identification, and responds only to an application associated with the ID information of the mobile terminal device 50. The application of the mobile terminal device 50 registers the same identifier as the beacon master unit 40. The application (beacon slave unit 52) of the mobile terminal device 50 stands by in the background by executing an application equipped with the i-beacon function, and excites a predetermined action when approaching the beacon of the beacon master unit 40.

[Mobile terminal device 50]
The mobile terminal device 50 is carried by a family member or the like. The mobile terminal device 50 is, for example, a smartphone 50a, a tablet 50b, a notebook computer 50c, or the like. The mobile terminal device 50 is also a mobile phone, a PHS (Personal Handy-Phone System), a PDA (Personal Digital Assistants), a dedicated terminal, or the like. In the present embodiment, the mobile terminal device 50 can be used by a family member or the like in various places (that is, the current position), and receives an e-mail or a video including a moving image from the monitoring device 100 via a telephone line (not shown). It is possible.

In the present embodiment, the mobile terminal device 50 assumes the use of the smartphone 50a (ID terminal), and each individual can use it in various places (that is, the current position). One of the mobile terminal devices 50 is located at a security company (not shown).

The smartphone 50a has a digital search security application (hereinafter referred to as “security application”). If the security application is started by the background processing of each application, for example, it is via the communication carrier network (fixed network), the web service cloud 300 (see Fig. 3 below), or the Internet 303 (see Fig. 3 below). , The cloud server 301 on the web service cloud 300 (see FIG. 3 below) can be connected, and the telop about the suspicious person can be received from the cloud server 301. The smartphone 50a can notify a telop in which a suspicious person is detected on a standby screen or the like.

The smartphone 50a includes a Wi-Fi individual identification device (hereinafter referred to as a "Wi-Fi slave unit") 51, and a GPS 53 that captures the positions of related parties.
The smartphone 50a may be provided with the beacon slave unit 52. Alternatively, the smartphone 50a may include any one of the Wi-Fi slave unit 51, the beacon slave unit 52, and the GPS 53.

<Wi-Fi handset 51>
The Wi-Fi slave unit 51 receives and individually identifies the radio waves of the Wi-Fi master unit 30 installed in the business facility. The monitoring device 100 stores the arrangement information of the Wi-Fi master unit 30 installed in the facility as safety-related information. When the Wi-Fi slave unit 51 approaches the Wi-Fi master unit 30, it is possible to determine the ID and position of the person who carries the mobile terminal device 50.

<Beacon slave unit 52>
The beacon slave unit 52 is an application of the mobile terminal device 50 that supports reception of radio waves from the beacon master unit 40. The beacon master unit 40 transmits a beacon (unique ID information necessary for identification), and the application (beacon slave unit 52) of the mobile terminal device 50 excites a predetermined action when approaching the beacon of the beacon master unit 40. To do.

<GPS53>
The GPS 53 receives radio waves of position information from GPS satellites and the like. The GPS 53 acquires the position information by calculating the current position information as three parameters of latitude, longitude and altitude from the information received via the GPS antenna. The acquired position information is timely transmitted to the monitoring device 100.

In the present embodiment, an example using GPS satellites is shown as a means for acquiring position information, but a method using a positional relationship with a base station other than GPS may also be used. For example, when an Android (registered trademark) smartphone or a high-performance mobile phone with a camera is used as the mobile terminal device 50, which is a mobile terminal, a base station and a mobile phone communication network (not shown) are used in place of or in combination with GPS53. It is also possible to send and receive information to and from the mobile phone company server via the mobile phone company and acquire the current location information of the own terminal from the proximity confirmation.

Further, location information acquisition by Wi-Fi positioning, that is, location information acquisition using a Wi-Fi access point and a predetermined location information service may be used.

[Monitoring device 100]
The monitoring device 100 is installed in the residence of a person concerned (for example, a family member) and centrally manages the security area. The monitoring device 100 may be a general server computer, a personal computer, or the like.

The monitoring device 100 includes a control unit 110 (control means), an input unit 120, a storage unit 130, a display unit 140, an output unit 150, a recording unit 160 (recording means), and a face information DB (database) 165. The image processing unit 170, the interface (I / F) unit 180, and the communication unit 190 are provided, and each unit is connected by a bus 195.

After that, when the subject is described as "○○ part", the control unit 110 reads each program from the ROM as necessary, loads it into the RAM, and executes each function (described later). Each program may be stored in the storage unit 130 in advance, or may be incorporated into the monitoring device 100 when necessary via another storage medium or communication medium.

The control unit 110 is composed of a CPU (Central Processing Unit) or the like, controls the entire monitoring device 100, executes a control program, and functions as a digital search security system. The detailed configuration of the control unit 110 will be described later.

The input unit 120 has an operation panel 120a, and is an input device for the user of the monitoring device 100 to input instructions and the like, such as a keyboard, a mouse, and a touch panel or a microphone provided on the screen of the display unit 140.

The storage unit 130 includes a non-volatile memory such as a ROM (Read Only Memory), a RAM (Random Access Memory), or an EEPROM (Electrically Erasable Programmable Read-Only Memory), and stores various data and programs used by the control unit 110. .. The storage unit 130 stores still images or moving images received from the surveillance camera 11, various data and programs used by the control unit 110, and the like.

The display unit 140 displays the operating status of the monitoring device 100, an image received from the monitoring camera 11, a GUI (Graphical User Interface) for operating the monitoring device 100, and the like.

The output unit 150 is, for example, an audio interface, and outputs an audio signal from the monitoring device 100 to the sound system 158 in the security area. As the audio signal output from the monitoring device 100 to the acoustic system 158, for example, an audio signal input from an audio input device such as a microphone provided in the input unit 120, or music data stored in the storage unit 130 is controlled by the control unit. 110 may be the reproduced audio signal. The sound system 158 includes an amplifier and a plurality of speakers arranged on the premises, and emits a signal input from the monitoring device 100 on the premises.

The recording unit 160 is composed of an external storage device such as an HDD (Hard Disk Drive), and records an image in the security area taken by the surveillance camera 11. The recording unit 160 records high image quality for a predetermined short time after shooting, converts it to low image quality after a predetermined short time elapses, records it for a predetermined long time, and erases and controls the high quality shot image. When the unit 110 detects a suspicious person, it saves a predetermined short-time high-quality photographed image up to that time.

The face information DB 165 accumulates face images (face information) of suspicious persons, store personnel, and the like. Basic data such as the face of a person registered in the face information DB 165 is obtained from the headquarters, the head office, a security company, etc. (not shown) via the I / F unit 180, and the face information DB 165 is constructed. In addition, mutual face information DB can be updated to the latest face image (face information) by exchanging information with the headquarters, head office, security company, etc. (not shown).

The image processing unit 170 is composed of a DSP (Digital Signal Processor) or the like, and performs predetermined processing on the received image. Predetermined processing includes contour extraction, image resizing, resolution conversion processing, and the like.

When the image captured by the surveillance camera 11 in 1 second is, for example, a 5-frame image, the 1/5 second image, the 2/5 second image, the 3/5 second image, the 4/5 second image, and the 5/5 second image. By inputting the outer shape line of the object in the movement, the size of the moving object can be known.
The image processing unit 170 processes the image data taken by the surveillance camera 11 and outputs the image in the security area.

The I / F unit 180 connects each surveillance camera 11 arranged in the security area with the surveillance device 100. Further, the I / F unit 180 is connected to a headquarters, a head office, a security company, etc. (not shown) by a network or a dedicated line.
The communication unit 190 transmits / receives data to / from the mobile terminal device 50 via the base station. The communication unit 190 has a wireless communication function, and is connected to a control board using, for example, a UART (Universal Asynchronous Receiver Transmitter).

[Control unit 110]
FIG. 2 is a block diagram of the control unit 110 of the monitoring device 100 of the digital search security system according to the first embodiment of the present invention.
As shown in FIG. 2, the control unit 110 is configured by a CPU (Central Processing Unit) or the like, controls the entire monitoring device 100, and executes a control program to function as a digital search security system.
The control unit 110 includes a person detection unit 111 (human detection means), a behavior detection unit 112 (behavior detection means), a tool detection unit 113 (tool detection means), and an ID terminal detection unit 114 (ID terminal detection means). It includes a face recognition function unit 115.

The control unit 110 reports the existence of a suspicious person when the tool detection unit 113 detects a tool for an intrusion crime (picking pick, crowbar, glass cutter, knife, burner, or thumb turn).
The control unit 110 determines the risk of intrusion crime based on the type, combination or number of tools.
The control unit 110 makes the shooting direction of the surveillance camera 11 (shooting means) variable, and determines a person at a plurality of positions.
The control unit 110 excludes the person who possesses the ID terminal detected by the ID terminal detection unit 114 from the suspicious person.
The control unit 110 cancels the suspicious person determination at a predetermined time zone in which people come and go in the security area.
The control unit 110 reports the existence of a suspicious person in response to the fact that the behavior detection unit 112 detects a suspicious human behavior. Human behavior includes body movements by skeleton extraction, arm movements, palm condition, and eye contact.

The control unit 110 records in the recording unit 160 with high image quality for a predetermined short time after shooting, converts the image quality to low image quality after the predetermined short time elapses, and records the captured image with high image quality for a predetermined long time. When the image is erased and a change in the shooting direction is detected, the high-quality shot image for a predetermined short time up to that time is saved.

The person detection unit 111 detects a person in the security area by using the AI accelerator 200 (described later). The person detection unit 111 issues a person detection request to the AI accelerator 200, and the AI accelerator 200 executes an AI calculation other than the CPU and transmits the person detection result to the person detection unit 111. Since high speed is required for human detection, the AI accelerator 200 is used for human detection.

In the present embodiment, the person detection unit 111 detects a person by using the AI accelerator 200, but the person may be detected by the thermo camera (or the motion sensor 20). That is, the motion sensor 20 detects the temperature in the security area. Then, the human detection unit 111 detects the presence of a person (suspicious person candidate) by detecting the body temperature of the person by the motion sensor 20 and detecting the change in the captured image by the surveillance camera 11.

The person detection unit 111 may combine the person detection using the AI accelerator 200 and the person detection using the thermo camera (or motion sensor). For example, in the security area, AI accelerator 200 is used for the entrance, veranda, and windows where high speed of human detection is required, and the site of the boundary between the site and the outside where high speed of human detection is not required is a thermo. Use a camera (or motion sensor).

The behavior detection unit 112 detects the behavior of a person. Human behavior includes body movements by skeleton extraction, arm movements, palm condition, and eye contact.
Since the behavior detection is not required to be as fast as the human detection by the human detection unit 111, the behavior of the human is detected by CPU processing. Specific examples of human behavior will be described later in FIGS. 10 to 13.

In the present embodiment, the AI accelerator 200 is used for human detection and the behavior is detected by CPU processing, but the AI accelerator 200 may also be used for the behavior detection. Alternatively, the person detection and the behavior detection may be performed by CPU processing (execution of the control program of the control unit 110).

The tool detection unit 113 detects that a person in the security area detected by the person detection unit 111 possesses a tool for an intrusion crime.
The ID terminal detection unit 114 detects an ID (Identification) terminal possessed by a non-intruder.

The face recognition function unit 115 performs face recognition for collating the face image of a person taken by the surveillance camera 11 with the face image of a suspicious person registered in the face information DB 165. The face recognition function unit 115 performs face recognition for determining whether or not the face of a person included in the image received from the surveillance camera 11 is the face of a suspicious person registered in the face information DB 165. Face recognition can be performed based on the degree of similarity between the face information extracted from the received image and the face information of the suspicious person registered in the face information DB 165.

[AI Accelerator 200]
The AI accelerator 200 is a dedicated processor that detects a person, and uses a computing resource other than the CPU. The AI accelerator 200 is, for example, an accelerator for image processing by a processor enhanced with a GPU (Graphics Processing Unit) and signal processing using an FPGA (Field-Programmable Gate Array). Further, the AI accelerator 200 executes the calculation of AI (Artificial Intelligence) on a dedicated hardware (for example, GPU).

In computer processing by a normal PC, it takes about 1.5 seconds to detect a person (human body) per digital image. Therefore, in the present embodiment, by using the AI accelerator 200, which is a human detection processor, the performance of the computer processing by the PC is about 10 times higher, and the intrusion detection is executed quickly. Further, in the present embodiment, the calculation of AI, which has a high calculation load, is left to the AI accelerator 200, which is a dedicated hardware. This proved that it is possible to detect a person in real time and register a suspicious person even with a configuration using a commercially available camera and inexpensive equipment.

[Digital Search Security System]
FIG. 3 is a configuration diagram showing a digital search security system using the mobile terminal device 50 according to the embodiment of the present invention.
As shown in FIG. 3, the digital search security system 1000 actively sends information on the web service cloud 300 in cooperation with the cloud server (commercial server) 301 and the cloud server 301 that provide the digital search security service. It has a Push notification server 302 that acquires and notifies the user's smartphone 50a (mobile terminal device; ID terminal). The web service cloud 300 is connected to the Internet 303. The smartphone 50a can transmit texts and images to the cloud server 301 on the web service cloud 300 via the Internet 303. Further, the smartphone 50a receives the push notification from the push notification server 302 via the Internet 303. Further, the cloud server 301 and the push notification server 302 are connected to the smartphone 50a equipped with the security application via a communication carrier network (fixed network) (not shown) such as an LTE / 3G network.

As shown in FIG. 3, the digital search security system 1000 detects an intrusion of a suspicious person into the premises and push-notifies the smartphone 50a owned by the person, his / her family, or the like.
When the user taps the notification that comes to the smartphone 50a, the security application is activated and the zoom screen of the suspicious person is displayed, and the telop "A suspicious person has been detected at the entrance on the 1st floor" is played. At the same time, the content is read aloud. In this way, the suspicious person is notified by the telop and voice on the screen of the smartphone 50a. Further, by operating the smartphone 50a user, for example, in an emergency, the related organizations (police / fire department) are notified. In this case, the security company and the head office of the affiliated company are also automatically notified. Also, if there is no urgency or if you want to confirm, only the security company will be notified.

[Security app operation]
FIG. 4 is a diagram showing the operation of the security application of the smartphone 50a.
As shown on the left side of FIG. 4, a telop that detects a suspicious person is notified on the standby screen or the like of the smartphone 50a. By tapping the user, as shown in FIG. 4, the display of the smartphone 50a is switched to the security application operation display, the zoom screen of the suspicious person is displayed, and the position and situation of the suspicious person "suspicious person at the entrance on the 1st floor" is displayed. .. In addition, this telop is read aloud by automatic voice. Further, by tapping the user, the display of the smartphone 50a is displayed on four time-series screens as shown on the right side of FIG.

Hereinafter, the operation of the digital search security system configured as described above will be described.
[Suspicious person registration process]
First, the registration process of a suspicious person in the digital search / security system will be described.

5A and 5B are flowcharts showing a suspicious person registration process of the control unit 110 of the monitoring device 100 of the digital search security system. This flow is executed by the control unit 110 (see FIG. 2) of the monitoring device 100.

In step S1, the person detection unit 111 of the control unit 110 requests the AI accelerator 200 to perform [human detection processing] by AI, and the person detection unit 111 waits for the detection result of the person from the AI accelerator 200. As described above, the person detection unit 111 may detect a person with a thermo camera (or motion sensor 20).

If no person is detected in the security area (step S1: No), this flow ends. When a person is detected in the security area (step S1: Yes), the process proceeds to step S2.

In step S2, the face of a person detected in the security area (in the monitoring area) is photographed. The surveillance camera 11 is, for example, a PTZ camera and is remotely controlled by the surveillance device 100. The surveillance camera 11 is an authentication camera that captures the face of a person. The image taken by the surveillance camera 11 is output to the surveillance device 100.
In step S3, the control unit 110 temporarily stores the photographed image of the human face in the storage unit 130 (see FIG. 1).

In step S4, the face recognition function unit 115 determines whether or not the photographed person's face image matches the person's face image stored in the face information DB 165.
If the photographed person's face image does not match the person's face image stored in the face information DB 165 (step S4: No), the process proceeds to step S6. Further, when the photographed image of the person's face matches the image of the person's face stored in the face information DB 165 (step S4: Yes), the process proceeds to step S5.

In step S5, the control unit 110 determines whether the image of the photographed person's face is a well-meaning person (house owner, friend, acquaintance or courier) or a malicious intruder.
If the photographed person's face image is a well-meaning person, the process is terminated as it is, and if it is a malicious intruder, the process proceeds to step S6.

If the image of the human face taken in step S4 does not match the image of the human face stored in the face information DB 165 (step S4: No), is the person who entered the security area a suspicious person? This is a case where we do not know whether or not. In this case, in step S6, the behavior detection unit 112 of the control unit 110 detects the behavior of a person.

In step S7, the control unit 110 determines whether or not the person detected by the behavior detection unit 112 is suspicious of the behavior.
When it is determined that the behavior is suspicious (step S7: Yes), in step S8, the tool detection unit 113 of the control unit 110 has a tool for an intrusion crime (burglary) by a person (suspicious behavior person) in the security area. Judge whether or not. The tools are picking picks, crowbars, glass cutters, knives, burners or thumb-turners. In addition, the tools include weapons such as knives, clubs and pistols.

When it is not determined that the behavior is suspicious (step S7: No), in this case as well, the tool detection unit 113 of the control unit 110 determines whether or not a person in the security area possesses the tool.

Here, in the present embodiment, priority is given to determining whether or not a person in the security area possesses a tool. That is, if a person is detected in the security area, it is determined whether or not the person in the security area possesses a tool regardless of the determination result of the suspicious behavior of the person in step S7. This is because possession of the above-mentioned tool has a high probability of intrusion (it is considered that such a tool is not possessed unless the purpose of invasion is intended), whereas a suspicious behavior person is not necessarily an intruder. However, if the person is suspicious of behavior and possesses a tool, it can be determined that the risk of intrusion is extremely high.

When it is determined in step S8 that the tool is possessed (step S8: Yes), the person in the security area is a behavior suspicious person and possesses the tool in step S10, and the intruder is dangerous and urgent. Judge and notify the high-ranked telop about the suspicious person.
Further, when it is determined in step S9 that the tool is possessed (step S9: Yes), the person in the security area is not a behavior suspicious person and possesses the tool in step S11, and is a telop with a low rank regarding the suspicious person. Notify.

If it is determined in step S8 or step S9 that the tool is not possessed, the process proceeds to step S12.

In step S12, the control unit 110 registers the face (appearance) of the suspicious person in the security area in the storage unit 130. Here, if the image of the human face taken in step S4 matches the image of the face of the suspicious person in the face information DB 165, the above behavior suspicious or tool possession information is added to the face information as an attribute. Register as information. As a result, the face of the suspicious person and the intrusion information of the intruder are linked, which is useful information for crime prevention.
According to this flow, a suspicious person is registered in the face information DB 165 together with the attribute information.

[Digital search and security processing]
Next, the digital search security process of the digital search security system will be described.
6A and 6B are flowcharts showing digital search security processing of the control unit 110 of the monitoring device 100 of the digital search security system. This flow is mainly executed by the control unit 110 (see FIG. 2) of the monitoring device 100.

First, in step S21, the person detection unit 111 of the control unit 110 requests the AI accelerator 200 to perform [human detection processing] by AI, and the person detection unit 111 waits for the detection result of the person from the AI accelerator 200. .. As described above, the person detection unit 111 may detect a person with a thermo camera (or motion sensor 20).

If no person is detected in the security area, this flow ends. If a person is detected in the security area, the process proceeds to step S22.

When there is a person in the security area, in step S22, the ID terminal detection unit 114 is a person who has an ID terminal (mobile terminal device 50; smartphone 50a) (a person who has a predetermined iBeacon such as a family member). ) Whether or not. The control unit 110 excludes the person who possesses the ID terminal detected by the ID terminal detection unit 114 from the suspicious person (step S22: Yes), and proceeds to step S23.

If the person does not have the ID terminal (step S22: No), in step S23, the control unit 110 determines whether or not it is a predetermined time zone in which people come and go frequently, such as in the evening, in the security area. The control unit 110 cancels the suspicious person determination (step S23: Yes) in a predetermined time zone in which people come and go in the security area in the evening, and proceeds to step S24.

When it is not a predetermined time zone where people come and go frequently (step S23: No) such as in the evening, the surveillance camera 11 takes an image in the security area (inside the surveillance area) in step S24, and the image is recorded in the recording unit 160. ..

In step S25, the control unit 110 changes the shooting direction of the surveillance camera 11, and the face authentication function unit 115 detects the faces of people (suspicious person candidates) at a plurality of positions.

In step S26, the control unit 110 determines whether or not a suspicious person has been detected from the registration information of the suspicious person's face. If the face of a suspicious person is detected (step S26: Yes), the process proceeds to step S33, and if the face of a suspicious person is not detected (step S26: No), the process proceeds to step S27. That is, when the face of a suspicious person is detected, the control unit 110 immediately determines that the suspicious person has been detected in step S33, skips the subsequent processing, and speeds up the processing.

In step S27, the behavior detection unit 112 of the control unit 110 detects the behavior of a person.
In step S28, the control unit 110 determines whether or not the behavior of the person detected by the behavior detection unit 112 is suspicious.

When it is determined that the behavior is suspicious (step S28: Yes), in step S29, the tool detection unit 113 of the control unit 110 has a tool for an intrusion crime (burglary) by a person (behavior suspicious person) in the security area. Judge whether or not.
When it is not determined that the behavior is suspicious (step S28: No), in this case as well, the tool detection unit 113 of the control unit 110 determines in step S30 whether or not a person in the security area possesses the tool.

If the detected person has suspicious behavior and possesses a tool in step S29 (step S29: Yes), the control unit 110 determines in step S31 that the person has the highest risk. If the behavior is suspicious and the tool is not possessed in step S29 (step S29: No), the control unit 110 determines in step S33 that the person is a suspicious person with a normal degree of risk.

If the detected person is not suspicious of behavior in step S30 and possesses a tool (step S30: Yes), the control unit 110 determines in step S32 that the person is a suspicious person with a high degree of risk.
In step S30, if the detected person is not suspicious of behavior and does not have the tool (step S30: No), this flow ends.

In step S34, the control unit 110 classifies the risk of intrusion crime (classification of the degree of danger) based on the type, combination, and number of tools possessed by a person in the security area detected by the tool detection unit 113. ) Is judged. For example, if the tools you have are a pistol or knife, the risk and urgency are the highest, and the burner and picking picks are also the highest. In addition, if there are multiple tools, it is judged that the risk of intrusion crime is high. In addition to the risk to humans, the risk of intrusion crime is judged to be higher when the intrusion speed is high.

If there are multiple suspicious persons, it can be judged that the risk of intrusion crime is higher. If there are multiple suspicious persons, it is expected that the types and number of tools possessed by each intruder will increase. In addition, even if the number of tools does not increase, it is expected that the invasion speed will increase if one person is in charge of the lookout and the other is devoted to the invasion work.
Further, if the person is suspicious of behavior and possesses a tool (step S31), it can be determined that the risk of intrusion is extremely high.

In the present embodiment, the degree of danger is determined by determining a person at a plurality of positions in step S25, determining suspicious behavior in step S28, and determining the degree of danger using a tool possessed in step S34. It is possible to respond to intrusion crimes more quickly according to the situation. On the other hand, by excluding the person who possesses the mobile terminal device 50 in step S22 and excluding the predetermined time zone in step S23, unnecessary processing can be eliminated, and as a result, unnecessary processing can be eliminated. Can reduce costs and speed up processing by reducing computing resources.

Upon receiving the determination of the risk of intrusion crime in step S34, the control unit 110 reports the existence of a suspicious person in step S35. The report transmits information about the suspicious person to, for example, the smartphone 50a of the person concerned. This report should be given a message, mark, highlighted letters, and color coding indicating the degree of danger according to the degree of risk of intrusion crime (classification of the degree of danger). Also, change the report destination according to the degree of danger or urgency. For example, if the intruder's tool is a weapon such as a pistol or knife, the notification telop is the most prominent telop and the voice message also conveys the risk. In addition, the police will be included in the relevant organizations to be reported. And this report is given priority over the automatic registration process of the suspicious person.

As described above, the control unit 110 transmits a telop for notifying the existence of a suspicious person to the standby screen or the like of the smartphone 50a shown on the left side of FIG. By tapping the user, as shown in FIG. 4, the zoom screen of the suspicious person is displayed, the position and situation of the suspicious person "suspicious person at the entrance on the 1st floor" is displayed, and this telop is read out by automatic voice.

Returning to the flow of FIGS. 6A and 6B, in step S36, the recording unit 160 records with high image quality for a predetermined short time after shooting, and after the predetermined short time elapses, converts it to low image quality for a predetermined long time. When the control unit 110 detects a suspicious person, it saves a predetermined short-time high-quality shot image up to that time and ends the process of this flow.

In the present embodiment, the "suspicious person registration process" shown in the flows of FIGS. 5A and 5B and the "digital search security process" shown in the flows of FIGS. 6A and 6B have been described separately, but the processes are summarized. You may. Further, the "suspicious person registration process" shown in the flow of FIGS. 5A and 5B may be a mode in which only the registration process is performed and steps S10 and S11 are omitted.

[Automatic registration of suspicious persons]
Next, the automatic registration of a suspicious person in the digital search security system will be described. This is an additional explanation in the “suspicious person registration process” shown in the flow of FIG. The "automatic registration of suspicious persons" described below can be similarly applied to the "digital search security processing" shown in FIGS. 6A and 6B.
The automatic registration of a suspicious person includes <human detection> (including an object detection function by deep learning), <judgment of a suspicious person's face>, and <judgment of a suspicious person by behavior>. Hereinafter, they will be described in order.

<Detection of people>
The digital search security system 1000 detects an intruder in the security area by detecting a person using the AI accelerator 200.
In deep learning by the AI accelerator 200, the digital search security system 1000 has extremely high accuracy as compared with image recognition by a moving object detection type surveillance camera using a conventional image difference by monitoring only a person. Intruder can be detected in.

<Object detection function by deep learning>
The digital search security system 1000 uses the AI accelerator 200 to implement an object detection function by deep learning.
The outline of object detection will be described.
The AI accelerator 200 performs image processing (convolution operation) one after another by utilizing the two-dimensional property of the image, acquires an object that emphasizes the characteristics of the image, divides the image into parts, and associates the image with the correct label. The correct answer label is "person" in this system, but if it detects a ship in the picture, it is labeled as "ship". The above is just an example, and a person can be learned as a "person", a cat as a "cat", and a ship as a "ship".
Then, thousands of such images are processed and trained.

Learning is a parameter such as the order and degree of convolution operations and image division.
All the parameters of the learned result constitute the classifier. When a digital image is input to this classifier, an area that matches the degree of matching with the label is output. As a result of object detection, whether or not intrusion is possible is calculated for objects having a certain degree of agreement (for example, the degree of agreement is 60%). Whether or not intrusion is possible means that (A) a polygon as a concept representing the site is prepared for each surveillance camera 11 (for example, there are a maximum of three pentagonal regions), and that region and (B) a region where a person is detected are prepared. The intersection of the (rectangular region) is determined, and as a result, the intersection region (C) is obtained.
In the intersection determination, an image in which the two areas (A) and (B) are filled is created, and AND is obtained by bit operation. If the area of (C) as a result is obtained and the total is a ratio of a certain ratio or more of the area (B) where a person is detected, it is determined that an intruder has invaded. The recognition function may be, for example, 25% or more.
If the intrusion is detected as a result of the above determination, the digital search security system 1000 notifies the detection result. The digital search security system 1000 determines how long it takes to stay in the place for illegal invasion in the alert mode (outing security, home security, etc.), and if it is illegal intrusion, notifies the contractor of the illegal invasion by e-mail or the like. (See steps S10 to S11 in FIGS. 5A and 5B, and steps S35 in FIGS. 6A and 6B).

<Judgment of suspicious person's face>
When warning is executed by the fixed surveillance camera 11, in the digital search security system 1000, the face recognition function unit 115 (FIG. 2) of the control unit 110 is stored in the photographed person's face image and the face information DB 165. Judge the match of the image of the person's face. Then, the control unit 110 determines whether the image of the photographed person's face is a well-meaning person (house owner or his friend / acquaintance, courier) or a malicious intruder (steps 5A and 5B). S4 to S5, see steps S24 to S26 of FIGS. 6A and 6B).
However, with face recognition alone, there are cases where the face image is not accumulated in the face information DB 165 or the face is not photographed well. Therefore, in the digital search security system 1000, in addition to face recognition, <determination of suspicious person by behavior> is executed.

<Judgment of suspicious person by behavior>
Human behavior includes neck movement, human body movement, arm movement, palm condition, and eye contact.
FIG. 7 is a diagram for explaining the movement of the neck used in determining a suspicious person based on the behavior.
By extracting the contours of the eyes, eyebrows, nose, mouth, and chin with respect to the position of the entire face shown in Fig. 7 rectangular frame, the neck is moved so that the face turns to the left from the neutral state in the center of Fig. 7. The state and the state of moving the neck so as to turn the face to the right are determined step by step. In addition, using this technique, it is also possible to generate a face image in a neutral state based on a face image taken from a certain angle, and register the face image in the neutral state in the face information DB 165 of FIG. Will be possible.

<Skeletal tracking>
A skeletal tracking technique is used to know the movement of the human body.
Skeletal tracking uses a "skeletal model" that represents a person by the parts ("hands, forearms, upper arms", "knuckles") that connect the moving parts ("shoulders, elbows, wrists", "fingers"). It knows the movement of the body. The skeletal model is represented by the three-dimensional position of the joint and the degree of bending (angle) of the joint. It will be described later with reference to FIGS. 8 to 12.

Skeleton tracking is realized by using the AI accelerator 200 used in human detection for estimation (inference) for deep learning.
In the learning phase of deep learning, a digital image of a person and a pair of skeletal models (three-dimensional coordinates of joints and joint angles) are used as teachers for deep learning. By using this learning data, a skeletal model (body state) corresponding to the newly given digital image is obtained.

In the suspicious behavior model, the angle of the elbow and the angle of the knuckle were used. Intruders walking with their elbows extended are relaxed and non-malicious, and the bent elbows are a pre-action stage, potentially malicious, and require increased alert levels. did.

<Automatic registration of suspicious persons depending on the condition of the palm>
8 and 9 are diagrams showing a skeleton model in the case of automatically registering a suspicious person in the state of the palm, FIG. 8 is an example in a normal state, and FIG. 9 is an example in an abnormal state.
In the palm state, the thumb has a bending angle to be tracked on the first and second joints, and the index finger, middle finger, ring finger, and little finger are tracked on the first, second, and third joints. Bending angle occurs (even if you wear gloves).

As shown in FIG. 8, if the joint is stretched, it is relaxed and the alert level is low. On the other hand, as shown in FIG. 9, it was assumed that the alert level was high because the bent joint may take action. Here, if the elbow bend (angle) is above a certain level, the alert level shall be raised to raise the notification.

<Basic recognition of suspicious behavior by extracting the skeleton>
FIG. 10 is a diagram for explaining the basic recognition of suspicious behavior by skeleton extraction.
FIG. 10 (1) recognizes the uninflected word.
(2) in FIG. 10 recognizes that the person has turned sideways.
(3) in FIG. 10 recognizes that the hand has been put forward.
(4) in FIG. 10 recognizes that the doorknob has been touched.
FIG. 10 (5) shows an enlargement from above the hand.
FIG. 10 (6) shows an enlargement from the side of the hand.

<Example of suspicious behavior by skeleton extraction 1>
FIG. 11 is a diagram illustrating an example 1 of suspicious behavior due to skeleton extraction.
As shown in the upper part of FIG. 11, a suspicious person is illegally operating the keyhole at the entrance of the office during security. As shown in the middle of FIG. 11, the movement of the arm is suspicious. As shown in the lower part of FIG. 11, the movements of the hands and fingers are touching the doorknob.

<Example 2 of suspicious behavior due to skeleton extraction>
FIG. 12 is a diagram illustrating Example 2 of suspicious behavior due to skeleton extraction.
As shown in FIG. 12, during security, an intruder on the balcony is trying to enter the room through a window.
The behavior of suspicious behavior by skeleton extraction has been described above.

Next, the suspicious person determination based on the behavior will be described in more detail.
The behavior is judged from the three viewpoints of "quick movement of eyes", "movement of looking around the neck", and "unnatural behavior of hands".

<Fast movement of eyes>
Use the following method for quick movements.
Eye detection is one of the object detections. This algorithm is used to identify the position of the eyes (rectangular area) in a digital image.
Next, a circle detection technique (for example, a Hough transform technique for detecting a straight line or a circle from an image) is used to detect a circular portion included in the image. It is acceptable as long as the detected circle is included in the position of the eye and the radius of the circle is an appropriate value (for example, 1/6 of the width of the eye). This is equivalent to detecting the pupil.
Determine if the pupil is centered or biased within the eye rectangle.
If the deviation of the pupil moves from side to side in a short period of time, it is judged that the behavior is suspicious.

FIG. 13 is a diagram illustrating an example of suspicious behavior due to fast movement of the eyes.
As shown in the upper part of FIG. 13, a suspicious person is trying to invade the office during security. As shown in the lower middle of FIG. 13, the movement of the eyes and the movement of the arms are suspicious.

<Neck movement>
The movement of the neck is determined by combining the movement of the neck described in FIG. 7 and the detection of a person. It is determined that the movement of the neck is repeated in a short time from side to side in the same manner as in the case of the quick movement detection of the eyes. Behavior is suspicious when the movement of the neck is repeated from side to side in a short time with respect to the position of the human body.

<Hand behavior>
The following method is used for hand movements.
A 3D camera is used to detect specific shape changes of the hands, feet, neck, etc. A 3D camera is a camera having a function of taking a two-dimensional digital image and at the same time acquiring a distance in the depth direction to each pixel of the two-dimensional digital image. By the way, in the person detection, only the circumscribed rectangle of the presence / absence of the person and the position where the person exists can be acquired.
Based on the information detected by the 3D camera, a skeletal model is applied to obtain the status of each joint of a person. As a result, the state of suspicious hand behavior is detected. For example, a state in which a person reaches for a doorknob or a window opening is regarded as suspicious.
The suspicious person judgment based on the behavior has been described above.

[Automatic registration of suspicious persons by possession of tools (weapons)]
Next, the automatic registration of suspicious persons by possessing tools (weapons) will be described.
14 to 17 are diagrams showing a skeletal model (body state) when a suspicious person possessing a tool (weapon) is automatically registered. FIG. 14 is a diagram showing a skeleton model by possessing a key. FIG. 15 is a diagram showing a skeleton model by possessing a knife. FIG. 16 is a diagram showing a skeletal model with a pistol possession. FIG. 17 is a diagram showing a skeleton model by possessing a club.
As mentioned above, by extracting the skeleton, it is possible to recognize that the uninflected word, the person turned sideways, the hand was put forward, and the doorknob was touched.

<Key possession>
As shown on the left of FIG.
Right arm movement: Normal Left arm movement: Normal Right hand movement: Suspicious Left hand movement: Normal Right hand object: Key Left hand object: None. It is determined that the operation shown on the left side of FIG. 14 is generally normal.

The operation progresses to the state shown in the center of FIG.
As shown in the center of FIG.
Right arm movement: Normal Left arm movement: Normal Right hand movement: Suspicious Left hand movement: Normal Right hand object: Key Left hand object: None. The operation shown in the center of FIG. 14 is determined to be totally normal as in the case of the left side of FIG.

Further, the operation progresses to the state shown on the right side of FIG.
As shown on the right of FIG. 14,
Right arm movement: Suspicious left arm movement: Normal Right hand movement: Suspicious left hand movement: Normal Right hand object: Key Left hand object: None. In the movement shown on the right in FIG. 14, the movement of the right arm is also suspicious and the number of "suspicious" increases to two, but as a result of many of each movement, it is judged to be normal overall.
As mentioned above, in the case of possession of the key, it is judged to be normal regardless of the state of the skeletal model (body state), and the suspicious person is not automatically registered.

<Knife possession>
If you have a knife (weapon), it is basically judged to be abnormal and the suspicious person is automatically registered.
As shown on the left of FIG.
Right arm movement: Normal Left arm movement: Normal Right hand movement: Suspicious Left hand movement: Suspicious Right hand object: Knife Left hand object: Knife. Although there are two normal operations shown on the left side of FIG. 15, it is determined that they are abnormal as a result of a large number of each operation.

The operation progresses to the state shown in the center of FIG.
As shown in the center of FIG.
Right arm movement: Suspicious left arm movement: Suspicious right hand movement: Suspicious left hand movement: Suspicious right hand object: Knife Left hand object: Knife. In the operation shown in the center of FIG. 15, it is determined that each operation is suspicious and is totally abnormal.

Further, the operation progresses to the state shown on the right side of FIG.
As shown on the right of FIG.
Right arm movement: Suspicious left arm movement: Suspicious right hand movement: Suspicious left hand movement: Suspicious right hand object: Knife Left hand object: Knife. Even in the operation shown on the right side of FIG. 15, all the operations continue to be in a suspicious state, and it is determined that they are abnormal as a whole.
As described above, in the case of possessing a knife, it is determined that the abnormality is abnormal regardless of the state of the skeletal model (body state), and the suspicious person is automatically registered.
If you have a knife or the following pistol, the danger is great, so you should urgently notify the relevant organizations (police / fire department) without waiting for the operation of the smartphone 50a user.

<Pistol possession>
If you have a pistol (weapon), it will be judged as abnormal and the suspicious person will be automatically registered.
As shown on the left of FIG.
Right arm movement: Normal Left arm movement: Normal Right hand movement: Suspicious Left hand movement: Suspicious Right hand object: Pistol Left hand object: Pistol. Although there are two normal operations shown on the left side of FIG. 16, it is determined that they are abnormal as a result of a large number of each operation.

The operation progresses to the state shown in the center of FIG.
As shown in the center of FIG.
Right arm movement: Normal Left arm movement: Normal Right hand movement: Suspicious Left hand movement: Suspicious Right hand object: Pistol Left hand object: Pistol. Although there are two normal operations shown in the center of FIG. 16, it is determined that they are abnormal as a result of a large number of each operation.

Further, the operation progresses to the state shown on the right side of FIG.
As shown on the right of FIG.
Right arm movement: Suspicious Left arm movement: Suspicious Right hand movement: Suspicious Left hand movement: Abnormal Right hand object: Pistol Left hand object: Pistol. The operation shown on the right side of FIG. 16 is determined to be abnormal as a whole because the normality disappears and an abnormality appears.
As described above, in the case of possession of a pistol, it is determined that it is abnormal regardless of the state of the skeletal model (body state), and the suspicious person is automatically registered.
Since the risk of possessing a pistol is even greater than that of possessing a knife, the relevant organizations (police and fire department) are urgently notified without waiting for the operation of the smartphone 50a user. In addition, an emergency notification is given with priority over the above-mentioned automatic registration process for suspicious persons.

<Possession of a club>
If you have a club (weapon), it will be judged as abnormal and the suspicious person will be automatically registered.
As shown on the left of FIG. 17,
Right arm movement: Normal Left arm movement: Normal Right hand movement: Suspicious Left hand movement: Suspicious Right hand object: Club Left hand object: Club. Although there are two normal operations shown on the left side of FIG. 17, it is determined that they are abnormal as a result of a large number of each operation.

As the operation progresses, the club shown in the center of FIG. 17 is swung up.
As shown in the center of FIG. 17,
Right arm movement: Suspicious Left arm movement: Suspicious Right hand movement: Suspicious Left hand movement: Suspicious Right hand object: Club Left hand object: Club. In the operation shown in the center of FIG. 17, each operation is suspicious, and it is determined that the operation is abnormal as a whole.

Further, the operation progresses, and the club shown on the right side of FIG. 17 is swung up.
As shown on the right of FIG. 17,
Right arm movement: Suspicious Left arm movement: Suspicious Right hand movement: Suspicious Left hand movement: Suspicious Right hand object: Club Left hand object: Club. Even in the operation shown on the right side of FIG. 17, each operation continues to be in a suspicious state, and it is determined that the operation is abnormal as a whole.
As mentioned above, in the case of possession of a club, it is judged to be abnormal regardless of the state of the skeletal model (body state), and the suspicious person is automatically registered.

[Application example]
An application example of intrusion detection by a digital search / security system will be described.
FIG. 18 is a plan view showing a site (security area) to which the digital search security system 1000 is applied. As shown in FIG. 18, a dwelling 6 is built on the site 5, and the dwelling 6 has an entrance 6a on the north side, a door 6b on the west side, and a window 6c on the south side. Around the dwelling 6, there are a north entrance space 5a, a west space 5b, a south space 5c, and an east space 5d. It is envisioned that an intruder will invade through these spaces and use tools (picking picks, crowbars, glass cutters, thumb-turners, etc.) to enter the dwelling with the entrance 6a, door 6b, and window 6c.

FIG. 19 is a diagram showing an example of intrusion detection by the digital search security system 1000.
In the upper left of FIG. 19, there is a person riding a bicycle in the north entrance space 5a, and this person is not a person (family, etc.) who possesses the mobile terminal device 50. In addition, the head is hidden by the umbrella and face recognition is not possible. However, since he is not close to the entrance, it can be judged that he is not an intruder at this point. However, although there is still the possibility of previewing before the crime by bicycle, it is not possible to identify the person as a suspicious person because the face cannot be recognized.

In the upper right of FIG. 19, there is a person at the north entrance 6a, and this person is not a person (family, etc.) who owns the mobile terminal device 50, so the possibility of an intruder remains. As described with reference to FIGS. 11 and 12, suspicious operation by skeleton extraction can be determined, which can lead to detection of a suspicious person. In particular, if this person possesses a tool for an invasion crime (a burglary) (see step S30 in FIG. 6B), it is immediately determined to be a suspicious person.

In the lower left of FIG. 19, a person is crouching in the north entrance space 5a, and this person is not a person (family, etc.) who possesses the mobile terminal device 50. The crouching state can be determined by skeleton extraction. Also, face recognition is not possible. However, since it is not close to the entrance 6a, it can be determined that it is not an intruder at this point.

In the lower right of FIG. 19, there is a person at the north entrance 6a, and this person is not a person (family, etc.) who possesses the mobile terminal device 50. Moreover, the face is hidden by the mask and face recognition is not possible. Further, as described with reference to FIGS. 11 and 12, the movement of the arm is suspicious and the movement of the hand / finger is touching the doorknob due to the suspicious movement determination by the skeleton extraction. It can be judged that there is a high probability of being a suspicious person. In particular, if this person possesses a tool for an invasion crime (a burglary) (see step S30 in FIG. 6B), it is immediately determined to be a suspicious person.

As described in detail above, according to the present embodiment, in the digital search security system 1000 (see FIG. 1), the person detection unit 111 for detecting a person in the security area and the person detection unit 111 have detected the person. The existence of a suspicious person in response to the tool detection unit 113 that detects that a person in the security area possesses a tool for an intrusion crime and the tool detection unit 113 that detects a tool for an intrusion crime. It is provided with a control unit 110 for reporting the above.

With this configuration, when the tool detection unit 113 detects a tool for an intrusion crime, it reports the existence of a suspicious person, so that a thief (intruder) before the picking act or by a method different from the picking act. Can output an alarm when an attempt is made to break in. Further, the presence of a suspicious person can be detected more accurately based on the captured image of the surveillance camera 11 and the temperature of the motion sensor 20.

In the present embodiment, the control unit 110 determines the risk of intrusion crime based on the type, combination or number of tools. It is possible to respond to intrusion crimes more quickly according to the degree of risk. In addition, it is possible to send telops according to the degree of risk and report to related organizations. For example, if you have a knife or pistol, the danger is great, so you should urgently notify the relevant organizations (police and fire department) without waiting for the user's operation.

In the present embodiment, the digital search security system 1000 includes a surveillance camera 11 that captures an image in the security area, and the control unit 110 changes the imaging direction of the surveillance camera 11 to determine a person at a plurality of positions. To do. An abnormal situation can be grasped by saving the number of surveillance cameras 11 or by using a limited number of surveillance cameras 11. It is possible to reduce the cost by reducing the resources for monitoring.

In the present embodiment, the digital search security system 1000 includes an ID terminal detection unit 114 that detects an ID terminal possessed by a non-intruder, and the control unit 110 possesses an ID terminal detected by the ID terminal detection unit 114. Exclude those who are suspicious. By excluding the person who owns the ID terminal from the suspicious person, it is possible to reduce unnecessary reports and effectively monitor. In addition, resources for monitoring can be reduced to reduce costs.

In the present embodiment, the control unit 110 cancels the suspicious person determination at a predetermined time zone in which people frequently come and go in the security area. By canceling the suspicious person judgment in a predetermined time zone when people come and go in the security area, it is possible to reduce unnecessary reports and effectively monitor. In addition, resources for monitoring can be reduced to reduce costs.

In the present embodiment, in the digital search security system 1000, the behavior detection unit 112 captures the characteristics of the movements of the arms and hands, and the characteristics of the movements of the arms and hands are supplementary as a determination material for determining the psychological state of the intruder. Used for. In this way, it distinguishes between a well-meaning person and a malicious intruder, and in the case of an intruder, the risk of intrusion is determined, and the alert state is ranked according to the risk.

In the present embodiment, the digital search security system 1000 notifies the intruder of the normal alert and the alert of a higher rank to the monitoring area, so that the owner of the house can perform the corresponding operation. , The threat can be eliminated with higher accuracy.

In the present embodiment, the digital search security system 1000 includes an AI accelerator 200 which is a computing resource other than the CPU, and the person detection unit 111 of the control unit 110 detects a person in the security area by using the AI accelerator 200. To do. The AI accelerator 200 can detect a person existing in a wide security area in real time by executing a person detection process on a dedicated hardware separately from the CPU process. In addition, a person can be detected in real time even with a configuration using an inexpensive camera device.
In addition, the AI accelerator 200 can detect an intruder with extremely high accuracy as compared with image recognition by a moving object detection type surveillance camera using a conventional image difference.

In the present embodiment, the digital search security system 1000 includes a recording unit 160 that records a captured image of the surveillance camera 11, and the recording unit 160 records a predetermined short time after shooting with high image quality, and the predetermined short time. After a lapse of time, it is converted to low image quality and recorded for a predetermined long time, and the high-quality captured image is erased. When the control unit 110 detects a suspicious person, the high image quality for a predetermined short time up to that time is obtained. Save the captured image. As a result, it is possible to reduce the resources of the recording means by recording the captured image with high image quality required for digital search and security control and recording with low image quality after the lapse of time.

The above description is an example of a preferred embodiment of the present invention, and the scope of the present invention is not limited thereto.

Further, in the above embodiment, the names of the digital search security system and the method are used, but this is for convenience of explanation, and may be a monitoring system, a security system, a search security method, or the like.

The digital search security system and method of the present invention are also realized by a program for making a computer function as the digital search security system or method of the present invention. The program may be stored on a computer-readable recording medium.

The recording medium on which this program is recorded may be the ROM itself of the digital search security system, or a program reading device such as a CD-ROM drive is provided as an external storage device, and the recording medium is inserted therein. It may be a readable CD-ROM or the like.

Further, each of the above configurations, functions, processing units, processing means and the like may be realized by hardware by designing a part or all of them by, for example, an integrated circuit. In addition, each of the above configurations, functions, and the like may be realized by software for the processor to interpret and execute a program that realizes each function. Information such as programs, tables, and files that realize each function can be stored in memory, hard disks, recording devices such as SSDs (Solid State Drives), IC (Integrated Circuit) cards, SD (Secure Digital) cards, optical disks, etc. It can be held on a recording medium.

All publications, patents and patent applications cited herein are incorporated herein by reference only.

The digital search / security system, method and program according to the present invention are expected to be installed in business establishments requiring a high degree of management such as residences, corporate offices, factories, research institutes, information processing rooms, and money counting rooms. To. In addition, residential, commercial facilities, offices, hospitals, hotels, financial institutions, factories, research institutes, power plants, air terminals, meeting places, event halls, stadiums, museums and other buildings indoors and outdoors, transportation trains, ferries , The inside of an airplane, etc. is also applicable.

11 Surveillance camera (shooting means)
20 Motion sensor (suspicious person detection means)
30 Wi-Fi master unit 40 Beacon master unit 50 Mobile terminal device (ID terminal)
50a Smartphone (mobile terminal device; ID terminal)
51 Wi-Fi handset 52 Beacon handset 53 GPS
100 Monitoring device 110 Control unit (control means)
111 human detection unit (human detection means)
112 Behavior detection unit (behavior detection means),
113 Tool detection unit (tool detection means)
114 ID terminal detection unit (ID terminal detection means)
115 Face recognition function unit 120 Input unit 120a Operation panel 130 Storage unit 140 Display unit 150 Output unit 160 Recording unit (recording means)
165 Face information DB
170 Image processing unit 180 Interface (I / F) unit 190 Communication unit 200 AI accelerator (human detection means)
1000 Digital Search Security System

In the digital search security system according to the present invention, a person detecting means for detecting a person in a security area and a person in the security area detected by the person detecting means possess a tool for intrusion crime. and tools detecting means for detecting a behavior detection means for detecting a behavior of a person, including the behavior of the hand carrying the tool the tool detecting unit detects, tools and the behavior detecting means detects that the tool detecting means detects It is characterized by providing a control means for notifying the existence of a suspicious person in response to the determination that the person is a suspicious person as a result of comprehensively determining the behavior of the person .

With this configuration, the tool detection unit 113 detects the tool for the intrusion crime , the behavior detection unit 112 detects the behavior of the hand holding the tool, and comprehensively determines the existence of a suspicious person. Since it receives and reports the existence of a suspicious person, it is possible to output an alarm before the picking action or when a thief (intruder) tries to invade by a method different from the picking action.

The digital search security method of the present invention includes a human detection step of detecting a person in the security zone, human of the security zone in which the person detecting step has detected is in possession of tools for penetration crime and tools detection step of detecting that a behavior detection step of detecting the behavior of a person, including the behavior of the hand carrying the tool the tool detecting step detects said tool said motion detecting step detecting step is a tool that is detected It is characterized by including a control step for notifying the existence of a suspicious person in response to the determination that the person is a suspicious person as a result of comprehensively determining the detected behavior of the person .

Further, the present invention detects that a computer has a person detecting means for detecting a person in a security area and a person in the security area detected by the person detecting means possesses a tool for an intrusion crime. and tools detection means for the tools and behavior detection means for detecting a behavior of a person, including the behavior of the hand detection means carrying the tool detected, the said tool tool detecting means detects said motion detecting means detects In order to function as a digital search security system equipped with a control means for notifying the existence of a suspicious person in response to the fact that the person is determined to be a suspicious person as a result of comprehensively judging the behavior of the person. It is a program.

According to the present invention, it is possible to output an alarm before the picking action or when a thief tries to invade by a method different from the picking action. In addition, suspicious behavior can be detected in real time even with a configuration using an inexpensive camera device.

Claims (10)

Person detection means to detect people in the security area,
A tool detecting means for detecting that a person in the security area detected by the person detecting means possesses a tool for an intrusion crime,
A digital search security system including a control means for notifying the existence of a suspicious person in response to the detection of a tool for an intrusion crime by the tool detection means. The digital search security system according to claim 1, wherein the control means determines the risk of an intrusion crime based on the type, combination or number of the tools. A shooting means for capturing an image in the security area is provided.
The digital search security system according to claim 1, wherein the control means changes the shooting direction of the shooting means to determine a person at a plurality of positions. It is equipped with an ID terminal detecting means for detecting an ID (Identification) terminal possessed by a non-intruder.
The digital search security system according to claim 1, wherein the control means excludes a person who possesses the ID terminal detected by the ID terminal detecting means from a suspicious person. The digital search security system according to claim 1, wherein the control means cancels the suspicious person determination at a predetermined time zone in which people come and go frequently in the security area. A recording means for recording a captured image of the shooting means is provided.
The recording means records high-quality images for a predetermined short time after shooting, converts the image quality to low quality after a predetermined short time, records the images for a predetermined long time, and erases the high-quality shot images. The digital search security system according to claim 3, wherein when the control means detects a change in the shooting direction, a high-quality shot image is stored for a predetermined short time up to that time. The digital search security system according to claim 1, wherein the tool is a picking pick, a crowbar, a glass cutter, a knife, a burner, or a thumb turn. Equipped with a behavior detection means to detect human behavior
The digital search security system according to claim 1, wherein the control means further notifies the existence of a suspicious person in response to the behavior detecting means detecting a suspicious person's behavior. Steps to detect people in the security area and
A step of detecting that a person in the security area detected by the person detection means possesses a tool for an intrusion crime, and
A digital search security method comprising a step of notifying the existence of a suspicious person in response to the detection of a tool for an intrusion crime by the tool detecting means. Computer,
A person detecting means for detecting a person in a security area, a tool detecting means for detecting that a person in the security area detected by the person detecting means possesses a tool for an intrusion crime, and the tool detecting means. A program for functioning as a digital search security system characterized by having a control means for reporting the existence of a suspicious person in response to the detection of a tool for an intrusion crime.

Images