WO2020256152A1

WO2020256152A1 - Information processing device, information processing method, and program

Info

Publication number: WO2020256152A1
Application number: PCT/JP2020/024372
Authority: WO
Inventors: トニーシュウ
Original assignee: トニーシュウ
Priority date: 2019-06-21
Filing date: 2020-06-22
Publication date: 2020-12-24
Also published as: JP2021002229A

Abstract

The present invention addresses the problem of achieving a surveillance device (stand-alone surveillance device) that functions independently in a prescribed space in which an abnormality or the like is to be detected.　A captured image acquiring unit 101 acquires data relating to an image obtained as a result of imaging of at least a portion of a prescribed space in which an information processing device is disposed. On the basis of the acquired data relating to the image, a hazard/abnormality detecting unit 105 detects the possibility of a hazardous or abnormal object or phenomenon being present or occurring in the prescribed space. If the detection result obtained by the hazard/abnormality detecting unit 105 satisfies a predetermined condition, a warning notification control unit 106 controls notification of a prescribed warning. The problem described hereinabove is thus resolved.

Description

Information processing equipment, information processing methods, and programs

The present invention relates to an information processing device, an information processing method, and a program.

Conventionally, for example, there is a surveillance camera system in which a surveillance camera is installed inside or around a house, etc., and the data of an image captured by the surveillance camera is transmitted to a monitor or the like and used for crime prevention (for example, patent documents). 1).

JP-A-2019-068228

However, with only the prior art including the technique described in Patent Document 1 described above, the surveillance camera merely captures an image, and the data of the image is sent to an external device such as an external observer via the Internet or the like. It was transmitted, and an abnormality was detected by the external device.
As a result, for example, when an abnormality occurs on the Internet or the like, the detection of the abnormality or the like by the surveillance camera may not function. In addition, even if an abnormality or the like is detected in a remote external device, the measures for the abnormality or the like may be delayed and insufficient measures may be taken.
For this reason, there has been a demand for the realization of a monitoring device (stand-alone monitoring device) that functions independently within a predetermined space for detecting abnormalities, etc., but the situation is that such a request cannot be fully met. Met.

The present invention has been made in view of such a situation, and an object of the present invention is to realize a monitoring device (stand-alone monitoring device) that functions independently in a predetermined space for detecting an abnormality or the like.

The information processing device of one aspect of the present invention is
In an information processing device arranged in a predetermined space
An image acquisition means for acquiring image data obtained as a result of imaging at least a part of the predetermined space, and
A detection means for detecting the possibility that a dangerous or abnormal object or phenomenon exists or occurs in the predetermined space based on the acquired data of the image.
When the detection result by the detection means satisfies a predetermined condition, the notification control means for controlling the notification of a predetermined warning and the notification control means.
To be equipped.

Each of the information processing methods and programs of one aspect of the present invention is a method and program corresponding to the above-mentioned information processing device of one aspect of the present invention.

According to the present invention, it is possible to realize a monitoring device (stand-alone monitoring device) that functions independently in a predetermined space for detecting an abnormality or the like.

It is a figure explaining the principle of the security camera which concerns on one Embodiment of information processing of this invention. It is a figure which shows the example of the processing which can be realized by operating two or more security cameras of FIG. 1 in cooperation with each other. It is a perspective view which shows the structural example of the appearance of the security camera of FIG. It is a block diagram which shows the hardware configuration example inside the security camera of FIG. It is a functional block diagram which shows an example of the functional structure of the security camera of FIG. As an example of a dangerous or abnormal object or phenomenon detected by the security camera of FIG. 5, it is a figure explaining an example of a method of detecting a flame by a fire, and detecting a dangerous act by a knife.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a diagram illustrating the principle of a security camera according to an embodiment of information processing of the present invention.

One or more security cameras (in the example of FIG. 1, two security cameras 1-1, 1-2) are arranged in a predetermined space S such as in a store, and constantly monitor dangerous or abnormal objects or phenomena (in the example of FIG. 1). It is a stand-alone (independent function) AI-equipped security camera that monitors in real time (24 hours).

Here, the installation of AI (Artificial Intelligence) means the following. That is, machine learning is performed in advance based on the data of one or more images determined to actually contain a dangerous or abnormal object or phenomenon, and a predetermined algorithm generated or updated by the machine learning is installed. That is the meaning of installing AI.

Here, a dangerous or abnormal object or phenomenon can be arbitrarily set by the designer, the user, or the like. For example, in this example, the following is set. That is, solids such as blades and weapons, liquids such as powerful drugs, and gases such as flames generated in the event of a fire are set as dangerous or abnormal objects or phenomena. Dangerous acts such as swinging or fighting with a knife, violent acts, flood damage such as drowning in a pool, fire, etc. are set as dangerous or abnormal objects or phenomena. The predetermined space S does not have to be a closed space such as indoors, but may be an open space such as outdoors. That is, flood damage such as drowning people on the coast, forest fires, traffic accidents, etc. are also set as dangerous or abnormal objects or phenomena.

The security cameras 1-1 and 1-2 have a built-in battery so that the predetermined space S can be installed even in a place where power supply is difficult, such as outdoors.

Such security cameras 1-1 and 1-2 can execute the following series of processes.
That is, the security cameras 1-1 and 1-2 image at least a part of the predetermined space S, and the data of the captured image obtained as a result is subjected to predetermined image processing by the built-in GPU (Graphics Processing Unit). Give. Here, the image is a broad concept including a still image and a moving image. Further, the security cameras 1-1 and 1-2 record the voice of the predetermined space S, and perform the predetermined voice processing on the voice data obtained as a result by the built-in GPU (Graphics Processing Unit).
In the security cameras 1-1 and 1-2, a dangerous or abnormal object or phenomenon exists in the predetermined space S based on the data of the captured image (for example, the feature amount) subjected to the image processing in this way. The possibility of occurrence is determined using the predetermined algorithm (AI) described above. Here, the detection result of the possibility is not particularly limited, and may be, for example, a percentage indicating the possibility (a value having a certain range), or a binary value such as yes (100%) or no (0%). ..
When the detection result satisfies the predetermined condition, the security cameras 1-1 and 1-2 notify a predetermined warning. The predetermined conditions are not particularly limited, and for example, when the detection result is represented by a percentage indicating the possibility, a condition such as exceeding a threshold value such as 80% can be adopted, and for example, the detection result is present or absent. When represented by, conditions such as the output of Yes can be adopted.
Further, for example, when the detection result is expressed as a percentage indicating the possibility, the security cameras 1-1 and 1-2 sequentially notify a predetermined warning in multiple stages using a plurality of threshold values (for example, 60%). In the above cases, "Caution", and in the case of 80% or more, "Danger", etc. can be notified in sequence).

For example, in the example of FIG. 1, the security camera 1-1 images a predetermined space S including a phenomenon in which the person H-1 expresses the knife K with respect to the person H-2. The GPU of the security camera 1-1 performs predetermined image processing on the captured image data.
The security camera 1-1 detects the abnormal IR as a dangerous act by the blade itself or the blade based on the data of the captured image (for example, the feature amount) subjected to the image processing in this way.
Therefore, the security camera 1-1 sounds an alarm A in the predetermined space S or transmits an e-mail M-1 to the effect that an abnormal IR has occurred to an external device such as the police as a notification of the abnormal IR warning. be able to.
The notification method is not particularly limited to the example of FIG. 1, and may be arbitrary. For example, an instruction (signal) for operating the sprinkler may be adopted as a fire notification.

Here, in the example of FIG. 1, two security cameras 1-1 and 1-2 are shown, but the number of security cameras is not particularly limited to FIG. 1, and may be arbitrary, and of course, one. However, by distributing two or more security cameras (for example, security cameras 1-1 and 1-2 in the example of FIG. 1) in various places and making them function independently, it is expensive to manage a huge amount of data. You can eliminate the need for a server center.

Further, by operating two or more security cameras in cooperation with each other, various kinds of processing can be executed as shown in FIG. That is, for example, in the description of FIGS. 1 and 2, the security camera 1-1 is a cable connecting the security camera 1-2 and a predetermined cable (the cables connecting the security cameras 1-1 and 1-2 shown in FIGS. 1 and 2). ), And the security cameras 1-1 and 1-2 can exchange image data and information related to the detection of danger or abnormality with each other.
FIG. 2 is a diagram showing an example of processing that can be realized by operating two or more security cameras of FIG. 1 in cooperation with each other.

For example, as shown in A of FIG. 2, when the predetermined space S is wide, the imaging range (angle of view) becomes a blind spot with only one security camera 1-1, and a dangerous or abnormal object or phenomenon occurs. A delay in detection (there is a possibility that the presence or occurrence in the predetermined space S is lower than the actual value) may occur. On the contrary, with only one security camera 1-1, erroneous detection of a dangerous or abnormal object or phenomenon (there is a possibility that it exists or occurs in the predetermined space S is higher than it actually is) may occur.
In order to prevent these things, the security cameras 1-1, 1-2, etc., which are arranged at different positions (and therefore the imaging range and the object imaging direction are different), are obtained as a result of imaging from different viewpoints. Based on the data of each captured image, the possibility that a dangerous or abnormal object or phenomenon exists or occurs in the predetermined space S is detected by using the predetermined algorithm (AI) described above.
As a result, a warning based on at least one of the abnormal IR-1 detected by the security camera 1-1 and the abnormal IR-2 detected by the security camera 1-2 is notified.
By doing so, it is possible that the captured image output as data from at least one of the security cameras 1-1 and 1-2 includes a phenomenon in which the person H-1 expresses the knife K. However, it is more likely that it is included in the captured image output as data from only the security camera 1-1. As a result, when the security cameras 1-1 and 1-2 that capture images from different viewpoints are arranged, the detection of dangerous or abnormal objects or phenomena is compared with the case where only the security cameras 1-1 are arranged. It is possible to prevent the occurrence of delay and the occurrence of false detection.

For example, as shown in B of FIG. 2, the person H-1 may perform a dangerous act with a knife after destroying the security camera 1-2. In this case, when only the security cameras 1-2 are arranged, the dangerous act is not detected by the blade by the person H-1, and no warning may be notified.
In order to prevent this, that is, to reduce the risk that the security cameras 1-2 are destroyed, the two security cameras 1-1, 1-2, etc. are obtained as a result of independent imaging. Based on the data of the captured image, the possibility that a dangerous or abnormal object or phenomenon exists or occurs in the predetermined space S is detected by using the above-mentioned predetermined algorithm (AI).
As a result, even if the security camera 1-2 is destroyed, a warning based on the abnormal IR-2 detected by the security camera 1-1 is notified.

Also, for example, a person H-1 who has already performed a dangerous act may recognize that the image has been taken by the security camera 1-2 and destroy the security camera 1-2. The security camera 1-2 that detects the dangerous act by the person H-1 controls to copy the data of the image in which the dangerous act of the person H-1 is captured to the storage unit of the security camera 1-1. As a result, even if the security camera 1-2 is destroyed by the person H-1, the data of the image in which the dangerous act of the person H-1 is captured is retained.

For example, as shown in C of FIG. 2, it may be necessary to detect two or more dangerous or abnormal objects or phenomena. Specifically, for example, in the predetermined space S, it may be necessary to detect a flame F due to a fire in addition to detecting a dangerous act by a knife by a person H-1.
In such a case, when only one of the security cameras 1-2 is arranged, it becomes difficult to detect two or more dangerous or abnormal objects or phenomena at the same time due to the limitation of the processing capacity, and each warning (Fig. In the example of C of 2, at least one of the warnings based on the two abnormalities IR-4 and IR-5) may be delayed.
In order to prevent such a fear, the two security cameras 1-1 and 1-2 can distribute the responsibility for detection and warning that they are in charge of.
Specifically, for example, the security camera 1-1 is in charge of only the abnormal IR-5 regarding dangerous acts by the blade by the person H-1. On the other hand, the security camera 1-2 can be in charge of only detecting the abnormal IR-5 for the flame F caused by the fire, without being in charge of the abnormal IR-5 for the dangerous act by the blade by the person H-1. ..
As a result, the processing load for detecting each of the security cameras 1-1 and 1-2 is reduced, so that it is possible to easily detect two or more dangerous or abnormal objects or phenomena, and as a result, each of them. It is possible to prevent a delay in the notification of the warning.

In the following, when it is not necessary to distinguish the security cameras 1-1, 1-2, etc. individually, these are collectively referred to as "security camera 1".

FIG. 3 is a perspective view showing a configuration example of the appearance of the security camera of FIG.
As shown in FIG. 3, the security camera 1 has a housing 2 and a camera 3.
By installing the security camera 1 on the upper part (ceiling or the like) of the predetermined space S on the housing 2 side, the camera 3 can image the predetermined space S from the viewpoint from the upper side to the lower side, so that the blind spot is created. It is reduced and the accuracy of detecting dangerous or abnormal objects or phenomena is improved.

FIG. 4 is a block diagram showing an example of the internal hardware configuration of the security camera of FIG.

The security camera 1 contains a CPU (Central Processing Unit) 11, a GPU 12, a ROM (Read Only Memory) 13, a RAM (Random Access Memory) 14, and a bus 15 inside the housing 2 of FIG. It includes an output interface 16, an output unit 17, an input unit 18, a storage unit 19, a communication unit 20, a drive 21, and a battery 22.

The CPU 11 executes various processes according to the program recorded in the ROM 13 or the program loaded from the storage unit 19 into the RAM 14.
The GPU 12 executes various image processing according to the program recorded in the ROM 13 or the program loaded from the storage unit 19 into the RAM 14.
Data and the like necessary for the CPU 11 to execute various processes and the GPU 12 to execute various image processes are also appropriately stored in the RAM 14.

The CPU 11, GPU 12, ROM 13 and RAM 14 are connected to each other via the bus 15. An input / output interface 16 is also connected to the bus 15. An output unit 17, an input unit 18, a storage unit 19, a communication unit 20, and a drive 21 are connected to the input / output interface 16.

The output unit 17 is composed of a speaker 17 and the like. The speaker 17 outputs various sounds for warning notification.
The input unit 18 includes a camera 3, a microphone 41, and the like. The camera 3 captures at least a part of the predetermined space S and outputs the data of the captured image obtained as a result. The microphone 41 inputs the voice emitted in the predetermined space S and outputs it as voice data.
The storage unit 19 is composed of a DRAM (Dynamic Random Access Memory) or the like, and stores various data.
The communication unit 20 is connected to another security camera 1 installed nearby (for example, in the example of FIG. 1, when the security camera 1 shown in FIG. 4 is the security camera 1-1, the other security camera 1-2). Communicate with. The communication unit 20 transmits information for notifying a warning to another device (for example, a device managed by a police (not shown)) via a network N including the Internet, if necessary.

A removable medium 31 made of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately mounted on the drive 21. The program read from the removable media 31 by the drive 21 is installed in the storage unit 19 as needed.
In addition, the removable media 31 can also store various data stored in the storage unit 19 in the same manner as the storage unit 19.

The battery 22 is a power source that supplies sufficient power for the security camera 1 to function. That is, since the security camera 1 does not require a commercial power supply, it can be installed even in a place where the predetermined space S is outdoors or where power supply is difficult, as described above.

By collaborating with various hardware and various software of the security camera 1 shown in FIG. 4, it is possible to detect a dangerous or abnormal object or phenomenon and execute various processes for notifying a warning based on the detection. become.
FIG. 5 is a functional block diagram showing an example of the functional configuration of the security camera of FIG.

As shown in FIG. 5, in the CPU 11 or GPU 12 of the security camera 1, the captured image acquisition unit 101, the image processing unit 102, the voice acquisition unit 103, the voice processing unit 104, the danger abnormality detection unit 105, and the warning The notification control unit 106 and the other device collaborative control unit 107 function.
The storage unit 19 is provided with an image information DB (database) 200, an audio information DB 300, and a learning result DB 400.

As described above, the camera 3 of the security camera 1 images at least a part of the predetermined space S, and outputs the data of the captured image obtained as a result.
The captured image acquisition unit 101 acquires the captured image data output from the camera 3, outputs the data to the image processing unit 102, and appropriately performs compression processing or the like as log data for a certain period of time to store the data in the image information DB 200. ..
The image processing unit 102 performs various image processing (for example, extraction processing of the feature amount of the image) on the data of the captured image from the captured image acquisition unit 101, and outputs the data to the danger abnormality detection unit 105.

As described above, the microphone 41 of the security camera 1 inputs the voice emitted in the predetermined space S and outputs it as voice data.
The voice acquisition unit 103 acquires the voice data output from the microphone 41, outputs the voice data to the voice processing unit 104, and appropriately performs compression processing or the like as log data for a certain period of time to store the voice information DB 300.
The voice processing unit 104 performs various processes (for example, voice extraction processing, etc.) on the voice data from the voice acquisition unit 103, and outputs the data to the danger abnormality detection unit 105.

Here, as will be described later, the detection of the possibility that a dangerous or abnormal object or phenomenon exists or occurs in the predetermined space S by the danger abnormality detection unit 105 is based on the data of the captured image from the image processing unit 102. Is executed. However, if necessary, the voice data from the voice processing unit 104 is further taken into consideration, and the possibility that a dangerous or abnormal object or phenomenon exists or occurs in the predetermined space S is detected.
The reason why the voice data is considered for the detection of the possibility that a dangerous or abnormal object or phenomenon exists or occurs in the predetermined space S will be described below.

That is, the conventional surveillance camera system or the like detects an abnormality or the like (for example, an intrusion of a suspicious person) based only on the image data. However, for a predetermined type of abnormality (for example, a fight, etc.), not only an image but also a sound, information that contributes to detection of the predetermined type of abnormality, etc. (for example, an angry word that is often issued in a fight, etc. Etc.) may be included.
Therefore, the surveillance camera 1 of the present embodiment has a predetermined type of object or phenomenon (for example, a fight, etc.) based on audio data (for example, an angry number, etc.) in addition to image data (for example, an image of a fight, etc.). I am trying to detect about. That is, by considering not only the image but also the sound, more accurate detection becomes possible.
Further, for a predetermined type of abnormality (for example, a fight, etc.), the predetermined type of abnormality, etc. It is expected that there is a peculiar correlation with. However, conventionally, anomalies and the like have not been detected based on such a correlation.
On the other hand, in the present embodiment, a predetermined algorithm is used in which machine learning is performed in advance for the correlation between the image information and the audio information, and the result is generated or updated. Therefore, the accuracy of detection of a predetermined type of object or phenomenon (for example, a fight) is further improved.

The danger abnormality detection unit 105 has a dangerous or abnormal object or phenomenon in the predetermined space S based on the data of the captured image from the image processing unit 102 and the sound data from the sound processing unit 104 as needed. Or detect the possibility of occurrence.
Here, the learning result DB 400 contains data of a predetermined algorithm generated or updated by machine learning performed in advance based on the data of one or more images determined to actually contain a dangerous or abnormal object or phenomenon. It is stored.
Therefore, the danger abnormality detection unit 105 detects the possibility that a dangerous or abnormal object or phenomenon exists or occurs in the predetermined space S by using the predetermined algorithm.

Here, an example of specific detection of a dangerous or abnormal object or phenomenon will be described with reference to FIG.
FIG. 6 is a diagram illustrating an example of a method of detecting a flame due to a fire and a method of detecting a dangerous act by a knife as an example of a dangerous or abnormal object or phenomenon detected by the security camera of FIG.

FIG. 6A is a diagram illustrating an example of a method for detecting a flame caused by a fire.
As shown in A of FIG. 6, the flame F1 caused by artificially created fire (hereinafter referred to as "artificial flame F1") and the flame F2 caused by spontaneous combustion (hereinafter referred to as "natural flame F2") are different. There is.

Therefore, first, machine learning is performed in advance based on the data of one or more images determined to contain the artificial flame F1 and the data of one or more images determined to contain the natural flame F2. By the machine learning, the data of the predetermined algorithm for distinguishing and recognizing the artificial flame F1 and the natural flame F2 is generated or updated from the data of the captured image (appropriately image-processed), and the learning result DB 400 of FIG. Pre-stored in.

In this case, when a fire is generated from a match, a lighter, or the like in the predetermined space S, the captured image data including the artificial flame F1 (different from the learning time) is acquired by the captured image acquisition unit 101 and image processing is performed. Image processing is performed by unit 102.
Therefore, based on the data of the captured image, the danger abnormality detection unit 105 detects that it is very unlikely that a flame due to a fire exists or occurs in the predetermined space S by using the above-mentioned predetermined algorithm. As a result, the notification of a predetermined warning is prohibited (that is, the warning is not notified) by the control of the warning notification control unit 106 described later.

On the other hand, in the predetermined space S, when spontaneous combustion is caused by a fire, the data of the captured image including the natural flame F2 (different from the time of learning) is acquired by the captured image acquisition unit 101, and the image processing unit 102 Image processing is performed by.
Therefore, the danger abnormality detection unit 105 detects that there is a very high possibility that a flame due to a fire exists or occurs in the predetermined space S based on the data of the captured image by using the above-mentioned predetermined algorithm. As a result, a predetermined warning is notified by the control of the warning notification control unit 106, which will be described later.
Here, even in the case of a fire caused by a match, a lighter, or the like, a natural flame F2 (different from that at the time of learning) is generated. Therefore, even in this case, the danger abnormality detection unit 105 detects that the flame due to the fire is very likely to exist or occur in the predetermined space S by using the predetermined algorithm described above. As a result, a predetermined warning is notified by the control of the warning notification control unit 106, which will be described later.

As described above, the predetermined space S is not always a closed space in the room as shown in FIG. 1, and may be an open space such as a forest, although not shown. In this case, even if a fire occurs in a forest or the like, it is often difficult to visually recognize the actual natural flame F2 (different from the time of learning) from a distance. In other words, when the security camera 1 is installed in a place where it is determined to be far away, it is difficult to detect the natural flame F2 (different from the time of learning) based on the data of the captured image.
Therefore, in such a case, in addition to the natural flame F2, the data of one or more images determined to contain a flame (not shown) is further used, and machine learning is performed in advance, and the machine is performed. By learning, data of a predetermined algorithm that distinguishes and recognizes smoke in addition to artificial flame F1 and natural flame F2 is generated or updated from the data of the captured image (with appropriate image processing), and the learning of FIG. The result is stored in the DB 400 in advance.
In this case, when spontaneous ignition is caused by a fire in a forest or the like (predetermined space S), smoke (at the time of learning) at the first stage (the natural flame F2 is not included or is unrecognizable even if it is included). The captured image data including (different) is acquired by the captured image acquisition unit 101, and image processing is performed by the image processing unit 102.
Therefore, based on the data of the captured image, the danger abnormality detection unit 105 has a moderate possibility that a flame due to a fire exists or occurs in the predetermined space S (the possibility will increase in the future). Detect using an algorithm. As a result, under the control of the warning notification control unit 106, which will be described later, for example, "there is a risk of fire" is notified as a predetermined warning.
After that, when the fire further progresses, the data of the captured image including the natural flame F2 is acquired by the captured image acquisition unit 101, and the image processing unit 102 performs image processing.
Therefore, the danger abnormality detection unit 105 detects that there is a very high possibility that a flame due to a fire exists or occurs in the predetermined space S based on the data of the captured image by using the above-mentioned predetermined algorithm. As a result, under the control of the warning notification control unit 106, which will be described later, a notification such as "fire (warning one step higher than the above-mentioned danger)" is given as a predetermined warning.

In contrast to A in FIG. 6 for explaining an example of a method for detecting a flame due to a fire, FIG. 6B in FIG. 6 is a diagram for explaining an example of a method for detecting a dangerous act with a knife.
As shown in B of FIG. 6, the knife K1 that can be purchased at the store (hereinafter referred to as "purchasing knife K1") is packaged, while the knife K2 that is exposed for robbery (hereinafter referred to as "robbery"). (Called Knife K2) is not packaged.
Also, the way the person H-3 holds the purchase knife K1 when purchasing is different from the way the person H-1 holds the robbery knife K2 for robbery (to threaten the person).
Furthermore, regarding the person H-2 as a clerk, the act of selling the purchase knife K1 and the act of being struck by the robbery knife K2 (for example, the act of raising a hand) are different.

Therefore, first, the data of one or more images determined to include the person H-3 who purchases with the purchase knife K1 and the person H-2 as a clerk, and the person H as a clerk with the robbery knife K2. Machine learning is performed in advance based on the data of one or more images determined to include the appearance of threatening -3. Then, by the machine learning, from the data of the captured image (with appropriate image processing), the normal state for purchase (the state where there is no dangerous act by the knife) and the abnormal state due to theft etc. (the state of the dangerous act by the knife). ) And the data of the predetermined algorithm to be recognized separately are generated or updated and stored in advance in the learning result DB 400 of FIG.

Here, in the predetermined space S, the person H-3 (different from the time of learning) brings the purchase knife K1 (different from the time of learning) and the person H-2 as a clerk (different from the time of learning). It shall be handed over to.
In this case, when handed over to the person H-3, the data of the captured image including the person H-3 who purchases with the purchase knife K1 and the person H-2 as a clerk is the captured image acquisition unit 101. Image processing is performed by the image processing unit 102.
Therefore, the danger abnormality detection unit 105 detects that the possibility of a dangerous act by the blade is very low based on the data of the captured image by using the above-mentioned predetermined algorithm. As a result, the notification of a predetermined warning is prohibited (that is, the warning is not notified) by the control of the warning notification control unit 106 described later.

On the other hand, in the predetermined space S, the person H-1 (different from the time of learning) uses the robber knife K2 (different from the time of learning) to use the person H-3 as a clerk (different from the time of learning). Suppose you are threatening.
In this case, the captured image data including the appearance of threatening the person H-3 as a clerk with the robbery knife K2 is acquired by the captured image acquisition unit 101, and the image processing unit 102 performs image processing.
Therefore, the danger abnormality detection unit 105 detects that the possibility of a dangerous act by the blade is very high based on the data of the captured image by using the above-mentioned predetermined algorithm. As a result, a predetermined warning is notified by the control of the warning notification control unit 106, which will be described later.

Returning to FIG. 5, the warning notification control unit 106 executes control to notify a predetermined warning when the detection result of the danger abnormality detection unit 105 satisfies the predetermined condition.
For example, the warning notification control unit 106 issues an alarm (for example, alarm A in FIG. 1) as a predetermined warning from a speaker 17 included in the security camera 1 or another speaker independent of the security camera 1 via the communication unit 20. , Control for ringing in the predetermined space S can be executed. Further, for example, the warning notification control unit 106 sends an e-mail (for example, the e-mail M-1 of FIG. 1) to the effect that an abnormality (for example, the abnormal IR of FIG. 1) has occurred as a predetermined warning via the communication unit 20. It is possible to execute control to transmit to an external device such as the police (not shown).

The other device collaborative control unit 107 performs at least a part of the above-mentioned series of processes (for example, the processes of the danger abnormality detection unit 105 and the warning notification control unit 106) with another security camera 1 (for example, the security of the example of FIG. Cooperate with Camera 1 Perform control to collaborate with (at least some of the other unshown security cameras).
The method of collaboration is not particularly limited, but for example, the method of collaboration as described in each of A to C in FIG. 2 described above can be adopted.

The present invention is not limited to the above-described embodiment, and modifications, improvements, and the like within the range in which the object of the present invention can be achieved are included in the present invention.

For example, in the above-described embodiment, the power source of the security camera 1 is a built-in battery, but the power source is not particularly limited to this, and for example, a solar panel power source or the like may be used.

Further, for example, in the above description, the communication unit 20 communicates with another device (for example, a device managed by a police not shown) via a network N including the Internet, or communicates with another security camera via a predetermined cable. Was connected and communicated. However, the method related to the communication of the communication unit 20 is not particularly limited to this. For example, as the communication method of the security camera 1, one or both of a wired method and a wireless method may be adopted. Further, for example, as the communication method of the security camera 1, a method capable of communicating with a plurality of devices (security camera 1 or an external device) via the network N may be adopted, and one pair with another device without going through the network N. A method of communicating with 1 may be adopted.

For example, as the communication method of the security camera 1, a wireless method that does not go through the network N can be adopted. Specifically, for example, as a method related to communication of the communication unit 20, a short-range communication method using Bluetooth (registered trademark), an ad hoc communication method using Wi-Fi (registered trademark), or the like can be adopted. As a result, the warning notification control unit 106 can notify a predetermined warning as follows.

For example, the warning notification control unit 106 can transfer a predetermined warning to a surrounding camera. That is, for example, the warning notification control unit 106 of the security camera 1 transfers to the other security cameras 1 around the security camera 1 that an abnormality (for example, the abnormality IR in FIG. 1) has occurred as a predetermined warning. can do. At this time, the transfer is repeated until the security camera 1 capable of controlling transmission to an external device such as the police is transferred to the effect that an abnormality (for example, the abnormal IR in FIG. 1) has occurred. As a result, for example, even when the security camera 1 cannot connect to the network N including the Internet or the like, the security camera 1 is an external device such as the police that an abnormality (for example, the abnormality IR in FIG. 1) has occurred. Can be sent to.

Further, for example, with respect to the security camera 1, the appearance configuration shown in FIG. 3 and the hardware configuration shown in FIG. 4 are merely examples for achieving the object of the present invention, and are not particularly limited. Furthermore, the information processing device to which the present invention is applied does not need to be configured as a security camera 1, and if it can execute the above-mentioned series of processes, its appearance configuration and internal hardware configuration Is not particularly limited.

Further, for example, the functional block diagram shown in FIG. 4 is merely an example and is not particularly limited. That is, it suffices if the information processing system (not shown) including one or more security cameras 1 has a function capable of executing the above-mentioned series of processes as a whole, and what kind of function block is used to realize this function. Is not particularly limited to the example of FIG.

Further, the location of the functional block is not limited to FIG. 5, and may be arbitrary. For example, although not shown, when a control device for controlling each of one or more surveillance cameras 1 is arranged in or near the predetermined space S, the danger abnormality detection unit 105, the warning notification control unit 106, And the other device cooperative control unit 107 and the like may be provided in the control device.
Further, one functional block may be configured by a single piece of hardware, a single piece of software, or a combination thereof.
Further, each of the functions shown by the functional blocks of FIG. 5 may function in either or both of the CPU 11 and the GPU 12.

When the processing of each functional block is executed by software, the programs constituting the software are installed on a computer or the like from a network or a recording medium.
The computer may be a computer embedded in dedicated hardware. Further, the computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose smartphone or a personal computer in addition to a server.

The recording medium containing such a program is not only composed of removable media, which is distributed separately from the device main body to provide the program to each user, but also is preliminarily incorporated in the device main body to each user. It is composed of the provided recording media and the like.

In the present specification, the steps for describing a program recorded on a recording medium are not necessarily processed in chronological order according to the order, but are not necessarily processed in chronological order, but in parallel or individually. It also includes the processing to be executed.

Further, in the present specification, the term of the system means an overall device composed of a plurality of devices, a plurality of means, and the like.

Summarizing the above, the information processing apparatus to which the present invention is applied need only have the following configuration, and various various embodiments can be taken.

To summarize the above, the information processing device to which the present invention is applied is
An information processing device (for example, the surveillance camera 1 in FIGS. 3 and 4) arranged in a predetermined space (for example, the predetermined space S in FIG. 1).
An image acquisition means (for example, the captured image acquisition unit 101 in FIG. 5) for acquiring image data obtained as a result of imaging at least a part of a predetermined space, and
A detection means (for example, the danger abnormality detection unit 105 in FIG. 5) for detecting the possibility that a dangerous or abnormal object or phenomenon exists or occurs in the predetermined space based on the acquired data of the image.
When the detection result by the detection means satisfies a predetermined condition, the notification control means (for example, the warning notification control unit 106 in FIG. 5) that controls the notification of a predetermined warning,
To be equipped.
As a result, the information processing device can function as a monitoring device (stand-alone monitoring device) that functions independently in a predetermined space for detecting an abnormality or the like.

Further, the detection means is a predetermined algorithm generated or updated by machine learning performed based on the data of one or more images determined to actually contain a dangerous or abnormal object or phenomenon (for example, the learning of FIG. 5). The above possibility is detected by using a predetermined algorithm) stored as data in the result DB 400.
be able to.

Further, a voice acquisition means (for example, the voice acquisition unit 103 in FIG. 5) for acquiring voice data emitted in the predetermined space is further provided.
The detection means detects the possibility that a dangerous or abnormal object or phenomenon exists or occurs in the predetermined space based on the acquired audio data in addition to the acquired image data. To do,
be able to.

Collaborative control means (for example, other device collaborative control unit 107 in FIG. 5) that executes control for cooperating with other information processing devices for at least a part of the processing of the detection means and the notification control means.
Can be further prepared.

1,1-1,1-2 ... Security camera, 3 ... Camera, 11 ... CPU, 12 ... GPU, 19 ... Storage unit, 20 ... Communication unit, 101 ... -Captured image acquisition unit, 102 ... image processing unit, 103 ... voice acquisition unit, 104 ... voice processing unit, 105 ... danger abnormality detection unit, 106 ... warning notification control unit, 107.・・ Other device collaborative control unit, 200 ・・・ image information DB, 300 ・・・ voice information DB, 400 ・・・ learning result DB

Claims

In an information processing device arranged in a predetermined space
An image acquisition means for acquiring image data obtained as a result of imaging at least a part of the predetermined space, and
A detection means for detecting the possibility that a dangerous or abnormal object or phenomenon exists or occurs in the predetermined space based on the acquired data of the image.
When the detection result by the detection means satisfies a predetermined condition, the notification control means for controlling the notification of a predetermined warning and the notification control means.
Information processing device equipped with.
The detection means uses a predetermined algorithm generated or updated by machine learning based on the data of one or more images determined to actually contain a dangerous or anomalous object or phenomenon to obtain the possibility. To detect,
The information processing device according to claim 1.
A voice acquisition means for acquiring voice data emitted in the predetermined space,
With more
The detection means detects the possibility that a dangerous or abnormal object or phenomenon exists or occurs in the predetermined space based on the acquired audio data in addition to the acquired image data. To do,
The information processing device according to claim 1 or 2.
A collaborative control means that executes control for collaborating with another information processing apparatus at least a part of the processes of the detection means and the notification control means.
The information processing apparatus according to any one of claims 1 to 3, further comprising.
In the information processing method executed by the information processing device arranged in the predetermined space,
An image acquisition step of acquiring image data obtained as a result of imaging at least a part of the predetermined space, and
A detection step for detecting the possibility that a dangerous or abnormal object or phenomenon exists or occurs in the predetermined space based on the acquired data of the image.
When the detection result obtained by the processing of the detection step satisfies a predetermined condition, the notification control step for controlling the notification of a predetermined warning and the notification control step
Information processing methods including.
A computer that controls an information processing device placed in a predetermined space
An image acquisition step of acquiring image data obtained as a result of imaging at least a part of the predetermined space, and
A detection step for detecting the possibility that a dangerous or abnormal object or phenomenon exists or occurs in the predetermined space based on the acquired data of the image.
When the detection result obtained by the processing of the detection step satisfies a predetermined condition, the notification control step for controlling the notification of a predetermined warning and the notification control step
A program that executes control processing including.