JP2020113945A - Monitoring camera and detection method - Google Patents

Abstract

To provide a monitoring camera that allows a user to flexibly set a detection target to be detected.SOLUTION: A monitoring camera having artificial intelligence includes an imaging unit, a communication unit that receives a parameter relating to a detection target from a terminal device, and a processing unit that constructs the artificial intelligence based on the parameter, and uses the constructed artificial intelligence to detect the detection target from an image captured by the imaging unit.SELECTED DRAWING: Figure 3

Description

The present disclosure relates to a surveillance camera and a detection method.

Patent Document 1 discloses a mobile remote monitoring camera having artificial intelligence. The mobile remote monitoring camera of Patent Document 1 is an all-in-one monitoring camera in which a Web camera, a router, artificial intelligence, and the like are housed in a housing.

International Publication No. 2016/199192

The detection target detected by the surveillance camera may differ depending on the user who uses the surveillance camera. For example, a user detects a person using a surveillance camera. Another user uses a surveillance camera to detect the vehicle. Moreover, another certain user detects a pest by using a surveillance camera.

However, Patent Document 1 does not disclose any specific method of setting a detection target to be detected by the user in the surveillance camera.

The non-limiting example of the present disclosure contributes to the provision of a surveillance camera and a detection method that allow a user to flexibly set a detection target to be detected.

A surveillance camera according to an aspect of the present disclosure is a surveillance camera including artificial intelligence, and includes an imaging unit, a communication unit that receives a parameter related to a detection target from a terminal device, and the artificial intelligence based on the parameter. And a processing unit configured to detect the detection target from a video image captured by the imaging unit, using the constructed artificial intelligence.

A detection method according to an aspect of the present disclosure is a detection method for a surveillance camera having artificial intelligence, in which a parameter related to a detection target is received from a terminal device, and the artificial intelligence is constructed based on the parameter, The artificial intelligence is used to detect the detection target from the image captured by the imaging unit.

Note that these comprehensive or specific aspects may be realized by a system, a device, a method, an integrated circuit, a computer program, or a recording medium. The system, the device, the method, the integrated circuit, the computer program, and the recording medium. May be realized in any combination.

According to an aspect of the present disclosure, a user can flexibly set a detection target to be detected for a surveillance camera.

Further advantages and effects of one aspect of the present disclosure will be apparent from the specification and the drawings. Such advantages and/or effects are provided by the features described in several embodiments and in the specification and drawings, respectively, but not necessarily all to obtain one or more of the same features. There is no.

The figure which showed an example of the surveillance camera system which concerns on 1st Embodiment. FIG. 3 is a diagram for explaining a schematic operation example of a surveillance camera system. Figure showing a block configuration example of the surveillance camera Diagram showing an example of the block configuration of the terminal device Diagram illustrating an example of generation of a learning model and setting of the learning model in a surveillance camera The figure explaining an example of generation of a learning model The figure explaining an example of generation of a learning model The figure explaining an example of generation of a learning model The figure explaining an example of setting of a learning model Flowchart showing an operation example of learning model generation of the terminal device Flowchart showing an example of the operation of the surveillance camera The figure which showed an example of the surveillance camera system which concerns on 2nd Embodiment. The figure explaining an example of selection of the learning model in a server Flowchart showing an example of setting operation of the learning model in the surveillance camera in the terminal device Figure showing a modification of the surveillance camera system The figure which showed an example of the surveillance camera system which concerns on 3rd Embodiment. The figure which showed an example of the surveillance camera system which concerns on 4th Embodiment. Figure showing a modification of the surveillance camera system The flowchart which showed the operation example of the surveillance camera which concerns on 5th Embodiment. Diagram illustrating an example of detection of a detection target by switching learning models The figure explaining an example of setting of a learning model

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed description of well-known matters and duplicate description of substantially the same configuration may be omitted. This is for avoiding unnecessary redundancy in the following description and for facilitating understanding by those skilled in the art.

It should be noted that the accompanying drawings and the following description are provided for those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims by these.

(First embodiment)
FIG. 1 is a diagram showing an example of a surveillance camera system according to the first embodiment. As shown in FIG. 1, the surveillance camera system includes a surveillance camera 1, a terminal device 2, and a reporting device 3.

In addition to the surveillance camera system, FIG. 1 shows a part of the structure A1 and a user U1 who uses the terminal device 2. The structure A1 is, for example, an outer wall or an inner wall of a building. Alternatively, the structure A1 is a pillar or the like installed in a field or the like. The user U1 may be a purchaser who purchased the surveillance camera 1. Further, the user U1 may be a contractor who installs the surveillance camera 1 on the structure A1.

The surveillance camera 1 is installed in, for example, the structure A1, and photographs the surroundings of the structure A1. The surveillance camera 1 is equipped with artificial intelligence, and detects the detection target (predetermined video) from the captured image using the mounted artificial intelligence. In the following, artificial intelligence may be simply referred to as AI.

The detection target includes, for example, detection of a person (discrimination of whether a person is a person). The detection target includes, for example, detection of a specific person (face authentication). Further, the detection target includes, for example, detection of a vehicle such as a bicycle, an automobile, a motorcycle (discrimination as to whether or not it is a vehicle). Further, the detection target is, for example, detection of a car model or a motorcycle model. Further, the detection target includes, for example, detection of an animal (discrimination of whether or not it is an animal). Further, the detection target includes, for example, detection of types of animals such as bear, raccoon dog, deer, horse, cat, dog, and crow. Further, the detection target includes, for example, detection of an insect (distinguishing whether or not it is an insect). Further, the detection target includes, for example, detection of types of insects such as wasps, butterflies and caterpillars. Further, the detection target includes, for example, detection of flower bloom.

The user U1 can set the detection target of the surveillance camera 1 using the terminal device 2. For example, it is assumed that the user U1 wants to detect the car parked in the parking lot using the surveillance camera 1. In this case, the user U1 installs the surveillance camera 1 at a place where the parking lot can be photographed, and uses the terminal device 2 to set the detection target of the surveillance camera 1 to an automobile. Further, for example, the user U1 uses the monitoring camera 1 to detect a boar that appears in and out of the field. In this case, the user U1 installs the surveillance camera 1 at a place where the field can be photographed, and uses the terminal device 2 to set the detection target of the surveillance camera 1 to a boar.

The surveillance camera 1 notifies one or both of the terminal device 2 and the reporting device 3 of the detection result. For example, when the surveillance camera 1 detects an automobile from a video image of a parking lot, the surveillance camera 1 transmits information indicating that the automobile is detected to the terminal device 2. Further, for example, when the surveillance camera 1 detects a boar from a video image of a field, the surveillance camera 1 transmits information indicating that the boar has been detected to the alarm device 3.

The terminal device 2 is, for example, an information processing device such as a personal computer, a smartphone, or a tablet terminal. The terminal device 2 communicates with the surveillance camera 1 by wire or wirelessly.

The terminal device 2 is owned by the user U1, for example. The terminal device 2 sets the detection target of the monitoring camera 1 according to the operation of the user U1. Further, the terminal device 2 receives the detection result of the surveillance camera 1. The terminal device 2 displays the detection result on a display device, or outputs the detection result by voice with a speaker or the like, for example.

The alarm device 3 is installed, for example, on the structure A1 on which the surveillance camera 1 is installed. The alarm device 3 may be installed in a structure different from the structure A1 in which the surveillance camera 1 is installed. The reporting device 3 communicates with the surveillance camera 1 by wire or wirelessly.

The alarm device 3 is, for example, a speaker. The reporting device 3, for example, outputs a sound according to the detection result notified from the surveillance camera 1. For example, when the alarm device 3 receives from the surveillance camera 1 information indicating that a boar has been detected, it issues a sound for expelling the boar from the field.

The alarm device 3 is not limited to the speaker. The alarm device 3 may be, for example, a projector or the like. For example, when the surveillance camera 1 detects an intruder, the alarm device 3 (light projector) may emit light to warn the intruder.

A schematic operation example of the surveillance camera system in FIG. 1 will be described.

FIG. 2 is a diagram illustrating a schematic operation example of the surveillance camera system. 2, the same parts as those in FIG. 1 are designated by the same reference numerals.

A learning model M1 is stored in the terminal device 2. The learning model M1 is a parameter group that characterizes the function of the AI installed in the surveillance camera 1. That is, the learning model M1 is a parameter group that determines an AI detection target mounted on the surveillance camera 1. The AI of the surveillance camera 1 can change the detection target by changing the learning model M1.

The learning model M1 may be, for example, a parameter group that determines the structure of the neural network N1 of the surveillance camera 1. The parameter group that determines the structure of the neural network N1 of the surveillance camera 1 includes, for example, information indicating a connection relationship between units of the neural network N1 or a weighting coefficient. The learning model may be referred to as a learned model, an AI model, a detection model, or the like.

The terminal device 2 generates the learning model M1 according to the operation of the user U1. That is, the user U1 can use the terminal device 2 to set (select) a detection target to be detected by the surveillance camera 1.

For example, when the user U1 wants to detect a car in a parking lot with the monitoring camera 1, the user U1 uses the terminal device 2 to generate a learning model M1 for detecting a car. Further, for example, when the user U1 wants to detect boars appearing in the field with the surveillance camera 1, the user U1 uses the terminal device 2 to generate a learning model M1 for detecting boars. The generation of the learning model will be described in detail below.

When the user U1 generates a learning model using the terminal device 2, the user U1 transmits the generated learning model M1 to the monitoring camera 1. The surveillance camera 1 constructs (forms) an AI based on the learning model M1 transmitted from the terminal device 2. That is, the surveillance camera 1 forms the learned AI based on the learning model M1.

For example, when the learning model M1 received from the terminal device 2 is a learning model for detecting an automobile, the surveillance camera 1 forms a neural network for detecting an automobile from an image. Further, for example, when the learning model M1 received from the terminal device 2 is a learning model that detects boars, the monitoring camera 1 forms a neural network that detects boars from a video image.

In this way, the surveillance camera 1 receives from the terminal device 2 the learning model M1 for constructing the AI for detecting the detection target. Then, the monitoring camera 1 forms an AI based on the received learning model M1 and detects the detection target from the video.

Thereby, the user U1 can flexibly set the detection target to be detected in the surveillance camera 1. For example, when the user U1 wants to detect an automobile with the monitoring camera 1, the user U1 may use the terminal device 2 to generate a learning model M1 for detecting an automobile and transmit the learning model M1 to the monitoring camera 1. Further, for example, when the user U1 wants to detect a boar with the monitoring camera 1, the user U1 may use the terminal device 2 to generate a learning model M1 for detecting a boar and send the learning model M1 to the monitoring camera 1.

Although the learning model M1 is generated by the terminal device 2, the learning model M1 is not limited to this. For example, the learning model M1 may be generated by an information processing device different from the terminal device 2. The learning model M1 generated by the information processing device may be transferred to the terminal device 2 that communicates with the monitoring camera 1, and may be transmitted from the terminal device 2 to the monitoring camera 1.

FIG. 3 is a diagram showing a block configuration example of the surveillance camera 1. In addition to the surveillance camera 1, FIG. 3 also shows an external storage medium 31 that is inserted into and removed from the surveillance camera 1. The external storage medium 31 is, for example, a storage medium such as an SD card (registered trademark).

As shown in FIG. 3, the surveillance camera 1 includes a lens 11, an image sensor 12, a video processing unit 13, a control unit 14, a storage unit 15, an external signal output unit 16, an AI processing unit 17, and A communication unit 18, a TOF (Time of Flight) sensor 19, a microphone 20, a USB I/F (USB: Universal Serial Bus, I/F: Interface) unit 21, and an external storage medium I/F unit 22. Have

The lens 11 forms an image of a subject on the light receiving surface of the image sensor 12. As the lens 11, lenses having various focal lengths or shooting ranges can be used depending on the installation location of the surveillance camera 1, the shooting application, or the like.

The image sensor 12 converts the light received by the light receiving surface into an electric signal. The image sensor 12 is, for example, an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The image sensor 12 outputs an electric signal (analog signal) corresponding to the light received on the light receiving surface to the video processing unit 13.

The video processing unit 13 converts the analog signal output from the image sensor 12 into a digital signal (digital video signal). The video processing unit 13 outputs the digital video signal to the control unit 14 and the AI processing unit 17. The lens 11, the image pickup device 12, and the image processing unit 13 may be regarded as an image pickup unit.

The control unit 14 controls the entire surveillance camera 1. The control unit 14 may be configured by, for example, a CPU (Central Processing Unit) or a DSP (Digital signal Processor).

The storage unit 15 stores a program for operating the control unit 14 and the AI processing unit 17. The storage unit 15 also stores data for the control unit 14 and the AI processing unit 17 to perform calculation processing, or data for the control unit 14 and the AI processing unit 17 to control each unit. Further, the storage unit 15 stores the video data captured by the surveillance camera 1. The storage unit 15 may be configured by a storage device such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, and a HDD (Hard Disk Drive).

The external signal output unit 16 is an output terminal that outputs the video signal output from the video processing unit 13 to the outside.

The AI processing unit 17 detects a detection target from the video of the video signal output from the video processing unit 13. The AI processing unit 17 may be configured by, for example, a CPU or a DSP. Further, the AI processing unit 17 may be configured by a PLD (Programmable Logic Device) such as FPGA (Field-Programmable Gate Array).

The AI processing unit 17 has an AI calculation engine 17a, a decoding engine 17b, and a learning model storage unit 17c.

The AI calculation engine 17a forms an AI based on the learning model M1 stored in the learning model storage unit 17c. For example, the AI calculation engine 17a forms a neural network based on the learning model M1. The video signal output from the video processing unit 13 is input to the AI calculation engine 17a. The AI calculation engine 17a detects a detection target from the image of the input image signal by the neural network based on the learning model M1.

The terminal device 2 generates a learning model M1 as described in detail below. The terminal device 2 encrypts the generated learning model M1 and transmits it to the monitoring camera 1. The decoding engine 17b receives the learning model M1 transmitted from the terminal device 2 via the communication unit 18, decodes the received learning model M1 and stores it in the learning model storage unit 17c.

The learning model storage unit 17c stores the learning model M1 decoded by the decoding engine 17b. The learning model storage unit 17c may be configured by a storage device such as a RAM, a ROM, a flash memory, and an HDD.

The communication unit 18 has a data transmission unit 18a and a data reception unit 18b. The data transmission unit 18a transmits data to the terminal device 2 by short-range wireless communication such as Wi-Fi (registered trademark) or Bluetooth (registered trademark). The data receiving unit 18b receives data transmitted from the terminal device 2 by short-range wireless communication such as Wi-Fi or Bluetooth.

The data transmission unit 18a may transmit data to the terminal device 2 via a network cable (wired) such as an Ethernet (registered trademark) cable. The data receiving unit 18b may receive the data transmitted from the terminal device 2, for example, via a network cable such as an Ethernet cable.

The TOF sensor 19 measures the distance to the detection target, for example. The TOF sensor 19 outputs a signal (digital signal) of the measured distance to the control unit 14.

The microphone 20 converts voice into an electric signal (analog signal). The microphone 20 converts an analog signal into a digital signal and outputs the digital signal to the control unit 14.

A device such as a USB memory or an information processing device is connected to the USB I/F unit 21 via a USB connector. The USB I/F unit 21 outputs a signal transmitted from the device connected to the USB I/F unit 21 to the control unit 14. In addition, the USB I/F unit 21 transmits the signal output from the control unit 14 to the device connected to the USB I/F unit 21.

An external storage medium 31 such as an SD card is inserted into and removed from the external storage medium I/F unit 22.

The learning model M1 may be stored in the external storage medium 31 from the terminal device 2. The decoding engine 17b acquires the learning model M1 from the external storage medium 31 mounted on the external storage medium I/F unit 22, decodes the acquired learning model M1 and stores the learning model M1 in the learning model storage unit 17c. Good.

The learning model M1 may be stored in the USB memory from the terminal device 2. The decoding engine 17b may acquire the learning model M1 from the USB memory attached to the USB I/F unit 21, decode the acquired learning model M1, and store the learning model M1 in the learning model storage unit 17c. The USB memory may also be regarded as an external storage medium.

FIG. 4 is a diagram showing a block configuration example of the terminal device 2. As shown in FIG. 4, the terminal device 2 includes a control unit 41, a display unit 42, an input unit 43, a communication unit 44, an I/F unit 45, and a storage unit 46.

The control unit 41 controls the entire terminal device 2. The control unit 41 may be configured by a CPU, for example.

The display unit 42 is connected to a display device (not shown). The display unit 42 outputs the image data output from the control unit 41 to the display device.

The input unit 43 is connected to an input device (not shown) such as a keyboard or a touch panel superimposed on the screen of the display device, for example. The input unit 43 is connected to an input device such as a mouse, for example. The input unit 43 receives a signal output from the input device according to a user operation, and outputs the signal to the control unit 41.

The communication unit 44 communicates with the surveillance camera 1. The communication unit 44 may communicate with the surveillance camera 1 by short-range wireless communication such as Wi-Fi or Bluetooth. Further, the communication unit 44 may communicate with the surveillance camera 1 via a network cable such as an Ethernet cable, for example.

The external storage medium 31 is inserted into and removed from the I/F unit 45, for example. A USB memory is inserted into and removed from the I/F unit 45, for example.

The storage unit 46 stores a program for operating the control unit 41. In addition, the storage unit 46 stores data for the control unit 41 to perform calculation processing, data for the control unit 41 to control each unit, and the like. Further, the storage unit 46 stores the video data of the surveillance camera 1. The storage unit 46 may be configured by a storage device such as a RAM, a ROM, a flash memory, and an HDD.

FIG. 5 is a diagram illustrating an example of generation of a learning model and setting of the learning model in the monitoring camera 1. 5, the same parts as those in FIG. 1 are designated by the same reference numerals. The surveillance camera 1 is installed in the structure A1 so as to photograph, for example, a parking lot.

1. The terminal device 2 activates an application that generates a learning model according to the operation of the user U1. The terminal device 2 (the activated application for generating the learning model) receives the video data from the monitoring camera 1 according to the operation of the user U1. The received video data may be live data or recorded data.

2. The terminal device 2 displays the video of the video data received from the surveillance camera 1 on the display device. The user U1 searches the video displayed on the display device of the terminal device 2 for a video including a detection target to be detected by the surveillance camera 1.

For example, the user U1 wants to detect an automobile with the surveillance camera 1. In this case, the user U1 searches the video of the parking lot received from the surveillance camera 1 for a video including a car and creates a still image of the searched video. It is desirable to create a plurality of still images. The created still image is stored in the storage unit 46.

3. The terminal device 2 generates a learning model from the still image stored in the storage unit 46 according to the operation of the user U1. For example, the terminal device 2 generates a learning model for the surveillance camera 1 to detect an automobile. The generation of the learning model will be described in detail below.

4. The terminal device 2 transmits (sets) the generated learning model to the monitoring camera 1 according to the operation of the user U1. The surveillance camera 1 forms a neural network according to the learning model for detecting a vehicle transmitted from the terminal device 2. The monitoring camera 1 detects an automobile from the image of the image data captured by the image sensor 12 based on the formed neural network.

In addition, in FIG. 5, an example in which a learning model for detecting a vehicle is generated has been described, but a learning model for detecting another detection target can be similarly generated. For example, it is assumed that the surveillance camera 1 is installed on the structure A1 so as to photograph the fields. The user U1 wants to detect a boar with the surveillance camera 1. In this case, the user U1 generates a still image of a video containing a boar from the video of the video data captured by the surveillance camera 1. The terminal device 2 generates a learning model for detecting a boar from the still image stored in the storage unit 46 according to the operation of the user U1. Then, the terminal device 2 transmits the generated learning model to the surveillance camera 1.

FIG. 6 is a diagram illustrating an example of generation of a learning model. The screen 51 shown in FIG. 6 is displayed on the display device of the terminal device 2.

As described with reference to FIG. 5, the terminal device 2 (application for generating the learning model) displays the video of the video data received from the monitoring camera 1 on the display device. The user operates the terminal device 2 to search an image displayed on the display device of the terminal device 2 for an image including a detection target to be detected by the surveillance camera 1 and create a still image of the searched image.

In the image list 51a of the screen 51 of FIG. 6, the file names of the still images created by the user from the video of the surveillance camera 1 are displayed. In the example of FIG. 6, six still image files are generated.

When a still image file is selected from the image list 51a according to the user's operation, the terminal device 2 displays the image of the selected still image file on the display device of the terminal device 2. The still image 51b shown in FIG. 6 is an image of the still image file “0002.jpg” selected by the user.

The user selects a detection target to be detected by the surveillance camera 1 from the still image 51b. For example, the user wants to detect an automobile with the surveillance camera 1. In this case, the user selects (marks) a car on the still image 51b. For example, the user operates the terminal device 2 and surrounds the automobile with the frames 51c and 51d.

The user marks the automobile in all or part of the still image files displayed in the image list 51a, for example. When marking the vehicle in all or part of the still image file, the user clicks the "generate detection model" icon 51e.

When the icon 51e is clicked, the terminal device 2 transits to a screen for giving a label to the image marked with the still image file (the image surrounded by the frames 51c and 51d). That is, the terminal device 2 transitions to a screen that teaches that the image marked with the still image file is the detection target (automobile).

FIG. 7 is a diagram illustrating an example of generation of a learning model. The screen 52 shown in FIG. 7 is displayed on the display device of the terminal device 2. The screen 52 is displayed on the display device of the terminal device 2 when the icon 51e shown in FIG. 6 is clicked.

A label 52a is displayed on the screen 52. The user selects the check box displayed on the left side of the label 52a to give the label to the detection target marked with the still image.

In the example of FIG. 6, the user marks the automobile in the still image 51b. Therefore, the user selects the check box corresponding to the label 52a of the car (automobile) on the screen 52 of FIG.

After selecting the label, the user clicks the button 52b. When the button 52b is clicked, the terminal device 2 generates a learning model.

For example, when the button 52b is clicked, the terminal device 2 performs learning with the image marked with the still image and the label. The terminal device 2 generates, for example, a parameter group that determines the structure of the neural network of the surveillance camera 1 by learning the image marked with the still image and the label. That is, the terminal device 2 generates a learning model that characterizes the AI function of the surveillance camera 1.

FIG. 8 is a diagram illustrating an example of generation of a learning model. The screen 53 shown in FIG. 8 is displayed on the display device of the terminal device 2. The screen 53 is displayed on the display device of the terminal device 2 when the button 52b shown in FIG. 7 is clicked and a learning model is generated.

The user can give a file name to the learning model generated by the terminal device 2 on the screen 53. In the example of FIG. 8, the file name is “car.model”. After giving the learning model a file name, the user clicks the button 53a. When the button 53a is clicked, the terminal device 2 stores the generated learning model in the storage unit 46.

The terminal device 2 transmits (sets) the learning model stored in the storage unit 46 to the monitoring camera 1 according to the operation of the user.

FIG. 9 is a diagram illustrating an example of setting a learning model. In the screen examples of FIGS. 6 to 8, the screen of the terminal device 2 is described assuming the screen of the personal computer, but in FIG. 9, the screen of the smartphone is assumed. The screen 54 of FIG. 9 is displayed when the application that generates the learning model is started.

The learning model 54a indicates the file name of the learning model stored in the storage unit 46 of the terminal device 2. The learning model 54a is displayed on the display device of the terminal device 2 when the icon 54b on the screen 54 is tapped.

The user selects a learning model to be set in the surveillance camera 1. For example, the user selects the check box displayed on the left side of the learning model 54a and selects the learning model to be set in the surveillance camera 1. In the example of FIG. 9, the user has selected the file name “car.model”.

When selecting the learning model, the user taps the button 54c. When the button 54c is tapped, the terminal device 2 transmits the learning model selected by the user to the surveillance camera 1. Upon receiving the learning model, the surveillance camera 1 forms a neural network according to the received learning model.

FIG. 10 is a flowchart showing an operation example of the learning model generation of the terminal device 2. The control unit 41 of the terminal device 2 acquires the video data of the surveillance camera 1 (step S1). The video data may be live data or recorded data. The control unit 41 of the terminal device 2 may acquire the video data of the monitoring camera 1 from the recorder that records the video of the monitoring camera 1.

The user operates the terminal device 2 to search the video of the surveillance camera 1 for a video including the detection target, and creates a still image including the detection target.

The control unit 41 of the terminal device 2 receives the selection of the still image to be marked on the detection target from the user (step S2). For example, the control unit 41 of the terminal device 2 accepts selection of a still image to be marked on the detection target from the image list 51a of FIG.

The control unit 41 of the terminal device 2 accepts a marking operation for the detection target from the user. For example, the control unit 41 of the terminal device 2 receives the marking operation by the frames 51c and 51d shown in FIG. The control unit 41 of the terminal device 2 stores the still image marked by the user in the storage unit 46 (step S3).

The control unit 41 of the terminal device 2 determines whether or not a user has instructed to generate a learning model (step S4). For example, the control unit 41 of the terminal device 2 determines whether the icon 51e of FIG. 6 has been clicked. When the control unit 41 of the terminal device 2 determines that there is no instruction for generating the learning model from the user (“No” in S4), the process proceeds to step S2.

On the other hand, when the control unit 41 of the terminal device 2 determines that the user has instructed to generate the learning model (“Yes” in S4), the control unit 41 accepts the labeling operation from the user (see FIG. 7). Then, the control unit 41 of the terminal device 2 generates a learning model from the still image stored in the storage unit 46 and the machine learning algorithm (step S5). The machine learning algorithm may be, for example, deep learning.

The control unit 41 of the terminal device 2 transmits the generated learning model to the monitoring camera 1 according to the operation of the user (step S6).

FIG. 11 is a flowchart showing an operation example of the surveillance camera 1. The AI processing unit 17 of the surveillance camera 1 starts the detection operation of the detection target with the activation of the surveillance camera 1 (step S11). For example, the AI processing unit 17 of the surveillance camera 1 forms a neural network based on the learning model transmitted from the terminal device 2 and starts the detection operation of the detection target.

The image sensor 12 of the surveillance camera 1 captures one image (one frame) of an image (step S12).

The control unit 14 of the surveillance camera 1 inputs the video imaged in step S12 to the AI processing unit 17 (step S13).

The AI processing unit 17 of the surveillance camera 1 determines whether or not the video input in step S13 includes a detection target (step S14).

When it is determined in step S14 that the detection target is not included (“No” in S14), the control unit 14 of the surveillance camera 1 moves the process to step S12.

On the other hand, when it is determined in step S14 that the detection target is included (“Yes” in S14), the control unit 14 of the monitoring camera 1 determines whether or not the reporting condition is satisfied (step S15).

The reporting condition includes, for example, detection of parking of a car in a parking lot. For example, the control unit 14 of the surveillance camera 1 may determine that the alarm condition is satisfied when the automobile is detected by the AI processing unit 17.

In addition, the reporting condition includes, for example, detection of a boar that is a harmful beast. For example, the control unit 14 of the surveillance camera 1 may determine that the alerting condition is satisfied when the boar is detected by the AI processing unit 17.

Further, the reporting condition includes, for example, the number of visitors. For example, the control unit 14 of the surveillance camera 1 counts the number of people detected by the AI processing unit 17, and when the counted number of people reaches a preset number, it may be determined that the reporting condition is satisfied. Good.

In addition, the notification condition includes, for example, detection of a specific person. For example, the control unit 14 of the surveillance camera 1 may determine that the reporting condition is satisfied when the AI processing unit 17 detects a specific person (face of the specific person).

Further, the reporting condition includes, for example, detection of flower blooming. For example, the control unit 14 of the surveillance camera 1 may determine that the reporting condition is satisfied when the AI processing unit 17 detects the flowering of a flower.

When the control unit 14 of the monitoring camera 1 determines in step S15 that the notification condition is not satisfied (“No” in S15), the process proceeds to step S12.

On the other hand, when the control unit 14 of the monitoring camera 1 determines in step S15 that the reporting condition is satisfied (“Yes” in S15), the reporting device 3 is used to issue, for example, a sound or the like ( Step S16).

As described above, the communication unit 18 of the surveillance camera 1 receives the learning model related to the detection target from the terminal device 2. The AI processing unit 17 of the surveillance camera 1 constructs an AI based on the learning model received by the communication unit 18, and detects the detection target from the video imaged by the image sensor 12 using the constructed AI. As a result, the user can flexibly set the detection target to be detected in the surveillance camera 1.

Further, the learning model is generated using the video imaged by the surveillance camera 1 installed on the structure A1. As a result, the surveillance camera 1 constructs the AI based on the learning model generated by learning from the video imaged by the surveillance camera 1, so that the detection target can be detected with high accuracy.

(Modification)
The control unit 14 of the surveillance camera 1 may store the detection result in the external storage medium 14 inserted in the external storage medium I/F unit 22. The control unit 14 of the surveillance camera 1 may store the detection result in a USB memory inserted in the USB I/F unit 21. The control unit 14 of the surveillance camera 1 stores the detection result in the storage unit 15, and stores the detection result stored in the storage unit 15 into the external storage medium I/F unit 22 or the USB I/F unit. You may transfer to the USB memory inserted in 21. The control unit 41 of the terminal device 2 acquires the detection result stored in the external storage medium or the USB memory via the I/F unit 45, collects the acquired detection result statistics, and analyzes the statistical result. Good. The control unit 41 of the terminal device 2 may use the analysis result for creating the learning model.

Further, the control unit 14 of the surveillance camera 1 may transmit the detection result terminal device 2 via the communication unit 18. The control unit 41 of the terminal device 2 may take statistics of the detection result transmitted from the surveillance camera 1 and analyze the statistical result. The control unit 41 of the terminal device 2 may use the analysis result for creating the learning model.

(Second embodiment)
In the first embodiment, the learning model is generated in the terminal device 2. In the second embodiment, a case where the learning model is stored in a server connected to a public network such as the Internet will be described.

FIG. 12 is a diagram showing an example of a surveillance camera system according to the second embodiment. 12, the same parts as those in FIG. 1 are designated by the same reference numerals. In the following, parts different from the first embodiment will be described.

The surveillance camera system of FIG. 12 has a server 61 in addition to the surveillance camera system of FIG. The server 61 may have a block configuration similar to that shown in FIG. 4, for example. However, the communication unit of the server 61 is connected to the network 62 by wire, for example. The server 61 may be referred to as an information processing device.

The network 62 is, for example, a public network such as the Internet. The server 61 communicates with the terminal device 2 via the network 62, for example. The communication unit 44 of the terminal device 2 may be connected to the network 62 by wire, for example, or may be connected to the network 62 via a wireless communication network such as a mobile phone.

The server 61 implements an application that generates a learning model. The server 61 generates a learning model from the video of the surveillance camera 1 and stores it in the storage unit.

The server 61 may be managed by, for example, a manufacturer who manufactures the surveillance camera 1. For example, the manufacturer of the surveillance camera 1 receives the video data from the purchaser who purchased the surveillance camera 1. The manufacturer of the surveillance camera 1 uses the server 61 to generate a learning model from the video data provided by the purchaser of the surveillance camera 1. It is considered that the purchaser of the monitoring camera 1 shoots various detection targets using the monitoring camera 1, and the manufacturer of the monitoring camera 1 uses various types of learning models based on video data of shooting various detection targets. Can be generated. Further, the manufacturer of the surveillance camera 1 can generate a learning model from a large amount of video data, and can generate a learning model with high detection accuracy.

Further, the server 61 may be managed by, for example, a contractor who installs the surveillance camera 1 on the structure A1. The contractor of the surveillance camera 1 receives the video data from the purchaser of the surveillance camera 1, like the manufacturer. The contractor of the surveillance camera 1 can generate various types of learning models from video data obtained by photographing various detection targets. Further, the contractor of the surveillance camera 1 can generate a learning model from a large amount of video data, and can generate a learning model with high detection accuracy.

The contractor of the surveillance camera 1 may install the surveillance camera 1 on the structure A1 specializing in detection of a specific detection target, for example. For example, the contractor of the surveillance camera 1 may install the surveillance camera 1 on the structure A1 specializing in detecting harmful animals. In this case, the contractor of the surveillance camera 1 receives the video data on the harmful animal from the purchaser of the surveillance camera 1, so that the learning model specialized for the harmful animal detection can be generated.

The terminal device 2 accesses the server 61 and receives the learning model from the server 61 according to the operation of the user U1. The terminal device 2 transmits the learning model received from the server 61 to the monitoring camera 1 via short-range wireless communication such as Wi-Fi or Bluetooth. Further, the terminal device 2 may transmit the learning model received from the server 61 to the surveillance camera 1 via, for example, a network cable.

Further, the terminal device 2 may store the learning model received from the server 61 in the external storage medium 31 via the I/F unit 45 according to the operation of the user U1. The user U1 may insert the external storage medium 31 into the external storage medium I/F unit 22 of the surveillance camera 1 and set the learning model stored in the external storage medium 31 in the surveillance camera 1.

FIG. 13 is a diagram illustrating an example of learning model selection in the server 61. The screen 71 of FIG. 13 is displayed on the display device of the terminal device 2. The screen 71 is displayed when the application that generates the learning model is started.

The learning model 71a on the screen 71 indicates the name of the learning model stored in the server 61. The learning model 71a is displayed on the display device of the terminal device 2 when the icon 71b on the screen 71 is tapped.

The user selects a learning model to be set in the surveillance camera 1. For example, the user selects the check box displayed on the left side of the learning model 71a and selects the learning model to be set in the surveillance camera 1. In the example of FIG. 13, the user has selected the learning model name “dog”.

When selecting the learning model, the user taps the button 71c. When the button 71c is tapped, the terminal device 2 receives the learning model selected by the user from the server 61, and transmits the received learning model to the monitoring camera 1. Upon receiving the learning model, the surveillance camera 1 forms a neural network based on the received learning model.

FIG. 14 is a flowchart showing an example of setting operation of the learning model in the surveillance camera 1 in the terminal device 2.

The control unit 41 of the terminal device 2 activates the application for setting the learning model in the monitoring camera 1 according to the user's operation (step S21).

The control unit 41 of the terminal device 2 connects to the monitoring camera 1 that sets the learning model according to the user's operation (step S22).

The control unit 41 of the terminal device 2 connects to the server 61 connected to the network 62 according to the user's operation (step S23).

The control unit 41 of the terminal device 2 displays the name of the learning model corresponding to the surveillance camera 1 connected in step S22 among the learning models stored in the server 61 on the display device (step S24). For example, the control unit 41 of the terminal device 2 displays the name of the learning model on the display device, as shown in the learning model 71a in FIG.

Note that the server 61 stores learning models corresponding to various types of surveillance cameras. The control unit 41 of the terminal device 2 displays the name of the learning model corresponding to the monitoring camera 1 connected to step S22 among the learning models corresponding to various types of monitoring cameras on the display device.

The control unit 41 of the terminal device 2 receives the learning model set in the surveillance camera 1 from the user (step S25). For example, the control unit 41 of the terminal device 2 accepts the learning model set in the surveillance camera 1 by the check box displayed on the left side of the learning model 71a in FIG.

The control unit 41 of the terminal device 2 receives the learning model received in step S25 from the server 61 and transmits the received learning model to the monitoring camera 1 (step S26).

As described above, the server 61 may generate and store learning data from video data of various surveillance cameras. The terminal device 2 may acquire the learning data stored in the server 61 and set the learning data in the surveillance camera 1. Thereby, the surveillance camera 1 can build AI based on various types of learning models.

(Modification)
In the above description, the control unit 41 of the terminal device 2 transmits the learning model received from the server 61 to the monitoring camera 1 via the near field communication, the external storage medium 31, or the network cable, but the present invention is not limited to this. The control unit 41 of the terminal device 2 may transmit the learning model received from the server 61 to the monitoring camera 1 via the network 62.

FIG. 15 is a diagram showing a modification of the surveillance camera system. 15, the same components as those in FIG. 12 are designated by the same reference numerals.

In FIG. 15, the communication unit 18 of the surveillance camera 1 is connected to the network 62. The communication unit 18 of the surveillance camera 1 may be connected to the network 62 via a cable such as a network cable, or may be connected to the network 62 via a wireless communication network such as a mobile phone.

The control unit 41 of the terminal device 2 receives the learning model from the server 61 via the network 62, as indicated by an arrow B1 in FIG. The control unit 41 of the terminal device 2 transmits the learning model received from the server 61 to the monitoring camera 1 via the network 62, as indicated by an arrow B2 in FIG.

In this way, the control unit 41 of the terminal device 2 may transmit the learning model received from the server 61 to the surveillance camera 1 via the network 62.

The control unit 41 of the terminal device 2 may instruct the server 61 to transmit the learning model to the surveillance camera 1. That is, the surveillance camera 1 may receive the learning data from the server 61 without passing through the terminal device 2.

(Third Embodiment)
In the third embodiment, the surveillance camera 1 sends an e-mail to a preset address when the notification condition is satisfied. That is, when the surveillance camera 1 satisfies the reporting condition, the surveillance camera 1 notifies the user by e-mail that the reporting condition is satisfied.

FIG. 16 is a diagram showing an example of a surveillance camera system according to the third embodiment. 16, the same components as those in FIG. 15 are designated by the same reference numerals.

In FIG. 16, the mail server 81 is shown. The mail server 81 is connected to the network 62.

When the notification condition is satisfied, the control unit 14 of the surveillance camera 1 transmits a mail addressed to the terminal device 2 to the mail server 81 as indicated by an arrow A11. The email may include the content indicating that the report condition is satisfied and the image of the detection target detected by the monitoring camera 1.

The mail server 81 notifies the terminal device 2 that the mail has been received from the surveillance camera 1. The mail server 81 transmits the mail transmitted from the monitoring camera 1 to the terminal device 2 as indicated by an arrow A12, in response to the request from the terminal device 2 which has received the reception notification of the mail.

The surveillance camera 1 may have a destination address of an email set by the terminal device 2. The address of the terminal device other than the terminal device 2 may be set as the destination address of the mail. For example, the address of the terminal device other than the terminal device 2 used by the user U1 may be set as the destination address of the mail. Further, there may be a plurality of mail destination addresses.

As described above, when the surveillance camera 1 satisfies the reporting condition, the surveillance camera 1 may notify the user that the reporting condition is satisfied by email. Thereby, the user can recognize that the detection target is detected by the monitoring camera 1, for example.

(Fourth Embodiment)
In each of the above embodiments, an example in which the learning model is set in one surveillance camera 1 has been described. In the fourth embodiment, an example of setting a learning model in a plurality of surveillance cameras will be described.

FIG. 17 is a diagram showing an example of the surveillance camera system according to the fourth embodiment. As shown in FIG. 17, the surveillance camera system includes surveillance cameras 91a to 91d, a terminal device 92, a recorder 93, and a mail server 94. The monitoring cameras 91a to 91d, the terminal device 92, the recorder 93, and the mail server 94 are each connected to a LAN (Local Area Network) 95.

The surveillance cameras 91a to 91d have functional blocks similar to the functional blocks of the surveillance camera 1 shown in FIG. The terminal device 92 has the same functional blocks as the terminal device 2 shown in FIG. The same learning model may be set for the monitoring cameras 91a to 91d, or different learning models may be set.

The recorder 93 stores the video data of the surveillance cameras 91a to 91d. The terminal device 92 may generate the learning model of the monitoring cameras 91a to 91d from the live video data of the monitoring cameras 91a to 91d. Further, the terminal device 92 may generate the learning model of the monitoring cameras 91a to 91d from the recorded video data of the monitoring cameras 91a to 91d stored in the recorder 93. The terminal device 92 transmits the generated learning model to the monitoring cameras 91a to 91d via the LAN 95.

When the alarm conditions are satisfied, the surveillance cameras 91a to 91d send a mail addressed to the terminal device 92 to the mail server 94. The mail server 94 transmits the mail transmitted from the monitoring camera 1 to the terminal device 2 in response to the request from the terminal device 2.

In this way, the plurality of surveillance cameras 91a to 91d, the terminal device 92, and the mail server 94 may be connected by the LAN 95. Then, the terminal device 92 may generate a learning model of the plurality of monitoring cameras 91a to 91d and transmit (set) the learning model to the monitoring cameras 91a to 91d. Thereby, the user can detect the detection target by the plurality of monitoring cameras 91a to 91d.

The types of the AIs (AI arithmetic engines) of the monitoring cameras 91a to 91d may be different in each of the monitoring cameras 91a to 91d. In this case, the terminal device 92 generates a learning model suitable for the type of AI in each of the surveillance cameras 91a to 91d.

(Modification)
Although the terminal device 92 generates the learning model in the above, the learning model may be stored in a server connected to a public network such as the Internet.

FIG. 18 is a diagram showing a modified example of the surveillance camera system. 18, the same parts as those in FIG. 17 are designated by the same reference numerals. The surveillance camera system of FIG. 18 has a server 101. The server 101 is connected to the LAN 95 via a network 103, which is a public network such as the Internet, and a gateway 102.

The server 101 has the same function as the server 61 described in FIG. The server 101 generates and stores a learning model based on video data of various monitoring cameras other than the monitoring cameras 91a to 91d. The terminal device 92 may access the server 101, acquire the learning data stored in the server 101, and set the learning data in the monitoring cameras 91a to 91d.

(Fifth Embodiment)
In the fifth embodiment, the surveillance camera 1 stores a plurality of learning models. Then, the monitoring camera 1 selects one of the plurality of learning models according to the instruction from the terminal device 2, and detects the detection target based on the selected learning model. In the following, parts different from the first embodiment will be described.

FIG. 19 is a flowchart showing an operation example of the surveillance camera 1 according to the fifth embodiment. The learning model storage unit 17c of the surveillance camera 1 stores a plurality of learning models.

The surveillance camera 1 is activated, for example, by turning on the power (step S31).

The AI processing unit 17 of the surveillance camera 1 sets one learning model of the plurality of learning models stored in the learning model storage unit 17c in the AI calculation engine 17a (step S32).

It should be noted that the AI processing unit 17 of the surveillance camera 1 sets, for example, the learning model, which was set at the time of the previous startup, among the plurality of learning models stored in the learning model storage unit 17c, in the AI calculation engine 17a. Good. In addition, the AI processing unit 17 of the surveillance camera 1 sets, for example, the learning model initialized by the terminal device 2 in the AI calculation engine 17a among the plurality of learning models stored in the learning model storage unit 17c. Good.

The AI processing unit 17 of the surveillance camera 1 determines whether or not there is an instruction to switch the learning model from the terminal device 2 (step S33).

When the AI processing unit 17 of the monitoring camera 1 determines that the instruction to switch the learning model is given (“Yes” in S33), the terminal device 2 is selected from the plurality of learning models stored in the learning model storage unit 17c. The learning model instructed from is set in the AI calculation engine 17a (step S34).

The AI calculation engine 17a detects a detection target from the image of the image data by the set learning model (forming a neural network according to the set learning model) (step S35).

When it is determined in step S33 that there is no instruction to switch the learning model (“No” in S33), the AI calculation engine 17a of the monitoring camera 1 does not switch the learning model and sets the learning model already set. Thus, the detection target is detected from the image of the image data (step S35).

FIG. 20 is a diagram illustrating an example of detection of a detection target by switching learning models. It is assumed that the learning model storage unit 17c of the monitoring camera 1 stores a learning model A, a learning model B, and a learning model C. The learning model A is a learning model for detecting a person from the video output from the video processing unit 13. The learning model B is a learning model for detecting a dog from the video output from the video processing unit 13. The learning model C is a learning model for detecting boar from the video output from the video processing unit 13.

The AI processing unit 17 receives from the terminal device 2 a notification instructing the use of the learning model A. The AI processing unit 17 sets the learning model A stored in the learning model storage unit 17c in the AI calculation engine 17a according to an instruction from the terminal device 2. As a result, the AI calculation engine 17a detects a person from the image output from the image processing unit 13, as shown in “when using the learning model A” in FIG. 20, for example.

The AI processing unit 17 receives from the terminal device 2 a notification instructing the use of the learning model B. The AI processing unit 17 sets the learning model B stored in the learning model storage unit 17c in the AI calculation engine 17a in response to an instruction from the terminal device 2. As a result, the AI calculation engine 17a detects a dog from the image output from the image processing unit 13, as shown in “when using the learning model B” in FIG. 20, for example.

The AI processing unit 17 receives from the terminal device 2 a notification instructing the use of the learning model C. The AI processing unit 17 sets the learning model C stored in the learning model storage unit 17c in the AI calculation engine 17a according to an instruction from the terminal device 2. As a result, the AI calculation engine 17a detects a boar from the image output from the image processing unit 13, as shown in “when using the learning model C” in FIG. 20, for example.

The AI processing unit 17 receives from the terminal device 2 a notification instructing the use of the learning models A, B, and C. The AI processing unit 17 sets the learning models A, B, and C stored in the learning model storage unit 17c in the AI calculation engine 17a according to the instruction from the terminal device 2. As a result, the AI calculation engine 17a detects a person, a dog, or a boar from the image output from the image processing unit 13, as shown in "Learning model A + learning model B + learning model C" in FIG. To do.

FIG. 21 is a diagram illustrating an example of setting the learning model. 21, the same parts as those in FIG. 9 are designated by the same reference numerals.

The user selects a learning model to send to the surveillance camera 1. For example, the user selects the check box displayed on the left side of the learning model 54a and selects the learning model to be set in the surveillance camera 1. In the example of FIG. 21, the user has selected three learning models.

When the user selects the three learning models, the user taps the button 54c. When the button 54c is tapped, the terminal device 2 transmits the three learning models selected by the user to the surveillance camera 1. When the monitoring camera 1 receives the three learning models, the monitoring camera 1 stores the received three learning models in the learning model storage unit 17c.

After transmitting the three learning models to the monitoring camera 1, the user instructs the monitoring camera 1 on the learning model to be set in the AI calculation engine 17a. The AI processing unit 17 of the surveillance camera 1 sets the learning model instructed by the terminal device 2 in the AI calculation engine 17a among the three learning models stored in the learning model storage unit 17c.

The terminal device 2 can add, change, or update the learning model stored in the learning model storage unit 17c according to the user's operation. In addition, the terminal device 2 can delete the learning model stored in the learning model storage unit 17c according to the operation of the user.

In this way, the surveillance camera 1 may store a plurality of learning models. Then, the monitoring camera 1 may select one of the plurality of learning models according to an instruction from the terminal device 2 and form an AI based on the selected learning model. As a result, the user can easily change the detection target of the surveillance camera 1.

Each functional block used in the description of the above embodiment is typically realized as an LSI which is an integrated circuit. These may be individually made into one chip, or may be made into one chip so as to include some or all of them. The name used here is LSI, but it may also be called IC, system LSI, super LSI, or ultra LSI depending on the degree of integration.

Further, the method of circuit integration is not limited to LSI, and it may be realized by a dedicated circuit or a general-purpose processor. A field programmable gate array (FPGA) that can be programmed after the LSI is manufactured, or a reconfigurable processor capable of reconfiguring the connection and setting of circuit cells inside the LSI may be used.

Furthermore, if integrated circuit technology comes out to replace LSI's as a result of the advancement of semiconductor technology or a derivative other technology, it is naturally also possible to carry out function block integration using this technology. The application of biotechnology is possible. Further, the respective embodiments may be combined.

The present disclosure is useful in surveillance cameras with AI.

1, 91a to 91d Monitoring camera 2, 92 Terminal device 3 Notification device 11 Lens 12 Image sensor 13 Video processing unit 14, 41 Control unit 15, 46 Storage unit 16 External signal output unit 17 AI processing unit 17a AI arithmetic engine 17b Decoding Engine 17c Learning model storage unit 18 Communication unit 18a Data transmission unit 18b Data reception unit 19 TOF sensor 20 Microphone 21 USB I/F unit 22 External storage medium I/F unit 31 External storage medium 42 Display unit 43 Input unit 44 Communication unit 45 I /F section 61,101 server 62,103 network 81,94 mail server 93 recorder 95 LAN
102 gateway A1 structure U1 user M1, M11 learning model

Claims (6)

A surveillance camera with artificial intelligence,
An imaging unit,
A communication unit that receives parameters related to the detection target from the terminal device,
Constructing the artificial intelligence based on the parameters, using the constructed artificial intelligence, a processing unit that detects the detection target from the image captured by the imaging unit,
Surveillance camera having. The parameter is generated using an image captured by the image capturing unit,
The surveillance camera according to claim 1. Based on the detection result of the detection target, it is determined whether or not the alarm condition is satisfied, and if the alarm condition is satisfied, the control unit that outputs alarm information to the alarm device or the terminal device,
The surveillance camera according to claim 1, further comprising: The communication unit receives a plurality of different parameters from the terminal device,
The processing unit, among the plurality of different parameters, constructs the artificial intelligence based on the parameters specified from the terminal device,
The surveillance camera according to any one of claims 1 to 3. An interface unit that receives the parameter from an external storage medium that stores the parameter,
The surveillance camera according to any one of claims 1 to 4, further comprising: A method of detecting a surveillance camera having artificial intelligence,
Receives parameters related to the detection target from the terminal device,
Build the artificial intelligence based on the parameters,
Using the artificial intelligence, detecting the detection target from the image captured by the imaging unit,
Detection method.

Images