JP7162412B2 - detection recognition system - Google Patents

Description

The present invention relates to sensing and recognition systems.

Conventionally, a camera that detects/recognizes an object such as a person or an object is known (see Patent Documents 1 and 2, for example). Such a camera is used, for example, as a surveillance camera for the purpose of crime prevention, and issues an alarm when an abnormality is detected through detection/recognition.

In recent years, machine learning has been widely used in fields such as image recognition. Deep learning, for example, is known as a machine learning method. Deep learning uses multi-layered neural networks to learn features of data, and is known to enable highly accurate image recognition.

JP 2012-208851 A JP 2010-160743 A

By the way, surveillance cameras are often used for a long period of time after installation. Technology for detecting and recognizing objects from images is advancing day by day. The detection/recognition technology used may become obsolete.
In addition, since the optimum algorithm for detection/recognition changes depending on the environment such as the location where it is used and the subject to be photographed, the detection/recognition firmware that was originally installed in the camera was installed before the camera was installed. The detection/recognition algorithms used may not provide sufficient detection/recognition.

The present invention has been made in view of the above-mentioned circumstances, and the detection/recognition performance of detecting a feature included in an image and recognizing a recognition target set from this feature is improved by updating the firmware for detection/recognition. It is an object of the present invention to provide a detection and recognition system that can be improved by

To achieve the above object, the detection and recognition system of the present invention comprises:
Equipped with imaging means for imaging, detection/recognition means, and a server,
The detection/recognition means includes detection/recognition firmware, and detects features included in the image from the image acquired by the imaging means under the control of the detection/recognition firmware, and recognizes a set recognition target. Also, the detection/recognition firmware can be updated to new detection/recognition firmware generated by the detection/recognition firmware generating means,
The server generates machine learning means for generating a detection/recognition algorithm by machine learning using the image acquired by the imaging means as teacher data, and generates new detection/recognition firmware for the detection/recognition means from the detection/recognition algorithm. and the detection/recognition firmware generation means for generating the detection/recognition firmware.

In the present invention, the imaging means takes an image. Then, the detection/recognition means detects features included in the image from the image obtained by the image pickup means under the control of the detection/recognition firmware, and recognizes the set recognition target. Also, the machine learning means of the server generates a detection/recognition algorithm by machine learning using the image acquired by the imaging means as teacher data. Then, the detection/recognition firmware generation means of the server converts the generated detection/recognition algorithm into firmware adapted to the detection/recognition means (detection/recognition firmware). Then, the detection/recognition firmware of the detection/recognition means is updated to the new detection/recognition firmware generated by the detection/recognition firmware generating means.
Therefore, a detection/recognition algorithm that enables more accurate detection and recognition is generated by machine learning from the image obtained by the imaging means, and this detection/recognition algorithm is converted into firmware suitable for the detection/recognition means. However, since the detection/recognition firmware of the detection/recognition means can be updated, the performance of detection/recognition can be improved.

In the configuration of the present invention, it is preferable that the machine learning means perform machine learning using an image obtained when the detection/recognition means incorrectly recognizes the set recognition target as teacher data.

According to such a configuration, the machine learning means learns the image when the detection/recognition means misrecognizes the recognition target so as not to misrecognise the recognition target. Since detection/recognition algorithms can be generated and new detection/recognition firmware can be generated, detection/recognition performance can be reliably improved.

Further, in the configuration of the present invention,
with at least one camera,
Preferably, the camera comprises the imaging means and the detection/recognition means.

According to such a configuration, since it is not necessary to provide a terminal or the like having detection/recognition means in addition to the camera, the size of the entire system can be reduced. Also, the camera can update the detection/recognition firmware of the detection/recognition means with new detection/recognition firmware created as a result of machine learning on the server, thus improving the detection/recognition performance of the camera. be able to. Therefore, it is possible to easily improve the detection/recognition performance of the camera even after installation.

Further, in the configuration of the present invention,
It is preferable to provide a plurality of cameras in which at least a part of the imaging range of the imaging means and the set recognition target overlap.

According to such a configuration, it is possible to perform detection/recognition by capturing an image of a predetermined range with a plurality of cameras. Therefore, since the same object or the same phenomenon can be detected/recognized by a plurality of cameras, the accuracy of detection/recognition can be improved.

Further, in the configuration of the present invention,
The machine learning means, when some cameras among the plurality of cameras recognize overlapping recognition targets, other cameras among the plurality of cameras do not recognize the overlapping recognition targets. Machine learning is preferably performed using the obtained image as teacher data.

According to such a configuration, when at least one camera recognizes overlapping recognition targets, machine learning is performed using an image in which the overlapping recognition targets could not be recognized by other cameras as teacher data. be able to. Therefore, it is possible to perform machine learning using an image that is highly likely to have been unable to recognize a recognition target even though the recognition target was desired to be recognized, as training data, thereby improving the efficiency of machine learning.

Moreover, in the above configuration of the present invention, it is preferable that at least one of the plurality of cameras is a camera with a different imaging means.

According to such a configuration, it is difficult to perform detection/recognition from an image captured by a certain image capturing means, and even if the recognition target cannot be recognized by a camera equipped with the image capturing means, the recognition target can be detected by another image capturing means. It becomes possible to recognize a recognition target with a camera provided with means. As a result, it is possible to easily know that the recognition target could not be recognized, and machine learning can be performed using the image that could not be recognized as teacher data, so that detection/recognition can be performed. Machine learning can be performed so that it can recognize even from difficult images.

According to the present invention, the detection/recognition performance of detecting a feature included in an image and recognizing a recognition target set from this feature can be improved by updating the firmware for detection/recognition.

BRIEF DESCRIPTION OF THE DRAWINGS It is a block diagram which shows embodiment of this invention and which shows a detection recognition system. It is a block diagram showing a camera of the detection recognition system. It is a block diagram which shows the server of a detection recognition system same. 10 is a flowchart for explaining a method of updating detection recognition firmware by the detection recognition system.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings.
The detection/recognition system of this embodiment is used, for example, to notify when a recognition target set from an image captured by a camera is recognized.
In the following description, simply referring to an image basically includes both a moving image and a still image.

The detection recognition system 1 includes a plurality of cameras 2, a server 3, and a terminal 4, as shown in FIG. Also, the plurality of cameras 2, the server 3, and the terminal 4 are connected by a wired or wireless network 5, respectively.

For example, the detection recognition system 1 is installed in a building such as a convenience store or outdoors as a surveillance camera, and recognizes an object photographed by the camera 2 as a suspicious person based on its appearance and movement. In addition, it can be used to notify a terminal 4 at another location. Further, when a suspicious person is recognized in this way, the terminal or system management device owned by the administrator 6 of the detection and recognition system 1 may be informed.

The camera 2 includes an imaging means 20, a detection/recognition means 21, a recording means 22, a communication means 23, and a control means 24, as shown in FIG.
The imaging means 20 has, for example, a lens and a solid-state imaging device, and acquires an image by imaging. Also, the detection/recognition means 21 includes an arithmetic processing unit and a memory, and performs image recognition. Specifically, the detection/recognition firmware provided in the memory of the detection/recognition means 21 detects features included in the image captured by the imaging means 20, and recognizes the set recognition target based on these features. do. It should be noted that hereinafter, when simply referred to as "detection/recognition", basically, it means detecting a feature included in an image captured by the imaging means 20 and recognizing a set recognition target based on this feature. Say.

The recording means 22 also stores reference images and other information for detection/recognition by the detection/recognition means 21, and images at the time of abnormality (for example, when the detection/recognition means 21 recognizes a set recognition target). Other information (eg, voice, etc.) is recorded. Further, the communication means 23 communicates with the server 3 via the network 5 to transmit images and other information at the time of abnormality to the server 3 and receive commands and detection recognition firmware from the server 3 . The communication means 23 is also connected to the terminal 4 and a terminal owned by the administrator 6 via the network 5, and transmits an alarm signal or the like to these terminals and the server 3 in the event of an abnormality. Also, the terminal 4 or a terminal held by the manager 6 receives this alarm signal or sounds an alarm in response to an instruction to sound an alarm from the server 3 that has received the alarm signal.

Also, the control means 24 has an arithmetic processing unit and a memory, and controls the imaging means 20 , the detection/recognition means 21 , the recording means 22 and the communication means 23 . Note that the control means 24 may share an arithmetic processing unit or memory with the detection/recognition means 21 .

Note that the camera 2 does not have to include all of the imaging means 20 , the detection/recognition means 21 , the recording means 22 , the communication means 23 and the control means 24 . For example, the detection and recognition system 1 is connected to the camera 2 by wire or wirelessly, and has a terminal outside the camera 2 that can control the camera 2 and display images taken by the camera 2. The terminal is provided with detection/recognition means 21, recording means 22, communication means 23, and control means 24, and the terminal detects/recognizes an image taken by an imaging means 20 provided in the camera 2. You may do so.

The camera 2 has, for example, a configuration similar to that of a general monitoring camera. The cameras 2 of the detection and recognition system 1 may be of the same type or may be of different types. Also, the imaging ranges of the respective cameras 2 may overlap or may be completely different.
In this embodiment, as the cameras 2, a total of four cameras 2 of different types are used: two stereo cameras 2a, one infrared camera 2b, and one monocular camera 2c. , and the imaging ranges of the four cameras 2 overlap each other.
By using a stereo camera 2a capable of calculating distance, size, 3D structure, etc. from parallax as camera 2, it is possible to calculate distance, size, 3D structure, etc. from parallax. Therefore, detection/recognition can be easily performed even if the camera does not have a high-performance processor or the like.

In addition, by using an infrared camera (near-infrared camera or far-infrared camera) 2b as the camera 2, it is possible to capture near-infrared or far-infrared images, and detect things that cannot be seen by the human eye. / can be recognized. It also facilitates detection/recognition in a dark environment such as at night.

Also, the types of cameras 2 are not limited to these. For example, a distance image sensor may be used as the camera 2 . For example, a TOF (Time Of Flight) can be used as the distance image sensor. TOF measures distance from the time it takes a projected laser to make a round trip to the target.

In other words, the camera 2 may be one in which the image pickup means 20 picks up one two-dimensional image and performs detection/recognition from this image. The detection/recognition may be performed by calculating the distance, size, 3D structure, etc., or the imaging means 20 may capture a 3D distance image using a TOF sensor or the like, and the detection/recognition may be performed from this 3D distance image. Alternatively, the imaging means 20 may capture near-infrared or far-infrared images, and detection/recognition may be performed from these images. Moreover, one camera 2 may be provided with a plurality of the imaging means 20 described above. That is, one camera 2 may have imaging functions of, for example, a stereo camera and an infrared camera, and detection/recognition may be performed from images obtained by these functions.

The detection/recognition means 21 recognizes a set recognition target, and the recognition target may be a concrete object (including humans and objects other than humans) or an abstract phenomenon. Conceivable. In other words, the recognition target may be a person such as a robber, a thief, or an arsonist, or an object such as a handgun, or a phenomenon such as a crime or fire.
For example, in a state where a robbery is set as a recognition target, when an image of a person holding a kitchen knife or a pistol is captured by the imaging means 20 of the camera 2 installed in the convenience store, the detection/recognition means It is conceivable that 21 detects a person holding a kitchen knife or a pistol from this image, or detects the person's movement, and recognizes this person as a robbery. Also, for example, in a state where a fire is set as a recognition target, it is possible to detect that the temperature in a certain place is abnormally high from the image obtained by the infrared camera, and to recognize that a fire is occurring. can be considered. Also, for example, if the infrared camera uses far infrared rays, it can detect the temperature. It is also conceivable to detect weapons by image recognition. However, since the detection/recognition firmware of the detection/recognition means 21 is generated by machine learning in the machine learning means 30 described later, the detection/recognition means 21 is actually easy for such people to understand ( comprehensible) way of recognition.
In other words, the detection/recognition means 21 detects features included in the image captured by the imaging means 20 under the control of the detection/recognition firmware, and recognizes the set recognition target based on these features.
Note that the detection/recognition means 21 may perform detection/recognition using not only images but also sounds. For example, the camera 2 has an audio input means such as a microphone, and the accuracy of detection/recognition can be improved by performing detection/recognition using audio acquired by this audio input means. Also, the detection/recognition by the server-side detection/recognition means 32, which will be described later, may similarly use voice.

The detection/recognition firmware of the detection/recognition means 21 is updated by new detection/recognition firmware generated by the machine learning means 30 and the detection/recognition firmware generation means 31, which will be described later. The detection/recognition firmware initially provided in the detection/recognition means 21 may be generated by the machine learning means 30 and the detection/recognition firmware generation means 31, or may be generated by other equipment capable of machine learning. / It may be one incorporated in the recognition means 21 . Alternatively, detection/recognition firmware generated by a method other than machine learning may be provided in the detection/recognition means 21 first.

It is also assumed that the detection/recognition firmware includes settings for objects to be recognized by the detection/recognition means 21 . For example, in the case where the detection/recognition firmware is generated by the machine learning means 30 and the detection/recognition firmware generation means 31, if the object to be recognized is a robbery at a convenience store, the teacher data for machine learning may be, for example, a convenience store Provide a machine learning means 30 with a plurality of images showing a robbery who committed a robbery and information that these images are images showing the robbery as training data (tag the image with the robbery). . Then, machine learning learns where to focus in the given image (teaching data) to recognize the robbery. Machine learning then produces a detection/recognition algorithm with a high probability of recognizing the burglar from the image. Then, this detection/recognition algorithm is converted by the detection/recognition firmware generating means 31 to generate detection/recognition firmware. In other words, the detection/recognition firmware (detection/recognition algorithm) obtained through this learning can recognize whether or not a robbery is included in an image by focusing on where in the image. , it can be said that the robbery is set. Note that it is not always necessary to tag images when performing this machine learning. For example, if only an image of a robbery is given as training data, an image with features similar to those given as training data will be recognized even if there is no information that the image shows the robbery. By creating an algorithm, it is possible to create an algorithm that recognizes robbers.
Note that the number of recognition targets (targets recognized by the detection/recognition firmware) set in the detection/recognition firmware is not limited to one, and a plurality of targets may be set.

As described above, the detection/recognition firmware recognizes a specific target, and when the detection/recognition firmware recognizes the specific target, the detection/recognition means 21 sends a signal or the like indicating recognition. (for example, an alarm signal). A signal or the like indicating that the setting object has been recognized is sent to the server 3, the terminal 4, the terminal owned by the administrator 6, or the like via the communication means 23, and the terminal or the like is notified that the setting object has been recognized. be. A signal or the like indicating that this recognition has been performed is sent only to the server 3, and after comprehensively judging the information from each camera 2 in the server 3, the recognition target is sent from the server 3 to the terminal 4 or the like. An e-mail to that effect or alarm information such as an instruction to sound an alarm may be sent.

In addition, the imaging ranges of the four cameras 2 overlap with each other, and it is possible for the four cameras 2 to simultaneously recognize overlapping portions of the recognition targets set in the detection/recognition firmware. ing. In other words, if, for example, a robbery is set as an overlapping recognition target, four cameras can simultaneously recognize a specific robbery who commits a specific robbery.

The server 3, as shown in FIG. A side control means 35 is provided. The machine learning means 30, the detection/recognition firmware generation means 31, the server-side detection/recognition means 32, and the server-side control means 35 each have an arithmetic processing unit and a memory. may have one or may share a processor or memory.

Machine learning means 30 performs machine learning such as deep learning to generate a detection/recognition algorithm. Here, the detection/recognition algorithm is an algorithm for recognizing a set recognition target from an image captured by the imaging means 20 of the camera 2 .

The detection/recognition firmware generating means 31 converts the detection/recognition algorithm generated by the machine learning means 30 into firmware executable by each camera 2, and generates detection/recognition firmware. Each camera 2 has the resolution of an image that can be acquired by the imaging means 20, the performance of the arithmetic processing unit of the detection / recognition means 21, the presence or absence of a GPU (Graphics Processing Unit) for the detection / recognition means 21, and the voice input means such as a microphone. , the type of camera (stereo camera, TOF sensor, etc.), etc., the firmware that can be executed by each camera 2 is also different. A detection/recognition firmware generating means 31 converts a detection/recognition algorithm generated by machine learning into firmware that can be executed by each camera 2, enabling each camera 2 to implement a new detection/recognition program. becomes.

The server-side detection/recognition means 32 performs detection/recognition by comprehensively judging the situation from the images and information of each camera 2 . For example, the detection/recognition means 21 of each camera 2 performs detection/recognition using the image acquired by the imaging means 20 of that camera 2, while the server-side detection/recognition means 32 acquires images with a plurality of cameras 2. Detection/recognition is performed using the captured image. If the processing is too heavy for each camera 2 to perform, the server-side detection/recognition means 32 may perform part of the processing. Detection/recognition firmware for the server-side detection/recognition means 32 is stored in the memory of the server-side detection/recognition means 32 . The detection/recognition firmware of the server-side detection/recognition means 32 can also be updated by the detection/recognition firmware generated by the machine learning means 30 and detection/recognition firmware generation means 31 .

In addition, the server-side detection/recognition means 32 determines whether the recognition of the recognition target in each camera 2 is correct, or The probability that the recognition of the recognition target in each camera 2 is correct may be determined. Then, based on this determination result, alarm information or the like may be sent to the terminal 4 or the like. For example, when there is a notification that a setting target (for example, a robbery) has been recognized from all four cameras, the server-side detection/recognition means 32 determines that the recognition of the setting target is correct, and an alarm is sent to the terminal 4 or the like. may be ordered to ring. Also, the content of the alarm information may be changed depending on the number of cameras that recognize the setting target. For example, when all four cameras recognize the setting target, it is determined that the recognition is correct, and the server-side detection/recognition means 32 commands the terminal 4 to sound a loud alarm sound. If only the following cameras recognize the setting target, it is determined that the recognition may be correct, and the server-side detection/recognition means 32 instructs the terminal 4 to sound a small alarm sound. can be

Further, the server-side detection/recognition means 32 determines erroneous recognition (recognition error) of the camera 2 from the recognition results of the detection/recognition means 21 of the plurality of cameras 2 . For example, out of four cameras 2, if three cameras 2 notify that the detection/recognition means 21 has recognized a setting target, and one camera 2 does not notify, this one camera The camera 2 of 1 judges that erroneous recognition (recognition error) has occurred. Conversely, when three cameras 2 out of four cameras 2 did not notify that the detection/recognition means 21 recognized the setting target, but one camera 2 did. Alternatively, it may be determined that this one camera 2 made an erroneous recognition (recognition error).
Even if the result of detection/recognition by the server-side detection/recognition means 32 and the result of detection/recognition by each camera 2 are compared to determine misrecognition (recognition error) of each camera 2, good.

The server-side recording means 33 records teacher data and the like for machine learning performed by the machine learning means 30 . In addition, the server-side communication means 34 communicates with each camera 2 via the network 5, receives images and other information from each camera 2, transmits commands and detection/recognition firmware to each camera 2, (when the setting target is recognized) is transmitted to the terminal 4 and the administrator 6.

Next, a method for updating the detection recognition firmware of the detection recognition system 1 will be described with reference to the flowchart of FIG.
The camera 2 acquires an image by the imaging means 20 and performs recognition (detection/recognition) of a recognition target set by control by the detection/recognition firmware of the detection/recognition means 21 . Then, when the recognition is erroneous, the image at the time of the erroneous recognition is transmitted to the server 3 (step S1). It should be noted that audio data or the like at the time of the recognition error may be transmitted together with the image at the time of the recognition error.
It should be noted that the server-side detection/recognition means 32 determines whether or not recognition is erroneous based on the recognition results of the plurality of cameras 2, as described above. For example, in a system that notifies the server 3 of recognition (for example, recognition of a robbery) when the camera 2 recognizes a set recognition target (for example, a robbery), cameras 2a and 2b If the server 3 receives a notification of recognition, but does not receive a notification of recognition from the camera 2c, the server-side detection/recognition means 32 recognizes the camera 2c incorrectly based on these notification results ( I could not recognize it). At this time, the control means of the server 3 instructs the camera 2c at the same time as the time when the camera 2a and the camera 2b acquire the image in which the recognition target is recognized, or at a time before or after this time (for example, several seconds to several minutes before or after). The camera 2c is instructed to transmit the image acquired by the camera 2c to the server 3 as the image obtained when the recognition is erroneous. In response to this command, the camera 2c transmits to the server 3 the image of the time when the recognition was erroneous.
It should be noted that a person may judge whether or not the recognition is erroneous. For example, the detection/recognition system 1 includes display means for displaying an image taken by the camera 2 and a terminal equipped with input means such as a pointing device and a keyboard. If not, the person checks the image taken by the camera 2 from the display means of this terminal, selects the image that the person wants the robbery to be recognized using the input means of this terminal, and recognizes it. It may be transmitted to the server 3 as an image at the time of error.

The server-side control means 35 records the image sent from the camera 2 when the recognition is erroneous in the server-side recording means 33 as teacher data (education data). In addition to the recording of the image when the recognition is erroneous, the server-side recording means 33 stores the recognition result (for example, the request that the detection/recognition means 21 should recognize the robbery from the image) as teaching data. Record.
Note that the recognition result desired to be output by the detection/recognition means 21 to be recorded as teacher data may be generated by the server 3 or may be sent from the camera 2 . For example, when the server-side detection/recognition means 32 determines whether or not recognition is erroneous based on the recognition results of a plurality of cameras 2, the server-side detection/recognition means 32 determines the likely correct recognition results (detection/recognition means 21) is created as teacher data, and this teacher data may be recorded in the server-side recording means 33. FIG. Further, for example, when a person checks an image taken by the camera 2 and determines whether or not the recognition is incorrect, when the person selects an image from the above-mentioned terminal that the person wants to recognize the robbery, A request to recognize the robbery (that the image shows the robbery) is also input using the input means of this terminal, and is transmitted to the server 3 together with the image when the recognition is erroneous. The obtained data may be recorded in the server-side recording means 33 by the server-side control means 35 as teacher data.

The machine learning means 30 reads the teacher data recorded in the server-side recording means 33 (step S2). Then, the machine learning means 30 extracts feature points from the misrecognized image contained in the read teacher data by convolution operation (step S3). The machine learning means 30 performs machine learning based on the information of the extracted feature points and the recognition results that the detection/recognition means 21 wanted to output (step S4). As a result of machine learning, a detection/recognition algorithm, which is a neural network for performing detection/recognition processing, is generated (step S5).

Machine learning by the machine learning means 30 is performed so that the detection/recognition algorithm (detection/recognition firmware) is optimized for each camera 2 . Each camera 2 may be of a different type, or even cameras with exactly the same characteristics may differ in installation location or usage environment. Because it may come. Based on the original detection/recognition algorithm and the teacher data, the machine learning means 30 recognizes the recognition desired by the detection/recognition means 21 contained in the teacher data from the image in which the recognition was erroneously included in the teacher data. Generate new detection/recognition algorithms that can produce results. Note that the original detection/recognition algorithm used for machine learning may be recorded in the server-side recording means 33, and detection/recognition firmware is transmitted from the camera 2, and this detection/recognition firmware is detected/recognised. You may convert it into an algorithm and use it. In other words, the machine learning means 30 generates a new detection/recognition algorithm from the detection/recognition algorithm used in the detection/recognition firmware of the camera 2 that made a detection/recognition error and the teacher data.

The detection/recognition firmware generating means 31 converts the detection/recognition algorithm generated by the machine learning means 30 into detection/recognition firmware, which is detection/recognition software for each camera (step S6). That is, the detection/recognition algorithm is converted by the detection/recognition firmware generating means 31 into software in a format executable by each camera.
The server-side communication means 34 transmits the detection/recognition firmware, which is the detection/recognition software generated by the detection/recognition firmware generating means 31, to the camera 2 (step S7). Then, when the camera 2 receives the detection/recognition firmware, the control means 24 of the camera 2 updates the firmware of the detection/recognition means 21 to new detection/recognition firmware.

According to the detection/recognition system of the present embodiment, the detection/recognition firmware of the detection/recognition means 21 of the camera 2 is new detection/recognition firmware generated by the machine learning means 30 and the detection/recognition firmware generation means 31 of the server 3. can be updated to
The machine learning by the machine learning means 30 is performed using an image when the detection/recognition means 21 of the camera 2 misrecognizes the set recognition target as teacher data. Therefore, in machine learning using this teacher data, The detection/recognition algorithm is improved so as not to misrecognise the recognition target set for the image. Therefore, the detection/recognition performance of the camera 2 can be improved.

Machine learning is performed by the server 3, and the camera 2 only needs to execute the detection recognition firmware generated by the server 3. Therefore, even if the computing power of the camera 2 is not so high, the detection recognition firmware can be updated. can be used to perform highly accurate detection/recognition. Also, as the camera ages, the performance of the camera does not become relatively lower than that of other cameras. Also, the performance of the camera 2 can be improved so that detection/recognition suitable for the environment in which the camera 2 is used can be performed.

In addition, by allowing the machine to learn by itself, it becomes possible to recognize the set recognition target even when a human cannot notice it. For example, as educational data, instead of giving an image of the robber actually doing the robbery, giving an image of the robber before the actual robbery was given. , rather than an algorithm that recognizes robbers when a robbery is actually taking place, the behavior of people loitering in and around convenience stores is used to identify people who are likely to commit robberies in the future. It would also be possible to generate a detection/recognition algorithm that would find the features and recognize such a person as a burglar (a likely burglar). It should be noted that the machine learning means 30 determines what features are actually focused on for recognition, so it is not always possible to recognize a person who is highly likely to commit robbery based on behavior.

Further, according to the detection/recognition system of the present embodiment, the imaging ranges of the four cameras 2 overlap each other. They can be simultaneously recognized by the camera 2 on the stand. Therefore, even if some of the four cameras 2 cannot recognize the recognition target, other cameras out of the four cameras 2 can recognize the recognition target. / It is possible to increase the possibility of being able to recognize, and to improve the accuracy of detection / recognition as the whole system.
The four cameras 2 include cameras with different types of imaging means 20, such as a stereo camera 2a, an infrared camera 2b, and a monocular camera 2c. Therefore, for example, even if it is difficult to detect/recognize with the stereo camera 2a, the infrared camera 2b can detect/recognize. It is possible to improve the accuracy of detection/recognition as.
Note that the plurality of cameras 2 may be installed in locations with completely different imaging ranges, or may recognize completely different recognition targets.

Further, the server-side detection/recognition means 32 judges whether the recognition of the recognition target in each camera 2 is correct from the recognition results of the detection/recognition means 21 of the four cameras 2, and determines whether the recognition target in each camera 2 is correct. It is possible to determine the probability that the recognition of the camera 2 is correct, or to determine the erroneous recognition (recognition error) of the camera 2 . Therefore, only when the server-side detection/recognition means 32 determines that the recognition is correct based on the detection/recognition results of the individual cameras 2, the terminal 4 or the like can sound an alarm.
In addition, it is possible to automatically determine erroneous recognition of the camera 2 and automatically perform machine learning so as to improve the detection/recognition capability of the erroneously recognized camera 2 . At this time, the image used as teacher data for machine learning can be the image at the time of erroneous recognition. becomes possible. Therefore, erroneous recognition can be determined automatically, and the accuracy of detection/recognition can be improved as the camera 2 is used.

Note that the timing of performing machine learning may be adjusted as appropriate. For example, training data may be stored in the recording means 22 or the server-side recording means 33, and machine learning may be performed when a certain number or more of training data are accumulated or after a certain period of time has elapsed.
Also, machine learning may be performed using images other than the images captured by the imaging means 20 . If only the images captured by the imaging means 20 are insufficient in the number and quality of teacher data, other images can be given to the machine learning means 30 to improve the effect of machine learning.

Note that the object to be recognized by the camera 2 is not limited to those described above, and may be anything as long as it can be detected/recognized from the image captured by the imaging means 20 .

1 detection recognition system 2 camera 2a stereo camera (camera)
2b infrared camera (camera)
2c monocular camera (camera)
3 server 20 imaging means 21 detection/recognition means 30 machine learning means 31 recognition firmware generation means

Claims (6)

comprising a plurality of cameras and a server,
The camera comprises imaging means for imaging and detection/recognition means,
the server comprises server-side detection/recognition means, machine learning means, and detection/recognition firmware generation means;
The detection/recognition means includes detection/recognition firmware, and detects features included in the image from the image acquired by the imaging means under the control of the detection/recognition firmware, and recognizes a set recognition target. , the detection/recognition firmware can be updated to new detection/recognition firmware generated by the detection/recognition firmware generating means;
The server-side detection/recognition means includes server-side detection/recognition firmware, and detects features included in the image from the image acquired by the imaging means under the control of the server-side detection/recognition firmware, and sets The recognition target can be recognized and the server-side detection/recognition firmware can be updated to new server-side detection/recognition firmware generated by the detection/recognition firmware generating means,
The machine learning means performs a process of generating a detection/recognition algorithm by machine learning using the image acquired by the imaging means as teacher data,
The detection/recognition firmware generation means generates new detection/recognition firmware for the detection/recognition means from the detection/recognition algorithm, and a new server for the server-side detection/recognition means from the detection/recognition algorithm. performing a process of generating side detection/recognition firmware;
at least a part of the imaging range of the imaging means and the set recognition targets of the plurality of cameras overlap;
Based on the recognition of the recognition target by the server-side detection/recognition means, the server transmits to the terminal information for notifying the user of the terminal that the recognition target has been recognized. A detection recognition system characterized by outputting . 2. The detection/recognition system according to claim 1, wherein said server-side detection/recognition means judges whether recognition of a recognition target by said detection/recognition means is correct or wrong. at least one of the plurality of cameras includes audio input means;
3. The detection/recognition means or the server-side detection/recognition means according to claim 1 or 2, which recognizes a set recognition target using the voice obtained by the voice input means. detection recognition system. The machine learning means, when some cameras among the plurality of cameras recognize overlapping recognition targets, other cameras among the plurality of cameras do not recognize the overlapping recognition targets. 4. The detection and recognition system according to any one of claims 1 to 3, wherein machine learning is performed using the image obtained as teacher data. comprising a plurality of cameras and a server,
The camera comprises imaging means for imaging and detection/recognition means,
the server comprises machine learning means and detection/recognition firmware generation means;
The detection/recognition means includes detection/recognition firmware, and detects features included in the image from the image acquired by the imaging means under the control of the detection/recognition firmware, and recognizes a set recognition target. , the detection/recognition firmware can be updated to new detection/recognition firmware generated by the detection/recognition firmware generating means;
The machine learning means performs a process of generating a detection/recognition algorithm by machine learning using the image acquired by the imaging means as teacher data,
The detection/recognition firmware generation means executes processing for generating new detection/recognition firmware for the detection/recognition means from the detection/recognition algorithm,
at least a part of the imaging range of the imaging means and the set recognition targets of the plurality of cameras overlap;
The machine learning means, when some cameras among the plurality of cameras recognize overlapping recognition targets, other cameras among the plurality of cameras do not recognize the overlapping recognition targets. A detection and recognition system characterized by performing machine learning using images as training data. 6. The detection and recognition system according to any one of claims 1 to 5, wherein at least one of said plurality of cameras is a camera having a different imaging means.

Images