KR101789690B1 - System and method for providing security service based on deep learning - Google Patents

System and method for providing security service based on deep learning Download PDF

Info

Publication number
KR101789690B1
KR101789690B1 KR1020170087862A KR20170087862A KR101789690B1 KR 101789690 B1 KR101789690 B1 KR 101789690B1 KR 1020170087862 A KR1020170087862 A KR 1020170087862A KR 20170087862 A KR20170087862 A KR 20170087862A KR 101789690 B1 KR101789690 B1 KR 101789690B1
Authority
KR
South Korea
Prior art keywords
image
deep learning
monitored
information
learning server
Prior art date
Application number
KR1020170087862A
Other languages
Korean (ko)
Inventor
이동식
이원경
Original Assignee
(주)블루비스
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by (주)블루비스 filed Critical (주)블루비스
Priority to KR1020170087862A priority Critical patent/KR101789690B1/en
Application granted granted Critical
Publication of KR101789690B1 publication Critical patent/KR101789690B1/en

Links

Images

Classifications

    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00Burglar, theft or intruder alarms
    • G08B13/18Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength
    • G08B13/189Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19654Details concerning communication with a camera
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/40Image enhancement or restoration by the use of histogram techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/136Segmentation; Edge detection involving thresholding
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/60Analysis of geometric attributes
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00Burglar, theft or intruder alarms
    • G08B13/18Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength
    • G08B13/189Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19602Image analysis to detect motion of the intruder, e.g. by frame subtraction

Abstract

More particularly, the present invention relates to a system and method for providing a deep learning-based security service, and more particularly, to a system and method for providing a deep learning-based security service by analyzing a monitoring target object appearing in an image captured through a camera, It is possible to improve the accuracy of identification of objects and thereby reduce the rate of false alarms caused by objects other than the monitored object. At the same time, among the plurality of images constituting the image, the monitored object repeatedly learns the image characteristics of the identified image, The present invention relates to a deep learning-based security service providing system and method for assuring reliable and accurate identification of an object to be monitored by assuring that an image of an object to be monitored is provided so that omissions of a monitoring object detected by the sensor do not occur.

Description

[0001] SYSTEM AND METHOD FOR PROVIDING DEEP LEUNNING-BASED SECURITY SERVICE [0002]

More particularly, the present invention relates to a system and method for providing a deep learning-based security service, and more particularly, to a system and method for providing a deep learning-based security service by analyzing a monitoring target object appearing in an image captured through a camera, It is possible to improve the accuracy of identification of objects and thereby reduce the rate of false alarms caused by objects other than the monitored object. At the same time, among the plurality of images constituting the image, the monitored object repeatedly learns the image characteristics of the identified image, The present invention relates to a deep learning-based security service providing system and method for assuring reliable and accurate identification of an object to be monitored by assuring that an image of an object to be monitored is provided so that omissions of a monitoring object detected by the sensor do not occur.

In addition to the development of communication and image analysis technologies, various security systems are provided to provide security for the target area through image analysis and object identification of images received from cameras located at remote locations Through this, it provides various services such as notifying the manager of the danger when the object of the monitoring target area is detected and recording the image according to the object detection, thereby enhancing the convenience of management of the monitoring target area.

Generally, a security system detects a surveillance object located in a surveillance target area through a sensor located in a surveillance target area, and when the surveillance target is sensed, So that the surveillance object detected through the sensor can be confirmed through the image.

However, when an intrusion signal is generated according to the detection of a sensor of the existing security system, an alarm is provided to the administrator separately from the confirmation through the image. Thus, an alarm is provided to the object not to be monitored, There is a problem in that it occurs continuously.

In order to prevent this, in recent security systems, when an intrusion signal is generated through a sensor, an alarm is provided to an administrator when a monitored object corresponding to the monitored object is identified in an image photographed through the camera We want to minimize the misinformation of the security system.

However, since the monitoring target area generally has a low-light environment, a recent security system can not detect the object of the sensor object because the object is not clearly displayed in the image generated through the camera operated in the low- There is a problem that the obligation rate can not be greatly improved due to the frequent occurrence of the alarm being provided or the monitoring object being missed because the object is not identified through the image even when the monitored object appears.

Due to the above-described problem, there is a problem that the reliability and accuracy of the intrusion detection report for the monitoring target area of the security system is lowered.

Korean Patent No. 10-1002712

The object of the present invention is to provide a method and apparatus for analyzing a surveillance target object appearing in an image photographed by a camera at a point of time when a sensing signal is sensed according to sensing by a depth learning method, The monitoring object is repeatedly learned from the image characteristics of the identified image among the plurality of images constituting the image so that an optimal image capable of identifying the monitored object is provided, And to ensure the reliability and accuracy of the security of the monitoring target area by identifying the target to be monitored.

The object of the present invention is to repeatedly learn an image identified by the object to be monitored and to provide an optimal image capable of identifying the object to be monitored by adapting to the environmental condition of the object to be monitored, And to increase the reliability of security in the target area.

The system for providing a deep learning-based security service according to an embodiment of the present invention includes a sensing management device that transmits sensing signals received from a sensor unit that senses an object to be sensed and transmits a sensing signal to a predetermined signal server, A signal server for transmitting the moving image information generated by the imaging unit to the deep running server when receiving the sensing signal from the sensing management apparatus; An image analysis apparatus for extracting and transmitting an image corresponding to the image request among a plurality of images constituting the image, and an image analyzing apparatus for requesting an image from the image analyzing apparatus upon receiving the sensing signal from the signal server, Deep learning method is used to analyze the pre- The method further comprises the steps of: generating event information when the monitoring object is not identified; re-requesting the image analyzing apparatus for another image related to the reception time of the sensing signal when the monitoring object is not identified; And a depth learning server for transmitting result information on the identified normal image to the image analyzing apparatus, wherein the image analyzing apparatus is configured to determine, when the image re-request of the deep learning server is made, And transmits the optimal image to the deep learning server in accordance with a predetermined selection criterion that belongs to a predetermined time range and that has one or more other images in which the object is detected, and updates the selection criterion according to the result information .

In one embodiment of the present invention, the deep learning server transmits the event information to a predetermined control server through the signal server when generating the event information.

As an example related to the present invention, the image analysis apparatus may be configured as a DVR or an NVR.

According to an embodiment of the present invention, the deep learning server may further include, when the monitored object is identified in at least one of the at least one optimal image received according to the image re-request, Wherein the image analysis apparatus transmits result information including image identification information to the image analysis apparatus, and the image analysis apparatus extracts image features through image analysis of the normal image corresponding to the result information, Attribute of each attribute constituting the selection criterion or assigns a predetermined weight to any one of the attributes.

As an example related to the present invention, the attribute includes at least one of a size of an object position on an image, a boundary or an outline of the object, an illuminance or a color difference between the object and the background, and a histogram distribution variation .

In one embodiment of the present invention, the deep learning server analyzes the image received from the image analysis apparatus through a deep learning-based neural network, identifies the monitored object, and stores the normal image And comparing the similarity between the specific image and the at least one normal image when a specific object other than the monitored object is identified in the process of identifying the monitored object in the specific image received from the image analysis apparatus, And generating error information on an error between the output value of the image processing based on the neural network and the object information with respect to the specific image when the similarity degree is equal to or greater than a preset reference value, Based on the error information, And adjusting the parameters constituting the network.

According to an embodiment of the present invention, the deep learning server calculates a weight of connection strength between the input layer, the at least one hidden layer and the output layer constituting the neural network through the back propagation algorithm based on the error information, The bias of the unit configured in the input layer, the hidden layer, and the output layer may be varied to learn that the identification error of the monitored object is minimized.

A deep learning server for receiving a sensing signal of a sensor unit according to an embodiment of the present invention and a deep learning based security service for an image analysis apparatus communicating with the deep learning server, The method includes the steps of transmitting, to the image analysis apparatus, request information for requesting an image corresponding to a reception time of the sensing signal when the deep learning server receives the sensing signal, Extracting an image corresponding to the request information from the moving image information, and transmitting the extracted image to the deep learning server; and the deep learning server analyzes the image received from the image analysis apparatus by a deep learning method, And generates event information when it is identified, and when the monitoring object is not identified, The method comprising the steps of: transmitting re-request information for re-requesting another image related to the reception time of the sensing signal to the analyzing device; and transmitting the re- Analyzing the at least one other image belonging to the time range within a predetermined time range according to a preset selection criterion and transmitting the optimum image to the deep learning server.

In one embodiment of the present invention, the deep learning server generates result information including image identification information on the best image when identifying a monitored object from the best image, Analyzing an image corresponding to the result information when the image analysis apparatus receives the result information, extracting an image feature, and updating the selection criterion based on the image feature .

The deep learning-based security service providing system according to the present invention not only enhances the accuracy of the monitoring object object in a still image (image) through deep learning-based learning, but also enhances the image characteristic extracted from the identified image It is possible to provide an optimal image that can be easily identified in the object detection process of the moving image provided by the camera unit according to the environmental characteristics of the monitored area by performing iterative learning based on the environment characteristic of the monitored area, It is possible to increase the accuracy of identification of objects and improve the accuracy of identification of objects by extracting optimal images that are easy to identify objects in a moving image. Further, when the object is detected through the sensor, Helps ensure that objects are accurately identified to prevent misleading It is possible to prevent a report on the monitored object sensed by the sensor from being missed, thereby greatly enhancing the reliability and accuracy of the security service.

In addition, the present invention not only improves the accuracy of identification of a monitored object through iterative learning of a monitored object, but also learns a feature change of a monitored object appearing in a moving image or an image reflecting environmental characteristics of the monitored area Based on this, based on the characteristics of the operation of the security system, it is possible to optimize the deep learning algorithm according to the characteristics of the object to be observed in the monitored area so as to accurately identify the object to be photographed through a camera operated in a low- So that it is possible to accurately distinguish objects other than the object to be monitored, and thereby to prevent an object from being misunderstood as an object to be monitored.

1 and 2 are a configuration diagram and a basic operation example of a deep learning-based security service providing system according to an embodiment of the present invention;
3 is a diagram illustrating an operation of object detection and image provisioning of an image analysis apparatus constituting a deep learning-based security service providing system according to an embodiment of the present invention;
FIG. 4 and FIG. 5 are diagrams illustrating an operation example of a learning process for increasing the accuracy of identification of a monitored object in a deep learning-based security service providing system according to an embodiment of the present invention;
6 is a flowchart illustrating a method of providing a deep learning-based security service according to an embodiment of the present invention;

Hereinafter, detailed embodiments of the present invention will be described with reference to the drawings.

1 and 2 are block diagrams of a system for providing a deep learning-based security service according to an embodiment of the present invention. As shown in the figure, the sensor unit 10 includes various sensors, A sensing management device 20 that receives a sensing signal of the sensor unit 10 and a sensing unit 20 that is connected to the sensing management device 20 and receives a sensing signal of the sensor unit 10 from the sensing management device 20 An image analyzer 70 connected to the camera unit 70 and performing image analysis on an image generated through the camera unit 70, And a deep learning server 50 connected to the signal server 30 to receive the sensing signal and to be connected to the image analysis device 60.

At this time, the connection between the components constituting the deep learning-based security service providing system may be connected through a wired / wireless communication network, and various widely known communication methods may be applied to the wired / wireless communication network.

The sensing management device 20 and the signal server 30 may be constituted by one component. For example, the sensing management device 20 may be included in the signal server 30, 30 may be included in the detection management device 20.

In addition, the image analysis apparatus 60 may include a DVR (Digital Video Recorder) or an NVR (Network Video Recorder).

The sensor unit 10 may include various sensors such as an infrared sensor, a human body sensor, a magnetic sensor, and the like, which are located at various places in the monitored area. In addition, And transmit the signal to the detection management device 20 through a communication network.

In addition, the sensing management device 20 receiving the sensing signal from the sensor unit 10 may transmit the sensing signal to the signal server 30 through the communication network.

In addition, the signal server 30 may transmit the sensed signal to the predetermined deep learning server 50 through a communication network.

Meanwhile, the camera unit 70 may transmit the moving image information generated by photographing the monitoring target area to the image analysis apparatus 60. [0033] FIG.

At this time, the camera unit 70 may include various kinds of cameras such as a visible light camera, an infrared camera, a depth sensing camera, and the like, and may be composed of one or more cameras located at different places in a monitored area.

In addition, the image analyzing apparatus 60 analyzes the moving image information according to a predetermined image analysis algorithm when the moving image information is received from the camera unit 70, and detects an object to be monitored located in the monitoring area can do.

At this time, the image analyzing apparatus 60 may store the moving image information received from the camera unit 70 in a storage unit configured in the image analyzing apparatus 60.

In order to identify an object corresponding to the object to be monitored when the sensing signal is sensed by the image analysis device 60 upon receiving a sensing signal from the sensing management device 20, And transmit request information for requesting an image corresponding to the sensing signal to the controller 60.

Accordingly, when receiving the request information for the image request of the deep learning server 50, the image analyzing apparatus 60 may receive the sensing information from the plurality of images constituting the moving image information, The corresponding image can be extracted from the moving image information and transmitted to the deep learning server 50.

In addition, the image analyzer 60 may analyze the at least one image belonging to the predetermined time range based on the reception time of the sensing signal among the plurality of images constituting the moving image information upon receiving the request information, And may transmit the extracted one or more images to the deep learning server 50 by extracting the one or more images from the moving image information.

At this time, the image analyzer 60 separates the foreground and the background in the image analysis of the plurality of images constituting the moving image information, detects the moving object moving in the foreground, and performs the deep running To the server (50).

In addition, the image analysis apparatus 60 may include an image analysis algorithm such as a differential image method, a model of Gaussian (MOG) algorithm using Gaussian Mixture Models (GMM), a codebook algorithm, , And a hash image analysis algorithm can be applied to object detection.

Meanwhile, the deep learning server 50 analyzes the image received from the image analysis device 60 by a deep learning method and identifies a monitoring target object when a monitoring target object for a predetermined monitoring target is identified And transmits the generated event information to the signal server 30.

In addition, the signal server 30 may transmit the event information to a predetermined control server 40 through a communication network to notify the control server 40 of an event occurrence of the monitored object, It is possible to notify the monitoring server 40 of a notice of the monitored object and report on the appearance of the monitored object.

At this time, the deep learning server 50 can directly transmit the event information to a client terminal, which is set in advance, through a communication network when generating the event information, and the client terminal The event information can be displayed through an application or a web browser.

As described above, in general, when an existing security system operates in a low-illuminance environment and an object is detected during an image analysis process of a moving image generated when a monitoring target is detected through a sensing signal of the sensor, If an object is not detected but the object is not recognized due to low illuminance even if the object is detected through the sensor, it is possible to detect an error rate In order to improve the problem, the deep learning-based security service providing system according to the present invention continuously learns the characteristics of the monitored object appearing in the environmental characteristic (environmental condition) of the monitored region through the deep learning, It is possible to increase the accuracy of identification of the object, By continuously learning the gong, it is possible to judge the optimal image which is easy to identify the object, and by selectively providing the image in which the object to be monitored is accurately identified, it is possible to distinguish the object to be monitored accurately from other objects, At the same time, it is possible to prevent the report from being missed when it is detected.

The detailed configuration of the present invention will be described below with reference to the drawings based on the above-described configuration.

FIG. 3 is a diagram illustrating an operation of the image analysis apparatus 60 according to an embodiment of the present invention. As shown in FIG. 3, the image analysis apparatus 60 receives request information from a deep learning server 50 (S1) An image corresponding to the reception time point of the sensing signal included in the request information among the plurality of images constituting the moving picture information received from the camera unit 70 may be extracted and transmitted to the deep learning server 50 (S2).

At this time, in order to increase the accuracy of identification of the object to be monitored of the deep learning server 50, the image analysis apparatus 60 transmits the extracted image corresponding to the request information to the deep learning server 50 The image analysis may be performed on the extracted image before transmission.

For example, the image analysis apparatus 60 extracts an associated image adjacent to the extracted image in the temporal order from the extracted moving image, and associates the extracted image and the associated image with an image analysis algorithm such as a differential image method So as to determine whether or not the object of the extracted image is detected.

In addition, a selection criterion for determining whether the object can be identified is preset for the extracted image. If the object is detected (S3), the image analyzer 60 analyzes the extracted image according to the selection criterion (S5) If the extracted image satisfies the selection criteria (S6), the extracted image may be transmitted to the deep learning server 50 (S7).

For example, the image analysis apparatus 60 determines whether or not an object detected in the extracted image has a size equal to or larger than a predetermined size according to the selection criterion in accordance with the selection criterion, Whether or not the object is identifiable or not can be judged comprehensively.

If the object is not detected in the extracted image or the selection criterion is not satisfied (S3, S6), the image analyzer 60 determines whether the sensing signal is received from among the plurality of images included in the moving image information (S3, S5, and S6). If the object is detected and the object satisfying the selection criterion satisfies the selection criterion An image may be selected and transmitted to the deep learning server 50 (S7).

Accordingly, the image analysis apparatus 60 itself determines whether or not the monitoring object can be identified with respect to the extracted image corresponding to the reception timing of the sensing signal. If it is difficult to identify the monitoring target object, To the deep learning server 50 by providing an optimum image capable of identifying the object to be monitored, so that the object to be monitored can be identified through another image close to the reception time of the sensing signal, .

Meanwhile, the deep learning server 50 can identify an object through a deep learning method (or a deep learning algorithm) according to the request information, from the image received from the image analysis device 60.

In this case, the deep learning server 50 may include a controller. The controller sets a neural network according to a Deep Learning Network (DNN) based deep learning algorithm. The neural network includes an input layer, , One or more hidden layers (Hidden Layers), and an output layer (Output Layer).

Here, the deep learning algorithm may be applied to a neural network other than DNN. For example, a neural network such as CNN (Convolution Neural Network) or RNN (Recurrent Neural Network) may be applied.

Accordingly, the controller of the deep learning server 50 transmits request information to the image analysis apparatus 60 every time a sensing signal is received from the signal server 30, An image received from the analysis apparatus 60 may be stored in the DB included in the deep learning server 50. [

The control unit of the deep learning server 50 analyzes the images received from the image analysis apparatus 60 based on the neural network to identify and classify one or more different objects, It is possible to generate object information including parameters for various object characteristics such as star size, color distribution, outline, etc., and store the object information in the DB.

In addition, the control unit of the deep learning server 50 may update the object information by repeatedly learning the image continuously received from the image analysis apparatus 60 through the neural network, It is possible to accurately identify the object corresponding to the object characteristic parameter included in the object information in the image by adapting to the object characteristic change appearing in the image according to the environmental change (environmental condition change (for example, illumination, obstacle)) .

Meanwhile, the deep learning server 50 may include a user input unit or a communication unit. The control unit of the deep learning server 50 may receive user input related control information through the user input unit, One or more objects classified by the control unit based on the control information can be set as a monitoring target object and the setting information can be stored.

At this time, the setting information may include object information about an object set as the monitored object.

In addition, the controller of the deep learning server 50 may identify the object to be monitored through the deep learning method in the image received from the image analysis apparatus 60 according to the setting information, Information can be generated.

At this time, the control unit of the deep learning server 50 can distinguish the normal image in which the monitored object is identified from other images, and store the normal image in the DB.

If a specific object other than the monitored object is identified in the process of identifying the monitored object in the specific image received from the image analysis device 60 according to the setting information, the controller of the deep learning server 50 It is possible to compare the similarity between the specific image and one or more normal images stored in the DB. If the similarity is equal to or greater than a predetermined reference value, it can be determined that an error has occurred in the object identification process for the specific object.

At this time, the controller of the deep learning server 50 may perform a histogram matching based on a similarity comparison method between images, a template matching specified to an object identified in the image, The similarity can be compared using the feature point comparison extracted through the feature point comparison.

Accordingly, the controller of the deep learning server 50 can generate error information on the error between the output value of the image processing based on the neural network and the object information with respect to the specific image, the parameters configuring the neural network can be adjusted (varied) on the basis of the error information through the backpropagation algorithm, thereby optimizing the neural network to improve the identification accuracy of the monitored object.

The parameters constituting the neural network include a weight for connection strength between the input layer, the at least one concealment layer and the output layer constituting the neural network or the bias of the unit configured at the input layer, . ≪ / RTI >

Through the above-described configuration, the controller of the deep learning server 50 can increase the accuracy of the identification of the monitored object through the iterative learning of the monitored object, In order to accurately identify the objects to be photographed by a camera operated in a low-light environment, it is necessary to learn the characteristics of the object to be observed It is possible to improve the accuracy of object identification by optimizing the deep learning algorithm according to the characteristics of the object. By doing so, it is possible to precisely distinguish objects other than the object to be monitored, .

In the meantime, even when the monitoring object is sensed by the sensor unit 10 and the sensing signal is received, the deep learning server 50 may degrade the illuminance of the surveillance target area instantaneously or due to appearance of an obstacle, 60 may be difficult to identify the monitored object in the received image corresponding to the request information. Even if the monitored object is detected by the sensor unit 10, the event is not generated and the report is missed Can occur.

4 and 5, the deep learning server 50 receives the sensing signal from the image analysis device 60 in response to a reception of the sensing signal, If the identification of the monitored object fails in the image, the image analysis apparatus 60 may transmit the re-request information for requesting another image again.

In addition, the image analysis apparatus 60 performs image analysis for each of a plurality of images (existing) within a predetermined time range based on the reception time of the sensing signal in the moving image information upon receiving the re-request information And transmit the one or more images in which the object is detected among the plurality of images to the deep learning server 50.

Accordingly, the deep learning server 50 analyzes each of the one or more images received corresponding to the re-request information from the image analysis apparatus 60 through the deep learning algorithm to determine whether or not the monitored object is identified And if the monitored object has an identified image, it can generate identification related event information of the monitored object and transmit the generated identification related event information to the signal server 30.

Also, the deep learning server 50 may transmit the result information on the image in which the monitored object is identified to the image analysis apparatus 60.

5, the image analysis apparatus 60 may extract a normal image corresponding to the result information from the moving image information stored in the storage unit of the image analysis apparatus 60 when the result information is received have.

At this time, the image analyzing apparatus 60 can identify the moving image information including the normal image corresponding to the result information based on the result information, extracts the normal image from the moving image information identified corresponding to the result information, Can be extracted.

Thereafter, the image analyzing apparatus 60 extracts image features of the normal image in which the monitored object corresponding to the result information transmitted from the deep learning server 50 is identified, and stores feature information on the image feature have.

That is, when the deep-learning server 50 fails to identify the object, the image analysis device 60 identifies the object in another specific image at a time point close to the reception time of the sensing signal through the image re-request, It can be judged that the image characteristic is easy to identify.

On the other hand, in the image analysis apparatus 60, a selection criterion for selecting and transmitting an optimal image easy to identify the object to be monitored is set in advance to the deep learning server 50, and each time the feature information is received, The selection criterion can be continuously renewed.

That is, the image analysis apparatus 60 can extract and collect a normal image in which the object to be monitored is correctly identified, based on the result information transmitted from the deep learning server 50, Based on feature information of a normal image according to the result information continuously collected so as to optimize a predetermined selection criterion in the image analysis apparatus 60 for selecting an optimal image so as to easily identify an object in the image analysis unit 50, It is possible to perform iterative learning by continuously updating the selection criteria.

At this time, the image analysis apparatus 60 can extract an image feature through image analysis of a normal image corresponding to the image identification information extracted from the moving image information, and based on the extracted image feature, And may update the selection criterion by assigning a weight to any one of a plurality of the predetermined attributes.

The selection criterion may be set in an image analysis algorithm previously set in the image analysis apparatus 60. The image analysis apparatus 60 may acquire feature information generated from the normal image through a predetermined machine learning algorithm, It can be updated by applying to selection criteria.

In addition, the image analysis apparatus 60 may repeatedly update the selection criteria based on the result information continuously received from the deep learning server 50, and may generate a plurality of attribute-specific parameters or a weight value Value can be operated so that the deep learning server 50 converges to an optimal value for easily identifying the monitored object.

For example, the image analysis apparatus 60 analyzes a histogram distribution, which is an image characteristic of a normal image according to the result information received from the deep learning server 50, or a histogram variation between a normal image and a normal image of the normal image , It is possible to adjust parameters relating to one or more attributes (for example, color values, color distribution, gamma values, etc.) related to the corresponding histogram distribution or the histogram variation according to the machine learning algorithm, or to assign weights to the attributes.

Accordingly, the image analyzing apparatus 60 selects an optimal image that is easy to identify the monitored object and has the state that the image characteristic of the specific attribute is emphasized in the subsequent image analysis process according to the selection criteria, 50, and can support the monitoring object 50 to be easily identified by the deep learning server 50.

In this case, in the above-described configuration, the selection criterion may include a plurality of property-specific parameters, and the attribute may include a size of an object on an image, a distribution of a boundary or an outline on the object, A color difference, a histogram distribution change amount between the normal image and the previous image of the normal image, and the like.

Thereafter, the image analysis apparatus 60 receives the sensing signal from the deep learning server 50 and, when an object is detected in the image corresponding to the reception timing of the sensing signal at the time of the image request, analyzes the image according to the selection criteria If the selection criterion is satisfied, it is transmitted to the deep learning server 50. If the selection criterion is not satisfied, the time of reception of the sensing signal among the plurality of images constituting the moving picture information (Or comparing with the selection criterion) the selected one or more images in a predetermined time range and extracting one or more images in which the object is detected through image analysis, The optimum image can be transmitted to the deep learning server 50.

At this time, when the image is requested by the deep learning server 50, the image analyzer 60 extracts an image corresponding to the reception time of the sensing signal according to the request information from the moving image information and transmits the extracted image to the deep learning server 50 And an image selection unit that selects one of the plurality of images included in the moving image information based on whether the object is detected and whether the selection criterion is satisfied, May be selected and transmitted to the deep learning server 50.

Accordingly, the deep learning server 50 can easily identify an object to be monitored by receiving an optimal image from the image analysis device 60, which is easy to identify the object to be monitored, from the image analysis device 60 If it is difficult to identify the object to be monitored in the image photographed through the camera unit 70 at the time of receiving the sensing signal, the image captured through the camera unit 70 within the time range adjacent to the reception time of the sensing signal An optimal image capable of identifying the object to be monitored is selected and supported so that the object to be monitored is easily identified through the optimal image so that the object to be monitored detected by the sensor unit 10 can be accurately identified in the image It is possible to prevent a case where the report is omitted.

For example, the deep learning server 50 receives an image from the image analysis device 60 in response to the sensing signal, and the monitoring target area is configured in a low-illuminance environment so that an object exists in the image, It is possible to identify the type of the object, and if the identification of the object to be monitored fails, the image analysis apparatus 60 can request another image again.

Accordingly, the image analyzing apparatus 60 responds to the re-request and detects one or more other images in the predetermined time range based on the reception time of the sensing signal, 50), and the deep learning server (50) selects only the image in which the monitored object is identified from at least one of the other images, and transmits the result information including the selected image-based image identification information to the image analysis device ).

At this time, the deep learning server 50 may select only a normal image in which the eyeball ratio is distinct from one or more of the other images if the person to be monitored is a person, and provide the result information including image identification information to the image analyzer 60 have.

Accordingly, based on the result information transmitted from the deep learning server 50, the image analysis apparatus 60 can extract and collect only the normal image, which is easy to identify the monitored object, from the video information, , An image characteristic of an image in which the edge points are distinctly detected can be extracted through image analysis for each of a plurality of different normal images.

In addition, the image analyzing apparatus 60 reflects and updates the predetermined selection criterion based on the feature information according to the image feature, and updates the feature information generated based on the image feature of the normal image, which is continuously collected, The selection criterion can be optimized so as to facilitate the selection of the image in which the eyeball ratio is clearly detected.

Accordingly, when the deep processing server 50 requests or requests the image analysis device 60 to respond to the reception of the sensing signal at a later time, the image analysis device 60 corresponds to the reception time of the sensing signal It is possible to determine whether or not the object is detected and whether the selection criterion is satisfied in the image 6 which is one of the plurality of images constituting the moving image information.

Accordingly, if the object is detected in the image 6 but the image 6 is not normally identified as a result of analyzing the image 6 according to the selection criterion, Extracting a plurality of images belonging to a predetermined time range on the basis of a receiving time point from the moving image information and analyzing the image in which the object is detected among the plurality of images in accordance with the selection criterion, It is possible to provide the deep running server 50 with the image 7 in which the odds ratio is apparent.

Accordingly, the deep learning server 50 receives an optimal image selected from the image analysis device 60 as an optimal image that is easy to identify the monitored object according to a selection criterion, Can be easily identified.

As described above, the deep learning-based security service providing system according to the present invention not only improves the accuracy of identification of a monitored object in a still image (image) through deep learning-based learning, By performing the iterative learning based on the extracted image characteristic, it is possible to select and provide an optimal image that is easy to identify the object in the object detection process for the moving image provided by the camera unit 70 according to the environmental characteristic of the monitored region , It is possible to improve the accuracy of identification of objects through the iterative learning through still images, and at the same time, it is possible to enhance the accuracy of identification of objects by supporting the extraction of the optimal image which is easy to identify objects in moving images. At the point of time, the object to be monitored is accurately identified through the image of the camera, And at the same time, the reliability and accuracy of the security service can be greatly increased by preventing the report on the monitored object sensed by the sensor from being missed.

That is, according to the present invention, the object identification function based on still images and the object detection function based on moving images can be interlocked and supplemented to optimize detection and identification of a monitored object, The rate can be greatly improved.

FIG. 6 is a flowchart illustrating a method of providing a deep learning-based security service according to an embodiment of the present invention. As shown in FIG. 6, a deep learning server 50 for receiving a sensing signal of a sensor unit 10, The image analysis apparatus 60 receiving the moving image information from the camera unit 70 may transmit the request information for requesting the image corresponding to the reception time of the sensing signal (S11).

The image analysis apparatus 60 may extract an image corresponding to the request information of the deep learning server 50 from the moving image information and transmit the extracted image to the deep learning server 50 (S12).

Thereafter, the deep learning server 50 analyzes the image in a deep learning manner (S13). If a predetermined monitoring object is identified (S14), the deep learning server 50 generates event information (S15) (S14), the image analysis apparatus 60 may transmit re-request information for re-requesting another image related to the reception time of the sensing signal (S18).

Upon receiving the re-request information, the image analyzer 60 may transmit one or more other images in which the object is detected within a preset time range to the deep learning server 50 based on the reception time of the sensing signal S19).

At this time, the image analysis apparatus 60 belongs to (corresponds to) a preset time range based on the reception time of the sensing signal corresponding to the re-request information among the plurality of images included in the moving image information, It is possible to determine whether or not the object is detected based on one or more images except for the transmitted image, and transmit the detected image to the deep learning server 50. [

In addition, the image analyzing apparatus 60 may be configured such that a selection criterion (including a plurality of parameters for each property) composed of a plurality of predetermined parameters for each attribute necessary for identification of the monitored object is set in advance, An optimal image satisfying the selection criterion may be transmitted to the deep learning server 50 in step S20.

Accordingly, the deep learning server 50 generates event information when the monitored object is identified among the at least one other image (S14) (S15), and the deep learning server 50 generates the image identification information To the image analysis apparatus 60 (S16).

In addition, the image analyzing apparatus 60 may analyze the normal image corresponding to the result information upon receiving the result information, extract image features, and update the selection criteria based on the image characteristics (S17).

Then, the image analyzing apparatus 60 receives the re-request information from the deep learning server 50, and selects an object among a plurality of images included in the moving image information within a predetermined time range based on the reception time of the sensing signal Selects an optimal image that satisfies the selection criteria, and transmits the selected optimal image to the deep learning server 50, thereby enabling the deep learning server 50 to easily identify the monitored object.

The various devices and components described herein may be implemented by hardware circuitry (e.g., CMOS-based logic circuitry), firmware, software, or a combination thereof. For example, it can be implemented utilizing transistors, logic gates, and electronic circuits in the form of various electrical structures.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or essential characteristics thereof. Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

10: sensor unit 20: detection management device
30: signal server 40: control server
50: Deep Learning Server 60: Image analysis device
70:

Claims (9)

  1. A sensing management device for sensing the object to be monitored and transmitting the sensing signal received from a sensor unit for transmitting a sensing signal to a predetermined signal server;
    A camera unit for capturing the surveillance target area and transmitting the generated video information;
    A signal server for transmitting the sensing signal from the sensing management device to a preset deep learning server upon receipt of the sensing signal;
    An image analyzer for extracting and transmitting an image corresponding to the image request among a plurality of images constituting the moving image information received from the camera unit upon receiving an image request; And
    Requesting an image from the image analysis apparatus when receiving the sensing signal from the signal server, analyzing the received image in a deep learning manner according to the image request, and generating event information when a predetermined monitored object is identified, Requesting another image related to the reception time of the sensing signal to the image analyzing device when the monitoring object is not identified, and outputting result information on the normal image in which the monitoring object is identified, And a deep learning server for transmitting to the image analysis apparatus,
    Wherein the image analyzing apparatus includes a plurality of images that are within a predetermined time range based on a reception time point of the sensing signal among the plurality of images when the image re-request of the deep learning server is requested, And transmits the optimal image to the deep learning server, and updates the selection criterion according to the result information.
  2. The method according to claim 1,
    Wherein the deep learning server transmits the event information to a preset control server through the signal server when generating the event information or transmits the event information to a preset client terminal.
  3. The method according to claim 1,
    Wherein the image analysis apparatus comprises a DVR or an NVR.
  4. The method according to claim 1,
    Wherein the deep learning server is configured to generate the result information including image identification information for each normal image identified by the monitored object when the monitored object is identified in at least one of the received optimal images according to the image re- To the image analysis apparatus,
    Wherein the image analysis device extracts an image feature through image analysis of the normal image corresponding to the result information, and changes at least one of attributes-specific parameters constituting the selection criterion based on the extracted image feature, Attribute of the deep learning-based security service to a predetermined value.
  5. The method of claim 4,
    Wherein the attribute includes at least one of a size of an object position on the image, a boundary or an outline of the object, an illuminance or a color difference between the object and the background, and a histogram distribution variation. system.
  6. The method according to claim 1,
    Wherein the deep learning server analyzes the image received from the image analysis apparatus through a deep learning based neural network to identify the monitored object, stores the normal image in which the monitored object is identified, When a specific object other than the monitored object is identified in the process of identifying the monitored object in the specific image, the comparison of the similarity between the specific image and the at least one normal image can be performed. If the similarity is equal to or greater than a preset reference value And generating error information on an error between the output value of the image processed based on the neural network and the object information with respect to the specific image and outputting the error information based on the error information through a predetermined back propagation algorithm, Adjust the parameters that make up the neural network Deep learning-based security service system wherein the.
  7. The method of claim 6,
    The deep learning server calculates a weight of connection strength between the input layer, the at least one hidden layer and the output layer constituting the neural network through the back propagation algorithm based on the error information or the weight of the input layer, Wherein learning is performed such that an identification error of the monitored object is minimized by varying a bias of the configured unit.
  8. A method for providing a deep learning-based security service of an image analyzing apparatus which receives moving image information from a deep learning server and a camera unit receiving a sensing signal of a sensor unit according to monitoring of a monitored object and communicates with the deep learning server,
    Transmitting, to the image analysis apparatus, request information for requesting an image corresponding to a reception time of the sensing signal when the deep learning server receives the sensing signal;
    Extracting an image corresponding to the request information of the deep learning server from the video information and transmitting the extracted image to the deep learning server;
    Wherein the deep learning server analyzes the image received from the image analysis apparatus by a deep learning method to generate event information when a predetermined monitored object is identified, Transmitting re-request information for re-requesting another image associated with a reception time of the signal;
    Wherein the image analysis device analyzes the one or more other images belonging to a time range within a predetermined time range based on the reception time of the sensing signal among the plurality of images in response to the re- Transmitting an image to the deep learning server
    Based security service.
  9. The method of claim 8,
    Generating the result information including the image identification information for the best image when the deep learning server identifies the monitored object from the optimal image, and transmitting the result information to the image analysis apparatus; And
    Analyzing an image corresponding to the result information when the image analysis apparatus receives the result information, extracting an image feature, and updating the selection criterion based on the image feature
    The method of claim 1, further comprising:
KR1020170087862A 2017-07-11 2017-07-11 System and method for providing security service based on deep learning KR101789690B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020170087862A KR101789690B1 (en) 2017-07-11 2017-07-11 System and method for providing security service based on deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020170087862A KR101789690B1 (en) 2017-07-11 2017-07-11 System and method for providing security service based on deep learning

Publications (1)

Publication Number Publication Date
KR101789690B1 true KR101789690B1 (en) 2017-10-25

Family

ID=60300165

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020170087862A KR101789690B1 (en) 2017-07-11 2017-07-11 System and method for providing security service based on deep learning

Country Status (1)

Country Link
KR (1) KR101789690B1 (en)

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101942808B1 (en) 2018-11-13 2019-01-29 주식회사 인텔리빅스 Apparatus for CCTV Video Analytics Based on Object-Image Recognition DCNN
KR101964683B1 (en) * 2018-10-22 2019-04-02 주식회사 인텔리빅스 Apparatus for Processing Image Smartly and Driving Method Thereof
KR102005956B1 (en) * 2018-03-19 2019-07-31 김대훈 Image recognition apparatus in the inside of vehicle
KR20190101692A (en) * 2018-02-23 2019-09-02 주식회사 삼알글로벌 Video watch method based on transfer of learning
KR102020845B1 (en) * 2018-04-04 2019-09-11 주식회사 자비스넷 Calibration system and method for camera using object identification
KR20190108855A (en) * 2018-03-15 2019-09-25 엘지전자 주식회사 Washing machine configuring function based on object sensing, cloud server and method of configuring thereof
WO2019199043A1 (en) * 2018-04-10 2019-10-17 엘지전자 주식회사 Mobile terminal
KR20190123661A (en) * 2018-04-10 2019-11-01 엘지전자 주식회사 Mobile terminal
KR102058452B1 (en) * 2019-06-28 2019-12-23 가온플랫폼 주식회사 IoT Convergence Intelligent Video Analysis Platform System
KR102069085B1 (en) * 2019-04-02 2020-01-22 주식회사 자비스넷 Security service system and method supporting active response based on deep learning
WO2020017814A1 (en) * 2018-07-19 2020-01-23 엘지이노텍 주식회사 Abnormal entity detection system and method
KR102094027B1 (en) * 2018-10-02 2020-03-26 (주)인스웨이브시스템즈 Web standard UI source transformation device and method using deep learning
WO2020091215A1 (en) * 2018-10-31 2020-05-07 Lg Electronics Inc. Terminal and method for connecting to target devices
KR20200058260A (en) 2019-01-18 2020-05-27 주식회사 인텔리빅스 Apparatus for CCTV Video Analytics Based on Object-Image Recognition DCNN and Driving Method Thereof
KR102087926B1 (en) * 2019-11-06 2020-05-27 주식회사엔클라우드 Iot detection control apparatus

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101748121B1 (en) 2016-08-25 2017-06-16 델리아이 주식회사 System and method for detecting image in real-time based on object recognition

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101748121B1 (en) 2016-08-25 2017-06-16 델리아이 주식회사 System and method for detecting image in real-time based on object recognition

Cited By (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102110375B1 (en) * 2018-02-23 2020-05-14 주식회사 삼알글로벌 Video watch method based on transfer of learning
KR20190101692A (en) * 2018-02-23 2019-09-02 주식회사 삼알글로벌 Video watch method based on transfer of learning
KR102111110B1 (en) * 2018-03-15 2020-05-14 엘지전자 주식회사 Washing machine configuring function based on object sensing using artificial intelligence, cloud server and method of configuring thereof
KR20190108855A (en) * 2018-03-15 2019-09-25 엘지전자 주식회사 Washing machine configuring function based on object sensing, cloud server and method of configuring thereof
KR102005956B1 (en) * 2018-03-19 2019-07-31 김대훈 Image recognition apparatus in the inside of vehicle
KR102020845B1 (en) * 2018-04-04 2019-09-11 주식회사 자비스넷 Calibration system and method for camera using object identification
WO2019199043A1 (en) * 2018-04-10 2019-10-17 엘지전자 주식회사 Mobile terminal
KR102102396B1 (en) * 2018-04-10 2020-04-21 엘지전자 주식회사 Mobile terminal
KR20190123661A (en) * 2018-04-10 2019-11-01 엘지전자 주식회사 Mobile terminal
WO2020017814A1 (en) * 2018-07-19 2020-01-23 엘지이노텍 주식회사 Abnormal entity detection system and method
KR102094027B1 (en) * 2018-10-02 2020-03-26 (주)인스웨이브시스템즈 Web standard UI source transformation device and method using deep learning
KR101964683B1 (en) * 2018-10-22 2019-04-02 주식회사 인텔리빅스 Apparatus for Processing Image Smartly and Driving Method Thereof
KR102137194B1 (en) * 2018-10-31 2020-07-24 엘지전자 주식회사 Mobile terminal
WO2020091215A1 (en) * 2018-10-31 2020-05-07 Lg Electronics Inc. Terminal and method for connecting to target devices
KR101942808B1 (en) 2018-11-13 2019-01-29 주식회사 인텔리빅스 Apparatus for CCTV Video Analytics Based on Object-Image Recognition DCNN
KR20200058260A (en) 2019-01-18 2020-05-27 주식회사 인텔리빅스 Apparatus for CCTV Video Analytics Based on Object-Image Recognition DCNN and Driving Method Thereof
KR102069085B1 (en) * 2019-04-02 2020-01-22 주식회사 자비스넷 Security service system and method supporting active response based on deep learning
KR102058452B1 (en) * 2019-06-28 2019-12-23 가온플랫폼 주식회사 IoT Convergence Intelligent Video Analysis Platform System
KR102087926B1 (en) * 2019-11-06 2020-05-27 주식회사엔클라우드 Iot detection control apparatus

Similar Documents

Publication Publication Date Title
US10546186B2 (en) Object tracking and best shot detection system
US9286778B2 (en) Method and system for security system tampering detection
US9836933B2 (en) Data processing apparatus to generate an alarm
AU2013200450B2 (en) System and method for face capture and matching
US8300890B1 (en) Person/object image and screening
JP6127152B2 (en) Security monitoring system and corresponding alarm triggering method
JP3812985B2 (en) Automatic monitoring device
JP4629090B2 (en) Camera tampering detection
EP1435170B1 (en) Video tripwire
US8243987B2 (en) Object tracking using color histogram and object size
EP1364351B8 (en) Method and device for detecting fires based on image analysis
KR101826897B1 (en) Method and camera for determining an image adjustment parameter
US9928707B2 (en) Surveillance system
CN102521578B (en) Method for detecting and identifying intrusion
US7542588B2 (en) System and method for assuring high resolution imaging of distinctive characteristics of a moving object
US20120321138A1 (en) Suspicious behavior detection system and method
US5091780A (en) A trainable security system emthod for the same
US9277165B2 (en) Video surveillance system and method using IP-based networks
US7583815B2 (en) Wide-area site-based video surveillance system
KR101168760B1 (en) Flame detecting method and device
JP2006523043A (en) Method and system for monitoring
US20200034631A1 (en) Image processing apparatus, image processing method, and storage medium
KR20090021613A (en) Apparatus and method for inferencing topology of multiple cameras network by tracking movement
US6774905B2 (en) Image data processing
US20080291278A1 (en) Wide-area site-based video surveillance system

Legal Events

Date Code Title Description
E902 Notification of reason for refusal
E701 Decision to grant or registration of patent right
GRNT Written decision to grant