KR102002812B1 - Image Analysis Method and Server Apparatus for Detecting Object - Google Patents

Abstract

Disclosed is an image analysis server apparatus and method for object detection.
According to an embodiment of the present invention, there is provided an object detection apparatus comprising: an object detection unit that is learned to detect and recognize an object from an input image; And a re-learning unit for re-learning the object detection unit such that an object is not detected from the input image when an error of a detection result output from the object detection unit exceeds a predetermined threshold value without detecting motion in the input image And provides an image analysis server apparatus including the image analysis server apparatus.

Description

TECHNICAL FIELD The present invention relates to an image analysis server apparatus and method for detecting an object,

The present invention relates to an image analysis server apparatus and method for object detection.

The contents described in this section merely provide background information on the present invention and do not constitute the prior art.

Surveillance systems using CCTV (Closed Circuit Television) and DVR (Digital Video Recorder) are widening their scope as an intelligent surveillance system in addition to the technology of simple monitoring and image storage by the administrator.

The intelligent video surveillance system is a system that can detect, recognize, classify, and trace objects by analyzing the video information inputted from the camera in real time. The intelligent image detection system can determine whether an object has generated security related events and provide information to the administrator, or store data and event contents, and maximize the efficiency of post-preventive management and retrieval.

CCTV video surveillance system adopts Deep Learning technology such as Convolutional Neural Network (CNN) for object detection, recognition, classification, and tracking.

When a convolution neural network (CNN) -based technique is used, it is possible to detect an object for each image frame and utilize information about the detected object. However, even if there is no object in the image, Due to the factors, the object may be detected frequently.

False alarms due to erroneous detection preclude correct recognition and rapid response of dangerous situations. Especially, CCTV images have a similar background image. Therefore, once false detection occurs, there arises a problem that false alarms are continuously generated for similar images.

Embodiments of the present invention provide an image analysis server apparatus and method capable of minimizing false detection of objects in a fixed camera environment such as CCTV.

According to an embodiment of the present invention, there is provided an object detection apparatus comprising: an object detection unit that is learned to detect and recognize an object from an input image; And a re-learning unit for re-learning the object detection unit such that an object is not detected from the input image when an error of a detection result output from the object detection unit exceeds a predetermined threshold value without detecting motion in the input image And provides an image analysis server apparatus including the image analysis server apparatus.

According to another embodiment of the present invention, there is provided a method of acquiring an input image, Generating a detection result by a learned object detector to detect and recognize an object from the input image; And re-learning the object detector so that an object is not detected from the input image if motion in the input image is not detected and the error of the detection result exceeds a preset threshold value do.

As described above, according to the embodiments of the present invention, it is possible to prevent the occurrence of false positives continuously in the video surveillance system by determining whether the neural network is re-learned for object detection and recognition using motion information in the input image There is an effect that can be minimized.

1 is a block diagram of a video surveillance system according to an embodiment of the present invention.
2 is a block diagram of an image analysis server apparatus according to an embodiment of the present invention.
3 is a flowchart illustrating an image analysis method according to an embodiment of the present invention.
4 is a flowchart illustrating an example of an image analysis method according to an embodiment of the present invention.

Hereinafter, some embodiments of the present invention will be described in detail with reference to exemplary drawings. It should be noted that, in adding reference numerals to the constituent elements of the drawings, the same constituent elements are denoted by the same reference numerals even though they are shown in different drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

In describing the components of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are intended to distinguish the constituent elements from other constituent elements, and the terms do not limit the nature, order or order of the constituent elements. Throughout the specification, when an element is referred to as being "comprising" or "comprising", it means that it can include other elements as well, without excluding other elements unless specifically stated otherwise . In addition, '... Quot ;, " module ", and " module " refer to a unit that processes at least one function or operation, and may be implemented by hardware or software or a combination of hardware and software.

1 is a block diagram of a video surveillance system 100 in accordance with an embodiment of the present invention. The video surveillance system 100 includes a video photographing apparatus 110, an image analysis server apparatus 120, and a monitoring apparatus 130.

The image capturing apparatus 110 generates an image to be input to the image analysis server apparatus 120 in real time. The image capturing apparatus 110 may be implemented as a fixed type camera such as a CCTV camera, but may include all apparatuses capable of generating an input image of the image analysis server apparatus 120. [

The image analysis server apparatus 120 detects and recognizes an object by analyzing an input image received from the image capturing apparatus 110 in real time, and detects one or more events defined in advance based on the analysis result.

Also, the image analysis server device 120 transmits information according to the detected event to the monitoring device 130. [ The image analysis server device 120 trains a neural network designed to detect and recognize objects, and detects and recognizes objects in the input image based on the learned neural network.

For example, a convolutional neural network (CNN) can be used as a neural network. The convolutional neural network (CNN) includes at least one convolution layer for performing a convolution operation on the input image and at least one pooling layer for sampling the output of the convolution layer.

However, the convolutional neural network (CNN) is merely an example, and the present invention is not limited thereto, and various other neural networks such as a Recurrent Neural Network (RNN) or a combination of CNN and RNN may be used.

The monitoring device 130 can receive the image photographed from the image photographing device 110 and display it in real time. In addition, the monitoring device 130 may generate an appropriate event such as an alarm according to the information received from the image analysis server device 120.

2 is a block diagram of an image analysis server apparatus 120 according to an embodiment of the present invention. The image analysis server apparatus 120 includes an input image acquisition unit 210, an object detection unit 220, and a re-learning unit 230. Each component shown in Fig. 2 may be implemented as a hardware chip, or may be implemented as software and a microprocessor may be implemented to execute the functions of software corresponding to each component.

The input image obtaining unit 210 receives an input image from the image capturing apparatus 110 and captures the input image. The input image obtaining unit 210 resizes the captured input image to the size of the input layer of the object detection and recognition neural network. The adjusted input image is transmitted to the object detection unit 220. In addition, the adjusted input image is also transmitted to the re-learning unit 230 when the object detection unit 220 needs to update.

The object detection unit 120 detects and recognizes an object from an input image based on the learned neural network. The object detection unit 120 derives a detection candidate through calculation in several layers (e.g., a convolution layer and a max-pooling layer) included in the neural network. The object detection unit 120 generates and outputs a final detection result by applying a threshold to the detection candidates.

The output detection result includes at least one of position information of the object, class information of the object, and reliability of the object. Here, the position information of the object represents information (eg, position coordinates, area width and height) of the area (or box) in which the object is detected, and the class information of the object indicates that the object is a certain class ).

Also, the reliability of an object indicates the probability that the object belongs to the finally determined class. For example, if there is no object, the ideal confidence value is zero. When a plurality of detection regions exist in one input image, the above detection result can be output for each detection region.

Although not shown in FIG. 2, the image analysis server apparatus 120 may further include a component for determining a situation based on a detection result. A component (not shown) for determining the situation can detect a situation occurring in the input image using the detected object.

For example, the component may determine whether a person is detected in a situation where a person is to be detected, and may determine whether a person and a vehicle are detected at the same time have. The component may also make a situation determination based on the confidence values of each detected object.

The re-learning unit 230 determines whether re-learning of the object detection unit 220 is necessary based on whether motion is detected in the input image and an error of the detection result output from the object detection unit 220. [

As a result of the determination, if the re-learning is necessary, the re-learning unit 230 re-learns the object detection unit 220 so that the object is not detected from the input image. Re-learning of the object detection unit 220 means updating the learned parameters (or weights) between the nodes included in the neural network. First, a method for determining whether the re-learning unit 230 needs re-learning is described.

The re-learning unit 230 calculates the motion in the input image to determine whether re-learning is necessary. There are various methods for calculating the motion in the input image, such as using two consecutive image differences, using Gaussian Mixture Model (GMM), and difference of background image modeled based on VIBE Can be used.

The re-learning unit 230 calculates an error of the detection result output from the object detection unit 220 to determine whether re-learning is necessary. The error of the detection result means a loss between the target result (i.e., the target output) and the output (i.e., the estimated output) output from the object detection unit 220 with respect to the input image.

For example, in the embodiment of the present invention, since the object is not detected in the background image, it is ideal that the reliability value of the detected object is 0 in the background image.

However, according to the detection result actually outputted, there is a case where the reliability value is larger than 0, so that an error of the detection result occurs. The re-learning unit 230 can calculate the error of the detection result based on the reliability of the detection result and the number of one or more classes into which the detected object is classified.

When the input image is a background image, the larger the difference between the actual reliability value and the desired reliability value is, the larger the error becomes. Therefore, the error of the detection result can be calculated using the mean square error (MSE).

Specifically, the error of the detection result can be calculated using a mean square error between a target reliability value of 0 and the reliability value of the detection result. The calculation formula for calculating the error of the detection result according to this embodiment is expressed by Equation (1).

In the above equation, L represents the error of the detection result, n represents the number of classes in which the object is classified, and Confidence (i) represents the reliability for the i-th region among one or more detection regions in the input image.

The re-learning unit 230 determines whether re-learning of the object detection unit 220 is necessary by using the motion in the input image and the error of the detection result. The re-learning unit 230 first determines whether to re-learn the object detection unit 220 using the calculated motion information in the input image.

An object of the present invention is to prevent an object from being continuously misdetected in the absence of motion in an input image, so that in the case where motion is calculated, there is a high probability that the detection result is not wrong.

On the contrary, when the motion is calculated to be free, it is necessary to update the neural network so that the reliability of the detection result is low (converge to 0, for example) because the probability that the input image is background is high.

If the re-learning unit 230 determines that there is no motion in the input image, the re-learning unit 230 determines whether re-learning of the object detection unit 220 is necessary using the error of the detection result. Specifically, the re-learning unit 230 can determine that the re-learning of the object detection unit 220 is not necessary if the error of the detection result is equal to or less than a preset threshold value.

Since the error of the detection result is less than the preset threshold value, it means that the false detection does not occur in the background image, and it is not necessary to update the neural network. On the contrary, if the error of the detection result is equal to or greater than a preset threshold value, the re-learning unit 230 can determine that the object detection unit 220 needs re-learning.

This is because the probability of false positives is high because objects with high reliability are detected despite the absence of motion in the input image. Therefore, in this case, it is necessary to update the neural network so as to lower the reliability of the detection result (for example, to converge to zero).

If the re-learning of the object detection unit 220 is determined to be necessary, the re-learning unit 230 may generate labeled data including information indicating that no detected object exists in the input image, (I.e., the detected background image) as learning data, and re-learns the object detection unit 220 (i.e., updates the neural network). The label data may be, for example, a vector having information indicating that there is no detected object in the input image.

The updating of the neural network is performed in the same manner as the method used for learning the neural network performed in advance. Specifically, the re-learning unit 230 re-learns the object detection unit 220 so that the error of the detection result for the false detected input image is lowered (or the reliability is lowered) using the back propagation technique .

For example, the re-learning unit 230 may detect an error (or an error) of the detection result in the reverse direction from the output layer of the neural network in the object detection unit 220 to the input layer via the hidden layer Reliability) can be propagated. The parameters (or connection weights) between the nodes can be updated so that the error (or reliability) value is reduced in the process of propagating the error (or reliability) of the detection result in the reverse direction.

The re-learning unit 230 may variably set the learning rate (update rate) for one input image when the object detection unit 220 re-learns it. When the learning rate is high, the neural network converges quickly, whereas when the learning rate is low, the neural network converges slowly. When the neural network converges, it is no longer updated.

The re-learning unit 230 can determine how often the neural network should be updated by adjusting the learning rate. As the learning rate increases, the neural network converges quickly, so the neural network can be updated quickly with fewer images (eg, 1 frame per second or 1 frame per minute).

Hereinafter, an image analysis method according to an embodiment of the present invention will be described with reference to FIGS. 3 and 4. FIG. 3 is a flowchart illustrating an image analysis method according to an embodiment of the present invention.

Referring to FIG. 3, an input image is acquired in step S310. Specifically, the input image transmitted from the image capturing device is captured, and the captured input image is resized to the size of the input layer of the object detection and recognition neural network.

When the input image is acquired, the detection result is generated by the learned object detector to detect and recognize the object from the input image (S320). Object detectors derive detection candidates through computation in several layers (eg, Convolution Layer, Max-pooling Layer, etc.) included in the neural network. The object detector generates and outputs a final detection result by applying a threshold to the detection candidate.

The output detection result includes at least one of position information of the object, class information of the object, and reliability of the object. Here, the position information of the object represents information (eg, position coordinates, area width and height) of the area (or box) in which the object is detected, and the class information of the object indicates that the object is a certain class ).

The reliability of an object indicates the probability that the object belongs to the finally determined class. For example, if there is no object, the ideal confidence value is zero. When a plurality of detection regions exist in one input image, the above detection result can be output for each detection region.

In step S330, it is determined whether or not re-learning of the object detector is necessary based on whether motion detection is performed in the input image and an error in the detection result of step S320. If it is determined that re-learning is required, the object detector is re-learned in step S340 so that the object is not detected from the input image.

Re-learning the object detector means updating the learned parameters (or weights) between nodes included in the neural network. First, a process of determining whether re-learning is necessary will be described.

The process of determining re-learning necessity includes a process of calculating an error of a motion and detection result in the input image and a process of determining whether re-learning is necessary using the calculation result.

There are various methods for calculating the motion in the input image, such as using two consecutive image differences, using Gaussian Mixture Model (GMM), and difference of background image modeled based on VIBE Can be used.

The error of the detection result means a loss between the target result (i.e., the target output) and the output (i.e., the estimated output) output from the object detection unit 220 with respect to the input image.

For example, in the embodiment of the present invention, since the object is not detected in the background image, it is ideal that the reliability value of the detected object is 0 in the background image. However, according to the detection result actually outputted, there is a case where the reliability value is larger than 0, so that an error of the detection result occurs.

The error of the detection result can be calculated based on the reliability of the detection result and the number of one or more classes into which the detected object is classified. When the input image is a background image, the larger the difference between the actual reliability value and the desired reliability value is, the larger the error becomes. Therefore, the error of the detection result can be calculated using the mean square error (MSE).

Specifically, the error of the detection result can be calculated using a mean square error between a target reliability value of 0 and the reliability value of the detection result. The calculation formula for calculating the error of the detection result according to the present embodiment is expressed by Equation (1).

After calculating the motion in the input image and the error of the detection result, it is determined whether or not the object detection unit 220 needs re-learning using the calculation result. First, it is determined whether the object detector should be re-learned using motion information in the calculated input image.

An object of the present invention is to prevent an object from being continuously misdetected in the absence of motion in an input image, so that in the case where motion is calculated, there is a high probability that the detection result is not wrong. However, in the case where it is calculated that there is no motion in the contrary, it is necessary to update the neural network so that the reliability of the detection result is low (converge to 0, for example) since the probability that the input image is background is high.

If it is determined that there is no motion in the input image, it is determined whether or not re-learning of the object detector is necessary using the error of the detection result. Specifically, if the error of the detection result is equal to or less than a preset threshold value, it can be determined that re-learning of the object detector is not necessary.

Since the error of the detection result is less than the preset threshold value, it means that the false detection does not occur in the background image, and it is not necessary to update the neural network. Conversely, if the error of the detection result is greater than or equal to a preset threshold value, it can be determined that re-learning of the object detector is necessary.

This is because the probability of false positives is high because objects with high reliability are detected despite the absence of motion in the input image. Therefore, in this case, it is necessary to update the neural network so as to lower the reliability of the detection result (for example, to converge to zero).

In summary, if motion in the input image is not detected and the error of the detection result is greater than or equal to a threshold value, it is determined that re-learning of the object detector is necessary.

If it is determined in step S330 that the re-learning of the object detector is required, the labeled data and the input image (i.e., the false-detected background image) including information indicating that the detected object is not present in the input image, (I.e., updates the neural network). The label data may be, for example, a vector having information indicating that there is no detected object in the input image.

The updating of the neural network is performed in the same manner as the method used for learning the neural network performed in advance. Specifically, using the back propagation technique, the object detector can be re-learned so that the error of the detection result for the false-detected input image is lowered (or the reliability is lowered).

For example, the error (or reliability) of the detection result can be propagated from the output layer of the neural network in the object detector through the hidden layer to the input layer in the reverse direction. The parameters (or connection weights) between the nodes can be updated so that the error (or reliability) value is reduced in the process of propagating the error (or reliability) of the detection result in the reverse direction.

In the re-learning of step S340, the learning rate (update rate) for one input image can be variably determined. When the learning rate is high, the neural network converges quickly, whereas when the learning rate is low, the neural network converges slowly. When the neural network converges, it is no longer updated.

In step S340, it is possible to determine how often the neural network should be updated by adjusting the learning rate. As the learning rate increases, the neural network converges quickly, so the neural network can be updated quickly with fewer images (eg, 1 frame per second or 1 frame per minute).

4 is a flowchart illustrating an example of an image analysis method according to an embodiment of the present invention.

After the input image is acquired in step S410, an object in the input image is detected and recognized to generate a detection result (S420). Then, the status can be determined using the generated detection result (S430). That is, it is possible to detect a situation occurring in the input image using the detected object.

For example, when a situation to detect is a situation where a person appears, it is determined whether a person is detected. When a situation where a person wants to detect a situation where a person is leaving the vehicle is detected, it can be determined whether a person and a vehicle are detected simultaneously. It is also possible to determine the situation according to the reliability value of each detected object.

 Meanwhile, in order to determine whether to re-learn the object detector, motion in the input image is calculated (S440), and the error of the detection result generated in operation S420 is additionally calculated (S450).

In step S460, it is determined whether the motion is detected based on the motion in the input image calculated in step S440. If there is no motion in the input image, it is determined whether the error of the detection result calculated in step S450 is greater than a predetermined threshold (S470). As a result of the determination, if the error of the detection result is equal to or greater than a preset threshold value, the object detector is re-learned (S480).

In FIGS. 3 and 4, it is described that each process is sequentially executed, but the present invention is not limited thereto. In other words, it can be applied to changing the processes described in FIG. 3 and FIG. 4 or executing one or more processes in parallel. Thus, FIGS. 3 and 4 are not limited to time series.

The image analysis method according to the embodiment described in FIGS. 3 and 4 can be implemented as a program and recorded in a computer-readable recording medium. A program for implementing the image analysis method according to the present embodiment is recorded, and a computer-readable recording medium includes all kinds of recording devices for storing data that can be read by a computing system.

The foregoing description is merely illustrative of the technical idea of the present embodiment, and various modifications and changes may be made to those skilled in the art without departing from the essential characteristics of the embodiments. Therefore, the present embodiments are to be construed as illustrative rather than restrictive, and the scope of the technical idea of the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment should be construed according to the following claims, and all technical ideas within the scope of equivalents thereof should be construed as being included in the scope of the present invention.

100: video surveillance system 110:
120: Image analysis server 130: Monitoring device
210: input image acquisition unit 220: object detection unit
230: re-learning unit

Claims (8)

An object detection unit for detecting and recognizing an object from an input image; And
And a re-learning unit that re-learns the object detection unit so that an object is not detected from the input image if the motion in the input image is not detected and the error of the detection result output from the object detection unit exceeds a predetermined threshold value and,
The re-
An error of the detection result is calculated based on the number of one or more classes in which the detected object is classified and the reliability of the detection result
Image analysis server device. delete The method according to claim 1,
The re-
Wherein an error of the detection result is calculated using a mean square error (MSE) between a target reliability value of 0 and a reliability value of the detection result. The method according to claim 1,
The re-
Wherein the information analyzing unit re-learns the object detecting unit using information indicating that no object exists in the input image and the input image as learning data. Acquiring an input image;
Generating a detection result by a learned object detector to detect and recognize an object from the input image; And
And re-learning the object detector so that an object is not detected from the input image if motion in the input image is not detected and the error of the detection result exceeds a preset threshold value,
The error of the detection result is,
The detected object is calculated based on the number of one or more classes to be classified and the reliability of the detection result
Image analysis method. delete 6. The method of claim 5,
The error of the detection result is,
Wherein a mean square error (MSE) between a target reliability value of 0 and a reliability value of the detection result is used. 6. The method of claim 5,
The process of re-learning the object detector comprises:
And the object detector is re-learned using information indicating that no object exists in the input image and the input image as learning data.

Images