WO2021020866A1 - Image analysis system and method for remote monitoring - Google Patents

Image analysis system and method for remote monitoring Download PDF

Info

Publication number
WO2021020866A1
WO2021020866A1 PCT/KR2020/009954 KR2020009954W WO2021020866A1 WO 2021020866 A1 WO2021020866 A1 WO 2021020866A1 KR 2020009954 W KR2020009954 W KR 2020009954W WO 2021020866 A1 WO2021020866 A1 WO 2021020866A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
unit
images
event
monitored
Prior art date
Application number
PCT/KR2020/009954
Other languages
French (fr)
Korean (ko)
Other versions
WO2021020866A9 (en
Inventor
심재술
Original Assignee
(주)유디피
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by (주)유디피 filed Critical (주)유디피
Publication of WO2021020866A1 publication Critical patent/WO2021020866A1/en
Publication of WO2021020866A9 publication Critical patent/WO2021020866A9/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G06T7/001—Industrial image inspection using an image reference approach
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/60—Editing figures and text; Combining figures or text
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/254—Analysis of motion involving subtraction of images
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30232—Surveillance

Definitions

  • the present invention relates to an image analysis system and method for remote monitoring, and more particularly, when detecting an object to be monitored by analyzing an object identified in an image periodically received from a camera unit located in a remote location instead of in real time, based on deep learning.
  • the present invention relates to an image analysis system and method for remote monitoring capable of reducing time and cost for analyzing a monitored object by providing an event.
  • the present invention applies a deep learning-based image analysis method to an image received from a camera located at a remote location and has a very small number of transmission frames per second to easily detect an object to be monitored in the image.
  • the purpose of this is to increase the efficiency and reliability of the system for video monitoring by providing support to provide events and shortening the time required for video analysis during deep learning-based video analysis.
  • an object of the present invention is to support to reduce system configuration cost by supporting reliable detection of an object to be monitored even when a low-cost camera with a very small number of transmission frames per second is used.
  • An image analysis system for remote monitoring includes an image collection unit that receives a plurality of images from a camera unit, and an object in which movement occurs by analyzing each of the plurality of images according to a preset image analysis algorithm is detected.
  • An object extraction unit for extracting an object region for the object from the plurality of images; an image synthesis unit for generating a composite image obtained by combining at least one object region extracted from the object extraction unit into a single image; and the composite image
  • a deep learning unit that identifies the object to be monitored by analyzing through a deep learning algorithm in which a pattern for the object to be monitored is set in advance, and object information on the identified object from the deep learning unit is received, and the object information is It may include an event determination unit that determines that an event occurs when a set event occurrence condition is satisfied.
  • the object information includes a type of an object and a degree of similarity with the object to be monitored
  • the event determination unit includes a type of the object related to the object to be monitored in advance, and the type of the object according to the object information. If they coincide and the degree of similarity is equal to or greater than a preset reference value, it may be determined that an event occurs.
  • the event determination unit may further include an event notification unit for generating and outputting event information when an event occurs as a result of the determination of the event determination unit or transmitting it to a preset external device.
  • the object extraction unit generates a median image obtained by synthesizing the plurality of images, and the movement occurs among the plurality of images through a difference image from the median image for each of the plurality of images. It may be characterized in that the object region of the object is extracted for each image in which the object is detected.
  • the object extracting unit extracts the object region from the specific image along an outline of the region determined as the object from the specific image in which the object is detected, and the image combining unit It may be characterized in that one or more object regions extracted by the object extracting unit are synthesized into one composite image through a preset box filling problem.
  • the plurality of images may be images corresponding to an object detected by detection of a sensor configured in the camera unit or an image analysis by the camera unit.
  • An image analysis method for remote monitoring of a monitoring server that communicates with a camera unit through a communication network includes receiving a plurality of images from the camera unit, and analyzing each of the plurality of images in advance. Extracting an object region for the object from the plurality of images when an object in which movement has occurred by analyzing according to an algorithm is detected, and generating a composite image by combining the extracted one or more object regions into one image; Identifying an object to be monitored by analyzing the composite image through a deep learning algorithm in which a pattern for a predetermined object to be monitored is learned, receiving object information on the identified object, and generating an event in which the object information is preset When the condition is satisfied, it may include determining that an event has occurred.
  • the present invention provides an image analysis algorithm based on a real-time image at a number of frames per second due to a large data transmission distance and a low performance of the camera unit when transmitting a plurality of images to a monitoring server according to object detection from an image in a remote location. Even if an image is transmitted in the form of a snapshot that is insufficient to identify the object to be monitored through the camera unit, only the object region is separated and extracted from the plurality of images transmitted from the camera unit and synthesized into a single image.
  • the camera unit located in a remote location can be configured as a low-cost camera, thereby reducing system configuration costs and improving the reliability of the object analysis results. There is a guarantee effect.
  • the present invention does not analyze the entire area of each of the plurality of images received from the camera unit in the monitoring server through a deep learning algorithm, but separates only the object area where movement has occurred in each of the plurality of images, and then converts it into one image.
  • the analysis time required for object identification of the deep learning algorithm can be greatly shortened by analyzing a single composite image synthesized by using a deep learning algorithm.Through this, even if the number of camera units communicating with the monitoring server is large, In addition to supporting rapid event determination based on identification, even if the number of camera units increases, the number of monitoring servers for accommodating them and hardware performance can be reduced, thereby reducing system configuration cost.
  • FIG. 1 is a block diagram of an image analysis system for remote monitoring according to an embodiment of the present invention.
  • FIG. 2 is a detailed configuration diagram of a monitoring server configuring an image analysis system for remote monitoring according to an embodiment of the present invention.
  • 3 to 5 are diagrams illustrating operations for a process of extracting an object area and generating a composite image of a monitoring server according to an embodiment of the present invention.
  • 6 and 7 are diagrams illustrating operations of a monitoring server for identification of an object to be monitored and an event generation process according to an embodiment of the present invention.
  • FIG. 8 is a flowchart of an image analysis method for remote monitoring according to an embodiment of the present invention.
  • FIG. 1 is a configuration diagram of an image analysis system for remote monitoring according to an embodiment of the present invention, including a camera unit 10 located at a remote location and a monitoring server 100 communicating through a communication network as shown. Can be.
  • the camera unit 10 may be configured as an IP (Internet Protocol) camera.
  • the camera unit 10 may include a sensor unit such as a passive infrared sensor (PIR) or perform self-image analysis, and when detecting an object that satisfies a preset condition through the sensing signal or image analysis of the sensor unit A plurality of images in which the object is detected may be transmitted to the monitoring server 100 based on the object detection time point.
  • a sensor unit such as a passive infrared sensor (PIR) or perform self-image analysis
  • each of the plurality of images may be composed of a frame that is a snapshot.
  • the camera unit 10 has difficulty in transmitting a large amount of data related to a real-time high-definition image due to the distance from the monitoring server 100, so that the data capacity of the image, the data transmission speed of the camera unit 10, and the network environment Considering that, an image composed of a very small number of the snapshots per second may be generated and transmitted to the monitoring server 100 periodically during the time when an object is detected.
  • the camera unit 10 may generate and transmit an image composed of a snapshot of 2 frames per second.
  • the monitoring server 100 may receive a plurality of images according to the detection of one or more objects from the camera unit 10.
  • the monitoring server 100 detects and extracts an object moving in the image targeting a plurality of images in the form of a snapshot that is periodically transmitted instead of a real-time image, and selects an object to be monitored among one or more objects displayed in the plurality of images.
  • an event can be generated when a monitored object that satisfies a preset event condition is identified.
  • a moving person or vehicle may be set as the object to be monitored in the monitoring server 100.
  • the monitoring server 100 can easily and accurately identify the object to be monitored for 10 frames or less, not a real-time image of several tens of frames per second, and provide an event for this, and through this, a camera unit located at a remote location ( Even if 10) is configured as a low-cost, low-performance camera, it is possible to reduce system configuration cost and increase system reliability by supporting the monitoring server 100 to easily monitor the object to be monitored.
  • FIG. 2 is a configuration diagram of the monitoring server 100, as shown, an image collection unit 110, an object extraction unit 120, an image synthesis unit 130, a deep learning unit 140, and , An event determination unit 150 and an event notification unit 160 may be included.
  • At this time, at least one of the image collection unit 110, the object extraction unit 120, the image synthesis unit 130, the deep learning unit 140, the event determination unit 150, and the event notification unit 160 May be configured as a control unit that controls the monitoring server 100, and a component part of the monitoring server 100 other than the control unit may be controlled by the control unit.
  • control unit executes an overall control function of the monitoring server 100 using programs and data stored in the monitoring server 100.
  • the control unit may include RAM, ROM, CPU, GPU, and bus, and RAM, ROM, CPU, GPU, and the like may be connected to each other through a bus.
  • 3 to 5 are diagrams illustrating operations of a process of extracting an object area and generating a composite image of the monitoring server 100 according to an embodiment of the present invention.
  • the image collection unit 110 may receive a plurality of images from a camera unit 10 located at a remote location through a communication network.
  • the image collection unit 110 may store the plurality of images in the DB 101 included in the monitoring server 100.
  • the object extraction unit 120 receives a plurality of images from the image collection unit 110, detects an object in which movement has occurred in the plurality of images, and determines the object region of the detected object from the plurality of images. Can be extracted.
  • the object extraction unit 120 may analyze each of the plurality of images received from the image collection unit 110 through a preset image analysis algorithm to detect an object in which movement has occurred.
  • the object extracting unit 120 may apply a differential image method, a Model of Gaussian (MOG) algorithm using Gaussian Mixture Models (GMM), a codebook algorithm, and the like as the image analysis algorithm.
  • MOG Model of Gaussian
  • GMM Gaussian Mixture Models
  • the object extracting unit 120 may extract an object region corresponding to the specific object from each of at least one or more images in which the specific object is detected from among the plurality of images.
  • the object extraction unit 120 detects an object that has moved when the image analysis algorithm is applied to the plurality of images. Errors may occur in detection.
  • the object extraction unit 120 generates a median image obtained by synthesizing the plurality of images, and detects an object through a difference image from the median image for each of the plurality of images.
  • the object area of the object may be extracted for each image in which the moving object is detected among the plurality of images.
  • the object extracting unit 120 may prevent an error when analyzing an image targeting a plurality of images having a very small number of frames per second.
  • the object extraction unit 120 may prevent an error by generating an analysis target image obtained by obtaining a horizontal edge and a vertical edge for each of a plurality of images, and calculating the difference between the analysis target images to detect the object. have.
  • the object extraction unit 120 may detect one or more objects from each of the plurality of images.
  • the image synthesizing unit 130 may interwork with the object extracting unit 120 to collect an object area for each object extracted from the plurality of images, and combine the object area for each object into a single image. have.
  • the image synthesis unit 130 synthesizes one or more object regions for each object extracted by the object extraction unit 120 from at least one of a plurality of images through a preset bin packing problem. Can be combined into images.
  • the object extracting unit 120 for a plurality of images collected by the image collection unit 110 as shown in FIG. 4 When an object related to a moving vehicle is detected through a predetermined image analysis algorithm for a plurality of images, an object region for each object may be extracted from the plurality of images.
  • the object extracting unit 120 may also detect nonsensical objects such as a shadow of a vehicle to be monitored or a light reflected by the vehicle as a moving object, together with the object to be monitored, and detect an object area according to such a meaningless object.
  • the object region may be extracted from a specific image in which a moving object is detected along an outline of an area determined as the moving object.
  • the object extraction unit 120 is the object of the meaningless object, which is regarded as noise such as a change in a shadow or light attached to the object to be monitored when extracting the object area to a bounding box in the process of extracting the object-related object area. Due to the problem that the size of the bounding box of the object to be monitored increases more than necessary depending on the region, only the region related to the object in which movement has occurred can be extracted roughly along the outline without using the bounding box.
  • the image synthesis unit 130 may collect one or more object regions extracted from the plurality of images by the object extraction unit 120 and combine them into one image.
  • the image synthesizing unit 130 may apply one or more object regions to a preset box filling problem-related algorithm as shown in Fig. 5(a) and synthesize it into one image. As shown in ), a composite image in which the one or more object regions are combined may be generated.
  • the object extraction unit 120 may provide the position of the object in the image for each image in which the object is detected among a plurality of images to the image synthesis unit 130, and the image synthesis unit 130 ) May also record the position of the object in the image in the composite image.
  • FIGS. 6 and 7 are exemplary diagrams of the operation of the monitoring server 100 for identification of the object to be monitored and the event generation process according to the embodiment of the present invention.
  • the deep learning unit ( 140) receives a composite image from the image synthesizing unit 130, analyzes the composite image through a deep learning algorithm in which a preset pattern for the object to be monitored is learned, and identifies the object to be monitored from the composite image. have.
  • the deep learning unit 140 may continuously learn the composite images generated by the deep learning algorithm in response to images provided whenever an object is detected by the camera unit 10, and through the learning It is possible to learn the pattern of the object to be monitored in the learning algorithm.
  • the deep learning unit 140 may output object information for each object identified in the composite image through the deep learning algorithm, connected to the monitoring server 100 or through a separately configured output unit.
  • the deep learning algorithm receives feedback information selected by a user as an object to be monitored from among object information output from the deep learning unit 140 through the user interface unit 170 to be received, and the deep learning algorithm based on the feedback information By modifying, the identification error of the object to be monitored can be reduced so that a pattern for the object to be monitored can be learned.
  • the deep learning unit 140 may learn a pattern of an object to be monitored, such as a person or vehicle, to the deep learning algorithm.
  • the deep learning algorithm is preferably Regions with Convolutional Neural Network (R-CNN), but is not limited thereto, and various neural network models may be applied.
  • R-CNN Regions with Convolutional Neural Network
  • the user interface unit 170 may be included in the monitoring server 100 and configured.
  • the deep learning unit 140 analyzes one or more object regions included in the composite image through a deep learning algorithm, and among objects corresponding to the object region, the object to be monitored An object is identified through a deep learning algorithm, and object information including the object type of the object to be monitored and the similarity between the object type is generated in correspondence to the object area identified as the object to be monitored, and then the event determination unit 150 Can provide.
  • the deep learning unit 140 corresponds to a specific object area among one or more object areas included in the composite image, and when a specific object identified is a person-related monitoring object, the object type is The object information including a degree of similarity to a person may be generated in relation to the specific object set as a person and identified as a monitoring target object.
  • the object identified in correspondence with the specific object area is a vehicle-related monitoring object
  • the object type is set as a vehicle
  • the object type is set as a monitoring object.
  • the object information including a degree of similarity to a vehicle may be generated.
  • the composite image may include location information on a location (or location of an object) corresponding to the object area in the image corresponding to the object area for each object area, and the deep learning unit 140 Among the location information for each object area included in the image, location information of an object area corresponding to the specific object identified as the object to be monitored may be included in the object information corresponding to the specific object.
  • the deep learning unit 140 matches the object information generated for each object area identified as the object to be monitored among the one or more object areas with the object area corresponding to the object information in the composite image and adds it to the composite image.
  • a composite image including the object information may be provided to the event determination unit 150.
  • the event determination unit 150 receives object information on the identified object from the deep learning unit 140, and when the object information satisfies a preset event occurrence condition, a plurality of images transmitted from the camera unit 10 It can be determined as the occurrence of an event corresponding to the image of
  • the event determination unit 150 may determine that an event has occurred.
  • the event determination unit 150 sets an object type related to a person or vehicle as a monitoring target object among one or more object information provided from the deep learning unit 140, and If there is object information having a similarity with the person or vehicle equal to or greater than a preset reference value, it is determined that a preset event condition is satisfied, and the event is determined to have occurred in response to the camera unit 10 that transmitted the plurality of images. I can.
  • event notification unit 160 may interwork with the event determination unit 150 to generate event information when the event determination unit 150 determines that an event has occurred and output it through the output unit.
  • the event notification unit 160 may include a plurality of images corresponding to the event in the event information and output through the output unit.
  • the event notification unit 160 may transmit the event information to a preset external device through a communication network.
  • the event notification unit 160 may identify object information that satisfies the event occurrence condition in conjunction with the event determination unit 150, and monitor among a plurality of images based on the location information included in the object information. For each image in which the target object exists, a plurality of images in which the location of the object to be monitored is marked with a preset mark may be generated and then included in the event information and transmitted.
  • the monitoring server 100 may communicate with a plurality of different camera units 10 through a communication network.
  • the image collection unit 110 configured in the monitoring server 100 may allocate different channels to each of the plurality of camera units 10 and receive a plurality of images for each channel.
  • the monitoring server 100 may classify the plurality of camera units 10 through a channel, and individually determine whether an event occurs for each of the plurality of camera units 10 as described above.
  • the present invention provides an event by determining whether the object is a monitoring target object in a monitoring server for an object detected as an event by the camera unit, according to object detection in an image by a camera unit located at a remote location.
  • the data transmission distance is significant and the number of frames per second is transmitted in the form of a snapshot that is insufficient to identify the object to be monitored through an image analysis algorithm based on real-time images due to the low performance of the camera unit.
  • whether the object detected by the camera is easily detected by the camera unit through a deep learning algorithm after separating and extracting only the object area from the plurality of images transmitted from the camera unit and combining it into one image.
  • the present invention does not analyze the entire area of each of the plurality of images received from the camera unit in the monitoring server through a deep learning algorithm, but separates only the object area where movement has occurred in each of the plurality of images, and then converts it into one image.
  • the analysis time required for object identification of the deep learning algorithm can be greatly shortened by analyzing a single composite image synthesized by using a deep learning algorithm.Through this, even if the number of camera units communicating with the monitoring server is large, In addition to supporting rapid event determination based on identification, even if the number of camera units increases, the number of monitoring servers and hardware performance for accommodating them can be reduced, thereby reducing system configuration cost.
  • FIG. 8 is a flowchart illustrating an image analysis method for remote monitoring of a monitoring server communicating with a camera unit through a communication network according to an embodiment of the present invention.
  • the monitoring server 100 may receive a plurality of images from the camera unit 10 (S1).
  • the monitoring server 100 analyzes each of the plurality of images according to a preset image analysis algorithm (S2) and extracts an object region for the object from the plurality of images when an object in which movement has occurred (S3) is detected. I can (S4).
  • the monitoring server 100 may generate a composite image obtained by combining the extracted one or more object regions into one image (S5).
  • the monitoring server 100 may identify the object to be monitored by analyzing the composite image through a deep learning algorithm in which a pattern for the object to be monitored is set in advance (S6).
  • the monitoring server 100 may receive object information on the identified object, determine that an event has occurred when the object information satisfies a preset event occurrence condition, and output event information according to the event occurrence or
  • the event information may be transmitted to a set external device (S7).
  • CMOS-based logic circuitry CMOS-based logic circuitry
  • firmware software
  • software or a combination thereof.
  • transistors logic gates, and electronic circuits in the form of various electrical structures.

Abstract

The present invention relates to an image analysis system and method for remote monitoring, whereby objects identified in images received periodically rather than in real time from a camera unit positioned at a remote location can be analyzed on the basis of deep learning to provide an event when an object to be monitored is detected, and thus the time required to analyze the object to be monitored can be reduced and costs can be lowered.

Description

원격 모니터링을 위한 영상 분석 시스템 및 방법Video analysis system and method for remote monitoring
본 발명은 원격 모니터링을 위한 영상 분석 시스템 및 방법에 관한 것으로서, 더욱 상세히는 원격지에 위치하는 카메라부로부터 실시간이 아닌 주기적으로 수신되는 영상에서 식별된 객체를 딥러닝 기반으로 분석하여 감시 대상 객체 검출시 이벤트를 제공하여 감시 대상 객체의 분석에 소요되는 시간을 단축하고 비용을 절감할 수 있는 원격 모니터링을 위한 영상 분석 시스템 및 방법에 관한 것이다.The present invention relates to an image analysis system and method for remote monitoring, and more particularly, when detecting an object to be monitored by analyzing an object identified in an image periodically received from a camera unit located in a remote location instead of in real time, based on deep learning. The present invention relates to an image analysis system and method for remote monitoring capable of reducing time and cost for analyzing a monitored object by providing an event.
최근 영상 전송 및 영상 분석 관련 기술과 이를 지원하는 카메라를 비롯한 다양한 영상 관련 장치의 발전과 더불어 원격지에 위치하는 카메라로부터 고화질의 영상을 수신하고, 해당 고화질의 영상을 분석하여 감시 대상 객체를 검출하는 모니터링 시스템의 발전이 지속적으로 이루어지고 있다.With the recent development of video transmission and video analysis related technologies and various video related devices, including cameras that support them, monitoring that receives high-quality images from cameras located in remote locations and analyzes the high-quality images to detect objects to be monitored. The development of the system is continuing.
그러나, 기존의 모니터링 시스템에 적용되는 감시 대상 객체의 추적 및 검출을 위한 영상 분석 방식은 대부분 실시간으로 수신되는 영상을 전제로 하기 때문에 수km 이상 원격지에 위치하는 카메라가 고성능이라 하더라도 초당 수십 프레임이 요구되는 고화질의 실시간 영상을 전송하는데 어려움이 있을 뿐만 아니라 이를 지원하기 위한 카메라 구성에 상당한 비용이 요구되므로 모니터링 시스템을 구성하는데 있어서 영상을 모니터링하는 모니터링 서버에 영상을 전송하도록 원거리에 위치하는 카메라를 운영하는 사용자는 이러한 고성능의 카메라를 채택하는데 어려움이 있다.However, video analysis methods for tracking and detection of objects to be monitored applied to existing monitoring systems presuppose images received in real time, so even if cameras located in remote locations over several kilometers are high performance, tens of frames per second are required. Not only is it difficult to transmit high-definition real-time images, but also a considerable cost is required to configure cameras to support them, so in configuring a monitoring system, a remotely located camera is operated to transmit images to a monitoring server that monitors the images. Users have difficulty in adopting such a high-performance camera.
이에 따라, 사용자는 모니터링 서버와 연결되는 카메라로 저가의 카메라를 사용하며, 이러한 저가의 카메라는 초당 매우 작은 프레임수로 영상을 전송하므로 모니터링 서버에서 실시간 영상을 전제로 하는 영상 분석 방식을 기초로 영상 분석시 잘못된 알람을 빈번하게 출력하게 되고, 이로 인해 모니터링 시스템에 대한 신뢰도를 저하시키는 문제가 발생한다.Accordingly, users use low-cost cameras as cameras connected to the monitoring server, and these low-cost cameras transmit images at a very small frame rate per second. During analysis, false alarms are frequently output, which causes a problem of lowering the reliability of the monitoring system.
상술한 문제를 해결하기 위해, 본 발명은 원격지에 위치하여 초당 전송 프레임수가 매우 적은 카메라로부터 수신되는 영상에 대해 딥러닝 기반의 영상 분석 방식을 적용하여 영상에서 감시 대상 객체를 용이하게 검출하고 이에 따른 이벤트를 제공할 수 있도록 지원함과 아울러 딥러닝 기반의 영상 분석시 영상 분석에 소요되는 시간을 단축하여 영상 모니터링에 대한 시스템의 효율성 및 신뢰도를 높이는데 그 목적이 있다.In order to solve the above-described problem, the present invention applies a deep learning-based image analysis method to an image received from a camera located at a remote location and has a very small number of transmission frames per second to easily detect an object to be monitored in the image. The purpose of this is to increase the efficiency and reliability of the system for video monitoring by providing support to provide events and shortening the time required for video analysis during deep learning-based video analysis.
또한, 본 발명은 초당 전송 프레임수가 매우 적은 저가의 카메라를 이용하더라도 감시 대상 객체에 대한 검출이 신뢰성 있게 이루어지도록 지원하여 시스템 구성 비용을 절감할 수 있도록 지원하는데 그 목적이 있다.In addition, an object of the present invention is to support to reduce system configuration cost by supporting reliable detection of an object to be monitored even when a low-cost camera with a very small number of transmission frames per second is used.
본 발명의 실시예에 따른 원격 모니터링을 위한 영상 분석 시스템은 카메라부로부터 복수의 영상을 수신하는 영상 수집부와, 상기 복수의 영상 각각을 미리 설정된 영상 분석 알고리즘에 따라 분석하여 이동이 발생한 객체 검출시 상기 객체에 대한 객체 영역을 상기 복수의 영상에서 추출하는 객체 추출부와, 상기 객체 추출부에서 추출된 하나 이상의 객체 영역을 하나의 이미지에 합성한 합성 이미지를 생성하는 이미지 합성부와, 상기 합성 이미지를 미리 설정된 감시 대상 객체에 대한 패턴이 학습된 딥러닝 알고리즘을 통해 분석하여 감시 대상 객체를 식별하는 딥러닝부 및 상기 딥러닝부로부터 식별된 객체에 대한 객체 정보를 수신하고, 상기 객체 정보가 미리 설정된 이벤트 발생 조건 만족시 이벤트 발생으로 판단하는 이벤트 판단부를 포함할 수 있다.An image analysis system for remote monitoring according to an embodiment of the present invention includes an image collection unit that receives a plurality of images from a camera unit, and an object in which movement occurs by analyzing each of the plurality of images according to a preset image analysis algorithm is detected. An object extraction unit for extracting an object region for the object from the plurality of images; an image synthesis unit for generating a composite image obtained by combining at least one object region extracted from the object extraction unit into a single image; and the composite image A deep learning unit that identifies the object to be monitored by analyzing through a deep learning algorithm in which a pattern for the object to be monitored is set in advance, and object information on the identified object from the deep learning unit is received, and the object information is It may include an event determination unit that determines that an event occurs when a set event occurrence condition is satisfied.
본 발명과 관련된 일 예로서, 상기 객체 정보는 객체의 종류 및 감시 대상 객체와의 유사도를 포함하고, 상기 이벤트 판단부는 상기 객체 정보에 따른 객체의 종류가 미리 설정된 상기 감시 대상 객체 관련 객체의 종류와 일치하고, 상기 유사도가 미리 설정된 기준치 이상인 경우 이벤트 발생으로 판단하는 것을 특징으로 할 수 있다.As an example related to the present invention, the object information includes a type of an object and a degree of similarity with the object to be monitored, and the event determination unit includes a type of the object related to the object to be monitored in advance, and the type of the object according to the object information. If they coincide and the degree of similarity is equal to or greater than a preset reference value, it may be determined that an event occurs.
본 발명과 관련된 일 예로서, 상기 이벤트 판단부의 판단 결과 이벤트 발생시 이벤트 정보를 생성하여 출력하거나 미리 설정된 외부 장치로 전송하는 이벤트 알림부를 더 포함하는 것을 특징으로 할 수 있다.As an example related to the present invention, the event determination unit may further include an event notification unit for generating and outputting event information when an event occurs as a result of the determination of the event determination unit or transmitting it to a preset external device.
본 발명과 관련된 일 예로서, 상기 객체 추출부는 상기 복수의 영상을 합성한 메디안 이미지를 생성하고, 상기 복수의 영상 각각에 대해 상기 메디안 이미지와의 차분 영상을 통해 상기 복수의 영상 중 상기 이동이 발생한 객체가 검출되는 영상별로 상기 객체의 객체 영역을 추출하는 것을 특징으로 할 수 있다.As an example related to the present invention, the object extraction unit generates a median image obtained by synthesizing the plurality of images, and the movement occurs among the plurality of images through a difference image from the median image for each of the plurality of images. It may be characterized in that the object region of the object is extracted for each image in which the object is detected.
본 발명과 관련된 일 예로서, 상기 객체 추출부는 상기 객체가 검출된 특정 영상에서 상기 객체로 판단되는 영역의 외곽선을 따라 상기 특정 영상에서 상기 객체 영역을 추출하고, 상기 이미지 합성부는 복수의 영상 중 적어도 하나로부터 상기 객체 추출부에 의해 추출되는 하나 이상의 객체 영역을 미리 설정된 상자 채우기 문제를 통해 하나의 합성 이미지로 합성하는 것을 특징으로 할 수 있다.As an example related to the present invention, the object extracting unit extracts the object region from the specific image along an outline of the region determined as the object from the specific image in which the object is detected, and the image combining unit It may be characterized in that one or more object regions extracted by the object extracting unit are synthesized into one composite image through a preset box filling problem.
본 발명과 관련된 일 예로서, 상기 복수의 영상은 상기 카메라부에 구성된 센서의 감지 또는 상기 카메라부에 의한 영상 분석에 따라 검출된 객체에 대응되는 영상인 것을 특징으로 할 수 있다.As an example related to the present invention, the plurality of images may be images corresponding to an object detected by detection of a sensor configured in the camera unit or an image analysis by the camera unit.
본 발명의 실시예에 따른 카메라부와 통신망을 통해 통신하는 모니터링 서버의 원격 모니터링을 위한 영상 분석 방법은, 상기 카메라부로부터 복수의 영상을 수신하는 단계와, 상기 복수의 영상 각각을 미리 설정된 영상 분석 알고리즘에 따라 분석하여 이동이 발생한 객체 검출시 상기 객체에 대한 객체 영역을 상기 복수의 영상에서 추출하는 단계와, 상기 추출된 하나 이상의 객체 영역을 하나의 이미지에 합성한 합성 이미지를 생성하는 단계와, 상기 합성 이미지를 미리 설정된 감시 대상 객체에 대한 패턴이 학습된 딥러닝 알고리즘을 통해 분석하여 감시 대상 객체를 식별하는 단계 및 상기 식별된 객체에 대한 객체 정보를 수신하고, 상기 객체 정보가 미리 설정된 이벤트 발생 조건 만족시 이벤트 발생으로 판단하는 단계를 포함할 수 있다.An image analysis method for remote monitoring of a monitoring server that communicates with a camera unit through a communication network according to an embodiment of the present invention includes receiving a plurality of images from the camera unit, and analyzing each of the plurality of images in advance. Extracting an object region for the object from the plurality of images when an object in which movement has occurred by analyzing according to an algorithm is detected, and generating a composite image by combining the extracted one or more object regions into one image; Identifying an object to be monitored by analyzing the composite image through a deep learning algorithm in which a pattern for a predetermined object to be monitored is learned, receiving object information on the identified object, and generating an event in which the object information is preset When the condition is satisfied, it may include determining that an event has occurred.
본 발명은 원격지에 위치하는 카메라부에서 영상에서 객체 검출에 따라 복수의 영상을 모니터링 서버에 전송시 데이터 전송거리가 상당하고 카메라부의 낮은 성능으로 인해 초당 프레임수가 실시간 영상을 기반으로 하는 영상 분석 알고리즘을 통해 감시 대상 객체를 식별하는데 충분치 않은 스냅샷 형태로 영상을 전송하는 경우라도 카메라부에서 전송한 복수의 영상에서 객체 영역만을 분리 추출하여 하나의 이미지로 합성한 후 딥러닝 알고리즘을 통해 용이하게 카메라부에서 검출한 객체가 이벤트 관련 감시 대상 객체인지 여부를 정확하게 검출할 수 있도록 지원함으로써, 원격지에 위치하는 카메라부를 저가의 카메라로 구성할 수 있도록 지원하여 시스템 구성 비용을 절감하면서도 객체 분석 결과에 대한 신뢰성을 보장하는 효과가 있다.The present invention provides an image analysis algorithm based on a real-time image at a number of frames per second due to a large data transmission distance and a low performance of the camera unit when transmitting a plurality of images to a monitoring server according to object detection from an image in a remote location. Even if an image is transmitted in the form of a snapshot that is insufficient to identify the object to be monitored through the camera unit, only the object region is separated and extracted from the plurality of images transmitted from the camera unit and synthesized into a single image. By supporting the ability to accurately detect whether the object detected by the device is an object to be monitored related to an event, the camera unit located in a remote location can be configured as a low-cost camera, thereby reducing system configuration costs and improving the reliability of the object analysis results. There is a guarantee effect.
또한, 본 발명은 모니터링 서버에서 카메라부로부터 수신된 복수의 영상 각각의 전체 영역을 대상으로 딥러닝 알고리즘을 통해 분석하는 것이 아닌 복수의 영상 각각에서 움직임이 발생한 객체 영역만을 분리한 후 이를 하나의 이미지로 합성한 합성 이미지 1장을 대상으로 딥러닝 알고리즘을 통해 분석함으로써 딥러닝 알고리즘의 객체 식별에 필요한 분석 시간을 크게 단축시킬 수 있으며, 이를 통해 모니터링 서버와 통신하는 카메라부의 수가 상당하더라도 용이하게 객체의 식별에 따른 이벤트 판단이 신속하게 이루어지도록 지원함과 아울러 카메라부의 수가 증가하더라도 이를 수용하기 위한 모니터링 서버의 개수와 하드웨어적인 성능을 낮출 수 있어 시스템 구성 비용을 절감시키는 효과가 있다.In addition, the present invention does not analyze the entire area of each of the plurality of images received from the camera unit in the monitoring server through a deep learning algorithm, but separates only the object area where movement has occurred in each of the plurality of images, and then converts it into one image. The analysis time required for object identification of the deep learning algorithm can be greatly shortened by analyzing a single composite image synthesized by using a deep learning algorithm.Through this, even if the number of camera units communicating with the monitoring server is large, In addition to supporting rapid event determination based on identification, even if the number of camera units increases, the number of monitoring servers for accommodating them and hardware performance can be reduced, thereby reducing system configuration cost.
도 1은 본 발명의 실시예에 따른 원격 모니터링을 위한 영상 분석 시스템의 구성도.1 is a block diagram of an image analysis system for remote monitoring according to an embodiment of the present invention.
도 2는 본 발명의 실시예에 따른 원격 모니터링을 위한 영상 분석 시스템을 구성하는 모니터링 서버의 상세 구성도.2 is a detailed configuration diagram of a monitoring server configuring an image analysis system for remote monitoring according to an embodiment of the present invention.
도 3 내지 도 5는 본 발명의 실시예에 따른 모니터링 서버의 객체 영역 추출 및 합성 이미지 생성 과정에 대한 동작 예시도.3 to 5 are diagrams illustrating operations for a process of extracting an object area and generating a composite image of a monitoring server according to an embodiment of the present invention.
도 6 및 도 7은 본 발명의 실시예에 따른 모니터링 서버의 감시 대상 객체의 식별 및 이벤트 발생 과정에 대한 동작 예시도.6 and 7 are diagrams illustrating operations of a monitoring server for identification of an object to be monitored and an event generation process according to an embodiment of the present invention.
도 8은 본 발명의 실시예에 따른 원격 모니터링을 위한 영상 분석 방법에 대한 순서도.8 is a flowchart of an image analysis method for remote monitoring according to an embodiment of the present invention.
이하, 도면을 참고하여 본 발명의 상세 실시예를 설명한다.Hereinafter, detailed embodiments of the present invention will be described with reference to the drawings.
도 1은 본 발명의 실시예에 따른 원격 모니터링을 위한 영상 분석 시스템의 구성도로서, 도시된 바와 같이 원격지에 위치하는 카메라부(10)와 통신망을 통해 통신하는 모니터링 서버(100)를 포함하여 구성될 수 있다.1 is a configuration diagram of an image analysis system for remote monitoring according to an embodiment of the present invention, including a camera unit 10 located at a remote location and a monitoring server 100 communicating through a communication network as shown. Can be.
이때, 상기 카메라부(10)는 IP(Internet Protocol) 카메라로 구성될 수 있다.In this case, the camera unit 10 may be configured as an IP (Internet Protocol) camera.
또한, 상기 카메라부(10)는 PIR 센서(passive infrared sensor)와 같은 센서부를 포함하거나 자체 영상 분석을 수행할 수 있으며, 상기 센서부의 센싱 신호 또는 영상 분석을 통해 미리 설정된 조건을 만족하는 객체 검출시 객체 검출 시점을 기준으로 상기 객체가 검출되는 복수의 영상을 상기 모니터링 서버(100)에 전송할 수 있다.In addition, the camera unit 10 may include a sensor unit such as a passive infrared sensor (PIR) or perform self-image analysis, and when detecting an object that satisfies a preset condition through the sensing signal or image analysis of the sensor unit A plurality of images in which the object is detected may be transmitted to the monitoring server 100 based on the object detection time point.
이때, 상기 복수의 영상 각각은 스냅샷(snapshot)인 프레임으로 구성될 수 있다.In this case, each of the plurality of images may be composed of a frame that is a snapshot.
즉, 상기 카메라부(10)는 모니터링 서버(100)와의 거리로 인해 실시간 고화질 영상 관련 대용량의 데이터를 전송하는데 어려움이 있어, 영상의 데이터 용량과 카메라부(10)의 데이터 전송 속도 및 네트워크 환경을 고려하여 초당 매우 적은 수의 상기 스냅샷으로 구성된 영상을 생성하여 객체가 검출되는 시간 동안 주기적으로 상기 모니터링 서버(100)로 전송할 수 있다.That is, the camera unit 10 has difficulty in transmitting a large amount of data related to a real-time high-definition image due to the distance from the monitoring server 100, so that the data capacity of the image, the data transmission speed of the camera unit 10, and the network environment Considering that, an image composed of a very small number of the snapshots per second may be generated and transmitted to the monitoring server 100 periodically during the time when an object is detected.
일례로, 상기 카메라부(10)는 초당 2프레임씩 스냅샷으로 구성된 영상을 생성하여 전송할 수 있다.As an example, the camera unit 10 may generate and transmit an image composed of a snapshot of 2 frames per second.
또한, 상기 모니터링 서버(100)는 상기 카메라부(10)로부터 하나 이상의 상기 객체 검출에 따른 복수의 영상을 수신할 수 있다.In addition, the monitoring server 100 may receive a plurality of images according to the detection of one or more objects from the camera unit 10.
상기 모니터링 서버(100)는 실시간 영상이 아닌 주기적으로 전송되는 스냅샷 형태의 복수의 영상을 대상으로 영상에서 이동하는 객체를 검출하여 추출하고, 상기 복수의 영상에 나타난 하나 이상의 객체 중 감시 대상 객체를 딥러닝(deep learning) 기반으로 식별하여 미리 설정된 이벤트 조건을 만족하는 감시 대상 객체 식별시 이벤트를 발생시킬 수 있다.The monitoring server 100 detects and extracts an object moving in the image targeting a plurality of images in the form of a snapshot that is periodically transmitted instead of a real-time image, and selects an object to be monitored among one or more objects displayed in the plurality of images. By identifying based on deep learning, an event can be generated when a monitored object that satisfies a preset event condition is identified.
이때, 본 발명에서 설명하는 감시 대상 객체의 일례로서, 이동하는 사람이나 차량 등이 상기 모니터링 서버(100)에 감시 대상 객체로 설정될 수 있다.In this case, as an example of the object to be monitored described in the present invention, a moving person or vehicle may be set as the object to be monitored in the monitoring server 100.
즉, 모니터링 서버(100)는 초당 수십 프레임의 실시간 영상이 아닌 10개 이하의 프레임에 대해서도 감시 대상 객체를 용이하고 정확하게 식별하여 이에 대한 이벤트를 제공할 수 있으며, 이를 통해 원격지에 위치하는 카메라부(10)를 저가의 저성능 카메라로 구성하더라도 용이하게 모니터링 서버(100)에서 감시 대상 객체에 대한 모니터링이 이루어지도록 지원하여 시스템 구성 비용을 절감하고 시스템 신뢰도를 높일 수 있다.That is, the monitoring server 100 can easily and accurately identify the object to be monitored for 10 frames or less, not a real-time image of several tens of frames per second, and provide an event for this, and through this, a camera unit located at a remote location ( Even if 10) is configured as a low-cost, low-performance camera, it is possible to reduce system configuration cost and increase system reliability by supporting the monitoring server 100 to easily monitor the object to be monitored.
상술한 구성을 토대로, 이하 도면을 통해 본 발명의 실시예에 따른 원격 모니터링을 위한 영상 분석 시스템을 구성하는 모니터링 서버(100)의 상세 구성을 설명한다.Based on the above-described configuration, a detailed configuration of the monitoring server 100 constituting an image analysis system for remote monitoring according to an embodiment of the present invention will be described below with reference to the drawings.
도 2는 상기 모니터링 서버(100)의 구성도로서, 도시된 바와 같이, 영상 수집부(110)와, 객체 추출부(120)와, 이미지 합성부(130)와, 딥러닝부(140)와, 이벤트 판단부(150) 및 이벤트 알림부(160)를 포함하여 구성될 수 있다.2 is a configuration diagram of the monitoring server 100, as shown, an image collection unit 110, an object extraction unit 120, an image synthesis unit 130, a deep learning unit 140, and , An event determination unit 150 and an event notification unit 160 may be included.
이때, 영상 수집부(110)와, 객체 추출부(120)와, 이미지 합성부(130)와, 딥러닝부(140)와, 이벤트 판단부(150) 및 이벤트 알림부(160) 중 적어도 하나가 상기 모니터링 서버(100)를 제어하는 제어부로서 구성될 수 있으며, 상기 제어부가 아닌 상기 모니터링 서버(100)의 구성부가 상기 제어부에 의해 제어될 수 있다.At this time, at least one of the image collection unit 110, the object extraction unit 120, the image synthesis unit 130, the deep learning unit 140, the event determination unit 150, and the event notification unit 160 May be configured as a control unit that controls the monitoring server 100, and a component part of the monitoring server 100 other than the control unit may be controlled by the control unit.
또한, 상기 제어부는 모니터링 서버(100)에 저장된 프로그램 및 데이터를 이용하여 상기 모니터링 서버(100)의 전반적인 제어 기능을 실행한다. 제어부는 RAM, ROM, CPU, GPU, 버스를 포함할 수 있으며, RAM, ROM, CPU, GPU 등은 버스를 통해 서로 연결될 수 있다.In addition, the control unit executes an overall control function of the monitoring server 100 using programs and data stored in the monitoring server 100. The control unit may include RAM, ROM, CPU, GPU, and bus, and RAM, ROM, CPU, GPU, and the like may be connected to each other through a bus.
상술한 모니터링 서버(100)의 각 구성부의 상세 동작 구성을 도 3 내지 도 7을 통해 설명한다.Detailed operation configurations of each component of the monitoring server 100 will be described with reference to FIGS. 3 to 7.
도 3 내지 도 5는 본 발명의 실시예에 따른 모니터링 서버(100)의 객체 영역 추출 및 합성 이미지 생성 과정에 대한 동작 예시도이다.3 to 5 are diagrams illustrating operations of a process of extracting an object area and generating a composite image of the monitoring server 100 according to an embodiment of the present invention.
우선, 도 3에 도시된 바와 같이, 상기 영상 수집부(110)는 원격지에 위치하는 카메라부(10)로부터 통신망을 통해 복수의 영상을 수신할 수 있다.First, as shown in FIG. 3, the image collection unit 110 may receive a plurality of images from a camera unit 10 located at a remote location through a communication network.
이때, 상기 영상 수집부(110)는 상기 모니터링 서버(100)에 포함된 DB(101)에 상기 복수의 영상을 저장할 수도 있다.In this case, the image collection unit 110 may store the plurality of images in the DB 101 included in the monitoring server 100.
또한, 상기 객체 추출부(120)는 상기 영상 수집부(110)로부터 복수의 영상을 수신하고, 상기 복수의 영상에서 이동이 발생한 객체를 검출하여 상기 복수의 영상으로부터 상기 검출된 객체의 객체 영역을 추출할 수 있다.In addition, the object extraction unit 120 receives a plurality of images from the image collection unit 110, detects an object in which movement has occurred in the plurality of images, and determines the object region of the detected object from the plurality of images. Can be extracted.
이때, 상기 객체 추출부(120)는 상기 영상 수집부(110)로부터 수신된 복수의 영상 각각을 미리 설정된 영상 분석 알고리즘을 통해 분석하여 이동이 발생한 객체를 검출할 수 있다.In this case, the object extraction unit 120 may analyze each of the plurality of images received from the image collection unit 110 through a preset image analysis algorithm to detect an object in which movement has occurred.
또한, 상기 객체 추출부(120)는 상기 영상 분석 알고리즘으로 차분영상 방법, GMM(Gaussian Mixture Models)을 이용하는 MOG(Model of Gaussian) 알고리즘, 코드북(Codebook) 알고리즘 등을 적용할 수 있다.In addition, the object extracting unit 120 may apply a differential image method, a Model of Gaussian (MOG) algorithm using Gaussian Mixture Models (GMM), a codebook algorithm, and the like as the image analysis algorithm.
또한, 상기 객체 추출부(120)는 이동이 발생한 특정 객체 검출시 상기 복수의 영상 중 상기 특정 객체가 검출된 적어도 하나 이상의 영상 각각에서 상기 특정 객체에 대응되는 객체 영역을 추출할 수 있다.In addition, when a specific object in which movement has occurred is detected, the object extracting unit 120 may extract an object region corresponding to the specific object from each of at least one or more images in which the specific object is detected from among the plurality of images.
이때, 상기 카메라부(10)로부터 전송된 복수의 영상은 초당 프레임수가 2프레임 정도로 매우 적으므로, 상기 객체 추출부(120)는 복수의 영상을 대상으로 상기 영상 분석 알고리즘 적용시 이동이 발생한 객체를 검출하는데 있어서 오류가 발생할 수 있다.At this time, since the plurality of images transmitted from the camera unit 10 has a very small number of frames per second of about 2 frames, the object extraction unit 120 detects an object that has moved when the image analysis algorithm is applied to the plurality of images. Errors may occur in detection.
이러한 오류를 방지하기 위해, 상기 객체 추출부(120)는 상기 복수의 영상을 합성한 메디안(median) 이미지를 생성하고, 상기 복수의 영상 각각에 대해 상기 메디안 이미지와의 차분 영상을 통해 객체를 검출할 수 있으며, 상기 복수의 영상 중 상기 이동이 발생한 객체가 검출되는 영상별로 상기 객체의 객체 영역을 추출할 수 있다.In order to prevent such an error, the object extraction unit 120 generates a median image obtained by synthesizing the plurality of images, and detects an object through a difference image from the median image for each of the plurality of images. The object area of the object may be extracted for each image in which the moving object is detected among the plurality of images.
이를 통해, 상기 객체 추출부(120)는 초당 프레임수가 매우 적은 복수의 영상을 대상으로 영상 분석시 오류를 방지할 수 있다.Through this, the object extracting unit 120 may prevent an error when analyzing an image targeting a plurality of images having a very small number of frames per second.
이외에도, 상기 객체 추출부(120)는 복수의 영상 각각에 대해 수평 엣지와 수직 엣지를 구한 분석 대상 영상을 생성하고, 상기 분석 대상 영상들간의 차이를 산출하여 상기 객체를 검출함으로써 오류를 방지할 수도 있다.In addition, the object extraction unit 120 may prevent an error by generating an analysis target image obtained by obtaining a horizontal edge and a vertical edge for each of a plurality of images, and calculating the difference between the analysis target images to detect the object. have.
또한, 상기 객체 추출부(120)는 상기 복수의 영상 각각에서 하나 이상의 객체를 검출할 수도 있다.Also, the object extraction unit 120 may detect one or more objects from each of the plurality of images.
한편, 상기 이미지 합성부(130)는 상기 객체 추출부(120)와 연동하여 상기 복수의 영상에서 추출된 하나 이상의 객체별 객체 영역을 수집하고, 상기 객체별 객체 영역을 하나의 이미지로 합성할 수 있다.Meanwhile, the image synthesizing unit 130 may interwork with the object extracting unit 120 to collect an object area for each object extracted from the plurality of images, and combine the object area for each object into a single image. have.
이때, 상기 이미지 합성부(130)는 복수의 영상 중 적어도 하나로부터 상기 객체 추출부(120)에 의해 추출되는 하나 이상의 객체별 객체 영역을 미리 설정된 상자 채우기 문제(Bin packing problem)를 통해 하나의 합성 이미지로 합성할 수 있다.At this time, the image synthesis unit 130 synthesizes one or more object regions for each object extracted by the object extraction unit 120 from at least one of a plurality of images through a preset bin packing problem. Can be combined into images.
상술한 동작 내용에 대한 상세 일례를 도 4 및 도 5를 통해 설명하면, 도 4에 도시된 바와 같이, 영상 수집부(110)에 의해 수집된 복수의 영상에 대해 상기 객체 추출부(120)는 복수의 영상을 대상으로 미리 설정된 영상 분석 알고리즘을 통해 이동하는 차량 관련 객체 검출시 객체별 객체 영역을 상기 복수의 영상에서 추출할 수 있다.When a detailed example of the above-described operation content is described with reference to FIGS. 4 and 5, the object extracting unit 120 for a plurality of images collected by the image collection unit 110 as shown in FIG. 4 When an object related to a moving vehicle is detected through a predetermined image analysis algorithm for a plurality of images, an object region for each object may be extracted from the plurality of images.
이때, 객체 추출부(120)는 감시 대상 객체인 차량의 그림자나 차량에 의해 반사되는 불빛과 같은 무의미한 객체 역시 이동하는 객체로서 감시 대상 객체와 함께 검출할 수 있으며, 이러한 무의미한 객체에 따른 객체 영역을 최소화하기 위해 이동 객체가 검출된 특정 영상에서 상기 이동 객체로 판단되는 영역의 외곽선을 따라 상기 객체 영역을 추출할 수 있다.In this case, the object extracting unit 120 may also detect nonsensical objects such as a shadow of a vehicle to be monitored or a light reflected by the vehicle as a moving object, together with the object to be monitored, and detect an object area according to such a meaningless object. To minimize, the object region may be extracted from a specific image in which a moving object is detected along an outline of an area determined as the moving object.
즉, 상기 객체 추출부(120)는 객체 관련 객체 영역의 추출 과정에서 바운딩 박스(bounding box)로 객체 영역 추출시 감시 대상 객체에 붙은 그림자나 불빛의 변화와 같은 노이즈로 간주되는 상기 무의미한 객체의 객체 영역에 따라 감시 대상 객체의 바운딩 박스의 크기가 필요 이상으로 커지는 문제로 인해 이러한 바운딩 박스를 이용하지 않고 움직임이 발생한 객체와 관련된 영역만을 외곽선을 따라 러프(rough)하게 추출할 수 있다.That is, the object extraction unit 120 is the object of the meaningless object, which is regarded as noise such as a change in a shadow or light attached to the object to be monitored when extracting the object area to a bounding box in the process of extracting the object-related object area. Due to the problem that the size of the bounding box of the object to be monitored increases more than necessary depending on the region, only the region related to the object in which movement has occurred can be extracted roughly along the outline without using the bounding box.
또한, 도 5에 도시된 바와 같이, 상기 이미지 합성부(130)는 상기 객체 추출부(120)에 의해 상기 복수의 영상에서 추출된 하나 이상의 객체 영역을 수집하여 하나의 이미지에 합성할 수 있다.In addition, as shown in FIG. 5, the image synthesis unit 130 may collect one or more object regions extracted from the plurality of images by the object extraction unit 120 and combine them into one image.
이때, 상기 이미지 합성부(130)는 도 5(a)에 도시된 바와 같이 하나 이상의 객체 영역을 미리 설정된 상자 채우기 문제 관련 알고리즘에 적용하여 하나의 이미지에 합성할 수 있으며, 이를 통해 도 5(b)에 도시된 바와 같이 상기 하나 이상의 객체 영역이 합성된 합성 이미지를 생성할 수 있다.In this case, the image synthesizing unit 130 may apply one or more object regions to a preset box filling problem-related algorithm as shown in Fig. 5(a) and synthesize it into one image. As shown in ), a composite image in which the one or more object regions are combined may be generated.
상술한 구성에서, 상기 객체 추출부(120)는 복수의 영상 중 객체가 검출된 영상별로 영상 내에서의 객체의 위치를 상기 이미지 합성부(130)에 제공할 수 있으며, 상기 이미지 합성부(130)는 상기 영상 내에서의 객체의 위치 또한 상기 합성 이미지에 기록할 수 있다.In the above-described configuration, the object extraction unit 120 may provide the position of the object in the image for each image in which the object is detected among a plurality of images to the image synthesis unit 130, and the image synthesis unit 130 ) May also record the position of the object in the image in the composite image.
한편, 도 6 및 도 7은 본 발명의 실시예에 따른 모니터링 서버(100)의 감시 대상 객체의 식별 및 이벤트 발생 과정에 대한 동작 예시도로서, 도 6에 도시된 바와 같이, 상기 딥러닝부(140)는 상기 이미지 합성부(130)로부터 합성 이미지를 수신하고, 상기 합성 이미지를 미리 설정된 감시 대상 객체에 대한 패턴이 학습된 딥러닝 알고리즘을 통해 분석하여 상기 합성 이미지에서 감시 대상 객체를 식별할 수 있다.On the other hand, FIGS. 6 and 7 are exemplary diagrams of the operation of the monitoring server 100 for identification of the object to be monitored and the event generation process according to the embodiment of the present invention. As shown in FIG. 6, the deep learning unit ( 140) receives a composite image from the image synthesizing unit 130, analyzes the composite image through a deep learning algorithm in which a preset pattern for the object to be monitored is learned, and identifies the object to be monitored from the composite image. have.
이때, 상기 딥러닝부(140)는 상기 카메라부(10)에서 객체 검출시마다 제공되는 영상들에 대응되어 생성된 합성 이미지들을 상기 딥러닝 알고리즘에 지속적으로 학습시킬 수 있으며, 상기 학습을 통해 상기 딥러닝 알고리즘에 감시 대상 객체에 대한 패턴을 학습시킬 수 있다.In this case, the deep learning unit 140 may continuously learn the composite images generated by the deep learning algorithm in response to images provided whenever an object is detected by the camera unit 10, and through the learning It is possible to learn the pattern of the object to be monitored in the learning algorithm.
일례로, 상기 딥러닝부(140)는 상기 딥러닝 알고리즘을 통해 상기 합성 이미지에서 식별된 객체별로 객체 정보를 상기 모니터링 서버(100)와 연결되거나 별도 구성된 출력부를 통해 출력할 수 있으며, 사용자 입력을 수신하는 사용자 인터페이스부(170)를 통해 상기 딥러닝부(140)가 출력하는 객체 정보 중에서 감시 대상 객체로 사용자에 의해 선택된 피드백(feedback) 정보를 수신하고, 상기 피드백 정보를 기초로 상기 딥러닝 알고리즘을 수정하여 상기 감시 대상 객체의 식별 오차를 감소시켜 감시 대상 객체에 대한 패턴이 학습되도록 할 수 있다.As an example, the deep learning unit 140 may output object information for each object identified in the composite image through the deep learning algorithm, connected to the monitoring server 100 or through a separately configured output unit. The deep learning algorithm receives feedback information selected by a user as an object to be monitored from among object information output from the deep learning unit 140 through the user interface unit 170 to be received, and the deep learning algorithm based on the feedback information By modifying, the identification error of the object to be monitored can be reduced so that a pattern for the object to be monitored can be learned.
이때, 상기 딥러닝부(140)는 사람이나 차량과 같은 감시 대상 객체에 대한 패턴을 상기 딥러닝 알고리즘에 학습시킬 수 있다.In this case, the deep learning unit 140 may learn a pattern of an object to be monitored, such as a person or vehicle, to the deep learning algorithm.
또한, 상기 딥러닝 알고리즘은 R-CNN(Regions with Convolutional Neural Network)인 것이 바람직하나, 이에 한정되지 않고 다양한 신경망 모델이 적용될 수 있다.In addition, the deep learning algorithm is preferably Regions with Convolutional Neural Network (R-CNN), but is not limited thereto, and various neural network models may be applied.
또한, 상기 사용자 인터페이스부(170)는 상기 모니터링 서버(100)에 포함되어 구성될 수 있다.In addition, the user interface unit 170 may be included in the monitoring server 100 and configured.
상술한 구성에 따라, 도 6에 도시된 바와 같이, 상기 딥러닝부(140)는 합성 이미지에 포함된 하나 이상의 객체 영역을 딥러닝 알고리즘을 통해 분석하고, 상기 객체 영역에 대응되는 객체 중 감시 대상 객체를 딥러닝 알고리즘을 통해 식별하여 감시 대상 객체로 식별된 객체 영역에 대응되어 감시 대상 객체의 객체 종류와 해당 객체 종류와의 유사도를 포함하는 객체 정보를 생성한 후 상기 이벤트 판단부(150)에 제공할 수 있다.According to the above configuration, as shown in FIG. 6, the deep learning unit 140 analyzes one or more object regions included in the composite image through a deep learning algorithm, and among objects corresponding to the object region, the object to be monitored An object is identified through a deep learning algorithm, and object information including the object type of the object to be monitored and the similarity between the object type is generated in correspondence to the object area identified as the object to be monitored, and then the event determination unit 150 Can provide.
일례로, 도 6에 도시된 바와 같이, 상기 딥러닝부(140)는 합성 이미지에 포함된 하나 이상의 객체 영역 중 특정 객체 영역에 대응되어 식별된 특정 객체가 사람 관련 감시 대상 객체인 경우 객체 종류가 사람으로 설정되며 감시 대상 객체로 식별된 상기 특정 객체와 관련하여 사람과의 유사도를 포함하는 상기 객체 정보를 생성할 수 있다.As an example, as shown in FIG. 6, the deep learning unit 140 corresponds to a specific object area among one or more object areas included in the composite image, and when a specific object identified is a person-related monitoring object, the object type is The object information including a degree of similarity to a person may be generated in relation to the specific object set as a person and identified as a monitoring target object.
또 다른 일례로, 도 7에 도시된 바와 같이, 상기 딥러닝부(140)는 상기 특정 객체 영역에 대응되어 식별된 객체가 차량 관련 감시 대상 객체인 경우 객체 종류가 차량으로 설정되며 감시 대상 객체로 식별된 상기 객체와 관련하여 차량과의 유사도를 포함하는 상기 객체 정보를 생성할 수 있다.As another example, as shown in FIG. 7, when the object identified in correspondence with the specific object area is a vehicle-related monitoring object, the object type is set as a vehicle, and the object type is set as a monitoring object. In relation to the identified object, the object information including a degree of similarity to a vehicle may be generated.
이때, 상기 합성 이미지는 객체 영역별로 객체 영역에 대응되는 영상 내에서의 객체 영역에 대응되는 위치(또는 객체의 위치)에 대한 위치 정보를 포함할 수 있으며, 상기 딥러닝부(140)는 상기 합성 이미지에 포함된 객체 영역별 위치 정보 중 상기 감시 대상 객체로 식별된 특정 객체에 대응되는 객체 영역의 위치 정보를 상기 특정 객체에 대응되는 객체 정보에 포함시킬 수 있다.In this case, the composite image may include location information on a location (or location of an object) corresponding to the object area in the image corresponding to the object area for each object area, and the deep learning unit 140 Among the location information for each object area included in the image, location information of an object area corresponding to the specific object identified as the object to be monitored may be included in the object information corresponding to the specific object.
또한, 상기 딥러닝부(140)는 상기 하나 이상의 객체 영역 중 감시 대상 객체로 식별된 객체 영역별로 생성한 객체 정보를 상기 합성 이미지에서 상기 객체 정보에 대응되는 객체 영역과 매칭하여 상기 합성 이미지에 추가할 수 있으며, 상기 객체 정보를 포함하는 합성 이미지를 상기 이벤트 판단부(150)에 제공할 수도 있다.In addition, the deep learning unit 140 matches the object information generated for each object area identified as the object to be monitored among the one or more object areas with the object area corresponding to the object information in the composite image and adds it to the composite image. Alternatively, a composite image including the object information may be provided to the event determination unit 150.
한편, 상기 이벤트 판단부(150)는 상기 딥러닝부(140)로부터 식별된 객체에 대한 객체 정보를 수신하고, 상기 객체 정보가 미리 설정된 이벤트 발생 조건 만족시 상기 카메라부(10)로부터 전송된 복수의 영상에 대응되어 이벤트 발생으로 판단할 수 있다.On the other hand, the event determination unit 150 receives object information on the identified object from the deep learning unit 140, and when the object information satisfies a preset event occurrence condition, a plurality of images transmitted from the camera unit 10 It can be determined as the occurrence of an event corresponding to the image of
일례로, 상기 이벤트 판단부(150)는 상기 객체 정보에 따른 객체의 종류가 미리 설정된 상기 감시 대상 객체 관련 객체의 종류와 일치하고, 상기 유사도가 미리 설정된 기준치 이상인 경우 이벤트 발생으로 판단할 수 있다.For example, when the type of the object according to the object information matches the type of the object related to the object to be monitored previously set, and the similarity is greater than or equal to a preset reference value, the event determination unit 150 may determine that an event has occurred.
즉, 도 6 및 도 7에 도시된 바와 같이, 상기 이벤트 판단부(150)는 상기 딥러닝부(140)로부터 제공되는 하나 이상의 객체 정보 중에서 감시 대상 객체인 사람이나 차량과 관련된 객체 종류가 설정되고 상기 사람이나 차량과의 유사도가 미리 설정된 기준치 이상인 객체 정보가 존재하는 경우 미리 설정된 이벤트 조건을 만족하는 것으로 판단하여, 상기 복수의 영상을 전송한 카메라부(10)에 대응되어 이벤트가 발생한 것으로 판단할 수 있다.That is, as shown in FIGS. 6 and 7, the event determination unit 150 sets an object type related to a person or vehicle as a monitoring target object among one or more object information provided from the deep learning unit 140, and If there is object information having a similarity with the person or vehicle equal to or greater than a preset reference value, it is determined that a preset event condition is satisfied, and the event is determined to have occurred in response to the camera unit 10 that transmitted the plurality of images. I can.
또한, 상기 이벤트 알림부(160)는 상기 이벤트 판단부(150)와 연동하여 상기 이벤트 판단부(150)가 이벤트 발생으로 판단시 이벤트 정보를 생성하여 상기 출력부를 통해 출력할 수 있다.In addition, the event notification unit 160 may interwork with the event determination unit 150 to generate event information when the event determination unit 150 determines that an event has occurred and output it through the output unit.
이때, 상기 이벤트 알림부(160)는 이벤트 발생시 상기 이벤트에 대응되는 복수의 영상을 상기 이벤트 정보에 포함시켜 상기 출력부를 통해 출력할 수도 있다.In this case, when an event occurs, the event notification unit 160 may include a plurality of images corresponding to the event in the event information and output through the output unit.
또한, 상기 이벤트 알림부(160)는 상기 이벤트 정보를 미리 설정된 외부 장치로 통신망을 통해 전송할 수도 있다.In addition, the event notification unit 160 may transmit the event information to a preset external device through a communication network.
또한, 상기 이벤트 알림부(160)는 이벤트 판단부(150)와 연동하여 이벤트 발생 조건을 만족하는 객체 정보를 식별할 수 있으며, 상기 객체 정보에 포함된 상기 위치 정보를 기초로 복수의 영상 중에서 감시 대상 객체가 존재하는 영상별로 감시 대상 객체의 위치를 미리 설정된 표식으로 마킹한 복수의 영상을 생성한 후 상기 이벤트 정보에 포함시켜 전송할 수도 있다.In addition, the event notification unit 160 may identify object information that satisfies the event occurrence condition in conjunction with the event determination unit 150, and monitor among a plurality of images based on the location information included in the object information. For each image in which the target object exists, a plurality of images in which the location of the object to be monitored is marked with a preset mark may be generated and then included in the event information and transmitted.
한편, 상술한 구성에서, 상기 모니터링 서버(100)는 복수의 서로 다른 카메라부(10)와 통신망을 통해 통신할 수 있다.Meanwhile, in the above-described configuration, the monitoring server 100 may communicate with a plurality of different camera units 10 through a communication network.
이때, 상기 모니터링 서버(100)에 구성된 영상 수집부(110)는 복수의 카메라부(10) 각각에 서로 다른 채널을 할당하고, 채널별로 복수의 영상을 수신할 수 있다.In this case, the image collection unit 110 configured in the monitoring server 100 may allocate different channels to each of the plurality of camera units 10 and receive a plurality of images for each channel.
또한, 상기 모니터링 서버(100)는 채널을 통해 복수의 카메라부(10)를 구분하여, 상기 복수의 카메라부(10) 각각에 대해 개별로 상술한 바와 같이 이벤트 발생 여부를 판단할 수 있다.In addition, the monitoring server 100 may classify the plurality of camera units 10 through a channel, and individually determine whether an event occurs for each of the plurality of camera units 10 as described above.
상술한 바와 같이, 본 발명은 카메라부에서 이벤트로 감지된 객체에 대해 모니터링 서버에서 해당 객체가 감시 대상 객체인지 여부를 판단하여 이벤트를 제공함으로써, 원격지에 위치하는 카메라부에서 영상에서 객체 검출에 따라 복수의 영상을 모니터링 서버에 전송시 데이터 전송거리가 상당하고 카메라부의 낮은 성능으로 인해 초당 프레임수가 실시간 영상을 기반으로 하는 영상 분석 알고리즘을 통해 감시 대상 객체를 식별하는데 충분치 않은 스냅샷 형태로 영상을 전송하는 경우라도 상술한 바와 같이 카메라부에서 전송한 복수의 영상에서 객체 영역만을 분리 추출하여 하나의 이미지로 합성한 후 딥러닝 알고리즘을 통해 용이하게 카메라부에서 검출한 객체가 이벤트 관련 감시 대상 객체인지 여부를 정확하게 검출할 수 있도록 지원함으로써, 원격지에 위치하는 카메라부를 저가의 카메라로 구성할 수 있도록 지원하여 시스템 구성 비용을 절감하면서도 객체 분석 결과에 대한 신뢰성을 보장할 수 있다.As described above, the present invention provides an event by determining whether the object is a monitoring target object in a monitoring server for an object detected as an event by the camera unit, according to object detection in an image by a camera unit located at a remote location. When transmitting multiple images to the monitoring server, the data transmission distance is significant and the number of frames per second is transmitted in the form of a snapshot that is insufficient to identify the object to be monitored through an image analysis algorithm based on real-time images due to the low performance of the camera unit. Even in the case of such a case, as described above, whether the object detected by the camera is easily detected by the camera unit through a deep learning algorithm after separating and extracting only the object area from the plurality of images transmitted from the camera unit and combining it into one image. By supporting to accurately detect the object, it is possible to configure the camera unit located in a remote location as a low-cost camera, thereby reducing the cost of system configuration and guaranteeing the reliability of the object analysis result.
또한, 본 발명은 모니터링 서버에서 카메라부로부터 수신된 복수의 영상 각각의 전체 영역을 대상으로 딥러닝 알고리즘을 통해 분석하는 것이 아닌 복수의 영상 각각에서 움직임이 발생한 객체 영역만을 분리한 후 이를 하나의 이미지로 합성한 합성 이미지 1장을 대상으로 딥러닝 알고리즘을 통해 분석함으로써 딥러닝 알고리즘의 객체 식별에 필요한 분석 시간을 크게 단축시킬 수 있으며, 이를 통해 모니터링 서버와 통신하는 카메라부의 수가 상당하더라도 용이하게 객체의 식별에 따른 이벤트 판단이 신속하게 이루어지도록 지원함과 아울러 카메라부의 수가 증가하더라도 이를 수용하기 위한 모니터링 서버의 개수와 하드웨어적인 성능을 낮출 수 있어 시스템 구성 비용을 절감시킬 수 있다.In addition, the present invention does not analyze the entire area of each of the plurality of images received from the camera unit in the monitoring server through a deep learning algorithm, but separates only the object area where movement has occurred in each of the plurality of images, and then converts it into one image. The analysis time required for object identification of the deep learning algorithm can be greatly shortened by analyzing a single composite image synthesized by using a deep learning algorithm.Through this, even if the number of camera units communicating with the monitoring server is large, In addition to supporting rapid event determination based on identification, even if the number of camera units increases, the number of monitoring servers and hardware performance for accommodating them can be reduced, thereby reducing system configuration cost.
도 8은 본 발명의 실시예에 따른 카메라부와 통신망을 통해 통신하는 모니터링 서버의 원격 모니터링을 위한 영상 분석 방법에 대한 순서도이다.8 is a flowchart illustrating an image analysis method for remote monitoring of a monitoring server communicating with a camera unit through a communication network according to an embodiment of the present invention.
우선, 상기 모니터링 서버(100)는 상기 카메라부(10)로부터 복수의 영상을 수신할 수 있다(S1).First, the monitoring server 100 may receive a plurality of images from the camera unit 10 (S1).
또한, 상기 모니터링 서버(100)는 상기 복수의 영상 각각을 미리 설정된 영상 분석 알고리즘에 따라 분석하여(S2) 이동이 발생한 객체 검출시(S3) 상기 객체에 대한 객체 영역을 상기 복수의 영상에서 추출할 수 있다(S4).In addition, the monitoring server 100 analyzes each of the plurality of images according to a preset image analysis algorithm (S2) and extracts an object region for the object from the plurality of images when an object in which movement has occurred (S3) is detected. I can (S4).
다음, 상기 모니터링 서버(100)는 상기 추출된 하나 이상의 객체 영역을 하나의 이미지에 합성한 합성 이미지를 생성할 수 있다(S5).Next, the monitoring server 100 may generate a composite image obtained by combining the extracted one or more object regions into one image (S5).
또한, 상기 모니터링 서버(100)는 상기 합성 이미지를 미리 설정된 감시 대상 객체에 대한 패턴이 학습된 딥러닝 알고리즘을 통해 분석하여 감시 대상 객체를 식별할 수 있다(S6).In addition, the monitoring server 100 may identify the object to be monitored by analyzing the composite image through a deep learning algorithm in which a pattern for the object to be monitored is set in advance (S6).
이후, 상기 모니터링 서버(100)는 상기 식별된 객체에 대한 객체 정보를 수신하고, 상기 객체 정보가 미리 설정된 이벤트 발생 조건 만족시 이벤트 발생으로 판단할 수 있으며, 이벤트 발생에 따른 이벤트 정보를 출력하거나 미리 설정된 외부 장치로 상기 이벤트 정보를 전송할 수 있다(S7).Thereafter, the monitoring server 100 may receive object information on the identified object, determine that an event has occurred when the object information satisfies a preset event occurrence condition, and output event information according to the event occurrence or The event information may be transmitted to a set external device (S7).
본 명세서에 기술된 다양한 장치 및 구성부는 하드웨어 회로(예를 들어, CMOS 기반 로직 회로), 펌웨어, 소프트웨어 또는 이들의 조합에 의해 구현될 수 있다. 예를 들어, 다양한 전기적 구조의 형태로 트랜지스터, 로직게이트 및 전자회로를 활용하여 구현될 수 있다.The various devices and components described herein may be implemented by hardware circuitry (eg, CMOS-based logic circuitry), firmware, software, or a combination thereof. For example, it may be implemented using transistors, logic gates, and electronic circuits in the form of various electrical structures.
전술된 내용은 본 발명이 속하는 기술 분야에서 통상의 지식을 가진 자라면 본 발명의 본질적인 특성에서 벗어나지 않는 범위에서 수정 및 변형이 가능할 것이다. 따라서, 본 발명에 개시된 실시예들은 본 발명의 기술 사상을 한정하기 위한 것이 아니라 설명하기 위한 것이고, 이러한 실시예에 의하여 본 발명의 기술 사상의 범위가 한정되는 것은 아니다. 본 발명의 보호 범위는 아래의 청구범위에 의하여 해석되어야 하며, 그와 동등한 범위 내에 있는 모든 기술 사상은 본 발명의 권리범위에 포함되는 것으로 해석되어야 할 것이다.The above contents may be modified and modified without departing from the essential characteristics of the present invention by those of ordinary skill in the technical field to which the present invention pertains. Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain the technical idea, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be interpreted as being included in the scope of the present invention.

Claims (7)

  1. 카메라부로부터 복수의 영상을 수신하는 영상 수집부;An image collection unit receiving a plurality of images from the camera unit;
    상기 복수의 영상 각각을 미리 설정된 영상 분석 알고리즘에 따라 분석하여 이동이 발생한 객체 검출시 상기 객체에 대한 객체 영역을 상기 복수의 영상에서 추출하는 객체 추출부;An object extracting unit that analyzes each of the plurality of images according to a preset image analysis algorithm and extracts an object region for the object from the plurality of images when a moving object is detected;
    상기 객체 추출부에서 추출된 하나 이상의 객체 영역을 하나의 이미지에 합성한 합성 이미지를 생성하는 이미지 합성부;An image synthesis unit for generating a composite image obtained by combining one or more object regions extracted by the object extraction unit into one image;
    상기 합성 이미지를 미리 설정된 감시 대상 객체에 대한 패턴이 학습된 딥러닝 알고리즘을 통해 분석하여 감시 대상 객체를 식별하는 딥러닝부; 및A deep learning unit that identifies the object to be monitored by analyzing the composite image through a deep learning algorithm in which a pattern for the object to be monitored is learned in advance; And
    상기 딥러닝부로부터 식별된 객체에 대한 객체 정보를 수신하고, 상기 객체 정보가 미리 설정된 이벤트 발생 조건 만족시 이벤트 발생으로 판단하는 이벤트 판단부An event determination unit that receives object information on the identified object from the deep learning unit and determines that an event occurs when the object information satisfies a preset event occurrence condition
    를 포함하는 원격 모니터링을 위한 영상 분석 시스템.Video analysis system for remote monitoring comprising a.
  2. 청구항 1에 있어서,The method according to claim 1,
    상기 객체 정보는 객체의 종류 및 감시 대상 객체와의 유사도를 포함하고, 상기 이벤트 판단부는 상기 객체 정보에 따른 객체의 종류가 미리 설정된 상기 감시 대상 객체 관련 객체의 종류와 일치하고, 상기 유사도가 미리 설정된 기준치 이상인 경우 이벤트 발생으로 판단하는 것을 특징으로 하는 원격 모니터링을 위한 영상 분석 시스템.The object information includes a type of an object and a degree of similarity with the object to be monitored, and the event determination unit matches the type of the object according to the object information to a type of the object related to the object to be monitored, and the degree of similarity is preset. An image analysis system for remote monitoring, characterized in that it determines that an event has occurred when the value exceeds the reference value.
  3. 청구항 1에 있어서,The method according to claim 1,
    상기 이벤트 판단부의 판단 결과 이벤트 발생시 이벤트 정보를 생성하여 출력하거나 미리 설정된 외부 장치로 전송하는 이벤트 알림부를 더 포함하는 것을 특징으로 하는 원격 모니터링을 위한 영상 분석 시스템.An image analysis system for remote monitoring, further comprising an event notification unit generating and outputting event information when an event occurs as a result of the determination of the event determination unit or transmitting it to a preset external device.
  4. 청구항 1에 있어서,The method according to claim 1,
    상기 객체 추출부는 상기 복수의 영상을 합성한 메디안 이미지를 생성하고, 상기 복수의 영상 각각에 대해 상기 메디안 이미지와의 차분 영상을 통해 상기 복수의 영상 중 상기 이동이 발생한 객체가 검출되는 영상별로 상기 객체의 객체 영역을 추출하는 것을 특징으로 하는 원격 모니터링을 위한 영상 분석 시스템.The object extraction unit generates a median image obtained by synthesizing the plurality of images, and for each of the plurality of images, the object for each image in which the moving object is detected through a difference image from the median image An image analysis system for remote monitoring, characterized in that extracting the object area of.
  5. 청구항 1에 있어서,The method according to claim 1,
    상기 객체 추출부는 상기 객체가 검출된 특정 영상에서 상기 객체로 판단되는 영역의 외곽선을 따라 상기 특정 영상에서 상기 객체 영역을 추출하고,The object extracting unit extracts the object region from the specific image along an outline of the region determined to be the object from the specific image in which the object is detected,
    상기 이미지 합성부는 복수의 영상 중 적어도 하나로부터 상기 객체 추출부에 의해 추출되는 하나 이상의 객체 영역을 미리 설정된 상자 채우기 문제를 통해 하나의 합성 이미지로 합성하는 것을 특징으로 하는 원격 모니터링을 위한 영상 분석 시스템.The image synthesizing unit synthesizes one or more object regions extracted by the object extracting unit from at least one of a plurality of images into one composite image through a preset box filling problem.
  6. 청구항 1에 있어서,The method according to claim 1,
    상기 복수의 영상은 상기 카메라부에 구성된 센서의 감지 또는 상기 카메라부에 의한 영상 분석에 따라 검출된 객체에 대응되는 영상인 것을 특징으로 하는 원격 모니터링을 위한 영상 분석 시스템.The image analysis system for remote monitoring, wherein the plurality of images are images corresponding to an object detected by detection of a sensor configured in the camera unit or an image analysis by the camera unit.
  7. 카메라부와 통신망을 통해 통신하는 모니터링 서버의 원격 모니터링을 위한 영상 분석 방법에 있어서,In the image analysis method for remote monitoring of a monitoring server communicating through a camera unit and a communication network,
    상기 카메라부로부터 복수의 영상을 수신하는 단계;Receiving a plurality of images from the camera unit;
    상기 복수의 영상 각각을 미리 설정된 영상 분석 알고리즘에 따라 분석하여 이동이 발생한 객체 검출시 상기 객체에 대한 객체 영역을 상기 복수의 영상에서 추출하는 단계;Analyzing each of the plurality of images according to a preset image analysis algorithm and extracting an object region for the object from the plurality of images when an object in which movement has occurred is detected;
    상기 추출된 하나 이상의 객체 영역을 하나의 이미지에 합성한 합성 이미지를 생성하는 단계;Generating a composite image obtained by combining the extracted one or more object regions into one image;
    상기 합성 이미지를 미리 설정된 감시 대상 객체에 대한 패턴이 학습된 딥러닝 알고리즘을 통해 분석하여 감시 대상 객체를 식별하는 단계; 및Analyzing the composite image through a deep learning algorithm in which a predetermined pattern for the object to be monitored is learned to identify the object to be monitored; And
    상기 식별된 객체에 대한 객체 정보를 수신하고, 상기 객체 정보가 미리 설정된 이벤트 발생 조건 만족시 이벤트 발생으로 판단하는 단계Receiving object information on the identified object, and determining that an event occurs when the object information satisfies a preset event occurrence condition
    를 포함하는 원격 모니터링을 위한 영상 분석 방법.Image analysis method for remote monitoring comprising a.
PCT/KR2020/009954 2019-07-31 2020-07-28 Image analysis system and method for remote monitoring WO2021020866A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR10-2019-0093150 2019-07-31
KR1020190093150A KR102247359B1 (en) 2019-07-31 2019-07-31 Image analysis system and method for remote monitoring

Publications (2)

Publication Number Publication Date
WO2021020866A1 true WO2021020866A1 (en) 2021-02-04
WO2021020866A9 WO2021020866A9 (en) 2021-04-01

Family

ID=74230394

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2020/009954 WO2021020866A1 (en) 2019-07-31 2020-07-28 Image analysis system and method for remote monitoring

Country Status (2)

Country Link
KR (1) KR102247359B1 (en)
WO (1) WO2021020866A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113537009A (en) * 2021-06-30 2021-10-22 上海晶赞融宣科技有限公司 Household isolation supervision system

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102305468B1 (en) * 2021-05-12 2021-09-30 씨티씨 주식회사 Landslide distributed detection system based on deep learning
KR102305467B1 (en) * 2021-05-12 2021-09-30 씨티씨 주식회사 Landslide distributed detecting method based on deep learning
KR102586144B1 (en) * 2021-09-23 2023-10-10 주식회사 딥비전 Method and apparatus for hand movement tracking using deep learning
KR102348233B1 (en) * 2021-12-03 2022-01-07 새빛이앤엘 주식회사 System of monitoring moving images using cctv video contrast optimization

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150117772A1 (en) * 2013-10-24 2015-04-30 TCL Research America Inc. Video object retrieval system and method
KR101789690B1 (en) * 2017-07-11 2017-10-25 (주)블루비스 System and method for providing security service based on deep learning
KR101932009B1 (en) * 2017-12-29 2018-12-24 (주)제이엘케이인스펙션 Image processing apparatus and method for multiple object detection
KR101954717B1 (en) * 2018-10-22 2019-03-06 주식회사 인텔리빅스 Apparatus for Processing Image by High Speed Analysis and Driving Method Thereof
KR101937272B1 (en) * 2012-09-25 2019-04-09 에스케이 텔레콤주식회사 Method and Apparatus for Detecting Event from Multiple Image

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101930049B1 (en) 2014-11-28 2018-12-17 한국전자통신연구원 Apparatus and Method for Interesting Object Based Parallel Video Analysis
JP2019003565A (en) * 2017-06-19 2019-01-10 コニカミノルタ株式会社 Image processing apparatus, image processing method and image processing program

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101937272B1 (en) * 2012-09-25 2019-04-09 에스케이 텔레콤주식회사 Method and Apparatus for Detecting Event from Multiple Image
US20150117772A1 (en) * 2013-10-24 2015-04-30 TCL Research America Inc. Video object retrieval system and method
KR101789690B1 (en) * 2017-07-11 2017-10-25 (주)블루비스 System and method for providing security service based on deep learning
KR101932009B1 (en) * 2017-12-29 2018-12-24 (주)제이엘케이인스펙션 Image processing apparatus and method for multiple object detection
KR101954717B1 (en) * 2018-10-22 2019-03-06 주식회사 인텔리빅스 Apparatus for Processing Image by High Speed Analysis and Driving Method Thereof

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113537009A (en) * 2021-06-30 2021-10-22 上海晶赞融宣科技有限公司 Household isolation supervision system
CN113537009B (en) * 2021-06-30 2024-02-13 上海晶赞融宣科技有限公司 Household isolation supervision system

Also Published As

Publication number Publication date
KR20210014988A (en) 2021-02-10
WO2021020866A9 (en) 2021-04-01
KR102247359B1 (en) 2021-05-04

Similar Documents

Publication Publication Date Title
WO2021020866A1 (en) Image analysis system and method for remote monitoring
WO2014051337A1 (en) Apparatus and method for detecting event from plurality of photographed images
WO2013115470A1 (en) Integrated control system and method using surveillance camera for vehicle
WO2016171341A1 (en) Cloud-based pathology analysis system and method
WO2017074005A1 (en) Cctv automatic selection monitoring system, and cctv automatic selection monitoring management server and management method
WO2017115905A1 (en) Human body pose recognition system and method
WO2017090892A1 (en) Osd information generation camera, osd information synthesis terminal (20), and osd information sharing system comprising same
WO2012124852A1 (en) Stereo camera device capable of tracking path of object in monitored area, and monitoring system and method using same
WO2012046899A1 (en) Image-monitoring device and method for detecting events therefor
WO2019124635A1 (en) Syntax-based method for sensing object intrusion in compressed video
WO2016099084A1 (en) Security service providing system and method using beacon signal
WO2018151503A2 (en) Method and apparatus for gesture recognition
KR20190038137A (en) Image Analysis Method and Server Apparatus for Per-channel Optimization of Object Detection
WO2014061922A1 (en) Apparatus and method for detecting camera tampering using edge image
CN112487891B (en) Visual intelligent dynamic identification model construction method applied to electric power operation site
WO2012137994A1 (en) Image recognition device and image-monitoring method therefor
WO2018135906A1 (en) Camera and image processing method of camera
WO2021167374A1 (en) Video search device and network surveillance camera system including same
WO2021100919A1 (en) Method, program, and system for determining whether abnormal behavior occurs, on basis of behavior sequence
WO2016064107A1 (en) Pan/tilt/zoom camera based video playing method and apparatus
WO2018097384A1 (en) Crowdedness notification apparatus and method
WO2023158068A1 (en) Learning system and method for improving object detection rate
WO2011043498A1 (en) Intelligent image monitoring apparatus
WO2023128186A1 (en) Multi-modal video captioning-based image security system and method
WO2022019601A1 (en) Extraction of feature point of object from image and image search system and method using same

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20847546

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 30.06.2022)

122 Ep: pct application non-entry in european phase

Ref document number: 20847546

Country of ref document: EP

Kind code of ref document: A1