WO2021117930A1 - Semantic filtering module system for improving duplicate area object detection - Google Patents

Semantic filtering module system for improving duplicate area object detection Download PDF

Info

Publication number
WO2021117930A1
WO2021117930A1 PCT/KR2019/017504 KR2019017504W WO2021117930A1 WO 2021117930 A1 WO2021117930 A1 WO 2021117930A1 KR 2019017504 W KR2019017504 W KR 2019017504W WO 2021117930 A1 WO2021117930 A1 WO 2021117930A1
Authority
WO
WIPO (PCT)
Prior art keywords
semantic
object detection
detection
improving
data
Prior art date
Application number
PCT/KR2019/017504
Other languages
French (fr)
Korean (ko)
Inventor
이영한
송혁
조충상
김제우
Original Assignee
전자부품연구원
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 전자부품연구원 filed Critical 전자부품연구원
Publication of WO2021117930A1 publication Critical patent/WO2021117930A1/en

Links

Images

Classifications

    • G06T5/70—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/30—Noise filtering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • the present invention relates to a semantic filtering module system, and more particularly, to a semantic filtering module system to which a semantic information-based post-processing technology for improving detection of overlapping region objects is applied.
  • the object detection engine detects irrespective of semantic information, there is a problem that various misrecognitions such as presence of a face in the ID card or detection of a human object in the face are highly likely to occur.
  • the present invention has been devised to solve the above problems, and an object of the present invention is to improve the performance of the object detection algorithm by improving the misrecognition due to the overlapping area object detection, and ultimately to improve the performance of the object detection algorithm and the data used for object detection learning.
  • An object of the present invention is to provide a semantic filtering module system and method that can improve performance without additional data construction cost by extracting semantic data from
  • a semantic filtering method for improving detection of an overlapping area object comprising: collecting object detection data; and separately extracting semantic information based on the collected object detection data.
  • semantic information about the location, size, and type of the object may be separately extracted.
  • the object may be at least one of a face, a person, a license plate, and an identification card in the image.
  • the extraction step can extract semantic information including a rule that the existence of a 'face' object is possible in a 'person' object, but that the existence of a 'person' object is impossible in an 'identity' object.
  • the extraction step may extract semantic information including a rule that the existence of the 'face' object is possible in the 'identity card' object, but the existence of the 'identity card' object is impossible in the 'face' object.
  • the extraction step may extract semantic information including a rule that the existence of the 'identification' object is possible in the 'person' object, but the existence of the 'person' object is impossible in the 'face' object.
  • semantic filtering method for improving detection of overlapping region objects is filtered based on semantic information extracted separately by inputting object detection data to a deep learning learning model and outputting object detection results It may further include;
  • the deep learning learning model may be implemented in a recurrent neural network (RNN) structure so that it can be learned regardless of the length and order of input information.
  • RNN recurrent neural network
  • data corresponding to an impossible existence among the object detection results may be removed to prevent a malfunction of the object detection technology.
  • a semantic filtering module system for improving detection of overlapping area objects includes: an input unit for collecting object detection data; and a processor that separately extracts semantic information based on the collected object detection data.
  • a computer-readable recording medium in which a computer program for performing a semantic filtering method for improving detection of overlapping area objects is recorded, the method comprising: collecting object detection data; and separately extracting semantic information based on the collected object detection data.
  • a semantic filtering method for improving detection of overlapping area objects includes: collecting object detection data; extracting semantic information separately based on the collected object detection data; and filtering an object detection result output by inputting object detection data into the deep learning learning model based on semantic information extracted separately.
  • 1 is a diagram illustrating an image and an object obtained in a lifetime
  • FIG. 2 is a flowchart provided in the description of a semantic filtering method for improving overlapping area object detection according to an embodiment of the present invention
  • 3 is a diagram illustrating an example of an erroneous detection that occurs in an object detection technology using deep learning
  • 5 is a diagram provided for the explanation of the method of operation of the semantic filtering module for improving detection of overlapping area objects
  • FIG. 6 is a diagram provided in the description of a semantic filtering module for improving overlapping area object detection
  • FIG. 7 is a diagram provided to explain a semantic filtering module system for improving overlapping area object detection according to an embodiment of the present invention.
  • FIG. 3 is a flowchart provided to explain a semantic filtering method (hereinafter, collectively referred to as a 'semantic filtering method') for improving detection of overlapping area objects according to an embodiment of the present invention
  • FIG. It is a diagram provided to explain the rule extraction process of
  • FIG. 5 is a diagram provided to explain the operation method of the semantic filtering module for improving detection of overlapping area objects
  • FIG. 6 is a diagram provided for improving the detection of overlapping area objects.
  • the drawings are provided in the description.
  • the semantic filtering method improves the misrecognition due to overlapping region object detection, ultimately improving the performance of the object detection algorithm, and extracting semantic data from the data used for object detection learning, thereby providing additional data Performance can be improved without the cost of deployment.
  • the present semantic filtering method includes a collection step (S310) of collecting object detection data, an extraction step (S320) of separately extracting semantic information based on the collected object detection data, and an object in a deep learning learning model.
  • a filtering step (S330) of filtering an object detection result output by inputting detection data based on semantic information extracted separately may be configured.
  • learning data for object detection consists of an image and object information (position and type) in the image.
  • a 'face' object may exist in the 'person' object, but it can be confirmed through object information that the existence of the 'person' object is impossible in the 'identification card' object.
  • semantic information of the location, size, and type of an object such as a face, a person, a license plate, and an identification card in the image is separately extracted so that it can be used for learning.
  • the existence of a 'face' object is possible in the 'person' object, but the rule that the existence of a 'person' object in the 'identity card' object is impossible, the 'face' object in the 'identity' object is possible, but the rule that the existence of an 'id' object in a 'face' object is not possible or/and the existence of an 'ident' object within a 'person' object is possible, but the existence of an 'person' object within a 'face' object It is possible to extract semantic information including a rule that says it is impossible.
  • the performance of the object detection module can be improved by overlapping the results of the object detection module by using semantic filtering in a manner that removes data corresponding to an impossible existence from among the object detection results. .
  • the deep learning learning model may be implemented in a recurrent neural network (RNN) structure so that it can be learned regardless of the length and order of input information.
  • RNN recurrent neural network
  • FIG. 7 is a diagram provided to explain a semantic filtering module system for improving overlapping area object detection according to an embodiment of the present invention.
  • the semantic filtering module system for improving detection of overlapping region objects may include an input unit 110 , a storage unit 120 , and a processor 130 .
  • the input unit 110 may be provided to collect object detection data.
  • the storage unit 120 may store programs and data necessary for the processor 130 to operate.
  • the storage unit 120 may store information about the collected data and the deep learning learning model.
  • the processor 130 separately extracts semantic information based on the collected object detection data, and filters the object detection result output by inputting the object detection data to the deep learning learning model based on the separately extracted semantic information. can do.
  • the performance of the object detection algorithm can be improved by improving the misrecognition caused by the detection of objects in the overlapping area, and by extracting semantic data from the data used for object detection learning, the performance is improved without additional data construction cost. can do it
  • the technical idea of the present invention can also be applied to a computer-readable recording medium containing a computer program for performing the functions of the apparatus and method according to the present embodiment.
  • the technical ideas according to various embodiments of the present invention may be implemented in the form of computer-readable codes recorded on a computer-readable recording medium.
  • the computer-readable recording medium may be any data storage device readable by the computer and capable of storing data.
  • the computer-readable recording medium may be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical disk, hard disk drive, or the like.
  • the computer-readable code or program stored in the computer-readable recording medium may be transmitted through a network connected between computers.

Abstract

Provided are a semantic filtering module system and method to which a semantic information-based post-processing technology for improving duplicate area object detection is applied. A semantic filtering method for improving duplicate area object detection, according to one embodiment of the present invention, comprises the steps of: collecting object detection data; and separately extracting semantic information on the basis of the collected object detection data. Therefore, misrecognition due to the duplicate area object detection is alleviated so that object detection algorithm performance can be ultimately improved, and semantic data is extracted from the data used for objection detection learning so that the performance thereof can be improved without additional data construction costs.

Description

중복 영역 객체 검출 개선을 위한 의미적 필터링 모듈 시스템Semantic filtering module system to improve detection of overlapping area objects
본 발명은 의미적 필터링 모듈 시스템에 관한 것으로, 더욱 상세하게는 중복 영역 객체 검출 개선을 위한 의미적 정보 기반의 후처리 기술이 적용되는 의미적 필터링 모듈 시스템에 관한 것이다.The present invention relates to a semantic filtering module system, and more particularly, to a semantic filtering module system to which a semantic information-based post-processing technology for improving detection of overlapping region objects is applied.
최근에는 도 1에 예시된 바와 같이 딥러닝 알고리즘을 이용하여 영상에서의 객체의 종류 및 위치를 판단하는 객체검출 연구는 활발히 진행되고 있다. Recently, as illustrated in FIG. 1 , object detection research for determining the type and location of an object in an image using a deep learning algorithm is being actively conducted.
그러나, 딥러닝을 적용하더라도 검출하고자 하는 종류의 객체에 따라서 도 2에 예시된 바와 같이 객체의 오검출이 발생하는데, 기존의 수식 및 알고리즘을 통한 기술과 달리 오검출의 분석 및 보정이 어렵기 때문에 상용화에 문제가 되고 있다.However, even when deep learning is applied, erroneous detection of an object occurs as illustrated in FIG. 2 depending on the type of object to be detected. Unlike techniques using existing formulas and algorithms, it is difficult to analyze and correct erroneous detection. Commercialization is a problem.
즉, 객체검출 엔진은 의미론적 정보와 상관없이 검출하기 때문에 아이디카드 내에 얼굴이 존재한다든지, 얼굴 내에 사람 객체를 검출하는 등 다양한 오인식이 일어날 가능성이 높다는 문제점이 존재한다. That is, since the object detection engine detects irrespective of semantic information, there is a problem that various misrecognitions such as presence of a face in the ID card or detection of a human object in the face are highly likely to occur.
이러한 오인식은 객체 검출 성능의 열화를 가지고 오기 때문에 객체 검출 엔진의 성능 향상을 시스템 개선이나 최적화가 필요하다. Since such misrecognition leads to deterioration of object detection performance, system improvement or optimization is required to improve the performance of the object detection engine.
본 발명은 상기와 같은 문제점을 해결하기 위하여 안출된 것으로서, 본 발명의 목적은, 중복 영역 객체 검출로 인한 오인식을 개선함으로써, 궁극적으로 객체 검출 알고리즘의 성능을 향상시키고, 객체검출 학습에 사용된 데이터로부터 의미론적 데이터를 추출함으로써, 추가적인 데이터 구축 비용없이 성능을 향상시킬 수 있는 의미적 필터링 모듈 시스템 및 방법을 제공함에 있다.The present invention has been devised to solve the above problems, and an object of the present invention is to improve the performance of the object detection algorithm by improving the misrecognition due to the overlapping area object detection, and ultimately to improve the performance of the object detection algorithm and the data used for object detection learning. An object of the present invention is to provide a semantic filtering module system and method that can improve performance without additional data construction cost by extracting semantic data from
상기 목적을 달성하기 위한 본 발명의 일 실시예에 따른, 중복 영역 객체 검출 개선을 위한 의미적 필터링 방법은, 객체 검출 데이터를 수집하는 단계; 및 수집된 객체 검출 데이터를 기반으로 의미적 정보를 별로도 추출하는 단계;를 포함한다.According to an embodiment of the present invention for achieving the above object, there is provided a semantic filtering method for improving detection of an overlapping area object, the method comprising: collecting object detection data; and separately extracting semantic information based on the collected object detection data.
이때, 추출 단계는, 객체의 위치 및 크기, 종류에 대한 의미적 정보를 별도로 추출할 수 있다.In this case, in the extraction step, semantic information about the location, size, and type of the object may be separately extracted.
그리고 객체는, 영상 내 얼굴, 사람, 번호판, 신분증 중 적어도 하나일 수 있다. And the object may be at least one of a face, a person, a license plate, and an identification card in the image.
또한, 추출 단계는, `사람` 객체 안에 `얼굴` 객체의 존재가 가능하지만, `신분증` 객체 안에 `사람` 객체의 존재가 불가능하다는 규칙이 포함된 의미적 정보를 추출할 수 있다. In addition, the extraction step can extract semantic information including a rule that the existence of a 'face' object is possible in a 'person' object, but that the existence of a 'person' object is impossible in an 'identity' object.
그리고 추출 단계는, '신분증' 객체 안에 '얼굴' 객체의 존재가 가능하지만, '얼굴' 객체 안에 '신분증' 객체의 존재가 불가능하다는 규칙이 포함된 의미적 정보를 추출할 수 있다.And the extraction step may extract semantic information including a rule that the existence of the 'face' object is possible in the 'identity card' object, but the existence of the 'identity card' object is impossible in the 'face' object.
또한, 추출 단계는, '사람' 객체 안에 '신분증' 객체의 존재가 가능하지만, '얼굴' 객체 안에 '사람' 객체의 존재가 불가능하다는 규칙이 포함된 의미적 정보를 추출할 수 있다.In addition, the extraction step may extract semantic information including a rule that the existence of the 'identification' object is possible in the 'person' object, but the existence of the 'person' object is impossible in the 'face' object.
그리고 본 발명의 일 실시예에 따른, 중복 영역 객체 검출 개선을 위한 의미적 필터링 방법은, 딥러닝 학습 모델에 객체 검출 데이터를 입력하여 출력되는 객체 검출 결과를 별도로 추출되는 의미적 정보를 기반으로 필터링하는 단계;를 더 포함할 수 있다.And the semantic filtering method for improving detection of overlapping region objects according to an embodiment of the present invention is filtered based on semantic information extracted separately by inputting object detection data to a deep learning learning model and outputting object detection results It may further include;
또한, 딥러닝 학습 모델은, 입력되는 정보의 길이 및 순서에 무관하게 학습 가능하도록, RNN(Recurrent Neural Network) 구조로 구현될 수 있다. In addition, the deep learning learning model may be implemented in a recurrent neural network (RNN) structure so that it can be learned regardless of the length and order of input information.
그리고 필터링 단계는, 객체 검출 기술의 오동작을 방지하도록, 객체 검출 결과 중 불가능한 존재에 해당하는 데이터를 제거할 수 있다.In the filtering step, data corresponding to an impossible existence among the object detection results may be removed to prevent a malfunction of the object detection technology.
한편, 본 발명의 다른 실시예에 따른, 중복 영역 객체 검출 개선을 위한 의미적 필터링 모듈 시스템은, 객체 검출 데이터를 수집하는 입력부; 및 수집된 객체 검출 데이터를 기반으로 의미적 정보를 별로도 추출하는 프로세서;를 포함한다.Meanwhile, according to another embodiment of the present invention, a semantic filtering module system for improving detection of overlapping area objects includes: an input unit for collecting object detection data; and a processor that separately extracts semantic information based on the collected object detection data.
또한, 본 발명의 다른 실시예에 따른, 중복 영역 객체 검출 개선을 위한 의미적 필터링 방법을 수행하는 컴퓨터 프로그램이 수록된 컴퓨터로 읽을 수 있는 기록매체는, 객체 검출 데이터를 수집하는 단계; 및 수집된 객체 검출 데이터를 기반으로 의미적 정보를 별로도 추출하는 단계;를 포함하는 방법을 수행하는 컴퓨터 프로그램이 수록된다.In addition, according to another embodiment of the present invention, there is provided a computer-readable recording medium in which a computer program for performing a semantic filtering method for improving detection of overlapping area objects is recorded, the method comprising: collecting object detection data; and separately extracting semantic information based on the collected object detection data.
그리고 본 발명의 다른 실시예에 따른, 중복 영역 객체 검출 개선을 위한 의미적 필터링 방법은, 객체 검출 데이터를 수집하는 단계; 수집된 객체 검출 데이터를 기반으로 의미적 정보를 별로도 추출하는 단계; 및 딥러닝 학습 모델에 객체 검출 데이터를 입력하여 출력되는 객체 검출 결과를 별도로 추출되는 의미적 정보를 기반으로 필터링하는 단계;를 포함한다.And according to another embodiment of the present invention, a semantic filtering method for improving detection of overlapping area objects includes: collecting object detection data; extracting semantic information separately based on the collected object detection data; and filtering an object detection result output by inputting object detection data into the deep learning learning model based on semantic information extracted separately.
이상 설명한 바와 같이, 본 발명의 실시예들에 따르면, 중복 영역 객체 검출로 인한 오인식을 개선함으로써, 궁극적으로 객체 검출 알고리즘의 성능을 향상시킬 수 있으며, 객체검출 학습에 사용된 데이터로부터 의미론적 데이터를 추출함으로써, 추가적인 데이터 구축 비용없이 성능을 향상시킬 수 있다. As described above, according to embodiments of the present invention, it is possible to improve the performance of the object detection algorithm by improving the misrecognition due to the overlapping area object detection, and semantic data from the data used for object detection learning can be improved. By extracting, performance can be improved without additional data construction costs.
도 1은 일생 생활에서 얻은 영상과 객체가 예시된 도면, 1 is a diagram illustrating an image and an object obtained in a lifetime;
도 2는 본 발명의 일 실시예에 따른 중복 영역 객체 검출 개선을 위한 의미적 필터링 방법의 설명에 제공된 흐름도, 2 is a flowchart provided in the description of a semantic filtering method for improving overlapping area object detection according to an embodiment of the present invention;
도 3은 딥러닝을 이용한 객체 검출 기술에서 발생하는 오검출 사례가 예시된 도면, 3 is a diagram illustrating an example of an erroneous detection that occurs in an object detection technology using deep learning;
도 4는 객체 정보 사이의 규칙 추출 과정의 설명에 제공된 도면, 4 is a diagram provided for the explanation of the rule extraction process between object information;
도 5는 중복 영역 객체 검출 개선을 위한 의미적 필터링 모듈의 동작 방식의 설명에 제공된 도면, 5 is a diagram provided for the explanation of the method of operation of the semantic filtering module for improving detection of overlapping area objects;
도 6은 중복 영역 객체 검출 개선을 위한 의미적 필터링 모듈의 설명에 제공된 도면, 그리고 6 is a diagram provided in the description of a semantic filtering module for improving overlapping area object detection;
도 7은 본 발명의 일 실시예에 따른 중복 영역 객체 검출 개선을 위한 의미적 필터링 모듈 시스템의 설명에 제공된 도면이다. 7 is a diagram provided to explain a semantic filtering module system for improving overlapping area object detection according to an embodiment of the present invention.
이하에서는 도면을 참조하여 본 발명을 보다 상세하게 설명한다.Hereinafter, the present invention will be described in more detail with reference to the drawings.
도 3은 본 발명의 일 실시예에 따른 중복 영역 객체 검출 개선을 위한 의미적 필터링 방법(이하에서는 '의미적 필터링 방법'으로 총칭하기로 함)의 설명에 제공된 흐름도이며, 도 4는 객체 정보 사이의 규칙 추출 과정의 설명에 제공된 도면이며, 도 5는 중복 영역 객체 검출 개선을 위한 의미적 필터링 모듈의 동작 방식의 설명에 제공된 도면이고, 도 6은 중복 영역 객체 검출 개선을 위한 의미적 필터링 모듈의 설명에 제공된 도면이다. 3 is a flowchart provided to explain a semantic filtering method (hereinafter, collectively referred to as a 'semantic filtering method') for improving detection of overlapping area objects according to an embodiment of the present invention, and FIG. It is a diagram provided to explain the rule extraction process of , FIG. 5 is a diagram provided to explain the operation method of the semantic filtering module for improving detection of overlapping area objects, and FIG. 6 is a diagram provided for improving the detection of overlapping area objects. The drawings are provided in the description.
본 실시예에 따른 의미적 필터링 방법은, 중복 영역 객체 검출로 인한 오인식을 개선함으로써, 궁극적으로 객체 검출 알고리즘의 성능을 향상시키고, 객체검출 학습에 사용된 데이터로부터 의미론적 데이터를 추출함으로써, 추가적인 데이터 구축 비용없이 성능을 향상시킬 수 있다.The semantic filtering method according to the present embodiment improves the misrecognition due to overlapping region object detection, ultimately improving the performance of the object detection algorithm, and extracting semantic data from the data used for object detection learning, thereby providing additional data Performance can be improved without the cost of deployment.
이를 위해, 본 의미적 필터링 방법은, 객체 검출 데이터를 수집하는 수집 단계(S310), 수집된 객체 검출 데이터를 기반으로 의미적 정보를 별로도 추출하는 추출 단계(S320) 및 딥러닝 학습 모델에 객체 검출 데이터를 입력하여 출력되는 객체 검출 결과를 별도로 추출되는 의미적 정보를 기반으로 필터링하는 필터링 단계(S330)로 구성될 수 있다. To this end, the present semantic filtering method includes a collection step (S310) of collecting object detection data, an extraction step (S320) of separately extracting semantic information based on the collected object detection data, and an object in a deep learning learning model. A filtering step (S330) of filtering an object detection result output by inputting detection data based on semantic information extracted separately may be configured.
일반적으로 객체 검출을 위한 학습 데이터는 영상과 영상 내의 객체 정보(위치 및 종류)로 구성되어 있다. In general, learning data for object detection consists of an image and object information (position and type) in the image.
본 의미적 필터링 방법에서는 영상에서부터 객체 정보 사이의 규칙을 데이터로부터 학습하는 것을 넘어서 도 4에 예시된 바와 같이 객체 정보간의 규칙을 데이터로부터 추출함으로써 후처리를 할 수 있도록 한다.In this semantic filtering method, beyond learning a rule between object information from an image from data, as illustrated in FIG. 4 , a rule between object information is extracted from data to enable post-processing.
예를 들어, 도 4의 경우 `사람` 객체 안에는 `얼굴` 객체가 존재할 수 있지만 `신분증` 객체 안에 `사람` 객체의 존재가 불가능하다는 것을 객체 정보들을 통해서 확인할 수 있다. For example, in the case of FIG. 4 , a 'face' object may exist in the 'person' object, but it can be confirmed through object information that the existence of the 'person' object is impossible in the 'identification card' object.
즉, 추출 단계(S320)에서는, 영상 내 얼굴, 사람, 번호판, 신분증과 같은 객체의 위치 및 크기, 종류가 가지는 의미적 정보를 별도로 추출함으로써 학습에 사용할 수 있도록 한다. That is, in the extraction step (S320), semantic information of the location, size, and type of an object such as a face, a person, a license plate, and an identification card in the image is separately extracted so that it can be used for learning.
구체적으로, 추출 단계(S320)에서는, `사람` 객체 안에 `얼굴` 객체의 존재가 가능하지만, `신분증` 객체 안에 `사람` 객체의 존재가 불가능하다는 규칙, '신분증' 객체 안에 '얼굴' 객체의 존재가 가능하지만, '얼굴' 객체 안에 '신분증' 객체의 존재가 불가능하다는 규칙 또는/및 '사람' 객체 안에 '신분증' 객체의 존재가 가능하지만, '얼굴' 객체 안에 '사람' 객체의 존재가 불가능하다는 규칙 등이 포함된 의미적 정보를 추출할 수 있다.Specifically, in the extraction step (S320), the existence of a 'face' object is possible in the 'person' object, but the rule that the existence of a 'person' object in the 'identity card' object is impossible, the 'face' object in the 'identity' object is possible, but the rule that the existence of an 'id' object in a 'face' object is not possible or/and the existence of an 'ident' object within a 'person' object is possible, but the existence of an 'person' object within a 'face' object It is possible to extract semantic information including a rule that says it is impossible.
이러한 정보를 이용하여 데이터가 가지고 있는 객체 정보간의 의미적 정보로 후처리함으로써, 객체 검출 기술의 오동작을 방지할 수 있다. 이에 대한 후처리 모듈의 학습 과정은 도 5와 같다.By using this information to post-process as semantic information between object information possessed by data, it is possible to prevent a malfunction of the object detection technology. A learning process of the post-processing module for this is shown in FIG. 5 .
정리하면, 필터링 단계(S330)에서는, 객체 검출 결과 중 불가능한 존재에 해당하는 데이터를 제거하는 방식으로, 의미적 필터링을 객체 검출 모듈의 결과와 겹쳐서 사용함으로써, 객체 검출 모듈의 성능을 향상시킬 수 있다. In summary, in the filtering step ( S330 ), the performance of the object detection module can be improved by overlapping the results of the object detection module by using semantic filtering in a manner that removes data corresponding to an impossible existence from among the object detection results. .
이때, 딥러닝 학습 모델은, 입력되는 정보의 길이 및 순서에 무관하게 학습 가능하도록, RNN(Recurrent Neural Network) 구조로 구현될 수 있다. In this case, the deep learning learning model may be implemented in a recurrent neural network (RNN) structure so that it can be learned regardless of the length and order of input information.
도 7은 본 발명의 일 실시예에 따른 중복 영역 객체 검출 개선을 위한 의미적 필터링 모듈 시스템의 설명에 제공된 도면이다. 7 is a diagram provided to explain a semantic filtering module system for improving overlapping area object detection according to an embodiment of the present invention.
도 7을 참조하면, 본 실시예에 따른 중복 영역 객체 검출 개선을 위한 의미적 필터링 모듈 시스템은, 입력부(110), 저장부(120) 및 프로세서(130)를 포함할 수 있다. Referring to FIG. 7 , the semantic filtering module system for improving detection of overlapping region objects according to the present embodiment may include an input unit 110 , a storage unit 120 , and a processor 130 .
입력부(110)는, 객체 검출 데이터를 수집하기 위해 마련될 수 있다. The input unit 110 may be provided to collect object detection data.
저장부(120)는, 프로세서(130)가 동작함에 있어 필요한 프로그램 및 데이터를 저장할 수 있다. The storage unit 120 may store programs and data necessary for the processor 130 to operate.
예를 들면, 저장부(120)는, 수집된 데이터들과 딥러닝 학습 모델에 대한 정보를 저장할 수 있다. For example, the storage unit 120 may store information about the collected data and the deep learning learning model.
프로세서(130)는, 수집된 객체 검출 데이터를 기반으로 의미적 정보를 별로도 추출하고, 딥러닝 학습 모델에 객체 검출 데이터를 입력하여 출력되는 객체 검출 결과를 별도로 추출되는 의미적 정보를 기반으로 필터링할 수 있다. The processor 130 separately extracts semantic information based on the collected object detection data, and filters the object detection result output by inputting the object detection data to the deep learning learning model based on the separately extracted semantic information. can do.
이를 통해, 중복 영역 객체 검출로 인한 오인식을 개선함으로써, 궁극적으로 객체 검출 알고리즘의 성능을 향상시킬 수 있으며, 객체검출 학습에 사용된 데이터로부터 의미론적 데이터를 추출함으로써, 추가적인 데이터 구축 비용없이 성능을 향상시킬 수 있다. Through this, the performance of the object detection algorithm can be improved by improving the misrecognition caused by the detection of objects in the overlapping area, and by extracting semantic data from the data used for object detection learning, the performance is improved without additional data construction cost. can do it
한편, 본 실시예에 따른 장치와 방법의 기능을 수행하게 하는 컴퓨터 프로그램을 수록한 컴퓨터로 읽을 수 있는 기록매체에도 본 발명의 기술적 사상이 적용될 수 있음은 물론이다. 또한, 본 발명의 다양한 실시예에 따른 기술적 사상은 컴퓨터로 읽을 수 있는 기록매체에 기록된 컴퓨터로 읽을 수 있는 코드 형태로 구현될 수도 있다. 컴퓨터로 읽을 수 있는 기록매체는 컴퓨터에 의해 읽을 수 있고 데이터를 저장할 수 있는 어떤 데이터 저장 장치이더라도 가능하다. 예를 들어, 컴퓨터로 읽을 수 있는 기록매체는 ROM, RAM, CD-ROM, 자기 테이프, 플로피 디스크, 광디스크, 하드 디스크 드라이브, 등이 될 수 있음은 물론이다. 또한, 컴퓨터로 읽을 수 있는 기록매체에 저장된 컴퓨터로 읽을 수 있는 코드 또는 프로그램은 컴퓨터간에 연결된 네트워크를 통해 전송될 수도 있다.On the other hand, it goes without saying that the technical idea of the present invention can also be applied to a computer-readable recording medium containing a computer program for performing the functions of the apparatus and method according to the present embodiment. In addition, the technical ideas according to various embodiments of the present invention may be implemented in the form of computer-readable codes recorded on a computer-readable recording medium. The computer-readable recording medium may be any data storage device readable by the computer and capable of storing data. For example, the computer-readable recording medium may be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical disk, hard disk drive, or the like. In addition, the computer-readable code or program stored in the computer-readable recording medium may be transmitted through a network connected between computers.
또한, 이상에서는 본 발명의 바람직한 실시예에 대하여 도시하고 설명하였지만, 본 발명은 상술한 특정의 실시예에 한정되지 아니하며, 청구범위에서 청구하는 본 발명의 요지를 벗어남이 없이 당해 발명이 속하는 기술분야에서 통상의 지식을 가진자에 의해 다양한 변형실시가 가능한 것은 물론이고, 이러한 변형실시들은 본 발명의 기술적 사상이나 전망으로부터 개별적으로 이해되어져서는 안될 것이다.In addition, although preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention as claimed in the claims Various modifications are possible by those of ordinary skill in the art, and these modifications should not be individually understood from the technical spirit or prospect of the present invention.

Claims (12)

  1. 객체 검출 데이터를 수집하는 단계; 및collecting object detection data; and
    수집된 객체 검출 데이터를 기반으로 의미적 정보를 별로도 추출하는 단계;를 포함하는 중복 영역 객체 검출 개선을 위한 의미적 필터링 방법.A semantic filtering method for improving detection of overlapping area objects, comprising: separately extracting semantic information based on the collected object detection data.
  2. 청구항 1에 있어서,The method according to claim 1,
    추출 단계는, The extraction step is
    객체의 위치 및 크기, 종류에 대한 의미적 정보를 별도로 추출하는 것을 특징으로 하는 중복 영역 객체 검출 개선을 위한 의미적 필터링 방법.A semantic filtering method for improving detection of overlapping area objects, characterized in that semantic information on the location, size, and type of the object is separately extracted.
  3. 청구항 2에 있어서,3. The method according to claim 2,
    객체는, object is,
    영상 내 얼굴, 사람, 번호판, 신분증 중 적어도 하나인 것을 특징으로 하는 중복 영역 객체 검출 개선을 위한 의미적 필터링 방법.A semantic filtering method for improving detection of overlapping area objects, characterized in that at least one of a face, a person, a license plate, and an identification card in an image.
  4. 청구항 3에 있어서,4. The method according to claim 3,
    추출 단계는, The extraction step is
    `사람` 객체 안에 `얼굴` 객체의 존재가 가능하지만, `신분증` 객체 안에 `사람` 객체의 존재가 불가능하다는 규칙이 포함된 의미적 정보를 추출하는 것을 특징으로 하는 중복 영역 객체 검출 개선을 위한 의미적 필터링 방법.For improving detection of overlapping area objects, characterized by extracting semantic information including a rule that the existence of a 'face' object is possible in a 'person' object, but that the existence of a 'person' object is not possible in an 'identification' object Semantic filtering method.
  5. 청구항 3에 있어서,4. The method according to claim 3,
    추출 단계는, The extraction step is
    '신분증' 객체 안에 '얼굴' 객체의 존재가 가능하지만, '얼굴' 객체 안에 '신분증' 객체의 존재가 불가능하다는 규칙이 포함된 의미적 정보를 추출하는 것을 특징으로 하는 중복 영역 객체 검출 개선을 위한 의미적 필터링 방법.For improving detection of overlapping area objects, characterized in that it extracts semantic information including a rule that the existence of a 'face' object is possible in the 'identity card' object, but the existence of an 'identity card' object in the 'face' object is impossible Semantic filtering method.
  6. 청구항 3에 있어서,4. The method according to claim 3,
    추출 단계는, The extraction step is
    '사람' 객체 안에 '신분증' 객체의 존재가 가능하지만, '얼굴' 객체 안에 '사람' 객체의 존재가 불가능하다는 규칙이 포함된 의미적 정보를 추출하는 것을 특징으로 하는 중복 영역 객체 검출 개선을 위한 의미적 필터링 방법.For improving detection of overlapping area objects characterized by extracting semantic information including a rule that the existence of an 'identity card' object is possible in a 'person' object, but the existence of a 'person' object is not possible in a 'face' object Semantic filtering method.
  7. 청구항 4 내지 6 중 어느 하나에 있어서,7. The method according to any one of claims 4 to 6,
    딥러닝 학습 모델에 객체 검출 데이터를 입력하여 출력되는 객체 검출 결과를 별도로 추출되는 의미적 정보를 기반으로 필터링하는 단계;를 더 포함하는 것을 특징으로 하는 중복 영역 객체 검출 개선을 위한 의미적 필터링 방법.Filtering the object detection result output by inputting object detection data into the deep learning learning model based on semantic information extracted separately; Semantic filtering method for improving detection of overlapping region objects, characterized in that it further comprises.
  8. 청구항 7에 있어서,8. The method of claim 7,
    딥러닝 학습 모델은, The deep learning learning model is
    입력되는 정보의 길이 및 순서에 무관하게 학습 가능하도록, RNN(Recurrent Neural Network) 구조로 구현되는 것을 특징으로 하는 중복 영역 객체 검출 개선을 위한 의미적 필터링 방법.A semantic filtering method for improving detection of overlapping region objects, characterized in that it is implemented in a Recurrent Neural Network (RNN) structure so that it can be learned regardless of the length and order of input information.
  9. 청구항 7에 있어서,8. The method of claim 7,
    필터링 단계는, The filtering step is
    객체 검출 기술의 오동작을 방지하도록, 객체 검출 결과 중 불가능한 존재에 해당하는 데이터를 제거하는 것을 특징으로 하는 중복 영역 객체 검출 개선을 위한 의미적 필터링 방법.A semantic filtering method for improving detection of an object in a duplicate area, characterized in that data corresponding to an impossible existence is removed from an object detection result to prevent a malfunction of the object detection technology.
  10. 객체 검출 데이터를 수집하는 입력부; 및an input unit for collecting object detection data; and
    수집된 객체 검출 데이터를 기반으로 의미적 정보를 별로도 추출하는 프로세서;를 포함하는 중복 영역 객체 검출 개선을 위한 의미적 필터링 모듈 시스템.A semantic filtering module system for improving detection of overlapping area objects, comprising: a processor for separately extracting semantic information based on the collected object detection data.
  11. 객체 검출 데이터를 수집하는 단계; 및collecting object detection data; and
    수집된 객체 검출 데이터를 기반으로 의미적 정보를 별로도 추출하는 단계; 를 포함하는 중복 영역 객체 검출 개선을 위한 의미적 필터링 방법을 수행하는 컴퓨터 프로그램이 수록된 컴퓨터로 읽을 수 있는 기록매체.extracting semantic information separately based on the collected object detection data; A computer-readable recording medium having a computer program for performing a semantic filtering method for improving detection of overlapping area objects, including a computer-readable recording medium.
  12. 객체 검출 데이터를 수집하는 단계; collecting object detection data;
    수집된 객체 검출 데이터를 기반으로 의미적 정보를 별로도 추출하는 단계; 및 extracting semantic information separately based on the collected object detection data; and
    딥러닝 학습 모델에 객체 검출 데이터를 입력하여 출력되는 객체 검출 결과를 별도로 추출되는 의미적 정보를 기반으로 필터링하는 단계;를 포함하는 중복 영역 객체 검출 개선을 위한 의미적 필터링 방법.A semantic filtering method for improving detection of overlapping regions, comprising: inputting object detection data to a deep learning learning model and filtering an output object detection result based on semantic information extracted separately.
PCT/KR2019/017504 2019-12-10 2019-12-11 Semantic filtering module system for improving duplicate area object detection WO2021117930A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR1020190163313A KR102264208B1 (en) 2019-12-10 2019-12-10 Semantic filtering module system for improved detection of duplicate area objects
KR10-2019-0163313 2019-12-10

Publications (1)

Publication Number Publication Date
WO2021117930A1 true WO2021117930A1 (en) 2021-06-17

Family

ID=76330408

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2019/017504 WO2021117930A1 (en) 2019-12-10 2019-12-11 Semantic filtering module system for improving duplicate area object detection

Country Status (2)

Country Link
KR (1) KR102264208B1 (en)
WO (1) WO2021117930A1 (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2004057537A1 (en) * 2002-12-20 2004-07-08 The Foundation For The Promotion Of Industrial Science Method and device for tracing moving object in image
KR20110103842A (en) * 2010-03-15 2011-09-21 오므론 가부시키가이샤 Object tracking apparatus, object tracking method, and controlling program

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2004057537A1 (en) * 2002-12-20 2004-07-08 The Foundation For The Promotion Of Industrial Science Method and device for tracing moving object in image
KR20110103842A (en) * 2010-03-15 2011-09-21 오므론 가부시키가이샤 Object tracking apparatus, object tracking method, and controlling program

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
LI, YIKANG ET AL.: "ViP-CNN: Visual Phrase Guided Convolutional Neural Network", PROCEEDINGS OF THE IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR, 26 July 2017 (2017-07-26), pages 1347 - 1356, Retrieved from the Internet <URL:hilps://openaccess.thecvf.com/contentcvpr2017/himl/LiViP-CNNVisualPhraseCVPR_2017_paper.html> [retrieved on 20200821] *
MORO. ANDREA ET AL.: "Semantic Rule Filtering for Web-Scale Relation Extraction", 12TH INTERNATIONAL SEMANTIC WEB CONFERENCE, 25 October 2013 (2013-10-25), Sydney, NSW, Australia, pages 347 - 362, XP047491112, Retrieved from the Internet <URL:https://link.springer.com/chapter/10.1007/978-3-642-41335-3_22> [retrieved on 20200820], DOI: 10.1007/978-3-642-41335-3_22 *
YANG, QINGXIONG: "Semantic Filtering", IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, 30 June 2016 (2016-06-30), pages 4517 - 4526, XP033021641, [retrieved on 20200821], DOI: 10.1109/CVPR.2016.489 *

Also Published As

Publication number Publication date
KR102264208B1 (en) 2021-06-11

Similar Documents

Publication Publication Date Title
CN111523480B (en) Method and device for detecting face obstruction, electronic equipment and storage medium
CN102945366B (en) A kind of method and device of recognition of face
CN110705405B (en) Target labeling method and device
WO2013048160A1 (en) Face recognition method, apparatus, and computer-readable recording medium for executing the method
WO2015126031A1 (en) Person counting method and device for same
CN110610127B (en) Face recognition method and device, storage medium and electronic equipment
US20210279450A1 (en) Person detection system
CN103902619B (en) A kind of network public-opinion monitoring method and system
CN109344886B (en) Occlusion number plate distinguishing method based on convolutional neural network
CN112150514A (en) Pedestrian trajectory tracking method, device and equipment of video and storage medium
CN109410172A (en) Detection method, device, storage medium and the processor of paper sheet thickness
CN108765935A (en) A kind of pedestrian running red light is broken rules and regulations processing system and its method
WO2021153861A1 (en) Method for detecting multiple objects and apparatus therefor
WO2020080571A1 (en) Method and system for de-identification of personal information for image information device
WO2020096111A1 (en) Deep learning-based abnormal behavior recognition device and method
WO2012121480A2 (en) Method for supporting a plurality of users to simultaneously perform collection, server, and computer readable recording medium
WO2021117930A1 (en) Semantic filtering module system for improving duplicate area object detection
CN112580531A (en) Method and system for identifying and detecting true and false license plates
WO2018212380A1 (en) Method by which wearable device detects ecg template for personal authentication
CN113553965B (en) Person identity recognition method combining face recognition and human body recognition
WO2022107925A1 (en) Deep learning object detection processing device
WO2022191380A1 (en) Blockchain-based image forgery and falsification preventing system and method, and computer program therefor
WO2014185691A1 (en) Apparatus and method for extracting high watermark image from continuously photographed images
CN112257768B (en) Method and device for identifying illegal financial pictures and computer storage medium
CN111683085B (en) External network identity authentication method, system, equipment and storage medium based on internal network connection

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19956119

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19956119

Country of ref document: EP

Kind code of ref document: A1