KR20170010543A - Object recognition method and device using visible light sensor and thermal imagery sensor - Google Patents

Object recognition method and device using visible light sensor and thermal imagery sensor Download PDF

Info

Publication number
KR20170010543A
KR20170010543A KR1020150102300A KR20150102300A KR20170010543A KR 20170010543 A KR20170010543 A KR 20170010543A KR 1020150102300 A KR1020150102300 A KR 1020150102300A KR 20150102300 A KR20150102300 A KR 20150102300A KR 20170010543 A KR20170010543 A KR 20170010543A
Authority
KR
South Korea
Prior art keywords
image
visible light
thermal
thermal image
sensor
Prior art date
Application number
KR1020150102300A
Other languages
Korean (ko)
Inventor
배원길
배연정
서승원
Original Assignee
서울대학교산학협력단
(주)케이스포유
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 서울대학교산학협력단, (주)케이스포유 filed Critical 서울대학교산학협력단
Priority to KR1020150102300A priority Critical patent/KR20170010543A/en
Publication of KR20170010543A publication Critical patent/KR20170010543A/en

Links

Images

Classifications

    • G06K9/6288—
    • G06K9/00362—
    • G06K9/6289—

Landscapes

  • Studio Devices (AREA)
  • Radiation Pyrometers (AREA)
  • Image Processing (AREA)

Abstract

An image sensing step (S10) for sensing a visible light image of an object by the visible light sensor (10) and sensing a thermal image of the object using the thermal image sensor (20); An image generation step S20 of processing the signal of the visible light sensor 10 by the operation processing unit 20 to generate a visible light image F1 and processing a signal of the thermal image sensor 20 to generate a thermal image F2 ; The arithmetic processing unit 30 adjusts the size of the thermal image F2 so that the thermal image F2 can be overlaid on the visible light image F1 and the visible image F1 and the thermal image F2, F2) an image adjustment step S30 of matching a view point / a field of view; The calculation processing unit 30 determines whether or not there is an area exceeding the threshold value C1 of the temperature inputted in advance in the column image F2 and if the column image F2 exceeds the threshold value C1 (S40) of transmitting the visible light image (F1) and the thermal image (F2) together with the fact to the server (40) and applying an alarm display signal to the server (40) And an image overlay presentation step (S50) in which the display unit (50) overlays the visible image (F1) and the thermal image (F2) transmitted from the operation processing unit (30) And more particularly, to a method of recognizing a fusion object based on a visible light sensor and a thermal image sensor.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for recognizing a fusion object based on a visible light sensor and a thermal image sensor,

The present invention relates to an apparatus and method for fusing individual image information obtained from a visible light sensor and a thermal image sensor into a single image in real time, and can integrate physical features such as spatial arrangement and thermal distribution of a subject.

Registered Patent No. 10-1313203: An apparatus and method for aligning an optical axis of a visible light sensor and a thermal image sensor includes a visible light sensor for photographing a subject; A thermal image sensor for photographing a thermal image of the subject; Reflecting means for transmitting visible light from the subject to the visible light sensor and selectively reflecting light to transmit infrared rays from the subject to the thermal image sensor; And a controller configured to compare an image photographed by the visible light sensor and an image photographed by the thermal image sensor to calculate a degree of an optical axis error, and to adjust at least one of the visible light sensor, the thermal image sensor, Wherein the control means compares the image photographed by the visible light sensor with the image photographed by the thermal image sensor to determine whether the optical axis is misaligned And adjusting the optical axis so that the degree of misalignment of the optical axis is less than or equal to a threshold value.

The present invention relates to a visible light sensor and a thermal image sensor-based fusion object capable of integrating individual image information obtained from a visible light sensor and a thermal image sensor into a single image in real time and integrating physical characteristics such as spatial arrangement and thermal distribution of a subject, And to provide a recognition method and apparatus.

An image sensing step (S10) in which the visible light sensor 10 senses a visible light image of an object and the column image sensor 20 senses a column image of the object;

An image generation step S20 of processing the signal of the visible light sensor 10 by the operation processing unit 20 to generate a visible light image F1 and processing a signal of the thermal image sensor 20 to generate a thermal image F2 ;

The arithmetic processing unit 30 adjusts the size of the thermal image F2 so that the thermal image F2 can be overlaid on the visible light image F1 and the visible image F1 and the thermal image F2, F2) an image adjustment step S30 of matching a view point / a field of view;

The calculation processing unit 30 determines whether or not there is an area exceeding the threshold value C1 of the temperature inputted in advance in the column image F2 and if the column image F2 exceeds the threshold value C1 (S40) of transmitting the visible light image (F1) and the thermal image (F2) together with the fact to the server (40) and applying an alarm display signal to the server (40)

An image overlay presentation step (S50) in which the display unit (50) overlays and displays the visible image (F1) and the thermal image (F2) transmitted from the operation processing unit (30);

And a thermal image sensor based fusion object recognition method.

According to the present invention, a visible light sensor and a thermal image sensor, which can integrate individual image information obtained from a visible light sensor and a thermal image sensor into a single image in real time, and can integrate physical features such as spatial arrangement and thermal distribution of a subject, Based fusion object recognition method and apparatus are provided.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram illustrating a method of recognizing a fusion object based on a visible light sensor and a thermal image sensor according to the present invention.
FIG. 2 and FIG. 3 are diagrams showing the visible light image and extraction by the visible light sensor of the present invention. FIG.
4 is an explanatory view of a thermal image by the thermal image sensor of the present invention.
5 (a), (b) and (c) are explanatory diagrams of thermal image adjustment collected and generated by the thermal image sensor of the present invention.
6 is a block diagram of a fusion object recognition apparatus based on a visible light sensor and a thermal image sensor according to the present invention.
7 is a flowchart of a method of recognizing a fusion object based on a visible light sensor and a thermal image sensor according to the present invention.

Hereinafter, a method and an apparatus for recognizing a fusion object based on a visible light sensor and a thermal image sensor according to the present invention will be described in detail with reference to the accompanying drawings. FIG. 1 is a diagram for explaining a method of recognizing a fusion object based on a visible light sensor and a thermal image sensor of the present invention, FIGS. 2 and 3 are conceptual diagrams of a visible light image and extraction by a visible light sensor of the present invention, 5 is a diagram illustrating a thermal image control generated by the thermal image sensor according to the present invention. FIG. 6 is a diagram illustrating a thermal image sensor according to an embodiment of the present invention. FIG. 7 is a flowchart illustrating a method of recognizing a fusion object based on a visible light sensor and a thermal image sensor according to the present invention.

As shown in the figure, the visible light sensor and the thermal image sensor based fusion object recognition method according to the present invention are characterized in that a visible light sensor 10 senses a visible light image of an object and a thermal image sensor 20 senses a thermal image of the object (S10); An image generation step S20 of processing the signal of the visible light sensor 10 by the operation processing unit 20 to generate a visible light image F1 and processing a signal of the thermal image sensor 20 to generate a thermal image F2 ; The arithmetic processing unit 30 adjusts the size of the thermal image F2 so that the thermal image F2 can be overlaid on the visible light image F1 and the visible image F1 and the thermal image F2, F2) an image adjustment step S30 of matching a view point / a field of view; The calculation processing unit 30 determines whether or not there is an area exceeding the threshold value C1 of the temperature inputted in advance in the column image F2 and if the column image F2 exceeds the threshold value C1 (S40) of transmitting the visible light image (F1) and the thermal image (F2) together with the fact to the server (40) and applying an alarm display signal to the server (40) And an image overlay displaying step S50 in which the display unit 50 overlays and displays the visible image F1 and the thermal image F2 transmitted from the operation processing unit 30. [

As shown in the figure, in the method of recognizing a fusion object based on a visible light sensor and a thermal image sensor according to the present invention, the calculation processing unit 30 determines whether the thermal image F2 is a visible light image F1 in the image control step S30, It is preferable to control the saturation of the color set for each of the heat temperature sections or to generate the display or transmission image by removing the thermal lag at a portion lower in temperature than the temperature region of interest Do.

As shown in the figure, in the method of recognizing a fusion object based on a visible light sensor and a thermal image sensor according to the present invention, the operation processing unit 30 generates a visible light image (S30) between the image generating step S20 and the image adjusting step S30 And extracting an image of a main part of the object so as to recognize the object or identify the object from the object image F1.

As shown in the figure, the visible light sensor and the thermal image sensor based fusion object recognition apparatus of the present invention include a visible light sensor 10 for sensing a visible light image of an object, a thermal image sensor 20 for sensing a thermal image of the object, A processing unit 20, a server 40, and a display unit 50. [

The operation processing unit 20 processes the signal of the visible light sensor 10 to generate a visible light image F1 and processes the signal of the thermal image sensor 20 to generate a thermal image F2, The size of the thermal image F2 is adjusted so that the thermal image F2 can be overlaid on the visible light image F1 and the visible light image F1 and the thermal image F2 view point, / Field of view. The server 40 determines whether the operation processing unit 30 has an area exceeding the threshold value C1 of the temperature set in advance in the thermal image F2 and determines whether the thermal image F2 has a threshold value C1), a visible light image F1 and a column image F2 are received together with the fact. The display unit 50 overlays and displays the visible light image F1 and the thermal image F2 transmitted from the operation processing unit 30. [ The arithmetic processing unit 30 adjusts the saturation of the color set for each heat temperature interval so that the thermal image F2 can be recognized on the visible light image F1 in the image adjusting step S30, And generates a display or transmission image by removing a thermal lag in a region where the temperature is lower than the temperature region of interest. In the present invention, it is possible to further include a tracking unit (60) for adjusting a photographing direction of the sensor so as to track the moving object.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, ≪ RTI ID = 0.0 > and / or < / RTI >

It is to be understood that the appended claims are intended to supplement the understanding of the invention and should not be construed as limiting the scope of the appended claims.

10: Visible light sensor
20: Thermal image sensor
30:
40: Server
50:

Claims (5)

An image sensing step (S10) in which the visible light sensor 10 senses a visible light image of an object and the column image sensor 20 senses a column image of the object;
An image generation step S20 of processing the signal of the visible light sensor 10 by the operation processing unit 20 to generate a visible light image F1 and processing a signal of the thermal image sensor 20 to generate a thermal image F2 ;
The arithmetic processing unit 30 adjusts the size of the thermal image F2 so that the thermal image F2 can be overlaid on the visible light image F1 and the visible image F1 and the thermal image F2, F2) an image adjustment step S30 of matching a view point / a field of view;
The calculation processing unit 30 determines whether or not there is an area exceeding the threshold value C1 of the temperature inputted in advance in the column image F2 and if the column image F2 exceeds the threshold value C1 (S40) of transmitting the visible light image (F1) and the thermal image (F2) together with the fact to the server (40) and applying an alarm display signal to the server (40)
An image overlay presentation step (S50) in which the display unit (50) overlays and displays the visible image (F1) and the thermal image (F2) transmitted from the operation processing unit (30);
And a thermal image sensor based fusion object recognition method. The method according to claim 1,
The arithmetic processing unit 30 adjusts the saturation of the color set for each of the thermal temperature intervals so that the thermal image F2 can be recognized on the visible light image F1 in the image control step S30, And a display or transmission image is generated by removing a thermal lag in a part of the image area lower in temperature than the temperature range of interest. The method according to claim 1,
Extracting an image of a main part of the object so that the arithmetic processing unit 30 recognizes the object from the visible light image F1 or confirms the identity between the image generating step S20 and the image adjusting step S30; And a thermal image sensor based on the acquired information. A visible light sensor 10 for sensing a visible light image of an object and a thermal image sensor 20 for sensing a thermal image of the object;
Processes a signal of the visible light sensor 10 to generate a visible light image F1 and processes the signal of the thermal image sensor 20 to generate a thermal image F2,
The arithmetic processing unit 30 adjusts the size of the thermal image F2 so that the thermal image F2 can be overlaid on the visible light image F1 and the visible image F1 and the thermal image F2, F2) an operation processing unit 20 for performing image adjustment for matching a view point / a field of view;

The calculation processing unit 30 determines whether or not there is an area exceeding the threshold value C1 of the temperature inputted in advance in the column image F2 and if the column image F2 exceeds the threshold value C1 A server 40 for receiving the visible light image F1 and the thermal image F2 together with the fact that the area is present;
And a display unit (50) for overlaying and displaying the visible light image (F1) and the thermal image (F2) transmitted from the operation processing unit (30) Based fusion object recognition apparatus. 5. The method of claim 4,
The arithmetic processing unit 30 adjusts the saturation of the color set for each of the thermal temperature intervals so that the thermal image F2 can be recognized on the visible light image F1 in the image control step S30, Wherein the thermal image of the image area is removed by removing a thermal lag in a part of the image area lower in temperature than the temperature range of interest to generate a display or transmission image.
KR1020150102300A 2015-07-20 2015-07-20 Object recognition method and device using visible light sensor and thermal imagery sensor KR20170010543A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020150102300A KR20170010543A (en) 2015-07-20 2015-07-20 Object recognition method and device using visible light sensor and thermal imagery sensor

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020150102300A KR20170010543A (en) 2015-07-20 2015-07-20 Object recognition method and device using visible light sensor and thermal imagery sensor

Publications (1)

Publication Number Publication Date
KR20170010543A true KR20170010543A (en) 2017-02-01

Family

ID=58109211

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020150102300A KR20170010543A (en) 2015-07-20 2015-07-20 Object recognition method and device using visible light sensor and thermal imagery sensor

Country Status (1)

Country Link
KR (1) KR20170010543A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20200043771A (en) * 2018-10-18 2020-04-28 엔씨엔스페이스(주) System and method for fire prediction using thermal image and visible image
CN114494186A (en) * 2022-01-25 2022-05-13 国网吉林省电力有限公司电力科学研究院 Fault detection method for high-voltage power transmission and transformation line electrical equipment
CN114584702A (en) * 2019-09-20 2022-06-03 广州速率信息技术有限公司 Method and system for shooting visible light and thermal imaging overlay

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20200043771A (en) * 2018-10-18 2020-04-28 엔씨엔스페이스(주) System and method for fire prediction using thermal image and visible image
CN114584702A (en) * 2019-09-20 2022-06-03 广州速率信息技术有限公司 Method and system for shooting visible light and thermal imaging overlay
CN114494186A (en) * 2022-01-25 2022-05-13 国网吉林省电力有限公司电力科学研究院 Fault detection method for high-voltage power transmission and transformation line electrical equipment

Similar Documents

Publication Publication Date Title
US9432633B2 (en) Visual command processing
EP3698230B1 (en) Field of view (fov) and key code limited augmented reality to enforce data capture and transmission compliance
US10165202B2 (en) Method and system for performing alignment of a projection image to detected infrared (IR) radiation information
US9835445B2 (en) Method and system for projecting a visible representation of infrared radiation
KR102022970B1 (en) Method and apparatus for sensing spatial information based on vision sensor
US20170054962A1 (en) Three-dimensional depth perception method and apparatus with an adjustable working range
US20150009295A1 (en) Three-dimensional image acquisition apparatus and image processing method using the same
JP6992035B2 (en) How to determine misalignment of the measurement range of an imager and a non-temporary computer-readable medium
KR101998639B1 (en) Intelligent system for ignition point surveillance using composite image of thermal camera and color camera
KR101411426B1 (en) Thermal image system capable of chasing specific position in thermal image and method for chasing specific position using thereof
KR102048411B1 (en) Wearable device and image processing method thereof
KR20170010543A (en) Object recognition method and device using visible light sensor and thermal imagery sensor
WO2017187694A1 (en) Region of interest image generating device
CN104090656A (en) Eyesight protecting method and system for smart device
KR102203687B1 (en) Stereo camera apparatus and rectification method of the same
KR20230017800A (en) Elevated Temperature Screening Systems and Methods
KR101961266B1 (en) Gaze Tracking Apparatus and Method
KR101242891B1 (en) Apparatus and method for extracting depth map image
KR102347226B1 (en) Fusion sensor data visualization device and method
CN106131448A (en) The 3 d stereoscopic vision system of brightness of image can be automatically adjusted
KR102354704B1 (en) Automatic color map expression method by removing outlier temperature for thermal image
JP2007094536A (en) Device and method for tracking physical object
KR102086594B1 (en) Apparatus for stereo matching
US11132579B2 (en) Contour recognition device, contour recognition system and contour recognition method
JP2013210778A (en) Imaging apparatus

Legal Events

Date Code Title Description
A201 Request for examination
E601 Decision to refuse application