WO2021260871A1 - Appareil et procédé de traitement d'image infrarouge - Google Patents

Appareil et procédé de traitement d'image infrarouge Download PDF

Info

Publication number
WO2021260871A1
WO2021260871A1 PCT/JP2020/024953 JP2020024953W WO2021260871A1 WO 2021260871 A1 WO2021260871 A1 WO 2021260871A1 JP 2020024953 W JP2020024953 W JP 2020024953W WO 2021260871 A1 WO2021260871 A1 WO 2021260871A1
Authority
WO
WIPO (PCT)
Prior art keywords
correction coefficient
thermal image
pixel
thermal
image processing
Prior art date
Application number
PCT/JP2020/024953
Other languages
English (en)
Japanese (ja)
Inventor
俊樹 藤野
康平 栗原
孝一 山下
大祐 鈴木
Original Assignee
三菱電機株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 三菱電機株式会社 filed Critical 三菱電機株式会社
Priority to JP2022532169A priority Critical patent/JP7282268B2/ja
Priority to PCT/JP2020/024953 priority patent/WO2021260871A1/fr
Priority to CN202080102194.XA priority patent/CN115836530A/zh
Priority to US17/925,829 priority patent/US20230177653A1/en
Priority to DE112020007355.6T priority patent/DE112020007355T5/de
Publication of WO2021260871A1 publication Critical patent/WO2021260871A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/30—Transforming light or analogous information into electric information
    • H04N5/33—Transforming infrared radiation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/20—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from infrared radiation only
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H04N23/81—Camera processing pipelines; Components thereof for suppressing or minimising disturbance in the image signal generation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/67—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response
    • H04N25/671—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response for non-uniformity detection or correction
    • H04N25/677—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to fixed-pattern noise, e.g. non-uniformity of response for non-uniformity detection or correction for reducing the column or line fixed pattern noise
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10048—Infrared image

Definitions

  • the present disclosure relates to an infrared image processing apparatus and an infrared image processing method for performing noise reduction processing of thermal images.
  • the infrared image processing device that receives infrared rays and generates a thermal image is equipped with a plurality of infrared detection elements for receiving infrared rays.
  • the infrared detection elements are arranged in a matrix so as to correspond to the generated image.
  • separate drive lines are connected for each row or column in which the elements are lined up, and power is supplied from each drive line. If the characteristics of these drive lines vary, the relationship between the input value to the infrared detection element and the output value from the infrared detection element changes for each row or column. This change may cause streaky noise extending in the row or column direction in the thermal image generated by the infrared image processor.
  • the image pickup apparatus of Patent Document 1 proposes to reduce the noise by the following method.
  • the image pickup apparatus of Patent Document 1 captures images in a light-shielded state at regular intervals, and measures output signals at the time of light-shielding and at the time of exposure.
  • the image pickup apparatus of Patent Document 1 reduces noise by generating a correction signal corresponding to a noise offset component and a light amount-dependent component from the measured value and the ideal value.
  • the present disclosure has been made to solve the above-mentioned problems, and it is possible to calculate a correction coefficient for correcting streak noise from a plurality of images, and the calculated correction coefficient is used with high accuracy. It is an object of the present invention to provide an infrared image processing apparatus capable of reducing streak noise.
  • the infrared image processing apparatus includes a thermal image sensor that receives infrared rays and outputs a signal corresponding to the infrared rays, a thermal image generation unit that generates a plurality of thermal images based on the signals, and a plurality of images.
  • Thermal image For each pixel of each image smoothing processing is performed using the pixel values of the surrounding pixels, and a plurality of smoothed images are calculated to calculate the smoothed pixel value, which is the smoothed pixel value.
  • the infrared image processing apparatus calculates the correction coefficient based on the difference in the pixel values of the pixels included in a plurality of images.
  • the calculated correction coefficient can accurately reduce streak noise.
  • FIG. 1 It is a block diagram which shows the structure of the infrared image processing apparatus which concerns on Embodiment 1.
  • FIG. It is a graph which shows the relationship between the input value and the output value of the thermal image sensor which concerns on Embodiment 1.
  • FIG. It is a conceptual diagram for demonstrating the smoothing process of the infrared image processing apparatus which concerns on Embodiment 1.
  • FIG. It is a conceptual diagram for demonstrating the thermal image and the target pixel value sequence in the infrared image processing apparatus which concerns on Embodiment 1.
  • FIG. It is a conceptual diagram for demonstrating the method of calculating the correction coefficient in the infrared image processing apparatus which concerns on Embodiment 1.
  • FIG. It is a flowchart up to showing the process of outputting the smoothed image of the infrared image processing apparatus 1 which concerns on Embodiment 1.
  • FIG. It is a flowchart up to showing the process of outputting the correction coefficient of the infrared image processing apparatus 1 which concerns on Embodiment 1.
  • FIG. It is a flowchart until the corrected thermal image of the infrared image processing apparatus 1 which concerns on Embodiment 1 is output.
  • It is a block diagram which shows the structure of the infrared image processing apparatus which concerns on Embodiment 2.
  • FIG. 1 It is a schematic diagram of the thermal image sensor and the sensor moving part of the infrared image processing apparatus which concerns on Embodiment 2, and is a schematic diagram of a thermal image. It is a figure which shows an example of the hardware configuration of the respiratory information estimation apparatus which concerns on Embodiment 2. FIG. It is a figure which shows another example of the hardware composition of the respiratory information estimation apparatus which concerns on Embodiment 2. FIG.
  • FIG. 1 is a block diagram showing a configuration of an infrared image processing apparatus according to the first embodiment.
  • the infrared image processing device 1 includes a thermal image sensor 2 and a thermal image processing device 3.
  • the thermal image processing device 3 includes a thermal image generation unit 31, a smoothing processing unit 32, a correction coefficient calculation unit 33, and a thermal image correction unit 34.
  • the thermal image sensor 2 sends a voltage signal VI to the thermal image generation unit 31.
  • the thermal image generation unit 31 sends the thermal image TI to the smoothing processing unit 32, the correction coefficient calculation unit 33, and the thermal image correction unit 34.
  • the smoothing processing unit 32 sends the smoothed image FI to the correction coefficient calculation unit 33.
  • the correction coefficient calculation unit 33 sends the correction coefficient c and the correction coefficient d to the thermal image correction unit 34 via a memory or the like.
  • the thermal image correction unit 34 sends the corrected thermal image TO to the output destination as the output of the thermal image processing device 3 and the infrared image processing device 1.
  • the thermal image sensor 2 is installed in an electric product installed in the room, and images the room at regular intervals when the electric product is in operation. Next, the thermal image sensor 2 receives infrared rays (electromagnetic waves of about 8 ⁇ m to 12 ⁇ m) radiated from an object existing in the room and outputs a voltage signal VI corresponding to the intensity of the infrared rays.
  • the thermal image sensor 2 is composed of a plurality of infrared detection elements that detect infrared rays radiated from an object and output a voltage signal VI corresponding to the intensity of the infrared rays.
  • the infrared detection element is, for example, a pyroelectric element.
  • the plurality of infrared detection elements are arranged in a matrix, and one drive line is connected to the elements arranged in the row direction to supply electric power.
  • the thermal image sensor 2 transmits the voltage signal VI output by the infrared detection element to the thermal image processing device 3.
  • the thermal image sensor 2 transmits a voltage signal VI corresponding to infrared rays to the thermal image processing device 3 in the order associated with the arrangement of the infrared detection elements.
  • the thermal image TI is generated by the thermal image processing device 3 based on the voltage signal VI of the thermal image sensor 2, and is an image obtained by capturing a plurality of regions having different temperatures in the room in an exposed state.
  • the thermal image TI includes a plurality of pixels having different pixel values from the first frame to the second frame at a specific pixel position. For example, the first frame and the second frame are adjacent frames.
  • the frame number is, for example, a number for expressing the image of frame number 1 as the image of frame number 10 when there are 10 images.
  • the pixel value is a value obtained by converting the voltage value of the voltage signal VI into a digital value. Specifically, the pixel value is obtained by the following method.
  • the thermal image processing device 3 stores the voltage value of the voltage signal VI in a memory or the like (not shown in FIG. 1).
  • the thermal image processing device 3 reads out the voltage value and the order information, and converts the voltage value into a pixel value associated in advance.
  • the thermal image processing device 3 arranges the pixel values using the order information.
  • the thermal image processing device 3 generates a single thermal image from the arranged pixel values and stores it in a memory or the like (not shown).
  • FIG. 2 is a graph showing the relationship between the input value and the output value of the thermal image sensor according to the first embodiment.
  • the infrared detection element described here is affected by, for example, manufacturing errors and usage environment. Therefore, each infrared detection element has a different relationship between the input value (infrared intensity) to the infrared detection element and the output value (pixel value of the thermal image) from the infrared detection element.
  • the coefficient a and the coefficient b of the linear function are different for each infrared detection element.
  • the infrared detection element E can be represented by a linear function using the coefficients a1 and the coefficient b1.
  • the infrared detection element F can be represented by a linear function using the coefficient a2 and the coefficient b2.
  • the infrared detection element G can be represented by a linear function using the coefficients a3 and the coefficient b3.
  • the output value Y3 of the infrared detection element G is larger than the output value Y1 of the infrared detection element E.
  • the infrared detection elements lined up in the row direction are connected to the same drive line and receive power. Therefore, the coefficient a indicating the sensitivity of the infrared detection elements arranged in the row direction is affected by the characteristics of the drive line. Infrared detection elements arranged in the row direction have higher sensitivity than other infrared detection elements if the inclination is larger than that of other infrared detection elements. Further, the coefficient b, which is an intercept component, also changes for each infrared detection element in the same manner as the coefficient a. From the above, the output values of the infrared detection elements arranged in the row direction fluctuate.
  • streaky noise (hereinafter referred to as streak noise) extending in the row direction from one end to the other end of the thermal image is generated in the thermal image.
  • the coefficient a indicating the sensitivity of the infrared detection elements arranged in the row direction or the coefficient b which is an intercept component changes due to a change over time in the characteristics of the drive line. Therefore, the intensity of the streak noise also changes with time.
  • the thermal image processing device 3 receives the voltage signal VI from the thermal image sensor 2. That is, in the thermal image processing device 3, the thermal image generation unit 31 closest to the input unit receives the voltage signal VI from the thermal image sensor 2. The thermal image generation unit 31 converts the voltage signal VI of the thermal image sensor 2 into pixel values. Next, the thermal image generation unit 31 generates a thermal image TI including a plurality of pixels having different pixel values in the direction in which streak noise is generated from the converted pixel values. Then, the thermal image generation unit 31 sends the generated thermal image TI to the smoothing processing unit 32, the correction coefficient calculation unit 33, and the thermal image correction unit 34.
  • the smoothing processing unit 32 performs smoothing processing on the thermal image TI for each target pixel value using adjacent pixel values.
  • the smoothing processing unit 32 generates a smoothed image FI that has undergone smoothing processing, and sends it to the correction coefficient calculation unit 33.
  • the target pixel described above is a pixel at a pixel position that serves as a reference when performing smoothing processing, correction coefficient calculation processing, or correction processing.
  • the target pixel is included in a pixel array which is a set of pixels arranged in a direction in which streak noise is generated in the thermal image TI generated by the thermal image generation unit 31.
  • the above-mentioned adjacent pixels are included in a pixel array adjacent to a direction intersecting a direction in which streak noise is generated.
  • the adjacent pixel is adjacent to the target pixel.
  • the smoothed pixel value is an average value of the target pixel value and the adjacent pixel value.
  • FIG. 3 is a conceptual diagram for explaining the smoothing process of the infrared image processing apparatus 1 according to the first embodiment.
  • FIG. 3A shows a thermal image of L rows and N columns.
  • FIG. 3 (b) shows a smoothing thermal image of L rows and N columns.
  • the row having the target pixel value is defined as the L1 row.
  • the smoothing processing unit 32 obtains the average value of the target pixel value PX1 and the corresponding adjacent pixel value PX2 and the adjacent pixel value PX3. This average value is obtained from the leftmost target pixel value and the leftmost adjacent pixel value in the pixel column of the L1 row.
  • the next average value is obtained by similarly processing the target pixel value and the adjacent pixel value from the second left to the left of the pixel column in the L1 row.
  • the average value is obtained, for example, in the range of the frame H in FIG. 3 (a).
  • a smoothing thermal image composed of smoothing pixel values in which streak noise is reduced in the L1 row can be obtained.
  • the correction coefficient calculation unit 33 calculates the correction coefficient c and the correction coefficient d.
  • the correction coefficient c may be rephrased as the first correction coefficient, the correction coefficient d as the second correction coefficient, and the first correction coefficient and the second correction coefficient as the correction coefficient group.
  • the pixel value z is the corrected pixel value, and the pixel value m is the pixel value of the thermal image.
  • the correction coefficient calculation unit 33 stores the calculated correction coefficient c and the correction coefficient d in a memory or the like (not shown).
  • the thermal image correction unit 34 corrects the image TI generated by the thermal image generation unit 31 using the correction coefficient c and the correction coefficient d read from the memory.
  • FIG. 4 is a conceptual diagram for explaining a thermal image and a target pixel value sequence in the infrared image processing apparatus according to the first embodiment.
  • FIG. 5 is a conceptual diagram for explaining a smoothed thermal image and a target pixel value sequence thereof in the infrared image processing apparatus according to the first embodiment. For example, a case where a coordinate value is set from the target pixel value and the smoothed pixel value in the L row and Nth column will be described.
  • the correction coefficient calculation unit 33 reads out the target pixel value column from the pixel in the L row and Nth column of the first frame of the thermal image to the pixel in the L row and Nth column of the second frame.
  • the target pixel value sequence (R1, R2, and so on) in the thermal image as shown in FIG. 4B. R3, ..., Rn) are set.
  • FIG. 6 is a conceptual diagram for explaining a method of calculating a correction coefficient in the infrared image processing apparatus according to the first embodiment.
  • the correction coefficient calculation unit 33 calculates the correction coefficient c and the correction coefficient d.
  • the correction coefficient c and the correction coefficient d are different for each pixel position.
  • the plurality of coordinate values are plotted at different positions. As described above, since the plurality of thermal images include a plurality of pixels having different pixel values in the direction (row direction) in which the streak noise is generated, the positions of the plots are deviated.
  • the correction coefficient calculation unit 33 calculates the correction coefficient c and the correction coefficient d based on this difference in position, that is, the difference in pixel value.
  • the correction coefficient calculation unit 33 uses, for example, the least squares method for calculating the correction coefficient c and the correction coefficient d.
  • the correction coefficient calculation unit 33 stores the correction coefficients c and d in a memory or the like (not shown).
  • the thermal image correction unit 34 acquires the pixel value z in the thermal image TI generated by the thermal image generation unit 31.
  • the pixel value z is a pixel value for a pixel sequence from the target pixel of the first frame to the target pixel of the second frame.
  • the thermal image correction unit 34 substitutes the pixel sequence from the target pixel of the first frame to the target pixel of the second frame into a correction formula using the stored correction coefficient c and correction coefficient d.
  • the thermal image correction unit 34 outputs the thermal image TO, which outputs the pixel value z for the image TI, to the output destination.
  • FIG. 7 is a flowchart up to showing a process of outputting a smoothed image FI of the infrared image processing apparatus 1 according to the first embodiment.
  • the operation of the thermal image sensor 2 in the infrared image processing apparatus 1 and the smoothing processing unit 32 of the thermal image processing apparatus 3 will be described with reference to the flowchart of FIG.
  • the infrared image processing device 1 generates a thermal image TI, performs smoothing processing on the thermal image TI for each target pixel value using adjacent pixel values, and outputs a smoothed image FI.
  • step S001 the thermal image generation unit 31 generates a thermal image TI from the voltage signal VI of the thermal image sensor 2.
  • step S002 the smoothing processing unit 32 determines the pixel values of the pixel columns of the Lth row, the L-1st row, and the L + 1st row in the plurality of thermal images generated by the thermal image generation unit 31. get.
  • L is a natural number.
  • the L-1th row in the thermal image does not exist. Therefore, for example, the pixel sequence similar to the Lth row or the L + 1st row is treated as existing in the L-1th row in the thermal image. That is, the pixel sequence of the Lth row or the L + 1st row read separately is treated as the pixel row of the L-1th row in the thermal image. Similarly, since the pixel string in the L + 1 row does not exist in the lowermost row of the thermal image, the pixel string in the Lth row or the L-1th row read separately is treated as the pixel string in the L + 1 row in the thermal image. Further, here, the pixel value included in the Lth row is the target pixel value, and the pixel values included in the L-1st row and the L + 1th row are adjacent pixel values.
  • step S003 the smoothing processing unit 32 smoothes each target pixel value in the Lth row with the adjacent pixel values that are the pixels in the L-1st row and the L + 1st row adjacent to each target pixel. do. Then, the smoothing processing unit 32 acquires the smoothing pixel value of each pixel in the Lth row.
  • step S004 the smoothing processing unit 32 replaces L with L + 1.
  • step S005 the smoothing processing unit 32 determines whether or not the pixel string exists in the L row replaced with L + 1 in the previous step. When the pixel sequence exists (YES in step S005), the processes of steps S002 to S004 are repeated. If the pixel sequence does not exist (NO in step S005), the process proceeds to the next step.
  • FIG. 8 is a flowchart up to showing the process of outputting the correction coefficient c and the correction coefficient d of the infrared image processing apparatus 1 according to the first embodiment.
  • the operation of the correction coefficient calculation unit 33 of the thermal image processing device 3 in the infrared image processing device 1 will be described with reference to the flowchart of FIG.
  • the pixel value z is the corrected pixel value, and the pixel value m is the pixel value of the thermal image.
  • the correction coefficient calculation unit 33 calculates the correction coefficient c and the correction coefficient d.
  • Step S006 is a step following step S005.
  • the correction coefficient calculation unit 33 acquires the pixel values in the L rows and N columns and the smoothed pixel values in the L rows and N columns, and a plurality of coordinate values in the corresponding target pixel strings and the smoothed pixel strings.
  • the pixel value in the L row and N column is a pixel value for each pixel column from the pixel in the L row and Nth column of the first frame to the pixel in the L row and Nth column of the second frame.
  • the smoothed pixel value in the L row and N column is the smoothed pixel value for each pixel column from the pixel in the L row and Nth column of the first frame to the pixel in the L row and Nth column in the second frame.
  • L and N are natural numbers.
  • the plurality of coordinate values are plotted at different positions.
  • the correction coefficient calculation unit 33 calculates the correction coefficients c and d based on this difference in position, that is, the difference in pixel value.
  • step S008 the correction coefficient calculation unit 33 performs a process of replacing L with L + 1.
  • step S009 the correction coefficient calculation unit 33 determines whether or not the pixel string exists in the L row replaced with L + 1 in the previous step. When the pixel column exists in the Lth row (YES in step S009), the processes of steps S006 to S009 are repeated. When the pixel string does not exist in the Lth row (NO in step S009), it means that the pixel string of the pixel in the Nth column has been corrected, and the process proceeds to the next step S010.
  • step S010 the correction coefficient calculation unit 33 performs a process of replacing N with N + 1.
  • step S011 the correction coefficient calculation unit 33 determines whether or not the pixel string exists in the Nth column replaced with N + 1 in the previous step. When the pixel row exists in the Nth row (YES in step S011), the processing of step S006 to step S011 is repeated.
  • step S011 When the pixel column does not exist in the Nth column (NO in step S011), it means that the calculation of the correction coefficients c and d is completed in all the rows of the thermal image, and the process proceeds to the next step.
  • FIG. 9 is a flowchart up to the output of the corrected thermal image TO of the infrared image processing apparatus 1 according to the first embodiment. The operation until the thermal image correction unit 34 in the infrared image processing apparatus 1 outputs the corrected thermal image TO will be described with reference to the flowchart of FIG.
  • Step S012 is a step following step S011.
  • the thermal image correction unit 34 acquires the pixel values of the pixels in the P row and Q column.
  • the pixel value of the pixel in the P row and Q column is each pixel of the pixel column from the pixel in the P row and Q column of the first frame to the pixel in the P row and Q column of the second frame in the thermal image.
  • the value. P and Q are natural numbers.
  • step S013 the thermal image correction unit 34 corrects the pixel values of the pixels in the P row and Q column using the corresponding correction formula.
  • the pixel values of the pixels in the P row and Q column are the pixel values of the pixels in the P row and Q column of the first frame to the pixel columns in the P row and Q column of the second frame.
  • step S014 the thermal image correction unit 34 performs a process of replacing P with P + 1.
  • step S015 the correction coefficient calculation unit 33 determines whether or not the pixel string exists in the P row replaced with P + 1 in the previous step. When the pixel column exists in the P row (YES in step S015), the processes of steps S012 to S014 are repeated. When the pixel string does not exist in the P row (NO in step S015), it means that the pixel string of the pixel in the Q column has been corrected, and the process proceeds to the next step.
  • step S016 the thermal image correction unit 34 performs a process of replacing Q with Q + 1.
  • step S017 determines whether or not the pixel string exists in the Q column replaced with Q + 1 in the previous step. When the pixel string exists in the Q column (YES in step S017), the processes of steps S012 to S016 are repeated. When the pixel string does not exist in the Qth column (NO in step S017), it means that the pixel string of the pixel in the Qth column has been corrected, and the process proceeds to the next step.
  • the thermal image correction unit 34 reconstructs the image using the corrected pixel value, and generates the corrected image. Specifically, the thermal image correction unit 34 rearranges the pixel rows from the corrected pixel values and pixel position information of each pixel row according to the pixel position information, and generates a plurality of thermal images.
  • step S019 the thermal image correction unit 34 outputs the corrected thermal image TO to the output destination.
  • the infrared image processing apparatus 1 is configured as described above, and has the following effects.
  • the infrared image processing apparatus 1 obtains a correction coefficient c and a correction coefficient d, and corrects the thermal image TI using them.
  • the infrared image processing apparatus 1 cannot sufficiently reduce the streak noise.
  • any input value can be used. Only correction that adds the difference value can be done.
  • the infrared image processing device 1 uses the smoothed pixel value obtained from the thermal image, the correction coefficient can be calculated using the thermal image during operation in which the temperature condition is not constant. Therefore, it is possible to update the correction coefficient while processing the infrared image processing device 1.
  • the infrared image processing device 1 sets coordinate values using the target pixel value of the thermal image and the smoothed pixel value of the smoothed thermal image, and calculates the correction coefficient by linearly approximating the coordinate values. .. Therefore, the correction coefficient can be calculated only by setting the coordinate value and performing the processing of one straight line approximation, and the correction coefficient can be updated efficiently.
  • the edge portion of the heat source body is located on the line for calculating the correction coefficient, the edge portion will be smoothed and the accuracy of the smoothed pixel value of the smoothed thermal image will decrease.
  • the thermal image processing apparatus 3 cannot obtain an appropriate thermal image.
  • the ratio of the smoothed pixel value of the edge portion to the entire pixel is reduced by approximating the plurality of coordinate values with a straight line. Therefore, the infrared image processing device 1 can suppress a decrease in correction accuracy due to the edge portion.
  • the target pixel value of the thermal image and the smoothed pixel value of the smoothed thermal image are input with data such as generation and correction of thermal images in the order of input to the thermal image sensor 2. It was decided to process in the order.
  • the order is the order of processing from the pixel at one end of the first row of the first column to the pixel to the other end, and then processing from the pixel at one end of the second row of the first column to the pixel to the other end.
  • the direction from the pixel at one end to the pixel at the other end is the row direction of the thermal image.
  • the thermal image processing apparatus 3 may sort the target pixel values of the thermal image and arrange them in descending order or ascending order in a straight line approximation.
  • a sorting unit for sorting the pixel values of a plurality of thermal images in descending order or ascending order in the frame direction is provided.
  • the thermal image processing apparatus 3 may perform a process of sorting the smoothed pixel values of the smoothed thermal image and arranging them in descending order or ascending order by linear approximation and correction.
  • a sorting unit is provided for sorting the pixel values of a plurality of smoothed thermal images in the descending order or the ascending order in the frame direction.
  • the thermal image processing apparatus 3 may set the sorting direction from the pixel at one end to the pixel at the other end.
  • the thermal image processing apparatus 3 sets the sorting direction to the frame direction, that is, the pixels in the first column and the first row, the pixels in the first column and the first row of the next frame, and the first row in the first column of the next frame. It may be in the direction of the pixel of.
  • the thermal image processing device 3 when the sorting direction is the direction from the pixel at one end to the pixel from the other end, the pixel located at the end of the pixel array in which the sorted pixels are arranged. The value may be removed. Specifically, in the thermal image processing apparatus 3, it is determined whether or not the first threshold value is exceeded in order from the largest value of the sorted pixel strings. Next, the thermal image processing device 3 deletes the pixel value exceeding the threshold value from the pixel sequence. Then, the thermal image processing apparatus 3 determines whether or not the second threshold value is in order from the smallest value of the sorted pixel sequence. Then, the thermal image processing device 3 deletes the pixel value below the second threshold value from the pixel sequence.
  • the first threshold value is larger than the second threshold value.
  • a pixel value below the designated first threshold value and a pixel value above the designated second threshold value are used.
  • a pixel value below the designated first threshold value or a pixel value above the designated second threshold value may be used.
  • the number of pixel values to be removed may be predetermined without using a threshold value, and the corresponding number of pixel values may be removed from the largest value in the pixel string and from the smallest value in the pixel string.
  • removing the pixel value located at the end of the pixel array in which the sorted pixels are arranged means that the large value and the small value in the pixel array are not used. For example, if there are light bulbs, gas stoves, people, etc. in the room, their contours will appear as edges in the thermal image. When the thermal image is smoothed, the edge portion is smoothed, so that the accuracy of the smoothed pixel value is lowered. The same applies to windows cooled by the outside air. By removing the edge portion in the thermal image, it is possible to suppress a decrease in the accuracy of the smoothed pixel value.
  • removing the pixel value located at the end of the pixel array in which the sorted pixels are arranged is not limited to this, for example, as long as it is a method of detecting the maximum pixel value and the minimum pixel value in the image.
  • the direction in which streak noise is generated is the row direction of the thermal image, but it may be the column direction or the diagonal direction. It was mentioned that the fact that the drive line is connected to the infrared detection element arranged in the row direction is the cause of the streak noise in the row direction, but when the drive line is connected in the column direction or the diagonal direction, the column Streaky noise is generated in the direction or diagonal direction.
  • streak noise is generated due to the difference in the characteristics of the drive line, but the cause of the streak noise is not limited to this.
  • infrared detection elements arranged in a row direction or the like are connected by the same output line (a line that transmits a voltage signal)
  • streak noise may occur due to variations in the characteristics of the output lines.
  • the characteristics of the A / D converter, amplifier, etc. connected to this output line vary.
  • noise reduction processing may be performed in addition to the processing of reducing streak noise. When this processing is performed line by line, the degree of noise reduction processing differs for each line, and streaky noise may occur.
  • the infrared image processing apparatus described above can similarly reduce these streaky noises.
  • FIG. 10 is a block diagram showing the configuration of the infrared image processing apparatus according to the second embodiment.
  • FIG. 11 is a schematic view of a thermal image sensor and a sensor moving portion of the infrared image processing apparatus according to the second embodiment, and a schematic diagram of a thermal image.
  • the same parts as the configuration and operation described in the first embodiment will be omitted, and the parts different from the first embodiment will be described below.
  • the thermal image sensor 2 in which the infrared detection elements are arranged in a matrix is used.
  • a thermal image sensor 502 in which a plurality of infrared detection elements are arranged in a row in the vertical direction is used.
  • the infrared image processing device generates a single thermal image by moving the thermal image sensor 502 in the lateral direction and synthesizing the voltage signal VI which is the output of the infrared detection element. By repeating this operation, the infrared image processing apparatus 501 generates a plurality of images.
  • the infrared image processing apparatus 501 of the second embodiment has a different configuration of the thermal image sensor 502 from the thermal image sensor 2 of the first embodiment, and as shown in FIG. 10, the infrared image of the first embodiment.
  • a sensor moving unit 504 is added to the configuration of the processing device 1.
  • the thermal image sensor 502 has a plurality of infrared detection elements arranged in a row in the vertical direction.
  • the sensor moving unit 504 moves the thermal image sensor 502 laterally so as to change the position of the thermal image sensor 502. That is, the sensor moving unit 504 moves the thermal image sensor 502 in the direction intersecting the direction in which the thermal image sensors 502 are lined up.
  • the sensor moving unit 504 is a motor.
  • the rotational operation of the motor is transmitted to the thermal image sensor 502.
  • the connection between the thermal image sensor 502 and the sensor moving unit 504 is shown by a dotted line connecting the two.
  • the sensor moving unit 504 includes a rotary encoder. The sensor moving unit 504 transmits a signal indicating the moving amount of the thermal image sensor 502, which is the rotation amount of the motor, to the thermal image processing device 503 (more specifically, the thermal image generation unit 531).
  • the thermal image generation unit 531 in the infrared image processing device 501 of the second embodiment is from the thermal image sensor 502, the voltage signal VI corresponding to the infrared rays received with the rotational movement of the thermal image sensor 502, and the sensor moving unit 504. Receives a signal indicating the amount of movement of.
  • the thermal image generation unit 531 rearranges the voltage signal VI to generate a single thermal image.
  • the thermal image sensor 502 rotates so as to draw an arc concentrically with the rotation axis, and transmits a voltage signal VI corresponding to the received infrared rays.
  • the thermal image sensor 502 and the sensor moving unit 504 are connected by a dotted line, but they are actually connected by a structure.
  • the thermal image sensor 502 combines a thermal image I1 generated from a voltage signal VI at a certain position in rotational movement and a thermal image I1 at a plurality of positions into a single sheet. Obtain a thermal image I2.
  • the infrared image processing device 501 is an infrared detection element in a matrix of several rows (three rows in FIG. 11 (c)) in an arc shape as the sensor moving unit 504 rotates. May be placed.
  • the infrared image processing device 1 receives a voltage signal VI corresponding to the infrared rays received with the rotational movement of the thermal image sensor 502b and a signal indicating the amount of movement from the sensor moving unit 504 to obtain a single thermal image. To generate.
  • the infrared image processing apparatus 501 according to the second embodiment of the present disclosure is configured as described above, and has the following effects.
  • the infrared image processing apparatus 501 of the second embodiment can capture a thermal image even with a small number of infrared detection elements. By reducing the number of infrared detection elements, it can be realized with a device of a small scale with respect to the configuration of the infrared image processing device 501 according to the first embodiment.
  • the infrared image processing apparatus 501 of the second embodiment can calculate the correction coefficient as in the first embodiment, it is possible to perform the process of reducing the streak noise. In addition, it has the same effect as that of the first embodiment.
  • the coefficient and the correction coefficient were calculated by using a linear function as a linear expression or a correction expression, but a polynomial function such as a cubic function may be used.
  • a polynomial function such as a cubic function
  • the pixel value can be approximated with high accuracy.
  • the correction coefficient is two or more.
  • the infrared image processing apparatus 501 may use an infrared detection element such as a thermopile type to which a thermocouple that causes the Seebeck effect is connected, or a bolometer type that utilizes a change in resistance value due to temperature information.
  • the infrared image processing device 501 may be of any type as long as it can detect infrared rays.
  • the thermal image generation unit 31 included in the thermal image processing device 3 receives the voltage signal VI of the thermal image sensor and generates a thermal image.
  • the thermal image generation unit 531 included in the thermal image processing device 503 receives the voltage signal VI of the thermal image sensor and generates a thermal image.
  • the thermal image generation unit 31 may be provided in the thermal image sensor 2
  • the thermal image generation unit 531 may be provided in the thermal image sensor 502.
  • the infrared image processing device 1 and the infrared image processing device 501 described above a process of obtaining an average value of a target pixel value and an adjacent pixel value was performed as a smoothing process.
  • the infrared image processing device 1 or the infrared image processing device 501 may obtain the average value by individually weighting the target pixel value and the adjacent pixel value for obtaining the average value.
  • the infrared image processing device 1 or the infrared image processing device 501 may obtain an average value only by the adjacent pixel value without using the target pixel value.
  • the infrared image processing device 1 and the infrared image processing device 501 performed smoothing processing using three pixel values including a target pixel value and two adjacent pixel values.
  • the infrared image processing device 1 or the infrared image processing device 501 may perform smoothing processing with five pixel values using two pixel values further adjacent to the adjacent pixel values.
  • the infrared image processing device 1 or the infrared image processing device 501 may perform smoothing processing with five or more pixel values using adjacent pixel values.
  • the above-mentioned infrared image processing device 1 and infrared image processing device 501 are assumed to be installed in an electric product in a room. However, by installing the infrared image processing device 1 or the infrared image processing device 501 on an electric product, the thermal image can be used for controlling the electric product. Further, the infrared image processing device 1 or the infrared image processing device 501 can use a thermal image to confirm the temperature condition in the room where the electric product is installed. Further, in the infrared image processing device 1 or the infrared image processing device 501, the infrared image processing device may be used not only as an electric product but also as a security or surveillance camera installed indoors or outdoors.
  • FIG. 12 is a diagram showing an example of the hardware configuration of the thermal image processing device 503 (thermal image processing device 3).
  • FIG. 13 is a diagram showing another example of the hardware configuration of the thermal image processing device 3.
  • the thermal image processing device 503 is composed of, for example, at least one processor 101a and a memory 101b.
  • the processor 101a is, for example, a Central Processing Unit (CPU) that executes a program stored in the memory 101b.
  • the function of the thermal image processing apparatus 503 is realized by software, firmware, or a combination of software and firmware.
  • the software and firmware are stored in the memory 101b as a program. Thereby, the program for realizing the function of the thermal image processing apparatus 503 is executed by the computer.
  • the memory 101b is a computer-readable recording medium, and is, for example, a volatile memory such as a RAM (Random Access Memory) and a ROM (Read Only Memory), a non-volatile memory, or a combination of a volatile memory and a non-volatile memory. Is.
  • a volatile memory such as a RAM (Random Access Memory) and a ROM (Read Only Memory)
  • ROM Read Only Memory
  • non-volatile memory or a combination of a volatile memory and a non-volatile memory.
  • the thermal image processing apparatus 503 may be configured by a processing circuit 101c as dedicated hardware such as a single circuit or a composite circuit. In this case, the function of the thermal image processing apparatus 503 is realized by the processing circuit 101c.
  • 1 Infrared image processing device 2 Thermal image sensor, 3 Thermal image processing device, 31 Thermal image generation unit, 32 Smoothing processing unit, 33 Correction coefficient calculation unit, 34 Thermal image correction unit, 504 Sensor movement unit.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Photometry And Measurement Of Optical Pulse Characteristics (AREA)
  • Radiation Pyrometers (AREA)
  • Transforming Light Signals Into Electric Signals (AREA)

Abstract

Un appareil de traitement d'image infrarouge capable de réduire le bruit de traînée en calculant un coefficient de correction pour corriger un bruit de traînée à partir d'une pluralité d'images est divulgué.　Cet appareil de traitement d'image infrarouge est caractérisé en ce qu'il comprend : un capteur d'image thermique qui reçoit des rayons infrarouges et émet un signal correspondant aux rayons infrarouges ; une unité de génération d'image thermique qui génère une pluralité d'images thermiques sur la base du signal ; une unité de traitement de lissage qui réalise un traitement de lissage pour chaque pixel de chaque image de la pluralité d'images thermiques en utilisant des valeurs de pixel de pixels environnants, calcule une pluralité d'images lissées et calcule une valeur de pixel lissée qui est une valeur de pixel lissée de chaque image ; une unité de calcul de coefficient de correction qui calcule un groupe de coefficients de correction comprenant un premier coefficient de correction et un second coefficient de correction à partir des images thermiques ou des images lissées ; et une unité de correction d'image thermique qui corrige une image thermique à l'aide du groupe de coefficients de correction.
PCT/JP2020/024953 2020-06-25 2020-06-25 Appareil et procédé de traitement d'image infrarouge WO2021260871A1 (fr)

Priority Applications (5)

Application Number Priority Date Filing Date Title
JP2022532169A JP7282268B2 (ja) 2020-06-25 2020-06-25 赤外線画像処理装置および赤外線画像処理方法
PCT/JP2020/024953 WO2021260871A1 (fr) 2020-06-25 2020-06-25 Appareil et procédé de traitement d'image infrarouge
CN202080102194.XA CN115836530A (zh) 2020-06-25 2020-06-25 红外线图像处理装置和红外线图像处理方法
US17/925,829 US20230177653A1 (en) 2020-06-25 2020-06-25 Infrared image processing device and infrared image processing method
DE112020007355.6T DE112020007355T5 (de) 2020-06-25 2020-06-25 Infrarotbild-verarbeitungseinrichtung und infrarotbild-verarbeitungsverfahren

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2020/024953 WO2021260871A1 (fr) 2020-06-25 2020-06-25 Appareil et procédé de traitement d'image infrarouge

Publications (1)

Publication Number Publication Date
WO2021260871A1 true WO2021260871A1 (fr) 2021-12-30

Family

ID=79282284

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/024953 WO2021260871A1 (fr) 2020-06-25 2020-06-25 Appareil et procédé de traitement d'image infrarouge

Country Status (5)

Country Link
US (1) US20230177653A1 (fr)
JP (1) JP7282268B2 (fr)
CN (1) CN115836530A (fr)
DE (1) DE112020007355T5 (fr)
WO (1) WO2021260871A1 (fr)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20010042825A1 (en) * 1996-08-30 2001-11-22 Young Ching-Ju J. Ditherless non-uniformity compensation for infrared detector arrays with recursive spatial low pass filtering
US20100220193A1 (en) * 2009-03-02 2010-09-02 Flir Systems, Inc. Systems and methods for processing infrared images
CN104657958A (zh) * 2015-03-18 2015-05-27 西安科技大学 一种红外图像条纹噪声消除方法
CN110796621A (zh) * 2019-10-29 2020-02-14 浙江大华技术股份有限公司 红外图像去横纹处理方法、处理设备和存储装置

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4868003B2 (ja) 2009-02-06 2012-02-01 ソニー株式会社 固定パターンノイズ除去回路、固定パターンノイズ除去方法、プログラムおよび撮像装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20010042825A1 (en) * 1996-08-30 2001-11-22 Young Ching-Ju J. Ditherless non-uniformity compensation for infrared detector arrays with recursive spatial low pass filtering
US20100220193A1 (en) * 2009-03-02 2010-09-02 Flir Systems, Inc. Systems and methods for processing infrared images
CN104657958A (zh) * 2015-03-18 2015-05-27 西安科技大学 一种红外图像条纹噪声消除方法
CN110796621A (zh) * 2019-10-29 2020-02-14 浙江大华技术股份有限公司 红外图像去横纹处理方法、处理设备和存储装置

Also Published As

Publication number Publication date
US20230177653A1 (en) 2023-06-08
DE112020007355T5 (de) 2023-04-20
JPWO2021260871A1 (fr) 2021-12-30
JP7282268B2 (ja) 2023-05-26
CN115836530A (zh) 2023-03-21

Similar Documents

Publication Publication Date Title
Budzier et al. Calibration of uncooled thermal infrared cameras
CN109870239B (zh) 非制冷红外焦平面探测器自适应定标方法
CN107532999B (zh) 基于波长带的无源红外气体成像
US20060279632A1 (en) Method for fixed pattern noise reduction in infrared imaging cameras
KR101827810B1 (ko) 보정데이터 생성장치 및 이의 보정데이터 생성방법, 그리고 적외선 카메라
US20200099825A1 (en) Error smoothing through global source non-uniformity correction
WO2004110053A1 (fr) Systeme et procede pour ameliorer efficacement la qualite de l'image dans des cameras
CN109791699B (zh) 辐射成像
US10803595B2 (en) Thermal-image based object detection and heat map generation systems and methods
US20210310941A1 (en) Gas lens filter systems and methods
WO2021260871A1 (fr) Appareil et procédé de traitement d'image infrarouge
JP6984766B2 (ja) 赤外線撮像装置および赤外線撮像プログラム
EP1483556B1 (fr) Ameliorations apportees a l'etalonnage de cameras infrarouges
US20150130972A1 (en) Self-Adaptive Lens Shading Calibration and Correction
US20050194539A1 (en) Method for improving measurement accuracy of infrared imaging radiometers
KR101731287B1 (ko) 적외선 디텍터의 출력 보정 방법 및 장치
Schramm et al. Compensation of the size-of-source effect of infrared cameras using image processing methods
JP7143558B2 (ja) 赤外線撮像装置及びそれに用いられるプログラム
CN104166960A (zh) 一种基于场景的自适应非均匀固定噪声去除方法
KR102297273B1 (ko) 열화상 카메라 온도 보정 방법 및 장치
JP2019095374A (ja) 検出器、ならびに、検出器の補正方法、校正方法、検出装置
KR101870351B1 (ko) 열화상 카메라를 이용한 열영상 처리 방법
CN117053928A (zh) 一种红外热像仪镜头畸变校正及成像非均匀性的标定方法
CN114414060A (zh) 测定装置
Chen et al. A novel non-uniformity correction method based on ROIC

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20941545

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022532169

Country of ref document: JP

Kind code of ref document: A

122 Ep: pct application non-entry in european phase

Ref document number: 20941545

Country of ref document: EP

Kind code of ref document: A1