JPWO2017179510A1 - Image processing apparatus, image processing method, image processing program, and gas detection system - Google Patents

Abstract

The image processing apparatus performs image processing on a plurality of images arranged in time series photographed by a camera. The image processing device calculates a value indicating a spatial change of pixel data and a value indicating a temporal change of the pixel data for the plurality of images, and the first calculation unit. A second calculation unit that statistically processes the value indicating the spatial change and the value indicating the time change, and calculates a value indicating each blur of the plurality of images caused by the camera shake; Is provided.

Description

  The present invention relates to an image processing technique that can cope with noise caused by camera shake.

  When a gas leak occurs, a slight temperature change occurs in the space where the leaked gas is drifting. As a technique for detecting gas using this principle, gas detection using an infrared camera is known.

  The camera shake may occur due to the shaking of the infrared camera during the shooting of the monitoring target (for example, the place where the gas transport pipes are connected) by the infrared camera. Thereby, since the infrared image image | photographed with the infrared camera contains the noise by the blurring of a camera, the detection accuracy of gas falls.

  An imaging device that detects the amount of camera shake by pattern matching between the previous frame image and the current frame image, and forms a low-resolution image from the current frame image based on the amount of camera shake as a technique for dealing with camera shake Has been proposed (see, for example, Patent Document 1).

  As one of the indexes indicating the gas concentration, there is a lower explosive limit (Low Explosive Limit). The unit is vol%. The lower explosion limit is the lowest concentration at which a combustible gas mixed with air will cause an explosion upon ignition. In the case of methane, the lower explosion limit is 5 vol%. Further,% LEL is known as a unit of concentration where the lower explosion limit concentration is 100. That is, a methane concentration of 5 vol% is equal to 100% LEL, and a methane concentration of 10 vol% is equal to 200% LEL.

  In gas detection using an infrared camera, the gas concentration in the space where the gas is drifting cannot be directly measured, but the gas thickness product is measured. The gas concentration / thickness product means a value obtained by integrating the gas concentration along the depth direction of the space where the gas drifts. The unit of the gas concentration thickness product is% LEL · m. For example, when the gas concentration is 200% LEL and the gas thickness is 2 m, the concentration thickness product is 400% LEL · m.

  In some cases, the object to be monitored is photographed remotely (for example, 50 m or more) with an infrared camera, and methane gas having a relatively low concentration (for example, 10% LEL · m) is detected using a gas detection algorithm. In such a case, even if the camera shake amount is extremely small (for example, 0.01 to 0.05 pixels), the present inventor has no gas detection accuracy unless noise from the camera shake is removed from the image. I found it to decline.

  In order to remove noise caused by camera shake from an image, it is necessary to obtain a value (for example, a blur amount of the entire image) indicating image blur caused by camera shake.

JP 2006-067291 A

  An object of the present invention is to provide an image processing apparatus, an image processing method, an image processing program, and a gas detection system that can calculate a value indicating image blur caused by camera shake.

  An image processing apparatus according to a first aspect of the present invention that achieves the above object is an apparatus that performs image processing on a plurality of images arranged in a time series photographed by a camera, and includes a first calculation unit, A second calculation unit. The first calculation unit calculates a value indicating a spatial change of pixel data and a value indicating a temporal change of the pixel data for the plurality of images. The second calculation unit statistically processes the value indicating the spatial change and the value indicating the time change calculated by the first calculation unit, and calculates the plurality of images generated by the camera shake. A value indicating each blur is calculated.

  The above and other objects, features and advantages of the present invention will become apparent from the following detailed description and the accompanying drawings.

It is explanatory drawing explaining the relationship between the value which shows the spatial change of the pixel value by the camera shake generate | occur | produced between the present frame and the following frame, and the value which shows the time change of a pixel value. It is three graphs which show the relationship between the value which shows the change of the x-axis direction of a pixel value, and the value which shows the time change of a pixel value about each of all the pixels which comprise the following flame | frame. It is a block diagram which shows the structure of the gas detection system to which the image processing apparatus which concerns on this embodiment was applied. It is a block diagram which shows the hardware constitutions of the image processing apparatus shown to FIG. 3A. It is explanatory drawing explaining time series pixel data. 4 is a flowchart for explaining the operation of the image processing apparatus according to the first embodiment. It is explanatory drawing explaining the frequency component data which comprise time series pixel data. It is explanatory drawing explaining the band pass filter used by step S1 of FIG. It is a graph which shows the predetermined frequency component data d extracted from the time series pixel data of the 1st pixel. It is explanatory drawing explaining the predetermined frequency component data d per frame. It is explanatory drawing explaining the process of step S2 of FIG. It is explanatory drawing explaining the process which calculates | requires Sx and Sy of the 1st frame in 1st Embodiment. In 1st Embodiment, it is explanatory drawing explaining the process which calculates | requires Sx and Sy of the 2nd-Kth frame. It is a flowchart explaining operation | movement of the image processing apparatus which concerns on 2nd Embodiment. It is explanatory drawing explaining the process which calculates ∂G / ∂t. In 2nd Embodiment, it is explanatory drawing explaining the process which calculates | requires Sx and Sy of the 1st frame. In 2nd Embodiment, it is explanatory drawing explaining the process which calculates | requires Sx and Sy of the 2nd-Kth flame | frame. In the third embodiment, it is a flowchart for explaining the determination process of Sx. It is explanatory drawing explaining the determination process of Sx. It is a schematic diagram of one frame divided into two or more regions. 14 is a flowchart illustrating a process of calculating Sx in the fourth embodiment. It is explanatory drawing explaining that ∂G / ∂ω = y · ∂G / ∂x + x · ∂G / ∂y is established. It is explanatory drawing explaining that ∂G / ∂m = x · ∂G / ∂x + y · ∂G / ∂y is established.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In each figure, the structure which attached | subjected the same code | symbol shows that it is the same structure, The description is abbreviate | omitted about the content which has already demonstrated the structure.

  The principle of this embodiment will be described. In the present embodiment, a value indicating an image blur (blurring of the entire image) caused by camera shake is obtained using image processing, and a signal change Ns caused by camera shake is calculated using this value and a predetermined formula. (That is, noise due to camera shake). FIG. 1 is an explanatory diagram for explaining the relationship between a value indicating a spatial change in the pixel value G due to camera shake occurring between the current frame and the next frame, and a value indicating a temporal change in the pixel value G. . A frame is one still image (image) that constitutes a moving image. In the present embodiment, a moving image is described as an example of a plurality of images arranged in time series photographed by a camera. The pixel value G is an example of pixel data of a pixel. The pixel data is numerical information that the pixel has. For example, in the case of an infrared image, the pixel data means the pixel value G or a temperature calculated using the pixel value G.

  In the current frame, the first to fifth pixels located in the j-th row are shown, and the pixels after the fifth are omitted. In the next frame, the first to fourth pixels located in the j-th row are shown, and the pixels after the fourth are omitted. The direction in which the pixels in the j-th row are arranged is the x-axis direction. A value (∂G / ∂x) indicating a change in the x-axis direction of the pixel value G is an example of a value indicating a spatial change of the pixel value G (pixel data) in one frame. The pixel value G indicates luminance, and the gradation is 256. In the j-th row of the current frame, the pixel value G gradually increases from the first pixel toward the fifth pixel. This means that the subject gradually becomes brighter from the first pixel toward the fifth pixel.

  Assume that a camera shake occurs in the positive direction of the x-axis between the current frame and the next frame without changing the subject. Since the subject gradually becomes brighter toward the first to fifth pixels, the pixel value G of the Nth pixel in the next frame is considered to be greater than or equal to the pixel value G of the Nth pixel in the current frame. It is done. For example, the pixel value G of the first pixel in the next frame is considered to be greater than or equal to the pixel value G of the first pixel in the current frame.

  Consider how much the pixel value G of the Nth pixel in the next frame is larger than the pixel value G of the Nth pixel in the current frame. Assume that a camera shake of +0.1 pixel occurs between the current frame and the next frame due to camera shake. The blur amount is an example of a value indicating blurring of a frame (image) caused by camera blurring.

  If the blur amount is within one pixel, it can be considered that the change amount of the pixel value G and the blur amount are directly proportional. Therefore, when the blur amount is +0.1 pixel, the pixel value G is considered to change as follows. The pixel value G of the first pixel of the current frame is 0. In the current frame, the value (∂G / ∂x) indicating the change in the x-axis direction of the pixel value G is 10 in the first pixel and the second pixel. Accordingly, in the next frame, the pixel value G of the first pixel is 1 (= 0 + 0.1 × 10).

  The pixel value G of the second pixel of the current frame is 10. In the current frame, the value (∂G / ∂x) indicating the change in the x-axis direction of the pixel value G is 0 in the second pixel and the third pixel. Accordingly, in the next frame, the pixel value G of the second pixel is 10 (= 10 + 0.1 × 0).

  The pixel value G of the third pixel in the current frame is 10. In the current frame, the value (∂G / ∂x) indicating the change in the x-axis direction of the pixel value G is 20 in the third pixel and the fourth pixel. Therefore, in the next frame, the pixel value G of the third pixel is 12 (= 10 + 0.1 × 20).

  The pixel value G of the fourth pixel in the current frame is 30. In the current frame, the value (∂G / ∂x) indicating the change in the x-axis direction of the pixel value G is 10 in the fourth pixel and the fifth pixel. Therefore, in the next frame, the pixel value G of the fourth pixel is 31 (= 30 + 0.1 × 10).

  A value indicating a temporal change of the pixel value G (pixel data) in two consecutive frames will be described. In the current frame and the next frame (in two consecutive frames), the value (∂G / ∂t) indicating the temporal change in the pixel value G of the first pixel is 1 (= 1-0) The value (∂G / ∂t) indicating the time change of the pixel value G of the second pixel is 0 (= 10−10), and the value indicating the time change of the pixel value G of the third pixel (∂G / ∂t) is 2 (= 12-10), and the value (∂G / ∂t) indicating the temporal change of the pixel value G of the fourth pixel is 1 (= 31-30).

  From the above, the relationship between the value (∂G / ∂x) indicating the change in the x-axis direction of the pixel value G (pixel data) due to camera shake and the value (∂G / ∂t) indicating the time change is as follows. It can be assumed that this is expressed by the following formula 1.

    ∂G / ∂t = Sx · ∂G / ∂x (Expression 1)

  Sx is a value indicating blur in the x-axis direction of a frame (image) caused by camera blur. In the case of FIG. 1, Sx is a blur amount in the x-axis direction between two consecutive frames (images).

  The value (∂G / ∂t) indicating the time change of the pixel value G due to camera shake is a signal change Ns caused by camera shake, and therefore the signal change Ns caused by camera shake is expressed by the following equation: It can be estimated that it is represented by 2.

    Ns = ∂G / ∂t = Sx · ∂G / ∂x (Expression 2)

  FIG. 2 shows a value (∂G / ∂x) indicating a change in the x-axis direction of the pixel value G and a value (∂G / ∂) indicating a temporal change of the pixel value G for each of all pixels constituting the next frame. It is three graphs which show the relationship with t). The horizontal axis of graphs 1, 2 and 3 indicates a value (∂G / ∂x) indicating the change in the x-axis direction of the pixel value G, and the vertical axis indicates a value indicating the time change of the pixel value G (∂ G / ∂t). The expression represented by the straight line SL is Expression 2. X indicates coordinates (∂G / ∂x, ∂G / ∂t) corresponding to each pixel constituting the next frame. For example, referring to FIG. 1, the coordinates corresponding to the first pixel of the next frame are (9, 1), and the coordinates corresponding to the second pixel are (2, 0). The coordinates corresponding to the pixel are (19, 2).

  Referring to FIG. 2, graph 1 shows a case where the subject has not changed and camera shake occurs in the x-axis direction. The coordinates do not include signal changes due to subject changes, but include signal changes Ns due to camera shake. For this reason, the coordinates are distributed on the straight line SL. The signal change caused by the subject change indicates, for example, an image of gas leaked from a gas leak monitoring target.

  Graph 2 shows a case where the subject changes and camera shake occurs in the x-axis direction. The coordinates include a signal change Ns caused by camera shake and a signal change caused by a subject change. For this reason, there are coordinates distributed above the straight line SL and coordinates distributed below the straight line SL.

  Since the change in the subject generally does not correlate with the camera shake in the x-axis direction, the signal change Ns caused by the camera shake and the signal change caused by the subject change can be separated. Thus, in the present embodiment, correction is performed to remove the signal change Ns caused by camera shake from the frame. Thereby, the signal change Ns caused by camera shake can be removed from the frame, and the signal change caused by the subject change can be left in the frame. Graph 3 shows a case where correction for removing signal change Ns caused by camera shake is performed on graph 2.

  As a method for obtaining Sx (a value indicating blur in the x-axis direction of a frame caused by camera shake), there are a method using an average value and a method using statistical processing. In the method using the average value, the average value of ∂G / ∂x and the average value of ∂G / ∂t are obtained at the coordinates (∂G / ∂x, ∂G / ∂t) shown in the graph 2, Sx is obtained from these average values and Equation 2. However, since the value of Sx becomes unstable in the vicinity of ∂G / ∂x = 0, the coordinates in the vicinity of ∂G / ∂x = 0 are excluded.

  In the method using statistical processing, Sx is obtained by performing statistical processing (for example, processing for obtaining a regression line) on the coordinates (∂G / ∂x, ∂G / ∂t) shown in the graph 2. Since the regression line obtained using the coordinates shown in the graph 2 is a straight line SL, Sx is obtained by obtaining the regression line using the coordinates shown in the graph 2.

  Statistical processing is a coordinate system composed of an axis indicating the direction of spatial change and an axis indicating the direction of temporal change. For each of a plurality of pixels constituting a frame (image), a value indicating spatial change and temporal change And a process for obtaining a regression equation that approximates the distribution of coordinates in the coordinate system. The figure shown by the regression equation includes a regression line, a regression curve, a regression plane, and a regression surface. FIG. 2 shows the case of a regression line.

  In the above, in order to simplify the description, the case where the camera shake occurs in the x-axis direction has been described. When camera shake occurs in the x-axis direction and y-axis direction, the signal change Ns caused by camera shake is assumed to be expressed by the following formula 3 in the same manner as in formulas 1 and 2. it can.

    Ns = ∂G / ∂t = Sx · ∂G / ∂x + Sy · ∂G / ∂y (Expression 3)

  ∂G / ∂y is a value indicating a change in the y-axis direction of the pixel value G (pixel data) (a value indicating a spatial change in the pixel value). Sy is a value indicating blur in the y-axis direction of a frame (image) caused by camera blur. As such a value, for example, there is a blur amount in the y-axis direction between two consecutive frames.

  The above is the description of the principle of the present embodiment.

  A gas detection system 1 to which the image processing apparatus 3 according to the present embodiment is applied will be described with reference to FIG. 3A. FIG. 3A is a block diagram showing the configuration of the system 1. The gas detection system 1 includes an infrared camera 2 and an image processing device 3.

  The infrared camera 2 captures a moving image of a monitoring target of gas leakage and a background infrared image (moving image to be monitored), and generates moving image data D1 indicating the moving image. Not only a moving image but also an infrared camera 2 may be used to capture a gas leak monitoring target and a background infrared image at a plurality of times. The infrared camera 2 includes an optical system 4, a filter 5, a two-dimensional image sensor 6, and a signal processing unit 7.

  The optical system 4 forms an infrared image of the subject (monitoring target and background) on the two-dimensional image sensor 6. The filter 5 is disposed between the optical system 4 and the two-dimensional image sensor 6, and allows only infrared light having a specific wavelength to pass through the light that has passed through the optical system 4. Of the infrared wavelength band, the wavelength band that passes through the filter 5 depends on the type of gas to be detected. For example, in the case of methane, a filter 5 that passes a wavelength band of 3.2 to 3.4 μm is used. The two-dimensional image sensor 6 is a cooled indium antimony (InSb) image sensor, for example, and receives the infrared rays that have passed through the filter 5. The signal processing unit 7 converts the analog signal output from the two-dimensional image sensor 6 into a digital signal and performs known image processing. This digital signal becomes the moving image data D1.

  The image processing apparatus 3 is a personal computer, a smartphone, a tablet terminal, or the like, and includes an image data input unit 8, an image processing unit 9, a display control unit 10, a display 11, and an input unit 12 as functional blocks.

  The image data input unit 8 is a communication interface that communicates with a communication unit (not shown) of the infrared camera 2. The image data input unit 8 receives the moving image data D1 sent from the communication unit of the infrared camera 2. The image data input unit 8 sends the moving image data D1 to the image processing unit 9.

  The moving image data D1 has a structure in which a plurality of frames are arranged in time series. Data obtained by arranging pixel data of pixels at the same position in time series in a plurality of frames (a plurality of infrared images) is referred to as time series pixel data. The time series pixel data will be specifically described. FIG. 4 is an explanatory diagram for explaining time-series pixel data. Let K be the number of frames of a moving image of an infrared image. One frame is composed of M pixels, that is, the first pixel, the second pixel,..., The M−1th pixel, and the Mth pixel. As described above, the pixel data is numerical information that the pixel has, for example, the pixel value G of the pixel. In the case of an infrared image, the pixel value G or the temperature calculated using the pixel value G is used.

  Pixels at the same position in a plurality (K) of frames mean pixels in the same order. For example, in the case of the first pixel, pixel data of the first pixel included in the first frame, pixel data of the first pixel included in the second frame,..., K−1th frame. The data obtained by arranging the pixel data of the first pixel included in the pixel data and the pixel data of the first pixel included in the Kth frame in time series becomes the time series pixel data of the first pixel. In the case of the Mth pixel, the pixel data of the Mth pixel included in the first frame, the pixel data of the Mth pixel included in the second frame,..., The (K−1) th frame. The data obtained by arranging the pixel data of the Mth pixel included in the pixel data and the pixel data of the Mth pixel included in the Kth frame in time series becomes the time series pixel data of the Mth pixel. The number of time-series pixel data is the same as the number of pixels constituting one frame.

  Returning to the description of FIG. The image processing unit 9 includes a first calculation unit 91, a second calculation unit 92, a third calculation unit 93, and a correction unit 94. These will be described later.

  The display control unit 10 displays a moving image indicated by the moving image data D1 and a moving image that has been subjected to predetermined processing by the image processing unit 9 (if another expression is used, a plurality of images corrected by the image processing unit 9 are time-sequentially displayed. Are displayed on the display 11.

  The input unit 12 receives various inputs related to gas detection.

  FIG. 3B is a block diagram showing a hardware configuration of the image processing apparatus 3 shown in FIG. 3A. The image processing apparatus 3 includes a CPU (Central Processing Unit) 3a, a RAM (Random Access Memory) 3b, a ROM (Read Only Memory) 3c, a HDD (Hard Disk Drive) 3d, a liquid crystal display 3e, a communication interface 3f, a keyboard 3g, and the like. And the bus 3h which connects these is provided. The liquid crystal display 3 e is hardware that implements the display 11. Instead of the liquid crystal display 3e, an organic EL display (Organic Light Emitting Diode display), a plasma display, or the like may be used. The communication interface 3 f is hardware that implements the image data input unit 8. The keyboard 3g is hardware for realizing the input unit 12. A touch panel may be used instead of the keyboard.

  The HDD 3d stores programs for realizing these functional blocks and various data (for example, moving image data D1) for the image processing unit 9 and the display control unit 10. The program that realizes the image processing unit 9 is an image processing program that acquires moving image data D1 (image data) and performs the predetermined processing on the moving image data D1. The program that realizes the display control unit 10 displays, for example, a display control that displays a moving image indicated by the moving image data D1 on the display 11, or displays a moving image that has been subjected to the predetermined processing by the image processing unit 9 on the display 11. It is a program. These programs are stored in advance in the HDD 3d, but are not limited thereto. For example, a recording medium (for example, an external recording medium such as a magnetic disk or an optical disk) in which these programs are recorded is prepared, and the program stored in the recording medium may be stored in the HDD 3d. . These programs may be stored in a server connected to the image processing apparatus 3 via a network, and these programs may be sent to the HDD 3d via the network and stored in the HDD 3d. These programs may be stored in the ROM 3c instead of the HDD 3d. The image processing apparatus 3 includes a flash memory instead of the HDD 3d, and these programs may be stored in the flash memory.

  The CPU 3a reads out these programs from the HDD 3d, expands them in the RAM 3b, and executes the expanded programs, whereby the image processing unit 9 and the display control unit 10 are realized. However, some or all of the functions of the image processing unit 9 and the display control unit 10 are realized by processing by a DSP (Digital Signal Processor) instead of or by processing by the CPU 3a. Also good. Similarly, a part or all of each function may be realized by processing by a dedicated hardware circuit instead of or together with processing by software.

  The image processing apparatus 3 includes the first to fourth embodiments as described below. Each of these embodiments is composed of a plurality of elements. Accordingly, the HDD 3d stores a program for realizing these elements. For example, the first embodiment of the image processing apparatus 3 includes a first calculation unit 91, a second calculation unit 92, a third calculation unit 93, and a correction unit 94 as elements. The HDD 3d stores a program for realizing each of these elements. These programs are expressed as a first calculation program, a second calculation program, a third calculation program, and a correction program.

  These programs are expressed using element definitions. The first calculation unit 91 and the first calculation program will be described as an example. The first calculation unit 91 calculates a value indicating the spatial change of the pixel data and a value indicating the temporal change of the pixel data for a plurality of time-series images (moving images) photographed by the infrared camera 2. The first calculation program is a program for calculating a value indicating a spatial change of pixel data and a value indicating a temporal change of pixel data for a plurality of images (moving images) arranged in time series photographed by the infrared camera 2. is there.

  A flowchart of these programs (first calculation program, second calculation program, third calculation program, and correction program) executed by the CPU 3a is FIG. 5 described later.

  As described above, the image processing apparatus 3 according to the present embodiment includes the first to fourth embodiments. In any of the embodiments, a case where camera shake occurs in the x-axis direction (first change direction) and the y-axis direction (second change direction) will be described as an example. In this case, the signal change Ns caused by camera shake is expressed by the above equation 3. Main terms used in the first to fourth embodiments will be described.

  As a value indicating a spatial change in the pixel value G (pixel data), a value (∂G / ∂x) indicating a change in the x-axis direction of the pixel value G or a change in the y-axis direction of the pixel value G (pixel data) is used. There is a value (∂G / ∂y) to indicate. ∂G / ∂x is obtained by directional differentiation of the change in the x-axis direction of the pixel value G. In other words, ∂G / ∂x is a difference between pixel values G of pixels adjacent in the x-axis direction. ∂G / ∂y is obtained by directional differentiation of a change in the y-axis direction of the pixel value G. In other words, ∂G / ∂y is a difference between pixel values G of pixels adjacent in the y-axis direction. Both ∂G / ∂x and ∂G / ∂y are values obtained by directional differentiation in the direction of spatial change with respect to the pixel degree G (pixel data).

  As a value indicating a temporal change of the pixel value G (pixel data), a value (∂G / 画素 t) obtained by subtracting the pixel values G (pixel data) of pixels in the same order in two consecutive frames (images), There is predetermined frequency component data d shown in FIGS. The former is used in the second and third embodiments. In other words, the former is a value obtained by differentiating the pixel value G in the direction of time change. The latter is used in the first embodiment, in other words, an AC component included in the pixel data of the first to Mth pixels.

  As a value indicating image blur caused by camera shake, a value (Sx) indicating a frame (image) shake caused by camera shake in the x-axis direction or a frame (image) caused by camera shake in the y-axis direction. There is a value (Sy) indicating blurring.

  Sx includes a blur amount in the x-axis direction between two consecutive frames (images) caused by camera shake and a frequency component of blur in the x-axis direction of each frame (image) caused by camera shake. The former is used in the second and third embodiments, and the latter is used in the first embodiment.

  As Sy, there are a blur amount in the y-axis direction between two consecutive frames (images) caused by camera shake, and a frequency component of a shake in the y-axis direction of each frame (image) caused by camera shake. The former is used in the second and third embodiments, and the latter is used in the first embodiment.

  The description will be made in order from the first embodiment. FIG. 5 is a flowchart for explaining the operation of the image processing apparatus 3 according to the first embodiment, which is executed by the CPU 3a of the image processing apparatus 3 according to the image processing program. With reference to FIG. 3A and FIG. 5, the first calculation unit 91 has a function of a bandpass processing unit. The first calculation unit 91 performs band-pass processing on the original moving image composed of the first to Kth frames shown in FIG. 4 (step S1).

  This will be described. FIG. 6 is an explanatory diagram for explaining the frequency component data constituting the time-series pixel data. The time-series pixel data is composed of the following frequency component data (1) to (5).

(1) Signal B indicating background temperature change (in other words, frequency component data indicating background temperature change)
(2) Signal change Ns caused by camera shake (in other words, frequency component data indicating noise caused by camera shake)
(3) Signal NL indicating temperature change due to leaked gas (in other words, frequency component data indicating temperature change due to leaked gas)
(4) Other noise Nc (in other words, frequency component data indicating other noise)
(5) High frequency noise Nh (in other words, frequency component data indicating high frequency noise)

  By (3), an image of the leaked gas (gas image) is formed. (2), (3) and (4) have a higher frequency than (1) and a lower frequency than (5). (1) is considerably larger than (3), and a value bias occurs with respect to the directional differentiation in the direction of the spatial change, and therefore needs to be removed. Although (2) and (5) are smaller than (1), they are non-negligible and need to be removed. (4) is much smaller than (3) and is ignored.

  The first calculation unit 91 processes the M time-series pixel data shown in FIG. 4 by using a bandpass filter, so that a signal indicating a background temperature change for each of the M time-series pixel data. B and high frequency noise Nh are removed. In other words, the first calculation unit 91 extracts predetermined frequency component data d from each of the M time-series pixel data by processing the M time-series pixel data using the bandpass filter. To do. The predetermined frequency component data d includes a signal change Ns caused by camera shake, a signal NL indicating a temperature change due to leaked gas, and other noise Nc. The predetermined frequency component data d is a moving image obtained by removing the signal B indicating the background temperature change and the high frequency noise Nh from the original moving image. Therefore, the predetermined frequency component data d includes the moving image of the gas image, but does not include the moving image of the background (monitoring target). Thus, the first calculation unit 91 functions as an extraction unit (not shown), and the extraction unit extracts a predetermined subject (here, a gas image) from the frame as subject extraction. Subject extraction is performed for each of the th to Kth frames. If another expression is used, the extraction unit extracts a predetermined subject from each of a plurality of images (a plurality of frames).

  The frequency of the signal NL indicating the temperature change due to the leaked gas is set to 0.3 to 3 Hz. For example, using a weighting coefficient that can extract 0.3 to 3 Hz, a weighted moving average is calculated with a predetermined number of frames smaller than the number of K frames shown in FIG. Thereby, predetermined frequency component data d is extracted from the time-series pixel data.

  FIG. 7 is an explanatory diagram for explaining the bandpass filter used in step S1. The horizontal axis indicates the frame, and the vertical axis indicates the weighting coefficient. The predetermined number of frames is, for example, 99 frames. The breakdown is a target frame, 49 consecutive frames before this, and 49 consecutive frames after this.

  Through the process in step S1, M pieces of predetermined frequency component data d are generated. The breakdown includes predetermined frequency component data d extracted from the time-series pixel data of the first pixel, predetermined frequency component data d extracted from the time-series pixel data of the second pixel,. This is predetermined frequency component data d extracted from pixel time-series pixel data.

  FIG. 8 is a graph showing predetermined frequency component data d extracted from the time-series pixel data of the first pixel. The vertical axis of the graph indicates the pixel value G, and the horizontal axis indicates the frame order. As described above, the predetermined frequency component data d is composed of the signal change Ns caused by camera shake, the signal NL indicating the temperature change caused by the leaked gas, and other noise Nc.

  FIG. 9 is an explanatory diagram for explaining the predetermined frequency component data d in units of frames. In each frame, the value of the predetermined frequency component data d of the first pixel is d (1), the value of the predetermined frequency component data d of the second pixel is d (2),..., The Mth pixel The value of the predetermined frequency component data d is d (M). Data in which d (1) of the first pixel of each frame is arranged in time series becomes predetermined frequency component data d extracted from the time series pixel data of the first pixel, and the second pixel of each frame Data in which d (2) is arranged in time series becomes predetermined frequency component data d extracted from the time series pixel data of the second pixel,..., d (M) of the Mth pixel in each frame Is a predetermined frequency component data d extracted from the time-series pixel data of the Mth pixel.

  d (1) to d (M) are AC components (in other words, frequency components) included in the pixel data of the first to Mth pixels, respectively. d (1) to d (M) are examples of values indicating temporal changes in the pixel value G (pixel data) in two consecutive frames. As described above, the first calculation unit 91 calculates a value indicating a temporal change in the pixel value G (pixel data).

  Referring to FIGS. 3A and 5, the first calculation unit 91 performs a process of calculating ∂G / ∂x and ∂G / ∂y for each of the first to Kth frames (Step S1). S2). FIG. 10 is an explanatory diagram for explaining this process. ∂G / ∂x is a value indicating a change in the x-axis direction of the pixel value G in one frame. ∂G / ∂y is a value indicating a change in the y-axis direction of the pixel value G in one frame. Both are values indicating the spatial change of the pixel value G (pixel data) in one frame.

  The order of the process of step S1 and the process of step S2 may be either first or may be performed simultaneously. As described above, the first calculation unit 91 calculates, for a plurality of images (a plurality of frames), a value indicating the spatial change of the pixel data (step S2) and a value indicating the temporal change of the pixel data. (Step S1).

  Referring to FIGS. 3A and 5, the second calculation unit 92 calculates Sx and Sy for each of the first to Kth frames using the result of step S1 and the result of step S2 (step S1). S3). This will be described.

  FIG. 11 is an explanatory diagram for explaining processing for obtaining Sx and Sy of the first frame. The two graphs shown in FIG. 11 have an axis indicating ∂G / ∂x, an axis indicating ∂G / ∂y, and an axis indicating d. These axes are orthogonal to each other. ∂G / ∂x (a value indicating the change of the pixel value G in the x-axis direction) is ∂G / ∂x of the first frame shown in FIG. ∂G / ∂y (a value indicating the change of the pixel value G in the y-axis direction) is ∂G / ∂y of the first frame shown in FIG. d (value indicating the temporal change of the pixel value G) is the frequency component data d (1) to d (M) of the first frame among the predetermined frequency component data d shown in FIG. The axis indicating ∂G / ∂x is the axis indicating the first change direction, the axis indicating ∂G / ∂y is the axis indicating the second change direction, and the axis indicating d is the time direction It is an axis which shows change of.

  The second calculator 92 calculates ∂G / ∂x and ∂G / ∂y of the first to Mth pixels constituting the first frame shown in FIG. 10, and the first frame shown in FIG. Using d (1) to d (M) of the first to Mth pixels constituting the coordinates, the coordinates (∂G / ∂x, ∂ for each of the first to Mth pixels constituting the first frame are used. G / ∂y, d (1)) to coordinates (∂G / ∂x, ∂G / ∂y, d (M)) are generated, and a regression surface for these coordinates is calculated. The calculation of the regression surface is an example of statistical processing.

  The second calculation unit 92 obtains an intersection line between a plane defined by the axis indicating ∂G / ∂x and the axis indicating d and the regression plane of the first frame, and indicates ∂G / ∂x The slope of the intersection line with respect to the axis is set as Sx of the first frame (Sx (1)). The second calculation unit 92 obtains the intersection line between the plane defined by the axis indicating ∂G / ∂y and the axis indicating d and the regression plane of the first frame, and indicates ∂G / ∂y. The inclination of the intersection line with respect to the axis is set as Sy of the first frame (Sy (1)). That is, the second calculation unit 92 sets the intersection line between the plane defined by the axis indicating the first change direction and the axis indicating the time change direction and the regression plane as the first intersection line, The first line of intersection with respect to the axis indicating the first change direction is defined as the second line of intersection between the plane defined by the axis indicating the change direction and the axis indicating the time change direction and the regression plane. Is the value indicating the blur of the frame (image) along the first change direction, and the inclination of the second intersection line with respect to the axis indicating the second change direction is the frame along the second change direction. The value indicates the blur of (image). Expressing this in a superordinate concept, when the graphic represented by the regression equation is a regression plane, the second calculation unit 92 is a plane defined by an axis indicating the direction of spatial change and an axis indicating the direction of time change. The inclination of the line of intersection with the regression plane with respect to the axis indicating the direction of spatial change is calculated.

  FIG. 12 shows the second frame Sx (Sx (2)) and the second frame Sy (Sy (2)) to the Kth frame Sx (Sx (K)) and the Kth frame Sy ( It is explanatory drawing explaining the process which calculates | requires (Sy (K)). The second calculation unit 92 obtains Sx and Sy of the second to Kth frames in the same manner as the first frame. As described above, the second calculation unit 92 statistically processes the value indicating the spatial change and the value indicating the time change calculated by the first calculation unit 91, and performs a plurality of processings caused by camera shake. A value indicating blur of each image (a plurality of frames) is calculated.

  Referring to FIGS. 3A and 5, the third calculation unit 93 calculates a signal change Ns caused by camera shake using the result of step S2 and the result of step S3 (step S4). If another expression is used, the third calculation unit 93 uses a value indicating spatial change and a value indicating image blur for each pixel of a plurality of images (a plurality of frames). The resulting signal change Ns is calculated. This will be described. The third calculator 93 calculates a signal change Ns caused by camera shake for each of the first to Mth pixels. A table explaining this is shown below. Ns is calculated using Equation 3 above.

  A description will be given from Ns regarding the first pixel. Ns related to the first pixel is Ns related to the first pixel included in the first frame, Ns related to the first pixel included in the second frame,..., The first included in the Kth frame. This is data in which Ns related to pixels are arranged in time series.

  Ns regarding the first pixel included in the first frame is ∂G / ∂x and ∂G / ∂y of the first pixel included in the first frame shown in FIG. 10 and 1 shown in FIG. It is obtained by substituting Sx (Sx (1)) and Sy (Sy (1)) of the second frame into Equation 3. Ns regarding the first pixel included in the second frame is ∂G / ∂x and ∂G / ∂y of the first pixel included in the second frame illustrated in FIG. 10, and 2 illustrated in FIG. 12. It is obtained by substituting Sx (Sx (2)) and Sy (Sy (2)) of the second frame into Equation 3. Ns regarding the first pixel included in the Kth frame is ∂G / ∂x and ∂G / ∂y of the first pixel included in the Kth frame illustrated in FIG. 10 and K illustrated in FIG. It is obtained by substituting Sx (Sx (K)) and Sy (Sy (K)) of the second frame into Equation 3.

  Ns related to the second pixel is Ns related to the second pixel included in the first frame, Ns related to the second pixel included in the second frame,..., The second included in the Kth frame. This is data in which Ns related to pixels are arranged in time series. Ns related to the Mth pixel is Ns related to the Mth pixel included in the first frame, Ns related to the Mth pixel included in the second frame,..., Mth included in the Kth frame. This is data in which Ns related to pixels are arranged in time series.

  With reference to FIGS. 3A and 5, the correction unit 94 calculates a difference between the predetermined frequency component data d and the signal change Ns caused by camera shake (step S <b> 5). This will be described. Referring to FIG. 8 and Table 1, the correction unit 94 calculates the difference between the predetermined frequency component data d extracted from the time-series pixel data of the first pixel and Ns related to the first pixel, the second pixel The difference between the predetermined frequency component data d extracted from the time-series pixel data and the Ns related to the second pixel,..., The predetermined frequency component data d extracted from the time-series pixel data of the M-th pixel And the difference between Ns for the Mth pixel.

  Referring to FIG. 6, the frequency component data obtained from the difference in step S5 is composed of a signal NL indicating a temperature change due to the leaked gas and other noise Nc. Since the other noise Nc is considerably smaller than the signal NL indicating the temperature change due to the leaked gas, the frequency component data obtained by the difference in step S5 can be regarded as the signal NL indicating the temperature change due to the leaked gas. it can. Accordingly, step S5 is a process for correcting camera shake for a moving gas image.

  As described above, the correction unit 94 uses the signal change Ns caused by camera shake as a second correction to correct a subject (here, a gas image), and forms the moving image. A second correction is performed on the subject extracted from each of the th frames. If another expression is used, the correction unit 94 corrects the subject extracted by the extraction unit using the signal change Ns calculated by the third calculation unit 93.

  With reference to FIGS. 3A and 5, the display control unit 10 displays a moving image (that is, a moving image of a gas image in which camera shake is corrected) indicated by the frequency component data obtained by the difference in step S <b> 5 on the display 11. (Step S6).

  The main effects of the first embodiment will be described. Referring to FIG. 1, (a) if camera shake occurs, a change in the spatial direction of the pixel value G and a change in the temporal direction of the pixel value G occur. (B) If the camera shake amount is within one pixel, the change amount of the pixel value G and the camera shake amount are considered to be directly proportional. Based on these assumptions, the present inventor provides a value indicating a spatial change of each of a plurality of pixels constituting the frame and a temporal change for each frame (image) constituting the moving image photographed by the infrared camera 2. It was found that Sx and Sy of each frame can be obtained by performing predetermined statistical processing (calculation of regression plane) on the indicated values (FIGS. 11 and 12).

  Referring to FIG. 5, the first embodiment calculates Sx and Sy of each frame based on the above (a) and (b) (step S3), so even if the camera shake is extremely small ( For example, 0.01 to 0.05 pixels), Sx and Sy of each frame can be calculated. In the first embodiment, the signal change Ns caused by camera shake is calculated using Sx and Sy (step S4), and the gas image is corrected using the signal change Ns (step S5). Therefore, according to the first embodiment, even if the camera shake is extremely small, noise caused by the camera shake can be removed, so that the gas detection accuracy can be improved.

  The above effect can be said also in the second to fourth embodiments.

  A second embodiment will be described. In the first embodiment, a moving image of a gas image in which camera shake is corrected is generated. On the other hand, 2nd Embodiment produces | generates the moving image (namely, moving image containing the image of a monitoring object, and a gas image) by which the camera shake was correct | amended with respect to the original moving image.

  FIG. 13 is a flowchart for explaining the operation of the image processing apparatus 3 according to the second embodiment, which is executed by the CPU 3a of the image processing apparatus 3 according to the image processing program. With reference to FIG. 3A and FIG. 13, the first calculation unit 91 performs ∂G / ∂x and ∂G / ∂ on the original moving image composed of the first to Kth frames shown in FIG. A process of calculating y and a process of calculating ∂G / ∂t are performed (step S11). Since the process of calculating ∂G / ∂x and ∂G / ∂y is the same as step S2 (FIG. 5) of the first embodiment, the description thereof is omitted.

  Processing for calculating ∂G / ∂t will be described. FIG. 14 is an explanatory diagram for explaining this. ∂G / ∂t is a value obtained by differentiating the pixel value G (pixel data) in the direction of time change. In the first embodiment, the predetermined frequency component data d described with reference to FIGS. 8 and 9 is used as a value indicating the time change. In the second embodiment, ∂G / ∂t is used as the value indicating the time change. Is used. ∂G / ∂t is a difference between pixel values G of pixels in the same order in two consecutive frames.

  Two consecutive frames are the first and second frames in the case of the second frame, the second and third frames in the case of the third frame, and so on. The K-1th and Kth frames. In the case of the first frame, since there are no two consecutive frames, a predetermined value is assigned to ∂G / ∂t of the first to Mth pixels constituting the first frame.

  For example, in the second frame, ∂G / ∂t of the first pixel is the pixel value G of the first pixel included in the first frame and the pixel of the first pixel included in the second frame. The difference from the value G, and 画素 G / の t of the second pixel is the pixel value G of the second pixel included in the first frame and the pixel of the second pixel included in the second frame Is the difference from the value G, and ∂G / ∂t of the Mth pixel is the pixel value G of the Mth pixel included in the first frame and the Mth pixel included in the second frame This is the difference from the pixel value G of this pixel.

  With reference to FIG. 3A and FIG. 13, the 2nd calculation part 92 calculates Sx and Sy about each of the 1st-Kth flame | frame using the result of step S11 (step S12). This will be described. FIG. 15 is an explanatory diagram for explaining processing for obtaining Sx and Sy of the first frame. The two graphs shown in FIG. 15 have an axis indicating ∂G / ∂x, an axis indicating ∂G / ∂y, and an axis indicating ∂G / ∂t. These axes are orthogonal to each other. ∂G / ∂x (a value indicating the change of the pixel value G in the x-axis direction) is ∂G / ∂x of the first frame shown in FIG. ∂G / ∂y (a value indicating the change of the pixel value G in the y-axis direction) is ∂G / ∂y of the first frame shown in FIG. ∂G / ∂t (a value indicating a temporal change in the pixel value G) is ∂G / ∂t of the first frame.

  The second calculation unit 92 uses the first to Mth pixels ∂G / ∂x, ∂G / ∂y, and ∂G / ∂t of the first frame to form the first frame. For each of the first to M-th pixels constituting, coordinates (∂G / ∂x, ∂G / ∂y, ∂G / ∂t) to coordinates (∂G / ∂x, ∂G / ∂y, ∂ G / ∂t) is generated, and a regression surface with respect to these coordinates is calculated. The calculation of the regression surface is an example of statistical processing.

  The second calculation unit 92 obtains an intersection line between the plane defined by the axis indicating ∂G / ∂x and the axis indicating ∂G / ∂t and the regression plane of the first frame, and ∂G / The slope of the intersection line with respect to the axis indicating ∂x is defined as Sx of the first frame (Sx1). The second calculation unit 92 obtains an intersection line between the plane defined by the axis indicating ∂G / ∂y and the axis indicating ∂G / ∂t and the regression plane of the first frame, and ∂G / The slope of the intersection line with respect to the axis indicating ∂y is defined as Sy of the first frame (Sy1).

  FIG. 16 illustrates the second frame Sx (Sx (2)) and the second frame Sy (Sy (2)) to the Kth frame Sx (Sx (K)) and the Kth frame Sy ( It is explanatory drawing explaining the process which calculates | requires (Sy (K)). The second calculation unit 92 obtains Sx and Sy of the second to Kth frames in the same manner as the first frame.

  With reference to FIG. 3A and FIG. 13, the 3rd calculation part 93 calculates the signal change Ns resulting from the camera shake using the result of step S11 and the result of step S12 (step S13). Since this is the same as step S4 (FIG. 5) of the first embodiment described using Table 1, the description thereof is omitted.

  With reference to FIG. 3A and FIG. 13, the 3rd calculation part 93 processes the signal change Ns resulting from a camera shake using the result of step S13 (step S14). A table explaining this is shown below.

  Referring to Tables 1 and 2, in the case of the first frame, the Ns integrated value related to the first pixel is Ns related to the first pixel included in the first frame, and the Ns integrated value related to the second pixel. The value is Ns related to the second pixel included in the first frame,..., The Ns integrated value related to the Mth pixel is Ns related to the Mth pixel included in the first frame.

  In the case of the second frame, the Ns integrated value related to the first pixel is Ns related to the first pixel included in the first frame + Ns related to the first pixel included in the second frame, and related to the second pixel. The Ns integrated value is Ns related to the second pixel included in the first frame + Ns related to the second pixel included in the second frame,..., And the Ns integrated value related to the Mth pixel is Ns related to the Mth pixel included in the frame + Ns related to the Mth pixel included in the second frame.

  In the case of the Kth frame, the Ns integrated value related to the first pixel is Ns + related to the first pixel included in the first frame + Ns +... + Kth related to the first pixel included in the second frame. Ns related to the first pixel included, and the Ns integrated value related to the second pixel is Ns related to the second pixel included in the first frame + Ns +... + K related to the second pixel included in the second frame. Ns for the second pixel included in the first frame, ..., the Ns integrated value for the Mth pixel is Ns for the Mth pixel included in the first frame + Mth included in the second frame Ns + related to the pixel No. + Ns related to the Mth pixel included in the Kth frame.

  With reference to FIG. 3A and FIG. 13, the correction | amendment part 94 calculates the difference of an original moving image and the result of step S14 (step S15). This will be described in detail. The correction unit 94 is associated with the pixel values G of the first to Mth pixels constituting the first frame of the original moving image and the first to Mth pixels constituting the first frame shown in Table 2. The difference from the Ns integrated value is calculated. The correction unit 94 respectively relates to the pixel values G of the first to Mth pixels constituting the second frame of the original moving image and the first to Mth pixels constituting the second frame shown in Table 2. The difference from the Ns integrated value is calculated. The correction unit 94 performs the same process for the third to Kth frames of the original moving image. Through the processing in step S15, camera shake can be corrected for the original moving image.

  As described above, the correction unit 94 sets the first correction to correct the frame using the signal change Ns caused by camera shake, and each of the first to Kth frames constituting the moving image. First correction is made. In other words, the correction unit 94 corrects a plurality of images (a plurality of frames) using the signal change Ns calculated by the third calculation unit 93.

  With reference to FIG. 3A and FIG. 13, the display control part 10 displays on the display 11 the moving image (namely, moving image with which the camera shake was correct | amended) obtained by the difference of step S15 (step S16).

  When generating a moving image of only a gas image, the image processing unit performs bandpass processing on the moving image obtained by the difference in step S15 (step S17). Since this process is the same as step S1 in FIG. The display control unit 10 causes the display 11 to display the moving image processed in step S17 (that is, the moving image of the gas image corrected for camera shake) (step S18).

  A third embodiment will be described. The third embodiment can cope with a case where the blur amount is large. Referring to FIG. 1, for example, the second pixel and the third pixel of the current frame have the same pixel value G. For this reason, when the blur amount exceeds one pixel, the change amount of the pixel value G and the blur amount are not directly proportional to each other, and the error of Sx becomes large (the error similarly increases for Sy). In the third embodiment, Ns is calculated by setting the blur amount between two consecutive frames to be one pixel or less.

  The third embodiment is different from the second embodiment in that a determination process of Sx and Sy is performed instead of step S12 shown in FIG. FIG. 17 is a flowchart for explaining Sx determination processing in the third embodiment. FIG. 18 is an explanatory diagram for explaining the determination process of Sx. With reference to FIG. 3A, FIG. 17 and FIG. 18, the second calculator 92 calculates the first frame Sx (Sx (1)) to the Kth frame Sx (Sx) calculated in step S12 of FIG. (K)) is set as an initial value of Sx (Sx (1)) of the first frame to Sx (Sx (K)) of the Kth frame (step S21). The initial value of Sx (Sx (1)) of the first frame is 0 pixels.

  The second calculator 92 calculates the Sx integrated value (step S22). This will be specifically described. The initial value of Sx (Sx (2)) of the second frame is, for example, 0.3 pixels. The Sx integrated value of the first frame is 0 pixel, and the Sx integrated value of the second frame is 0.3 pixel (= 0 + 0.3). The second calculation unit 92 converts the Sx integrated value of the first frame and the Sx integrated value of the second frame into integers (step S23). This is the same as the process of rounding off the Sx integrated value. Since the Sx integrated value of the first frame is 0 pixel, if it is converted to an integer, it becomes 0 pixel. Since the Sx integrated value of the second frame is 0.3 pixel, it becomes 0 pixel when converted to an integer.

  The second calculation unit 92 determines whether or not the integral values converted into integers are 0 pixels in two consecutive frames (step S24). Here, the two consecutive frames are the first and second frames. In these frames, since the integral values converted into integers are all 0 pixels (Yes in step S24), the second calculation unit 92 calculates the initial value (0 of Sx (Sx (1)) of the first frame. Pixel) is determined as Sx (Sx (1)) of the first frame, and the initial value (0.3 pixel) of Sx (Sx (2)) of the second frame is determined as Sx (Sx (2)) of the second frame. Sx (2)) is confirmed (step S25).

  The second calculation unit 92 determines whether or not the Sx of the last frame has been determined (step S26). Since Sx of the last frame is not fixed (No in Step S26), the second calculation unit 92 calculates the Sx integrated value of the next frame (Step S22). Here, the Sx integrated value of the third frame is calculated. The initial value of Sx (Sx (3)) of the third frame is, for example, 0.1 pixel. The Sx integrated value of the third frame is 0.4 pixels (= 0.3 + 0.1).

  The second calculation unit 92 converts the Sx integrated value of the third frame into an integer (step S23). Since the Sx integrated value of the third frame is 0.4 pixels, it becomes 0 pixels when converted to an integer.

  The second calculation unit 92 determines whether or not the integral values converted into integers are 0 pixels in two consecutive frames (step S24). Here, the two consecutive frames are the second and third frames. In these frames, since the integral values converted into integers are all 0 pixels (Yes in step S24), the second calculation unit 92 sets the initial value (0 of Sx (Sx (3)) of the third frame. .1 pixel) is determined as Sx (Sx (3)) of the third frame (step S25).

  The second calculation unit 92 determines whether or not the Sx of the last frame has been determined (step S26). Since Sx of the last frame is not fixed (No in Step S26), the second calculation unit 92 calculates the Sx integrated value of the next frame (Step S22). Here, the Sx integrated value of the fourth frame is calculated. The initial value of Sx (Sx (4)) of the fourth frame is, for example, 0.7 pixels. The Sx integrated value of the fourth frame is 1.1 pixels (= 0.4 + 0.7).

  The second calculation unit 92 converts the Sx integrated value of the fourth frame into an integer (step S23). Since the Sx integrated value of the fourth frame is 1.1 pixels, if it is converted to an integer, it becomes 1 pixel. The second calculation unit 92 determines whether or not the integral values converted into integers are 0 pixels in two consecutive frames (step S24). Here, the two consecutive frames are the third and fourth frames. For the fourth frame, the integerized Sx integrated value is one pixel (No in step S24). Therefore, the second calculation unit 92 calculates Sx (Sx (4)) of the fourth frame using the third frame and the fourth frame shifted by −1 pixel in the x-axis direction ( Step S27). In this example, Sx (4) is −0.3 pixel. The second calculation unit 92 determines Sx (Sx (4)) of the fourth frame as −0.3 pixel (step S28).

  The second calculation unit 92 determines whether or not the Sx of the last frame has been determined (step S26). Since Sx of the last frame is not fixed (No in Step S26), the second calculation unit 92 calculates the Sx integrated value of the next frame (Step S22). Here, the Sx integrated value of the fifth frame is calculated. The initial value of Sx (Sx (5)) of the fifth frame is assumed to be −0.8 pixels, for example. The Sx integrated value of the fifth frame is 0.3 pixels (= 1.1−0.8).

  The second calculation unit 92 converts the Sx integrated value of the fifth frame into an integer (step S23). Since the Sx integrated value of the fifth frame is 0.3 pixel, it becomes 0 pixel if it is converted to an integer. The second calculation unit 92 determines whether or not the integral values converted into integers are 0 pixels in two consecutive frames (step S24). Here, the two consecutive frames are the fourth and fifth frames. For the fourth frame, the integerized Sx integrated value is one pixel (No in step S24). Therefore, the second calculation unit 92 calculates Sx (Sx (5)) of the fifth frame using the fifth frame and the fourth frame shifted by −1 pixel in the x-axis direction ( Step S27). In this example, Sx (5) is 0.2 pixels. The second calculation unit 92 determines Sx (Sx (5)) of the fifth frame as 0.2 pixels (step S28).

  The second calculation unit 92 performs the same processing as described above for the sixth and subsequent frames. When the second calculation unit 92 determines that Sx of the Kth frame has been confirmed (Yes in step S26), the Sx determination process ends.

  The second calculation unit 92 performs a determination process for Sy as in the case of Sx. The image processing unit 9 performs the processing after step S13 shown in FIG. 13 using Sx and Sy on which the confirmation processing has been performed.

  A fourth embodiment will be described. During moving image shooting by the infrared camera 2 shown in FIG. 3A, in addition to camera shake, movement of a subject (for example, a car, a person, a tree branch, etc.) may occur. In the first to third embodiments, a value indicating a spatial change and a value indicating a temporal change of each of a plurality of pixels constituting the frame are statistically processed to calculate Sx and Sy of the frame. In this case, if the Sx and Sy of the frame are calculated including the pixels that constitute the region indicating the moving subject image, the accuracy of these values decreases. Therefore, in the fourth embodiment, the accuracy of the Sx and Sy of the frame is not lowered even when the subject moves.

  A moving subject appears as an image of a moving object in a moving image. In a frame divided into two or more areas, the value indicating the blur of the area indicating the image of the moving object is different from the value indicating the blur of the other areas. Therefore, in the fourth embodiment, in a frame divided into two or more regions, a value indicating the blur of each region is used to identify a region indicating the image of the moving object, and the image of the moving object is determined. A value indicating blurring of an image caused by camera blur is calculated using a value indicating blurring of two or more areas excluding the indicated area.

  FIG. 19 is a schematic diagram of one frame divided into two or more regions R. An arrow included in each of the two or more regions R indicates the magnitude and direction of the blur amount of the region R. In two or more regions R, there may be a region R in which the amount and direction of blurring are different from those of other regions R. For example, in FIG. 19, the magnitude and direction of the blur amount of the region Ra, the region Rb, the region Rc, the region Rd, the region Re, and the region Rf are different from the magnitude and direction of the blur amount of the other regions R. In the fourth embodiment, the regions Ra to Rf are regions that indicate the image of the moving body, and the region R such as the regions Ra to Rf is excluded from the calculation of the blur amount of one frame.

  FIG. 20 is a flowchart illustrating a process of calculating Sx and Sy in the fourth embodiment. The second calculation unit 92 sets i = 1 (step S41). This means processing for the first frame. In the first to third embodiments, Sx and Sy are calculated for one entire frame (for example, step S12 in FIG. 13), but the second calculation unit 92 of the fourth embodiment is the first The frame is divided into two or more regions R, and Sx and Sy are calculated for each of the two or more regions R (step S42). The method of calculating Sx and Sy of each region R is the same as the method of calculating Sx and Sy of one frame (for example, step S12 in FIG. 13). That is, the second calculation unit 92 calculates Sx and Sy of each of the two or more regions R using a method similar to the calculation method of Sx and Sy of the frame.

  The second calculation unit 92 calculates the average value of the vectors (Sx, Sy) calculated in step S42 (step S43).

  The second calculation unit 92 is farthest from the average value of the vectors (Sx, Sy) calculated in step S43 among the vectors (Sx, Sy) of the plurality of regions R calculated in step S42. Search for a vector (Sx, Sy) that is present (step S44).

  The second calculation unit 92 determines whether or not the absolute value of the difference between the vector (Sx, Sy) searched in step S44 and the average value calculated in step S43 is equal to or greater than a predetermined value ( Step S45).

  When the second calculation unit 92 determines that the difference between the vector (Sx, Sy) searched in step S44 and the average value calculated in step S43 is equal to or greater than a predetermined value (Yes in step S45). The region R having the vector (Sx, Sy) searched in step S44 is specified as the region indicating the image of the moving object, and the vector (Sx, Sy) searched in step S44 is used for calculating the average value. Determination is made (step S46). That is, the second calculation unit 92 specifies a region R indicating an image of the moving body included in the frame, using the vectors (Sx, Sy) of the two or more regions R. Then, the second calculation unit 92 returns to the process of step S43. Here, the second calculation unit 92 excludes the vector (Sx, Sy) determined in step S46 from the vector (Sx, Sy) calculated in step S42, and the vector (Sx, Sy). Is calculated (step S43). And the 2nd calculation part 92 performs the process after step S44.

  When the second calculation unit 92 determines that the difference between the vector (Sx, Sy) searched in step S44 and the average value calculated in step S43 is smaller than a predetermined value (No in step S45). ), The second calculation unit 92 determines whether i = k (step S47). Here, since i = 1 (No in step S47), the second calculation unit 92 determines the latest average value calculated in step S43 as the vector (Sx, Sy) of the i-th frame (step S43). S48). Here, the vector (Sx, Sy) of the first frame, that is, the vector (Sx (1), Sy (1)) is determined. That is, the second calculation unit 92 uses a vector (Sx, Sy) of two or more regions R excluding the region R indicating the image of the moving object (an average value of the vectors (Sx, Sy)), and a frame vector. (Sx, Sy) is calculated.

  The second calculation unit 92 sets i = i + 1 (step S49). Here, i = 2 is set. Then, the second calculation unit 92 returns to the process of step S42. Here, the second calculation unit 92 divides the second frame into a plurality of regions R, and calculates a vector (Sx, Sy) for each of the plurality of regions R (step S42).

  When the second calculation unit 92 determines that i = k (Yes in step S47), the second calculation unit 92 calculates the most recent average value calculated in step S43 as the i-th frame vector (Sx , Sy). Here, the vector (Sx, Sy) of the Kth frame, that is, the vector (Sx (K), Sy (K)) is determined.

  The above is the calculation processing of the frame vector (Sx, Sy) according to the fourth embodiment. This can be replaced with the calculation process of any vector (Sx, Sy) in the first to third embodiments.

  In this embodiment, as described with reference to FIGS. 11 and 12, and FIGS. 15 and 16, the regression surface is obtained, and Sx and Sy are obtained using the regression surface. This means that the following processing is performed.

  Referring to FIG. 6, let g be a signal obtained by subtracting signal B indicating background temperature change and high-frequency noise Nh from time-series pixel data. g is represented by the following formula.

    g = Ns + NL + Nc

  As described above, Ns is a signal change caused by camera shake, NL is a signal indicating a temperature change due to leaked gas, and Nc is other noise.

  The equation of g can be transformed into the following equation.

    NL + Nc = g−Ns

  As described above, when the camera shake occurs in the x-axis direction and the y-axis direction, it can be estimated that the signal change Ns caused by the camera shake is expressed by Expression 3. The signal change Ns caused by camera shake is independent of the signal g, the temperature change NL due to the leaked gas, and other noises Nc, and has no correlation. It is assumed that Sx and Sy are constant in one frame (one image). A combination of Sx and Sy that minimizes the signal value (NL + Nc = g−Ns) among various values of Sx and Sy is considered to be a correct value of Sx and Sy. This means that a combination of Sx and Sy that minimizes the square norm of the signal value represented by the following Expression 4 may be obtained.

  When Equation 4 is calculated, Equation 5 is obtained.

  At the point where the square norm of the signal value is minimum, it is considered that the value obtained by partial differentiation of Equation 5 with Sx and Sy is 0. The equation for partial differentiation of Equation 5 with Sx is Equation 6, and the equation for partial differentiation of Equation 5 with Sy is Equation 7.

  When Equation 6 is calculated (ie, Equation 6 is partially differentiated by Sx), Equation 8 is obtained, and when Equation 7 is calculated (ie, Equation 7 is partially differentiated by Sy), Equation 9 is obtained.

  From Expression 8 and Expression 9, Sx becomes Expression 10, and Sy becomes Expression 11.

  In the first to fourth embodiments, the signal change Ns caused by camera shake is calculated using Equation 3. This is Ns when camera shake occurs in the x-axis direction and the y-axis direction. Ns when the camera shake is rotational shake or the like will be described.

  When the camera shake is a rotational shake, the signal change Ns caused by the camera shake can be assumed to be expressed by the following Expression 12 when considered in the same manner as Expressions 1 and 2.

    Ns = ∂G / ∂t = Sω · ∂G / ∂ω = Sω (y · ∂G / ∂x + x · ∂G / ∂y) (Equation 12)

  Sω is a value indicating the rotational shake of the frame that occurs when the camera shake is a rotational shake. As such a value, for example, there is a blurring amount of the frame in the rotation axis direction. ∂G / ∂ω represents a value indicating a change in the rotation direction of the pixel value G. If ４G / ∂x and ∂G / ∂y are obtained from Equation 4, Ns can be obtained without obtaining ∂G / ∂ω.

  According to Equation 12, ∂G / ∂ω = y · ∂G / ∂x + x · ∂G / ∂y. Explain that this is true. FIG. 21 is an explanatory diagram for explaining these. When the camera rotates around the origin by an angle Δω, the change amount Δx of the x coordinate value and the change amount Δy of the y coordinate value of the pixel located at the coordinates (x1, y1) are as follows. .

Δx ≒ y1 ・ Δω
Δy ≒ x1 ・ Δω

  The change amount ΔG of the pixel value G of the pixel located at the coordinates (x1, y1) due to the rotation is as follows.

ΔG≈∂G / ∂x · Δx + ∂G / ∂y · Δy
≒ ∂G / ∂x ・ y1 ・ Δω + ∂G / ∂y ・ x1 ・ Δω

  The equation obtained by dividing ΔG by Δω is as follows.

    ΔG / Δω≈y1 · ∂G / ∂x · + x1 · ∂G / ∂y

  This equation holds when Δω is sufficiently small. Therefore, ∂G / ∂ω = y · ∂G / ∂x + x · ∂G / ∂y is established.

  In the case of enlargement / reduction blur (fluctuation in the size of the image that occurs when the lens is driven during focusing, or fluctuation in the size of the image that occurs when the distance to the subject changes), it is caused by camera shake. It can be estimated that the signal change Ns to be expressed is expressed by the following Expression 13 when considered in the same manner as Expressions 1 and 2.

    Ns = ∂G / ∂t = Sm · ∂G / ∂m = Sm (x · ∂G / ∂x + y · ∂G / ∂y) (Formula 13)

  Sm is a value indicating the enlargement / reduction blur of the frame that occurs in the case of enlargement / reduction blur. ∂G / ∂m indicates a value indicating a change in the pixel value G due to enlargement / reduction blur. From Equation 13, if ∂G / ∂x and ∂G / ∂y are obtained, Ns can be obtained without obtaining ∂G / ∂m.

  According to Equation 13, ∂G / ∂m = x · ∂G / ∂x + y · ∂G / ∂y. Explain that this is true. FIG. 22 is an explanatory diagram for explaining these. When the zoom of the camera is enlarged by (1 + Δm) with the origin as the center, the change amount Δx of the x coordinate value and the change amount Δy of the y coordinate value of the pixel located at the coordinates (x1, y1) are as follows: Street.

Δx = x1 · Δm
Δy = y1 ・ Δm

  The change amount ΔG of the pixel value G of the pixel located at the coordinates (x1, y1) due to the enlargement is as follows.

ΔG≈∂G / ∂x · Δx + ∂G / ∂y · Δy
≒ ∂G / ∂x ・ x1 ・ Δm + ∂G / ∂y ・ y1 ・ Δm

  The formula obtained by dividing ΔG by Δm is as follows.

    ΔG / Δm≈x1 · ∂G / ∂x · + y1 · ∂G / ∂y

  This equation holds when Δm is sufficiently small. Therefore, ∂G / ∂m = x · ∂G / ∂x + y · ∂G / ∂y is established.

  With reference to FIG. 3A, in the first to fourth embodiments, the output of the two-dimensional image sensor 6 is linear with respect to the amount of light received by the two-dimensional image sensor 6 in order to simplify the description. It explained on the assumption. If it is not linear, it can be dealt with in one of the following two ways. In the first coping method, the signal processing unit 7 linearly corrects the output of the two-dimensional image sensor 6, and then generates moving image data D1. In the second coping method, the second calculation unit 92 calculates a regression surface as statistical processing, and uses this regression surface to calculate Sx and Sy.

  With reference to FIG. 3A, in the first to fourth embodiments, the image processing unit 9 performs predetermined image processing on the moving image, and the display control unit 10 displays the moving image that has been subjected to the image processing. 11 is displayed. The present invention is not limited to this configuration, and includes the image processing unit 9, but may be configured not to include the display control unit 10 and the display 11, or includes the image processing unit 9 and the display control unit 10. The structure which is not provided may be sufficient.

  In the first to fourth embodiments, the infrared camera 2 has been described as an example of the camera, but the present invention can also be applied to a visible light camera.

(Summary of embodiment)
An image processing apparatus according to a first aspect of the present embodiment that achieves the above object is an apparatus that performs image processing on a plurality of images arranged in a time series photographed by a camera, and the plurality of images that constitute the image For each of the pixels, a first calculation is to calculate a value indicating a spatial change in the pixel data of the pixel and a value indicating a temporal change in the pixel data of the pixel. Statistical processing of the first calculation unit for performing the first calculation, the value indicating the spatial change and the value indicating the time change calculated for each of the plurality of pixels constituting the image. Calculating a value indicating blurring of the image caused by blurring of the camera as a second calculation, and a second calculation unit that performs the second calculation for each of the plurality of images, Is provided. In other words, the image processing apparatus according to the first aspect of the present embodiment is an apparatus that performs image processing on a plurality of images arranged in a time series photographed by a camera, and the plurality of images A value indicating the spatial change of the pixel data, a first calculation unit for calculating a value indicating the temporal change of the pixel data, a value indicating the spatial change calculated by the first calculation unit, and A second calculation unit that statistically processes a value indicating the temporal change and calculates a value indicating each blur of the plurality of images caused by the camera blur.

  If camera shake occurs, a change in the spatial direction of the pixel data and a change in the temporal direction of the pixel data occur. If the camera shake amount is within one pixel, the change amount of pixel data and the shake amount are considered to be directly proportional. Based on these assumptions, the present inventor, for a plurality of images arranged in a time series taken by a camera, for a value indicating a spatial change of each of a plurality of pixels constituting the image and a value indicating a temporal change. Thus, it has been found that if a predetermined statistical process is performed, a value indicating image blur caused by camera blur can be obtained. Therefore, in the image processing device according to the first aspect of the present embodiment, the first calculation unit includes, for each of a plurality of pixels constituting the image, a value indicating a spatial change in pixel data of the pixel, The second calculation unit calculates a value indicating the temporal change of the pixel data, and the second calculation unit calculates a value indicating the spatial change calculated by the first calculation unit and a temporal change for each of the plurality of pixels constituting the image. Predetermined statistical processing is performed on the indicated value. Therefore, according to the image processing apparatus according to the first aspect of the present embodiment, a value indicating image blur caused by camera blur can be obtained. The same can be said for the other expressions.

  Pixel data is numerical information possessed by a pixel, for example, a pixel value (luminance value) of the pixel. In the case of an infrared image, it means a pixel value or a temperature calculated using the pixel value.

  As a value indicating the spatial change, there is a value obtained by differentiating the pixel data of the pixel in the direction of the spatial change.

  As a value indicating time change, there is a value obtained by directional differentiation of the pixel data of the pixel in the direction of time change. Moreover, there exists an alternating current component contained in each pixel data of a some pixel as a value which shows a time change.

  In the above-described configuration, the statistical processing is performed for each of a plurality of pixels constituting the image in a coordinate system including an axis indicating the direction of spatial change and an axis indicating the direction of temporal change. And a process of constructing coordinates with a value indicating the time change and a process of obtaining a regression equation approximating the distribution of the coordinates in the coordinate system.

  This configuration is an example of statistical processing. The figure shown by the regression equation includes a regression line, a regression curve, a regression plane, and a regression surface.

  In the above configuration, when the graphic represented by the regression equation is a regression plane, the second calculation unit includes an axis indicating the direction of the spatial change, an axis indicating the first change direction, and the first An axis indicating a second change direction orthogonal to the change direction, a plane defined by the axis indicating the first change direction and the axis indicating the time change direction, and an intersection line of the regression plane The intersection line of the regression plane and the plane defined by the axis indicating the second change direction and the axis indicating the direction of time change, and the first intersection line. The inclination of the first intersection line with respect to the axis indicating the change direction of the image is a value indicating blurring of the image along the first change direction, and the second intersection with respect to the axis indicating the second change direction. The inclination of the line is a value indicating the blurring of the image along the second change direction. When another expression is used, the second calculation unit is defined by an axis indicating the direction of the spatial change and an axis indicating the direction of the time change when the graphic represented by the regression equation is a regression plane. An inclination of an intersection line between the plane and the regression plane with respect to an axis indicating the direction of the spatial change is calculated.

  When the graphic represented by the regression equation is a regression plane, the value indicating the image blur caused by camera blur is the value indicating the image blur along the first change direction and the image along the second change direction. It is divided into the value indicating the blur of the. The value indicating the blurring of the image along the first change direction is the slope of the first intersection line with respect to the axis indicating the first change direction. The value indicating the blur of the image along the second change direction is the inclination of the second intersection line with respect to the axis indicating the second change direction. The same can be said for the other expressions.

  In the above configuration, for each of the plurality of pixels constituting the image, a signal change caused by camera shake is calculated using a value indicating the spatial change and a value indicating the blur of the image. Is a third calculation, and further includes a third calculation unit that performs the third calculation for each of the plurality of images. When another expression is used, the third calculation unit uses a value indicating the spatial change and a value indicating the blur of the image for each pixel of the plurality of images, and causes the blur of the camera. The signal change to calculate is calculated.

  According to this configuration, it is possible to calculate a signal change caused by camera shake, that is, noise due to camera shake. The same can be said for the other expressions.

  The above configuration further includes a correction unit that corrects the image using the signal change as a first correction, and performs the first correction on each of the plurality of images. When another expression is used, the correction unit corrects the plurality of images using the signal change calculated by the third calculation unit.

  According to this configuration, camera shake can be corrected for a plurality of images arranged in time series (for example, images to be monitored for gas leakage). The same can be said for the other expressions.

  In the above configuration, extracting a predetermined subject included in the image is subject extraction, and for each of the plurality of images, an extraction unit that extracts the subject, and the subject using the signal change A correction unit that performs the second correction on the subject extracted from each of the plurality of images. When another expression is used, the extraction unit extracts a predetermined subject from each of the plurality of images. The correction unit corrects the subject extracted by the extraction unit using the signal change calculated by the third calculation unit.

  According to this configuration, a subject (for example, a gas image) included in a plurality of images arranged in time series can be extracted, and camera shake can be corrected for the subject. The same can be said for the other expressions.

  In the above-described configuration, the second calculation unit divides the image into two or more regions, and performs the second calculation (the statistical processing) for each of the two or more regions, so that the camera A value indicating each blur of two or more of the regions caused by blur is calculated, and the second calculation unit uses a value indicating the blur of each of the two or more regions to move included in the image. The region indicating the image of the body is specified, and the second calculation unit uses the value indicating the blur of the region excluding the region indicating the image of the moving body to generate the image generated by the camera shake. A value indicating the blur of the image is calculated.

  While taking an image with a camera, in addition to the occurrence of camera shake, movement of a subject (for example, a car, a person, a tree branch, etc.) may occur. An image processing apparatus according to a first aspect of the present embodiment statistically processes a value indicating a spatial change and a value indicating a temporal change of each of a plurality of pixels constituting an image, and blurs an image caused by camera shake. A value indicating is calculated. In this case, if a value indicating blurring of an image caused by camera blur is calculated including pixels that form a region indicating an image of a moving subject, the accuracy of this value decreases.

  A moving subject appears as an image of a moving body in a plurality of images arranged in time series. In an image divided into two or more regions, a value indicating blurring of a region indicating an image of a moving object is different from a value indicating blurring of other regions. Therefore, in this configuration, in an image divided into two or more regions, a region indicating an image of a moving object included in the image is specified by using a value indicating blur of each region, and the region indicating an image of the moving object is determined. A value indicating blurring of an image caused by camera blur is calculated using a value indicating blurring of a region excluding.

  The image processing method according to the second aspect of the present embodiment is a method for performing image processing on a plurality of images arranged in a time series photographed by a camera, and each of a plurality of pixels constituting the image. , Calculating a value indicating a spatial change of pixel data of the pixel and a value indicating a temporal change of pixel data of the pixel as a first calculation, and for each of the plurality of images, the first Statistical processing of the value indicating the spatial change and the value indicating the temporal change calculated for each of the plurality of pixels constituting the image by the first step of calculating And calculating a value indicating blurring of the image caused by blurring of the camera as a second calculation, and performing the second calculation for each of the plurality of images; Equipped with a. If another expression is used, the image processing method according to the second aspect of the present embodiment is a method of performing image processing on a plurality of images arranged in a time series photographed by a camera, wherein the plurality of images A first step of calculating a value indicating a spatial change of pixel data and a value indicating a temporal change of the pixel data, a value indicating the spatial change calculated by the first step, and And a second step of statistically processing a value indicating a temporal change to calculate a value indicating each blur of the plurality of images caused by the camera blur.

  The image processing method according to the second aspect of the present embodiment defines the image processing apparatus according to the first aspect of the present embodiment from the viewpoint of the method, and the image processing according to the first aspect of the present embodiment. It has the same effect as the device. The same can be said for the other expressions.

  The image processing program according to the third aspect of the present embodiment is a program for performing image processing on a plurality of images arranged in a time series photographed by a camera, and each of a plurality of pixels constituting the image. , Calculating a value indicating a spatial change of pixel data of the pixel and a value indicating a temporal change of pixel data of the pixel as a first calculation, and for each of the plurality of images, the first Statistical processing of the value indicating the spatial change and the value indicating the temporal change calculated for each of the plurality of pixels constituting the image by the first step of calculating The second calculation is to calculate a value indicating blurring of the image caused by the camera blur, and the second calculation is performed for each of the plurality of images. A step, the causes the computer to execute. If another expression is used, the image processing program according to the third aspect of the present embodiment is a program for performing image processing on a plurality of images arranged in time series photographed by a camera, and the plurality of images A first step of calculating a value indicating a spatial change of pixel data and a value indicating a temporal change of the pixel data, a value indicating the spatial change calculated by the first step, and A computer is caused to execute a second step of statistically processing a value indicating a temporal change and calculating a value indicating each blur of the plurality of images caused by the camera blur.

  The image processing program according to the third aspect of the present embodiment defines the image processing apparatus according to the first aspect of the present embodiment from the viewpoint of the program, and the image processing according to the first aspect of the present embodiment. It has the same effect as the device. The same can be said for the other expressions.

  The gas detection system according to the fourth aspect of the present embodiment captures a moving image to be monitored and outputs moving image data in which a plurality of images are arranged in a time series, and the input from the infrared camera. For a plurality of images, calculate a value indicating a spatial change in pixel data and a value indicating a temporal change in the pixel data, and statistically process the value indicating the spatial change and the value indicating the temporal change, An image processing unit that calculates a value indicating each blur of the plurality of images caused by the blur of the infrared camera, and corrects each of the plurality of images based on a value indicating each blur of the plurality of images; A display for displaying a moving image in which the plurality of images corrected by the image processing unit are arranged in time series.

  The gas detection system according to the fourth aspect of the present embodiment defines the image processing apparatus according to the first aspect of the present embodiment from the viewpoint of the gas detection system, and relates to the first aspect of the present embodiment. It has the same effect as the image processing apparatus.

  This application is based on Japanese Patent Application No. 2006-078578 filed on Apr. 11, 2016, the contents of which are included in the present application.

  In order to express the present invention, the present invention has been properly and fully described through the embodiments with reference to the drawings. However, those skilled in the art can easily change and / or improve the above-described embodiments. It should be recognized that this is possible. Therefore, unless the modifications or improvements implemented by those skilled in the art are at a level that departs from the scope of the claims recited in the claims, the modifications or improvements are not covered by the claims. To be construed as inclusive.

  According to the present invention, it is possible to provide an image processing apparatus, an image processing method, an image processing program, and a gas detection system.

Claims (13)

An apparatus for performing image processing on a plurality of images arranged in time series photographed by a camera,
A first calculation unit that calculates a value indicating a spatial change in pixel data and a value indicating a temporal change in the pixel data for the plurality of images;
The value indicating the spatial change calculated by the first calculation unit and the value indicating the temporal change are statistically processed to calculate a value indicating each blur of the plurality of images caused by the camera shake. An image processing apparatus.   The statistical processing includes a value indicating the spatial change for each of a plurality of pixels constituting the image in a coordinate system including an axis indicating the direction of spatial change and an axis indicating the direction of temporal change. The image processing apparatus according to claim 1, further comprising: processing for forming coordinates with values indicating the time change; and processing for obtaining a regression equation that approximates the distribution of the coordinates in the coordinate system.   When the graphic represented by the regression equation is a regression plane, the second calculation unit includes a plane defined by an axis indicating the direction of the spatial change and an axis indicating the direction of the time change, and the regression plane The image processing apparatus according to claim 2, wherein an inclination of a line of intersection with respect to an axis indicating a direction of the spatial change is calculated.   The image processing apparatus further includes a third calculation unit configured to calculate a signal change due to camera shake using a value indicating the spatial change and a value indicating the blur of the image for each pixel of the plurality of images. Item 4. The image processing device according to any one of Items 1 to 3.   The image processing apparatus according to claim 4, further comprising: a correction unit that corrects the plurality of images using the signal change calculated by the third calculation unit. An extraction unit for extracting a predetermined subject from each of the plurality of images;
The image processing apparatus according to claim 4, further comprising: a correction unit that corrects the subject extracted by the extraction unit using the signal change calculated by the third calculation unit. The second calculation unit divides the image into two or more regions, and performs the statistical processing for each of the two or more regions, thereby each of the two or more regions generated by camera shake. Calculate the value indicating blur,
The second calculation unit specifies the region indicating the image of the moving body included in the image using a value indicating the blur of each of the two or more regions,
The second calculation unit calculates a value indicating blurring of the image caused by blurring of the camera using a value indicating the blurring of the area excluding the area indicating the image of the moving body. The image processing apparatus as described in any one of -6.   The image processing apparatus according to claim 1, wherein the value indicating the spatial change is a value obtained by performing a directional differentiation in the direction of the spatial change with respect to the pixel data.   The image processing apparatus according to claim 1, wherein the value indicating the time change is a value obtained by directional differentiation of the pixel data in the direction of the time change.   The image processing apparatus according to claim 1, wherein the value indicating the time change is an AC component included in the pixel data. A method of performing image processing on a plurality of images arranged in time series photographed by a camera,
A first step of calculating a value indicating a spatial change of pixel data and a value indicating a temporal change of the pixel data for the plurality of images;
The value indicating the spatial change calculated by the first step and the value indicating the temporal change are statistically processed to calculate a value indicating each blur of the plurality of images caused by the camera shake. And a second step. A program for performing image processing on a plurality of images arranged in time series photographed by a camera,
A first step of calculating a value indicating a spatial change of pixel data and a value indicating a temporal change of the pixel data for the plurality of images;
The value indicating the spatial change calculated by the first step and the value indicating the temporal change are statistically processed to calculate a value indicating each blur of the plurality of images caused by the camera shake. An image processing program for causing a computer to execute the second step. An infrared camera that shoots a video to be monitored and outputs video data in which multiple images are arranged in time series;
For the plurality of images input from the infrared camera, a value indicating a spatial change of pixel data and a value indicating a temporal change of the pixel data are calculated,
Statistically processing the value indicating the spatial change and the value indicating the time change, and calculating a value indicating each blur of the plurality of images caused by the blur of the infrared camera,
An image processing unit that corrects each of the plurality of images based on a value indicating blur of each of the plurality of images;
A gas detection system comprising: a display that displays a moving image in which the plurality of images corrected by the image processing unit are arranged in time series.

Images