JP2016174205A - Program, image processing device and image processing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a program, an image processing device and an image processing system that can reduce deterioration of image quality of pickup images due to heat haze.SOLUTION: A program makes an image processing device for processing pickup images use plural pickup images captured at different height imaging positions to calculate, every imaging position, the intensity of vibration of an imaging device when the pickup images are generated, and use the plural pickup images to calculate, every imaging position, the strength of the heat haze occurring between the imaging device generating the pickup images and a target object. Then, the program makes the image processing device calculate, every imaging position, an image quality deterioration score representing the degree of image quality deterioration of a pickup image due to the heat haze on the basis of the intensity of the vibration and the strength of the heat haze at each imaging position, and the distance between the imaging device and the target object, and specify the optimum position for minimizing the image quality deterioration of the pickup image due to the heat haze on the basis of the image quality deterioration score.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing technique for improving the image quality of a captured image, and more particularly to a program, an image processing apparatus, and an image processing system that reduce image quality deterioration of a captured image due to a hot flame.

  Conventionally, when an object is photographed in the case where a flame, which is a refraction phenomenon of light due to a mixture of gases having a temperature difference between the photographing apparatus and the object, is photographed, there is a problem that the photographed image fluctuates due to the flame. Exists. There has been proposed an image processing technique for improving the image quality of a photographed image deteriorated by such a hot flame so that an object in the photographed image can be perceived satisfactorily.

  As an example of an image processing technique for correcting a positive flame, Patent Document 1 creates a reference image from images sequentially input from a photographing apparatus, calculates an optical flow between the latest input image and the reference image, and calculates the optical flow. An image processing method for correcting fluctuations in the latest input image is disclosed.

  However, since the image quality of a captured image is greatly deteriorated when a strong flame is generated, the conventional image processing method disclosed in Patent Document 1 can sufficiently correct a captured image whose image quality has been degraded by the flame. There was a problem that I could not.

  The present invention has been made in view of the above-described problems of the prior art, and an object of the present invention is to provide a program, an image processing apparatus, and an image processing system that can reduce image quality degradation of a captured image due to a hot flame. .

  The program of the present invention uses a plurality of photographed images taken at photographing positions with different heights to an image processing apparatus that processes a photographed image, and provides strong vibration of the photographing apparatus when the photographed image is generated. For each shooting position, the intensity of the heat generated between the shooting device that generated the shot image and the object is calculated for each shooting position. Then, the program according to the present invention is based on the intensity of vibration and the intensity of the flame at each shooting position, and the distance between the imaging device and the object, and the image quality degradation indicating the degree of image quality degradation of the photographed image due to the flame. A score is calculated for each shooting position, and based on the image quality deterioration score, an optimum position that minimizes the image quality deterioration of the shot image due to the hot flame is specified.

  In the present invention, by adopting the above-described configuration requirements, it is possible to specify an optimal shooting position when a hot flame occurs, and therefore, it is possible to reduce deterioration in image quality of a shot image due to the hot flame.

The figure which shows one Embodiment of the image processing system of this invention. The figure which shows one Embodiment of the hardware constitutions of the control unit with which the image processing apparatus of this invention is provided. The figure which shows one Embodiment of the function structure which the control unit of this invention has. 6 is a flowchart illustrating an embodiment of processing executed by the image processing apparatus of the present invention. The flowchart which shows one Embodiment of the calculation process of the strength of a vibration for every imaging | photography position. The flowchart which shows one Embodiment of the calculation process of the intensity of the hot flame for every imaging | photography position. 9 is a flowchart illustrating an embodiment of a process for calculating an image quality degradation score for each shooting position. The figure which shows one Embodiment of an image quality degradation table. The figure which shows one Embodiment of the continuous function which fits the image quality degradation score of a discrete imaging | photography position.

  FIG. 1 is a diagram showing an embodiment of an image processing system of the present invention. An image processing system 100 illustrated in FIG. 1 includes an image processing device 110 and a support 120. The image processing apparatus 110 captures an object at a plurality of predetermined positions along the support 120, and uses the captured images at the plurality of capturing positions to identify an optimal position at which the influence of the hot flame on the captured image is minimized. It is. The image processing apparatus 110 can move to the identified optimum position and photograph the object.

  The image processing device 110 includes a control unit 111, a photographing device 112, and a distance measuring device 113. As illustrated in FIG. 2, the control unit 111 includes a processor 200, a ROM 201, a RAM 202, and a device interface 203.

  The processor 200 is an arithmetic processing device that executes the program of the present invention. The ROM 201 is a storage device that stores various data such as the program of the present invention. The RAM 202 is a storage device that provides a program execution space. In the present embodiment, the processor 200 reads out the program of the present invention from the ROM 201 under the management of various OSs, develops the program in the RAM 202, and executes it, thereby realizing the functions described later on the image processing apparatus 110. In another embodiment, the functions described later may be realized by the image processing apparatus 110 by a semiconductor integrated circuit that executes the program of the present invention.

  The device interface 203 is an interface that connects the control unit 111 to the photographing device 112 and the distance measuring device 113. The photographing device 112 and the distance measuring device 113 operate under the control of the control unit 111.

  The imaging device 112 is an apparatus that captures an object and generates a captured image. The distance measuring device 113 is a device that measures the distance between the object and the photographing device 112. As the distance measuring device 113, for example, various distance measuring means such as a laser measuring instrument and a stereo camera can be employed.

  Further, the image processing apparatus 110 includes a pair of motors (not shown) and a support 120 as a lifting device. The pair of motors are attached so as to sandwich the support 120, and the image processing apparatus 110 can be moved up and down along the support 120 by rotating the motor. In another embodiment, an elevator driving mechanism installed on the column 120 may be adopted as the lifting device, and the image processing apparatus 110 may be moved up and down along the column 120 by the elevator driving mechanism.

  In the embodiment shown in FIG. 1, the control unit 111, the imaging device 112, and the distance measuring device 113 are integrally configured as the image processing device 110. However, in other embodiments, the control unit 111 and the imaging device The device 112 and the distance measuring device 113 may be configured separately, and the control unit 111 that is an image processing device may control the photographing device 112 and the distance measuring device 113 via wireless communication.

  FIG. 3 is a diagram illustrating an embodiment of a functional configuration of the control unit 111 of the image processing apparatus 110. Hereinafter, functions of the control unit 111 will be described with reference to FIG.

  The control unit 111 includes a control unit 300, a lift control unit 301, a distance measurement unit 302, a photographing unit 303, a vibration intensity calculation unit 304, a hot flame intensity calculation unit 305, a score calculation unit 306, and an optimum position specification. A unit 307, a lifting device control unit 308, a measurement device control unit 309, and a photographing device control unit 310.

  The control unit 300 is a unit that performs overall control of the image processing apparatus 110 and controls other functional units included in the control unit 11. The lifting control unit 301 is a unit that controls the lifting device via the lifting device control unit 308 and controls the lifting operation of the image processing apparatus 110. The distance measurement unit 302 is a unit that controls the distance measurement device 113 via the measurement device control unit 309 and measures the distance between the image processing device 110 and the object. The photographing unit 303 is a unit that controls the photographing device 112 via the photographing device control unit 310 to photograph a target object and generate a photographed image.

  The vibration intensity calculation unit 304 uses a plurality of photographed images taken at photographing positions with different heights, and calculates the strength of vibration of the photographing device 112 when the photographed image is generated for each photographing position. Means. The flame intensity calculation unit 305 uses a plurality of photographed images photographed at photographing positions with different heights, and the intensity of the flame generated between the photographing apparatus 112 that has generated the photographed image and the object. Is a means for calculating for each photographing position.

  The score calculation unit 306 is a means for calculating a score (hereinafter referred to as “image quality degradation score”) indicating the degree of image quality degradation of a photographed image due to the heat generated between the imaging device 112 and the object. . The score calculation unit 306 includes a vibration intensity for each imaging position calculated by the vibration intensity calculation unit 304, a heat intensity for each imaging position calculated by the flame intensity calculation unit 305, and the imaging device 112 calculated by the distance measurement unit 302. The image quality degradation score is calculated for each photographing position based on the distance between the object and the object.

  The optimum position specifying unit 307 is a means for specifying, as the optimum position, a photographing position that minimizes image quality degradation of a photographed image due to the heat generated between the photographing device 112 and the object.

  The lifting device control unit 308 is means for controlling a lifting device that lifts and lowers the image processing device 110. The measurement device control unit 309 is means for controlling the distance measurement device 113 that measures the distance between the image processing device 110 and the object. The imaging device control unit 310 is a unit that controls the imaging device 112 included in the image processing device 110.

  FIG. 4 is a flowchart illustrating an embodiment of processing executed by the image processing apparatus 110. Hereinafter, with reference to FIG. 4, a process of photographing an object at a plurality of discrete photographing positions and specifying an optimum position using the photographed images will be described.

  The processing in FIG. 4 starts from step S400. In step S401, the lifting control unit 301 controls the lifting device via the lifting device control unit 308 to move the image processing device 110 to a predetermined photographing position. In step S402, the distance measuring unit 302 causes the distance measuring device 113 to measure the distance between the image processing device 110 and the object via the measuring device control unit 309, and the distance is set to an image quality as shown in FIG. It is stored in a data table (hereinafter referred to as “image quality deterioration table”) in which information relating to deterioration is recorded. In step S <b> 402, the photographing unit 303 photographs a target object a plurality of times via the photographing device control unit 310, and generates a plurality of photographed images at the photographing position.

  In step S404, the control unit 300 determines whether or not the processes in steps S401 to S403 have been executed for all predetermined shooting positions (for example, heights 0 m, 15 m, and 30 m). When the process is not executed for all predetermined shooting positions (no), the process returns to step S401, and the processes of steps S401 to S403 are executed for the next shooting position. On the other hand, when the process is executed for all predetermined shooting positions (yes), the process branches to step S405.

  In step S405, the vibration strength calculation unit 304 calculates the strength of vibration of the photographing device 112 when the photographed image is generated for each photographing position. The processing for calculating the vibration intensity of the photographing apparatus 112 will be described in detail with reference to FIG.

  In step S406, the flame intensity calculation unit 305 calculates the intensity of the flame generated between the imaging device 112 and the object for each imaging position. The calculation process of the intensity of the hot flame will be described in detail with reference to FIG.

  In step S407, the score calculation unit 306 calculates an image quality degradation score for each shooting position. The image quality degradation score calculation process will be described in detail with reference to FIG.

  In step S <b> 408, the optimum position specifying unit 307 specifies the shooting position at which image quality degradation of the shot image due to the heat is minimized as the optimum position. In the present embodiment, the optimum position specifying unit 307 refers to an image quality deterioration table as shown in FIG. 8 and specifies the shooting position (for example, height 15 m) at which the image quality deterioration score is minimum as the optimum position.

  In step S409, the lifting control unit 301 controls the lifting device via the lifting device control unit 308, and moves the image processing apparatus 110 to the optimum position. In step S410, the imaging unit 303 captures an object and generates a captured image via the imaging device control unit 310, and the process ends in step S411.

  FIG. 5 is a flowchart illustrating an embodiment of a process for calculating, for each shooting position, the strength of vibration of the shooting device 112 when a shot image is generated.

The process shown in FIG. 5 starts from step S500. In step S501, the vibration intensity calculation unit 304 performs a plurality of operations at a predetermined one shooting position (for example, height 0 m) generated in the process of step S403 shown in FIG. The captured images (L _{1 to} L _N ) are acquired. Here, N indicates the number of captured images.

In step S502, the vibration intensity calculation unit 304 performs template matching using the latest captured image (L _N ) and other captured images (L _{1 to} L _N-1 ) among these captured images, calculating a latest photographed image _{(L N),} the variation vector of the captured image _{(L 1 ~L N-1)} to _{(J 1 ~J N-1)} .

In step S503, the vibration intensity calculation unit 304 calculates the standard deviation of these variation vectors (J _{1 to} J _N-1 ) as the vibration intensity K at the photographing position (for example, height 0 m). More specifically, the vibration intensity calculation unit 304 can calculate the vibration intensity K using Equation 1.

Here, K _x indicates the standard deviation of the value _x of the variation vector (J _{1 to} J _N-1 ). K _y represents the standard deviation of the y value of the variation vector (J _{1 to} J _N-1 ).

  In step S504, the vibration strength calculation unit 304 stores the vibration strength K calculated in step S503 in the image quality deterioration table. In step S505, the vibration intensity calculation unit 304 determines whether or not the vibration intensity K has been calculated for all the shooting positions (for example, heights 0 m, 15 m, and 30 m).

  If the vibration intensity K has not been calculated for all shooting positions (no), the process returns to step S501, and the vibration intensity K is calculated for the next shooting position (for example, height 15 m or height 30 m). To do. On the other hand, when the vibration intensity K is calculated for all predetermined shooting positions (yes), the process ends in step S506.

  FIG. 6 is a flowchart illustrating an embodiment of a process for calculating the strength of the heat generated between the image processing apparatus 110 and the object for each photographing position.

The process shown in FIG. 6 starts from step S600. In step S601, the hot flame intensity calculation unit 305 is in a predetermined one shooting position (for example, height 0 m) generated in the process of step S403 shown in FIG. A plurality of captured images (L _{1 to} L _N ) are acquired. In step S602, the hot flame intensity calculation unit 305 extracts the feature points (P _{1_1} , P _{1_2} ,... P _{1_M} ) of the oldest captured image (L ₁ ) among these captured images. Here, M indicates the number of feature points included in the captured image.

In step S603, shimmer intensity calculator 305 tracks the feature points of the captured image _(L 2 ~L _N) for shooting the image _{(L 1),} the variation of the feature points in the captured image _(L 2 ~L _N) Vectors (A _{2_1} , A _{2_2} ,... A _{2_M} , A _{3_1,} A _{N_M} ) are calculated. Here, for example, the variation vector (A _{2_1} ... A _{2_M} ) indicates the variation vector of the feature point in the captured image (L ₂ ). The hot flame intensity calculation unit 305 can calculate the variation vector (A _{2_1} ... A _{N_M} ) of each feature point of the captured image (L ₁ ) in the captured image (L _{2 to} L _N ) using Equation 2.

Here, g (2 ≦ g ≦ N) indicates an identification number of the captured image. Further, i (1 ≦ i ≦ M) indicates an identification number of a feature point in the captured image (L ₁ ).

At step S604, the shimmer intensity calculator 305, the captured image _(L 2 ~L _N) for the net variation vector of each feature point of the captured image _{(L 1) (C 2_1,} C 2_2, ... C 2_M, C 3_1 ... C _{N_M} ) is calculated. More specifically, the _hot flame intensity calculation unit 305 includes the variation vector (A _{2_1} ... A _{N_M} ) of the feature point in the captured image (L _{2 to} L _N ) and the vibration of the feature point calculated by the vibration intensity calculation unit 304. a variation vector _(J 1 ~J _N) is substituted into equation 3, it is possible to calculate the net variation vector _{(C 2_1} ... _{C N_M)} of each feature point of the captured image _{(L 1)} by. Note that _JN is 0.

Here, g (2 ≦ g ≦ N) indicates an identification number of the captured image. Further, i (1 ≦ i ≦ M) indicates an identification number of a feature point in the captured image (L ₁ ).

In step S605, the _hot flame intensity calculation unit 305 calculates the standard deviation (S _{1 to} S _M ) of the feature point based on the net variation vector (C _{2_1} ... C _{N_M} ) of each feature point of the captured image (L ₁ ). calculate. More specifically, the hot flame intensity calculation unit 305 can calculate the standard deviation (S _{1 to} S _M ) of the feature points using Equation 4.

Here, S _i represents the standard deviation of the net variation vector (C _{2_1} ... C _{N_M} ) of each feature point of the captured image (L ₁ ). S _iX represents the standard deviation of the variation vector in the x-axis direction, and S _iY represents the standard deviation of the variation vector in the y-axis direction. i (1 ≦ i ≦ M) indicates the identification number of the feature point in the captured image (L ₁ ).

In step S606, the hot flame intensity calculation unit 305 uses the average value of the standard deviations (S _{1 to} S _M ) of the feature points calculated in step S605 as the hot flame intensity D at the photographing position (for example, height 0 m). calculate. In step S607, the flame intensity calculation unit 305 stores the flame intensity D calculated in step S606 in the image quality deterioration table.

  In step S608, the hot flame intensity calculation unit 305 determines whether or not the hot flame intensity D has been calculated for all photographing positions (for example, heights of 0 m, 15 m, and 30 m). When the flame intensity D has not been calculated for all shooting positions (no), the process returns to step S601, and the flame intensity D is calculated for the next shooting position (for example, height 15 m or height 30 m). To do. On the other hand, when the intensity D of the hot flame is calculated for all the predetermined shooting positions (yes), the process ends in step S609.

  As described above, in the embodiment shown in FIG. 6, the intensity of the hot flame can be calculated in consideration of the strength of vibration of the imaging device 112 when the captured image is generated.

FIG. 7 is a flowchart illustrating an embodiment of a process for calculating an image quality degradation score for each shooting position. The processing shown in FIG. 7 starts from step S700, and in step S701, the score calculation unit 306 calculates the spread (dispersity) of the apparent points at predetermined shooting positions (for example, 0 m, 15 m, and 30 m). More specifically, the score calculation unit 306 calculates the apparent point spread (D _{SP_H} ) by substituting the strength of the _hot flame and the strength of vibration registered in the image quality deterioration table into Equation 5.

Here, _DH indicates the strength of the positive flame at the H meter point from the ground, and K _H indicates the strength of the vibration at the H meter point. _{DSP_H} indicates the spread of the apparent point at the H meter point. In the present embodiment, H is a real number greater than or equal to zero.

In step S702, the score calculation unit 306, the spread of the apparent point in a predetermined photographing position _{(D SP_H)} is normalized to calculate the extent of the apparent point of 0 m point _{(D SPGL_H).} More specifically, the score calculation unit 306 can calculate the apparent point spread (D _{SPGL_H} ) at the 0-meter ground using Equation 6.

Here, L _H indicates the distance between the imaging device 112 and the object at the H meter point from the ground, and L ₀ indicates the distance between the imaging device 112 and the object at the 0 meter point from the ground. D _{SPGL_H} indicates the spread of an apparent point at a point of 0 meters above the ground. In the present embodiment, H is a real number greater than or equal to zero.

In step S703, the score calculation unit 306 calculates an image quality degradation score at each shooting position. More specifically, the score calculation unit 306 can calculate the image quality degradation score (Q _H ) using Equation 7.

Here, Q _H indicates an image quality deterioration score at the H meter point on the ground. In the present embodiment, H is a real number greater than or equal to zero.

In another embodiment, a correction coefficient (a value of 0 or more and less than 1) may be applied to the vibration intensity _DH in Expression 7. In other embodiments, other factors that degrade image quality (for example, the amount of smoke or obstacles) may be introduced into Equation 7.

  In step S704, the score calculation unit 306 stores the image quality deterioration score calculated in step S703 in the image quality deterioration table, and the process ends in step S705.

  In the above-described embodiment, the optimal position specifying unit 307 refers to the image quality deterioration table as illustrated in FIG. 8 and specifies the shooting position with the minimum image quality deterioration score. In other embodiments, the optimal position specifying unit 307 In step 307, a continuous function such as a quadratic polynomial, a cubic polynomial, or a cubic spline curve is used to fit an image quality deterioration score at a discrete shooting position, thereby specifying a shooting position that minimizes the image quality deterioration score. it can.

  For example, an image quality degradation score at an arbitrary shooting position can be approximated using a second-order polynomial function as shown in FIG. When the number of actual shooting positions is greater than the number obtained by adding 1 to the degree of the polynomial, a curve passing through all points corresponding to the height of the shooting position and the image quality degradation score cannot be calculated. Thus, a curve defined by a cubic polynomial that minimizes the amount of error as shown in FIG. 9 can be calculated.

  Furthermore, it is possible to approximate the image quality degradation score at an arbitrary shooting position using a spline curve as shown in FIG. By using such a continuous function to approximate the image quality degradation score at an arbitrary shooting position, a more optimal shooting position can be calculated.

  Although the present embodiment has been described so far, the present invention is not limited to the above-described embodiment, and the constituent elements of the above-described embodiment are changed or deleted, or other than the constituent elements of the above-described embodiment. It is possible to make modifications within the range that can be conceived by those skilled in the art, such as adding the above-described components. Any aspect is included in the scope of the present invention as long as the effects of the present invention are exhibited.

  DESCRIPTION OF SYMBOLS 100 ... Image processing system, 110 ... Image processing apparatus, 111 ... Control unit, 112 ... Imaging | photography apparatus, 113 ... Distance measuring device, 120 ... Support | pillar

JP2012-104018

Claims (9)

A program that can be executed by an image processing apparatus that processes a captured image.
Using a plurality of photographed images photographed at photographing positions with different heights to calculate the strength of vibration of the photographing device when the photographed image is generated for each photographing position;
Using each of the plurality of captured images to calculate the intensity of the heat generated between the imaging device that generated the captured image and the object for each imaging position;
An image quality degradation score indicating the degree of image quality degradation of the photographed image due to the heat flame is determined for each photographing position based on the strength of vibration and the strength of the heat flame for each photographing position and the distance between the photographing device and the object. A calculating step;
And a step of specifying an optimum position that minimizes image quality degradation of a photographed image due to the hot flame based on the image quality degradation score.   The step of identifying the optimum position includes fitting an image quality degradation score at discrete shooting positions using a continuous function to identify an optimum position that minimizes image quality degradation of the shot image. The listed program. The program further provides to the image processing apparatus.
The program according to claim 1, wherein the step of moving the photographing device to the identified optimum position is executed. An image processing apparatus for processing a captured image,
Vibration intensity calculating means for calculating, for each shooting position, the strength of vibration of the shooting device when the shot image is generated using a plurality of shot images shot at different shooting positions;
A flame intensity calculating means for calculating the intensity of the flame generated between the imaging device that generated the captured image and the object using the plurality of captured images for each imaging position;
An image quality degradation score indicating the degree of image quality degradation of the photographed image due to the heat flame is determined for each photographing position based on the strength of vibration and the strength of the heat flame for each photographing position and the distance between the photographing device and the object. A score calculating means for calculating;
An image processing apparatus comprising: an optimum position specifying unit that specifies an optimum position that minimizes image quality deterioration of a photographed image due to the hot flame based on the image quality deterioration score.   5. The optimum position specifying unit is configured to fit an image quality deterioration score at discrete shooting positions using a continuous function, and specify an optimum position that minimizes image quality deterioration of a shot image. The image processing apparatus described.   The image processing apparatus according to claim 4, further comprising a lift control unit that moves the photographing apparatus to the specified optimum position. An image processing system including an image processing device including a photographing device and a lifting device,
The image processing apparatus includes:
Vibration intensity calculating means for calculating, for each shooting position, the strength of vibration of the shooting device when the shot image is generated using a plurality of shot images shot at different shooting positions;
A flame intensity calculating means for calculating the intensity of the flame generated between the imaging device that generated the captured image and the object using the plurality of captured images for each imaging position;
An image quality degradation score indicating the degree of image quality degradation of the photographed image due to the heat flame is determined for each photographing position based on the strength of vibration and the strength of the heat flame for each photographing position and the distance between the photographing device and the object. A score calculating means for calculating;
An image processing system comprising: an optimum position specifying unit that specifies an optimum position that minimizes image quality deterioration of a photographed image due to the hot flame based on the image quality deterioration score.   8. The optimum position specifying unit is configured to fit an image quality deterioration score at discrete shooting positions using a continuous function, and specify an optimum position that minimizes image quality deterioration of a shot image. The image processing system described.   The image processing system according to claim 7, further comprising a lifting control unit that controls the lifting device to move the photographing device to the specified optimum position.