JP2006138735A - Method, device and program of detecting camera angle change, and method of image processing, facility monitoring, surveying and setting stereoscopic camera using same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily and accurately detect the difference in camera angle between two images. <P>SOLUTION: For a standard image and an input image, individual power spectrum is calculated (S1), for individual power spectrum of the standard image and the input image, log polar transformation is carried out (S2). Rectangular regions corresponding to a fan-shaped region in a specific direction of the standard image and the input image concerning the power spectrum after log polar transformation are extracted by varying the specific direction of the fan-shaped region in order (S3). A phase-limited correlation between the rectangular region in the extracted standard image and the rectangular region in the input image is calculated to obtain a shift quantity in radial direction and in arc direction (S4). Rotation angle α is obtained from the arc direction shift (S5), and the expansion/contraction rate of input image to the standard image in every direction is obtained from the radial direction shift in each direction. From the ratio of maximum to minimum of the expansion/contraction rate, a plane inclination angle β is obtained (S6) and the direction minimizing the expansion/contraction rate is defined as a plane inclination angle γ (S7). <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a camera angle change detection method, apparatus, and program. More specifically, the present invention relates to a method, an apparatus, and a program for detecting a relative camera angle difference between two images using an image processing technique, and an image processing method, an equipment monitoring method, and a survey method using the same. And a stereo camera setting method.

  As a method for detecting the tilt of the camera by image processing, there is a method using a marker or the like. For example, Patent Document 1 discloses that a calibration chart having an artificial marker is used as a stereo camera calibration method.

  Non-Patent Document 1 proposes a method for obtaining the rotation and inclination angle of a texture plane between two images to be compared using a spectral moment.

JP 2003-242485 A Toshio Tsuji, Ikuo Yoshida: Detection of texture plane rotation and tilt angle using spectral moments: IEICE Transactions, Vol. J85-D-II, No. 3, pp.406--415 (2002).

  It is very difficult for a patrolman to visually check the progress of cracks and rust on the monitored facility on site in inspection work of the monitoring target facility by a patrolman. On the other hand, due to the low price of digital devices and the advancement of information processing technology, the environment for digital recording and storage of on-site situations is being established in on-site operations. Therefore, the patrolman takes a picture of the equipment to be monitored with a digital camera, records and saves the taken picture, and compares the newly taken picture of the equipment surface with the past picture of the equipment surface. It may be possible to accurately grasp the minute progress of rust and rust. For example, by creating a differential image between an image of a past monitoring target facility and a newly captured monitoring target facility, it is possible to accurately detect the minute progress of cracks and rust from the difference information of the image. it can.

  However, for this purpose, it is necessary to accurately align the images to be compared, and it is necessary to always fix the monitoring object and the camera in the same positional relationship. If the positional relationship between the monitoring object and the camera, that is, the camera angle, changes between the past shooting time and the current shooting time, it is recorded as an image in which the optical axis is shifted. Comparing images that do not match the camera angle, it will be processed as if it had been transformed to image information that did not change originally, greatly reducing the accuracy of detection of the minute progress of cracks and rust End up. For example, reference numeral 101 in FIG. 15 is an image showing the cracked state of the monitoring target equipment photographed in the past, and reference numeral 102 is an image showing the cracking state of the monitored target equipment photographed this time. The camera angle changes between the past captured image 101 and the current captured image 102. Reference numeral 103 is a difference image between the past captured image 101 and the current captured image 102. Even if the difference image 103 between the images 101 and 102 whose camera angles do not coincide with each other is created in this way, even the image information that has not changed originally appears as a difference, and a minute progress state of cracks and rust is displayed. It cannot be detected accurately.

  On the other hand, since it is difficult physically and costly to keep the positional relationship between the subject to be photographed and the camera, it is very difficult to realize photographing from the same camera position as the past. Therefore, it is conceivable to detect a difference in camera angle between two images by image processing technology and perform correction so that the camera angles match.

  However, the technique of Patent Document 1 is troublesome because it is necessary to prepare and set markers manually. Further, the method using the spectral moment of Non-Patent Document 1 has a problem that the error becomes large when the inclination angle of the plane is very small (for example, 5 ° or less).

  In other words, the change in the camera angle is a change in the optical axis. By accurately detecting the amount of deviation of the optical axis between the two images, for example, highly accurate triangulation, stereo camera settings, camera lenses, etc. It can be applied to distortion detection and correction. However, as described above, in the conventional image processing technique, there is a problem that the complexity of using the marker and the amount of deviation of the minute optical axis cannot be accurately detected. For this reason, in the corresponding point search between images by the conventional image processing, correction of a minute change in the direction of the camera is insufficient, the association between the images becomes ambiguous, and sufficient measurement accuracy cannot be obtained. .

  Therefore, the present invention provides a camera angle change detection method and apparatus and program capable of easily and accurately detecting a difference in camera angle between two images, an image processing method, an equipment monitoring method, and a survey method using the same. An object of the present invention is to provide a stereo camera setting method.

  In order to achieve such an object, the invention described in claim 1 uses a standard image and an input image obtained by photographing regions that can be regarded as the same plane of a subject to be photographed, and a relative camera angle between the standard image and the input image. A method for detecting a difference, wherein the plane area of the imaging target in the standard image is a standard plane, the plane area of the imaging target in the input image is an input plane, and the plane inclination of the input plane with respect to the standard plane Obtaining a direction γ, a plane tilt angle β of the input plane with respect to the standard plane about an axis orthogonal to the plane tilt direction γ and the plane of the standard image, and the input plane relative to the standard plane A rotation direction α about an axis orthogonal to the plane of the standard image, and a sector in a specific direction selected from the standard image and the input image with a fixed point as the center Each of the areas is extracted, and a correlation between the two fan-shaped areas extracted is obtained to obtain a radial shift amount of the fan-shaped area extracted from the input image with respect to the fan-shaped area extracted from the standard image, The radial direction deviation amount is obtained for all directions by sequentially changing the specific direction of the region, the enlargement / reduction ratio of the input image with respect to the standard image for each direction is obtained from the radial direction deviation amount for each direction, and the enlargement / reduction is performed The plane inclination angle β is obtained from the ratio of the maximum value and the minimum value, and the direction in which the enlargement / reduction ratio is minimum is set as the plane inclination direction γ.

  According to a twelfth aspect of the present invention, there is provided an apparatus for detecting a difference in relative camera angle between the standard image and the input image by using a standard image and an input image obtained by photographing regions that can be regarded as the same plane of the photographing target. The plane area of the imaging target in the standard image is a standard plane, the plane area of the imaging target in the input image is an input plane, and the plane inclination direction γ of the input plane with respect to the standard plane, Means for determining a plane tilt direction γ and a plane tilt angle β of the input plane with respect to the standard plane about an axis orthogonal on the plane of the standard image; and a plane of the standard image of the input plane with respect to the standard plane; Means for obtaining a rotation angle α around an orthogonal axis, and extracting each sector area in a specific direction selected around a certain point from the standard image and the input image, By obtaining a correlation between the two fan-shaped areas extracted, a radial shift amount of the fan-shaped area extracted from the input image with respect to the fan-shaped area extracted from the standard image is obtained, and a specific direction of the fan-shaped area is sequentially changed. And determining the radial displacement amount for all directions, obtaining the enlargement / reduction ratio of the input image with respect to the standard image for each direction from the radial displacement amount for each direction, and the maximum and minimum values of the enlargement / reduction ratio The plane tilt angle β is obtained from the ratio of the above, and the direction in which the enlargement / reduction ratio is minimized is set as the plane tilt direction γ.

  According to a thirteenth aspect of the present invention, a standard image obtained by photographing an area that can be regarded as the same plane of an imaging target and an input image are used, and the planar area of the imaging target in the standard image is set as a standard plane. The plane area of the object to be imaged is set as an input plane, the plane inclination direction γ of the input plane with respect to the standard plane, and the standard plane around the axis orthogonal to the plane inclination direction γ on the plane of the standard image A program that causes a computer to function as means for obtaining a plane inclination angle β of the input plane, and means for obtaining a rotation angle α about an axis orthogonal to the plane of the standard image of the input plane with respect to the standard plane, Extract a sector area in a specific direction selected around a certain point from the image and the input image, and obtain the correlation between the two extracted sector areas Thus, the radial deviation amount of the sector area extracted from the input image with respect to the sector area extracted from the standard image is obtained, and the specific direction of the sector area is sequentially changed to change the radial deviation amount in all directions. Finding the enlargement / reduction ratio of the input image with respect to the standard image for each direction from the radial deviation amount for each direction, obtaining the plane inclination angle β from the ratio of the maximum value and the minimum value of the enlargement / reduction ratio, The computer is caused to function as means for setting the direction in which the enlargement / reduction ratio is minimized to the plane inclination direction γ.

  When the texture plane is tilted in the γ direction by an angle β, the plane is reduced by cos β in the γ direction. Therefore, the deviation amount between the two images is obtained for each direction with respect to all directions around a certain point, thereby obtaining the enlargement / reduction ratio of the image for each direction, and calculating the plane from the ratio between the maximum value and the minimum value of the enlargement / reduction ratio of the image. Accurate γ and β can be obtained by obtaining the inclination angle β and setting the direction in which the enlargement / reduction ratio of the image is minimum as the plane inclination direction γ.

  According to a second aspect of the present invention, in the camera angle change detection method according to the first aspect, a power spectrum is calculated for each of the standard image and the input image, and a direct current of the power spectrum is calculated for each of the power spectra. The amount of deviation in the radial direction for each direction around the component is obtained.

  Therefore, by using the power spectra respectively calculated for the standard image and the input image, it is possible to cancel the vertical / horizontal misalignment between the standard image and the input image. In addition, by selecting the DC component of the power spectrum as the origin on the spatial frequency and selecting the direction of the sector area around the DC component (origin on the spatial frequency), the direction of the sector area is selected for the standard image and the input image. This saves you the trouble of setting the center point.

  The invention according to claim 3 is the camera angle change detection method according to claim 2, wherein log power conversion is performed on each of the power spectra of the standard image and the input image, and the power spectrum after the log polar conversion is performed. In this case, a rectangular area equivalent to the fan-shaped area is extracted, and the phase-only correlation between the two extracted rectangular areas is calculated to obtain the radial shift amount. Therefore, by using the log polar conversion, the enlargement / reduction ratio can be easily obtained using the logarithmic characteristics.

  According to a fourth aspect of the present invention, in the camera angle change detection method according to any one of the first to third aspects, the correlation between the two fan-shaped regions respectively extracted from the standard image and the input image is calculated. By obtaining, the arc direction deviation amount of the sector area extracted from the input image with respect to the sector area extracted from the standard image is obtained, and the rotation angle α is obtained based on the arc direction deviation amount. In this case, the calculation for obtaining the rotation angle α can be performed in parallel with the calculation for obtaining the plane inclination angle β and the plane inclination direction γ.

  According to a fifth aspect of the present invention, in the camera angle change detection method according to any one of the first to third aspects, the input image is obtained in the direction of the obtained plane tilt direction γ. The primary correction image is enlarged by 1 / cos β using the plane inclination angle β to obtain a primary correction image, and the correlation between the primary correction image and the standard image is obtained, thereby rotating the primary correction image relative to the standard image. An amount is obtained, and the obtained rotation deviation amount is set as the rotation angle α. In this case, after obtaining the plane inclination angle β and the plane inclination direction γ, the rotation angle α is obtained.

  An image processing method according to a sixth aspect uses the rotation angle α, the plane inclination angle β, and the plane inclination direction γ obtained by the camera angle change detection method according to any one of the first to fifth aspects. The input image is enlarged by 1 / cos β in the direction of the plane inclination direction γ determined above, and rotated by the rotation angle α determined above so that the camera angles of the input image and the standard image coincide with each other. I try to let them. Therefore, the input image and the standard image can be accurately compared.

  In addition, the facility monitoring method according to claim 7 creates a difference image between the input image and the standard image in which the camera angle is matched by the image processing method according to claim 6 with the monitoring target facility as a photographing target. The progress of rust and cracks in the monitored equipment is detected. Therefore, it is possible to accurately detect the minute progress of cracks and rust from the difference information of the image by creating a difference image between the past image of the monitoring target facility and the newly captured image of the monitoring target facility. it can.

  Further, the surveying method according to claim 8 shoots each of the measurement areas that can be regarded as the same plane by two imaging means, and the camera angle change detection method according to any one of claims 1 to 5, A rotation angle α, a plane inclination angle β, and a plane inclination direction γ between the two captured images are obtained, and the obtained rotation angle α and the plane are obtained by triangulation using the two imaging units and the measurement region as vertices of a triangle. An inclination angle β and a plane inclination direction γ are used. In this case, the obtained α, β, and γ are parameters for correcting the physical position of the imaging unit with respect to the measurement region, or parameters for executing information processing for correcting an image captured by the imaging unit, or the measurement region. It can be used as a parameter for measuring the position of the triangulation with high accuracy.

  Further, the image processing method according to claim 9 uses a standard image prepared in advance and an input image obtained by photographing the same subject to be photographed by an imaging unit under the same camera angle condition as the standard image, By determining the rotation angle α, the plane tilt angle β, and the plane tilt direction γ between the standard image and the input image by the camera angle change detection method according to claim 1, Image distortion caused by the imaging means is detected. In this case, if the obtained α, β, and γ are not 0, it can be determined that the input image is distorted due to, for example, the camera lens. When image distortion is detected, a warning or error can be notified to the user, for example, in order to promote replacement of parts of the imaging means.

  According to a tenth aspect of the present invention, in the image processing method according to the ninth aspect, the distortion of the detected image is corrected by using the rotation angle α, the plane tilt angle β, and the plane tilt direction γ determined as described above. I am doing so. In this case, for example, a high-performance image can be obtained even with a general-purpose imaging means including an inexpensive lens having a large lens aberration.

  The stereo camera setting method according to claim 11 is a rotation angle α between two images taken by two image pickup means by the camera angle change detection method according to any one of claims 1 to 5. The plane inclination angle β and the plane inclination direction γ are obtained, and the obtained rotation angle α, plane inclination angle β, and plane inclination direction γ are used for the setting for causing the two imaging units to function as a stereo camera. ing. In this case, for example, if two inexpensive general-purpose digital cameras are prepared, these two cameras can be made to function as a stereo camera which has conventionally been an expensive dedicated camera.

  Thus, according to the detection method of the camera angle change according to any one of claims 1 to 5, the apparatus according to claim 12, and the program according to claim 13, an artificial marker is obtained by obtaining an enlargement / reduction ratio of an image for each direction. The relative camera angle difference between the two images can be detected easily and with high accuracy even if the inclination angle of the image is very small.

  Furthermore, according to the image processing method of the sixth aspect, the input image and the standard image can be accurately compared by correcting the input image and the standard image so that the camera angles match.

  Furthermore, according to the image processing method according to claim 7, a minute progress state of cracks and rust is created by creating a difference image between the past image of the monitoring target facility and the newly captured image of the monitoring target facility. Can be accurately detected from the difference information of the image.

  Furthermore, according to the surveying method of claim 8, accurate surveying can be performed by using the obtained α, β, γ as parameters for performing the triangulation.

  Furthermore, according to the image processing method of the ninth aspect, it is possible to detect camera lens distortion and the like, and to notify the user of a warning or an error in order to prompt parts replacement or the like.

  Furthermore, according to the image processing method of claim 10, since the distortion of the detected image is corrected, a high-performance image can be obtained even with a general-purpose imaging means equipped with an inexpensive lens having a large lens aberration. Can do.

  Furthermore, according to the stereo camera setting method of claim 11, for example, if two inexpensive general-purpose digital cameras are prepared, these two cameras function as a stereo camera which has conventionally been an expensive dedicated camera. Can be made.

  Hereinafter, the configuration of the present invention will be described in detail based on embodiments shown in the drawings.

  1 to 14 show an embodiment of a camera angle change detection method and apparatus, a program, and an equipment monitoring method using the same according to the present invention. This detection method of the camera angle change is to detect a difference in relative camera angle between the standard image and the input image by using the standard image and the input image obtained by photographing the area that can be regarded as the same plane of the photographing target, The plane area of the shooting target in the standard image is the standard plane, the plane area of the shooting target in the input image is the input plane, the plane tilt direction γ of the input plane with respect to the standard plane, the plane tilt direction γ and the plane of the standard image And determining a plane inclination angle β of the input plane with respect to the standard plane around an axis orthogonal to each other, and determining a rotation angle α about the axis orthogonal to the plane of the standard image of the input plane with respect to the standard plane. Yes.

  Then, in the step of obtaining the plane inclination direction γ and the plane inclination angle β, fan-shaped areas in a specific direction selected around a fixed point are extracted from the standard image and the input image, respectively, and the correlation between the two fan-shaped areas extracted is obtained. Thus, the radial deviation amount of the fan-shaped area extracted from the input image with respect to the fan-shaped area extracted from the standard image is obtained, and the specific direction of the fan-shaped area is sequentially changed to obtain the radial deviation amount in all directions. Obtain the enlargement / reduction ratio of the input image with respect to the standard image for each direction from the amount of deviation in the radial direction, obtain the plane inclination angle β from the ratio of the maximum value and the minimum value of the enlargement / reduction ratio, and incline the direction where the enlargement / reduction ratio is minimum The direction γ is set.

  The standard image and the input image correspond to, for example, images obtained by photographing the same subject at different camera angles at the same time, or images obtained by photographing the same subject at different times. Even if the subject is the same subject, if the subject is photographed at a different time, the subject may change slightly, and it may occur that the subject areas in the standard image and the input image are not exactly the same. It suffices if it can be regarded as almost the same region. For example, if the object to be photographed is a concrete wall, cracks may occur over time, and it may be difficult to photograph the exact same area, but the standard image and the input image are exactly the same. It is not always necessary to photograph the area, and it is sufficient that an area that can be regarded as almost the same is photographed. In some cases, different objects having the same model (the same appearance such as the outer shape and pattern) may be regarded as the same subject. In addition, the object to be imaged does not necessarily have to be a complete plane, and includes a case where it can be regarded as a substantially plane. For example, when an object at a very long distance is an object to be photographed, conversely, when the object to be photographed is very enlarged, or when the relative camera angle change between the standard image and the input image is extremely small In this case, even if the surface of the object to be imaged is a curved surface or uneven, the object to be imaged can be regarded as being almost flat.

  For example, the imaging target in the present embodiment is a flat concrete wall surface that is a monitoring target facility. The monitoring image of the concrete wall is taken for every periodic inspection by a patrolman using an imaging means such as a digital camera, for example. For example, an image taken in the past inspection is set as the standard image, and an image newly taken in the current check is set as the input image. Note that the standard image and the input image are, for example, grayscale images (grayscale images) in the present embodiment. When the standard image or the input image is a color image, the color image is converted into a grayscale image (grayscale image). Here, as a method of converting a color image into a grayscale image, for example, each value of RGB values is R, G, B, and a grayscale luminance value is Y. Y = (0.298912 * R + 0.586611) * G + 0.114478 * B) is generally used. There is also a method of setting Y = (2 * R + 4 * G + B) / 7 to increase the processing speed. Further, there is a method for simplifying the calculation by using only the G value of the RGB value. The grayscale image is generated by the existing or new grayscale image generation method as described above. It is desirable that the grayscale image generation method to be adopted is such that the difference in the grayscale of the image is as large as possible. Further, an ultraviolet image or an infrared image can be used as a standard image or an input image by converting it into a grayscale image.

  Here, for example, in the present embodiment, the power spectrum is calculated for each of the standard image and the input image, and the above-described radial shift amount for each direction is obtained for each power spectrum with the DC component of the power spectrum as the center. I have to. The power spectrum has an invariant property with respect to the translation of the original image. Therefore, by using the power spectra respectively calculated for the standard image and the input image, it is possible to cancel the vertical / horizontal misalignment between the standard image and the input image. In addition, by selecting the DC component of the power spectrum as the origin on the spatial frequency and selecting the direction of the sector area around the DC component (origin on the spatial frequency), the direction of the sector area is selected for the standard image and the input image. This saves you the trouble of setting the center point.

  Further, for example, in the present embodiment, log polar conversion is performed for each power spectrum of the standard image and the input image, and a rectangular area equivalent to the sector area is extracted from the power spectrum after the log polar conversion, and these two extracted rectangles are extracted. By calculating the phase-only correlation of the region, the above-described radial shift amount is obtained. Log polar conversion (log-polar conversion) is a conversion that models the property that the resolution in the vicinity of the center of the human retina is high and the resolution decreases as the distance from the center increases. By using this log polar conversion, the enlargement / reduction ratio can be easily obtained. That is, when the radius that is enlarged or reduced with respect to the original radius R is sR, s is the enlargement / reduction ratio. The logarithm of the radius R is log (R), and the logarithm of the radius sR is log (sR) = log (s) + log (R). The amount of deviation in the radial direction after log polar conversion is obtained by log (s), and the enlargement / reduction ratio s can be easily obtained based on log (s). The logarithm is generally a natural logarithm, but the base of the logarithm may be changed.

  Further, for example, in the present embodiment, the sector shape extracted from the input image for the sector region extracted from the standard image is obtained by calculating the correlation between two sector regions (rectangular regions after log polar conversion) extracted from the standard image and the input image, respectively. The amount of deviation in the arc direction of the region is obtained, and the rotation angle α is obtained based on the amount of deviation in the arc direction.

  Hereinafter, the principle of calculating the plane tilt direction γ, the plane tilt angle β, and the rotation angle α will be described in detail. A plane to be imaged, which is a subject in a three-dimensional space, is projected onto a two-dimensional image projection plane through a camera that is an imaging unit. In order to briefly describe the change in the relative camera angle between the standard image and the input image, that is, the change in the positional relationship between the subject to be photographed and the camera, here, the camera is fixed and the subject to be photographed (plane) is I will explain it as moving.

  As an input image, the same texture plane as that of the standard image is tilted by an angle β in the γ direction, and then the image is obtained by rotating the angle α. Considering the power spectrum of such an image, the power spectrum rotates by α in the same direction when the texture plane rotates α due to the nature of the power spectrum. When the texture plane is inclined in the γ direction by an angle β, the image is reduced by cos β in the γ direction, and the power spectrum is enlarged by 1 / cos β. Therefore, by utilizing these power spectrum properties, it is possible to detect the rotation angle α of the texture plane in the camera optical axis direction, the plane tilt angle β, and the plane tilt direction γ. At this time, the axial direction of the inclination of the plane is γ + π / 2.

Further details will be described. A plane in the three-dimensional space is projected onto a two-dimensional image projection plane through a camera. Here, a pinhole camera having a focal length f is considered as an image generation model. As shown in FIG. 2, the standard image is an image obtained by arranging the texture plane on the optical axis separated from the pinhole by a distance d and being parallel to the image plane. Here, _assuming that the coordinate system of the texture plane is (x _s , y _s , z _s ), the coordinate system of the image projection plane is (x _i , y _i , z _i ), and the optical axes are z _s and z _i , the texture The relational expression of Formula 1 is established between the density value T (x _s , y _s ) of the plane and the density value F _s (x _i , y _i ) of the standard image.

here,

Here, as an input image, then the same texture plane as standard image tilted x _s -y _s angle to the angle γ direction relative x _s-axis in the plane beta, taken while rotating angle α around the optical axis Image. Furthermore, it is assumed that the distance from the pinhole is slightly changed to d ′. Assuming that the coordinate system of the texture plane after this rotation is (x _t , y _t , z _t ), the vertical axis z _t is an angle β with respect to the optical axis (z _s ), as shown in FIG. Take. Also, when projecting the axis z _t to the image plane, the angle of the projection line and the x _i axis is alpha + gamma. At this time, the relational expression of Expression 3 is established between the density value T (x _t , y _t ) of the texture plane and the density value F _i (x ′ _i , y ′ _i ) of the input image.

here,
Or
However,

  Expressions 9 and 10 are expressions indicating the perspective generated when the plane is inclined. The central projection includes terms represented by these formulas, but the weak central projection does not include terms represented by these formulas. Here, the central projection is a projection when a pinhole camera is assumed, and is a non-linear function, so that the calculation is generally complicated. For this reason, linear approximation is desirable. Linear approximation can be performed when a specific condition is satisfied, such as when the size of the target object is sufficiently small compared to the distance from the camera to the object. The zero-order approximation of the central projection is the weak central projection. Specifically, when shooting with a telephoto lens having a focal length f longer than the shooting range, or when the image is divided into small areas, an image with a narrow shooting range compared to the distance to the object, or the inclination angle of the plane This corresponds to a case where an image with a very small change is weak center projection. In weak center projection, the direction of orthographic projection is parallel to the optical axis, and as a result, the image does not change with distance from the optical axis.

In actual image processing, since an image captured by a camera is used, the coordinate system (x _i , y _i ) in the continuous space is sampled in the sampling space T to be a coordinate system (n, m) of the digital image. Here, the relational expressions 11 and 12 are established between the coordinate system (n, m) of the digital image and the coordinate system (x _i , y _i ) of the continuous space.
<Equation 11>
n = x _i / T
<Equation 12>
m = y _i / T

Further, the following relational expression is established from Expression 1 and Expression 3.

Here, the assumption of omitting H ′ in Expression 15 is introduced, and the case of weak center projection is considered. The power spectrum of the standard image is S _s (u, v), the power spectrum of the input image is S _i (u, v), and u, v (−π ≦ u, v ≦ π) are in the n and m directions, respectively. Assuming the spatial frequency, the following relational expression is established.
here,

  Due to the nature of the power spectrum, when the texture plane rotates α, the power spectrum also rotates α in the same direction. When the texture plane is inclined by β in the γ direction, the image is reduced by (d cos β) / d ′ in the γ direction, and the power spectrum is enlarged by d ′ / (d cos β). Therefore, by utilizing these properties of the power spectrum, the rotation angle α of the texture plane in the camera optical axis direction, the plane tilt angle β, and the plane tilt direction γ can be detected. In other words, the plane inclination angle β and the inclination direction γ can be detected by obtaining the magnification direction and magnification ratio of the power spectrum in the frequency space. At this time, the axial direction of the inclination of the plane is γ + π / 2. Further, when the rotation angle of the power spectrum in the frequency space is obtained, the rotation angle α of the plane can be detected.

FIG. 1 is a flowchart showing an example of a procedure for calculating the plane tilt direction γ, the plane tilt angle β, and the rotation angle α. First, a power spectrum is obtained for each of the standard image and the input image (S1). Next, log-polar conversion centering on the origin is performed for each power spectrum (S2). Here, the origin indicates the direct current component of the power spectrum, that is, the origin on the spatial frequency. Next, for each power spectrum after log-polar conversion, a sector area having an appropriate area size at a specific direction angle is selected (S3). Since it is after log-polar conversion, the sector area is a rectangular area having two sides perpendicular to the radius (r) direction and the angle (θ) direction. Here, the radius of the sector region is, for example, half the number of quantizations in this embodiment. Thereby, the largest circle can be created in the spatial frequency. However, it is not always necessary to take the above value as the radius of the sector area. Next, each of the rectangular regions described above selected to calculate the phase-only correlation, to calculate the displacement amount d _a fan radial displacement amount d _r and sector arc direction (angular direction) (S4). Further, the direction angle of the sector area is changed, and the selection of the sector area (rectangular area after log-polar conversion) (S3) and the calculation of the phase-only correlation (S4) are repeated. Specifically, the central angle of the sector area is fixed, the direction angle of the sector area is increased from 0 ° by a certain angle, and S3 and S4 are repeated until the direction angle reaches 360 °. The deviation d _a in the sector arc direction (angular direction) corresponds to the rotation angle α. Therefore, it is possible to determine the rotation angle α by obtaining the shift amount d _a (S5). On the other hand, the deviation d _r sector radial direction corresponds to the scaling factor s image. The plane inclination angle β can be obtained from the maximum value s _max and the minimum value s _{min of the} enlargement / reduction ratio s (S6). In addition, the direction angle of the sector area where the image enlargement / reduction ratio s has the minimum value corresponds to γ. Therefore, the plane inclination direction γ can be obtained by obtaining the direction angle of the fan-shaped region where the enlargement / reduction ratio s has the minimum value (S7). When the texture plane is inclined by β in the γ direction, the image is reduced by (d cos β) / d ′ in the γ direction, and the power spectrum is enlarged by d ′ / (d cos β). Therefore, when the “power spectrum scaling ratio” is determined instead of the “image scaling ratio”, the direction angle of the fan-shaped area where the “power spectrum scaling ratio” is the maximum value corresponds to γ. Become.

The contents of the process in FIG. 1 will be described in more detail. The log-polar conversion (S2) for the power spectrum is performed according to the following equations 19 and 20.

After the selection of the direction component to mask the sector region to the power spectrum (S3), by the phase-only correlation calculation, obtaining the k _r direction shift amount d _r and k _a direction of the deviation amount d _a (S4). Furthermore, the sector region serving as a mask is changed little by little, we obtain the direction-of k _r direction displacement amount d _r and k _a direction of the deviation amount d _a from the power spectrum center.

Rotation angle α can be determined by Equation 21 below the k _a direction of the deviation amount d _a. However, N _a is the number of quantized angular direction when log-polar transform.

Scaling factor s image can be obtained by equation 22 it follows from k _r direction shift amount d _r. Here, _Nr is the quantization number in the radial direction at the time of log-polar conversion.

The graph of FIG. 4 shows the image enlargement / reduction ratio s thus obtained on the vertical axis and the direction angle in the frequency domain on the horizontal axis. As shown in FIG. 4, the enlargement / reduction ratio s is a periodic function having _a period N _a / 2. In this graph, the direction angle of the sector area where the enlargement / reduction ratio s is minimum is γ. Further, the ratio of the magnification s _max when the magnification is the largest and the magnification s _min when the magnification is the smallest corresponds to cos β. Therefore, the calculation formula for obtaining the plane inclination angle β is the following Formula 23.

Furthermore, the use of coordinates d _rmin when the most reduced with the coordinates d _rmax when it is most expanded, equations 23 becomes Equation 24.

Here, k _a direction of the deviation amount d _a is because it is calculated for each selected direction, it can be rotational angle α is calculated for each direction. In this case, the rotation angle α is determined as follows. For example, an average value of the rotation angles α calculated for each direction is obtained, and this average value is set as the final rotation angle α. Alternatively, the rotation angle α in the direction in which the enlargement / reduction ratio is maximum is employed. Alternatively, the rotation angle α in the direction where the enlargement / reduction ratio is closest to 1 is adopted. Alternatively, the rotation angle α in the direction in which the correlation value at the time of correlation calculation for each direction is maximized is adopted.

In the above example, there has been an arc direction displacement amount d _a fan-shaped area to determine the rotation angle alpha, not necessarily limited to this example, for example, first determine the plane tilt direction γ and the planar inclination angle beta, the After correcting the input image using the obtained γ and β, the rotation angle α may be obtained using the corrected input image and the standard image. Specifically, the input image is 1 / cos using the obtained plane inclination angle β in the direction of the obtained plane inclination direction γ around the arbitrary point in the image, for example, the center point of the image. The primary correction image is enlarged by β times, the power spectrum of the primary correction image and the standard image is calculated, polar coordinate conversion is performed, rotational deviation is obtained by phase-only correlation, and α is calculated from the obtained rotational deviation amount. .

The procedure for obtaining the rotation angle α by calculating the correlation between the primary correction image and the standard image is shown in the flowchart of FIG. First, the standard image and the primary correction image are each subjected to two-dimensional Fourier transform (S101A, S101B). Then, with respect to the amplitude component (power spectrum) of the two-dimensional signal after Fourier transform, polar coordinate transformation is performed around the origin (DC component) of the frequency component (S102A, S103A, S102B, S103B). This processing is performed for each of the standard image (signal) and the primary corrected image (signal). Then, Fourier transformation is performed again on the two-dimensional signal (only the amplitude component) converted into the respective polar coordinates (S104A, S104B). Thereafter, for each frequency component after Fourier transform, the conjugate complex component of the signal created from the standard image and the complex component of the signal created from the primary correction image are respectively multiplied and synthesized (S105). When synthesis is performed, division is performed by the amplitude of each signal (a phase difference is calculated using only phase components). There are cases where the amplitude component becomes too small when performing this synthesis calculation, specifically, division. In such a case, the real component of the combined signal = 0 and the complex component = 0 may be used. Also, a random number may be assigned to the combined signal component when the amplitude component is too small. After the synthesis calculation, the inverse Fourier transform of the signal component is calculated (S106). A coordinate (r ₁ , θ ₁ ) that maximizes the correlation value is searched for in the converted signal (S107). Since there is a difference in the rotation angle between the standard image and the primary correction image by an angle corresponding to θ ₁ , the searched θ ₁ becomes the rotation angle α. In addition, when Fourier transform is performed on the standard image and the primary correction image, image processing such as an appropriate window function can be applied to the standard image and the primary correction image so that the angle deviation can be measured with high accuracy.

Furthermore, in order to improve the accuracy of the tilt detection angle, highly accurate misalignment detection described below may be performed. That is, it is possible to obtain the peak coordinates of r (n ₁ , n ₂ ) with high accuracy by numerically fitting the following Equation 25 to the actually calculated phase-only correlation equation.

The principle of the high-accuracy misregistration detection method will be described below. First, consider an image (two-dimensional signal) f _c (x ₁ , y ₁ ), x ₁ , y ₁ ∈R defined in a continuous space. _Let δ _x and δ _{y be} real numbers representing the amount of minute movement in the x ₁ and y ₁ directions, respectively. An image obtained by minutely moving f _c (x ₁ , y ₁ ) by (δ _x , δ _y ) in continuous space is f _c (x ₁ −δ _x , y ₁ −δ _y ). A discrete space image (two-dimensional signal) obtained by sampling these images f _c (x ₁ , y ₁ ) and f _c (x ₁ −δ _x , y ₁ −δ _y ) in the sampling spaces T ₁ and T _2. These are defined as f (n ₁ , n ₂ ) and g (n ₁ , n ₂ ), respectively, and are defined by the following formulas 26 and 27.

For the phase-only correlation for the images f (n ₁ , n ₂ ) and g (n ₁ , n ₂ ) defined in the discrete space, the small movement amounts δ _x and δ _y in the continuous space are estimated. At this time, in general, δ _x and δ _y can be considered to correspond to a movement amount of one pixel or less in a discrete space.

First, the two-dimensional DFT (discrete Fourier transform) of the images f (n ₁ , n ₂ ) and g (n ₁ , n ₂ ) are respectively expressed as F (k ₁ , k ₂ ) and G (k ₁ , k ₂ ). Then, the following approximation is established. However, N _{1 in} Expression 28 is “the number of quantization in the n ₁ direction in the image”, and N ₂ is “the number of quantization in the n ₂ direction in the image”.

The above equation 28 is approximate because of the difference between the continuous space and the discrete space. Specifically, the two-dimensional DFT assumes periodicity with respect to a discrete space. _{F (k 1, k 2)} , G (k 1, k 2) of the phase-only synthesis _{R (k} 1, _{k 2)} is given by the following equation 29, the two-dimensional IDFT (Inverse Discrete Fourier Transform) is This can be expressed as Equation 25.

Equation 25 shows the general form of the phase-only correlation function for the same image when there are small movement amounts δ _x and δ _y . Therefore, it is possible to obtain the peak coordinates of r (n ₁ , n ₂ ) with high accuracy by numerically fitting Equation 25 to the actually calculated phase-only correlation equation.

  For example, in the present embodiment, the input image is enlarged by 1 / cos β times using the obtained plane inclination angle β in the direction of the obtained plane inclination direction γ around the arbitrary point in the image, In addition, the camera angle of the input image and the standard image is made to coincide with each other by rotating by the obtained rotation angle α. Furthermore, in the equipment monitoring method of the present embodiment, a difference image between the input image and the standard image that match the camera angles is created, and the development of rust and cracks in the equipment to be monitored (especially in this embodiment, the monitoring imaging target is Since it is a concrete wall surface, the crack progress is detected.

Actually, there is also an effect of enlarging / reducing the image that occurs when the camera zoom or the camera position approaches or moves away from the object to be photographed. Therefore, the magnification s _max when the entire image is most enlarged is used. 1 / s _max reduction (enlargement in some cases) is performed, and thereafter, an enlargement of 1 / cos β times is performed in the direction of the plane inclination direction γ determined above about an arbitrary point in the image. good. In addition, as the interpolation method for determining the luminance value of the pixel to be added when enlarging the image, the nearest neighbor method (nearest neighbor method), the linear interpolation method (bilinear method), the cubic convolution method (bicubic method), etc. The existing method or new method may be used as appropriate.

Further, an image after correcting the camera angle to match the input image using α, β, and γ is used as a secondary correction image, and the vertical and horizontal positions between the secondary correction image and the standard image are further determined. The deviation may be corrected. The processing procedure in this case is shown in the flowchart of FIG. First, two-dimensional Fourier transform is performed on the standard image and the secondary correction image (S201A, S201B). Then, for each frequency component after Fourier transform, the conjugate complex component of the signal created from the standard image and the complex component of the signal created from the secondary correction image are respectively multiplied and synthesized (S202). When combining, division is performed by the amplitude of each signal. Then, inverse Fourier transform is performed on the synthesized signal (S203). Coordinates (x ₁ , y ₁ ) having the maximum correlation value are searched for in the converted signal wave (S204). Second supplementary image x ₁ pixel in the x-axis direction compared with the standard image, considered are shifted by y ₁ pixel in the y-axis direction. Therefore, it is possible to correct the vertical / horizontal misalignment between the secondary corrected image and the standard image using the searched x ₁ and y ₁ .

  The difference image between the input image corrected as described above and the standard image is, for example, a grayscale image (grayscale image) displayed whiter as the difference is larger and black as the difference is smaller. In the difference image, a portion displayed in white (a portion having a large difference) is considered to be a portion where cracks have progressed on the concrete wall surface to be monitored and photographed.

  The camera angle change detection method and facility monitoring method described above are implemented as an image processing apparatus shown in FIG. 7, for example. For example, as shown in FIG. 7, the image processing apparatus 1 includes an image capturing unit 2 that captures a standard image and an input image, each of which captures an area that can be regarded as the same plane as a capturing target, a plane tilt direction γ, and a plane tilt angle β. .Gamma..beta. Calculating means 3 and 4 for obtaining the angle .alpha. Calculating means 5 for obtaining the rotation angle .alpha.

  More specifically, for example, the image processing apparatus 1 according to the present embodiment performs power polar calculation for the power spectrum of the standard image and the input image and the log polar conversion for the power spectrum of the standard image and the input image. A log polar conversion unit 7; and a correlation calculation region extraction unit 8 for extracting a rectangular region corresponding to a sector region in a specific direction of the standard image and the input image for the power spectrum after the log polar conversion by sequentially changing the specific direction of the sector region; The phase-only correlation calculating means 9 for calculating the phase-specific correlation between the extracted rectangular area of the standard image and the rectangular area of the input image to obtain the radial direction deviation amount and the arc direction deviation amount, and the direction-specific radius Enlargement / reduction ratio calculating means for obtaining the enlargement / reduction ratio of the input image with respect to the standard image for each direction from the direction deviation amount 0, β calculating means 3 for determining the plane inclination angle β from the ratio of the maximum value and the minimum value of the enlargement / reduction ratio, γ determining means 4 for determining the direction in which the enlargement / reduction ratio becomes the minimum as the plane inclination direction γ, and the arc direction Α calculating means 5 for obtaining the rotation angle α based on the deviation amount. As described above, the α calculating means 5 enlarges the input image in the direction of the plane inclination direction γ determined above by 1 / cos β magnification using the calculated plane inclination angle β. By calculating the correlation between the primary correction image and the standard image as a corrected image, the rotational deviation amount of the primary correction image with respect to the standard image is obtained, and the obtained rotational deviation amount is used as the rotation angle α. Also good.

  Further, for example, the image processing apparatus according to the present embodiment enlarges the input image in the direction of the obtained plane inclination direction γ by 1 / cos β using the obtained plane inclination angle β and the obtained rotation angle. A camera angle correction unit 11 that matches the camera angle of the input image and the standard image by rotating by α, and a difference image output unit 12 that outputs a difference image between the input image and the standard image that match the camera angle. Yes.

  The image processing apparatus 1 described above is realized using, for example, an existing or new computer (computer). For example, as shown in FIG. 8, the computer 20 includes a central processing unit (CPU) 21, a storage device 22 such as a RAM, a ROM, and a hard disk, an output device 23 such as a display, and an interface for communicating with a digital camera 25. Hardware resources such as 24 are connected by a bus 26. When the image processing program according to the present invention is executed on the computer 20, the computer 20 functions as the image processing apparatus 1 described above. The interface 24 of the computer 20 functions as the image capturing unit 2, and an image recorded in the digital camera 25 is captured via the interface 24 into the storage device 22 of the computer 20. The storage device 22 stores standard images and input images, and work data and programs necessary for executing image processing. By the calculation function and the control function for hardware resources provided in the CPU 21, the above-described power spectrum calculation means 6, log polar conversion means 7, correlation calculation area extraction means 8, phase-only correlation calculation means 9, enlargement / reduction ratio calculation means 10, β calculation Means 3, γ determination means 4, α calculation means 5, camera angle correction means 11, and difference image output means 12 are realized. The differential image automatically created by the computer 20 is displayed on a display as the output device 23, for example. It should be noted that the computer 20 automatically detects the progress of cracks and rust of the monitoring target based on the created difference image. For example, when there are a certain number of pixels whose difference value is equal to or greater than a threshold value, monitoring is performed. It may be determined that the development of cracks and rust of the imaging target is remarkable, and an error message or warning sound is issued to alert the administrator or the like.

  For example, reference numeral 30 in FIG. 14 shows an example of a standard image, reference numeral 31 shows an example of an input image, reference numeral 32 shows an input image corrected so that the camera angle matches the standard image, and reference numeral 33 is corrected. A difference image between the input image 32 and the standard image 30 is shown. According to the difference image 33, unlike the difference image between the input image and the standard image created without correcting the camera angles to match each other as shown in FIG. 15, the progress of cracks to be monitored is accurately extracted. Can do.

  According to the present invention configured as described above, since the minute rotation and expansion / contraction are generated in the frequency spectrum of the photographed image due to the change in the three-dimensional camera angle, the change in the frequency spectrum is detected. The camera angle change can be detected with high accuracy, and based on the detected camera angle change, the camera angle of the newly taken image (input image) and the image taken in the past (standard image) are corrected to match. be able to. Further, in the present invention, since the frequency spectrum of the entire image is used, the present invention can be applied to an object having poor surface characteristics, such as a concrete surface of a dam, for example, compared to a method of extracting and correcting image characteristics.

<Application to electron microscope image>
Images were taken with an electron microscope with different plane tilts. The inclination angles of the plane were 0 °, 2.5 °, 5 °, 7.5 °, and 10 °. In these images, the stage was moved so that almost the same area was photographed, and at the same time, the focus was manually adjusted. In this way, electron microscope images in which substantially the same area was photographed were prepared, and the present invention was applied to these images. A partial image of 512 × 512 pixels including almost the same region was cut out from the photographed image and used. In the verification, an image having an inclination of 0 ° was used as a standard image, and the remaining image was used as an input image, and the present invention was applied to determine the inclination angle of the plane. As a result, the error was within 0.6 °, the average error was 0.40 °, and the maximum error was 0.55 °.

<Application to cork board images>
A cork board standing on a turntable was photographed with a camera fixed on a tripod, and its inclination was detected. In this case, the rotation angle of the turntable is the inclination angle of the plane to be obtained. The digital camera was fixed with a tripod so that the center of the cork board and the camera lens were at the same height. In such an environment, the orientation of the camera was manually adjusted by visual observation so that the center of the cork board appears in the vicinity of the center of the image. The turntable was moved by 0.5 ° from + 15.5 ° to −15.5 °, and a total of 63 images were taken. A partial image including substantially the same region was cut out from this image. The present invention was applied with an image taken from the front as a standard image and the remaining images as input images. As a result, the average error was 1.44 ° and the maximum error was 3.36 °.

<Application to concrete wall image>
I photographed the concrete wall of the building with a digital camera. Assuming a general patrol, the camera was fixed by hand without using a tripod, and two images were taken at different times (more precisely, about 5 minutes). A partial image of 256 × 256 pixels including substantially the same region was cut out from the photographed image. A standard image (first image) created in this way is shown in FIG. 9, and an input image (second image) is shown in FIG. FIG. 11 is a graph in which the present invention is applied to these images, and the enlargement / reduction ratio, which is the calculation result, is shown on the vertical axis and the direction angle in the frequency domain is shown on the horizontal axis.

  From the graph of FIG. 11, it was found that the input image was 0.966 times larger than the standard image, and was tilted by 9.22 ° about the x-axis direction of the image. The input image was corrected to have the same camera angle as the standard image, and the difference from the standard image was calculated. FIG. 12 shows a difference image between the input image subjected to the camera angle correction and the standard image. Note that FIG. 12 shows an image displaying the absolute value of the luminance difference between the two images so that the difference at the time of superimposition becomes clear.

  On the other hand, for comparison, FIG. 13 shows a difference image between an input image and a standard image to which only the translational correction of the image is applied without correcting the camera angle. In the difference image of FIG. 12 and FIG. 13, the part where the difference of the corresponding pixel of both images is large is drawn more white. In FIG. 13, the difference between the camera angles is drawn white as the difference between the pixel values. However, in FIG. 12, no drawing showing such a difference is seen, and it can be seen that the correction for matching the camera angles was accurately performed. . Obviously, the image correction according to the present invention is effective.

  In the present embodiment, the difference between the shooting time of the input image and the standard image is about 5 minutes, and no progress of cracks was seen in the concrete, so there was no difference in the difference image. According to the present invention, for example, as shown in FIG. 14, the portion where the crack has progressed is clearly displayed.

  The above-described embodiment is an example of a preferred embodiment of the present invention, but is not limited thereto, and various modifications can be made without departing from the gist of the present invention. For example, in the above-described embodiment, the camera angle change detection method / device / program according to the present invention is used for monitoring a facility to be monitored such as a concrete wall throat. However, the present invention is applicable to triangulation, stereo camera setting, camera lens. The present invention can also be applied to the detection and correction of distortion such as.

  For example, measurement areas that can be regarded as the same plane are photographed by two imaging means (for example, digital cameras), respectively, and the rotation angle α, the plane tilt angle β, and the plane between the two captured images are detected by the above-described camera angle change detection method. The inclination direction γ is obtained, and the obtained rotation angle α, plane inclination angle β, and plane inclination direction γ may be used for triangulation with the two imaging means and the measurement region as the vertices of a triangle. The obtained α, β, and γ are parameters for correcting the physical position of the imaging unit with respect to the measurement region, parameters for executing information processing for correcting an image captured by the imaging unit, or the position of the measurement region. It can be used as a parameter for measurement. That is, by using the obtained α, β, and γ, the direction and position of the imaging means with respect to the measurement area are corrected, or the correspondence between the two images taken by the two imaging means is accurately taken. Triangulation can be performed with high accuracy. When estimating the distance to an object with high accuracy by using the images taken at different camera angles by the principle of triangulation, it is possible to perform high-accuracy measurement by associating the images with high accuracy. By using the present invention to calculate the difference in camera angle and to search for corresponding points between images in consideration of the effect of the difference in camera angle, more accurate processing can be expected. Further, an image in a state in which the measurement area serving as the reference plane is viewed from the front is obtained by, for example, reading the surface of the measurement area with a scanner or the like, and this image is set as a standard image. Then, by using this standard image and using the image obtained by the imaging means as an input image and adjusting the position of the imaging means so that α = β = γ = 0, the imaging means is installed in front of the measurement region. be able to. Furthermore, if one imaging means is installed in front of the measurement area, and another imaging means is installed in a direction perpendicular to the optical axis connecting the imaging means and the measurement area, one imaging means and the measurement area And the angle formed by the optical axis connecting the other imaging means and the measurement region (in other words, the amount of deviation of the optical axes of the two imaging means with respect to the measurement region) and the two imaging means From the interval, the position of the measurement area (the distance between the imaging means installed in front of the measurement area and the measurement area) can be accurately obtained.

  In addition, the standard image prepared in advance and the input image obtained by photographing the same subject to be photographed with the imaging means under the same camera angle conditions as the standard image, the standard method is used to detect the change in camera angle. The image distortion caused by the imaging means may be detected by obtaining the rotation angle α, the plane tilt angle β, and the plane tilt direction γ between the image and the input image. For example, an image in a state in which the reference plane is viewed from the front is obtained by, for example, reading the reference plane with a scanner or the like, and this image is set as a standard image. On the other hand, an imaging unit is arranged in front of the reference plane using a jig or the like, and an image obtained by photographing the reference plane from the front is used as an input image. When α, β, and γ calculated between the standard image and the input image are not 0, it can be determined that the input image is distorted due to, for example, camera lens distortion. . When image distortion is detected, for example, a warning or error may be output in order to prompt replacement of parts of the imaging means. In the above description, the reference plane is used. However, the plane is not particularly required if appropriate digital data can be created.

  Furthermore, the distortion of the image detected as described above may be corrected using the obtained rotation angle α, plane tilt angle β, and plane tilt direction γ. For example, with respect to a normal lens, the lens incorporated in the imaging unit recognizes that distortion represented by α, β, and γ has occurred, and the image captured by the imaging unit A correction that enlarges 1 / cos β times by using the obtained plane inclination angle β in the direction of the obtained plane inclination direction γ around the arbitrary point in the above and rotates by the obtained rotation angle α. You may make it do. In this case, a high-performance image can be obtained even with an imaging means equipped with an inexpensive lens.

  Further, the rotation angle α, the plane tilt angle β, and the plane tilt direction γ between the two images taken by the two image pickup means are obtained by the above-described camera angle change detection method, and the two image pickup means are used as a stereo camera. The rotation angle α, the plane tilt angle β, and the plane tilt direction γ determined above may be used for setting for functioning. In stereo measurement, the two cameras are separated from each other, and it is very difficult to accurately set the directions of the cameras. Therefore, before starting the measurement, α, β between the two cameras is used by using a standard plane (even if there is unevenness or an object that can be regarded as a plane when viewed locally). , Γ are obtained and used for setting the stereo camera. Further, correction may be made if necessary when photographing the object to be measured. Furthermore, when there is digital data that can be considered to be taken from the front (or a known angle) by reading a reference plane with a scanner or printing digital data in advance, the plane and each By obtaining α, β, and γ with the camera image, the camera can be installed in front of the plane. According to the present invention, for example, if two inexpensive general-purpose digital cameras are prepared, these two cameras can be made to function as a stereo camera which has conventionally been an expensive dedicated camera.

It is a flowchart which shows an example of the processing of the camera angle change detection method and apparatus of this invention, and a program. It is a conceptual diagram for demonstrating the principle of the detection method of the camera angle change of this invention. It is a conceptual diagram for demonstrating the principle of the detection method of the camera angle change of this invention. It is a graph which shows the enlargement / reduction rate of the image according to direction, a vertical axis | shaft shows the enlargement / reduction rate of an image, and a horizontal axis shows a direction angle. It is a flowchart which shows an example of the process which calculates | requires rotation angle (alpha), after calculating | requiring the plane inclination direction (gamma) and the plane inclination angle (beta). It is a flowchart which shows an example of the process which detects the position shift of the vertical and horizontal direction of the input image which correct | amended the standard image and the camera angle. It is a block diagram which shows the structural example of the detection apparatus of the camera angle change of this invention. It is a block diagram which shows the structural example of the computer by which the detection program of the camera angle change of this invention is mounted. It is a figure which shows an example of a standard image. It is a figure which shows an example of an input image. FIG. 11 is a graph showing an enlargement / reduction ratio of an image for each direction obtained from the standard image of FIG. 9 and the input image of FIG. 10, with the vertical axis showing the enlargement / reduction ratio of the image and the horizontal axis showing the direction angle. FIG. 10 shows a difference image between the input image of FIG. 10 corrected to match the camera angle according to the present invention and the standard image of FIG. 9. A difference image between the input image of FIG. 10 and the standard image of FIG. 9 is shown (correction for matching camera angles is not performed). It is a conceptual diagram which shows a mode that the progress of the crack of an installation can be correctly detected as a difference image by correct | amending which makes camera angles correspond. It is a conceptual diagram which shows a mode that the image information which is not changing appears as a difference when correction | amendment which makes a camera angle correspond is not performed, and the progress of the crack of an installation cannot be detected correctly.

Explanation of symbols

1 Image processing device (camera angle change detection device)
2 Image capture means 3 β calculation means 4 γ determination means 5 α calculation means 6 power spectrum calculation means 7 log polar conversion means 8 correlation calculation area extraction means 9 phase-only correlation calculation means 10 enlargement / reduction ratio calculation means 11 camera angle correction means 12 Difference image output means 20 Computer

Claims (13)

  A method for detecting a relative camera angle difference between the standard image and the input image using a standard image and an input image each of which captures an area that can be regarded as the same plane of the imaging target, the imaging in the standard image The plane area of the target is a standard plane, the plane area of the imaging target in the input image is the input plane, the plane tilt direction γ of the input plane with respect to the standard plane, the plane tilt direction γ, and the plane of the standard image Obtaining a plane tilt angle β of the input plane with respect to the standard plane about an axis orthogonal to the upper side, and obtaining a rotation angle α about an axis of the input plane with respect to the standard plane perpendicular to the plane of the standard image. Each of which extracts a fan-shaped area in a specific direction centered on a certain point from the standard image and the input image, and extracts the two fans By obtaining the correlation of the areas, the radial deviation amount of the fan-shaped area extracted from the input image with respect to the fan-shaped area extracted from the standard image is obtained, and the specific direction of the fan-shaped area is sequentially changed for the omnidirectional Obtaining a radial deviation amount, obtaining an enlargement / reduction ratio of the input image with respect to the standard image for each direction from the radial deviation amount for each direction, and determining the plane inclination from a ratio between a maximum value and a minimum value of the enlargement / reduction ratio A method for detecting a change in camera angle, characterized in that an angle β is obtained and a direction in which the enlargement / reduction ratio is minimum is set as the plane inclination direction γ.   2. The power spectrum is calculated for each of the standard image and the input image, and the radial shift amount for each direction around the DC component of the power spectrum is obtained for each of the power spectra. The detection method of the camera angle change as described.   For each power spectrum of the standard image and the input image, log polar conversion is performed, and for the power spectrum after log polar conversion, a rectangular area equivalent to the fan-shaped area is extracted, and two of the extracted rectangular areas are extracted. 3. The camera angle change detection method according to claim 2, wherein the radial shift amount is obtained by calculating a phase-only correlation.   By obtaining the correlation between the two fan-shaped areas extracted from the standard image and the input image, the arc-direction deviation amount of the fan-shaped area extracted from the input image with respect to the fan-shaped area extracted from the standard image is obtained, The method of detecting a camera angle change according to any one of claims 1 to 3, wherein the rotation angle α is obtained based on the deviation amount in the arc direction.   The input image is enlarged by 1 / cos β times using the obtained plane inclination angle β in the direction of the obtained plane inclination direction γ to obtain a primary correction image, and the primary correction image and the standard image 4. The rotation deviation amount of the primary correction image with respect to the standard image is obtained by obtaining the correlation of the standard image, and the obtained rotation deviation amount is set as the rotation angle α. 5. The detection method of the camera angle change described in 3.   6. The plane tilt direction obtained by using the rotation angle α, the plane tilt angle β, and the plane tilt direction γ determined by the camera angle change detection method according to claim 1. An image processing method comprising: enlarging 1 / cos β times in the direction of γ and rotating the obtained rotation angle α to match the camera angles of the input image and the standard image.   The monitoring target equipment is photographed, and a difference image between the standard image and the input image with the camera angle matched is created by the image processing method according to claim 6 to detect the progress of rust and cracks in the monitoring target equipment A facility monitoring method characterized by:   6. The measurement areas that can be regarded as the same plane are photographed by two imaging means, respectively, and the rotation angle α and the plane between the two photographed images are detected by the camera angle change detection method according to any one of claims 1 to 5. An inclination angle β and a plane inclination direction γ are obtained, and the obtained rotation angle α, plane inclination angle β, and plane inclination direction γ are used for triangulation with the two imaging means and the measurement region as the vertices of a triangle. Surveying method characterized by   6. The standard image prepared in advance and an input image obtained by photographing the same subject to be photographed by an imaging unit under the same camera angle condition as the standard image, according to any one of claims 1 to 5. By detecting a camera angle change detection method, a rotation angle α, a plane tilt angle β, and a plane tilt direction γ between the standard image and the input image are detected to detect image distortion caused by the imaging means. An image processing method.   10. The image processing method according to claim 9, wherein the detected distortion of the image is corrected by using the obtained rotation angle α, plane tilt angle β, and plane tilt direction γ.   A rotation angle α, a plane inclination angle β, and a plane inclination direction γ between two images photographed by two imaging means are obtained by the camera angle change detection method according to any one of claims 1 to 5, A setting method of a stereo camera, wherein the obtained rotation angle α, plane tilt angle β, and plane tilt direction γ are used for the setting for causing the two imaging means to function as a stereo camera.   An apparatus for detecting a difference in relative camera angle between the standard image and the input image using a standard image and an input image each of which captures an area that can be regarded as the same plane of the imaging target, and the imaging in the standard image The plane area of the target is a standard plane, the plane area of the imaging target in the input image is the input plane, the plane tilt direction γ of the input plane with respect to the standard plane, the plane tilt direction γ, and the plane of the standard image Means for determining a plane inclination angle β of the input plane with respect to the standard plane about an axis orthogonal above, and a rotation angle α about an axis orthogonal to the plane of the standard image of the input plane with respect to the standard plane Means for extracting a sectoral area in a specific direction selected from the standard image and the input image with a fixed point as a center, and extracting the two fan-shaped areas extracted from the standard image and the input image. By obtaining a function, a radial displacement amount of the sector area extracted from the input image with respect to the sector area extracted from the standard image is obtained, and the specific direction of the sector area is sequentially changed to change the radial direction for all directions. A displacement amount is obtained, an enlargement / reduction rate of the input image with respect to the standard image for each direction is obtained from the radial displacement amount for each direction, and the plane inclination angle β is calculated from a ratio between a maximum value and a minimum value of the enlargement / reduction rate. And a direction in which the enlargement / reduction ratio is minimized is defined as the plane inclination direction γ.   Using a standard image and an input image obtained by photographing areas that can be regarded as the same plane of the shooting target, the shooting target plane area in the standard image is set as a standard plane, and the shooting target plane area in the input image is input. As the plane, a plane inclination direction γ of the input plane with respect to the standard plane, and a plane inclination angle β of the input plane with respect to the standard plane around an axis orthogonal to the plane inclination direction γ on the plane of the standard image. A program for causing a computer to function as a means for obtaining and a means for obtaining a rotation angle α about an axis orthogonal to the plane of the standard image of the input plane with respect to the standard plane, and centering a fixed point from the standard image and the input image Extract from the standard image by extracting the fan-shaped areas in the specific direction selected as In addition, the radial displacement amount of the sector region extracted from the input image with respect to the sector region is obtained, the specific direction of the sector region is sequentially changed to obtain the radial displacement amount in all directions, and the radius for each direction is determined. The enlargement / reduction ratio of the input image with respect to the standard image for each direction is obtained from the amount of direction deviation, the plane inclination angle β is obtained from the ratio between the maximum value and the minimum value of the enlargement / reduction ratio, and the enlargement / reduction ratio is minimized. A camera angle change detection program that causes a computer to function as means for setting the direction to the plane tilt direction γ.