JP6568722B2 - Image processing system, image processing method, and program - Google Patents

Description

  The present invention relates to an image processing system, an image processing method, and a program.

  Attempts have been made to nondestructively inspect structures such as buildings and facilities (hereinafter referred to as buildings) using an image processing system. As a related technique, there is one that acquires the state of the surface of the facility based on an image of the facility (see, for example, Patent Document 1). In the technique of Patent Document 1, a surface of a steel material that forms a steel tower is detected from a captured image, and the position of a degraded portion in the surface is specified based on the position of a side of the detected surface.

Japanese Patent Laid-Open No. 11-132962

  However, when a building or the like is an object to be evaluated, a wall surface without a structure such as a window or a monotonous wide-area wall surface may be an object of inspection. It is difficult to extract a structure or the like that can specify the position in the surface from the image of the wall surface. In the technique of Patent Document 1, it is necessary that the imaged image includes a side of the surface of the steel material forming the steel tower, but the method of Patent Document 1 is used to detect the state of the wall surface of the building. Even if it is applied, it is difficult to detect the state at a position relatively far from the side.

  The present invention has been made in view of such circumstances. An image processing system, an image processing method, and an image processing system that capture an image of an object by an imaging unit and obtain an image capable of detecting the state of a detection target surface of the object. One purpose is to provide a program.

One aspect of the present invention for solving the above problems, on the surface of the object to be imaged by the imaging unit from different positions, the light projecting from the light emitting portion provided in not affected position the movement of the imaging unit One or a plurality of indexes based on the captured image, an extraction unit for extracting the one or more indexes and feature points of the surface of the target object from the image captured by the image capturing unit; and the target object of the imaging unit for changing the position in the different positions relative to the position of the object in along Migihitsuji the surface is captured from the different positions to obtain a plurality of images, the plurality of images the resulting A detection unit that detects relative positions of the plurality of images based on one or more of the extracted indices and feature points of the surface of the object, and the light projecting unit includes the imaging unit Formed independently of the light emitting portion, and An image processing system, characterized in that location is fixed relative to the position of the object.
Further, according to one aspect of the present invention, the surface of the object imaged by the imaging unit from different positions is 1 or by the image projected from the light projecting unit provided at a position not affected by the movement of the imaging unit. A plurality of indices are drawn, and from the image captured by the imaging section, an extraction unit that extracts the one or more indices drawn linearly in the image and feature points of the surface of the object; The imaging unit that changes the position along the surface of the object captures images from the different positions to obtain a plurality of images, and the one or more extracted from each of the obtained plurality of images A detection unit configured to detect a relative position of the plurality of images based on an index and a feature point of the surface of the object; and first data based on the feature point of the surface of the object; Geometric features of the one or more indicators drawn in a shape A second data is generated based, an image processing system and generating data for adjusting the image on the basis the first data into the second data.

  Further, according to one aspect of the present invention, based on the distance between the extracted index and the feature point of the surface of the object, or the size of the feature point of the extracted index and the surface of the object, The image processing apparatus includes a correction unit that adjusts one or a plurality of images so that the scales of the plurality of images match.

  One embodiment of the present invention includes a correction unit that corrects distortion of the image based on at least the extracted index.

In the aspect of the invention described above, the detection unit may extract the one or more indices that can adjust distortion of the captured image from the captured image, and the extracted one or more indices. The direction of the optical axis of the imaging unit with respect to the surface of the object is detected based on the above.

Another embodiment of the invention, the correcting unit based on the orientation of the detection result of the optical axis of the imaging unit with respect to the plane of the object, to correct the distortion of an image captured by the imaging unit Features.

Another embodiment of the present invention, the surface of the object to be imaged by the imaging unit from different positions, 1 by projected by image from the light projecting portion provided in not affected position the movement of the imaging unit or a plurality of indices are drawn, a step of extracting section extracts the feature points of the surface of the one or the object with a plurality of indicators from the image captured by the imaging unit, earthenware pots on a surface of the object As described above, the image capturing unit that changes the position to the position different from the position of the object captures images from the positions different from each other to obtain a plurality of images, and is extracted from each of the obtained plurality of images. and one or more indicators and on the basis of the feature points of the surface of the object, viewed including the steps of detecting the relative positions of the plurality of images, the light projecting unit, independently from said image pickup unit And the position of the light projecting portion is the pair. An image processing method characterized in that it is fixed relative position of the object.
Further, according to one aspect of the present invention, the surface of the object imaged by the imaging unit from different positions is 1 or by the image projected from the light projecting unit provided at a position not affected by the movement of the imaging unit. A plurality of indices are drawn, and the extraction unit extracts the one or more indices drawn in a line shape on the image and the feature points of the surface of the object from the image captured by the imaging unit. And the imaging unit that changes the position along the surface of the object captures images from the different positions to obtain a plurality of images, and the extracted one or each of the plurality of obtained images Detecting relative positions of the plurality of images based on a plurality of indices and feature points of the surface of the object; first data based on the feature points of the surface of the object; and the linear shape Based on the geometric features of the indicators drawn in Generating a second data, an image processing method which comprises the steps of generating data for adjusting an image on the basis of the first data into the second data.

Another embodiment of the present invention, the surface of the object to be imaged by the imaging unit from different positions, 1 by projected by image from the light projecting portion provided in not affected position the movement of the imaging unit or A plurality of indices are drawn, and the extraction unit extracts the one or more indices drawn in a line shape on the image and the feature points of the surface of the object from the image captured by the imaging unit. a step to obtain a plurality of images the image pickup unit to change the position in along Migihitsuji the surface of the object is imaged from the different positions, which is the extracted from each of the plurality of images the resulting 1 Or detecting a relative position of the plurality of images based on a plurality of indices and feature points of the surface of the object; first data based on the feature points of the surface of the object; and the line The geometry of the one or more indicators drawn in a shape Program for executing the steps of generating a second data based on the feature, and generating data for adjusting an image on the basis of the first data and the second data to the computer of the image processing system It is.

  As described above, according to the present invention, there are provided an image processing system, an image processing method, and a program for capturing an image of an object by an image capturing unit and obtaining an image capable of detecting the state of the detection target surface of the object. be able to.

It is explanatory drawing which shows the outline of the image processing system in the 1st Embodiment of this invention. It is a block diagram of the image processing system 1 (1A) in this embodiment. It is explanatory drawing shown about adjustment of the relative position of the several image of this embodiment. It is explanatory drawing shown about adjustment of the scale of the several image of this embodiment. It is a flowchart which shows the procedure of the process in the image processing system of this embodiment in this embodiment. It is explanatory drawing which shows the outline of the image processing system in 2nd Embodiment. It is explanatory drawing shown about adjustment of the scale of the some image in this embodiment. It is explanatory drawing shown about the correction | amendment of the trapezoid distortion of the image in this embodiment. It is a flowchart which shows the procedure of the process in the image processing system of this embodiment. It is explanatory drawing which shows the outline of the image processing system in 3rd Embodiment. It is explanatory drawing which shows the outline of the modification of the image processing system shown to the said embodiment.

An overview of an image processing system according to an embodiment of the present invention will be described.
<First Embodiment>
FIG. 1 is an explanatory diagram showing an outline of an image processing system according to the first embodiment of the present invention. The image processing system 1 shown in the figure is an example. In the following description, the case of checking the situation of the wall surface 21 of the building 2 will be described as an example. Note that the image processing system 1 of the present embodiment can also be used for purposes other than checking the condition of the wall surface 21 of the building 2.

  The imaging unit 220 shown in the present embodiment is mounted on the flying object 200, for example, and configured to be movable to a desired position with respect to the surface of the object by the movement of the flying object 200. The objects include structures such as buildings and facilities. Hereinafter, the case where the building 2 is used as an object will be described as an example. In the vicinity of the wall surface (surface) 21 of the building 2, a light projecting unit 31 that projects light from a position different from that of the flying object 200, for example, a position fixed relative to the position of the wall surface 21 of the building 2 is provided. Yes. An image projected from the light projecting unit 31 provided on the wall surface 21 (surface of the object) of the building 2 captured by the imaging unit 220 mounted on the flying object 200 so as not to be affected by the movement of the flying object 200. An index 41 is drawn.

  The image processing system 1 (image processing apparatus 100, extraction unit) extracts the index 41 and the feature point 5 of the wall surface 21 (surface of the object) of the building 2 from the image captured by the imaging unit 220. The imaging unit 220 captures a plurality of images from different positions by changing the position along the wall surface 21 (surface of the object) of the building 2. The image processing apparatus 100 acquires a plurality of images captured by the imaging unit 220. The image processing apparatus 100 (detection unit) uses the index 41 extracted from each of the acquired plurality of images and the feature point 5 of the wall surface 21 (the surface of the object) of the building 2 as a reference, and Detect relative position.

  The index may be a figure having an area such as a line (including a line segment), a point, a circle, or a polygon. For example, the light projecting unit 31 may draw a vertical line on the surface of the object like an optical blackout device. In the following description of the present embodiment, a case where the index is a line will be described as an example.

With reference to FIG. 2, the configuration of the image processing system 1 (1A) in the present embodiment will be described. FIG. 1 is a configuration diagram of the image processing system 1 (1A) in the present embodiment.
The image processing system 1 (1A) includes an image processing apparatus 100 and a flying object 200.
The image processing apparatus 100 includes an acquisition unit 111, an extraction unit 112, a detection unit 113, a correction unit 114, an input / output unit 115, and a storage unit 120.

  Note that the image processing apparatus 100 may be configured to include the flight control unit 119 when controlling the flying object 200. For example, the flight control unit 119 instructs the flying object 200 to specify a two-dimensional position of geographic coordinates and move in the vertical direction based on the position. Note that the two-dimensional position of the geographic coordinates is a position suitable for detecting the state of the wall surface 21 of the building 2.

  The storage unit 120 stores an image acquired from the imaging unit 220, an image (composite image) generated from an acquired image, data generated in the course of each process, and the like. The storage unit 120 stores the image acquired from the image capturing unit 220 in association with data such as identification information for identifying a target building, image shooting date and time, and image identification information indicating the shooting order.

The input / output unit 115 accepts an operation for instructing the execution of each process of the image processing apparatus 100 and instructs each part of the image processing apparatus 100 to perform a process to be performed according to the received operation. The input / output unit 115 displays the image on the display unit including an image acquired from the imaging unit 220, an image (composite image) generated from the acquired image, and the like.
For example, the image processing apparatus 100 may be configured to include a personal computer. In this case, the input / output unit 115 includes a keyboard, a mouse, a touch panel, and the like as an input unit, and a display such as a liquid crystal display panel as an output unit. A device may be provided.

The description of each part constituting the image processing apparatus 100 will be continued.
The acquisition unit 111 acquires the image captured by the imaging unit 220 by causing the flying object 200 to fly. There is no limitation on the medium or the like used when the acquisition unit 111 acquires an image, and means such as a means for using a communication line and a means for storing and reading in a storage medium can be appropriately selected. Note that when the acquisition unit 111 acquires an image, identification information for identifying the image may be attached in the order according to the imaging order, and the image may be identified by the identification information. The acquisition unit 111 stores the acquired image in the storage unit 120. The image stored in the storage unit 120 is referred to from each unit described below. In the following description, description of processing for referring to the storage unit 120 from each unit may be omitted.

  The extraction unit 112 extracts the index 41 and the feature point 5 of the wall surface 21 (the surface of the object) of the building 2 from the image captured by the imaging unit 220. As shown in the description of the outline above, on the wall surface 21 of the building 2, an index 41 based on an image projected from the light projecting unit 31 provided at a position different from the flying object 200 is drawn. The optical system of the imaging unit 220 is set so that the index 41 falls within the angle of view, and the index 41 is reflected in an image captured and acquired in a desired situation. For example, the feature point 5 of the wall surface 21 of the building 2 includes a structure (a veranda, a fence, a beam, etc.) existing on the wall surface 21, an accessory such as a window provided on the wall surface 21, tiles or paint for protecting the wall surface 21. , Etc. The shape and color etc. are mentioned. Although not permanent, as the feature point 5 of the wall surface 21 of the building 2, a portion where dirt, deterioration or damage of the wall surface 21 can be identified from the image may be used. As described above, the feature point 5 of the wall surface 21 of the building 2 may be a point indicating the feature in the wall surface 21 of the building 2.

  The detection unit 113 uses the index 41 extracted from each of the plurality of images obtained by the imaging unit 220 and the feature point 5 of the wall surface 21 (surface of the object) of the building 2 as a reference, and the plurality of the plurality of the plurality of images. Detect the relative position of the image. A plurality of images obtained by the imaging unit 220 are obtained by the imaging unit 220 capturing images from the moved position when the flying object 200 moves along the wall surface 21 (surface of the object) of the building 2. Is. The detection unit 113 extracts a plurality of indexes included in the image from the image, and the light of the imaging unit 220 with respect to the wall surface 21 (surface of the object) of the building 2 based on the extracted plurality of indexes 41. You may make it detect the direction of an axis | shaft.

  The correction unit 114 performs correction processing on the acquired image. The correction unit 114 may generate a composite image by combining the corrected images. Details of the correction processing will be described later.

(Adjustment of relative position of multiple images using index (line) and feature points of target surface)
With reference to FIG. 3, the adjustment of the relative positions of a plurality of images according to the present embodiment will be described.
This figure is an explanatory view showing the adjustment of the relative positions of a plurality of images of this embodiment.

As shown in FIG. 1 described above, the index 41 is drawn on the wall surface 21. Reference numerals 611 and 612 denote ranges that fall within the angle of view when the imaging unit 220 captures the wall surface 21 from different positions. There are overlapping areas in the range indicated by reference numerals 611 and 612. There is a feature point 5 at least in the overlapping region. In addition, the range indicated by reference numeral 611 and reference numeral 612 includes portions where the index 41 is drawn.
FIG. 3A shows an image P611 obtained by imaging the range indicated by reference numeral 611. FIG. 3B shows an image P612 obtained by imaging the range indicated by reference numeral 612. In the images shown in FIGS. 3A and 3B, common feature points on the surface of the object (the wall surface 21 of the building 2) are shown. FIG. 3C shows a composite image P631 generated from the plurality of images.

  The correcting unit 114 adjusts the relative positions of the plurality of images (the images P611 and P612) using the index 41 (line) and the feature points of the surface of the object (the wall surface 21 of the building 2). For example, the correction unit 114 adjusts the relative positions of the plurality of images (image P611 and image P612) to generate a composite image P631 illustrated in FIG. The relative position is adjusted based on the index 41 and the feature point 5 extracted from the images P611 and P612, as necessary, by adjusting the scale of the image and adjusting the rotational distortion as necessary, The relative position of each image is adjusted. In the image processing system 1, since the index 41 is included in each image as described above, adjustment is facilitated by adjusting the position and direction indicated by the index 41 as a reference.

(About adjusting the scale of multiple images)
Next, with reference to FIG. 3 and FIG. 4 described above, adjustment of scales of a plurality of images according to the present embodiment will be described. FIG. 4 is an explanatory diagram showing adjustment of the scales of a plurality of images according to the present embodiment.
4 (a), FIG. 4 (b), and FIG. 4 (c) correspond to FIG. 3 (a), FIG. 3 (b), and FIG. 3 (c), respectively. The image P611 shown in FIG. 4A and the image P612A shown in FIG. In FIG. 4B, the range of the image that has been obtained as an image of the same scale as the image P611 shown in FIG. 4A, for example, the range of the image P612 shown in FIG. 3B is a dotted line.

Here, based on the distance d1 between the index 41 and the feature point 5 extracted from each image, the correcting unit 114 adjusts the image P611 or / And the scale of the image of the image P612A is adjusted.
Alternatively, the correcting unit 114 may adjust the scale of the plurality of images (image P611 and image P612A) based on the index 41 extracted from each image and the size W1 of the feature point 5, and / or the image P611 and / or The scale of the image P612A is adjusted.
For example, the correction unit 114 adjusts the scale of the image P612A by the method described above to generate the image P613A.

(About correction of image rotation distortion)
With reference to FIG. 3 described above, correction of rotational distortion of an image will be described.
In the image P611 shown in FIG. 3A and the image P612 shown in FIG. 3B, the extending direction of the index 41 in each image is different. If the wall surface 21 is a plane, the index 41 is drawn on the wall surface 21 so as to be a straight line. If the imaging unit 220 moves in the extending direction of the index 41 along the wall surface 21 and combines images captured from different positions determined by the movement, the extending direction of the index 41 is aligned with each other. The relative angle of the image may be adjusted.
As described above, the correction unit 114 corrects distortion (rotational distortion) of the image based on at least the extracted index.

  With reference to FIG. 5, the process in the image processing system of this embodiment will be described. FIG. 5 is a flowchart showing a processing procedure in the image processing system of the present embodiment.

  First, the acquisition unit 111 acquires an image captured by the imaging unit 220 (step S11). The extraction unit 112 extracts an index from the image captured by the imaging unit 220 (step S12), and further extracts feature points (step S13).

  The detection unit 113 determines whether or not the scales of the plurality of images are uneven (step S14). If it is determined in step S14 that the scales of the plurality of images are not irregular (step S14: No), the process proceeds to step S17. On the other hand, when it is determined in step S14 that the scales of the plurality of images are not uniform (step S14: No), the correction unit 114 adjusts the scales of the plurality of images (step S15).

  After the determination in step S14 or after finishing the process in step S15, the correction unit 114 detects the relative positions of the plurality of images (step S17). The correcting unit 114 adjusts the positions of the plurality of images based on the detected relative positions, and generates a new image P613 (composite image) from the adjusted images (step S18).

  As described above, in the image processing system 1 according to the present embodiment, the imaging unit 220 images the building 2 (target object), and the state of the wall surface 21 (surface) that is the detection target of the building 2 (target object). A detectable image (composite image) can be obtained.

  In the above description, in order to simplify the description, the case where the optical axis of the imaging unit 220 is substantially orthogonal to the wall surface 21 is illustrated. The image processing system 1 collects information indicating the flight status of the flying object 200, information indicating the direction of the optical axis of the imaging unit 220, and the like, and performs a process of adjusting the image based on the collected information. Alternatively, the processes may be performed separately from the above processes, and the processes may be combined. For example, the image processing system 1 collects information indicating the flight status of the flying object 200, information indicating the direction of the optical axis of the imaging unit 220, and the like, and performs a process of adjusting the image based on each information. . In this case, in the process of adjusting the above image, the flight state of the flying object 200 is specified using information such as a gyro mounted on the flying object 200 and an acceleration sensor, and the optical axis of the imaging unit 220 is determined from the specified condition. Identify the direction. Through the calculation using the spatial coordinate system based on the flight status (position) of the flying object 200 and the direction of the optical axis of the imaging unit 220 as a result of the identification, the above-described image correction process can be performed. By performing such processing, it is possible to correct an image having trapezoidal distortion.

<Second Embodiment>
The second embodiment will be described with reference to FIGS.
FIG. 6 is an explanatory diagram illustrating an outline of an image processing system according to the second embodiment. The configuration shown in the figure shows an example to which the image processing system 1A is applied, and corresponds to the image processing system 1 described above. The description will focus on the differences from the image processing system 1 described above.

  In the vicinity of the wall surface (surface) 21 of the building 2, a light projecting unit 31 and a light projecting unit that project light from a position different from the flying object 200, for example, a position fixed relative to the position of the wall surface 21 of the building 2. 32 is provided.

The light projecting unit 31 provided on the wall surface 21 (surface of the object) of the building 2 that is imaged from a different position by the imaging unit 220 mounted on the flying object 200 so as not to be affected by the movement of the flying object 200. An index 41 based on the illuminated image is drawn, and similarly, an index 42 based on the image projected from the light projecting unit 32 provided so as not to be affected by the movement of the flying object 200 is drawn.
The image processing system 1A (the image processing apparatus 100A, the extraction unit) extracts the index 41, the index 42, and the feature point 5 of the wall surface 21 (the surface of the object) of the building 2 from the image captured by the imaging unit 220.
The imaging unit 220 captures a plurality of images from different positions by changing the position along the wall surface 21 (surface of the object) of the building 2. The image processing apparatus 100A acquires a plurality of images captured by the imaging unit 220. The image processing apparatus 100A (detection unit) uses the index 41 extracted from each of the acquired plurality of images and the feature point 5 of the wall surface 21 (the surface of the object) of the building 2 as a reference, and Detect relative position.
The index may be a figure having an area such as a line (including a line segment), a point, a circle, or a polygon. For example, each of the light projecting unit 31 and the light projecting unit 32 may draw a vertical line on the surface of the object, like an optical blackout device. Moreover, even if a plurality of indexes are included in one image by projecting indexes having different characteristics, the light projecting unit 31 and the light projecting unit 32 may post any of the projecting units. It becomes easy to specify whether or not. In addition, when three or more indicators are included in one image, in addition to making them different from each other, it is configured to repeat with a periodicity in the order in which a plurality of light projecting units are arranged along the wall surface 21. May be.
Alternatively, when three or more indicators (lines) are included in one image, the feature of the line is an odd-numbered line and an even-numbered line in the order in which a plurality of light projecting units are arranged along the wall surface 21. May be changed. For example, the line characteristics include one or more of line color, brightness, line type (solid line, broken line, dash-dot line, etc.), shape (double line, thickness, wavy line, etc.), etc. The combination is mentioned. In the following description of the present embodiment, a case where the index is a line having the same characteristics will be described as an example.

(Adjustment of relative position of multiple images using index (line) and feature points of target surface)
With reference to FIG. 3 and FIG. 6 described above, adjustment of the relative positions of a plurality of images according to the present embodiment will be described.

  As shown in FIG. 6, the index 41 and the index 42 are drawn on the wall surface 21. Reference numerals 621 and 622 denote ranges that fall within the angle of view when the imaging unit 220 images the wall surface 21 from different positions. There are overlapping areas in the range indicated by reference numerals 621 and 622. There is a feature point 5 at least in the overlapping region. In addition, the range indicated by the reference numerals 621 and 622 includes portions where the index 41 and the index 42 are drawn.

In this embodiment, prior to the adjustment of the relative positions of a plurality of images, the scale of the image is adjusted by a method described later. Further, at this stage, a correction process for the trapezoidal distortion of the image may be performed as necessary by a method described later. The feature point 5 is extracted from the image after the scale adjustment and the like after the scale adjustment and the like.
Thereafter, the relative positions of the plurality of images are adjusted by a process similar to the process shown in FIG. In adjusting the relative positions of the plurality of images, at least one of the index 41 and the index 42 can be used.
In the image processing system 1A, since the index 41 and the index 42 are included in each other image as described above, by adjusting the position and direction indicated by either the index 41 or the index 42 as a reference, Making adjustments easier.

(About adjusting the scale of multiple images)
Next, with reference to FIG. 7, the adjustment of the scale of a plurality of images according to the present embodiment will be described. This figure is an explanatory view showing the adjustment of the scale of a plurality of images of the present embodiment.
FIG. 7A shows an image P621 obtained by imaging the range indicated by reference numeral 621. FIG. FIG. 7B shows an image P621N obtained by similarly imaging from the position closer to the wall surface 21 than the position where FIG. FIG.7 (c) shows the image P621F obtained by image-capturing similarly from the position far from the wall surface 21 from the position which image | photographed Fig.7 (a).
The images shown in FIG. 7A, FIG. 7B, and FIG. 7C include portions where the index 41 and the index 42 are drawn, respectively.

  The correcting unit 114 uses the index 41 (line) and the index 42 (line) to adjust the scale of each image so that the scale of each image is uniform. For example, the correction unit 114 performs a process of enlarging or reducing each image so that the interval between the index 41 (line) and the index 42 (line) in a plurality of images is uniform. For example, if the interval between the index 41 (line) and the index 42 (line) is relatively wide as in the image P621N (FIG. 7B), the index 41 (line) and the index like the image P621 are used. 42 (line) interval (reference interval) is adjusted. On the other hand, if the interval between the index 41 (line) and the index 42 (line) is relatively narrow as in the image P621F (FIG. 7C), the index 41 (line) and the index like the image P621 are used. 42 (line) interval (reference interval) is adjusted. By the above enlargement or reduction process, an image in which the interval between the index 41 (line) and the index 42 (line) is aligned with a desired interval (reference interval) determined as a reference is obtained. With the above processing, the image processing apparatus 100 can correct the scales of a plurality of images.

(Regarding correction of trapezoidal distortion of the image that occurs according to the direction of the optical axis of the imaging unit)
With reference to FIG. 8, correction of trapezoidal distortion of an image in the present embodiment will be described.
This figure is an explanatory diagram showing correction of trapezoidal distortion of an image in the present embodiment.
The direction of the optical axis of the imaging unit 220 does not necessarily coincide with the normal direction of the wall surface 21. A plurality of indices (index 41 or index 42) are drawn on the wall surface 21 at desired intervals. Therefore, if the optical axis of the imaging unit 220 and the linearly drawn index are in a twisted position, when the depression angle or the angle of attack occurs on the optical axis of the imaging unit 220, the index in the obtained image is oblique. Drawn as a line.
Further, when the imaging unit 220 itself rotates around the optical axis, the obtained image is also rotated, and the index in the obtained image is drawn as an oblique line. In the latter case, for example, the distortion caused by the rotation can be eliminated by correcting the index so that it is in the vertical direction.

  In the former case, as shown in FIGS. 8A and 8B, the image captured by the imaging unit 220 includes two indices (index 41 and index 42), an upper side TL, and a lower side BL of the image. It becomes an enclosed figure (quadrangle). FIG. 8A shows an image P621L acquired when a depression angle occurs on the optical axis of the imaging unit 220. FIG. FIG. 8B shows an image P621U acquired when a depression angle occurs on the optical axis of the imaging unit 220. The correction unit 114 treats the above figure as a trapezoid and corrects the entire screen so as to correct the trapezoid to a rectangle. With the above processing, the image processing apparatus 100 can correct the trapezoidal distortion of the image that occurs according to the direction of the optical axis of the imaging unit 220.

  With reference to FIG. 9, the process in the image processing system of this embodiment will be described. FIG. 5 is a flowchart showing a processing procedure in the image processing system of the present embodiment.

  First, the acquisition unit 111 acquires an image captured by the imaging unit 220 (step S21). The extraction unit 112 extracts an index from the image captured by the imaging unit 220 (step S22).

  The detection unit 113 determines whether or not the scales of the plurality of images are uneven (step S24). If it is determined in step S24 that the scales of the plurality of images are not irregular (step S24: No), the process proceeds to step S26. On the other hand, when it is determined in step S24 that the scales of the plurality of images are not uniform (step S24: No), the correction unit 114 adjusts the scales of the plurality of images (step S25).

  After the determination in step S24 or after finishing the process in step S25, the extraction unit 112 extracts feature points from the image captured by the imaging unit 220 (step S26).

  The correcting unit 114 detects the relative positions of the plurality of images (step S27). The correcting unit 114 adjusts the positions of the plurality of images based on the detected relative positions, and generates a new image P613 (composite image) from the adjusted images (step S28).

  As described above, in the image processing system 1 according to the present embodiment, the imaging unit 220 images the building 2 (target object), and the state of the wall surface 21 (surface) that is the detection target of the building 2 (target object). A detectable image (composite image) can be obtained.

  As described above, the image processing system 1A using a plurality of indices performs the above-described processing in a procedure different from the processing of the above-described image processing system 1, but as with the processing of the image processing system 1, the imaging unit The building 2 (object) can be imaged by 220, and an image (composite image) that can detect the state of the wall surface 21 (surface) to be detected of the building 2 (object) can be obtained.

<Third Embodiment>
The third embodiment will be described with reference to FIG. FIG. 10 is an explanatory diagram showing an outline of an image processing system in the third embodiment. The configuration shown in the figure shows an example to which the image processing system 1B is applied, and corresponds to the image processing system 1 and the image processing system 1A described above. The description will focus on the differences between the image processing system 1 and the image processing system 1A.

A light projecting portion 31A that projects light from a position fixed relative to the position of the wall surface 21 of the building 2 is provided. The light projecting unit 31A provided on the wall surface 21 (surface of the object) of the building 2 that is imaged from different positions by the imaging unit 220 mounted on the flying object 200 so as not to be affected by the movement of the flying object 200. An index 43 and an index 44 based on the illuminated image are drawn.
As described above, the image posted from one light projecting unit 31A of the present embodiment is drawn so as to be separated into a plurality on the wall surface 21 (surface). These images are referred to as an index 43 and an index 44.
For example, the light projecting unit 31A projects light so that the direction from the index 43 to the index 44 is the vertical direction.
The imaging unit 220 captures an image so as to include the plurality of indices in the angle of view.
In the case of the present embodiment, a linear index as shown in the first embodiment or the like is not obtained in the image captured by the imaging unit 220. However, the vertical direction can be read from the positions of the indices 43 and 44 in the captured image. As described above, even when the feature of the index is configured differently from the configuration of the above-described embodiment, the image processing system 1B uses the imaging unit 220 in the same manner as the image processing system 1 (1A) described above. The building 2 (object) can be imaged, and an image (composite image) that can detect the state of the wall surface 21 (surface) that is the detection target of the building 2 (object) can be obtained.

<Modification of the first to third embodiments>
With reference to FIG. 11, an image processing system as a modified example of the first to third embodiments will be described. FIG. 1 is an explanatory diagram showing an outline of a modification of the image processing system 1. Unlike the configurations shown in the first to third embodiments of the present invention, the present modification is configured without including the flying body 200 in the configuration. For example, as shown in the figure, it is assumed that a building 7 having a plurality of floors stands in the vicinity of the building 2 where the height of the building is equal to or higher than the building 2.
If it is the building 2 comprised as mentioned above, the imaging part 220 can image the wall surface 21 of the building 2 from each floor of the building 7, or a one part floor. The imaging unit 220 moves in the height direction of the building 2 without depending on the flying body 200 and captures an image of the wall surface 21 of the building 2. Further, if the floor can be moved in the horizontal direction on the same floor of the building 7, the imaging unit 220 may acquire the image by moving the same floor of the building 7 in the horizontal direction.
The above modifications are not limited to being applied as modifications of the first embodiment, but can be applied as modifications of the second embodiment and the third embodiment.

  As described above, in the image processing system according to the modified example, the imaging unit 220 images the building 2 (target object) and detects the state of the wall surface 21 (surface) that is the detection target of the building 2 (target object). Possible images (composite images) can be obtained.

  Although the embodiment of the present invention has been described above, the image processing system 1 has a computer system inside. A series of processes related to the above-described process is stored in a computer-readable storage medium in the form of a program, and the above-described process is performed by the computer reading and executing this program. Here, the computer-readable storage medium refers to a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, and the like. Alternatively, the computer program may be distributed to the computer via a communication line, and the computer that has received the distribution may execute the program. The “computer system” here includes an OS and the like.

  Then, all or a part of each processing in the image processing apparatus 100, the flying object 200, and the imaging unit 220 in the image processing system 1 is performed by a central processing unit such as a CPU in the main storage device such as a ROM or a RAM. Is read out and information processing and calculation processing are executed. Of course, each processing unit constituting the image processing system 1 may be realized by dedicated hardware.

  Although the embodiments of the present invention have been described above, the image processing system of the present invention is not limited to the illustrated examples described above, and various modifications can be made without departing from the scope of the present invention. Of course.

(1) In the image processing system 1 shown in the present embodiment, the light projection provided on the surface (wall surface 21) of the object imaged by the imaging unit from different positions at a position not affected by the movement of the imaging unit. The index by the image projected from the part is drawn. The extraction unit 112 extracts the index 41 and the feature point 5 of the surface (wall surface 21) of the object from the image captured by the imaging unit 220. The detection unit 113 changes the position along the surface (wall surface 21) of the object. The index extracted from each of the plurality of images obtained by the imaging unit imaging from the different positions and the surface of the object The relative positions of the plurality of images are detected with reference to the feature points.
Thereby, the image processing system 1 can capture an image of the object by the image capturing unit 220 and obtain an image (composite image) that can detect the state of the wall surface 21 (surface) of the detection target of the object.

(2) Moreover, according to said image processing system 1, the correction | amendment part 114 is the distance d1 between the extracted parameter | index (index | index 41) and the feature point 5 of the surface (wall surface 21) of the said object, or Based on the extracted index (index 41) and the size W1 of the feature point 5 of the surface (wall surface 21) of the object, one or a plurality of images are adjusted so that the scales of the plurality of images match.
Thereby, the image processing system 1 images an object by the imaging unit 220, adjusts the scale of the captured image, and can detect the state of the wall surface 21 (surface) of the detection target of the object ( Composite image) can be obtained.

(3) Moreover, according to said image processing system 1, the correction | amendment part 114 correct | amends the distortion of the said image based on the said extracted parameter | index at least.
As a result, the image processing system 1 captures an object by the image capturing unit 220, corrects the distortion of the captured image, and can detect the state of the wall surface 21 (surface) of the detection target of the object ( Composite image) can be obtained.

  (4) Further, according to the image processing system 1 described above, the detection unit 113 extracts the plurality of indexes (the index 41 and the index 42, or the index 43 and the index 44) from the image, and the extracted plurality of the indexes. The direction of the optical axis of the imaging unit 220 with respect to the surface (wall surface 21) of the object is detected based on the index.

(5) According to the image processing system 1 described above, the correction unit 114 causes the imaging unit 220 to capture images based on the detection result of the direction of the optical axis of the imaging unit 220 with respect to the surface (wall surface 21) of the object. Correct the distortion of the image.
As a result, the image processing system 1 captures an object with the image capturing unit 220, corrects distortion of the captured image, and detects an image (the surface) of the detection target wall surface 21 (surface). Composite image) can be obtained.

1 image processing system, 2 building (object), 21 wall surface (surface), 7 building,
31, 32 Projection unit, 41, 42 Index, 5 Feature points,
100 image processing apparatus, 111 acquisition unit, 112 extraction unit, 113 detection unit, 114 correction unit, 115 input / output unit, 120 storage unit,
200 aircraft, 220 imaging unit,
611, 612 angle of view,
P611, P612, P612A, P612F, P612L, P612N, P612U, P613, P613A images

Claims (9)

On the surface of the object imaged by the imaging unit from different positions, one or a plurality of indices are drawn by the image projected from the light projecting unit provided at a position not affected by the movement of the imaging unit, An extraction unit that extracts the one or more indices and feature points of the surface of the object from an image captured by the imaging unit;
To obtain a plurality of images the image pickup unit for changing the position in the different positions relative to the position of the object in along Migihitsuji the surface of the object is imaged from the different positions, a plurality of the obtained A detection unit that detects relative positions of the plurality of images based on the one or more indices extracted from each of the images and feature points of the surface of the object ,
The light projecting unit is formed independently of the imaging unit, and the position of the light projecting unit is fixed relative to the position of the object . On the surface of the object imaged by the imaging unit from different positions, one or a plurality of indices are drawn by the image projected from the light projecting unit provided at a position not affected by the movement of the imaging unit,
An extraction unit that extracts the one or more indices drawn linearly in the image and feature points of the surface of the object from an image captured by the imaging unit;
To obtain a plurality of images the image pickup unit to change the position in along Migihitsuji the surface of the object is imaged from the different positions, from each of the plurality of images the obtained one or more of which is the extraction A detection unit that detects relative positions of the plurality of images on the basis of an index and a feature point of the surface of the object; and
First data based on the feature points of the surface of the object and second data based on geometric features of the one or more indices drawn in a line are generated, and the first data and the An image processing system for generating data for adjusting an image based on the second data . The distance between the feature points of the face of said extracted one or more indicators and the object, or, based on the size of the feature points of the face of said extracted one or more indicators and the object, wherein the image processing system according to claim 1 or claim 2, characterized in that it comprises a correction unit for adjusting the one or more images to fit the measures of the plurality of images. Based on one or more indicators is at least the extracted image processing system according to claim 1 or claim 2, characterized in that it comprises a correction unit for correcting the distortion of the image. The detector is
The one or more indices that can adjust the distortion of the captured image are extracted from the captured image, and the optical axis of the imaging unit with respect to the surface of the object based on the extracted one or more indices The image processing system according to claim 3 or 4 , wherein the direction of the image is detected. The correction unit is
The image processing system according to claim 5 , wherein distortion of an image captured by the imaging unit is corrected based on a detection result of an optical axis direction of the imaging unit with respect to the surface of the object. On the surface of the object imaged by the imaging unit from different positions, one or a plurality of indices are drawn by the image projected from the light projecting unit provided at a position not affected by the movement of the imaging unit, An extraction unit extracting the one or more indices and feature points of the surface of the object from an image captured by the imaging unit;
To obtain a plurality of images the image pickup unit for changing the position in the different positions relative to the position of the object to Migihitsuji the surface of the object is imaged from the different positions, the resulting plurality of Based on the one or more indices extracted from each of the images and the feature points of the surface of the object,
Look including the step of detecting the relative positions of the plurality of images,
The light projecting unit is formed independently of the imaging unit,
An image processing method, wherein the position of the light projecting unit is fixed relative to the position of the object . On the surface of the object imaged by the imaging unit from different positions, one or a plurality of indices are drawn by the image projected from the light projecting unit provided at a position not affected by the movement of the imaging unit,
An extraction unit extracting the one or more indices drawn linearly in the image and feature points of the surface of the object from an image captured by the imaging unit;
To obtain a plurality of images the image pickup unit to change the position in along Migihitsuji the surface of the object is imaged from the different positions, from each of the plurality of images the obtained one or more of which is the extraction Detecting relative positions of the plurality of images based on an index and a feature point of the surface of the object;
Generating first data based on the feature point of the surface of the object and second data based on a geometric feature of the index drawn in a line, the first data and the second data; Generating data for adjusting the image based on the image processing method. On the surface of the object imaged by the imaging unit from different positions, one or a plurality of indices are drawn by the image projected from the light projecting unit provided at a position not affected by the movement of the imaging unit,
An extraction unit extracting the one or more indices drawn linearly in the image and feature points of the surface of the object from an image captured by the imaging unit;
To obtain a plurality of images the image pickup unit to change the position in along Migihitsuji the surface of the object is imaged from the different positions, from each of the plurality of images the obtained one or more of which is the extraction Detecting relative positions of the plurality of images based on an index and a feature point of the surface of the object ;
Generating first data based on the feature points of the surface of the object and second data based on geometric features of the one or more indices drawn in a line;
A program for causing a computer of an image processing system to execute a step of generating data for adjusting an image based on the first data and the second data .

Images