JP2004030440A - Image processing method, image processing program, and computer readable recording medium with the program recorded thereon - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing method, an image processing program and a computer readable recording medium with the program recorded thereon for accurately producing a three-dimensional (3D) image. <P>SOLUTION: A prescribed area on the ground including a photographing target is photographed from different points of view by a line sensor to acquire image data (S1). Then, a 3D area and 3D line segments are produced from a feature area and feature line segments extracted from the photographing target (S4 and S9). The 3D line segments near the 3D area are collected to produce a 3D line segment group (S12), a height of the 3D area is corrected by using the 3D line segment group and based upon the corrected 3D area, a photographing target model of a 3D image is corrected (S16). <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image processing method, an image processing program, and a computer-readable recording medium storing the same.
[0002]
[Prior art]
2. Description of the Related Art In recent years, in the field of GIS (Geographic Information System), demand for a three-dimensional map has been increasing, particularly in an urban area. Examples of the usage form include road information used for car navigation and the like, urban disaster GIS, and the like. In addition, a detailed and highly accurate three-dimensional map is required for analyzes such as radio wave propagation analysis and flood analysis.
[0003]
[Problems to be solved by the invention]
The inventor thought that as one of the methods of creating such a three-dimensional map, there is a method of generating a three-dimensional image of the target area by using a plurality of two-dimensional images taken of the target area from different viewpoints in the sky.
[0004]
When such a three-dimensional image is generated, a spatial position of a characteristic portion such as a roof of the object to be photographed is obtained based on images of the same object to be photographed included in a plurality of two-dimensional images, and then the photographing is performed. Generate a three-dimensional image of the object. When generating a three-dimensional image of a shooting target using a plurality of two-dimensional images in this manner, it is preferable to generate the three-dimensional image with high accuracy.
[0005]
An object of the present invention is to provide an image processing method, an image processing program, and a computer-readable recording medium on which the image processing method and the image processing program can generate a three-dimensional image with high accuracy.
[0006]
[Means for Solving the Problems]
In order to achieve such an object, an image processing method according to the present invention provides a three-dimensional image based on a plurality of two-dimensional images obtained by photographing a predetermined area on the ground including a photographing object from different viewpoints in the sky. A first step of extracting a characteristic region corresponding to the imaging target from each of the plurality of two-dimensional images, and extracting the characteristic region corresponding to the same imaging target. A model having a second step of associating between two-dimensional images, and a third step of estimating a photographing object model that is a three-dimensional model of the photographing object using the shape and spatial position of the associated characteristic region An estimating step, a fourth step of extracting, from each of the plurality of two-dimensional images, a characteristic line corresponding to the imaging target, and extracting the characteristic line corresponding to the same imaging target into the two-dimensional image And a sixth step of calculating three-dimensional coordinates of the associated feature line to generate a three-dimensional line segment, and a fifth step of calculating the three-dimensional coordinates of the associated feature line, and a model estimation step. A line group generation step of grouping a plurality of the three-dimensional line segments present in the vicinity of the imaging target model to generate a three-dimensional line group, based on the three-dimensional line group generated by the line group generation step And a fitting step of correcting the photographing object model.
[0007]
In the image processing method described above, the imaging target object model is corrected by using a three-dimensional line group generated by grouping three-dimensional line segments existing near the imaging target object model. Since the three-dimensional line segment has better spatial position accuracy than the three-dimensional region, by correcting the photographing object model in this way, a photographing object model that is a three-dimensional image can be generated with high accuracy.
[0008]
In addition, the image processing method may further include a model merging step of combining imaging object models located near each other among the imaging object models corrected by the fitting step. As a result, it is possible to suitably generate an imaging target object model even for an imaging target having a complicated shape.
[0009]
The image processing method may be characterized in that the line group generation step generates a three-dimensional line group using height distribution information of a predetermined area. Accordingly, even when the three-dimensional region has a large error with respect to the shape of the surface of the imaging target corresponding to the three-dimensional region, the three-dimensional line segments can be appropriately grouped.
[0010]
Further, the image processing method collects a plurality of three-dimensional line segments related to the same side included in the imaging target among the three-dimensional line segments included in the three-dimensional line group generated in the three-dimensional line group generation step. The method may further include a smoothing step of generating one three-dimensional line segment. When a plurality of three-dimensional line segments are generated for one side of the object to be imaged, one three-dimensional line segment can be specified for one side by this smoothing step.
[0011]
Further, the image processing method may be characterized in that the third step of the model estimation step estimates a photographing target object model by calculating three-dimensional coordinates of the associated characteristic region and generating a three-dimensional region. Good. This makes it possible to appropriately estimate the photographing object model.
[0012]
In the image processing method, the fitting step corrects the height of the three-dimensional region based on the height of the three-dimensional line group grouped by the imaging target object model estimated from the three-dimensional region. It may be characterized in that the object model is modified. As a result, the upper surface of the object to be photographed can be modeled with high accuracy, so that the object model to be photographed can be generated with higher accuracy.
[0013]
In addition, the image processing method further includes a tilting step of tilting the three-dimensional region corresponding to the three-dimensional line segment group when the height difference between the three-dimensional line segments included in the three-dimensional line segment group is equal to or more than a predetermined value. It may be provided with a feature. Alternatively, a tilting step of giving a tilt to a three-dimensional region corresponding to a three-dimensional line segment group based on three-dimensional region or three-dimensional line segment tilt information obtained when associating an image of an imaging target included in a two-dimensional image. May be further provided. By providing such a tilting step, it is possible to suitably generate a shooting target model of a shooting target having a tilted surface.
[0014]
Further, an image processing program according to the present invention causes a computer to execute the above-described image processing method.
[0015]
Further, a recording medium according to the present invention is characterized by recording an image processing program for causing a computer to execute the above-described image processing method.
[0016]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, preferred embodiments of an image processing method, an image processing program, and a computer-readable recording medium storing the same according to the present invention will be described in detail with reference to the drawings. In the description of the drawings, the same elements will be denoted by the same reference symbols, without redundant description. Also, the dimensional ratios in the drawings do not always match those described.
[0017]
FIG. 1 is a diagram illustrating an image processing apparatus 1 that performs an image processing method according to the present embodiment. The image processing apparatus 1 includes a computer 10 such as a workstation or a personal computer, a monitor 11 as a display device, a keyboard 12 as an input device, and a mouse 13. The computer 10 may be equipped with a hard disk as an internal recording device, and may be connected to various external recording devices.
[0018]
Image data and height distribution information data recorded on a hard disk 21 in an imaging device 2 mounted on a flying object, which will be described later, are transferred to a recording device inside or outside the computer 10. This transfer may be performed by connecting the imaging device 2 and the image data processing device 1 wirelessly or by wire, or may be performed via a removable recording medium.
[0019]
FIG. 2 is a diagram illustrating a configuration of the imaging device 2. As shown in FIG. 2A, the imaging device 2 is attached to a lower part of a flying object 3 such as an aircraft, an airship, and a helicopter, and a predetermined area on the ground including an object to be photographed from above while the flying object 3 is flying. This is for acquiring a two-dimensional image.
[0020]
FIG. 2B is a diagram illustrating a configuration of the imaging device 2. In FIG. 2B, the normal traveling direction of the flying object 3 is shown as facing left. The imaging device 2 includes a data processing unit 20, a hard disk 21, an imaging unit (camera) 27, and a distance measurement unit 28.
[0021]
The photographing unit 27 is a device for acquiring a two-dimensional image of a predetermined area including a photographing target. Here, FIG. 3 is a diagram showing a concept in which the image capturing section 27 captures an image from above to below. The line sensors 27B, 27F, and 27N are configured by arranging a plurality of pixels b in a line, and continuously capture images as the flying object 3 advances. Further, each line sensor is provided at a predetermined interval with a longitudinal direction perpendicular to the traveling direction of the flying object 3 as a longitudinal direction. The shooting direction vector A1 of the line sensor 27F, the shooting direction vector A2 of the line sensor 27N, and the shooting direction vector A3 of the line sensor 27B intersect at one point. As a result, the line sensor 27F receives the image data D1, which is the two-dimensional image data immediately below the flying object 3, and the line sensor 27N sends the image data D2, which is the two-dimensional image data immediately below the flying object 3, to the line sensor 27F. Reference numeral 27B can acquire image data D3 which is two-dimensional image data below and behind the flying object 3, respectively.
[0022]
FIG. 4 shows the position of each line sensor at the moment when a certain point P on the photographing target 300 is photographed. As shown in FIG. 4, the line sensor 27F captures the photographing target 300 in a forward (downward direction) vector with respect to the traveling direction of the flying object 3 (arrow a in FIG. 4). The line sensor 27N captures the imaging target 300 directly below the flying object 3 (imaging direction vector A2). The line sensor 27B captures the object 300 to be photographed below the photographing direction vector A3 with respect to the traveling direction of the flying object 3. FIG. 5A shows an example of the image data D2 among the image data thus photographed. The image data D2 includes an image 300a obtained by photographing the photographing object 300 from the direction of the photographing direction vector A2 in FIG. FIG. 5B shows an example of the image data D3. The image data D3 includes an image 300b obtained by shooting the shooting target 300 from the direction of the shooting direction vector A3 in FIG. In both FIGS. 5A and 5B, the arrow a in the figure indicates the traveling direction of the flying object 3.
[0023]
The distance measuring unit 28 illustrated in FIG. 2B is a device that measures the distance from the imaging device 2 to a plurality of points in a predetermined area on the ground including the imaging target. The density of the plurality of points in the predetermined area may be a density suitable for obtaining the required resolution in consideration of the time required for measuring one point.
[0024]
The distance measuring unit 28 includes a laser profiler 28a. The laser profiler 28a measures the distance from a point set in a predetermined area on the ground including the photographing target by irradiating the laser toward the ground and detecting the time until the laser is reflected and returned. I do. Then, the laser profiler 28a generates the height distribution information data D4 of the predetermined area based on the measurement result. This height distribution information data D4 is random point cloud data having three-dimensional coordinate values. The laser profiler 28a may measure points at a density such that the resolution is about 2 m when measured from an altitude of 3000 m, for example.
[0025]
The data processing unit 20 inputs the image data D1 to D3 acquired by the imaging unit 27 and the height distribution information data D4 generated by the distance measurement unit 28. Then, the data processing unit 20 outputs the image data D1 to D3 and the height distribution information data D4 to the hard disk 21. The hard disk 21 records image data D1 to D3 and height distribution information data D4.
[0026]
Hereinafter, the image processing method according to the present embodiment will be specifically described. The image processing method according to the present embodiment uses a three-dimensional region and a three-dimensional line segment generated by matching between image data D1 to D3 and height distribution information data D4 to capture a three-dimensional image. This is a method for generating an object model.
[0027]
FIG. 6 is a flowchart illustrating a method of generating a three-dimensional image including the image processing method according to the present embodiment. In the first step S1, first, using the imaging device 2 including the line sensors 27F, 27N, and 27B, a predetermined area on the ground including the imaging target is imaged from the sky, and image data D1 to D3 that are two-dimensional image data are obtained. Generate. Then, the image data D1 to D3 are once recorded on a recording device such as the hard disk 21 and then loaded into the computer 10.
[0028]
In the following step S2, a characteristic region corresponding to the photographing target is extracted using the image data D2 and D3 taken into the computer 10, and region data D2a and D3a relating to the characteristic region are generated (first step). FIG. 7 is a diagram showing (a) the area data D2a and (b) the area data D3a generated in step S2. In step S2, a characteristic region characterized by a constant shading level is extracted from an image of the photographing target such as the image 300a included in the image data D2. At this time, one of the following two methods may be used as a method for extracting the characteristic region. One method is a region expansion method in which image data is divided into small regions whose features are considered to be uniform, features between adjacent small regions are checked, and successively integrated into small regions having similar features. is there. Another method is a level slicing method in which clustering is performed by using a valley of a grayscale histogram based on image data as a boundary of a feature region. In FIG. 7A, as an example of the characteristic region, the roof portion of the image 300a having a substantially constant shading level is set as the characteristic region 301a.
[0029]
In this way, in step S2, the area data D2a in which information on the characteristic area is added to the image data D2 is generated. In step S2, a characteristic region such as the characteristic region 301b is extracted from the image of the photographing target such as the image 300b shown in FIG. 7B, for example, in the same manner as the image data D2. Then, area data D3a is generated by adding information on the characteristic area to the image data D3.
[0030]
In step S3, the characteristic regions of the same photographic object included in both the region data D2a and the region data D3a are associated with each other (second step). For example, the region data D2a illustrated in FIG. 7A includes a characteristic region 301a extracted from the image 300a. On the other hand, the region data D3a shown in FIG. 7B includes a characteristic region 301b extracted from the image 300b. Since the image 300a and the image 300b are images relating to the building 300 (shown in FIG. 4) which is the same photographing target, the characteristic regions 301a and 301b extracted from these images depend on their shapes and sizes. Correlated. At this time, as a method of searching for a corresponding feature region, a straight line (the coplanar condition) where the projection center of the two images and the target point are included in the same plane (coplanar condition) is used. A method of searching on the epipolar line) may be used. Thus, the characteristic regions 301a and 301b are associated with each other as characteristic regions of the same object to be photographed.
[0031]
In step S4, the three-dimensional coordinates of the target region are calculated from the characteristic region associated in step S3 by the front rear association method based on the region data D2a and the region data D3a. The spatial position of the characteristic region is determined based on the target region for which the three-dimensional coordinates have been determined in this way, and a three-dimensional region is generated. Then, from the three-dimensional area, a photographing object model that is a three-dimensional image (solid model) of the photographing object including the three-dimensional area is estimated (third step). For example, a plane perpendicular to the ground including the sides of the three-dimensional region may be generated to estimate the imaging target object model.
[0032]
The first to third steps constitute a model estimation step S5.
[0033]
In step S6, using the image data D2 and D3 captured by the computer 10, an edge image related to an edge generated by shading of the image of the imaging target is generated. As a method of generating an edge image, it is preferable to use a Canny method or the like for extracting an edge by obtaining a maximum of a gradient of a gray level of an image using a differential coefficient of a Gaussian filter. Further, as for the edge having a small gradient of the gray level, the edge connected to the edge having the large gradient of the gray level may be set as the edge.
[0034]
In step S7, a feature line segment that is an edge having a predetermined length is extracted from the edge image generated in step S6 (fourth step). As a feature line segment extraction method, for example, several types of templates are prepared in which a plurality of pixels are arranged two-dimensionally and some of the pixels indicate a part of the line segment. It is good to determine that it is a minute and extract it. FIG. 8 is a diagram showing (a) line segment data D2b generated by extracting the characteristic line segment from the image data D2 and (b) line segment data D3b generated by extracting the characteristic line segment from the image data D3 generated in step S7. is there. In FIG. 8A, as an example of the feature line segment, sides that are linear components around the roof portion having a large change in shading level in the image 300a are extracted as feature line segments 302a1 to 302a4. Similarly, in FIG. 8B, the sides around the roof portion where the change in the gray level is large in the image 300b are extracted as the characteristic line segments 302b1 to 302b4.
[0035]
In step S8, the characteristic line segments of the same object to be photographed, which are included in both the line segment data D2b and the line segment data D3b, are associated (fifth step). For example, the line segment data D2b shown in FIG. 8A includes characteristic line segments 302a1 to 302a4 extracted from the image 300a. On the other hand, the line segment data D3b shown in FIG. 8B includes the characteristic line segments 302b1 to 302b4 extracted from the image 300b. Since the image 300a and the image 300b are images relating to the building 300 (shown in FIG. 4) as the same photographing target, the characteristic line segments 302a1 to 302a4 and 302b1 to 302b4 extracted from these images are the same. Are associated with each other as characteristic line segments at the same location of the object to be photographed. As a method of this association, the shape of the characteristic line, the shading level on both sides of the characteristic line, the inclination of the characteristic line, the positional relationship with the surrounding characteristic region, the length of the characteristic line, and the like are matched. The feature line segments are associated with each other. At this time, similarly to the association of the characteristic regions, a corresponding characteristic line segment may be searched using the epipolar line.
[0036]
In step S8, the feature line segments may be associated using the three-dimensional region generated in step S4. That is, since there are relatively many feature line segments in the image data, it takes a long time to associate them, and there are many mistakes in the association. In order to solve this, when searching for a feature line segment using an epipolar line, the height component of the three-dimensional coordinates of a three-dimensional region relating to the same shooting object as the shooting object corresponding to the feature line segment is used. To narrow the search range.
[0037]
In step S9, a point corresponding to a feature point (such as an end point of the feature line segment) of the imaging target is extracted from the feature line segment associated in step S8. Then, the three-dimensional coordinates of the feature point are calculated by the front rear association method based on the line segment data D2b and the line segment data D3b. Based on the feature points for which the three-dimensional coordinates have been determined in this way, the spatial position of the feature line is determined, and a three-dimensional line segment is generated (sixth step).
[0038]
The segment generation step S10 is configured by the above fourth to sixth processes.
[0039]
Here, height distribution information of a predetermined area on the ground including the imaging target is acquired using the distance measurement unit 28 illustrated in FIG. In step S11, the distance from a plurality of points in a predetermined area on the ground including the imaging target is measured by irradiating the laser toward the ground and detecting the time until the laser is reflected and returned. . Then, height distribution information data D4 relating to the height distribution of the predetermined area is generated from the measurement result.
[0040]
In step S12, using the imaging target object model estimated in step S5, one of the three-dimensional line segments generated in step S10 that is estimated to be a three-dimensional line segment related to one imaging target is corresponded. Grouping is performed (grouping), and a three-dimensional line group is generated (line group generation step). That is, a plurality of three-dimensional line segments existing in the vicinity of the object model are grouped into a three-dimensional line group. At this time, a three-dimensional line segment group may be generated using the height distribution information data D4 generated in step S11. For example, it is assumed that a part of a certain photographing object model is missing and has a shape different from that of an actual photographing object. Even in such a case, the height distribution near the imaging target included in the height distribution information data D4 well represents the shape of the actual imaging target. Therefore, it is preferable to use the height distribution information data D4 such that a three-dimensional line segment group is generated according to the height distribution. This makes it possible to suitably generate a three-dimensional line segment group even when the imaging target object model is not accurate.
[0041]
In step S13, the three-dimensional region generated in step S4 and the three-dimensional line segment generated in step S10 are projected onto a three-dimensional coordinate space. FIG. 9 is a diagram showing a three-dimensional line segment and a three-dimensional area projected on a three-dimensional coordinate space. In this figure, three-dimensional line segments 3021 to 3024 are determined to be in the vicinity of the three-dimensional region 301 and are grouped to form a three-dimensional line group. In FIG. 9, the three-dimensional line segments are clearly shown as one line segment 3021 to 3024, but the actual three-dimensional line segment is a fine line segment and is scattered near the line segments 3021 to 3024. ing.
[0042]
In step S14, among the three-dimensional line segments included in the three-dimensional line segment group generated in step S12, a plurality of the three-dimensional line segments related to the same side included in the object to be photographed are collected into one three-dimensional line. A minute is generated (smoothing step). FIG. 10 is a diagram illustrating an example of a spatial position when the three-dimensional region and the three-dimensional line group of the same imaging target projected on the three-dimensional coordinate space in step S13 are viewed from the X direction. The diagram shown in FIG. 10 shows a three-dimensional area 301 and a three-dimensional line segment, with the Y axis on the horizontal axis and the Z coordinate on the vertical axis. In FIG. 10, the three-dimensional line segments included in the three-dimensional line segment group are scattered in a range of about 42 m to about 47 m in height. In step 14, one line segment is determined for the scattered three-dimensional line segment by a predetermined method, and the line segment is set as a new three-dimensional line segment (smoothing). As a predetermined method, for example, the mode of the height of each of a plurality of three-dimensional line segments included in a certain three-dimensional line segment group may be set as the height of the smoothed three-dimensional line segment. In the case of the three-dimensional line group shown in FIG. 10, a new three-dimensional line segment may be generated at the height of the dotted line A (about 43 m).
[0043]
At this time, among the three-dimensional line segments smoothed in step S14, a plurality of three-dimensional line segments corresponding to the same object to be photographed may include those having different heights. In such a case, it is conceivable that the surface of the subject to be photographed is inclined. Therefore, in step S15, the height of the midpoint of each of the plurality of three-dimensional line segments is defined as the average height of the three-dimensional line segments. When the height difference between the respective average heights is larger than a predetermined value, May be determined to be inclined. Then, it is preferable to give the inclination obtained from the height difference to the corresponding three-dimensional area (inclination step).
[0044]
Also, the following method may be used as step S15. That is, the inclination is given to the three-dimensional region based on the inclination information obtained when associating the characteristic regions in step S3. Alternatively, the inclination is given to the three-dimensional area based on the inclination information obtained when associating the characteristic line segments in step S8. When associating characteristic lines, if there are characteristic lines having substantially the same position and different inclinations on the image data, for example, the inclination angle of the corresponding three-dimensional line is estimated. At the same time, an inclination angle at which the coordinates of the feature points at both ends match is obtained. Thus, the inclination of the three-dimensional line segment is obtained. Further, the same can be obtained for the three-dimensional area (inclination step).
[0045]
In step S16, the photographic object model generated in step S5 is corrected (fitting step), and a three-dimensional image (solid model) of the photographic object is generated by adding elements constituting a building. As an example of the method, first, the height of the corresponding three-dimensional region is corrected based on the height of the three-dimensional line group. As shown in FIG. 10, the height of the three-dimensional region is shifted on the order of meters from the actual height (the height of the smoothed three-dimensional line segment). Therefore, when generating a three-dimensional image, it is necessary to obtain the height of the three-dimensional region again. In step S16, using the three-dimensional line segment smoothed in step S13, the three-dimensional region is corrected so as to have the same height as the three-dimensional line segment. In the case of the three-dimensional area 301 shown in FIG. 10, the height of the three-dimensional area 301 is corrected to the same height as the smoothed three-dimensional line segment (dotted line A).
[0046]
Then, by using the three-dimensional area corrected based on the height of the three-dimensional line segment group, the imaging target model estimated in step S5 is corrected, and the three-dimensional image of the imaging target is added by adding the elements constituting the building. Generate an image (solid model). FIG. 11 shows an example of the corrected photographing object model. The imaging target object model 305 is formed by adding a side wall 304 which includes one side of the three-dimensional region 303 to the corrected three-dimensional region 303 and is a plane perpendicular to the ground (the plane of Z = 0). The three-dimensional line segments 3021a to 3024a are smoothed three-dimensional line segments used when correcting the three-dimensional region 303.
[0047]
In step S17, among the solid models generated in step S16, solid models having a close positional relationship are combined (model merging step). When a plurality of solid models are close to each other in the X direction and the Y direction, it can be said that those solid models relate to the same object to be photographed. In step S17, such solid models are combined with each other to generate a model that is substantially equal to the shape of the actual photographing target. At this time, it is customary to provide an interval of 50 cm or more due to restrictions on construction work and the like, and the proximity distance for discriminating a solid model relating to the same object to be photographed is generally set to 50 cm.
[0048]
The image processing method according to the present embodiment described above has the following effects. That is, in the image processing method according to the present embodiment, the imaging target object model is corrected by using the three-dimensional line group generated by grouping the three-dimensional line segments existing near the imaging target object model. Since the three-dimensional line segment has better spatial position accuracy than the three-dimensional region, by correcting the photographing object model in this way, a photographing object model that is a three-dimensional image can be generated with high accuracy.
[0049]
Further, it is preferable that the image processing method according to the present embodiment includes a model merging step of combining imaging object models located close to each other among imaging object models corrected by the fitting step. As a result, it is possible to suitably generate an imaging target object model even for an imaging target having a complicated shape.
[0050]
Further, in the image processing method according to the present embodiment, it is preferable that the line group generation step generates a three-dimensional line group using the height distribution information. Thus, even when the three-dimensional region has a large error with respect to the shape of the surface of the imaging target corresponding to the three-dimensional region, the three-dimensional line segments can be appropriately grouped.
[0051]
Further, the image processing method according to the present embodiment includes a plurality of three-dimensional lines related to the same side included in the imaging target among the three-dimensional line segments included in the three-dimensional line group generated in the three-dimensional line group generation step. And a smoothing step of generating one three-dimensional line segment by combining the minutes. The image processing method preferably includes a smoothing step. When a plurality of three-dimensional line segments are generated for one side of the object to be photographed, it is possible to specify one three-dimensional line segment for one side. it can.
[0052]
In the image processing method according to the present embodiment, the third step of the model estimation step generates a three-dimensional area by calculating three-dimensional coordinates of the associated characteristic area. Then, an imaging target object model is estimated from the three-dimensional area. The image processing method preferably includes such a third step, whereby the imaging target object model can be appropriately estimated.
[0053]
In the image processing method according to the present embodiment, it is preferable that the height of the three-dimensional region is corrected based on the height of the three-dimensional line group grouped by the imaging target model estimated from the three-dimensional region. . As a result, the upper surface of the object to be photographed can be modeled with high accuracy, so that a model of the object to be photographed can be generated with higher accuracy.
[0054]
Further, in the image processing method according to the present embodiment, when the height difference between the three-dimensional line segments included in the three-dimensional line group is equal to or more than a predetermined value, the three-dimensional region corresponding to the three-dimensional line group is inclined. Preferably, a tilting step is provided. Alternatively, the tilting step includes tilting to a three-dimensional region corresponding to the three-dimensional line segment group based on the three-dimensional region or the three-dimensional line segment tilt information obtained when associating the image of the imaging target included in the two-dimensional image. May be given. By providing such a tilting step, it is possible to suitably generate a shooting target model of a shooting target having a tilted surface.
[0055]
Here, an image processing program according to the above-described embodiment and a computer-readable recording medium (hereinafter, simply referred to as a recording medium) that stores the image processing program will be described. Here, the recording medium causes a change state of energy such as magnetism, light, electricity, or the like to occur in a reading device provided in a hardware resource of a computer in accordance with a description content of a program, and the recording medium corresponds to the change state. It is capable of transmitting the description content of the program to the reading device in the form of a signal. Examples of such a recording medium include a magnetic disk, an optical disk, a CD-ROM, and a memory built in a computer.
[0056]
FIG. 12 is a configuration diagram of the recording medium according to the present embodiment. The recording medium 50 has a program area 50a. The image processing program 51 is recorded in the program area 50a. The image processing program 51 is an image processing program for generating a three-dimensional image based on a plurality of two-dimensional images obtained by photographing a predetermined area on the ground including a photographing object from different viewpoints in the sky.
[0057]
As shown in FIG. 12, the image processing program 51 includes a main module 51a for controlling the processing, a first processing for extracting a characteristic region corresponding to the imaging target from each of a plurality of two-dimensional images, and the same imaging target. A second process for associating a feature region corresponding to the two-dimensional image with a two-dimensional image, and calculating a three-dimensional coordinate of the associated feature region to generate a three-dimensional region, thereby obtaining a three-dimensional model of the imaging target A model estimating module 51b for causing a computer to execute a model estimating process having a third process for estimating an object model, and a fourth process for extracting a characteristic line segment corresponding to a photographing object from each of a plurality of two-dimensional images, the same Fifth processing for associating a feature line segment corresponding to the object to be photographed between two-dimensional images, and calculating three-dimensional coordinates of the associated feature line segment to calculate the three-dimensional segment A line segment generation module 51c for causing a computer to execute a line segment generation process having a sixth process to be performed; and a plurality of three-dimensional line segments present in the vicinity of the imaging target model estimated by the model estimation process are set to the height of a predetermined area. A line group generation module 51d that causes a computer to execute a line group generation process of generating a three-dimensional line group by grouping using distribution information, and 3Ds included in the three-dimensional line group generated by the three-dimensional line group generation process. A smoothing module 51e that causes a computer to execute a smoothing process of generating a single three-dimensional line by combining a plurality of three-dimensional lines related to the same side included in the imaging target among the three-dimensional lines; When the height difference between the three-dimensional line segments included in the segment group is equal to or greater than a predetermined value, a gradient is given to the three-dimensional region corresponding to the three-dimensional line segment group. By correcting the height of the three-dimensional area based on the height of the three-dimensional line group grouped by the imaging target object model estimated from the three-dimensional area, the tilt module 51f that causes the computer to execute the tilt processing. A fitting module 51g for causing a computer to execute a fitting process for correcting a shooting object model, and a model merging process for combining shooting object models located close to each other among the shooting object models corrected by the fitting process. And a model merge module 51h to be executed.
[0058]
Here, instead of the above-described tilting process, the tilt module 51f performs the three-dimensional line segment group corresponding to the three-dimensional line segment group when the height difference between the three-dimensional line segments included in the three-dimensional line segment group is equal to or more than a predetermined value. The computer may execute a tilt process for giving a tilt to the region.
[0059]
The above-described image processing method is executed by causing a computer to read and execute the above-described image processing program.
[0060]
The image processing method, the image processing program, and the computer-readable recording medium storing the image processing program according to the present invention are not limited to the above-described embodiments, and various modifications are possible. For example, in the flowchart shown in FIG. 6, in step S12, three-dimensional line segments near the object model estimated in step S5 are grouped to generate a three-dimensional line group. When grouping the three-dimensional line segments, the three-dimensional line segments that exist near the three-dimensional region generated in step S4 may be grouped as the imaging target object model.
[0061]
In the above embodiment, the image data D2 and D3 are used for image processing. The image data may use D1 and D3 or D1 and D2 in addition to this combination. The present image processing method can be performed by selecting a suitable combination from D1 to D3.
[0062]
【The invention's effect】
An image processing method, an image processing program, and a recording medium for recording the image processing method according to the present invention use a three-dimensional line group generated by grouping three-dimensional line segments existing in the vicinity of the imaging target object model. Has been corrected. Since the three-dimensional line segment has better spatial position accuracy than the three-dimensional region, by correcting the photographing object model in this way, a photographing object model that is a three-dimensional image can be generated with high accuracy.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating an image processing apparatus 1 that implements an embodiment of an image processing method according to the present embodiment.
FIG. 2A is a diagram illustrating an arrangement of an imaging device on a flying object. FIG. 2B is a diagram illustrating an internal configuration of the imaging device 2.
FIG. 3 is a diagram illustrating a concept in which a photographing unit photographs a lower part from the sky.
FIG. 4 is a diagram showing the position of each line sensor at the moment when a certain point on a building is imaged.
FIG. 5A is an example of image data D2. FIG. 5B is an example of the image data D3.
FIG. 6 is a flowchart illustrating a method for generating a three-dimensional image, including the image processing method according to the present embodiment.
FIG. 7A is a diagram showing region data D2a generated in step S2. FIG. 7B is a diagram showing the area data D3a generated in step S2.
FIG. 8A is a diagram illustrating line segment data D2b obtained by extracting a characteristic line segment from image data D2 generated in step S7. FIG. 8A is a diagram showing line segment data D3b obtained by extracting a characteristic line segment from the image data D3 generated in step S7.
FIG. 9 is a diagram showing a three-dimensional line segment and a three-dimensional area projected on a three-dimensional coordinate space.
FIG. 10 is a diagram illustrating an example of a spatial position when a three-dimensional region and a three-dimensional line group of the same imaging target projected on a three-dimensional coordinate space in step S12 when viewed in the X direction.
FIG. 11 is a diagram illustrating an example of a corrected photographing target object model.
FIG. 12 is a configuration diagram of a recording medium according to the present embodiment.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... Image processing apparatus, 10 ... Computer, 11 ... Monitor, 12 ... Keyboard, 13 ... Mouse, 2 ... Imaging device, 20 ... Data processing part, 21 ... Hard disk, 27 ... Imaging part, 27F, 27N, 27B ... Line sensor , 28... Distance measuring unit, 28 a.

Claims (10)

An image processing method for generating a three-dimensional image based on a plurality of two-dimensional images obtained by shooting a predetermined area on the ground including a shooting target from different viewpoints in the sky,
A first step of extracting a characteristic region corresponding to the imaging target from each of the plurality of two-dimensional images, and a second step of associating the characteristic region corresponding to the same imaging target with the two-dimensional images A model estimation step including: a third step of estimating an imaging target object model that is a three-dimensional model of the imaging target object using the shape and spatial position of the associated characteristic region.
A fourth step of extracting, from each of the plurality of two-dimensional images, a characteristic line corresponding to the imaging target; and a fifth step of associating the characteristic line corresponding to the same imaging target with the two-dimensional images. A line segment generating step comprising: a step; and a sixth step of calculating three-dimensional coordinates of the associated characteristic line to generate a three-dimensional line segment.
A line group generation step of grouping a plurality of the three-dimensional line segments present in the vicinity of the imaging target model estimated by the model estimation step to generate a three-dimensional line group;
A fitting step of correcting the photographing object model based on the three-dimensional line segment group generated in the line group generation step. The image processing method according to claim 1, further comprising: a model merging step of combining the photographing object models located near each other among the photographing object models corrected by the fitting step. The image processing method according to claim 1, wherein in the line group generation step, the three-dimensional line group is generated using height distribution information of the predetermined area. Among the three-dimensional line segments included in the three-dimensional line segment group generated in the three-dimensional line segment group generation step, a plurality of the three-dimensional line segments related to the same side included in the imaging target are collectively collected into one. The image processing method according to claim 1, further comprising a smoothing step of generating a three-dimensional line segment. The method according to claim 1, wherein the third step of the model estimating step estimates the imaging target object model by calculating three-dimensional coordinates of the associated characteristic region and generating a three-dimensional region. 2. The image processing method according to 1., The fitting step corrects the height of the three-dimensional region based on the height of the three-dimensional line group grouped by the three-dimensional object model estimated from the three-dimensional region, and thereby adjusts the height of the three-dimensional region. The image processing method according to claim 5, wherein the model is modified. A step of inclining the three-dimensional area corresponding to the three-dimensional line segment group when a height difference between the three-dimensional line segments included in the three-dimensional line segment group is equal to or more than a predetermined value. The image processing method according to claim 5, wherein: Tilting the three-dimensional region corresponding to the three-dimensional line segment group based on the three-dimensional region or the three-dimensional line segment tilt information obtained when associating the image of the imaging target included in the two-dimensional image The image processing method according to claim 5, further comprising a tilting step of: An image processing program for causing a computer to execute the image processing method according to claim 1. A computer-readable recording medium having recorded thereon an image processing program for causing a computer to execute the image processing method according to claim 1.

Images