JP4099776B2 - 3D model creation device, 3D model creation method, and 3D model creation program - Google Patents

Description

  The present invention relates to a three-dimensional model creation apparatus, a three-dimensional model creation method, and a three-dimensional model creation program, and more particularly to a system, method, and program for creating a three-dimensional polygon of a building by performing stereo processing using aerial photographs.

  A technique is known in which the height of a building is calculated by performing stereo processing on image data generated from an aerial photograph, and a three-dimensional polygon model of a city is automatically generated (see, for example, Patent Document 1). . FIG. 1 is a block diagram showing a configuration of a conventional city model generation system described in Patent Document 1. Referring to FIG. 1, a conventional city model generation system performs a stereo matching process on a satellite image storage unit 101 that stores satellite image data and satellite image data obtained from the satellite image storage unit 101. A stereo processing unit 102 that generates data, and erroneous data such as noise and loss in the three-dimensional data obtained by the stereo processing unit 102 are automatically used by using outline information such as buildings obtained from map data. A DEM data automatic correction unit 103 for correction and a map data storage unit 104 for providing map data such as the external shape of a building to the DEM data automatic correction unit 103 are configured.

  In this city model generation system, the satellite image storage unit 101 stores a plurality of images obtained by photographing the same point on the ground from the satellite from different viewpoints, that is, satellite stereo images. These satellite stereo images are stored in the stereo processing unit 102. Given to. The stereo processing unit 102 automatically performs stereo matching processing on the obtained satellite stereo image, and generates three-dimensional data indicating the topography around the captured point. Specifically, this is indicated by a height value corresponding to each point on the two-dimensional map.

  Here, the stereo matching process is to obtain corresponding points in each image capturing the same point for two images taken from different viewpoints, and use the parallax to obtain the target by the principle of triangulation. Determining depth and shape. In general, various methods such as a method of obtaining and associating feature amounts and a correlation method using correlation between left and right images have been proposed.

  The DEM data automatic correction unit 103 mainly uses the outline information of the building stored in the map data storage unit 104 and included in the region corresponding to the three-dimensional data, with respect to the three-dimensional data obtained by the stereo matching process. Incorrect data in the three-dimensional data is automatically corrected using the map data. In the DEM data automatic correction unit 103, the alignment unit 31 superimposes the three-dimensional data and the map data so that the points indicating the same coordinates coincide. If the geodetic system used in the three-dimensional data and the satellite image used as the source is different from the geodetic system used in the map data, the longitude and latitude values at the same point will be different, so they will be equivalent. Is converted to, and the conversion parameters are obtained using the information of the predetermined points with the same latitude and longitude, and the transformation such as affine transformation is applied to either, and the three-dimensional data and the map data are overlapped so that the same point matches. Match. For each building area in the overlapped map data, the three-dimensional data included therein is set as a building candidate area. Further, a statistical analysis such as obtaining a histogram for the value of the three-dimensional data included in each of the set building candidate areas is performed.

  For each region, select several values that indicate high frequency, and replace the selected point values with the most frequent values in the three-dimensional data in the vicinity of the selected points. To correct the 3D data. That is, select some values with high frequency from the histogram of each region as the representative value of the region, and among the pixels in the region, those having the value of DEM data within a predetermined threshold range from the representative value, A label corresponding to the representative value is added.

  After that, for each pixel remaining without being associated with the representative value, the distribution of labels added to neighboring pixels is examined, and a label with high frequency is selected as the label of the pixel. When labels are determined for all pixels in the building area, the value of each pixel is replaced with a representative value set in the label. As described above, since the three-dimensional data is automatically obtained on the computer by the stereo processing unit 102 and the DEM data automatic correction unit 103, no operator operation is required. The three-dimensional building polygon generated by the conventional technique has a large divergence from the actual landscape. In order to use the created three-dimensional building polygon for visualization application, an image of the building wall surface is obtained, and the 3 It needs to be pasted on a 3D building polygon.

  In order to use the created three-dimensional building polygon for visualization application, a technique for automatically generating image data of a building wall surface is desired.

JP 2002-157576 A

  The problem to be solved by the present invention is to provide a system that can automatically generate a city model that reflects information on a building wall surface from only aerial photographs, without requiring the operation of taking and cutting out a wall image. There is to do.

  The means for solving the problem will be described below using the numbers used in [Best Mode for Carrying Out the Invention]. These numbers are added to clarify the correspondence between the description of [Claims] and [Best Mode for Carrying Out the Invention]. However, these numbers should not be used to interpret the technical scope of the invention described in [Claims].

  An image data storage unit (1) that stores image data (1a) obtained by converting a plurality of continuously shot images into electronic data, and the adjacent image data (1a) of the images includes images of the same subject. Use what includes. Then, a side data output unit that extracts and outputs side data (7a) that is image data of a portion corresponding to a surface constituting the depth direction of the subject from the image data (1a), and the side data (7a). ) To generate a texture, and a 3D model creation device including a texture generation unit that attaches the texture to the 3D polygon data generated based on the image data (1a). A three-dimensional stereoscopic image corresponding to the captured image is generated.

  In the three-dimensional model creation device, the side data output unit includes a stereo processing unit (2) and an orthoprocessing unit (31), and the stereo processing unit (2) calculates the depth information of the subject. . The orthoprocessing unit (31) generates an ortho image that is an orthographic image of the subject based on the image data (1a) and the depth information, and the contour of the subject of the image data (1a). The side data (7a) is generated based on the pixels of the portion and the pixels of the contour portion of the subject of the ortho image. The texture generation unit includes a label image generation unit (8), a side surface information generation unit (10), and a wall surface texture generation unit (12), and the label image generation unit (8) includes the depth information. Generating reference image data based on the three-dimensional polygon data (6a) of the subject generated based on the image, and specifying a side region corresponding to the side surface of the three-dimensional polygon data (6a) in the reference image data; A label image (9a) is generated by adding individual label values to the pixels in the side region. The side information creation unit (10) generates side information (11a) used for generating the texture based on the side data (7a) and the label image (9a). The wall surface texture generation unit (12) generates the texture based on the side surface information (11a), and attaches the texture to the side surface of the three-dimensional polygon data (6a). A three-dimensional stereoscopic image is generated, and the three-dimensional stereoscopic image is used for a car navigation screen or the like.

  In the three-dimensional model creation device, the side data output unit includes a stereo processing unit (2) and an orthoprocessing unit (31), and the stereo processing unit (2) calculates the depth information of the subject. . The orthoprocessing unit (31) generates an ortho image that is an orthographic image of the subject based on the image data (1a) and the depth information, and the contour of the subject of the image data (1a). The side data (7a) is generated based on the pixels of the portion and the pixels of the contour portion of the subject of the ortho image. The texture generation unit includes a label image generation unit (8), a side surface information generation unit (10), and a wall surface texture generation unit (12), and the label image generation unit (8) includes the depth information. Generating reference image data based on the three-dimensional polygon data (6a) of the subject generated based on the image, and specifying a side region corresponding to the side surface of the three-dimensional polygon data (6a) in the reference image data; A label image (9a) is generated by adding individual label values to the pixels in the side region. The side information creating unit (10) generates side information (11a) used for generating the texture based on the label image (9a). The wall surface texture generation unit (12) generates the texture based on the side surface information (11a), and uses the 3D model creation device to attach the texture to the side surface of the 3D polygon data (6a). A stereoscopic image is generated, and the three-dimensional stereoscopic image is used for a car navigation screen or the like.

  In the three-dimensional model creation apparatus, the texture generation unit further includes a standard side image storage unit (13), and the standard side image storage unit (13) stores a plurality of side image data (13a) in advance. The side surface information creation unit (10) selects data most similar to the side surface information (11a) from the plurality of side surface image data (13a) as the most similar data, and the texture is based on the most similar data. A three-dimensional model creating apparatus that uses the generated one generates a three-dimensional stereoscopic image, and uses the three-dimensional stereoscopic image for a car navigation screen or the like.

  In the three-dimensional model creation device, the stereo processing unit (2) specifies each of the adjacent image data (1a) as a first image and a second image, and determines the first image and the second image. The depth information is calculated by performing a stereo matching process. The orthoprocessing unit (31) generates first side data (7a) corresponding to the first image and second side data (7a) corresponding to the second image. Further, the label image generation unit (8) generates a first label image corresponding to the first image and a second label image corresponding to the second image. The side information creation unit (10) creates first side information based on the first side data (7a) and the first label image, and the second side data (7a) and the second label. Based on the image, the second side information is created, the first side information is compared with the second side information, and a three-dimensional model creation device that creates the side information generates a three-dimensional stereoscopic image The three-dimensional stereoscopic image is used for a car navigation screen or the like.

  Corresponding to a step of reading adjacent image data (1a) including an image of the same subject from image data (1a) obtained by converting a plurality of continuously photographed images into electronic data, and a plane constituting the depth direction of the subject Extracting and outputting side data (7a); generating a texture using the side data (7a); and three-dimensional polygon data generated based on the image data (1a). The above three-dimensional stereoscopic image is generated by a program capable of executing the method including the step attached to (6a) by a computer.

  In the program, calculating the depth information of the subject, generating an ortho image that is an orthographic image of the subject based on the image data (1a) and the depth information, and the image data (1a) Generating the side data (7a) based on the pixel of the contour portion of the subject and the pixel of the contour portion of the subject of the ortho image, and 3 of the subject generated based on the depth information Generating reference image data based on the three-dimensional polygon data (6a), identifying a side region corresponding to the side surface of the three-dimensional polygon data (6a) in the reference image data, Adding a label value to generate a label image (9a), the side data (7a) and the label image (9 ) Generating side information (11a) used for generating the texture, generating the texture based on the side information (11a), and converting the texture into the three-dimensional polygon data (6a). 3D images are generated by a program that can be executed by a computer.

  In the program, calculating the depth information of the subject, generating an ortho image that is an orthographic image of the subject based on the image data (1a) and the depth information, and the image data (1a) Generating the side data (7a) based on the pixel of the contour portion of the subject and the pixel of the contour portion of the subject of the ortho image, and 3 of the subject generated based on the depth information Generating reference image data based on the three-dimensional polygon data (6a), identifying a side region corresponding to the side surface of the three-dimensional polygon data (6a) in the reference image data, Adding a label value to generate a label image (9a), and based on the label image (9a), A step of generating side information (11a) used for generating a tea; and the wall surface texture generation unit (12) generates the texture based on the side surface information (11a), and the texture is converted into the three-dimensional polygon. A three-dimensional stereoscopic image is generated by a computer-executable program that includes the step of attaching to the side of the data (6a).

  In the program, a step of selecting data most similar to the side information (11a) as the most similar data from a plurality of side image data (13a) stored in advance, and a step of generating based on the most similar data A three-dimensional stereoscopic image is generated by a program capable of executing the provided method by a computer.

  In the program, each of adjacent image data (1a) is specified as a first image and a second image, and the depth information is calculated by performing stereo matching processing on the first image and the second image. Generating first side data (7a) corresponding to the first image, second side data (7a) corresponding to the second image, and a first label image corresponding to the first image; Generating a second label image corresponding to the second image, creating first side information based on the first side data (7a) and the first label image, and generating the second side image. Creating the second side information based on the data (7a) and the second label image, and comparing the first side information and the second side information to create the side information; Compu method By a program executable by a data, to generate a three-dimensional stereoscopic images.

  Corresponding to a step of reading adjacent image data (1a) including an image of the same subject from image data (1a) obtained by converting a plurality of continuously photographed images into electronic data, and a plane constituting the depth direction of the subject Extracting and outputting side data (7a); generating a texture using the side data (7a); and three-dimensional polygon data generated based on the image data (1a). A three-dimensional stereoscopic image is generated by the three-dimensional model creation method including the step attached to (6a).

  In the three-dimensional model creation method, calculating the depth information of the subject, generating an ortho image that is an orthographic image of the subject based on the image data (1a) and the depth information, and the image Generating the side data (7a) based on the pixel of the contour portion of the subject of the data (1a) and the pixel of the contour portion of the subject of the ortho image, and generated based on the depth information Generating reference image data based on the three-dimensional polygon data (6a) of the subject; identifying a side region corresponding to the side surface of the three-dimensional polygon data (6a) in the reference image data; A step of generating a label image (9a) by adding individual label values to the pixels, the side data (7a) and the label Generating the side information (11a) used for generating the texture based on the image (9a), generating the texture based on the side information (11a), and converting the texture into the three-dimensional polygon A three-dimensional stereoscopic image is generated by a three-dimensional model creation method including the step of attaching to the side of the data (6a).

  In the three-dimensional model creation method, calculating the depth information of the subject, generating an ortho image that is an orthographic image of the subject based on the image data (1a) and the depth information, and the image Generating the side data (7a) based on the pixel of the contour portion of the subject of the data (1a) and the pixel of the contour portion of the subject of the ortho image, and generated based on the depth information Generating reference image data based on the three-dimensional polygon data (6a) of the subject; identifying a side region corresponding to the side surface of the three-dimensional polygon data (6a) in the reference image data; Generating a label image (9a) by adding individual label values to the pixels, and based on the label image (9a) The step of generating side surface information (11a) used for generating the texture, and the wall surface texture generation unit (12) generate the texture based on the side surface information (11a), and the texture is the three-dimensional A three-dimensional stereoscopic image is generated by a three-dimensional model creation method including the step of attaching to the side of the polygon data (6a).

  In the three-dimensional model creation method, a step of selecting data most similar to the side information (11a) as the most similar data from a plurality of side image data (13a) stored in advance, and based on the most similar data A three-dimensional stereoscopic image is generated by a three-dimensional model creation method including a generating step.

  In the three-dimensional model creation method, each of adjacent image data (1a) is specified as a first image and a second image, and the depth information is obtained by performing stereo matching processing on the first image and the second image. A first side data (7a) corresponding to the first image, a second side data (7a) corresponding to the second image, and a first side corresponding to the first image. Based on the step of generating one label image and a second label image corresponding to the second image, the first side data (7a) and the first label image, first side information is created, Based on the second side data (7a) and the second label image, second side information is created, and the first side information is compared with the second side information to create the side information. Tertiary with steps The modeling method, for generating a 3D stereoscopic image.

  In the three-dimensional model creation device, the program, and the three-dimensional model creation method, the plurality of continuously photographed images are photographed by an imaging device mounted on a flying object flying in the sky, in particular, an aircraft having fixed wings or rotary wings. Use what was done. The subject is a structure on the ground photographed by the imager, and the depth information is information indicating the height of the structure. Further, the side surface data (7a) generates a three-dimensional stereoscopic image by a three-dimensional model creation device, a program, and a three-dimensional model creation method, which are data indicating the state of the wall surface of the structure, particularly color and texture. The three-dimensional stereoscopic image is used for a car navigation screen or the like.

  In the three-dimensional model creation device, program, and three-dimensional model creation method, the surface constituting the depth direction of the subject is a surface parallel to the optical axis direction for capturing the image, the program, and 3 A three-dimensional stereoscopic image is generated by a three-dimensional model creation method, and the three-dimensional stereoscopic image is used for a car navigation screen or the like.

  According to the present invention, it is possible to automatically generate a city model reflecting information on a building wall surface from only an aerial photograph, without requiring a wall surface image capturing or clipping operation.

Next, the best mode for carrying out the invention will be described in detail with reference to the drawings.
[Configuration of First Embodiment]
The present invention will be described using an example in which an aerial photograph of a city taken from above is used as a two-dimensional image.
Referring to FIG. 2, the first embodiment of the present invention includes an aerial photograph data storage unit 1, a stereo processing unit 2, an orthoprocessing unit 31, a polygon creation unit 4, a map data storage unit 5, 3D polygon data storage unit 6, wall surface candidate image storage unit 7, label image generation unit 8, label image storage unit 9, wall surface area information extraction unit 10, building wall surface attribute storage unit 11, wall surface texture It is comprised from the production | generation part 12. FIG.

  The aerial photograph data storage unit 1 is an information storage function block. A plurality of aerial photographs taken consecutively with an overlap are stored. The stored aerial photographs are stored as aerial photograph data 1a each converted into image data. In the following description, the aerial photograph data 1a stored in the aerial photograph data storage unit 1 is described as an example of image data generated from an aerial photograph taken by a photographing device mounted on an aircraft. It is not intended to specify a photographing apparatus for taking a photograph of the invention and an object on which the photographing apparatus is mounted.

  The stereo processing unit 2 reads the aerial photograph data 1a stored in the aerial photograph data storage unit 1, performs stereo matching of overlapping regions, and generates a DEM (Digital Elevation Model: height information image) in units of pixels. It is a functional block. Stereo matching processing refers to finding the corresponding points in each image capturing the same image for two images taken from different viewpoints, and using the parallax to determine the depth to the target, This is a process for obtaining a shape. There are various methods for performing stereo matching processing, such as a method for obtaining a general feature value to be associated, and a method for obtaining a correlation between left and right images, and the stereo matching in the present embodiment. There is no limit to the technique used for processing.

  The orthoprocessing unit 31 is an information processing function block that executes the orthosing process on the aerial photograph data based on the DEM generated by the stereo processing unit 2. The DEM calculated by the stereo processing unit 2 is given to the orthoprocessing unit 31 together with the original aerial photograph data. The ortho-processing unit 31 obtains the three-dimensional position of each pixel by DEM. Therefore, the ortho-processing unit 31 performs conversion for rearranging the pixel by pixel so that it is orthogonally projected to the ground. Then, new image data is generated by re-projecting the aerial photograph data as viewed from above at all points in the image. Hereinafter, this conversion is referred to as ortho-rectification, the DEM after the ortho-processing is referred to as ortho-DEM, and the aerial photograph data after the ortho-processing is referred to as ortho-image.

  In addition, the orthorectification processing unit 31 generates the ortho image and generates wall surface candidate data 7a indicating a region in the original image that is not used in the ortho image as a wall surface candidate region. The wall surface candidate area referred to here is an area of the original image that is not used because it overlaps in the correspondence relationship between the original image and the ortho image obtained at the time of performing orthorectification.

  The wall surface candidate image storage unit 7 is an information storage function block that stores the wall surface candidate data 7 a created by the orthoprocessing unit 31. The map data storage unit 5 is an information storage function block that stores map data 5a. The polygon creation unit 4 superimposes the DEM created by the stereo processing unit 2 and the map data 5a representing the shape of the building with a plane polygon, and statistically processes the height of the DEM data inside each building area. The three-dimensional polygon data 6a of the building is generated by setting the height of the polygon. The three-dimensional polygon data storage unit 6 is an information storage function block that stores the three-dimensional polygon data 6a.

  The label image generation unit 8 is an information processing function block that creates label image data 9a corresponding to the three-dimensional polygon data 6a. The label image generation unit 8 draws a city model image from a viewpoint that matches the aerial photograph using the three-dimensional polygon data 6a of the building created by the polygon creation unit 4 and terrain information (not shown), and each building polygon model Label image data 9 a in which individual labels are written in the wall surface area of the image data is created and output to the label image storage unit 9. The label image storage unit 9 is an information storage function block that stores the label image data 9 a output from the label image generation unit 8.

  The wall surface area information extraction unit 10 superimposes the wall surface candidate data 7a stored in the wall surface candidate image storage unit 7 and the label image data 9a created by the label image generation unit 8 to obtain a wall surface area for each building. This is an information processing function block that determines and extracts a texture image of each building wall surface or a wall surface attribute such as a representative color. The wall surface area information extraction unit 10 outputs the extracted wall surface attribute to the building wall surface attribute storage unit 11 as wall surface attribute data 11a. The attribute here is a texture image or a region representative color. The building wall surface attribute storage unit 11 is an information storage function block that stores wall surface attribute data 11a that is a wall surface attribute such as a representative color or a texture image of each building obtained by the wall surface region information extraction unit 10.

  The wall surface texture generation unit 12 is an information processing function block that processes the 3D polygon data 6 a created by the polygon creation unit 4 with reference to the building wall surface attribute information included in the building wall surface attribute storage unit 11. Specifically, operations such as attaching or coloring a wall texture to the corresponding building model are performed.

[Operation of First Embodiment]
Next, the overall operation of the present embodiment will be described in detail with reference to the flowchart of FIG.
In step S101, two corresponding aerial photograph data 1a photographed with an overlap are provided to the stereo processing unit 2 from a plurality of aerial photograph data 1a stored in the aerial photograph data storage unit 1. The stereo processing unit 2 performs conversion (parallelization) so that all corresponding points in the two images are on the same scanning line through orientation calculation for calculating the geometry such as the shooting position and direction based on the acquired corresponding points. The correspondence calculation is performed by DP matching, and the DEM for each pixel is calculated.

  In step S102, the DEM calculated by the stereo processing unit 2 is given to the orthoprocessing unit 31 together with the original aerial photograph data 1a. Since the three-dimensional position of each pixel is obtained by the DEM, conversion is performed for both the DEM and the aerial photograph so that the pixel is rearranged in units of pixels so as to be orthogonally projected to the ground. In combination with this orthorectification processing, a region in the original image that has not been used in the ortho image is extracted as a wall surface candidate region, and wall surface candidate image creation processing for generating wall surface candidate data 7a as an image indicating this is performed (step S103). ).

  When orthorectification is performed in a state where the DEM is correctly obtained, a building having a side surface perpendicular to the ground is converted so that the roof surface moves to a position where it overlaps the bottom surface, and the side surface becomes invisible in the image. Since the pixels constituting the side surface move in a concentrated manner on the side of the upper surface of the building, they are unnecessary in the ortho image. For example, a binary image created by giving 1 as a significant pixel value to such a pixel and giving 0 as a different pixel value to another pixel is generated, and this is used as wall surface candidate data 7a. The generated wall surface candidate data 7 a is stored in the wall surface candidate image storage unit 7.

  In step S104, processing for generating three-dimensional polygon data 6a of the building is performed using the DEM created by the stereo processing unit 2 and the map data 5a representing the shape of the building with a plane polygon. Specifically, a plane polygon indicating each building shape is superimposed on the DEM, and statistical processing such as taking a mode value from the DEM data inside each building area is performed to obtain the height of each building. Reflecting this height, the three-dimensional polygon of each building is generated as a polygonal column by diverting the map polygon to the planar shape.

  In step S105, using the generated three-dimensional building polygon and a separately prepared terrain polygon indicating the terrain of the surrounding area, an image in which each polygon is drawn from the same viewpoint as the aerial photograph shooting viewpoint is generated. A label image generation process for generating an image in which individual label values are written is performed on the pixel at the position where the polygonal wall surface is drawn. The topographic model data used at this time is a three-dimensional polygon that shows the shape of the surrounding area of the building. For example, the topographic model data is created from polygons based on a 50m mesh elevation published by the Geospatial Information Authority of Japan, or from measurement data obtained by a laser profiler Can be used. In addition, the aerial photography viewpoint position information uses a value determined by the orientation process performed during the stereo process.

  In step S106, the wall surface candidate information 7a created in step S103 and the label image data 9a created in step S105 are overlapped to determine the wall pixel area of each building, and the wall surface area information extraction for extracting attribute information Processing is performed. Specifically, an area having a label corresponding to each building in the label image and having a significant value in the wall surface candidate data 7a is determined as the wall surface pixel area of the building, and is useful for texture generation from the area. The correct attributes. The attribute here refers to a texture image or a region representative color, and any or a plurality of attributes are acquired and stored in the building wall surface attribute storage unit 11 according to the area or color distribution of the region.

  In step S107, the corresponding building polygon data in the three-dimensional polygon data 6a is processed using the wall surface attribute information created in step S106 and stored in the building wall surface attribute storage unit 11, and a polygon close to a real landscape is obtained. A wall texture generation process is performed. A specific processing method is selected according to the obtained wall surface attribute data. For example, for a building with a texture, the texture is directly attached to the polygon, including the wall surface in the direction that is not reflected in the aerial photograph. For buildings that have only representative colors as attributes, either directly change the material data of the building polygons according to the representative color and color the polygons themselves, or color the default building wall image prepared in advance with the representative colors. A polygon is processed by attaching a texture in the same manner as described above.

  As a result, a pixel area that is not used for ortho-rectification using a dense DEM in pixel units obtained by stereo processing from an aerial photograph, and a three-dimensional city model generated from the DEM and map data, are the same as the aerial photograph. Because it is configured to specify the wall area of each building from the viewpoint label image and to process the wall surface of the 3D polygon data of the building using the wall attribute extracted from that area, the wall image in the aerial photograph It is possible to automatically obtain 3D polygon data of a building having a wall texture or color that makes use of the above-mentioned attributes, and a city model close to a real landscape can be generated.

  Hereinafter, the above operation will be described in detail using a specific example. FIG. 4 is a schematic diagram showing an aerial photography method used in the present embodiment. Referring to FIG. 4, an aerial photograph similar to that used for aerial survey is taken from above the target urban area. At this time, as shown in FIG. 4, a series of aerial photographs in which the imaging range including the target area is overlapped by about 60% are taken continuously while keeping a constant altitude and flying at a constant course and speed. obtain. When these are obtained as silver salt photographs, they are scanned at the same resolution to obtain image data.

  For the obtained aerial photograph image pair, the coordinates of corresponding points necessary for the orientation processing for determining the shooting angle of the aerial photograph are acquired. Specifically, in the overlapping areas in the left and right images, six sets of points where the same object exists are selected, and coordinate values in the respective images are acquired. It is desirable to obtain these coordinate pairs as dispersedly as possible, for example, at the four corners and edges of the overlapping area. For these four points, the latitude and longitude values at the corresponding points are acquired and stored using, for example, map software capable of displaying longitude and latitude coordinates in the Tokyo geodetic system.

  By using the orientation calculation similar to the aerial survey based on these orientation point information, orientation information, that is, the shooting viewpoint position of the aerial photograph pair, the vector between the shooting viewpoints, the direction of the shooting axis with respect to the ground coordinate system, etc. Can be obtained. There are no particular restrictions on this orientation calculation method.

  Using the obtained photographing position relationship, the aerial photograph pair is converted so that an arbitrary pair of corresponding points is located on the same scanning line. This operation is called parallelization, and is applied by considering the common plane parallel to the vector between the viewpoints and projecting the original aerial photograph pair to that plane. The method of parallelization is not limited to this, and for example, epipolar lines may be obtained for each point in the image, and rearrangement may be performed so that these are parallel in pixel units.

  Since the parallel image pair can limit the correspondence search to one dimension on the same scanning line, stereo matching in units of scanning lines can be applied. Although there is no restriction on the method used for matching, for example, if DP (Dynamic Programming) is used, a pixel unit correspondence can be obtained between image pairs for each scanning line of the left and right images. . The position of each point in the three-dimensional space is calculated using the parallax determined for each pixel by this DP matching, and rearranged as an image parallel to the ground surface to generate a DEM (Digital Elevation Model) for each pixel To do.

Since this pixel-based DEM can obtain the three-dimensional position of each pixel in the original aerial photograph, it can be rearranged so that each pixel is orthogonally projected with respect to the ground. This conversion is an operation called orthorectification, and the aerial photograph after orthorectification processing is called an ortho image.
Since an aerial photograph is an image of a central projection taken from a single viewpoint, the photographed building is taken down so as to move away from the center of the camera, and application such as overlaying with a map is not possible. By performing the above-mentioned orthorectification processing in units of pixels, an image that can be superimposed on a map can be generated, and the present invention can be used for computer graphics applications such as obtaining textures on the ground or a building roof.

  In the situation where the DEM is correctly obtained, when performing the orthorectification processing in units of pixels, the pixels in which the building wall surface vertical to the ground is imaged in the original image are as shown in FIG. Therefore, most of the pixels are not used in the ortho image. Pixels that are not mapped due to duplication during such ortho-conversion are regarded as wall surface candidates, detected during ortho-rectification, and for example, 1 is assigned as a significant pixel value to the wall surface candidate pixels, and 0 is assigned to other pixels, for example. Create and store a binary image. A binary image created based on this original image is referred to as wall surface candidate data 7a.

  Also, the DEM itself obtained in pixel units is arranged in an orthogonal projection with respect to the ground surface, and can be overlaid with the map data. For this reason, it is possible to superimpose the shape of the building on the vector map data having planar polygon data, and obtain the height of each building using the DEM value inside each area. Specifically, a plane polygon indicating each building shape is superimposed on the DEM, and statistical processing such as obtaining a mode value is performed on the DEM data inside each building area, for example, to obtain the height of each building. By using this height as the height of a polygonal column having the planar shape of the map data as the bottom surface, a three-dimensional polygon of each building can be generated.

By using polygon data indicating the topography of the surrounding area together with the polygons of each building, a model expressing a city with a three-dimensional polygon can be generated. The terrain model data used in this case is a three-dimensional polygon indicating the shape of the surrounding area of the building. For example, the coordinates are obtained based on numerical data representing the terrain such as a numerical map 50 m mesh elevation published by the Geospatial Information Authority of Japan. Can be connected in order to create a triangular patch and convert it into a terrain polygon. FIG. 6 shows a schematic diagram of processing from the above DEM and map data to creation of city model data.
Based on these city model data, an image in which each polygon is drawn from the same viewpoint as the aerial photography viewpoint is generated, and the individual label values that differ for each building for the pixel at the position where the wall of each building polygon is drawn The image which wrote is generated. This is called a label image. As the aerial photograph photographing viewpoint position information at this time, a value determined by the orientation process performed during the stereo process is used.

  As shown in FIG. 7, this label image and the already created wall surface candidate data 7a are superimposed, have pixels associated with individual buildings in the label image, and have a significant value in the wall surface candidate data 7a. The area where the pixels are present is determined as the wall surface pixel area of each building. The original aerial photograph pixels corresponding to the area are analyzed, and attributes useful for texture generation such as texture images and area representative colors are acquired. There is no restriction on the analysis means and its selection rule. For example, when a wall surface pixel area having a predetermined number of pixels or more is obtained, the entire wall surface area is converted into a rectangle and appropriately interpolated to obtain a texture image. When the predetermined number of pixels is not exceeded, the pixel value distribution of the pixels in the area is examined, the average value is calculated, and the area representative color can be obtained. A plurality of pixel attributes may be acquired by a plurality of analysis means. The wall surface area information of each building obtained by such a method is stored in the building wall surface attribute storage unit 11 in a form associated with each building.

  Finally, according to the wall surface attribute information obtained by the above method, the created building polygon is processed to obtain data close to the actual landscape. The specific processing method is selected depending on the acquired wall surface attribute data. For example, in the case of a building where a texture image is obtained, including the wall surface in the direction that does not appear in the aerial photograph, Attach the texture to the polygon wall as it is. For buildings where texture images are not obtained and only representative colors are obtained, the material data indicating the material of the building polygons is directly changed according to the representative color and the polygons themselves are colored, or in advance Polygon data is processed by a method of creating a texture image by coloring the default building wall image prepared in step 1 with a representative color and attaching it to the building wall surface. This makes it possible to automatically generate a city model that reflects building wall information from only aerial photographs.

[Configuration of Second Embodiment]
Next, a second embodiment of the present invention will be described in detail with reference to the drawings. FIG. 8 is a block diagram showing the configuration of the second exemplary embodiment of the present invention. Referring to FIG. 8, the configuration of the second embodiment is a configuration in which a standard wall surface image storage unit 13 is added to the same configuration as that of the first embodiment. The standard wall surface image storage unit 13 stores in advance a plurality of types of standard wall surface texture images corresponding to high-rise buildings, office buildings, condominiums, individual houses, and the like together with IDs corresponding to the respective standard wall surface texture images. It is an information storage function block.

  Similar to the case of the first embodiment, the wall surface area information extraction unit 14 superimposes the wall surface candidate data 7a stored in the wall surface candidate image storage unit 7 and the label image data 9a, and creates a wall surface for each building. The area is determined, the representative color of each area is determined, each wall area is compared with the standard wall texture image stored in the standard wall image storage unit 13, and the standard wall texture most similar to each wall area is determined. Determine the image. The determined texture image ID is stored in the building wall surface attribute storage unit 11 as an attribute of each building together with the representative color of each region. The building wall surface attribute storage unit 11 stores the attribute information of each building obtained by the wall surface region information extraction unit 14, that is, the ID of the standard wall surface texture image most similar to the representative color.

  The wall surface texture generation unit 12 colors the standard wall surface texture image with the representative color of each region determined by the wall surface region information extraction unit 14 from the ID of the standard wall surface texture image determined by the wall surface region information extraction unit 10. Then, it is attached to the 3D polygon wall surface of the building in the 3D polygon data 6a. The other functional blocks in the best mode for carrying out the second invention of the present invention operate in the same manner as the corresponding functional blocks in the first embodiment.

[Operation of Second Embodiment]
Next, the operation of the second embodiment will be described in detail with reference to the flowchart of FIG.
Since the processing from step S201 to step S205 is the same as that from step S101 to step S105 in the first embodiment, a description thereof will be omitted.

  Subsequent to step S205, the wall surface candidate information 7a and the label image data 9a are overlapped to determine the wall surface pixel area of each building, and wall surface area information extraction processing for extracting attribute information is performed (step S206). In step S <b> 206, the wall surface area information extraction unit 14 performs a process of necessarily extracting the area representative color attributes for all buildings. The extracted representative color attribute is stored in the building wall surface attribute storage unit 11.

  In step S207, the wall surface pixel region of each building associated in step S206 is compared with the standard wall surface texture image group stored in advance in the standard wall surface image storage unit 13, and is considered to be most similar to the wall surface region. A standard wall surface image comparison process for determining a standard wall surface texture image to be performed is performed. The standard wall surface texture image included in the standard wall surface image storage unit 13 has one or more images for each type such as a high-rise building, an office building of several floors, a condominium, and an individual house, and each has a unique ID. And stored in advance. Each of these images and the wall surface pixel region converted into a rectangle are compared by performing texture analysis, and the ID of the standard wall surface texture image that seems to be the most similar is stored in the building wall surface attribute storage unit 11 as corresponding to the building. Remember. The texture analysis method used at this time is not particularly limited, and for example, a method of selecting a standard wall texture image having the closest statistics such as average and variance obtained from the density histogram may be used.

  In step S208, the corresponding building polygon data in the three-dimensional polygon data 6a is processed using the wall surface attribute information created in step S206 and step S207 and stored in the building wall surface attribute storage unit 11 to obtain a real landscape. The wall surface texture generation process for making a near polygon is performed. Specifically, a texture image in which a pixel having a value close to the representative color in the standard wall texture image closest to each region is replaced with a representative color in the wall attribute is generated, and the three-dimensional polygon of the building is converted into an aerial photograph. Attach the wall surface in the direction not shown.

  As a result, the standard wall image facilitated in advance is colored with the representative color attribute obtained from the wall area of the aerial image and attached to the building polygon as a wall texture, so the wall area is sufficient for texture generation. Even if it does not have a good resolution, it is possible to attach a texture of sufficient resolution that does not give a sense of incongruity to each building polygon according to the actual landscape, and a more natural cityscape model can be created.

[Configuration of Third Embodiment]
Next, a third embodiment of the present invention will be described in detail with reference to the drawings. FIG. 10 is a block diagram showing the configuration of the third exemplary embodiment of the present invention. Referring to FIG. 10, the third embodiment of the present invention has substantially the same configuration as the mode for carrying out the first invention of the present invention, but some components are the same as those of the first embodiment. It operates differently from the form.

  The orthoprocessing unit 32 is an information processing function block that executes the orthosing process on the aerial photograph data based on the DEM generated by the stereo processing unit 2. The DEM calculated by the stereo processing unit 2 is given to the orthoprocessing unit 32 together with the original aerial photograph data. Since the three-dimensional position of each pixel is obtained by the DEM, the orthorectification processing unit 32 executes conversion for rearranging the pixel by pixel so that it is an orthogonal projection with respect to the ground, and for both the DEM and the aerial photograph. Then, new image data is generated by re-projecting the aerial photograph data as viewed from above at all points in the image. Hereinafter, this conversion is referred to as ortho-rectification, the DEM after the ortho-processing is referred to as ortho-DEM, and the aerial photograph data after the ortho-processing is referred to as ortho-image.

  Further, the orthorectification processing unit 32 generates the ortho image, and for each of the left and right original images obtained by stereo processing the wall surface candidate data 7a indicating the area in the original image that is not used in the ortho image as the wall surface candidate area. Generate. The wall surface candidate area referred to here is an area of the original image that is not used because it overlaps based on the correspondence between the original image and the ortho image that are obtained when performing the orthorectification.

  The label image generation unit 33 is an information processing function block that creates label image data 9a corresponding to the three-dimensional polygon data 6a. The label image generation unit 33 draws a city model image from a viewpoint that matches the aerial photograph using the three-dimensional polygon data 6a of the building created by the polygon creation unit 4 and the terrain information (not shown), and each building polygon model The label image data 9a in which individual labels are written on the wall surface area is created for each of the left and right aerial photographs.

  The wall surface area information extraction unit 34 superimposes the wall surface candidate data 7a stored in the wall surface candidate image storage unit 7 and the label image data 9a created by the label image generation unit 33, so that each building in each of the left and right images The wall surface area of the building is confirmed, the left and right wall surface areas are compared, the area including the wall surface attribute information is further selected, and the wall surface attribute such as the texture image of each building wall surface or the representative color is extracted to store the building wall surface attribute Store in unit 11. The other functional blocks in the other embodiments of the present invention operate in the same manner as the corresponding functional blocks in the first embodiment.

[Operation of Third Embodiment]
Next, the overall operation of the present embodiment will be described in detail with reference to the flowchart of FIG.
Since the processing of step S301 and step S302 is the same as that of step S101 and step S102 in the first embodiment, detailed description thereof is omitted.

  Along with the orthorectification process in step S302, a wall surface candidate image creation process for extracting the area in the original image that has not been used in the ortho image as a wall surface candidate area and generating the wall surface candidate data 7a as an image indicating this is performed in stereo. The processing is performed on each of the processed left and right original images (step S303). By this processing, the left image wall surface candidate data 7a and the right image wall surface candidate data 7a are respectively created. These wall surface candidate data 7 a are all stored in the wall surface candidate image storage unit 7.

  As in step S104 in the first embodiment, the polygon creation process in step S304 uses the DEM created by the stereo processing unit 2 and the map data 5a representing the shape of the building as a planar polygon. Processing for generating the three-dimensional polygon data 6a is performed. Specifically, a plane polygon indicating each building shape is superimposed on the DEM, and statistical processing such as taking a mode value from the DEM data inside each building area is performed to obtain the height of each building. Reflecting this height, the three-dimensional polygon of each building is generated as a polygonal column by diverting the map polygon to the planar shape.

  In step S305, using the three-dimensional building polygon generated in step S304 and the separately prepared terrain polygon indicating the terrain of the surrounding area, an image in which each polygon is drawn at the same viewpoint as the shooting viewpoints of the left and right original images And a label image generation process for generating an image in which individual label values are written for the pixel at the position where the wall surface of each building polygon is drawn. The topographic model data used in this case is a three-dimensional polygon indicating the shape of the surrounding area of the building, as in the first embodiment, for example, a polygon based on a numerical map 50 m mesh elevation issued by the Geospatial Information Authority of Japan. Alternatively, those created from measurement data obtained by a laser profiler can be used. In addition, the aerial photography viewpoint position information uses a value determined by the orientation process performed during the stereo process. By this label image generation process, a label image of the left image and a label image of the right image are created.

  In step S306, the left and right wall surface candidate data 7a created in step S303 and the left and right label image data 9a created in step S305 are overlapped to determine wall pixel areas of each building, and left and right wall pixel areas The wall surface area selection process for selecting the one that contains a lot of attribute information is executed. In step S307, wall surface area information extraction processing for extracting attribute information is performed. Specifically, the wall surface pixel region of each building in the left and right aerial photographs is obtained from the wall surface candidate data 7a corresponding to the left and right aerial photographs and the label image by the same method as in the first embodiment. Further, the wall surface pixel regions in the left and right images for the same building are compared, and the one having a larger number of pixels than that which seems to include more wall surface attribute information is determined as the wall surface pixel region of the building. An attribute useful for texture generation is acquired from the region by the same method as in the first embodiment. The attribute here refers to a texture image or a region representative color, and any or a plurality of attributes are acquired and stored in the building wall surface attribute storage unit 11 according to the area or color distribution of the region.

  Since the subsequent processing in step S308 is the same as that in step S307 in the first embodiment, detailed description thereof is omitted. By performing such a configuration and operation, the processing for determining the wall area of each building is performed for each of the left and right original images, and then the wall attribute information is extracted by selecting the one having a sufficient number of pixels. Because it is configured, it is directly under the viewpoint and the wall surface is not visible in the aerial photograph. Area attributes can be extracted and reflected in the generated city model.

[Configuration of Fourth Embodiment]
Next, a fourth embodiment of the present invention will be described in detail with reference to the drawings. Referring to FIG. 12, the fourth embodiment of the present invention has a configuration in which the wall surface candidate image storage unit 7 is excluded from the configuration of the first embodiment.

  The orthoprocessing unit 31 is an information processing function block that executes the orthosing process on the aerial photograph data based on the DEM generated by the stereo processing unit 2. The DEM calculated by the stereo processing unit 2 is given to the orthoprocessing unit 31 together with the original aerial photograph data. The ortho-processing unit 31 obtains the three-dimensional position of each pixel by DEM. Therefore, the ortho-processing unit 31 performs conversion for rearranging the pixel unit so that the three-dimensional position is orthogonal to the ground, for both the DEM and the aerial photograph. Then, new image data is generated by re-projecting the aerial photograph data as viewed from above at all points in the image. Hereinafter, this conversion is referred to as ortho-rectification, the DEM after the ortho-processing is referred to as ortho-DEM, and the aerial photograph data after the ortho-processing is referred to as ortho-image.

  The wall surface area information extraction unit 10 determines a wall surface area for each building from only the label image data 9a created by the label image generation unit 8, and extracts a wall surface attribute such as a texture image of each building wall or a representative color. To the building wall surface attribute storage unit 11. The other functional blocks in the fourth embodiment of the present invention perform the same operations as the corresponding functional blocks in the first embodiment.

[Operation of Fourth Embodiment]
Next, the overall operation of the present embodiment will be described in detail with reference to the flowchart of FIG.
Since the processing of step S401 and step S402 is the same as that of step S101 and step S102 in the first embodiment, detailed description thereof is omitted. Also, the processing in step S403 and step S404 is the same as that in step S104 and step S105 in the first embodiment, and thus detailed description thereof is omitted.

  In step S405, wall surface area information extraction processing is performed for determining the wall surface pixel area of each building in the label image data 9a created in step S404 and extracting attribute information. Specifically, an area having a label corresponding to each building in the label image is determined as a wall surface pixel area of the building, and attributes useful for texture generation are acquired from the area. The attribute here refers to a texture image or a region representative color, and any or a plurality of attributes are acquired and stored in the building wall surface attribute storage unit 11 according to the area or color distribution of the region.

  In step S406, the corresponding building polygon data in the three-dimensional polygon data 6a is processed using the wall surface attribute information stored in the building wall surface attribute storage unit 11 as in step S107 in the first embodiment. The wall surface texture generation process is performed to make the polygon close to the real landscape.

  As a result, the wall surface area in the aerial photograph is specified using the label image created from the three-dimensional polygon data of the building, and the texture is attached to the building polygon generated by extracting the attribute. A wall attribute can also be acquired for a building having a wall surface and using a wall surface pixel in the ortho image during the orthoprocessing.

In addition, although several embodiment of this invention was shown above, embodiment of this invention is not limited to these, Implementation on various conditions is possible. For example, the target image is not limited to an aerial photograph, and can be applied to, for example, a satellite image. In addition, some of the embodiments of the present invention described above can be applied in combination within a range where no contradiction occurs.
In the third embodiment of the present invention, the wall surface area is determined for each of the left and right images, and either one is selected to acquire the wall surface attribute. It is also possible to extract the wall surface area using the above method, obtain three to four or more wall surface areas for a certain building, and extract the wall surface information using them.

  In addition, there is no particular restriction on the method of extracting information from a plurality of wall areas for a building, and not only selecting the attribute with the largest area but also performing attribute extraction, after obtaining the attributes individually, You can use a method that compares the pixel distribution etc. to select a more reliable and more reliable one, or a method that uses all the pixels in multiple areas as they are to obtain statistical attributes such as representative colors, or It is also possible to select individual information extraction methods for each attribute and use them in combination.

FIG. 1 is a block diagram showing a configuration of city model generation by conventional aerial photo stereo processing. FIG. 2 is a block diagram showing the configuration of the first embodiment. FIG. 3 is a flowchart showing the operation of the first embodiment. FIG. 4 is a schematic diagram showing an aerial photography method used in the present embodiment. FIG. 5 is a schematic diagram showing an aerial photograph orthoprocessing. FIG. 6 is a schematic diagram showing a city model creation process from DEM data and map data. FIG. 7 is a schematic diagram showing wall surface area determination processing based on the wall surface candidate data 7a and the label image. FIG. 8 is a block diagram showing the configuration of the second embodiment. FIG. 9 is a flowchart showing the operation of the second embodiment. FIG. 10 is a block diagram illustrating a configuration of the third embodiment. FIG. 11 is a flowchart showing the operation of the third embodiment. FIG. 12 is a block diagram illustrating the configuration of the fourth embodiment. FIG. 13 is a flowchart showing the operation of the fourth embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Aerial photograph data storage part, 1a ... Aerial photograph data 2 ... Stereo processing part 3 ... Orthogonalization processing part 4 ... Polygon preparation part 5 ... Map data storage part, 5a ... Map data 6 ... Three-dimensional polygon data storage part, 6a 3D polygon data 7 ... Wall surface candidate image storage unit 7a ... Wall surface candidate image data 8 ... Label image generation unit 9 ... Label image storage unit 9a ... Label image 10 ... Wall surface area information extraction unit 11 ... Building wall surface attribute storage unit 11a ... Building wall surface attribute data 12 ... Wall surface texture generation unit 13 ... Standard wall surface image storage unit 13a ... Standard wall surface image data 14 ... Wall surface area information extraction unit 31, 32 ... Orthosing processing unit 33 ... Label image generation unit 34 ... Wall surface area information extraction unit 101 ... satellite image storage unit 102 ... stereo processing unit 103 ... DEM data automatic correction unit 104 ... map data storage unit

Claims (21)

An image data storage unit that stores image data obtained by converting a plurality of continuously shot images into electronic data, and adjacent images of the images include images of the same subject,
A side data output unit that extracts and outputs side data corresponding to the surface constituting the depth direction of the subject from the image data;
A texture generating unit that generates a texture using the side data and attaches the texture to the three-dimensional polygon data generated based on the image data ;
The side data output unit includes a stereo processing unit and an orthoprocessing unit,
The stereo processing unit calculates depth information of the subject,
The orthorectification processing unit generates an ortho image that is an orthographic image of the subject based on the image data and the depth information, the pixel of the contour portion of the subject of the image data, and the subject of the ortho image And generating the side data based on the pixels of the contour portion of
The texture generation unit includes a label image generation unit, a side surface information generation unit, and a wall surface texture generation unit,
The label image generation unit generates reference image data based on the three-dimensional polygon data of the subject generated based on the depth information, and a side region corresponding to a side surface of the three-dimensional polygon data in the reference image data And generating a label image by adding individual label values to the pixels of the side region,
The side surface information creating unit identifies a region having a label corresponding to the subject in the label image as a wall surface pixel region of the subject, and based on a texture image or a region representative color acquired from the wall surface pixel region Generate side information used to generate the texture,
The wall surface texture generation unit generates a texture based on the side surface information, and attaches the texture to a side surface of the 3D polygon data . The three-dimensional model creation device according to claim 1,
Before SL side information creation unit, on the basis of the side data and said labeled image, 3-dimensional modeling apparatus for generating side information used to generate the texture. The three-dimensional model creation device according to claim 2,
The texture generation unit includes a standard side image storage unit,
The standard side image storage unit stores a plurality of side image data in advance,
The side information creation unit selects, as the most similar data, data that is most similar to the side information from the plurality of side image data,
The texture is generated based on the most similar data. In the three-dimensional model creation device according to any one of claims 1 to 3 ,
The stereo processing unit specifies each of adjacent image data as a first image and a second image, calculates the depth information by performing a stereo matching process on the first image and the second image,
The orthoprocessing unit generates first side data corresponding to the first image and second side data corresponding to the second image,
The label image generation unit generates a first label image corresponding to the first image and a second label image corresponding to the second image;
The side information creation unit creates first side information based on the first side data and the first label image, and creates a second side based on the second side data and the second label image. A three-dimensional model creation device that creates information and creates the side information by comparing the first side information and the second side information. In the three-dimensional model creation device according to any one of claims 1 to 3 ,
The plurality of continuously shot images are taken by an imaging device mounted on a flying object flying in the sky,
The subject is a structure on the ground,
The depth information is information indicating the height of the structure,
The side surface data is data indicating a state of a wall surface of the structure body. The three-dimensional model creation device according to any one of claims 1 to 5 ,
The surface constituting the depth direction of the subject is a surface parallel to the optical axis direction for capturing the image.   An image data storage unit that stores image data obtained by converting a plurality of continuously shot images into electronic data, and adjacent images of the images include images of the same subject,
  A side data output unit that extracts and outputs side data corresponding to the surface constituting the depth direction of the subject from the image data;
  A texture generating unit that generates a texture using the side data and attaches the texture to the three-dimensional polygon data generated based on the image data;
  Comprising
  The side data output unit includes a stereo processing unit and an orthoprocessing unit,
  The stereo processing unit
  Each of adjacent image data is identified as a first image and a second image, and the depth information of the subject is calculated by performing stereo matching processing on the first image and the second image,
  The orthoprocessing unit is:
  An ortho image that is an orthographic image of the subject is generated based on the image data and the depth information, and the pixel of the contour portion of the subject of the image data and the pixel of the contour portion of the subject of the ortho image Based on the first side data corresponding to the first image and the second side data corresponding to the second image,
  The texture generation unit includes a label image generation unit, a side surface information generation unit, and a wall surface texture generation unit,
  The label image generator
  Reference image data is generated based on the three-dimensional polygon data of the subject generated based on the depth information, a side region corresponding to a side surface of the three-dimensional polygon data is specified in the reference image data, and the side region To generate a first label image corresponding to the first image and a second label image corresponding to the second image,
  The side information creation unit
  First side information is created based on the first side data and the first label image, second side information is created based on the second side data and the second label image, Based on the comparison between the one side information and the second side information, the side information used for generating the texture is generated,
  The wall surface texture generator is
  Generate the texture based on the side information and attach the texture to the side of the 3D polygon data
  3D model creation device. An image data reading step of reading adjacent image data including an image of the same subject from image data obtained by converting a plurality of continuously shot images into electronic data,
A side data output step of extracting and outputting side data corresponding to the surface constituting the depth direction of the subject;
Using said side data to generate a texture, the texture comprises a texture generation step attached to the three-dimensional polygon data generated based on the image data,
The side data generation step includes:
Calculating depth information of the subject;
Generating an ortho image that is an orthographic image of the subject based on the image data and the depth information;
Generating the side data based on pixels of the contour portion of the subject of the image data and pixels of the contour portion of the subject of the ortho image;
Including
The texture generation step includes
Generating reference image data based on the three-dimensional polygon data of the subject generated based on the depth information;
Identifying a side region corresponding to a side surface of the three-dimensional polygon data in the reference image data, adding a label value to each pixel of the side region, and generating a label image;
Generating side information used to generate the texture based on the label image;
Generating the texture based on the side information and attaching the texture to the side of the three-dimensional polygon data
A program capable of executing a method comprising: The program according to claim 8 , wherein
The step of generating the side information includes:
Generating side information used for generating the texture based on the side data and the label image;
A program that can be executed on a computer. The program according to claim 8 or 9 ,
The texture generation step includes
Selecting the most similar data as the most similar data from the plurality of side image data stored in advance,
A program capable of executing on a computer a method comprising the step of generating based on the most similar data. The program according to any one of claims 8 to 10 ,
Specifying each of adjacent image data as a first image and a second image, and calculating the depth information by performing a stereo matching process on the first image and the second image;
Generating first side data corresponding to the first image and second side data corresponding to the second image;
Generating a first label image corresponding to the first image and a second label image corresponding to the second image;
First side information is created based on the first side data and the first label image, second side information is created based on the second side data and the second label image, A computer-executable program comprising a step of comparing one side information and the second side information to create the side information. The program according to any one of claims 8 to 11 ,
The plurality of continuously shot images are taken by an imaging device mounted on a flying object flying in the sky,
The subject is a structure on the ground,
The depth information is information indicating the height of the structure,
The side data is data indicating a state of a wall surface of the structure. The program according to any one of claims 8 to 12 ,
The surface constituting the depth direction of the subject is a surface parallel to the optical axis direction for capturing the image.   An image data reading step of reading adjacent image data including an image of the same subject from image data obtained by converting a plurality of continuously shot images into electronic data,
  A side data output step of extracting and outputting side data corresponding to the surface constituting the depth direction of the subject;
  A texture generating step of generating a texture using the side data and attaching the texture to 3D polygon data generated based on the image data;
  Comprising
  The side data output step includes
  Identifying each of adjacent image data as a first image and a second image, and calculating depth information of the subject by performing stereo matching processing on the first image and the second image;
  Generating an ortho image that is an orthographic image of the subject based on the image data and the depth information;
  Based on the pixels of the contour portion of the subject of the image data and the pixels of the contour portion of the subject of the ortho image, the first side data corresponding to the first image and the second side corresponding to the second image. Generating two side data; and
  Including
  The texture generation step includes
  Generating reference image data based on the three-dimensional polygon data of the subject generated based on the depth information;
  In the reference image data, a side region corresponding to the side surface of the three-dimensional polygon data is specified, individual label values are added to the pixels of the side region, a first label image corresponding to a first image, and the first image Generating a second label image corresponding to the two images;
  First side information is created based on the first side data and the first label image, second side information is created based on the second side data and the second label image, Generating side information used for generating the texture based on a comparison between the one side information and the second side information;
  The wall surface texture generation unit generates the texture based on the side surface information, and attaches the texture to the side surface of the three-dimensional polygon data.
  A program capable of executing a method comprising: An image data reading step of reading adjacent image data including an image of the same subject from image data obtained by converting a plurality of continuously shot images into electronic data,
A side data output step of extracting and outputting side data corresponding to the surface constituting the depth direction of the subject;
Using said side data to generate a texture, the texture comprises a texture generation step attached to the three-dimensional polygon data generated based on the image data,
The side data generation step includes:
Calculating depth information of the subject;
Generating an ortho image that is an orthographic image of the subject based on the image data and the depth information;
Generating the side data based on pixels of the contour portion of the subject of the image data and pixels of the contour portion of the subject of the ortho image;
Including
The texture generation step includes
Generating reference image data based on the three-dimensional polygon data of the subject generated based on the depth information;
Identifying a side region corresponding to a side surface of the three-dimensional polygon data in the reference image data, adding a label value to each pixel of the side region, and generating a label image;
Generating side information used to generate the texture based on the label image;
Generating the texture based on the side information and attaching the texture to the side of the three-dimensional polygon data
A three-dimensional model creation method comprising : The three-dimensional model creation method according to claim 15,
The step of generating the side information includes:
A three-dimensional model creation method including a step of generating side information used for generation of the texture based on the side data and the label image . The three-dimensional model creation method according to claim 15 or 16 ,
The texture generation step includes
Selecting the most similar data as the most similar data from the plurality of side image data stored in advance,
A three-dimensional model creation method comprising the step of generating based on the most similar data. The three-dimensional model creation method according to any one of claims 15 to 17 ,
Specifying each of adjacent image data as a first image and a second image, and calculating the depth information by performing a stereo matching process on the first image and the second image;
Generating first side data corresponding to the first image and second side data corresponding to the second image;
Generating a first label image corresponding to the first image and a second label image corresponding to the second image;
First side information is created based on the first side data and the first label image, second side information is created based on the second side data and the second label image, A method of creating a three-dimensional model, comprising: comparing one side information and the second side information to create the side information. The three-dimensional model creation method according to any one of claims 15 to 18 ,
The plurality of continuously shot images are taken by an imaging device mounted on a flying object flying in the sky,
The subject is a structure on the ground,
The depth information is information indicating the height of the structure,
The side data is data indicating a state of a wall surface of the structure. In the three-dimensional model creation method according to any one of claims 15 to 19 ,
The surface constituting the depth direction of the subject is a surface parallel to the optical axis direction for capturing the image. An image data reading step of reading adjacent image data including an image of the same subject from image data obtained by converting a plurality of continuously shot images into electronic data,
A side data output step of extracting and outputting side data corresponding to the surface constituting the depth direction of the subject;
A texture generating step of generating a texture using the side data and attaching the texture to 3D polygon data generated based on the image data;
Comprising
The side data output step includes
Identifying each of adjacent image data as a first image and a second image, and calculating depth information of the subject by performing stereo matching processing on the first image and the second image;
Generating an ortho image that is an orthographic image of the subject based on the image data and the depth information;
Based on the pixels of the contour portion of the subject of the image data and the pixels of the contour portion of the subject of the ortho image, the first side data corresponding to the first image and the second side corresponding to the second image. Generating two side data; and
Including
The texture generation step includes
Generating reference image data based on the three-dimensional polygon data of the subject generated based on the depth information;
In the reference image data, a side region corresponding to the side surface of the three-dimensional polygon data is specified, individual label values are added to the pixels of the side region, a first label image corresponding to a first image, and the first image Generating a second label image corresponding to the two images;
First side information is created based on the first side data and the first label image, second side information is created based on the second side data and the second label image, Generating side information used for generating the texture based on a comparison between the one side information and the second side information;
The wall surface texture generation unit generates the texture based on the side surface information, and attaches the texture to the side surface of the 3D polygon data .

Images