JP2022163920A - Three-dimensional model generation device, three-dimensional model generation method, and program - Google Patents

Abstract

To allow for generating a three-dimensional model regardless of the presence or absence of image data.SOLUTION: A three-dimensional model generation device is provided, comprising: an image data acquisition unit for acquiring image data of an object including time information; a three-dimensional model generation unit configured to generate a three-dimensional model based on the image data of the object acquired by the image data acquisition unit; and a three-dimensional model correction unit configured to estimate a degree of change in the object during a given time period with respect to the three-dimensional model to correct the three-dimensional model.SELECTED DRAWING: Figure 4

Description

The present invention relates to a three-dimensional model generation device, a three-dimensional model generation method, and a program.

As a technique for easily retrieving desired image data from a plurality of image data, for example, there is a technique described in Patent Document 1 below. The technique described in Japanese Patent Application Laid-Open No. 2002-200002 stores a plurality of image data to which the photographing positions and photographing dates and times are attached as photographing information in an image exchange server, and the photographing information attached to the image data from a terminal device. Search for desired image data based on Further, as a technique for photograph mapping a plurality of images, there is a technique described in Patent Document 2 below, for example. The technology described in Patent Literature 2 uses meta information attached to a plurality of photo data to organically utilize search, search for information on buildings within sight in real time, search for information about It searches for photos that are marked as , and performs photo mapping.

Japanese Patent Application Laid-Open No. 2004-220420 JP 2009-176262 A

2. Description of the Related Art There is a technique called photogrammetry in which a plurality of photographs are taken while changing the photographing position of an object, and a three-dimensional model is generated based on data of the photographed photographs. However, conventional photogrammetry processes a plurality of actual photographic data to generate a three-dimensional model. Therefore, a three-dimensional model cannot be generated without actual photo data.

SUMMARY OF THE INVENTION It is an object of the present invention to make it possible to generate a three-dimensional model regardless of the presence or absence of image data.

In order to solve the above-described problems and achieve the object, a three-dimensional model generation apparatus according to the present invention includes an image data acquisition unit for acquiring image data of an object including time information; a three-dimensional model generating unit that generates a three-dimensional model based on the image data of the object that has been obtained; and a three-dimensional model correction unit.

A three-dimensional model generating method according to the present invention comprises the steps of obtaining image data of an object including time information, generating a three-dimensional model based on the obtained image data of the object, and and estimating the degree of change of the object at a predetermined time to correct the three-dimensional model.

A program according to the present invention comprises the steps of acquiring image data of an object including time information, generating a three-dimensional model based on the acquired image data of the object, and performing predetermined operations on the three-dimensional model. a step of estimating the degree of change of the object at the time of and correcting the three-dimensional model.

ADVANTAGE OF THE INVENTION According to this invention, it is effective in the ability to produce|generate a three-dimensional model regardless of the presence or absence of image data.

FIG. 1 is a block diagram showing an image acquisition device in a three-dimensional model generation device according to this embodiment. FIG. 2 is a schematic diagram showing a recording format for image data. FIG. 3 is a schematic diagram for explaining position information and azimuth information for an object. FIG. 4 is a block diagram showing the three-dimensional model generation device according to this embodiment. FIG. 5 is a flow chart representing a three-dimensional model generation method. FIG. 6 is a block diagram showing a specific configuration of the 3D model generation unit. FIG. 7 is an explanatory diagram showing the positional relationship between two images to which the photogrammetry principle is applied. FIG. 8 is an explanatory diagram showing the positional relationship between two images.

Exemplary embodiments of a three-dimensional model generation device, a three-dimensional model generation method, and a program according to the present invention will be described below in detail with reference to the accompanying drawings. In addition, the present invention is not limited by the following embodiments.

[Image acquisition device]
FIG. 1 is a block diagram showing an image acquisition device in a three-dimensional model generation device according to this embodiment. The image acquisition device 10 generates format image data, which will be described later, and transmits it to the database 26 in order to cause a 3D model generation device, which will be described later, to generate a 3D model.

As shown in FIG. 1, the image acquisition device 10 includes a camera (imaging device) 11, a GPS receiver (GNSS signal receiver) 12, an orientation sensor 13, a time information acquisition unit 21, and a position information acquisition unit 22. , an orientation information acquisition unit 23 , a format unit 24 , and a storage unit 25 .

A camera 11 photographs an object. Image data captured by the camera 11 is compressed into a format such as JPEG or PNG to such an extent that deterioration is not noticeable. The camera 11 sends the captured image data to the format unit 24, and also sends it to the time information acquisition unit 21, the position information acquisition unit 22, and the direction information acquisition unit 23 as trigger information.

The GPS receiver 12 receives radio waves from satellites and obtains accurate time information and position information. The GPS receiving section 12 sends the time information to the time information obtaining section 21 and sends the position information to the position information obtaining section 22 .

The time information acquisition unit 21 acquires time information when the camera 11 captured the image data based on the time information from the GPS reception unit 12 . The time information acquisition unit 21 sends time information when the camera 11 captures the image data to the format unit.

The position information acquisition unit 22 acquires position information when the camera 11 captures the image data based on the information from the GPS reception unit 12 . The position information acquisition unit 22 sends position information when the camera 11 captures the image data to the format unit.

The azimuth sensor 13 measures the tilt (elevation angle) of the camera 11 using a gyro (angular velocity sensor). The azimuth sensor 13 sends the measured result to the azimuth information acquisition unit 23 .

Based on the measurement result of the orientation sensor 13, the orientation information acquisition unit 23 sends the orientation information when the camera 11 captures the image data to the format unit.

The image data captured by the camera 11, the time information of the image data acquired by the time information acquisition unit 21, the position information of the image data acquired by the position information acquisition unit 22, and the orientation information of the image data acquired by the orientation information acquisition unit 23 are , is input to the format unit 24 .

The format unit 24 is set with the number of bytes per parameter, converts various types of input information into a predetermined format, and generates format image data. The format image data is data obtained by converting the image data, the time information of the image data, the position information of the image data, and the orientation information of the image data into a predetermined format. be done. Format unit 24 sends the formatted image data to storage unit 25 .

Note that the time information acquisition unit 21, the position information acquisition unit 22, the azimuth information acquisition unit 23, and the format unit 24 are configured by an arithmetic circuit such as a CPU (Central Processing Unit), for example.

The storage unit 25 temporarily stores the format image data. The storage unit 25 is an external storage device such as a HDD (Hard Disk Drive), a memory, or the like. The storage unit 25 transmits the formatted image data to a database 26 connected by a network such as the Internet via a transmission unit (not shown). The formatted image data temporarily stored in the storage unit 25 is stored in a database 26 on the Internet on a large scale.

FIG. 2 is a schematic diagram showing a recording format for image data.

As shown in FIG. 2, the format image data consists of a fixed-length portion in the first half and a variable-length portion in the second half. The fixed-length portion has time information, position information, and orientation information. The time information is year/month/date information, and consists of 2 bytes of year data, 1 byte of month data, 1 byte of day data, 1 byte of time data, 1 byte of minute data, and 1 byte of second data. The variable length portion is composed of image data. The position information consists of 8 bytes of latitude data and 8 bytes of longitude data. The positional information represents the position where the object was photographed, and information other than the latitude and longitude may be an address, an area name, a building name, or the like. Note that the position information may be defined as an area obtained by dividing a map into a predetermined size, and may be described as relative coordinates on the horizontal axis and the vertical axis within the area.

The azimuth information consists of 8 bytes for horizontal azimuth and 8 bytes for elevation angle data. The azimuth information of the image data acquired by the camera 11 is, for example, expressed using an analog clock, with the hands clockwise at 3 o'clock (90°), south at 6 o'clock (180°), and west at 9 o'clock. (270°) and North at 12 o'clock (360° (0°)). As for the elevation angle azimuth, the horizontal is 0 degrees and the top is 90 degrees. The variable-length portion is image information, and the image information is an image compression file in which an image is compressed in JPG format or PNG format.

FIG. 3 is a schematic diagram for explaining position information and azimuth information for an object.

As shown in FIG. 3, the map 100 is a section of a certain city, and the map 100 is, for example, a 100 m square and is sectioned into a plurality of areas. In FIG. 3, predetermined sections of the map 100 are defined as area 0010-2324, area 0010-2325, area 0010-2326, area 0010-2455, area 0010-2456, area 0010-2457, area 0010-2586, area 0010- 2587 and areas 0010 to 2588 divided into nine areas.

Object T is in an area numbered, for example, area 0010-2456. The camera 11 shoots an object T at five positions A1, A2, A3, A4, and A5. The position A1 is the front position with respect to the object T, the position A2 is the left oblique front position with respect to the object T, the position A3 is the left oblique rear position with respect to the object T, and the position A4 is the object T. , and the position A5 is a right oblique front position with respect to the object T.

Here, the lines extending from the positions A1, A2, A3, A4, and A5 toward the object T are azimuths. The azimuth information is in the horizontal direction, and the direction directly above (true north) on the map is the 12 o'clock direction (0 degrees). For example, the direction of position A1 is 00:00:00, and the direction of position A2 is 02:30:00. The elevation angle of the camera 11 that cannot be displayed on the map 100 is, for example, 12 degrees at position A1, 7 degrees at position A2, 10 degrees at position A3, 25 degrees at position A4, and 35 degrees at position A5. as described in the format.

[3D model generation device]
FIG. 4 is a block diagram showing the three-dimensional model generation device according to this embodiment.

As shown in FIG. 4 , the three-dimensional model generation device 30 includes an image processing device 31 , an input device 32 and a display device 33 .

The input device 32 is operated by the user. The input device 32 sends an operation command signal input by the user to the image processing device 31 . Here, the operation command signal is the desired place (name or address of the object) where the user wants to generate the three-dimensional model, the desired time when he wants to generate the three-dimensional model, and the like. The input device 32 is, for example, a keyboard or a touch panel.

The image processing device 31 is connected to the database 26 via the Internet, for example, via a transmitter/receiver (not shown). The image processing device 31 acquires the format image data stored in the database 26 based on the operation command signal sent from the input device 32, and performs image processing on the format image data. That is, the image processing device 31 acquires format image data corresponding to the object stored in the database 26 based on the object sent from the input device 32, and performs image processing on the format image data. The image processing device 31 sends image data that has undergone image processing to the display device 33 .

The display device 33 displays image data to the user. The display device 33 displays the three-dimensional model and image data image-processed by the image processing device 31 . The display device 33 is, for example, a liquid crystal display or a head mounted display.

The image processing device 31 has an image data acquisition unit 34 , a 3D model generation unit 35 , and a 3D model correction unit 36 . The image data acquisition unit 34, the 3D model generation unit 35, and the 3D model correction unit 36 are configured by, for example, a CPU.

The image data acquisition unit 34 receives an operation command signal input by the user through the input device 32 . The image data acquisition unit 34 searches and acquires a large number of corresponding image data based on this operation command signal. That is, the image data acquisition unit 34 acquires the format image data of the object input from the input device 32 from the format image data stored in the database 26 . The image data acquisition unit 34 sends out the acquired format image data to the three-dimensional model generation unit 35 .

The three-dimensional model generation unit 35 generates image data included in a large number of format image data acquired by the image data acquisition unit 34, time information when the image data included in the format image data was captured, and position information of the image data. , a three-dimensional model is generated based on the orientation information of the image data. The 3D model generation unit 35 sends the generated 3D model to the 3D model correction unit 36 .

The 3D model correction unit 36 corrects the 3D model generated by the 3D model generation unit 35 based on the time information input from the input device 32 . That is, the three-dimensional model correction unit 36 estimates the time information input from the input device 32, that is, the degree of change of the object at a designated predetermined time, and corrects the three-dimensional model. The 3D model correction unit 36 sends the corrected 3D model to the display device 33 .

At this time, the three-dimensional model generation unit 35 performs an averaging process of averaging the luminance value of a certain pixel with respect to a plurality of image data whose time information and orientation information match each other. A three-dimensional model is generated based on a plurality of image data in different orientations that have undergone the transformation processing. Then, the 3D model correction unit 36 corrects the 3D model based on the amount of change in the 3D model at different times.

[3D model generation method]
FIG. 5 is a flow chart representing a three-dimensional model generation method.

As shown in FIGS. 4 and 5, in step S11, the user uses the input device 32 to input a desired location for generating a three-dimensional model as an operation command signal. The desired location is the name, address, or the like of the object. At step S12, the user uses the input device 32 to input a desired time to generate a three-dimensional model as an operation command signal. Enter the desired time in AD and year/month/day. In step S13, the image data acquisition unit 34 retrieves and acquires from the database 26 a large number of format image data corresponding to the desired location and desired time as the retrieval conditions. In this case, the image data acquisition unit 34 searches for a predetermined period as the desired time. When, for example, July 1, 2000 is input as a desired time, the image data acquisition unit 34 acquires a plurality of images taken during the predetermined period from January 1, 1995 to July 1, 2005. Get format image data.

In step S14, the 3D model generator 35 generates a 3D model based on the image data included in the plurality of format image data, the time information of the image data, the position information of the image data, and the orientation information of the image data. to generate Details of the generation of the three-dimensional model in step S14 will be described later. In step S15, the three-dimensional model correction unit 36 estimates the degree of change of the target object in a predetermined time period with respect to the three-dimensional model and corrects the three-dimensional model. Then, in step S16, the display device 33 displays the corrected three-dimensional model.

That is, first, the image data acquisition unit 34 acquires a plurality of format image data from the database 26 via the Internet, thereby acquiring a plurality of image data and time information, position information, and azimuth information in each image data. do. The image data acquisition unit 34 acquires a plurality of image data of the object T photographed from a plurality of different positions and orientations at a plurality of different times.

Next, the three-dimensional model generation unit 35 performs affine transformation with degrees of freedom such as enlargement/reduction, rotation, and left/right trapezoidal correction on the plurality of image data while maintaining the aspect ratio, and converts the image characteristics into Positioning calculation is performed so that the points correspond to each other, that is, the matching process makes the difference extremely small. Left and right trapezoidal correction is based on the principle of perspective. When you shoot a building from the left side, the left vertical line of the same height becomes closer, making it longer, and the right vertical line becomes farther away, making it appear shorter. This is because, in the case of camera shooting, the number of shots taken at many angles increases because it is possible to move in the horizontal direction, but in the vertical direction, the angle of elevation changes only slightly, and the shooting point does not move vertically. By extracting groups with matching large features, the connections between images can be found and expanded in memory. These processes are a technique of generating a three-dimensional shape of the object T from images taken from a plurality of angles using a photogrammetry technique, and are the principle of triangulation. Whether or not a feature point can be taken is preferably determined by setting two sets of images among the selected images, in which the fields of view overlap each other by about 60%.

Then, the three-dimensional model correction unit 36 estimates the degree of change of the three-dimensional model generated by the three-dimensional model generation unit 35 so as to correspond to a desired time, and corrects the three-dimensional model. Then, a three-dimensional model of the object at a predetermined time desired by the user can be generated.

Here, a specific example of the three-dimensional model generation method in step S14 will be described. FIG. 6 is a block diagram showing the specific configuration of the 3D model generation unit, FIG. 7 is an explanatory diagram showing the positional relationship between two images to which the principle of photogrammetry is applied, and FIG. 8 is the positions of the two images. It is an explanatory view showing relation.

As shown in FIG. 6, the three-dimensional model generation unit 35 includes an epipolar line direction calculator 41, an epipolar line orthogonal direction calculator 42, a search range determiner 43, a corresponding point detector 44, and a distance calculator 45. and

The epipolar line direction calculator 41 calculates the direction of the epipolar line connecting the corresponding pixel points of the image data of the object T based on the image data acquired by the image data acquisition unit 34. . The epipolar line direction calculator 41 sends the calculated epipolar line direction to the epipolar line orthogonal direction calculator 42 .

An epipolar line orthogonal direction calculator 42 calculates an orthogonal direction orthogonal to the epipolar line. The epipolar line orthogonal direction calculator 42 outputs the calculated orthogonal direction to the epipolar line to the search range determiner 43 .

A search range determiner 43 determines a two-dimensional search range on the screen so as to include a plurality of pixel points corresponding to the direction of the epipolar line and the direction orthogonal to the epipolar line. The search range determiner 43 outputs the determined two-dimensional search range to the corresponding point detector 44 .

The corresponding point detector 44 searches for corresponding points based on the plurality of image data acquired by the image data acquisition unit 34 and the determined two-dimensional search range to obtain parallax vectors. The corresponding point detector 44 sends the obtained parallax vector to the distance calculator 45 .

The distance calculator 45 maps the parallax vector to the epipolar line to obtain the epipolar line direction component of the parallax vector, and calculates the distance to the object T based on the obtained epipolar line direction component. The distance calculator 45 sends the calculated distance to the object T to the three-dimensional model correction unit 36 .

The generation of a three-dimensional model will be described in detail below, but here, the case of generating a three-dimensional model from two pieces of image data will be described.

The image data acquisition unit 34 acquires two pieces of image data from the database 26 . For example, as shown in FIG. 3, among the data registered and formatted in the area 0010-2456, the second image randomly selected based on one image is set as a set, and the corresponding points are set to each other. Search for common photographic images. A three-dimensional point group is obtained by performing triangulation of this set of images, and by collecting these, the relative positions and point group images are expanded. Also, the texture of the portion corresponding to this image is associated and managed in memory for later mapping.

First, two sets of image data are obtained with respect to the object T by the field image camera 11A and the field image camera 11B (see FIG. 7 for both). Next, the corresponding point detector 44 searches for corresponding points of feature points based on the two sets of image data. The corresponding point detector 44 performs, for example, matching for each pixel, and searches for the position where the difference is the minimum. Here, as shown in FIGS. 6 and 7, the cameras 11A and 11B, which are supposed to exist at two viewpoints at the same time, are arranged so that the optical axes Ol and Or are included on the same XZ coordinate plane. = Yr. Using the corresponding points searched by the corresponding point detector 44, a disparity vector corresponding to the angle difference for each pixel is calculated.

Since the obtained parallax vector corresponds to the distance from the cameras 11A and 11B in the depth direction, the distance calculator 45 calculates the distance proportional to the magnitude of the parallax by the perspective method. Assuming that the photographer's cameras 11A and 11B move only approximately horizontally, by arranging the cameras 11A and 11B so that their optical axes Ol and Or are included on the same XZ coordinate plane, , corresponding points can be searched for only on the scanning lines, which are the epipolar lines Epl and Epr. The distance calculator 45 generates a three-dimensional model of the object T using the two image data of the object T and the respective distances from the cameras 11A and 11B to the object T. FIG.

On the other hand, when the point Ql (Xl, Yl) on the left image corresponds to the point Qr (Xr, Yr) on the right image, the parallax vector at the point Ql (Xl, Yl) is Vp (Xl-Xr, Yl- Yr). Here, since the two points Ql and Qr are on the same scanning line (epipolar line), Yl=Yr and the parallax vector is expressed as Vp(Xl-Xr, 0). The epipolar line direction calculator 41 obtains such a parallax vector Vp for all pixel points on the image and creates a parallax vector group to obtain information in the depth direction of the image. By the way, for a set whose epipolar line is not horizontal, one camera position may have a different height (low probability). In this case, the epipolar line orthogonal direction calculator 42 uses the epipolar line direction and the direction orthogonal to the epipolar line in a large search range as compared to the case of searching for corresponding points in a large two-dimensional area without being conscious of corresponding point matching. By searching within a rectangle that is about the degree of deviation from the horizontal, the minimum amount of calculation of the rectangle is reduced and becomes rational. Then, as shown in FIG. 8, the search range determiner 43 sets the epipolar line direction search range for the minimum rectangle to a to b=c to d, and sets the orthogonal direction search range to b to c=d to a. Indicates the search range for the case. In this case, the search width in the epipolar line direction is ΔE, and the search width in the direction T orthogonal to the epipolar line is ΔT. The minimum non-tilted rectangle ABCD containing the minimum tilted rectangle abcd is the desired region.

In this way, the 3D model generating unit 35 obtains parallax vectors from corresponding points of the feature points of the cameras 11A and 11B under the epipolar constraint condition, obtains information on the depth direction of each point, and obtains information on the surface of the 3D shape. Map the texture above to generate a 3D model. As a result, the model of the part in the image data used for calculation can reproduce the space seen from the front hemisphere. In addition, when there is a part that is not captured in the image data of the three-dimensional model, and when the surrounding texture lines or surfaces are extended and connected, the same texture is used to interpolate between them. The 3D model generation unit 35 sends the generated 3D model to the 3D model correction unit 36 .

Note that the method of generating the three-dimensional model is not limited to the one described above, and other methods may be used.

Then, the 3D model correction unit 36 calculates the average value of the image data when correcting the 3D model generated by the generation of the 3D model to the 3D model of the time when the image data does not exist, so that the image The degree of change of the object at a predetermined time when no data exists is estimated, and the three-dimensional model is corrected.

That is, the plurality of pieces of image data are image data captured at desired times during a predetermined period (for example, from January 1, 1995 to July 1, 2005). The plurality of image data are grouped by year, and the average value of the color difference signals of the image data in a predetermined object portion constituting the surface of the object forming the target T is obtained.

The three-dimensional model correcting unit 36 determines whether image data at a desired time of an object for which the user wants to generate a three-dimensional model exists in the acquired plural image data. When a plurality of image data exist for, for example, July 1, 2000 as a desired time of an object for which the user wants to generate a three-dimensional model, the three-dimensional model generation unit 35 acquires a plurality of image data. Since the three-dimensional model as of July 1, 2000 can be generated by using the three-dimensional model, the three-dimensional model correction unit 36 does not correct the three-dimensional model. That is, when a plurality of image data exist on July 1, 2000, the three-dimensional model generation unit 35 calculates the average value of the plurality of image data of the object T from the same time, the same position, and the same direction, A three-dimensional model is generated using color difference signals of average values of image data.

On the other hand, if there is no image data on July 1, 2000 as the desired time of the object for which the user wishes to generate a 3D model, then the 3D model generating unit 35 generates a plurality of images that are not at the desired time. The 3D model correction unit 36 generates a 3D model at a desired time by correcting the 3D model generated based on the image data. In this case, the 3D model correction unit 36 corrects the 3D model based on a plurality of image data before Jun. 30, 2000 and after Jul. 2, 2000. FIG. The data of the average value of the color difference signals of the image data at this time is the average value of the image data in the future direction (before June 30, 2000) and in the past direction (after July 2, 2000) at a desired time. . The three-dimensional model correction unit 36 corrects the three-dimensional model using the average value of the color difference signals of the image data from the same position and direction at different times. That is, the three-dimensional model correction unit 36 estimates and synthesizes values obtained by interpolating a plurality of image data at different times as color difference signals.

In addition, the past direction (prior to June 30, 2000) or the future direction (July 2000) of the image data in which July 1, 2000, which is the desired time of the object for which the user wants to generate a three-dimensional model, exists. 2nd day or later), the average value data of the color difference signals of the image data at this time is the average value of the image data before June 30, 2000, or the average value of the image data after July 2, 2000. Average value. The 3D model correction unit 36 estimates and corrects a 3D model in the future direction or the past direction using the average values of the color difference signals of the image data from the same position and the same direction at different times. That is, a value obtained by interpolating a plurality of image data at different times is estimated as a color difference signal and synthesized. That is, the three-dimensional model correction unit 36 estimates and synthesizes values obtained by extrapolating image data as color difference signals based on the amount of change in the annual average value of the color difference signals.

Also, when one day (24 hours) is divided into 1-hour intervals and the color difference signals of the image data in the divided time are used, the brightness changes depending on the time of day and the height of the sun, which affects the color difference signals. there is a reason to remove it. Therefore, the 3D model correcting unit 36 acquires weather information from the outside, refers to the weather forecast for the area where the image data was captured, estimates the degree of sunlight on that day for each hour, and corrects the 3D model. You may

That is, if there is no image data on July 1, 2000, which is the desired time of the object for which the user wishes to generate a three-dimensional model, the three-dimensional model correction unit 36 The three-dimensional model is corrected using the average value and amount of change of the image data from July 2, 2000 onwards, or the average value and amount of change of the image data after July 2, 2000. At this time, the three-dimensional model correction unit 36 estimates the weather at a desired time based on the weather before or after July 1, 2000, for example. Then, the three-dimensional model correction unit 36 calculates the average value of the image data of the same weather as the weather on July 1, 2000 from the plurality of image data before June 30, 2000 and after July 2, 2000. is used to correct the 3D model.

Also, the 3D model correction unit 36 uses part of the image data of the time when the image data exists to determine the angle from the camera 11 with respect to the object T on July 1, 2000, which is the desired time, the weather, Image data is generated by machine learning using a training data set that takes as input the parameters of the height of the sun that takes into account the season by month and day, and the captured image data, and outputs the corresponding exemplary captured image data. Before June 30, 2000, or after July 2, 2000, when there is no T, the image data of the object T at a specific time may be synthesized using a predicted image.

As the image data used for correction by the three-dimensional model correction unit 36, image data obtained by predicting the future object T, that is, future predicted image data may be used. Predicted future image data includes space and image data of moving images such as movies, and 3D space and image data created as a virtual future. Image data for the entire construction space can also be used, including partial confined space (inset space). Assuming urban redevelopment or rebuilding of buildings, the spatial and image data of the city before development are combined into new city data by combining the date, time, location, and direction information of the space. It is also possible to perform image processing for extrapolation or interpolation and display. Future-predicted image data also includes spatial and image data generated mechanically (such as CAD data) or by artificial intelligence. If the area, building coverage ratio, type of building, etc. are used as input information, spatial image data of standard structures can be obtained and utilized.

That is, the database 26 has not only the image data of the object T from the past to the present, but also the image data of the predicted future (future prediction image data). The future prediction image data is image data obtained by predicting the state of the target object T on and after March 2, 2021 with respect to March 1, 2021, which is the present. When the user uses the input device 32 to input a future time, for example March 1, 2031, as a desired time, the image data acquisition unit 34 retrieves only the image data before March 1, 2021. In addition, future prediction image data after March 2, 2021 are also acquired. The three-dimensional model correction unit 36 corrects the three-dimensional model using the image data before March 1, 2021 and the future prediction image data after March 2, 2021.

[Action and effect of the present embodiment]
In this embodiment, an image data acquisition unit 34 that acquires image data of the target object T including time information, and a three-dimensional model that generates a three-dimensional model based on the image data of the target object T acquired by the image data acquisition unit 34. A model generating unit 35 and a three-dimensional model correcting unit 36 for estimating the degree of change of the object T at a predetermined time with respect to the three-dimensional model and correcting the three-dimensional model.

Therefore, the 3D model generation unit 35 generates a 3D model based on a plurality of image data and time information in each image data, and the 3D model correction unit 36 applies the 3D model at a predetermined time. The degree of change of the object T is estimated to correct the three-dimensional model. That is, first, a large number of existing image data are acquired, and a three-dimensional model of a predetermined time is generated from the acquired large number of image data. Next, based on the degree of change (degree of deterioration) of the shape, color, etc. estimated from time in the 3D model at a plurality of times, the 3D model is corrected to generate a 3D model at times where there is no image data. . As a result, a three-dimensional model can be generated regardless of the presence or absence of image data.

Further, in this embodiment, the image data of the target object T includes position information and orientation information, and the three-dimensional model generation unit 35 generates image data of a plurality of target objects T whose time information, position information, and orientation information match. Generate a three-dimensional model based on Therefore, a three-dimensional model of the object T at a predetermined time can be generated with high accuracy.

Further, in the present embodiment, the 3D model correction unit 36 corrects the 3D model based on the amount of change in the 3D model at different times. Therefore, even if there is no image data at the desired time, it is possible to generate a three-dimensional model at the desired time without image data based on the amount of change in the image data at times other than the desired time.

Although the three-dimensional model generation device according to the present invention has been described so far, it may be implemented in various different forms other than the above-described embodiments.

Each component of the illustrated three-dimensional model generation device is functionally conceptual, and does not necessarily have to be physically configured as illustrated. In other words, the specific form of each device is not limited to the illustrated one, and all or part of it may be functionally or physically distributed or integrated in arbitrary units according to the processing load and usage conditions of each device. may

The configuration of the three-dimensional model generation device is implemented by, for example, a program loaded into a memory as software. In the above embodiments, functional blocks realized by cooperation of these hardware or software have been described. That is, these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

The components described above include those that can be easily assumed by those skilled in the art and those that are substantially the same. Furthermore, the above configurations can be combined as appropriate. Also, various omissions, substitutions, or modifications of the configuration are possible without departing from the gist of the present invention.

10 image acquisition device 11 camera (imaging device)
12 GPS receiver 13 direction sensor 21 time information acquisition unit 22 position information acquisition unit 23 direction information acquisition unit 24 format unit 25 storage unit 26 database 30 three-dimensional model generation device 31 image processing device 32 input device 33 display device 34 image data acquisition Part 35 Three-dimensional model generation part 36 Three-dimensional model correction part T Object

Claims (5)

an image data acquisition unit that acquires image data of an object including time information;
a three-dimensional model generation unit that generates a three-dimensional model based on the image data of the object acquired by the image data acquisition unit;
a three-dimensional model correction unit for estimating the degree of change of the object at a predetermined time with respect to the three-dimensional model and correcting the three-dimensional model;
3D model generation device. the image data of the object includes position information and orientation information;
The three-dimensional model generating unit generates a three-dimensional model based on image data of a plurality of objects whose time information, position information, and orientation information match.
3. The three-dimensional model generation device according to claim 1. The three-dimensional model correction unit corrects the three-dimensional model based on the amount of change in the three-dimensional model at different times.
3. The three-dimensional model generation device according to claim 1 or 2. obtaining image data of the object including time information;
generating a three-dimensional model based on the acquired image data of the object;
estimating the degree of change of the object at a predetermined time with respect to the three-dimensional model and correcting the three-dimensional model;
A three-dimensional model generation method comprising: obtaining image data of the object including time information;
generating a three-dimensional model based on the acquired image data of the object;
estimating the degree of change of the object at a predetermined time with respect to the three-dimensional model and correcting the three-dimensional model;
A program that causes a computer that operates as a three-dimensional model generation device to execute.

Images