JP2022081224A - Image generating apparatus and program - Google Patents

Abstract

To provide an image generating apparatus and a program therefor which can easily correct a rendering image to fit an entire celestial image.SOLUTION: The present invention is directed to an image generating apparatus 1 for generating a superimposed image by superimposing a rendering image simulating an architecture on a captured image obtained by capturing a scheduled construction site A. The apparatus has an image acquiring unit 11 for acquiring the captured image as an entire celestial image and the rendering image, a capturing position acquiring unit 12 for acquiring a capturing position P including height at which the captured image is captured, a superimposing position determination unit 13 for determining a superimposition position of the captured image and the rendering image based on the acquired capturing position P, an image converting unit 14 for converting the rendering image into the entire celestial image based on the determined superimposition position, and an image generating unit 15 for generating the superimposed image by superimposing the converted rendering image on the captured image.SELECTED DRAWING: Figure 5

Description

The present invention relates to an image generator and a program.

Conventionally, it has been practiced to show an image after completion by creating an external view of a building to be built. In this way, showing an external view is useful in the business of home sales and the like.

Nowadays, as an external view, a 3D rendered image (hereinafter, also simply referred to as a rendered image) is used to show an image after completion. By using the rendered image, the appearance can be confirmed from various viewpoints. Furthermore, by superimposing the rendered image on the live-action image of the planned construction site, it is possible to confirm the harmony with the neighboring landscape. For example, there has been proposed a three-dimensional image of a building (a building image display military method in which an external image 9 is superimposed on a photographed image of a planned location of a building (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2014-89697

By the way, by converting the live-action image of the planned construction site into an omnidirectional image, it is possible to confirm all directions of 360 degrees from a predetermined viewpoint. On the other hand, the rendered image also needs to be corrected according to the spherical image. Therefore, it is preferable if the rendered image can be easily corrected according to the spherical image.

An object of the present invention is to provide an image generation device and a program capable of easily correcting a rendered image according to a spherical image.

The present invention is an image generation device that superimposes a rendered image imitating a building on an captured image of a planned construction site to generate a superimposed image, and is an all-sky image, the captured image and the rendered image. Based on the image acquisition unit that acquires the image, the imaging position acquisition unit that acquires the imaging position including the height at which the captured image is captured, and the acquired imaging position, the superimposed position of the captured image and the rendered image is determined. A superimposition position determination unit to be determined, an image conversion unit that converts the rendered image into an all-sky image based on the determined superimposition position, and a superimposition position in which the captured image and the converted rendered image are determined. The present invention relates to an image generation device including an image generation unit that superimposes and generates a superposed image.

Further, the image generation device includes a reference point acquisition unit that acquires a predetermined reference point in the captured image, and a feature point acquisition unit that acquires a feature point indicating a position in the rendered image corresponding to the acquired reference point. It is preferable that the image pickup position acquisition unit further includes an estimation unit that estimates the image pickup position based on the acquired reference point and the feature point, and the image capture position acquisition unit acquires the estimated image pickup position.

Further, the image generation device has acquired a reference point acquisition unit that acquires a predetermined reference point in the captured image, and a point information acquisition unit that acquires the acquired position of the reference point on the map as point information. It is preferable that the estimation unit for estimating the image pickup position based on the reference point and the point information is further provided, and the image pickup position acquisition unit acquires the estimated image pickup position.

Further, the image acquisition unit acquires a captured moving image including the captured image, and the image generation unit adjusts the temporal length according to a preset temporal length of the captured moving image. It is preferable to superimpose the rendered moving image including the rendered moving image on the captured moving image to generate a superimposed moving image.

Further, the present invention is a program that causes a computer to function as an image generation device that generates a superimposed image by superimposing a rendered image imitating a building on an captured image of a planned construction site. Based on the image acquisition unit that acquires the captured image and the rendered image, which are celestial sphere images, the imaging position acquisition unit that acquires the imaging position including the height at which the captured image is captured, and the acquired imaging position. A superimposition position determination unit that determines the superimposition position of the captured image and the rendered image, an image conversion unit that converts the rendered image into an all-sky image based on the determined superimposition position, and the rendering converted with the captured image. The present invention relates to a program that functions as an image generation unit that superimposes an image at a determined superimposition position to generate a superimposition image.

According to the present invention, it is possible to provide an image generation device and a program capable of easily correcting a rendered image according to a spherical image.

It is a schematic diagram which shows the relationship between the image pickup position of the image capture image synthesized by the image generation apparatus which concerns on 1st Embodiment of this invention, and the planned construction site included in the image pickup object. It is a schematic diagram which shows the superposed image superposed by the image generation apparatus of 1st Embodiment. It is a schematic diagram which shows the captured image synthesized by the image generation apparatus of 1st Embodiment. It is a schematic diagram which shows the rendered image synthesized by the image generation apparatus which concerns on 1st Embodiment. It is a block diagram which shows the structure of the image generation apparatus of 1st Embodiment. It is a block diagram which shows the structure of the image generation apparatus which concerns on 2nd Embodiment of this invention. It is a block diagram which shows the structure of the image generation apparatus which concerns on 3rd Embodiment of this invention.

Hereinafter, the image generator 1 and the program according to each embodiment of the present invention will be described with reference to FIGS. 1 to 7.
First, in explaining the image generation device 1 and the program according to each embodiment, the generated images (hereinafter, also referred to as superimposed images) will be described with reference to FIGS. 1 to 4.

The superimposed image is, for example, as shown in FIGS. 1 and 2, an image in which a rendered image imitating a building is superimposed on an captured image obtained by capturing an image of a planned construction site A from an imaging position P. Specifically, the superimposed image is an image in which a rendered image of a building to be built is superimposed on a live-action image of the planned building site A. By superimposing the rendered image on the live-action image, it is possible to confirm the harmony between the building to be built and the adjacent building.

In each of the following implementation systems and others, a spherical image is used as the captured image as shown in FIG. This makes it possible to check the entire circumference from a predetermined viewpoint. Therefore, it is possible to confirm the harmony with the surrounding environment as well as the adjacent building.

On the other hand, the rendered image is, for example, a 3D model as shown in FIG. Rendered images are made using, for example, meshes and textures. Therefore, the rendered image needs to be converted so that it can be superimposed on the spherical image from a predetermined viewpoint. The image generation device 1 according to each of the following embodiments performs conversion of the rendered image based on the image pickup position P including the height. As a result, the rendered image can be easily corrected according to the spherical image.

[First Embodiment]
Next, the image generation device 1 and the program according to the first embodiment of the present invention will be described with reference to FIGS. 1 to 5.
The image generation device 1 is, for example, a device that acquires and superimposes a captured image and a rendered image. As shown in FIG. 5, the image generation device 1 includes an image acquisition unit 11, an image pickup position acquisition unit 12, a superposition position determination unit 13, an image conversion unit 14, an image generation unit 15, and an output unit 16. To prepare for.

The image acquisition unit 11 is realized, for example, by operating the CPU. The image acquisition unit 11 acquires a captured image and a rendered image, which are spherical images. The image acquisition unit 11 acquires, for example, a spherical image captured by the user as a captured image, and acquires a spherical image captured from a visible imaging position P of the planned construction site A. Further, the image acquisition unit 11 acquires, for example, an image previously created as a 3D model of a building as a rendered image.

The image pickup position acquisition unit 12 is realized, for example, by operating the CPU. The imaging position acquisition unit 12 acquires the imaging position P including the height at which the captured image is captured. The imaging position acquisition unit 12 acquires, for example, latitude, longitude, and height (altitude) as the imaging position P. In the present embodiment, the image pickup position acquisition unit 12 acquires an image pickup position P to be input by using an input device (not shown) such as a keyboard.

The superimposition position determining unit 13 is realized, for example, by operating the CPU. The superimposition position determination unit 13 determines the superimposition position of the captured image and the rendered image based on the acquired image pickup position P. The superimposition position determination unit 13 determines, for example, the position of the planned construction site A included in the captured image as the superimposition position of the rendered image with the image pickup position P as the viewpoint. Further, the superimposition position determination unit 13 determines the feature points of the rendered image to be superimposed on the reference point of the planned construction site A. The superimposition position determination unit 13 determines, for example, the feature points of the rendered image corresponding to the reference points, using a plurality of points at the boundary of the planned construction site A with the road as reference points. That is, the superimposition position determination unit 13 determines the feature points corresponding to the reference points among the plurality of points at the boundaries with the road included in the rendered image.

The image conversion unit 14 is realized, for example, by operating the CPU. The image conversion unit 14 converts the rendered image into a spherical image based on the determined superimposed position. The image conversion unit 14 converts, for example, a rendered image with the image pickup position P as a viewpoint into a spherical image so that the position of the reference point and the position of the feature point overlap each other.

The image generation unit 15 is realized, for example, by operating the CPU. The image generation unit 15 superimposes the captured image and the converted rendered image at a determined superimposition position to generate a superimposition image. The image generation unit 15 generates a superposed image by superimposing the captured image and the converted rendered image, for example, by superimposing the position of the feature point on the position of the reference point.

The output unit 16 is realized, for example, by operating the CPU. The output unit 16 outputs the generated superimposed image to the outside. The output unit 16 displays the superimposed image on a display device (not shown) such as a display, for example. Further, the output unit 16 outputs (transmits) the superimposed image to, for example, another terminal or the like.

Next, the operation of the image generation device 1 according to the present embodiment will be described.
First, the image acquisition unit 11 acquires a captured image and a rendered image. Next, the imaging position acquisition unit 12 acquires the imaging position P of the captured image. Next, the superimposition position determination unit 13 determines the superimposition position of the rendered image with respect to the captured image.

Next, the image conversion unit 14 converts the rendered image into a spherical image based on the determined superimposed position. Next, the image generation unit 15 superimposes the captured image and the converted rendered image to generate a superimposed image. Next, the output unit 16 outputs the generated superimposed image.

Next, the program according to this embodiment will be described.
Each configuration included in the image generation device 1 can be realized by hardware, software, or a combination thereof. Here, what is realized by software means that it is realized by a computer reading and executing a program.

Programs can be stored and supplied to a computer using various types of non-transitory computer readable medium. Non-temporary computer-readable media include various types of tangible storage media. Examples of non-temporary computer-readable media include magnetic recording media (eg, flexible disks, magnetic tapes, hard disk drives), optomagnetic recording media (eg, optomagnetic disks), CD-ROMs (Read Only Memory), CD- Includes R, CD-R / W, semiconductor memory (eg, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (random access memory)). The display program may also be supplied to the computer by various types of transient computer readable medium. Examples of temporary computer readable media include electrical, optical, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

According to the image generation device 1 and the program according to the above embodiment, the following effects are obtained.
(1) An image generation device 1 that superimposes a rendered image imitating a building on an captured image of a planned construction site A to generate a superimposed image, and includes an captured image and a rendered image that are all-sky images. The superimposed position of the captured image and the rendered image is determined based on the image acquisition unit 11 for acquiring the image, the imaging position acquisition unit 12 for acquiring the imaging position P including the height at which the captured image is captured, and the acquired imaging position P. A superimposition position determination unit 13 to be determined, an image conversion unit 14 to convert a rendered image into an all-sky image based on the determined superimposition position, and a superimposition of the captured image and the converted rendered image at the determined superimposition position. An image generation unit 15 for generating a superimposed image is provided.
Further, it is a program that causes a computer to function as an image generation device 1 that generates a superimposed image by superimposing a rendered image imitating a building on an image captured by capturing a planned construction site A, and the computer is used as an all-sky image. Image acquisition unit 11 for acquiring the captured image and the rendered image, the imaging position acquisition unit 12 for acquiring the imaging position P including the height at which the captured image was captured, and the captured image and the captured image based on the acquired imaging position P. The superimposition position determination unit 13 that determines the superimposition position of the rendered image, the image conversion unit 14 that converts the rendered image into an all-sky image based on the determined superimposition position, and the captured image and the converted rendered image are determined. It functions as an image generation unit 15 that superimposes at a superimposing position to generate a superimposing image.
As a result, the rendered image can be easily corrected according to the spherical image. That is, the rendered image can be easily corrected according to the captured image. Therefore, it is possible to easily obtain a superimposed image in which the captured image and the rendered image are superimposed. Further, since the captured image and the rendered image are superimposed based on the imaging position P including the height, it is possible to obtain a superimposed image with less discomfort. Further, since the reference point and the feature point are associated with each other by using the height information of the imaging position, the accuracy of superimposition can be improved as compared with the case where there is no height information of only the latitude and longitude.

[Second Embodiment]
Next, the image generator 1 and the program according to the second embodiment of the present invention will be described with reference to FIG. In the description of the second embodiment, the same components as those of the above-described embodiment are designated by the same reference numerals, and the description thereof will be omitted or simplified. The image generation device 1 and the program according to the second embodiment acquire reference points and feature points, and estimate the image pickup position P. Further, the image generation device 1 according to the second embodiment is different from the first embodiment in that the image pickup position acquisition unit 12 acquires the estimated image pickup device. As shown in FIG. 6, the image generation device 1 according to the second embodiment further includes a reference point acquisition unit 17, a feature point acquisition unit 18, and an estimation unit 19.

The reference point acquisition unit 17 is realized, for example, by operating the CPU. The reference point acquisition unit 17 acquires a predetermined reference point in the captured image. The reference point acquisition unit 17 acquires, for example, a position as a reference point in the captured image. The reference point acquisition unit 17 acquires the reference point by accepting the designation of the position of the captured image. In the present embodiment, the reference point acquisition unit 17 acquires a plurality of points at the boundary with the road and the boundary with the land of the neighboring house as reference points.

The feature point acquisition unit 18 is realized, for example, by operating the CPU. The feature point acquisition unit 18 acquires feature points indicating positions in the rendered image corresponding to the acquired reference points. The feature point acquisition unit 18 acquires feature points, for example, by accepting the designation of the position of the rendered image.

The estimation unit 19 is realized, for example, by operating the CPU. The estimation unit 19 estimates the imaging position P based on the acquired reference points and feature points. The estimation unit 19 estimates the imaging position P including the height from, for example, the position of the reference point and the position of the corresponding feature point. The estimation unit 19 estimates the imaging position P from, for example, the degree of deviation between the position of the reference point and the position of the corresponding feature point.

Next, the operation of the image generation device 1 according to the present embodiment will be described.
First, the image acquisition unit 11 acquires a captured image and a rendered image. Next, the reference point acquisition unit 17 acquires the reference point. Further, the feature point acquisition unit 18 acquires feature points. Next, the estimation unit 19 estimates the imaging position P including the height from the acquired reference point and feature point. The imaging position acquisition unit 12 acquires the imaging position P of the captured image.

Next, the superimposition position determination unit 13 determines the superimposition position of the rendered image with respect to the captured image. Next, the image conversion unit 14 converts the rendered image into a spherical image based on the determined superimposed position. Next, the image generation unit 15 superimposes the captured image and the converted rendered image to generate a superimposed image. Next, the output unit 16 outputs the generated superimposed image.

According to the image generation device 1 and the program according to the above embodiment, the following effects are obtained.
(2) The image generation device 1 has a reference point acquisition unit 17 that acquires a predetermined reference point in the captured image, and a feature point acquisition unit that acquires a feature point indicating a position in the rendered image corresponding to the acquired reference point. 18 and an estimation unit 19 for estimating the image pickup position P based on the acquired reference points and feature points are further provided, and the image pickup position acquisition unit 12 acquires the estimated image pickup position P. As a result, even if the imaging position P cannot be acquired, the imaging position P can be estimated if the reference point and the feature point can be obtained. Therefore, it is possible to suppress a sense of discomfort in superimposing the captured image and the rendered image.

[Third Embodiment]
Next, the image generator 1 and the program according to the third embodiment of the present invention will be described with reference to FIG. 7. In the description of the third embodiment, the same components as those in the above-described embodiment are designated by the same reference numerals, and the description thereof will be omitted or simplified. The image generation device 1 and the program according to the third embodiment estimate the image pickup position P including the height by acquiring the reference point and accepting the designation of the image pickup point on the map. Further, the image generation device 1 according to the third embodiment is different from the first embodiment in that the image pickup position acquisition unit 12 acquires the estimated image pickup device. As shown in FIG. 7, the image generation device 1 according to the third embodiment is different from the second embodiment in that it further includes a point information acquisition unit 20. Further, the image generation device 1 according to the third embodiment is different from the second embodiment in that the feature point acquisition unit 18 is not provided. Further, the image generation device 1 according to the third embodiment is different from the second embodiment in that the estimation unit 19 estimates the imaging position P based on the acquired reference point and point information.

The point information acquisition unit 20 is realized, for example, by operating the CPU. The point information acquisition unit 20 acquires the position of the acquired reference point on the map as point information. The point information acquisition unit 20 acquires point information using, for example, EXIF (Exchangeable image file format) included in the captured image.

Next, the operation of the image generation device 1 according to the present embodiment will be described.
First, the image acquisition unit 11 acquires a captured image and a rendered image. Next, the reference point acquisition unit 17 acquires the reference point. Further, the point information acquisition unit 20 acquires the point information. Next, the estimation unit 19 estimates the imaging position P including the height from the acquired reference point and point information. The imaging position acquisition unit 12 acquires the imaging position P of the captured image.

Next, the superimposition position determination unit 13 determines the superimposition position of the rendered image with respect to the captured image. Next, the image conversion unit 14 converts the rendered image into a spherical image based on the determined superimposed position. Next, the image generation unit 15 superimposes the captured image and the converted rendered image to generate a superimposed image. Next, the output unit 16 outputs the generated superimposed image.

According to the image generation device 1 and the program according to the above embodiment, the following effects are obtained.
(3) The image generation device 1 acquires a reference point acquisition unit 17 that acquires a predetermined reference point in a captured image, a point information acquisition unit 20 that acquires the position of the acquired reference point on a map as point information, and an acquisition unit 20. The estimation unit 19 for estimating the image pickup position P based on the reference point and the point information is further provided, and the image pickup position acquisition unit 12 acquires the estimated image pickup position P. As a result, even if the height of the imaging position P cannot be acquired, the height of the imaging position P can be estimated if the reference point and the point information can be obtained. Therefore, it is possible to suppress a sense of discomfort in superimposing the captured image and the rendered image.

Although the preferred embodiments of the image generator and the program of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments and can be appropriately modified.
For example, in the above embodiment, the imaging position acquisition unit 12 acquires the imaging position P by input from the input device, but the present invention is not limited to this. For example, the image pickup position acquisition unit 12 may be acquired from an image pickup position storage unit (not shown) that stores the image pickup position P in advance. Further, the image pickup position acquisition unit 12 may acquire the image pickup position P from an external terminal or the like.

Further, in the second or third embodiment, the reference point acquisition unit 17, the feature point acquisition unit 18, and the point information acquisition unit 20 acquire the reference point, feature point, or point information input by the input device. You may. Further, the reference point acquisition unit 17 and the feature point acquisition unit 18 may acquire the reference point or the feature point from the position of the edge (the position where the spatial frequency is large).

Further, in the above embodiment, the captured image and the rendered image have been described as an example, but the captured moving image including the captured image and the rendered captured image including the rendered image may be superimposed. The image acquisition unit 11 acquires a captured moving image including a captured image. The image generation unit 15 superimposes a rendered moving image including a rendered image, whose time length is adjusted according to a preset time length of the captured moving image, on the captured moving image, and superimposes the moving image. To generate. Here, the image generation device 1 may further include a time acquisition unit (not shown) for acquiring the time (for example, disclosure time and end time) used for superimposition among the acquired captured moving images. Further, the image generation device 1 may further include a rendering moving image generation unit (not shown) that adjusts or generates a rendering moving image according to the acquired time.

1 Image generation device 11 Image acquisition unit 12 Image acquisition position acquisition unit 13 Superimposition position determination unit 14 Image conversion unit 15 Image generation unit 17 Reference point acquisition unit 18 Feature point acquisition unit 19 Estimation unit 20 Point information acquisition unit A Planned construction site P Imaging position

Claims (5)

It is an image generation device that generates a superimposed image by superimposing a rendered image imitating a building on an captured image of a planned construction site.
An image acquisition unit that acquires the captured image and the rendered image, which are spherical images,
An imaging position acquisition unit that acquires an imaging position including the height at which the captured image is captured, and an imaging position acquisition unit.
A superimposition position determining unit that determines the superimposition position of the captured image and the rendered image based on the acquired image pickup position, and
An image conversion unit that converts the rendered image into an omnidirectional image based on the determined superimposed position, and
An image generation unit that generates a superposed image by superimposing the captured image and the converted rendered image at a determined superimposition position.
An image generator equipped with. A reference point acquisition unit that acquires a predetermined reference point in the captured image, and a reference point acquisition unit.
A feature point acquisition unit that acquires a feature point indicating a position in the rendered image corresponding to the acquired reference point, and a feature point acquisition unit.
An estimation unit that estimates the imaging position based on the acquired reference points and feature points, and an estimation unit.
Further prepare
The image generation device according to claim 1, wherein the image pickup position acquisition unit acquires the estimated image pickup position. A reference point acquisition unit that acquires a predetermined reference point in the captured image, and a reference point acquisition unit.
A point information acquisition unit that acquires the acquired position of the reference point on the map as point information, and
An estimation unit that estimates the imaging position based on the acquired reference point and the point information.
Further prepare
The image generation device according to claim 1, wherein the image pickup position acquisition unit acquires the estimated image pickup position. The image acquisition unit acquires a captured moving image including the captured image, and obtains the captured moving image.
The image generation unit superimposes a rendered moving image including the rendered image whose time length is adjusted according to a preset time length of the captured moving image on the captured moving image. The image generation device according to claim 1, which generates a superimposed moving image. It is a program that makes a computer function as an image generation device that generates a superimposed image by superimposing a rendered image that imitates a building on an captured image that captures a planned construction site.
The computer
An image acquisition unit that acquires the captured image and the rendered image, which are spherical images.
An imaging position acquisition unit that acquires an imaging position including the height at which the captured image is captured,
A superimposition position determining unit that determines the superimposition position of the captured image and the rendered image based on the acquired image pickup position.
An image conversion unit that converts the rendered image into a spherical image based on the determined superimposed position.
An image generation unit that generates a superposed image by superimposing the captured image and the converted rendered image at a determined superimposition position.
A program that functions as.

Images