JP6975370B1 - Image display method, program and data generation method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image display method, a program and a data generation method capable of making a user feel the actual state of a building via a communication network and enhancing the presence of the user while suppressing the amount of data. .. An image display method executed by an image display device for displaying an image representing an object is to detect a step of saving a 3D model that reproduces the object and a trigger operation to the image display device. , A step of transmitting a trigger signal, a step of receiving an image content based on the trigger signal, a step of synthesizing an image contained in the image content into a 3D model, and a step of displaying the combined image with the 3D model. To prepare for. The resolution of the 3D model is lower than the resolution of the image. [Selection diagram] Fig. 2

Description

The present invention relates to a technique for displaying an internal image of an object such as a building, and more particularly to an image display method, a program, and a data generation method.

Generally, when selecting a real estate property, the desired property is selected by selecting the property while looking at the floor plan and viewing the selected property. In addition, model rooms are exhibited in order to increase the likability of the target audience such as purchasers and renters.

In recent years, there has been a demand for a technique for guiding a property by displaying an image or a floor plan of a desired property on a computer via a communication network such as the Internet. It is said to be a remote guidance technology used for so-called real estate referral services.

In the conventional remote guidance technology, a 3D model technology that creates a 3D model that reproduces the structure of the property and displays the created 3D model on the net, and a panoramic technology that displays multiple panoramic images of buildings and properties one by one. It is divided into.

In the 3D model technology, the entire structure of the building is reproduced three-dimensionally, and the display of the image is changed as if the user is inside the actual building and is moving inside the building. Therefore, the amount of data of the 3D model becomes very large. In addition, it is desirable that the screen resolution is high in order to enhance the sense of presence. However, the higher the resolution, the larger the amount of data. Therefore, the 3D model has a problem of communication delay when played back on a mobile device such as a smartphone, and the usability on the mobile device is poor.

In the panorama technology, even if a high-resolution image is used, the amount of data is not large as compared with the 3D model. However, even if the panoramic images can be viewed one by one, the user cannot get the feeling of moving in the room. In addition, since the room is partially displayed, the sense of presence in the space is reduced.

The present invention has been made to solve the above-mentioned problems, and it is possible to make the user feel the actual state of the building through the communication network, and it is possible to enhance the user's sense of presence while suppressing the amount of data. It is an object of the present invention to provide an image display method, an image display program, and a data generation method.

In order to solve the above problems, the first aspect of the present invention is an image display method executed by an image display device to display an image expressing an object, and stores a 3D model that reproduces the object. Steps to perform, a step to send a trigger signal when a trigger operation to the image display device is detected, a step to receive image content based on the trigger signal, and a 3D model to synthesize the image contained in the image content. The resolution of the 3D model is lower than that of the image, including the step of displaying the combined image with the 3D model.

According to this aspect, the image display device can display a high image quality while suppressing the amount of data by synthesizing a 3D model having a low resolution and an image having a high resolution. Further, since the position of the image on the object can be grasped by the 3D model, the image display device can easily express the depth and the three-dimensional effect of the object. As a result, the user's sense of presence can be ensured.

A second aspect of the present invention is the image display method of the first aspect, wherein the image includes a plan view and a panoramic image, the plan view and the 3D model have a plurality of observation points, and the panoramic image is a panoramic image. , Has one or more other observation points different from its own observation point, each observation point is associated with a panoramic image, and when a trigger operation to each observation point is detected, the triggered observation Display a panoramic image corresponding to a point.

In this embodiment, the plan view and the panoramic image are combined with the 3D model when transmitted to the image display device. According to this aspect, when the observation point in the displayed plan view or panoramic image is triggered to switch to another panoramic image and displayed, the plan view or panoramic image is combined with the 3D model. Instead of just switching the images to be displayed, there is a sense of realism that you can jump into the space of the 3D model and stand at the corresponding observation point. Further, by synthesizing the 3D model and the image to be displayed, it becomes easy to express a sense of depth of space in the image to be displayed, and it is possible to further enhance the sense of presence. In addition, when switching between panoramic images, the sense of depth before and after the switching display changes, which has an impact and enhances the sense of presence.

A third aspect of the present invention is that when a trigger operation on the displayed image is detected in the image display method of the second aspect, the position where the trigger operation is received is a place where there is no observation point. The image display device acquires the position information of the closest observation point at the position where the trigger operation is received and includes it in the trigger signal.

According to this aspect, even if a trigger operation is performed at an arbitrary point on the display image, it is detected as a trigger operation, and it is easy to use.

A fourth aspect of the present invention is the image display method of the second or third aspect, wherein the image includes a 3D icon for switching to a 3D model, and when a trigger operation to the 3D icon is detected, If the displayed image is switched to the 3D model display and the displayed image is a panoramic image, a simplified diagram of the object is displayed in a part of the panoramic image, and the simplified diagram is the position in the object. The simulated movement display includes a simulated movement display indicating the image and the observation angle, and the simulated movement display is displayed so as to move in the simplified diagram according to the position and the observation angle of the displayed panoramic image. According to this aspect, it is possible to intuitively feel where the user is viewing in an object such as a building, and the sense of presence is enhanced.

A fifth aspect of the present invention is a panorama displayed after switching when switching from a panoramic image or a 3D model to the next panoramic image in the image display method according to any one of the second to fourth aspects. The image is displayed at the same observation angle as the panoramic image displayed before the switching or the 3D model. According to this aspect, since the images before and after the switching display are switched at the same observation angle, the images can be displayed as if they were "straight" from the previous observation point to the later observation point. It has a high impact on the user and can enhance the sense of presence.

A sixth aspect of the present invention is to display a pointer that moves according to the movement of the cursor in the displayed panoramic image in the image display method according to any one of the second to fifth aspects, and the shape and size of the pointer. However, it changes according to the structure and depth where the cursor touches.

According to this aspect, when the cursor is moved while the user is viewing the panoramic image, the shape and size of the pointer change, so that an object in space or a physically existing structure can be detected. You can get the feeling of touching. As a result, the sense of depth of the space and the three-dimensional effect of the object and the structure can be felt, and the sense of presence can be further enhanced.

According to the seventh aspect of the present invention, in the image display method according to any one of the first to sixth aspects, when a trigger operation to a part other than the 3D model is detected while the 3D model is displayed, the trigger operation is detected. Rotate the 3D model.

According to this aspect, by rotating the 3D model, the appearance and the overall structure of the building can be confirmed from various observation angles.

An eighth aspect of the present invention further comprises a step of downloading a 3D model from a server in any of the image display methods of the first to seventh aspects, and a step of receiving image content downloads the 3D model. Separately from the step, it is executed according to the trigger operation.

According to this aspect, the image content to be displayed is not transmitted together with the 3D model, but is transmitted to the image display device by a trigger signal separately from the storage of the 3D model. As a result, the amount of data transmitted at one time is suppressed, and the delay in communication is suppressed. In addition, the 3D model has a low resolution and cannot display the details of the object with high image quality, but since the image with high resolution is transmitted and displayed according to the trigger operation, high image quality display can be ensured and the amount of data is suppressed. Be done.

A ninth aspect of the present invention is an image display program for causing a computer to execute the image display method according to any one of the first to eighth aspects.

A tenth aspect of the present invention is a data generation method for displaying a panoramic image of an object on an image display device, in which a 3D model creation step of creating a 3D model that reproduces the object and a 3D model are rendered. A rendering step to generate, an image generation step to generate a panoramic image representing an object, a 3D model compression step to compress a rendered 3D model to generate a compressed 3D model, and observations within the compressed 3D model. It includes a panoramic image matching step for associating point and observation point position information with a panoramic image.

In the eleventh aspect of the present invention, in the data generation method of the tenth aspect, the resolution of the panoramic image is higher than the resolution of the compressed 3D model.

A twelfth aspect of the present invention comprises picking up a panoramic image from a rendered 3D model in the data generation method of the tenth or eleventh aspect, the image generation step.

According to the data generation method of this aspect, the compressed 3D model reproduces the object, although the amount of data is small. Therefore, when displaying the panoramic image combined with the 3D model, the depth and dimensions in the space can be easily expressed. Further, since the compressed 3D model and the panoramic image of each observation point can be separately transmitted to the image display device, the amount of data at the time of transmission can be suppressed.

According to the present invention, it is possible to make the user feel the actual state of the building via the communication network, and it is possible to enhance the presence of the user while suppressing the amount of data.

It is a schematic diagram which shows the structure of an image display system. It is a flowchart which shows the process of an image display method. It is a figure which displays the 3D model of a building. It is the figure which rotated the 3D model of FIG. 3 clockwise. It is a figure which displays the panoramic image at the observation point P1. It is a figure which displays the panoramic image at the observation point P2. It is a figure which displays the panoramic image at the observation point P2. It is a figure which displays the plan view of a building. It is a figure which shows the change of a pointer by moving a cursor in a panoramic image.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a schematic diagram showing a configuration of an image display system according to an embodiment. The image display system of the present embodiment includes a server 1 and an image display device 2.

The server 1 is connected to the image display device 2 so as to be able to transfer data via the communication network 100. The server 1 includes an information processing unit 11 and a content storage unit 12. The information processing unit 11 includes a processor such as a CPU and a memory such as a RAM and a ROM. The information processing unit 11 requests the image content stored in the content storage unit 12 in response to the signal from the image display device 2. The content storage unit 12 stores image content required for displaying an image, and transmits the image content to the image display device in response to a request from the information processing unit 11.

Examples of the image content include a two-dimensional image, a panoramic image, a 3D model, audio data such as a sound effect output together with the image and the 3D model, an introduction text, a moving image, and the like.

A two-dimensional image is a flat image such as a photograph. A panoramic image is an image including an image of all landscapes in a range of 360 degrees in the horizontal direction and 180 degrees in the vertical direction of an object. When taking a panoramic image, a group of pictures taken separately at predetermined angles are taken around the shooting point, and pictures with adjacent and overlapping areas are seamlessly connected by CG software or the like, and they are connected. Combine to a spherical panoramic image. The panoramic image may be not only a spherical panoramic image generated by a photograph taken by a camera, but also a panoramic image created by using an image generated by CG technology. The full observation angle panoramic image is an image that allows you to check the situation within the range of 360 degrees back and forth, left and right, and 180 degrees up and down to check the space, but the display screen (screen, etc.) of the computer is two-dimensional. Therefore, only a part of the panoramic image is displayed depending on the display angle (observation angle) to be displayed.

The 3D model of the present embodiment is a 3D model that proportionally reproduces elements in a space such as a building that is an object. The elements in the space of the building are the whole structure of the target building, the internal structure, the whole space, each child space partitioned from the whole space by walls, the relationship between each child space and the whole space, the depth of the space, It is a physical entity that can be visually confirmed, such as dimensions and objects inside space. The 3D model is composed of computer graphics (abbreviation: CG) by arranging a coordinate system for ordinary 3D technology. A 3D model is data that can be read and displayed by a computer. As an example of the 3D model, data composed of a large number of polygons such as OBJ file data and MTL file data can be mentioned. OBJ file data is one of the 3D model formats, and includes an ".obj" file that stores information such as vertices and normals, and a ".mtl" file that stores model color and texture information. It is composed of. Even if there is no ".mtl" file, it is possible to draw the shape of the 3D model as long as there is an ".obj" file.

A polygon is an element used in CG technology to reproduce an object by combining polygons such as triangles and quadrangles. By reducing the number of polygons, the amount of data becomes smaller and the resolution becomes lower.

When the object building has each room like an apartment, the 3D model may represent each room and the overall structure of each dwelling unit composed of each room. .. Figure 3 shows a 3D model of a two-story detached house. In the 3D model of FIG. 3, since there is no outer wall and ceiling, by displaying the outer wall and ceiling as if they are transparent, each room partitioned by the wall, the entire structure of the house composed of each room, and the inside are inside. You can confirm the existence of all furniture, home appliances, etc.

The image display device 2 is a computer that can communicate with the server 1 via the communication network 100 as a user terminal and can display an image. The image display device 2 is, for example, a mobile terminal such as a smartphone or a tablet, or a personal computer. Although one image display device 2 is shown in FIG. 1, the number of image display devices is not limited.

The image display device 2 includes a display control unit 21, a display unit 22, and an operation detection unit 23. The display control unit 21 includes a processor such as a CPU and a memory such as RAM and ROM. The display control unit 21 reads the image content transmitted from the content storage unit 12, synthesizes the display data by the trigger signal, and causes the display unit 22 to display the image to be displayed. The display unit 22 is a two-dimensional display screen such as a screen, and can display an image including a two-dimensional image and a three-dimensional panoramic image. Further, the display unit 22 may be an input means such as a touch panel.

The operation detection unit 23 is a means for receiving an input from the outside via a touch panel, a keyboard, a mouse, a microphone, and the like. When the operation detection unit 23 receives the trigger operation, a trigger signal indicating that the trigger operation has been received is transmitted to the information processing unit 11. The information processing unit 11 determines an image to be displayed based on the information included in the trigger signal, and requests the image content from the content storage unit 12 of the server 1 based on the determination result.

The trigger operation is not limited to clicking, but may be input such as tapping, sliding, dragging (drag-and-drop) with a mouse or a touch panel, preset recognizable gesture operation, voice operation, or the like.

The operation detection unit 23 reads the meaning of the trigger operation by the position or the icon of the trigger operation, and includes the position information, the observation angle information, the image data information, and the like in the trigger signal.

The image data information is image information such as a two-dimensional image, a panoramic image, and a plan view. The position information is information indicating each position in the 3D model by the coordinate system of the 3D model itself. Each position in the 3D model corresponds to each position in the physical space of the building. The observation angle is an angle (angle displayed on the display unit) when the user observes the building as an object. The direction indicating 0 degrees of the observation angle and the origin of the coordinate system of the 3D model itself are determined in advance when constructing the 3D model.

[Image display method]
Hereinafter, the display method in the present embodiment will be described in detail by taking the internal space of a building such as a condominium or a detached house as an example. Of the processes described below, a part or all of the processes executed by the display control unit 21 of the image display device 2 may be executed by the information processing unit 11 of the server 1.

FIG. 2 is a flowchart showing the processing of the image display method of the present embodiment. When the display is started, the image display device 2 accesses the server 1. Then, the OBJ file data of the 3D model is transmitted from the server 1 to the image display device 1. The image display device 2 downloads and saves the OBJ file data of the 3D model (step S0).

The display unit 22 displays the 3D model and is in the initial display state (step S1). When the operation detection unit 23 detects a trigger operation on the display unit 22, the trigger signal is transmitted to the information processing unit 11 (step S2). The information processing unit 11 determines the image content to be displayed based on the image data information, the position information, the observation angle information, etc. included in the trigger signal, and requests the content storage unit 12 (step S3). The content storage unit 12 transmits the image content to be displayed, the position information, the observation angle information, and the like to the display control unit 21 of the image display device 2 (step S4). The display control unit 21 reads the transmitted image content, analyzes it, synthesizes display data, and causes the display unit 22 to display an image to be displayed (step S5).

As will be described later, the download process of step S0 may be completed only once. The image display device may repeatedly execute the processes from steps S1 to S5 each time the image is switched and displayed. In the present embodiment, the display content in the initial display state is not limited. When the display unit 22 is in the initial display state, the displayed image may be an image relating to the target space. For example, the image in the initial display state may be a part of a panoramic image such as a photograph in the property, a floor plan, or a 3D model that reproduces the space of the property.

Next, switching from the display of the 3D model to the plan view will be described. The plan view is a projection view or a horizontal cross-sectional view from the upper surface of the object in order to express the structure, arrangement, etc. of the object, and in the case of a property, it is also called a floor plan.

FIG. 3 displays a 3D model of the object. The image display device 2 has already downloaded the 3D model in step S0 described above. FIG. 3 displays a 3D model viewed from a predetermined first observation angle V1 (eg, southward). Further, an icon such as a plan view is installed in a part of the display unit 22 (step S1). When the icon A2 in the plan view is clicked, the operation detection unit 23 transmits a trigger signal for detecting the trigger operation to the icon A2 in the plan view to the information processing unit 11 (step S2). Based on the trigger signal, the information processing unit 11 determines that the image to be displayed on the display unit is a plan view, and requests the content storage unit 12 for the plan view (step S3). The content storage unit 12 transmits the image content including the plan view to the display control unit 21 (step S4). The display control unit 21 of the image display device 2 reads the transmitted image content and synthesizes the plan view included in the image content into the 3D model. As a result, a composite 3D model image of the plan view and the 3D model is generated. The display control unit 21 causes the display unit 22 to display the plan view synthesized with the 3D model (step S5). Then, the display of the 3D model of FIG. 3 is switched to the display of the plan view as shown in FIG.

The 3D model of FIGS. 3 and 4, the panoramic image of FIGS. 5 to 7, and the plan view of FIG. 8 include a plurality of observation points P. The observation point is a place selected in advance as a viewing point inside the object, and each observation point has position information expressing the position of each observation point. Each observation point is associated with a panoramic image seen from each observation point by position information and a corresponding position in a 3D model or a plan view. The panoramic image associated with the observation point is an image in the object that can be viewed while standing at the observation point. From the panoramic image, the situation within the range of 360 degrees in front, back, left and right and 180 degrees up and down can be confirmed. Each panoramic image has the position information of the corresponding observation point. When the trigger operation to each observation point is detected, the display of the panoramic image is switched from the 3D model so as to be inside the 3D model. Then, the display unit displays a panoramic image at the observation point as if the state in the space of the object is being observed at the observation point where the rigger operation is detected.

It will be described that the 3D model is switched to the panoramic image of the observation point and displayed.
FIG. 4 displays a 3D model of the object. The image display device 2 has already downloaded the 3D model in step S0 described above. FIG. 4 displays a 3D model viewed from a predetermined second observation angle V2 (eg, southeast). As shown in FIG. 4, a plurality of observation points are installed in the 3D model, and a trigger operation to the 3D model is possible (step S1). When the operation detection unit 23 detects a trigger operation on the 3D model displayed on the display unit, the operation detection unit 23 transmits a trigger signal to the information processing unit 11 (step S2). For example, when the observation point P1 (first observation point) of the 3D model shown in FIG. 4 is clicked, the operation detection unit 23 sets the position information of the first observation point P1 and the observation angle of the currently displayed 3D model. A trigger signal including the observation angle information indicating (second observation angle V2) is transmitted to the information processing unit 11 (step S2). The information processing unit 11 determines that the image to be displayed on the display unit is a panoramic image of the first observation point P1 based on the position information of the first observation point P1 included in the trigger signal, and determines that the image should be displayed on the first observation point P1. A panoramic image having the position information of P1 is requested from the content storage unit 12 (step S3). The content storage unit 12 transmits the image content including the panoramic image of the first observation point P1 to the display control unit 21 (step S4). The display control unit 21 of the image display device 2 reads the transmitted image content, and synthesizes the panoramic image of the first observation point P1 at the 3D model and the first observation point P1 based on the position information of the first observation point P1. Let me. As a result, a synthetic 3D model image at the first observation point P1 is generated. The display control unit 21 causes the display unit 22 to display a panoramic image of the first observation point P1 synthesized with the 3D model at the second observation angle (step S5). Then, the display state of the 3D model of FIG. 4 is switched to the panoramic image of the first observation point P1 (FIG. 5).

When the first observation point P1 is clicked, the operation detection unit 23 reads the position information of the first observation point P1 and includes it in the trigger signal. When a trigger operation to a position other than the preset observation point is detected, the operation detection unit 23 determines the observation point closest to the position where the trigger is operated, and the observation point closest to the trigger operation (hereinafter referred to as “the nearest observation point”). The position information of the "closest observation point") is read and included in the trigger signal.

The operation detection unit 23 reads the second observation angle of the 3D model (observation angle V2 of the 3D model in FIG. 4) before the switching display as observation angle information and includes it in the trigger signal. Then, the display control unit 21 causes the display unit 22 to display the panoramic image of the first observation point P1 at the second observation angle V2. Therefore, the observation angle of the image displayed on the display unit is the same before and after the switching display between the 3D model and the panoramic image.

Hereinafter, the situation when switching from a 3D model having a different observation angle to a panoramic image at the same observation point will be described with reference to FIGS. 3 and 4.

Since the observation angle V2 (second observation angle) in FIG. 4 and the observation angle V1 (first observation angle) in FIG. 3 are different, the same 3D model is displayed in FIGS. 3 and 4, but each observation is performed. Even if the same observation point is clicked when switching to a panoramic image of a point, the image to be switched and displayed on the display unit is different. Specifically, the observation point P2 (second observation point) is taken as an example. When the second observation point P2 is clicked in the state of FIG. 4, as shown in FIG. 6, the panoramic image of the second observation point P2 is displayed at the same second observation angle V2 as the 3D model of FIG. On the other hand, when the second observation point P2 is clicked in the state of FIG. 3, as shown in FIG. 7, the panoramic image of the second observation point P2 is displayed at the same first observation angle V1 as the 3D model of FIG. .. 6 and 7 are both the same panoramic image of the second observation point P2, but different parts of the panoramic image of the second observation point P2 are displayed because the observation angles are different.

On the other hand, when the second observation point P2 in FIG. 4 is clicked, the operation detection unit 23 detects the position information of the second observation point P2 and the trigger signal including the second observation angle V2 in FIG. 4, and is shown in FIG. As described above, the panoramic image of the second observation point P2 is displayed at the second observation angle V2. When the first observation point P1 in FIG. 4 is clicked, the operation detection unit 23 detects the position information of the first observation point P1 and the trigger signal including the second observation angle V2 in FIG. 4, and as shown in FIG. , The panoramic image of the first observation point P1 is displayed at the second observation angle V2. When the trigger operation is performed on each observation point in FIG. 4, the displayed panoramic image differs depending on the different position information of each observation point, but the second observation angle in FIG. 4 before the observation angle of the displayed panoramic image is completely switched. It is V2.

Next, a case where the observation angle of the panoramic image at the same observation point is changed will be described.
6 and 7 are panoramic images of the same second observation point P2, but the observation angles are different. By dragging the display image of FIG. 6, the observation angle to be displayed can be arbitrarily changed. Therefore, it is possible to switch from the display image of FIG. 6 to the display image of FIG. 7. It is also possible to switch from the display image of FIG. 7 to the display image of FIG. 6 by dragging the display image of FIG. 7. Although it is the same panoramic image of the second observation point P2, different parts of the panoramic image are displayed by changing the observation angle.

For example, as shown in FIG. 6, it is assumed that the initial display state of the display unit 22 is the display of the panoramic image of the second observation point P2 at the second observation angle V2 (step S1). When the operation detection unit 23 detects a drag operation as a trigger operation to the display unit 22, the operation detection unit 23 analyzes the drag operation and determines whether or not the first observation angle V1 is requested. When the operation detection unit 23 determines that the first observation angle V1 is required, the operation detection unit 23 processes the trigger signal including the information for changing to the first observation angle V1 for the panoramic image of the second observation point P2. It is transmitted to the unit 11 (step S2). The information processing unit 11 requests the content storage unit 12 for a panoramic image of the second observation point P2 based on this trigger signal (step S3). The content storage unit 12 transmits the image content including the panoramic image of the second observation point P2 to the display control unit 21 (step S4). The display control unit 21 of the image display device 2 reads the transmitted image content, and uses the panoramic image of the second observation point P2 included in the image content as a 3D model based on the position information of the second observation point P2. It is synthesized at the second observation point P2. As a result, a synthetic 3D model image is generated. The display control unit 21 causes the display unit 22 to display a panoramic image of the second observation point P2 synthesized with the 3D model at the first observation angle V1 (step S5). Then, the panoramic image at the same observation point is changed from the display at the second observation angle V2 (FIG. 6) to the display at the first observation angle V1 (FIG. 7).

In order to reduce the amount of data to be transmitted, when displaying a panoramic image of a certain observation point (for example, P2), if the position information does not change, a trigger signal containing only the information of the observation angle to be changed is transmitted. It is also possible to transmit to the information processing unit 11. Then, each display control unit 21 displays the panoramic image at the changed observation angle according to the data transmitted from the server.
Next, it will be described that the panoramic images of different observation points are switched and displayed.

As shown in FIG. 5, the panoramic image of the first observation point P1 is displayed at the second observation angle V2. A plurality of different observation points (P2, P3, P4, P5, ...) Are also displayed in the panoramic image of the first observation point P1 (step 1).

When the trigger operation to the second observation point P2 is detected in the state of FIG. 5, the operation detection unit 23 includes the position information of the second observation point P2 and the second observation angle V2 of the screen currently displayed. The trigger signal is transmitted to the information processing unit 11 (step S2). The information processing unit 11 requests the content storage unit 12 for a panoramic image having the position information of the second observation point P2 (step S3). The content storage unit 12 transmits the image content including the panoramic image of the second observation point P2 to the display control unit 21 (step S4). The display control unit 21 of the image display device 2 reads the transmitted image content, and uses the panoramic image of the second observation point P2 included in the image content as a 3D model based on the position information of the second observation point P2. It is synthesized at the second observation point P2. As a result, a synthetic 3D model image of the second observation point P2 is generated. As shown in FIG. 6, the display control unit 21 causes the display unit 22 to display a panoramic image of the observation point P2 synthesized with the 3D model at the second observation angle V2 (step S5).

As a result, the panoramic image of the first observation point P1 is switched to the panoramic image of the second observation point P2. Since the observation angles before and after the switching display are the same, it is possible to give the user the feeling of moving from the first observation point P1 to the second observation point P2. As a result, the user can be given the feeling of moving in the space, and the user's sense of presence can be enhanced.

As shown in FIG. 5, when an arbitrary part P0 (a part other than a preset observation point) of the displayed image is clicked to perform a trigger operation, in step S2, the display control unit 21 performs this trigger. The position information of the second observation point P2 closest to the place where the operation occurred P0 is acquired and included in the trigger signal.

As shown in FIGS. 5 and 6, a simplified view K of the plan view of the property is displayed in a part of the displayed image. The simplified diagram K has a simulated movement display C. The simulated movement display C shows the position and the observation angle of the observation point of the currently displayed panoramic image at the property. That is, the simulated movement display C indicates the position of the user in the property and the direction of observation. For example, when the display image is switched from FIG. 5 to FIG. 6, the simulated movement display C moves from the observation points P1 to P2. FIG. 8 is a plan view, and is also an enlarged view of a simplified diagram (simplified diagram K in FIG. 6) when the simulated movement display C moves to the observation point P2.

When the panoramic image of FIG. 6 is displayed, the operation detection unit 23 detects a trigger signal requesting switching display to the plan view by a trigger operation to the icon A2 of the plan view or the simplified view K. The panoramic image shown in FIG. 6 is switched to and displayed in the plan view shown in FIG. The plan view has a plurality of observation points associated with each panoramic image. Therefore, by clicking one of the observation points in the plan view, it is possible to switch the display from the plan view to the panoramic image.

In this embodiment, the 3D model can be rotated by the user's operation. For example, when the trigger operation to the 3D icon A1 shown in FIGS. 5 to 8 is detected, the operation detection unit 23 reads as an instruction to display the 3D model and displays the 3D model stored in the image display device. To display. For example, when the 3D model is displayed at the first observation angle V1 as shown in FIG. 3, when the cursor G is used to drag a part other than the 3D model on the display screen, the 3D model is rotated as shown in FIG. The 3D model is displayed at a second observation angle V2 different from 3.

Next, as a means for expressing the detailed structure of the building in the panoramic image, the change of the pointer Y due to the movement of the cursor G will be described.

FIG. 9 is a diagram showing a part of the panoramic image of the observation point P2 at the observation angle V2. In a panoramic image, the user has a pointer that moves according to the movement of the cursor when the user operates the cursor G with a mouse or the like to move the cursor G in the display image. When the cursor G moves arbitrarily in the panoramic image displayed, the shape and size of the pointer change due to the difference in the structure and depth where the cursor G touches. For example, when the cursor G moves along the dotted line in FIG. 9, the pointer changes from Y0 to Y9 due to differences in the structure, angle, depth, etc. of the object touched by the cursor G, and its shape and size change. do. For example, when the cursor is placed on the upper surface of the table, the shape of the pointer Y0 becomes an ellipse along the upper surface of the table. As a result, the user feels that the portion where the cursor G is placed is the upper surface of the table. When the cursor G is placed on the table leg, the orientation and size of the ellipse of the pointer Y1 changes, and the user feels that the table leg is a vertical plane, perpendicular to the top surface of the table, and has two faces. Let me. Further, the pointers Y5 and Y7 have the same shape but different sizes. The pointer Y5 near the observation point P2 is displayed larger than the pointer Y7 far from the observation point P2. The same shape indicates that the planes on which the pointers Y5 and Y7 are placed are parallel to each other, and different sizes indicate different depths. Thereby, the user can be made to feel that the place where Y7 is located is a surface parallel to the place where Y5 is located, but is a separate surface having different depths. Further, since the shapes of the pointers Y5 and Y10 are almost the same in size, it can be shown that the distances from the observation points are on the same plane.

In the example shown in FIG. 9, the cursor G is displayed separately from the pointers Y0 to Y10, but the cursor itself may be a pointer. Further, the pointer is not limited to the circular shape and the elliptical shape as shown in FIG. 9, and may have other shapes.

In the present embodiment, synthesizing the image and the 3D model means associating the 3D model with the image corresponding to each part of the 3D model in correspondence with the position of each image in the 3D model. For this synthesis work, it is possible to use a conventional technique such as WebGL. When the image combined with the 3D model is displayed on a two-dimensional display unit 22 such as a screen, it looks like a two-dimensional image, but since the position of the image on the object can be grasped by the 3D model, the object is displayed. It becomes easier to express depth, dimensions, and three-dimensional effect.

[Image generation method]
Next, a method of generating image data in the present embodiment will be described.

First, using CG technology, a 3D model that reproduces the structure of the building as an object is created. The 3D model is mainly an OBJ file. Then, in order to enhance the sense of reality, the 3D model is rendered. Rendering reproduces the actual colors, patterns, textures, etc. of objects in the building on a 3D model. Then, the rendered 3D model is saved. Since the 3D model after rendering has a high resolution, the amount of data of the entire 3D model is large.

In the present embodiment, panoramic images at a plurality of observation points are picked up from the rendered 3D model and saved as panoramic images at each observation point. The saved panoramic image has the position information of its own observation point. By reducing the number of polygons for the rendered 3D model, the rendered 3D model is compressed. The compressed rendered 3D model is stored in the content storage of the server.

Then, the panoramic image of each picked-up observation point is associated with each observation point in the compressed 3D model by the position information of each observation point. Further, one or more other observation points are set in each panoramic image, and the panoramic image is associated with each observation point.

The compressed and rendered 3D model generated in this way is displayed on the image display device 2 as the 3D model of FIGS. 3 and 4 described above. The compressed 3D model has a lower resolution than the rendered 3D model. The resolution of the compressed 3D model is lower than each panoramic image picked up from the rendered 3D model. As a result, the amount of data obtained by the compressed 3D model is significantly reduced, and no time delay occurs even in a communication terminal such as a mobile phone. The panoramic image is picked up from a high resolution uncompressed, post-rendered 3D model. Therefore, it is possible to secure the user to display a panoramic image with high resolution. The image display device has accessed the server and downloaded the compressed 3D model after rendering.

In the above embodiment, a high-resolution panoramic image is picked up from the rendered 3D model, but the step of picking up is omitted, and the panoramic photograph created by taking a picture with a camera is compressed into a 3D model. It is also possible to associate it with each observation point of. Then, after creating the 3D model, it may be compressed immediately and the panoramic image prepared in advance may be associated with each observation point of the compressed 3D model. The work of picking up a high-resolution panoramic image from the rendered 3D model may be performed by generating a panoramic image by a known CG technique.

In the above embodiment, the target space of the building has been described mainly by taking a detached house as an example, but the target space is not limited to this. It may be an artificial or naturally formed space such as a museum, a department store sales space, an in-house space for automobiles, a factory, a natural cave, or a park.

In the above embodiment, in order to enhance the presence of the user, the observation angle of the image displayed on the display unit is the same before and after the switching display between the 3D model and the panoramic image, but the observation after switching is performed. The angle may be set arbitrarily.

Although the present invention has been described above using the embodiments, it goes without saying that the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various changes or improvements can be made to the above embodiments. It is also clear from the description of the claims that the form with such changes or improvements may be included in the technical scope of the present invention.

11 Information processing unit 12 Content storage unit 21 Display control unit 22 Display unit 23 Operation detection unit

Claims (7)

An image display method performed by an image display device to display an image representing an object.
A step to save a compressed 3D model that reproduces the object,
When the trigger operation to the image display device is detected, the step of transmitting the trigger signal and
The step of receiving the image content based on the trigger signal and
The image included in the image content is combined with the compressed 3D model, and the compressed 3D model and the combined image are displayed.
The compressed 3D model is generated by compressing a rendered rendered 3D model that reproduces the structure of the object and is rendered.
The resolution of the compressed 3D model, rather low than the resolution of the image,
The image includes a plan view and a panoramic image.
The plan view and the compressed 3D model have a plurality of observation points.
The panoramic image has one or more other observation points different from its own observation point.
Each of the observation points is associated with the panoramic image, respectively.
When a trigger operation to each of the observation points is detected, the panoramic image corresponding to the triggered observation point is displayed.
When switching from the panoramic image or the compressed 3D model to the next panoramic image, the panoramic image displayed after the switching is the panoramic image displayed before the switching or the compressed 3D. Displayed at the same viewing angle as the model,
Image display method. When the trigger operation to the displayed image is detected, if the position where the trigger operation is received is not the observation point, the image display device is closest to the position where the trigger operation is received. The position information of the observation point is acquired and included in the trigger signal.
The image display method according to claim 1. The image includes a 3D icon for switching to the compressed 3D model.
When the trigger operation to the 3D icon is detected, the displayed image is switched to the display of the compressed 3D model.
When the displayed image is the panoramic image, a simplified diagram of the object is displayed on a part of the panoramic image, and the simplified diagram shows a position and an observation angle in the object. Including moving display
The simulated movement display is displayed so as to move in the simplified diagram according to the position and observation angle of the displayed panoramic image.
The image display method according to claim 1 or 2. In the displayed panoramic image, a pointer that moves according to the movement of the cursor is displayed, and the shape and size of the pointer change according to the structure and depth of the touch of the cursor.
The image display method according to any one of claims 1 to 3. When a trigger operation to a part other than the compressed 3D model is detected while the compressed 3D model is displayed, the compressed 3D model is rotated.
The image display method according to any one of claims 1 to 4. Further with the step of downloading the compressed 3D model from the server,
The step of receiving the image content is executed in response to the trigger operation separately from the step of downloading the compressed 3D model.
The image display method according to any one of claims 1 to 5. An image display program for causing a computer to execute the image display method according to any one of claims 1 to 6.

Images