JP2021196851A - Image display method, image display system, and image display program - Google Patents

Abstract

To provide an image display method, an image display system, and an image display program that allow a plurality of remote users to view the same image as if they were in the same place in the same space and can enhance the sense of presence to the users.SOLUTION: An image display method includes the steps of: a plurality of image display devices accessing a server via a communication network; the server transmitting a three-dimensional model reproducing an object to the image display devices; when any image display device of the image display devices detects a trigger operation, the image display device transmitting a trigger signal to the server; the server transmitting image content containing an image and an observation angle included in the trigger signal to the image display devices based on the trigger signal; and each image display device incorporating the image contained in the image content with the three-dimensional model by the trigger signal and displaying the image incorporated with the three-dimensional model.SELECTED DRAWING: Figure 2

Description

The present invention relates to a technique for displaying an image of an object such as a building, and more particularly to an image display method, an image display system, and an image display program.

Generally, when selecting a real estate property, the desired property is selected by selecting the property while looking at the floor plan and viewing the selected property. In addition, model rooms are exhibited in order to increase the likability of the target audience such as purchasers and renters.

In recent years, there has been a demand for a technique for guiding a property by displaying an image or a floor plan of a desired property on a computer via a communication network such as the Internet. It is said to be a remote guidance technology used for so-called real estate referral services.

In the remote guidance technology, a 3D model technology that creates a 3D model that reproduces the structure of the property and displays the created 3D model on the net is used. In the 3D model technology, the entire structure of the building is reproduced three-dimensionally, and the display of the image is changed as if the user is inside the actual building and is moving inside the building.

In the above remote guidance technology, the user views the contents of the property on the Internet by himself. However, it is also desired to view the contents of the property on the Internet while having immediate communication not only with one person but also with a plurality of users who are far apart, or with a plurality of users and a real estate agent.

The present invention has been made to solve the above-mentioned problems, and it is possible to view the same image in a state where a plurality of remote users are in the same place in the same space, and an image that enhances the sense of presence to the user. A display method, an image display system, and an image display program are provided.

A first aspect of the present invention is an image display method for displaying an image representing an object on an image display device, in which a plurality of image display devices access a server via a communication network, and a server. However, when a step of transmitting a 3D model that reproduces an object to a plurality of image display devices and one of the plurality of image display devices detects a trigger operation, a trigger signal is transmitted to the server. A step, a step in which the server transmits an image content containing an image and an observation angle included in the trigger signal to a plurality of image display devices based on the trigger signal, and each image display device causes a trigger signal. This includes a step of synthesizing an image included in the image content into a 3D model and displaying the combined image with the 3D model.

According to this aspect, a plurality of remote users can view the same image in the space as if they were in the same place in the same space. Further, since the position of the image on the object can be grasped by the 3D model, each image display device can easily express the depth and the three-dimensional effect of the object. As a result, the user's sense of presence can be ensured.

A second aspect of the present invention is that in the image display method of the first aspect, when a trigger signal is received from one image display device, the server rejects the trigger signal from the other image display device within a predetermined time. Further prepare for the steps to be taken.

According to this aspect, when a trigger signal from one image display device is received for a plurality of image display devices connected communicably, the server receives a time from another image display device within a predetermined time. The later trigger signal is rejected. As a result, there is no confusion due to different instructions from each user, so that the user's sense of presence can be enhanced.

A third aspect of the present invention is that in the image display method of the first aspect or the second aspect, when any one of the plurality of image display devices makes a call application to the other image display device, If another image display device accepts the call request, the image display device that has applied for the call further comprises a step of being able to make a call with the image display device that has accepted the call request.

According to this aspect, the user of each image display device connected communicably can make a call. In particular, when there is an intermediary who provides a real estate information service among the users, communication between the intermediary and the user becomes easy.

A fourth aspect of the present invention is the image display method according to any one of the first to third aspects, wherein the image includes a plan view and a panoramic image, and the plan view and the 3D model include a plurality of observation points. , The panoramic image has one or more other observation points different from its own observation point, each observation point is associated with the panorama image, and when a trigger operation to each observation point is detected, the server Includes the panoramic image associated with the triggered observation point in the image content and sends it.

In this embodiment, the plan view and the panoramic image are transmitted to the image display device and combined with the 3D model. According to this aspect, when the observation point in the displayed plan view or panoramic image is triggered to switch to another panoramic image and displayed, the plan view or panoramic image is combined with the 3D model. Instead of just switching the images to be displayed, you can jump into the space of the 3D model and feel as if you are standing at the corresponding observation point. Further, by synthesizing the 3D model and the image to be displayed, it becomes easy to express a sense of depth of space in the image to be displayed, and it is possible to further enhance the sense of presence. In addition, when switching between panoramic images, the sense of depth before and after the switching display changes, which has an impact and enhances the sense of presence.

In the fifth aspect of the present invention, when the trigger operation for the displayed image is detected in the image display method of the fourth aspect, if the position where the trigger operation is received is a place without an observation point, the image is displayed. The display device acquires the position information of the closest observation point at the position where the trigger operation is received and includes it in the trigger signal.

According to this aspect, even if a trigger operation is performed at an arbitrary point on the display image, it is detected as a trigger operation, which is easy to use.

According to the sixth aspect of the present invention, in the image display method of the fourth or fifth aspect, when the image of the first observation point is switched to the image of the second observation point among the plurality of observation points, the image is displayed after the switching. The image of the second observation point to be displayed is displayed at the same observation angle as the image of the first observation point displayed before switching.

According to this aspect, since the images before and after the switching display are switched at the same observation angle, the images can be displayed as if they were "straight" from the previous observation point to the later observation point. It has a high impact on the user and can enhance the sense of presence.

A seventh aspect of the present invention is an image display system for displaying an image representing an object on an image display device, which is connected to a plurality of image display devices and a plurality of image display devices via a communication network. , A 3D model that reproduces an object, and a server that transmits an image. The server includes a content storage unit that stores an image and a 3D model, and an information processing unit that transmits to the image display device in response to a trigger signal from the image display device, and each of the plurality of image display devices has an image. When the image display device accesses the server, the server sends a 3D model to the image display device, and the operation detection unit of any of the image display devices includes a display unit, a display control unit, and an operation detection unit. However, when the trigger operation is detected, the trigger signal is transmitted to the information processing unit, the information processing unit requests the image content including the image in the content storage unit based on the trigger signal, and the content storage unit receives information. The image content requested by the processing unit is transmitted to each display control unit, and each display control unit synthesizes the image included in the image content into the 3D model from the trigger signal, and the combined image with the 3D model. Is displayed on the display unit at the observation angle.

An eighth aspect of the present invention is the image display system of the seventh aspect, wherein each of the plurality of image display devices further includes an audio input unit and an audio output unit, and one of the image display devices is the other. When a call application is made to the image display device, when another image display device accepts the call application, the image display device that applied for the call and the image display device that accepted the call application are a voice input unit and a voice output unit. Allows you to make calls.

A ninth aspect of the present invention is an image display program for causing a computer to execute the image display method according to any one of the first to sixth aspects.

According to the present invention, a plurality of remote users can view the same image in a state where they are in the same place in the same space, and the user can feel the presence of the image. In addition, a plurality of remote users can view and guide the space while communicating with each other.

It is a schematic diagram which shows the structure of an image display system. It is a flowchart which shows the process of an image display method. It is a figure which displays the 3D model of a building. It is the figure which rotated the 3D model of FIG. 3 clockwise. It is a figure which displays the panoramic image at the observation point P1. It is a figure which displays the panoramic image at the observation point P2. It is a figure which displays the panoramic image at the observation point P2. It is a figure which displays the plan view of a building. It is a flowchart which shows the process of an image display method.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a schematic diagram showing a configuration of an image display system according to an embodiment. The image display system of the present embodiment includes a server 1 and a plurality of image display devices 2a and 2b. The number of image display devices is not limited to two, and may be more than two.

The server 1 is connected to the image display devices 2a and 2b so as to be able to transfer data via the communication network 100. The server 1 includes an information processing unit 11 and a content storage unit 12. The information processing unit 11 includes a processor such as a CPU and a memory such as a RAM and a ROM. The information processing unit 11 requests the image content stored in the content storage unit 12 in response to the signal from the image display device. The content storage unit 12 stores image content required for displaying an image, and transmits the image content to the image display device in response to a request from the information processing unit 11.

Examples of the image content include a two-dimensional image, a panoramic image, a 3D model, audio data such as a sound effect output together with the image and the 3D model, an introduction text, a moving image, and the like.

A two-dimensional image is a flat image such as a photograph. The panoramic image is an image in which the situation within the range of 360 degrees to the front and back, left and right, and 180 degrees up and down of the object can be confirmed. When taking a panoramic image, a group of photographs taken separately at predetermined angles are taken around the camera, and photographs with adjacent and overlapping areas are seamlessly connected by CG software, etc., and they are spherical. Combine to panoramic image. The panoramic image includes not only a photograph taken by a camera but also a panoramic image created by using an image generated by CG technology. When a panoramic image representing a three-dimensional element is displayed on a display unit (screen, etc.) of a two-dimensional computer, only a part of the panoramic image is displayed depending on the display angle (observation angle) to be displayed.

The 3D model of the present embodiment is a proportional reproduction of elements in a space such as a building which is an object. The elements in the space of the building are the whole structure of the target building, the internal structure, the whole space, each child space partitioned from the whole space by walls, the relationship between each child space and the whole space, the depth of the space, It is a physical entity that can be visually confirmed, such as dimensions and objects built inside the space. The 3D model is composed of computer graphics (abbreviation: CG) by arranging a coordinate system for ordinary 3D technology. A 3D model is data that can be read and displayed by a computer. As an example of the 3D model, data composed of a large number of polygons such as OBJ file data and MTL file data can be mentioned. OBJ file data is one of the 3D model formats, and includes an ".obj" file that stores information such as vertices and normals, and a ".mtl" file that stores model color and texture information. It is composed of. Even if there is no ".mtl" file, it is possible to draw the shape of the 3D model as long as there is an ".obj" file.

A polygon is an element used in CG technology to reproduce an object by combining polygons such as triangles and quadrangles. By reducing the number of polygons, the amount of data becomes smaller and the resolution becomes lower.

When the object building has each room like an apartment, the 3D model may represent each room and the overall structure of each dwelling unit composed of each room. .. Figure 3 shows a 3D model of a two-story detached house. In the 3D model of FIG. 3, since there is no outer wall and ceiling, by displaying the outer wall and ceiling as if they are transparent, each room partitioned by the wall, the entire structure of the house composed of each room, and the inside are inside. You can confirm the existence of all furniture, home appliances, etc.

The image display devices 2a and 2b are computers that can communicate with the server 1 via the communication network 100 as terminals for users and can display images. The image display devices 2a and 2b are, for example, mobile terminals such as smartphones and tablets, or personal computers.

The image display device 2a includes a display control unit 21a, a display unit 22a, an operation detection unit 23a, a voice input unit 24a, and a voice output unit 25a. The image display device 2b includes a display control unit 21b, a display unit 22b, an operation detection unit 23b, a voice input unit 24b, and a voice output unit 25b. The display control units 21a and 21b include a processor such as a CPU and a memory such as a RAM and a ROM. The display control units 21a and 21b read the image content transmitted from the content storage unit 12 by the trigger signal, synthesize the display data, and display the image to be displayed on the display units 22a and 22b.

The display units 22a and 22b are two-dimensional display screens such as screens, and can display an image including a two-dimensional image and a three-dimensional panoramic image. Further, the display unit may be an input means such as a touch panel.

The operation detection units 23a and 23b are means for receiving inputs from input means such as a touch panel, a keyboard, a mouse, and a microphone. When the operation detection units 23a and 23b receive the trigger operation, a trigger signal indicating that the trigger operation has been received is transmitted to the information processing unit 11, and the information processing unit 11 determines the image content for display based on the trigger signal. Then, the content storage unit 12 of the server 1 is requested to display the image content to be displayed based on the determination result.

The trigger operation is not limited to clicking, but may be input such as tapping, sliding, dragging (drag-and-drop) with a mouse or a touch panel, preset recognizable gesture operation, voice operation, or the like.

The operation detection units 23a and 23b read the meaning of the trigger operation by the position and the icon of the trigger operation, and include the position information, the observation angle information, the image data information and the like in the trigger signal.

The image data information is image information such as a two-dimensional image, a panoramic image, and a plan view. The position information is information indicating each position in the 3D model by the coordinate system of the 3D model itself. Each position in the 3D model corresponds to each position in the physical space of the building. The observation angle is an angle (angle displayed on the display unit) when the user observes an object. The direction indicating 0 degrees of the observation angle and the origin of the coordinate system of the 3D model itself are determined in advance when constructing the 3D model.

The voice input units 24a and 24b are, for example, microphones and are used for voice input. The audio output units 25a and 25b are, for example, earphones and speakers, and are used for audio output. By interlocking the voice input unit 24 with the voice output unit 25, an immediate conversation function like a telephone is exhibited between a plurality of image display devices while checking an image.

A part or all of the processing executed by the display control units 21a and 21b may be executed by the server 1.
[Image display method]

Hereinafter, the image display method of the present embodiment will be described in detail by taking the internal space of a building such as a condominium or a detached house as an example.

FIG. 2 is a flowchart showing a process when one image display device 2a is accessing the server 1.

At the start of display, the image display device 2a accesses the server 1. The server 1 assigns setting information such as a user ID for identification to the image display device 2a. Then, the OBJ file data of the 3D model is transmitted from the server 1 to the image display device 2a. The OBJ file data of the 3D model is downloaded to the image display device 2a (step S0). The display unit 22a of the image display device 2a displays the 3D model and is in the initial display state (step S1). When the operation detection unit 23a detects a trigger operation on the display unit 22a, the trigger signal is transmitted to the information processing unit 11 (step S2). The information processing unit 11 determines the image content to be displayed on the display screen based on the image data information, position information, observation angle information, etc. included in the trigger signal, and requests the content storage unit 12 (step S3). ). The content storage unit 12 transmits the image content requested by the information processing unit 11 to the display control unit 21a of the image display device 2a (step S4). The display control unit 21a reads and analyzes the image content, synthesizes the display image included in the image content with the 3D model, and displays the display image on the display unit 22a (step S5).

The download process of step S0 may be completed only once. The image display device 2a may repeatedly execute the processes from steps S1 to S5 each time the image is switched and displayed.

Next, with reference to FIG. 9, a process when a plurality of image display devices (two image display devices 2a and 2b as an example) access the server 1 will be described.

With IM technology (Instant Messenger), when one image display device 2a is already connected to the server 1 and viewing the image of the property, another image display device 2b can be invited and linked with the image display device 2b. .. When the image display device 2a cooperates with the image display device 2b, the image display device 2a and the image display device 2b can view the same property. Further, the image display device 2a can also accept an invitation to view the same property from another image display device 2b and cooperate with the image display device 2a. Upon receiving the invitation from the image display device 2b, the image display device 2a can select whether or not to cooperate with the image display device 2b.

When the image display device 2a permits cooperation with the image display device 2b, the server 1 assigns a user ID for identification to the image display device 2b, and the server 1 transmits a 3D model to the image display device 2b. The 3D model is stored in the image display device 2b. Further, on the server 1, the image display devices 2a and 2b are set as a linked group (step S10).

When the image display device 2b is linked with the image display device 2a, the server 1 transmits the image content previously transmitted to the image display device 2a to the image display device 2b linked with the image display device 2a. Then, the image display device 2b executes the process shown in FIG. As a result, the linked image display devices 2a and 2b display the same image (step S11). In this state, when any of the operation detection units 23a and 23b detects a trigger operation on the display unit 22, a trigger signal is transmitted to the information processing unit 11 (step S12). When the information processing unit 11 receives the trigger signal, it identifies the user ID of the image display device 2a that is the source of the trigger signal, and is based on the image data information, position information, observation angle information, and the like included in the trigger signal. Then, the image content to be displayed is determined, and the image content is requested to the content storage unit 12 (step S13). Further, when the information processing unit 11 receives a trigger signal from a different image display device in a timely manner, it is received from another user ID in the same group within a predetermined time after receiving the previous trigger signal. Reject the trigger signal (step S14).

The content storage unit 12 transmits the image content to the display control units 21a and 21b of all the image display devices 2a and 2b of the group (step S15). The display control units 21a and 21b that have received the image content analyze the image content, synthesize the image to be displayed into the 3D model, and display the image combined with the 3D model on the display units 22a and 22b. (Step S16).

If the trigger signal from another image display device is rejected within a predetermined time in step 14, a rejection message may be transmitted to the rejected image display device.

The download process in step S10 may be completed only once. Each of the image display devices 2a and 2b may repeatedly execute the processes from steps S11 to S16 each time the image is switched and displayed.

In the present embodiment, the display image in the initial display state is not limited. When the display units 22a and 22b are in the initial display state, the displayed image may be an image relating to the target space. For example, a part of a panoramic image such as a photograph in the property, a floor plan, a 3D model, or the like may be used.

Next, a display method will be described by taking as an example the switching from the display of the 3D model to the plan view in the plurality of linked image display devices 2a and 2b. The plan view is a projection view or a horizontal cross-sectional view from the upper surface of the object in order to express the structure, arrangement, etc. of the object, and in the case of a property, it is also called a floor plan.

When the linkage is permitted, the server transmits the image content transmitted immediately before to the image display device 2a that was accessed earlier to the image display device 2b that was accessed later, so that the plurality of linked image display devices are linked. Is always in the same display state.

FIG. 3 displays a 3D model of the object. The image display devices 2a and 2b have accessed the server and downloaded the 3D model (step S10). As shown in FIG. 3, the plurality of linked image display devices 2a and 2b are in the same initial display state and display the 3D model from a predetermined first observation angle (for example, toward the south). Further, an icon A2 such as a plan view is installed in a part of each display unit 22a and 22b (step S11). For example, when the icon A2 in the plan view is clicked on the image display device 2a, the operation detection unit 23a of the image display device 2a transmits a trigger signal for detecting the trigger operation to the icon A2 in the plan view to the information processing unit 11 ( Step S12). Based on the trigger signal, the information processing unit 11 determines that the image to be displayed is a plan view, and requests the content storage unit 12 for the plan view (step S13). Further, when the information processing unit 11 receives the trigger signal, the information processing unit 11 identifies the user ID of the image display device 2a that is the source of the trigger signal. When the information processing unit 11 receives a trigger signal from another user ID in the same group within a predetermined time, the information processing unit 11 rejects the trigger signal and transmits a rejection message to the corresponding other image display device (step S14). .. The content storage unit 12 transmits the image content including the plan view to the display control units 21a and 21b of the image display devices 2a and 2b, respectively (step S15). Each of the display control units 21a and 21b of the image display devices 2a and 2b causes the 3D model to synthesize the plan view included in the image content, respectively. As a result, a composite 3D model image of the plan view and the 3D model is generated. Each display control unit 21a and 21b causes each display unit 22a and 22b to display a plan view synthesized with the 3D model (step S16). Then, the display units 22a and 22b switch from the display of the 3D model of FIG. 3 to the display of the plan view as shown in FIG.

The 3D model of FIGS. 3 and 4, the panoramic image of FIGS. 5 to 7, and the plan view of FIG. 8 include a plurality of observation points P. The observation point is a place selected in advance as a viewing point inside the object, and each observation point has position information expressing the position of each observation point. Each observation point is associated with a panoramic image from each observation point by position information and a corresponding position in a 3D model or a plan view. The panoramic image associated with the observation point is an image in the object that can be viewed while standing at the observation point. From the panoramic image, the situation within the range of 360 degrees in front, back, left and right and 180 degrees up and down can be confirmed. Each panoramic image has the position information of the corresponding observation point. When the trigger operation to each observation point is detected, the 3D model is switched to the panoramic image so as to be inside the 3D model. Then, the display units 22a and 22b display the panoramic image at each observation point as if they were looking at the state of the object in the space at each observation point.

Next, it will be described that the 3D model is switched to the panoramic image of the observation point and displayed in all the linked image display devices 2a and 2b.

The image display devices 2a and 2b have already downloaded the 3D model (step S10), and display the 3D model as shown in FIG. A plurality of observation points are installed in the 3D model displayed in FIG. 4, and a trigger operation to the 3D model is possible (step S11). When any of the operation detection units of the image display devices 2a and 2b detects a trigger operation on the 3D model displayed on its own display unit, the trigger signal is transmitted to the information processing unit 11 (step S12). .. For example, when the observation point P1 (first observation point) of the 3D model shown in FIG. 4 is clicked on the display unit 22a of the image display device 2a, the operation detection unit 23a displays the position information of the observation point P1 and the current display. A trigger signal including observation angle information (described later) indicating an observation angle (hereinafter referred to as “second observation angle”) of the 3D model is transmitted to the information processing unit 11 (step S12). The information processing unit 11 determines that the image to be displayed is a panoramic image of the observation point P1 based on the position information of the observation point P1 included in the trigger signal, and has a panoramic image having the position information of the observation point P1. Is requested from the content storage unit 12 (step S13). When the information processing unit 11 receives the trigger signal, the information processing unit 11 identifies the user ID of the image display device 2a that is the source of the trigger signal, and within a predetermined time, receives a trigger signal from another user ID in the same group. Reject (step S14). The content storage unit 12 transmits the image content including the panoramic image of the observation point P1 to all the linked display control units 21a and 21b (step S15). The display control units 21a and 21b of all the image display devices 2a and 2b read the transmitted image contents, and based on the position information of the observation point P1, the panoramic image of the observation point P1 is displayed on the 3D model and the observation point P1. Synthesize. As a result, a synthetic 3D model image of the observation point P1 is generated. Each display control unit 21a, 21b causes each display unit 22a, 22b to display a panoramic image of the observation point P1 synthesized with the 3D model at the second observation angle by the trigger signal (step S16). Then, all of the linked image display devices 2a and 2b switch from the display state of the 3D model of FIG. 4 to the panoramic image of the observation point P1 (FIG. 5).

When the observation point P1 is clicked, the operation detection unit 23a reads the position information of the observation point P1 and includes it in the trigger signal. When a trigger operation to a position other than the previously installed observation point is detected, the operation detection unit 23a determines the observation point closest to the position where the trigger operation was performed, and the nearest observation point (the nearest observation point ( Hereinafter, the position information of the "closest observation point") is read and included in the trigger signal.

In the present embodiment, the operation detection unit 23a reads the display angle of the 3D model before the switching display (the second observation angle of the 3D model in FIG. 4) as the observation angle information and includes it in the trigger signal. Then, the display control units 21a and 21b display the panoramic image of the observation point P1 synthesized at the observation point P1 with the 3D model at the second observation angle. Therefore, the observation angle of the displayed image is the same before and after the switching display between the 3D model and the panoramic image.

Next, it will be described that the panoramic images of different observation points are switched and displayed in all the linked image display devices 2a and 2b.

As shown in FIG. 5, each image display device 2a and 2b displays a panoramic image of the observation point P1 at the second observation angle. A plurality of different observation points (P2, P3, P4, P5, ...) Are also displayed in the panoramic image of the observation point P1 (first observation point) (step 11).

In the state of FIG. 5, for example, when a trigger operation to the observation point P2 (second observation point) is detected in the image display device 2b, the operation detection unit 23b displays the position information of the observation point P2 and the screen currently displayed. A trigger signal including the second observation angle of the above is transmitted to the information processing unit 11 (step S12). The information processing unit 11 requests the content storage unit 12 for a panoramic image having the position information of the observation point P2 (step S13). When the information processing unit 11 receives the trigger signal, it identifies the user ID of the image display device 2b that is the source of the trigger signal, and rejects the trigger signal from another user ID in the same group within a predetermined time. (Step S14). The content storage unit 12 transmits the image content including the panoramic image of the observation point P2 to all the display control units 21a and 21b (step S15). The display control units 21a and 21b of all the image display devices 2a and 2b read the transmitted image contents, and based on the position information of the observation point P2, generate a panoramic image of the observation point P2 with a 3D model and the observation point P2. Synthesize. As a result, the composite 3D model image of the observation point P2 is generated in each of the image display devices 2a and 2b, respectively. As shown in FIG. 6, the display control units 21a and 21b display the panoramic image of the observation point P2 synthesized in the 3D model by the trigger signal on each of the display units 22a and 22b at the second observation angle. (Step S16). As a result, in all the image display devices 2a and 2b, the panoramic image of the observation point P1 is switched to the panoramic image of the observation point P2. Since the observation angles before and after the switching display are the same, it is possible to give each user a sense of going straight from the first observation point P1 to the second observation point P2. As a result, the user can be given the feeling of moving in the space, and the user's sense of presence can be enhanced.

As shown in FIG. 5, when an arbitrary part P0 (a part other than a preset observation point) of the displayed image is clicked to perform a trigger operation, in step S2, the display control unit 21b performs this trigger. The position information of the second observation point P2 closest to the place where the operation occurred P0 is acquired and included in the trigger signal.

As shown in FIGS. 5 and 6, a simplified view K of the plan view of the building is displayed in a part of the displayed image. The simplified diagram has a simulated movement display C. The simulated movement display C shows the position and observation angle of the observation point of the currently displayed panoramic image in the building. That is, the simulated movement display C indicates the position of the user in the building and the direction of observation. For example, when the display image is switched from FIG. 5 to FIG. 6, the simulated movement display C moves from the observation points P1 to P2.

When the operation detection unit 23b detects a trigger signal requesting switching display to the plan view by operating the icon A2 of the plan view or the simplified view K while the panoramic image of FIG. 6 is displayed, FIG. 6 shows. The panoramic image shown is switched to the plan view shown in FIG. 8 for display. The plan view is also provided with a plurality of observation points associated with the panoramic image. Therefore, it is possible to switch from the plan view to the panoramic image by clicking an arbitrary place on the plan view.

Next, a case where the observation angle of the panoramic image at the same observation point is changed in all the linked image display devices 2a and 2b will be described.

6 and 7 are panoramic images of the same observation point P2, but the observation angles are different. By dragging the display image of FIG. 6, the observation angle to be displayed can be arbitrarily changed. Therefore, it is possible to switch from the display image of FIG. 6 to the display image of FIG. 7. It is also possible to switch from the display image of FIG. 7 to the display image of FIG. 6 by dragging the display image of FIG. 7. Although it is a panoramic image of the same observation point P2, different parts of the panoramic image are displayed by changing the observation angle.

For example, as shown in FIG. 6, it is assumed that the initial display state of each display unit 22a and 22b is the display of the panoramic image of the observation point P2 at the second observation angle (step S11). When the operation detection unit 23a detects a drag operation as a trigger operation to the display unit 22a, the operation detection unit 23a analyzes the drag operation and determines whether or not the first observation angle is requested. When the operation detection unit 23a determines that the first observation angle is required, the operation detection unit 23a transmits a trigger signal including information for changing to the first observation angle to the panoramic image of the observation point P2 to the information processing unit 11. (Step S12). The information processing unit 11 requests the content storage unit 12 for a panoramic image of the observation point P2 based on this trigger signal (step S13). When the information processing unit 11 receives the trigger signal, it identifies the user ID of the image display device 2a that is the source of the trigger signal, and rejects the trigger signal from another user ID in the same group within a predetermined time. (Step S14). The content storage unit 12 transmits the panoramic image of the observation point P2 and the image content including the first observation angle to the display control units 21a and 21b (step S15). The display control units 21a and 21b of all the image display devices 2a and 2b read the transmitted image contents, and based on the position information of the observation point P2, generate a panoramic image of the observation point P2 with a 3D model and the observation point P2. Synthesize. Then, each image display device 2a and 2b generates a composite 3D model image of the observation point P2. The display control units 21a and 21b display the panoramic image of the observation point P2 synthesized in the 3D model on the display units 22a and 22b at the first observation angle, respectively (step S16). Then, the image display devices 2a and 2b in the group change the display at the second observation angle (FIG. 6) to the display at the first observation angle (FIG. 7) for the panoramic image at the same observation point.

In order to reduce the amount of data to be transmitted, when displaying a panoramic image of a certain observation point (for example, P2), if the position information does not change, a trigger signal containing only the information of the observation angle to be changed is transmitted. It is also possible to transmit to the information processing unit 11. Then, each display control unit 21 displays the panoramic image at the changed observation angle according to the data transmitted from the server.

A process when any of the linked image display devices 2a and 2b tries to make a call to the other image display device 2b will be described.

With IM technology, it is possible to make a call while viewing an image. In the present embodiment, the image display device 2a may apply for a call to another image display device 2b by means of a call icon or the like while viewing an image. When the other image display device 2b accepts the call application, the voice input units 24a and 24b and the voice output units 25a and 25b are activated so that the image display devices 2a and 2b can make a call.

In this embodiment, the 3D model can be rotated by the user's operation. For example, when the image display device 2a detects a trigger operation on the 3D icon A1 shown in FIGS. 5 to 8, the operation detection unit 23a reads it as an instruction to display the 3D model and includes it in the trigger signal. Send the trigger signal to server 1. The server 1 transmits the trigger signal to the display control units 21a and 21b of the image display devices 2a and 2b. The display control units 21a and 21b display the stored 3D model on the display units 22a and 22b, respectively. While the 3D model is being displayed, if you drag a place other than the 3D model on the display screen to move the pointer, the 3D model will rotate and the 3D model will be displayed at each observation angle. As a result, as shown in FIG. 4, the 3D model is displayed at a second observation angle different from that in FIG.

In the present embodiment, synthesizing the image and the 3D model means associating the 3D model with the image corresponding to each part of the 3D model in correspondence with the position of each image in the 3D model. For this synthesis work, it is possible to use a conventional technique such as WebGL. When the image combined with the 3D model is displayed on a two-dimensional display unit 22 such as a screen, it looks like a two-dimensional image, but since the position of the image on the object can be grasped by the 3D model, the object is displayed. It becomes easier to express depth, dimensions, and three-dimensional effect.

Next, a method of generating image data in the present embodiment will be described.

First, using CG technology, a 3D model that reproduces the structure of the building as an object is created. The 3D model is mainly an OBJ file. Then, in order to enhance the sense of reality, the 3D model is rendered. Rendering reproduces the actual colors, patterns, textures, etc. of objects in the building on a 3D model. Then, the rendered 3D model is saved. Since the 3D model after rendering has a high resolution, the amount of data of the entire 3D model is large.

A panoramic image at a plurality of observation points is picked up from the rendered 3D model and saved as a panoramic image at each observation point. The saved panoramic image has the position information of its own observation point. By reducing the number of polygons for the rendered 3D model, the rendered 3D model is compressed. The compressed rendered 3D model is stored in the content storage unit 12 of the server 1.

Then, the panoramic image of each picked-up observation point is associated with each observation point in the compressed 3D model by the position information of each observation point. In addition, one or more other observation points are set in each panoramic image, and the panoramic image is associated with each observation point.

The compressed rendered 3D model thus generated is displayed on the image display devices 2a and 2b as the above-mentioned 3D models of FIGS. 3 and 4. The compressed 3D model has a lower resolution than the rendered 3D model. The resolution of the compressed 3D model is lower than each panoramic image picked up from the rendered 3D model. As a result, the amount of communication data by the compressed 3D model is significantly reduced, and no time delay occurs even in a communication terminal such as a mobile phone. The panoramic image is picked up from a high resolution uncompressed, post-rendered 3D model. Therefore, it is possible to secure the user to display a panoramic image with high resolution. Further, the image display device accesses the server and downloads the compressed 3D model after rendering.

In the above embodiment, a high-resolution panoramic image is picked up from the rendered 3D model, but the step of picking up is omitted, and the panoramic photograph created by taking a picture with a camera is compressed into a 3D model. It is also possible to associate it with each observation point of. Then, after creating the 3D model, it may be compressed immediately and the panoramic image prepared in advance may be associated with each observation point of the compressed 3D model. The task of picking up a high-resolution panoramic image from a rendered 3D model is to generate a panoramic image using conventional CG technology.

In the above embodiment, the space of the building is used as the target space, and the detached house is mainly used as an example, but the target space is not limited to this. Any space that can be reproduced with a 3D model, such as an office tower, a museum, a department store sales space, a car interior space, a factory, a natural cave, a park, or other artificial or naturally formed space.

In the above embodiment, in order to enhance the presence of the user, the observation angle of the image displayed on the display unit is the same before and after the switching display between the 3D model and the panoramic image, but the observation after switching is performed. The angle may be set arbitrarily.

Although the present invention has been described above using the embodiments, it goes without saying that the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various changes or improvements can be made to the above embodiments. It is also clear from the description of the claims that the form with such changes or improvements may be included in the technical scope of the present invention.

1 Server 2 Image display device 11 Information processing unit 12 Content storage unit 21 Display control unit 22 Display unit 23 Operation detection unit 24 Audio input unit 25 Audio output unit

Claims (9)

It is an image display method for displaying an image expressing an object on an image display device.
A step in which a plurality of the image display devices access a server via a communication network,
A step in which the server transmits a 3D model that reproduces the object to the plurality of image display devices.
A step of transmitting a trigger signal to the server when any one of the image display devices detects a trigger operation among the plurality of image display devices.
A step in which the server transmits the image content including the image and the observation angle included in the trigger signal to the plurality of image display devices based on the trigger signal.
Each of the image display devices synthesizes the image included in the image content into the 3D model by the trigger signal, and displays the image combined with the 3D model.
An image display method. Upon receiving the trigger signal from one image display device, the server further comprises a step of rejecting the trigger signal from the other image display device within a predetermined time.
The image display method according to claim 1. When any one of the plurality of image display devices makes a call application to the other image display device and the other image display device accepts the call application, the call application is made. The image display device further comprises a step of being able to make a call with the image display device that has accepted the call request.
The image display method according to claim 1 or 2. The image includes a plan view and a panoramic image.
The plan view and the 3D model include a plurality of observation points.
The panoramic image has one or more other observation points different from its own observation point.
Each of the observation points is associated with the panoramic image, respectively.
When a trigger operation to each of the observation points is detected, the server includes the panoramic image associated with the triggered observation point in the image content and transmits the panoramic image.
The image display method according to any one of claims 1 to 3. When the trigger operation for the displayed image is detected, if the position where the trigger operation is received is not the observation point, the image display device is closest to the position where the trigger operation is received. The position information of the observation point is acquired and included in the trigger signal.
The image display method according to claim 4. When the image of the first observation point is switched to the image of the second observation point among the plurality of observation points and displayed, the image of the second observation point displayed after the switching is the first observation displayed before the switching. Displayed at the same observation angle as the point image,
The image display method according to claim 4 or 5. An image display system for displaying an image representing an object on an image display device.
With the plurality of the image display devices,
A 3D model that reproduces the object and a server that transmits the image to the plurality of image display devices via a communication network.
Equipped with
The server includes a content storage unit that stores the image and the 3D model, and an information processing unit that transmits to the image display device in response to a trigger signal from the image display device.
Each of the plurality of image display devices includes a display unit for displaying the image, a display control unit, and an operation detection unit.
When the image display device accesses the server, the server transmits the 3D model to the image display device.
When the operation detection unit of any of the image display devices detects a trigger operation, the trigger signal is transmitted to the information processing unit.
Based on the trigger signal, the information processing unit requests the image content including the image in the content storage unit.
The content storage unit transmits the image content requested by the information processing unit to each display control unit, respectively.
From the trigger signal, each display control unit causes the 3D model to synthesize the image included in the image content, and causes the display unit to display the image combined with the 3D model at the observation angle. ,
Image display system. Each of the plurality of image display devices further includes an audio input unit and an audio output unit.
When any of the image display devices makes a call application to the other image display device, and the other image display device accepts the call application, the image display device that applied for the call and the call application are made. The accepted image display device can be talked to by the voice input unit and the voice output unit.
The image display system according to claim 7. An image display program for causing a computer to execute the image display method according to any one of claims 1 to 6.

Images