JP5682060B2 - Image composition apparatus, image composition program, and image composition system - Google Patents

Description

  The present invention is an apparatus, a program, and a system for displaying a virtual image superimposed on a real space image obtained by imaging a real space. More specifically, an image composition device that superimposes and displays a virtual space image created based on a space model on a real space image, and also displays the virtual space image according to the perspective of the real space image. , An image composition program and an image composition system.

  Many underground objects exist on roads due to their network and public characteristics. As underground buried objects under the road, water and sewage and gas are typical, and in recent years, undergrounding of electric wires has been actively performed, and an optical fiber cable called information BOX is also widely buried. FIG. 8 is an explanatory view schematically showing the state of burying under the road, in which the optical fiber cable A is buried on the left side of the road and the sewage pipe B is buried on the right side of the road. In this figure, the planar arrangement of the optical fiber cable A and the sewer pipe B is shown by a broken line, and the sectional arrangement is shown by a virtual sectional view, but of course, it cannot be actually seen in this way.

  FIG. 10 is an explanatory diagram showing a situation where a field survey is being conducted on a road. As shown in this figure, it is not possible to visually confirm the underground objects in daily life. For this reason, it is not easy to ascertain locally what kind of underground objects are present at which position. When excavating roads such as road improvement work and new buried object installation work, it is necessary to confirm the position and type of underground buried objects in advance, but it cannot be grasped by visual inspection at the site. You must collect and examine drawings. Based on the location information of the underground objects found as a result of the survey (usually information given in the plane coordinate system), surveys will be conducted on site, and will be clearly indicated by markers such as ridges or white lines. FIG. 10 is an explanatory diagram showing the planar arrangement of the underground object on the road with a white line C. FIG.

  As with underground buried objects under the road, wiring and piping contained in the wall of the building cannot be observed on site, so field surveys are conducted with reference to design drawings, etc., and markers are shown on the walls. It will be.

  Unlike these buried objects, there are some that cannot be seen locally because they do not exist in reality. For example, what was created based on the concept of people such as land boundaries, villages, towns and villages, unstructured roads and condominiums, structures at the planning stage, dangerous areas such as landslide warning areas and landslide designated areas (hereinafter referred to as "Conceptual object"). For such a conceptual object, there are not many situations that require confirmation on site. In this case, information indicating the position of the conceptual object (usually given in a plane coordinate system as well as the embedded object described above). Field survey based on the information, and clearly indicate with markers such as piles and white lines.

  In this way, in order to confirm on-site things that are not actually visible (hereinafter referred to as “invisible objects”), such as buried objects and conceptual objects, it is necessary to collect materials related to the invisible objects and to further survey them locally. is there. The labor required to confirm invisible objects on site is much larger than what can be visually observed (hereinafter referred to as “visible objects”).

  On the other hand, recently, a technology for experiencing a virtual space in a real space has been developed for practical use. This is a technique for superimposing and expressing digital information as a virtual image in a real space, and is called “Augmented Reality (AR)” in the sense of expanding the real world. Augmented reality starts with what is felt on a display (Head Mounted Display; HMD) that is worn on a person's head. Currently, what is displayed on a portable computer with a monitor such as a cellular phone is becoming mainstream.

  Hardware and software related to augmented reality are steadily advancing toward practical use, but there are currently not many proposals for how to apply this technology. In this sense, augmented reality has not been developed. It can be said that this technology is on the way. Proposals for technologies applying augmented reality are expected to increase in the future, and Patent Document 1 is one of them.

JP2010-238096

  Patent Document 1 continuously displays (live view) while photographing a real space with a terminal equipped with a camera, and displays virtual images according to the position of the camera and the direction of the camera (in this document, “augmented reality information”). Is acquired and superimposed on the captured image of the real space. Then, when the camera is moved, the virtual image is accurately displayed following the moving speed. A threshold is set for the moving amount of the camera, and the virtual image is compared with the threshold and the moving amount. It is determined whether or not to change.

  In Patent Document 1, as a method of measuring the position of the camera and the direction of the camera, various sensors are mounted on the camera-equipped terminal. In this way, it is becoming more common to use sensors to realize augmented reality, and technologies related to sensors such as technologies for improving measurement accuracy with sensors, technologies for shortening measurement time, and technologies for reducing the size of the sensor itself have greatly advanced. doing. This sensor can be broadly divided into those that measure the position (coordinates) of the camera and those that measure the posture of the camera. The former is representative of GPS (Global Positioning System), and the latter is exemplified by a geomagnetic sensor. it can.

  As mentioned above, when confirming “invisible objects” such as buried objects and conceptual objects that cannot actually be seen on the site, it is necessary to collect materials related to the invisible objects, and it is necessary to survey on the site. It required labor and time. In addition, in order to express invisible objects locally, they were displayed with markers such as ridges, stakes, or white lines, which was not preferable in terms of labor and aesthetics.

  In order to solve such a problem, it is also possible to use augmented reality as shown in Patent Document 1. That is, it is conceivable to take a picture with a camera-equipped terminal at the site and display a virtual image of an embedded object or a conceptual object on an actual image. However, the virtual image displayed in Patent Document 1 is a tag, and the user can perceive more than reality by the information shown on the tag. With such means, it is not possible to intuitively recognize where the buried objects exist locally.

  Furthermore, in the current augmented reality including Patent Document 1, there is a problem in positional accuracy when a virtual image is superimposed on an image in real space. Although measurement techniques using GPS and magnetic sensors have been improved, the accuracy is not yet sufficient, and a technique for correcting an erroneous display has not been established. In particular, when excavation is performed with a buried object under the road, if the display position accuracy of the virtual image (in this case, the underground object) displayed by augmented reality is not sufficient, an accident may occur that damages the underground object. There is a fear.

  The subject of the present invention is to solve the above-mentioned problems, and without inconvenience such as collection of materials and on-site surveying, it is possible to intuitively confirm invisible objects such as buried objects and conceptual objects at the site. is there. Another object of the present invention is to make the display position accuracy of the virtual image appropriate when superimposing the image on the real space. More specifically, it is an object of the present invention to provide an image composition device, an image composition program, and an image composition system that solve these problems.

  An image composition apparatus according to the present invention is an image composition apparatus that superimposes a virtual space image created based on a space model having space information on a real space image obtained by capturing a real space and displays the image on a display screen. Based on the image acquisition means for acquiring the real space image, the measurement means for measuring the position and orientation of the image acquisition means, the storage means for storing the space model, the measurement value by the measurement means and the space model The image creation means for creating the virtual space image so as to become a perspective view displayed in perspective on the display screen, the real space image displayed in perspective, and the virtual space image displayed in perspective And display means for superimposing and displaying on the display screen.

  In the image synthesizing apparatus according to the present invention, the spatial model includes spatial information of a visible object that is visible and spatial information of an invisible object that is not visible, and the image creating means includes the virtual space image of the visible object and the invisible object. And a display control unit that selects and displays or hides the virtual space image of the visible object and the virtual space image of the invisible object.

  The image composition device of the present invention displays the virtual space image and the real space image when the display positions of the virtual space image and the real space image for the same visible visible object do not match on the display screen. Display position adjusting means for moving the virtual space image so that the positions coincide or substantially coincide with each other on the display screen may be provided.

  In this case, the display position adjusting means automatically extracts the shape feature portion based on the image feature portion of the real space image, and automatically detects the matching portion to be matched with the shape feature portion from the virtual space image of the visible object. In addition, the display position of the virtual space image can be automatically adjusted so that the positions on the display image of the shape feature portion and the collation portion match or substantially match.

  In the image composition device according to the present invention, the display means displays a figure or / and attribute information relating to a virtual space image, and the display control means displays or hides the figure or / and attribute information. You can also

  The image composition program of the present invention is an image composition program that superimposes a virtual space image created based on a space model on a real space image obtained by capturing a real space and displays the image on a display screen. A model reading function for acquiring this spatial model and the position and orientation of the image acquisition means obtained by measurement are acquired from a storage means storing a spatial model including spatial information of visible objects and spatial information of invisible objects that cannot be viewed. Based on the measurement value reading function, the position and orientation of the image acquisition means, and the space model, the virtual space image of the visible object and the invisible object of the invisible object are displayed in a perspective view displayed on the display screen. An image creation function for creating a virtual space image, the real space image displayed in perspective, and the virtual space image displayed in perspective are superimposed on the display screen. A display function for displaying, a virtual space image for the visible object, and a display control function for selecting or displaying the virtual space image for the invisible object, the virtual space image for the same visible object, and the actual display. When the display position of the space image does not match on the display screen, the virtual space image is moved so that the display position of the virtual space image and the real space image match or substantially match on the display screen. The display position adjustment function is executed by a computer.

  In the image composition program of the present invention, the display position adjustment function includes a feature portion extraction function for extracting a shape feature portion based on an image feature portion of a real space image, and a matching portion for matching with the shape feature portion. A matching part detection function for detecting from a virtual space image of a visible object, and a position adjustment for adjusting the display position of the virtual space image so that the positions of the shape feature part and the matching part on the display image match or substantially match It is also possible to provide a function.

  In the image composition program of the present invention, the display function displays a figure or / and attribute information related to a virtual space image, and the display control function displays or hides the figure or / and attribute information. You can also

  An image composition system of the present invention is an image composition system that superimposes a virtual space image created based on a space model on a real space image obtained by capturing a real space, and displays the image on a display screen. The server includes one or two or more terminals connected to the server via a wireless or wired communication line, and the server includes space information of visible objects that are visible and space information of invisible objects that are not visible. Storage means for storing a model, image creation means for creating the virtual space image so as to become a perspective view displayed on the display screen of the terminal, and communication means, the terminal, Image acquisition means for acquiring a real space image, measurement means for measuring the position and orientation of the image acquisition means, display means for displaying the real space image and the virtual space image on a display screen, and the visible object Display control means for selecting and displaying a virtual space image and a virtual space image of the invisible object, and communication means, and when the terminal user requests the server, the terminal measurement The measurement value by the means is transmitted to a server, and the server, based on the request of the user of the terminal, based on the measurement value of the terminal and the space model, the virtual space image of the visible object and the invisible object And the virtual space image is transmitted to the terminal, and the display screen of the terminal that has received the virtual space image is displayed in perspective with the real space image displayed in perspective. When the display position of the virtual space image and the real space image for the same visible object are displayed in a superimposed manner and the display positions of the virtual space image do not match on the display screen, the display position included in the terminal The adjusting means such that said display position of the virtual space image the real space image matches or nearly matches on the display screen, it is capable to move the virtual space image.

  An image composition system of the present invention is an image composition system that superimposes a virtual space image created based on a space model on a real space image obtained by capturing a real space, and displays the image on a display screen. The server includes one or two or more terminals connected to the server via a wireless or wired communication line, and the server includes space information of visible objects that are visible and space information of invisible objects that are not visible. Storage means for storing a model, image creation means for creating the virtual space image so as to become a perspective view displayed on the display screen of the terminal, and communication means, the terminal, Image acquisition means for acquiring a real space image, measurement means for measuring the position and orientation of the image acquisition means, display means for displaying the real space image and the virtual space image on a display screen, and the visible object Display control means for selecting and displaying a virtual space image and a virtual space image of the invisible object, and communication means, and when the terminal user requests the server, the terminal measurement The measured value by the means and the real space image by the image acquisition means of the terminal are transmitted to the server, and the server determines the measured value of the terminal and the space model in response to a request of the user of the terminal. And the server creates a virtual space image of the visible object and a virtual space image of the invisible object, and the server displays the display position of the virtual space image and the real space image for the same visible object in the display. If they do not match on the screen, the shape feature is automatically extracted based on the image feature of the real space image, and the matching portion to be matched with the shape feature is automatically extracted from the virtual space image of the visible object. After detecting and automatically adjusting the display position of the virtual space image so that the position on the display image of the shape feature portion and the matching portion match or substantially match, the virtual space image of the visible object and the virtual space of the invisible object are In the display screen of the terminal that transmits the space image to the terminal and receives the virtual space image, the real space image displayed in perspective and the virtual space image displayed in perspective are superimposed and displayed. It can also be.

  In the image composition system of the present invention, the display means displays a figure or / and attribute information relating to a virtual space image, and the display control means displays or hides the figure or / and attribute information. You can also

The image composition device, the image composition program, and the image composition system of the present invention have the following effects.
(1) Invisible objects can be easily confirmed locally without the need for labor, such as data collection and field surveying.
(2) Moreover, since the marker for displaying an invisible object is not required locally, the local beauty can be maintained.
(3) Since the virtual space image is displayed with a perspective that matches the image of the real space image, it is possible to grasp an invisible object extremely intuitively.
(4) By matching the display position of the virtual space image and the real space image with the same visible object, the display position accuracy of the virtual space image of the invisible object is also improved. For example, even when excavating under the road, There is no damage.
(5) Displaying drawings and attribute information related to a virtual space image is preferable because it can confirm useful information such as the type and owner of an underground object in addition to the position where an invisible object exists.

Explanatory drawing which shows the condition which is image | photographing the road with a terminal device. (A) is a figure which shows the monitor side of a terminal, (b) is a figure which shows the back side (camera side) of a terminal. (A) is a figure which shows the virtual space image at the time of creating a site boundary as a line, (b) is a figure which shows the virtual space image at the time of creating the site boundary as a surface. The figure which shows the monitor with which the top view regarding the sewer pipe 3V was displayed. The figure which shows the monitor which superimposed and displayed the real space image and virtual space image (center line and the sewer pipe) which were image | photographed with the camera. Explanatory drawing for demonstrating the method of designating a formation element and moving the virtual space image on a monitor. Explanatory drawing for demonstrating the method to rotate and move a virtual space image. Explanatory drawing which showed the burial condition under the road typically. Explanatory diagram showing the situation of conducting field surveys on the road. Explanatory drawing which showed the planar arrangement | positioning of the underground burying object on the road with the white line.

(Embodiment)
One embodiment of an image composition device, an image composition program, and an image composition system of the present invention will be described below with reference to the drawings.

(Overview)
FIG. 1 is an explanatory view showing a situation where a road 1 is photographed with a terminal 1, FIG. 2 (a) is a diagram showing a monitor side of the terminal 1, and FIG. 1 (b) is a back side of the terminal 1 ( It is a figure which shows the camera side. As shown in FIG. 1, a sewage pipe 3 is buried under the road where the center line 2 is laid, and a user of the terminal 1 (hereinafter referred to as “user”) can see from the sidewalk. Shooting the previous sight. As shown in FIGS. 2A and 2B, the terminal 1 has a built-in camera 1 a (image acquisition means), and a monitor 1 b (display screen) is provided on the surface of the terminal 1. For convenience, the sewer pipe 3 is indicated by a broken line in FIG. 1, but the sewer pipe 3 is not visible to the user.

  As shown in FIG. 2 (a), the monitor 1b taken by the user has an image of what is present in the actual space (hereinafter referred to as "real thing"), that is, a roadway, a sidewalk, a utility pole, a building, etc. Hereinafter, an image of a real thing is referred to as “real space image”), and the center line 2 of the roadway is also displayed as a real space image (center line 2R). Of course, the actual thing is not limited to the centerline 2 of the roadway, the sidewalk, the power pole, the building, and the roadway exemplified here, but all the wires existing in the actual space such as the electric wires hung on the power pole and the reinforcing bars in the structure. It goes without saying that things are included. Also, an image of a virtual model (hereinafter referred to as “virtual space model”) created based on a space model described later (hereinafter, an image of the virtual space model is referred to as “virtual space image”). Is also displayed on the monitor 1b. In Fig.2 (a), the virtual space image (sewer pipe 3V) which imaged the virtual space model of the sewer pipe 3 is displayed. For the sake of convenience, the center line is represented by “centre line 2” and a symbol with only numerals when represented as a real object, and the “center line 2R” and symbol with R added when representing the center line as a real space image. As will be described later, when the center line is represented as a virtual space image, it is represented by a symbol with “center line 3V” and V appended thereto. Similarly, when the sewer pipe is expressed as an actual entity, it is expressed as “sewage pipe 3”, and when the sewer pipe is expressed as a virtual space image, it is expressed as “sewer pipe 3V”.

(Terminal)
The terminal 1 shown in FIG. 2 (a) will be described. The terminal 1 includes at least a camera 1a (image acquisition means), a sensor (measurement means) (not shown), a monitor 1b (display screen), and an electronic computer (hereinafter referred to as “computer”). The terminal 1 is preferably a small one that can be easily carried by a person, and a commercially available mobile phone or a portable computer with a built-in camera and sensor can be used. The created one can also be used.

  An image captured by the camera 1a of the terminal 1 is displayed on the monitor 1b, but a scene in a direction toward the camera 1a is always displayed, and is displayed in accordance with the movement of the camera 1a, so-called live view. Is displayed. Of course, it can also be acquired as a still image by operating the shutter.

  The sensors of the terminal 1 are roughly classified into two types: a position measurement sensor that measures position information of the camera 1a and an attitude measurement sensor that measures the attitude of the camera 1a.

  The position measuring sensor acquires the position coordinates of the camera 1a, and the position coordinates here mean three-dimensional coordinates (X, Y, Z) (of course, latitude, longitude, and altitude may be used). GPS is a representative position measurement sensor, but in addition, a positioning method using a wireless LAN access point (such as Place Engine), a positioning method using visible light communication that transmits LED blinking as a signal, indoors Positioning method to acquire position information from IMES (Indoor Messaging System), QR code, visual marker (marker used in ARTToolKit), RFID tag, etc., positioning using infrared communication For example, a known positioning technique can be used. Of course, various positioning methods can be used in combination, such as GPS for outdoor use and a wireless LAN system for indoor use.

  The posture measurement sensor acquires the posture of the camera 1a, and the posture here refers to the direction in which the camera 1a is facing and the tilt. More specifically, if the horizontal axis is the X axis, the horizontal axis is the Y axis, and the vertical axis is the Z axis, the rotation ω (pitch) around the X axis, the rotation φ (roll) around the Y axis, The rotation κ (yaw) around the Z axis is an element that determines the posture. Typical posture measurement sensors include a geomagnetic sensor (electronic compass) that measures rotation κ around the Z axis, that is, an orientation, and an acceleration sensor that measures rotation ω around the X axis and rotation φ around the Y axis. Other known techniques such as a gyro sensor can also be used. Since a mobile phone terminal incorporating a so-called 6-axis sensor and incorporating both an electronic compass and an acceleration sensor is also commercially available, it is preferable to use such a terminal.

(Virtual space image)
The virtual space image will be described. As described above, the virtual space image is a further image of a virtual space model created by processing the space model.

  A spatial model means a set of spatial information for expressing a feature with a specific position, a specific shape and size on the earth. The spatial information here is information for expressing the position of a feature such as coordinates and latitude / longitude, and two-dimensional coordinates (X, Y) and three-dimensional coordinates (X, Y, Z) are used. It can be illustrated. Further, the term “feature” as used herein means a level that can be expressed by spatial information, and is not limited to a real thing (real thing). For example, things created based on human concepts (concepts) such as land boundaries, villages, towns and villages, unstructured roads and condominiums, planned structures such as landslide disaster warning areas and landslide designated areas If a specific position and a specific shape / size can be expressed by spatial information, all are features.

  In addition, the actual objects can be divided into visible objects that can be visually observed like the center line 2 in FIG. 1 and invisible objects that are not actually visible such as the sewer pipe 3. Any of them is a feature because a specific position, a specific shape and size can be expressed by spatial information. That is, the spatial model is a set of spatial information related to features such as real objects (visible and invisible) and conceptual objects.

  As an example of the space model, a “numerical elevation model” in which the ground surface (feature) is modeled can be cited. The digital elevation model is a model composed of three-dimensional point cloud data, and DEM (Digital Elevation Model), DSM (Digital Surface Model), and the like are known. Of course, the spatial model is not limited to the one based on the three-dimensional point cloud data, but may be a plane mesh composed of the two-dimensional point cloud data, the two-point plane coordinates for representing the line segment, Also, it may be a set of spatial information such as the plane coordinates, direction, and length of one point that defines the line segment.

  The spatial model is stored as a database in a storage area of a computer (storage means). Necessary spatial information is acquired from the database of the spatial model, and predetermined calculation processing is performed to obtain a specific position, a specific shape, and a size of the feature. What is obtained here is a virtual space model. Note that as described above, features include real objects (visible objects and invisible objects), that is, real objects may be created as a virtual space model, but the term “virtual” means that it is obtained by calculation. Even if it is a real thing, what was created from the space model is called a virtual space model.

  An image to be displayed on the monitor 1b of the terminal 1, that is, a virtual space image is created from the virtual space model. If a virtual space image of a place taken by the camera 1a is displayed on the monitor 1b, information that cannot be obtained only by visual observation can be obtained at the same time, which is extremely useful. At this time, it is desirable that the virtual space image is displayed at a position that actually exists on the monitor 1b. For example, even though the sewage pipe 3 is actually buried on the left side of the road, if the sewage pipe 3V (virtual space image) is displayed on the right side of the roadway on the monitor 1b, it will be confused. .

  Further, as shown in FIG. 2A, the real space image displayed on the monitor 1b is displayed in an actually viewed state, that is, in a state expressing perspective (hereinafter, in a state expressing perspective). Therefore, if the virtual space image is not displayed in the perspective, there is a risk of misunderstanding. If only the sewage pipe 3V is displayed in parallel with the vertical frame of the monitor 1b in FIG. 2A, it is mistaken that the sewage pipe 3V is buried off the road. In the past, information related to actually photographed “things” has been displayed on the monitor 1b in the vicinity where the “thing” is displayed. Unlike the above information, when displaying the feature of “line” or “surface” in the real space image, not only the position but also the display method must be devised.

  Therefore, in the present invention, the virtual space image is also displayed on the monitor 1b in a state that would be displayed if it was actually taken. That is, in the case of the sewage pipe 3V (virtual space image) shown in FIG. 2A, the sewage pipe 3V is displayed at the plane position where it is actually embedded in the real space image displayed on the monitor 1b. At the same time, a perspective view is displayed like a real space image (for example, like a roadway or a sidewalk).

  At this time, since the virtual model has only spatial information (coordinates) in the real space, it is unclear at which position (which pixel) the monitor 1b should be displayed. Therefore, for example, an arbitrary coordinate system is set on the monitor 1b, and the real space coordinates of the virtual space model are converted into the arbitrary coordinate system of the monitor 1b. It is necessary to decide whether or not to express it.

  In addition, in order to display a virtual space image in perspective, it is necessary to grasp the perspective state in the real space image, and between the arbitrary coordinate system on the monitor 1b set in this perspective state and the coordinate system in the real space. It is necessary to obtain a geometric relationship. The perspective state in the real space image can generally be grasped by using the perspective method. This perspective is a technique that is used not only in paintings but also in various fields for displaying on a plane such as a design drawing or a camera. In recent years, it is also widely used in computer graphics (CG). A method of expressing a three-dimensional object existing in real space three-dimensionally (so that a three-dimensional effect can be obtained) on a two-dimensional plane is a perspective method. Called “Figure”. For example, the real space image of FIG. 2A is a perspective view displayed in perspective so that all lines converge toward one vanishing point.

  Since the feature expressed in the real space image is a real thing, it naturally has the coordinates of the real space, and by grasping how the real space coordinates of these real things are displayed on the monitor 1b, An arbitrary coordinate system on the monitor 1b can be determined. This can be optically determined from the real space coordinates of the camera 1a, the attitude of the camera 1a, and the specifications (view angle, focal length, etc.) of the camera 1a. The real space coordinates of the camera 1a can use three-dimensional coordinates (X, Y, Z) measured by the position measurement sensor, and the posture of the camera 1a can be values (ω, φ, κ) measured by the posture measurement sensor. ) Can be used.

  When an arbitrary coordinate system on the monitor 1b is determined based on real space coordinates of a real object represented by a real space image, the real space coordinates can be expressed by the arbitrary coordinate system, and similarly, a virtual space having spatial information. The model can also be expressed in an arbitrary coordinate system on the monitor 1b. Thus, the virtual space image is created by arranging the virtual space model in the arbitrary coordinate system on the monitor 1b and displaying it on the monitor 1b. The virtual space image is actually data to be processed by a computer, and is composed of data such as arrangement information (which pixel corresponds to) on the monitor 1b and luminance information (RGB, etc.). For the sake of convenience, an image that is visible as a result of being displayed on the monitor 1b is referred to as a virtual space image.

(Display on monitor)
Since the real space image arranged in the arbitrary coordinate system on the monitor 1b is in a perspective view displayed in perspective, the virtual space image in which the virtual space model is arranged in the arbitrary coordinate system on the monitor 1b is also perspective. It becomes the displayed perspective view. When the perspective-displayed real space image and the perspective-displayed virtual space image are superimposed and displayed on the monitor 1b, as shown in FIG. 2A, the virtual space image (the sewer pipe 3V), the roadway and the sidewalk are displayed. It is extremely useful because the positional relationship between the two can be grasped intuitively.

  When the user communicates to the terminal 1 that a virtual space image is to be displayed, such as by operating the “display button” of the terminal 1, based on the measurement values and the space model by the sensors (position measurement sensor, attitude measurement sensor). A virtual space image is created, and this virtual space image is displayed on the monitor 1b. The creation of the virtual space image may be performed by the terminal 1, or a server capable of communicating with the terminal 1 may be provided, and the virtual space image may be created by this server and distributed to the terminal 1. The virtual space image displayed on the monitor 1b is displayed until the next operation (for example, the next “display button” operation) is performed. Note that the display of the virtual space image on the monitor 1b is not limited to the timing according to the user's operation, and may be automatically performed at any time when the sensor acquires the measurement value.

  The virtual space image is not limited to the sewage pipe 3V created based on the actual object as shown in FIG. 2A, but based on the conceptual object such as the land boundary line 4V as shown in FIG. 3A. It can also be created. Further, by giving the site boundary line an arbitrary width (height), the site boundary line can be displayed as a plane, that is, as the site boundary surface 5V shown in FIG. Alternatively, if the spatial model has spatial information that can create a sectional view of a road, it can be displayed as if a part of the road was excavated as shown in FIG. In this way, if a virtual space image created as a perspective view adapted to the perspective display of the real space image can be displayed superimposed on the real space image, the land surveying can be performed only by holding the terminal 1 at the site. It is extremely useful because the land boundary can be visually confirmed without carrying out the inspection, and the road below can be visually confirmed without actually excavating.

  The spatial model is not limited to invisible objects and conceptual objects, and may have spatial information about visible objects. Since the visible object is photographed by the camera 1a and displayed on the monitor 1b as a real space image, when the space model has spatial information about the visible object, both the real space image and the virtual space image are the same for the visible object. It will be displayed on the monitor 1b. For example, when a certain building is photographed, a real space image of the building is displayed. However, when a virtual space image of the building is displayed in a superimposed manner, the display is complicated and difficult to understand. Therefore, it is preferable that various display methods can be selected for a virtual space image having a surface (or a combination of surfaces). For example, a method of pasting a texture on a virtual space image and overlaying it on a real space image, a method of displaying a virtual space image through translucency or the like, a method of displaying only the outline of a surface, or a mesh consisting of grid lines The method of displaying with is mentioned.

  Further, the virtual space image superimposed on the real space image on the monitor 1b can be selectively displayed. Specifically, when the user operates the terminal 1 and selects the display state of the virtual space image, the real space image and the virtual space image are superimposed on the monitor 1b, and the non-display state of the virtual space image is displayed. If selected, only the real space image is displayed on the monitor 1b. That is, the virtual space image can be displayed / hidden as required by the user.

  Furthermore, if the virtual space image can be identified for each feature, it can be displayed / hidden for each feature. For example, from the state in which a real space image of a building (visible object) is displayed on the monitor 1b and a virtual space image of the building and a virtual space image of a site boundary line are displayed, only the virtual space image of the building is selected. If it is not displayed, it is easy to grasp the building while leaving useful information such as the land boundary line.

  In general, since the spatial model is constructed over a wide geographical area, the spatial model has spatial information of the feature that greatly exceeds the range displayed on the monitor 1b. Therefore, when optically obtaining an arbitrary coordinate system on the monitor 1b based on the real space coordinates of the camera 1a, the attitude of the camera 1a, and the specifications of the camera 1a, a range that is simultaneously created as a virtual space image in the space model (That is, the drawing range) is extracted. Of course, this does not mean that extraction is performed in units of features, but what is to be drawn out of this feature when a part of the feature does not fit on the monitor 1b.

  If other information (data) associated with a feature or place (position) is provided as a database, various information related to the range (range in real space) displayed on the monitor 1b can be displayed. Examples of the various information include drawings such as detailed views and longitudinal views of the sewer pipe 3, and attribute information such as the owner and contact information of the site. FIG. 4 is a diagram showing the monitor 1b on which the plan view 6 regarding the sewer pipe 3V is displayed. As shown in this figure, the drawing and attribute information related to the virtual space image (such as the sewer pipe 3V and the site boundary line 4V) can be displayed as an overlay on the real space image or the virtual space image, or displayed as a separate window. It can also be displayed, or the screen can be divided and displayed. Moreover, it is desirable that the drawing and the attribute information regarding the virtual space image can be selected to be displayed or not displayed on the monitor 1b.

(Adjusting the display position of the virtual space image)
As described above, the virtual space image is subjected to space calculation processing based on the space information, the measurement value related to the camera 1a, and the like, and the display position on the monitor 1b is determined. However, depending on the measurement accuracy of the position measurement sensor and the posture measurement sensor, or depending on the position accuracy of the spatial information included in the spatial model, the image may not be displayed at the correct position on the monitor 1b. If only the virtual space image made up of invisible objects or conceptual objects is displayed on the monitor 1b, it cannot be immediately determined whether the display position is correct, but by displaying the virtual space image made up of visible objects on the monitor 1b. That is, the validity of the display position of the virtual space image can be determined by simultaneously displaying the real space image and the virtual space image for the same visible object.

  FIG. 5 is a diagram showing a monitor 1b in which a real space image captured by the camera 1a, a center line 2V and a sewer pipe 3V, which are virtual space images, are superimposed and displayed. As shown in this figure, when the real space image (center line 2R) and the virtual space image (center line 2V) of the center line 2 which is a visible object are displayed at the same time, the display position of the center line 2V from the “deviation” between them. Is immediately understandable.

  If the virtual space image is not displayed at the correct position on the monitor 1b, the correct position of the underground buried object and the land boundary cannot be grasped. Therefore, it is desirable to provide means for adjusting the virtual space image to the correct position. As this display position adjustment function, there are two means: a means for adjusting by a user operation and a means for automatically adjusting. Hereinafter, the two means will be described by taking the alignment of the real space image (center line 2R) and the virtual space image (center line 2V) with respect to the center line 2 as long as they are displayed on the monitor 1b. Needless to say, the alignment can be performed not only on the center line 2 but also on any other feature.

  The means for adjusting by user operation is a method in which the user displays the virtual space image at a correct position while checking the screen, and there are a method for moving the real space image and a method for moving the virtual space image.

  The method of moving the real space image is to move the terminal 1 while the user confirms the screen in a state where the virtual space image is displayed on the monitor 1b. As shown in FIG. 5, when a “deviation” occurs as if the virtual space image was translated relative to the real space image, the terminal 1 is also moved left and right and up and down with that posture (the user himself moves) You may). As shown in FIG. 6, when a “deviation” occurs as if the virtual space image is rotated with respect to the real space image, the terminal 1 is changed in its position as it is. Note that the posture here means rotation (ω, φ, κ) around three axes as described above, and it may be rotated (rolled) in the plane of FIG. It may be rotated (pitch) around the horizontal axis or rotated (yaw) around the vertical axis in FIG. In this way, the terminal 1 is moved left and right and up and down, or the attitude of the terminal 1 is changed, or these are combined as necessary, and the display position of the real space image is adjusted to the display position of the virtual space image. Display the aerial image at the correct position.

  The method of moving the virtual space image with respect to the method of moving the real space image is to move the virtual space image on the monitor 1b while the user confirms the screen while the virtual space image is displayed on the monitor 1b. There are various methods for moving the virtual space image. The movement of the virtual space image is to create a virtual space image by recalculation under new conditions and to redisplay it.

  The first method for moving the virtual space image is to designate the virtual space image on the monitor 1b with a finger or a touch pen and move it to a predetermined position by dragging. For example, in FIG. 5, the real space image (center line 2R) and the virtual space image (center line 2V) are superimposed on the visible center line 2, and the virtual space image is correctly displayed from the “deviation” between the two. I understand that it is not. Therefore, the center line 2V is designated using a touch pen or the like, and the center line 2V is moved to the position of the center line 2R by dragging as it is. If the display positions of the center line 2R that is the real space image and the center line 2V that is the virtual space image overlap, it can be considered that the virtual space image is displayed at the correct position. Note that the virtual space image can be moved using the arrow keys of the keyboard instead of dragging.

  The second method of moving the virtual space image is to form elements such as points, lines, and surfaces (hereinafter referred to as “forming elements”) that form the virtual space image on the monitor 1b, and the corresponding real space image. Specifies the forming element. For example, referring to FIG. 7, two points (Pv1 and Pv2) that form the center line 2V are designated using a touch pen or the like, and two points (Pr1 and Pr2) that form the center line 2R are designated. . At this time, first, Pv1 and Pr1 are specified in order, and then Pv2 and Pr2 are specified in order to recognize that Pv1 and Pr1 correspond and Pv2 and Pr2 correspond. As a result, the virtual space image is recalculated, and the virtual space image is displayed at the correct position on the monitor 1b. The forming element to be specified is not limited to a point but may be a line or a surface, and three or more forming elements may be specified.

  The third method for moving the virtual space image is to move the virtual space image on the monitor 1b by moving or rotating the terminal 1. In this case, it is necessary to temporarily change the real space image from the live view state to the still image state. For example, referring to FIG. 6, by performing a predetermined operation of the terminal 1 (for example, a button operation on the monitor 1b), the real space image is set to a still image state, and in this state, the terminal 1 is moved left and right and up and down. Or, change the attitude of the terminal 1 (combine these as necessary). The method of moving the terminal 1 left and right and up and the method of changing the attitude of the terminal 1 are the same as the above-described “method of moving a real space image”. A sensor of the terminal 1 measures the amount of movement of the terminal 1 and the changed amount of tilt, a virtual space image is created by recalculation based on the measured value, and this virtual space image is displayed on the monitor 1b. The In this case, the virtual space image may be moved by the user's operation (for example, “re-draw button” operation on the monitor 1b) after the terminal 1 is moved or the posture is changed. The virtual space image may be automatically moved as needed at the timing when the sensor acquires the measurement value in accordance with the movement or the posture change. This series of operations is performed until the virtual space image is displayed at the correct position (until the display positions of the real space image and the virtual space image match), and if the correct position of the virtual space image in the real space image can be confirmed, Cancel the still image state of the aerial image.

  Note that the above-described methods (first to third) for moving the virtual space image can be used alone or in combination of two or more. In the method of moving the virtual space image (first to third), the other virtual space image (for example, the sewer pipe 3V) is also moved in conjunction with the moved virtual space image (for example, the center line 2V). Accordingly, it can be considered that all the virtual space images on the monitor 1b are displayed at the correct positions.

  On the other hand, the automatically adjusted means is a method processed by a computer. In this case, the computer extracts a feature portion from the real space image by image recognition, and further, a partial feature of the feature portion (a feature portion on the shape or a feature portion on the image) based on the shape or image of the feature portion. Is extracted as a shape feature, matching processing (pattern matching, etc.) is performed on the virtual space image (visible object) with this shape feature as a reference, and the features of the virtual space image and real space image detected by the matching processing The entire virtual space image is displayed on the monitor 1b so that the positions of the parts match. This will be specifically described with reference to FIG. The center line 2R is extracted by image recognition from the real space image displayed on the monitor 1b. This image recognition can be performed using a known technique such as a method of automatically discriminating from a difference in luminance (or hue, saturation, or brightness) on an image. Next, a shape feature portion is extracted based on the shape formed by the outer edge (edge) of the center line 2R or an image (luminance, RGB, etc.) displaying the entire center line 2R, and the shape feature of the center line 2R serving as a reference is extracted. What is collated with the part (center line 2V) is detected from the virtual space image (visible object) displayed on the monitor 1b. Note that the collation determination performed here is not limited to a case where they completely match, but includes a case where a predetermined condition (threshold value) is satisfied. Then, the movement amount is calculated from the difference in display position on the monitor 1b between the center line 2R and the center line 2V, and the virtual space image is set to the correct position (position where the center line 2R and the center line 2V overlap) based on the movement amount. Redisplay. Of course, at this time as well, the display position is changed based on the calculated movement amount, not only for the center line 2V but for all virtual space images on the monitor 1b.

  As described above, when the real space image and the virtual space image are superimposed and displayed for the same visible object, it is possible to determine whether the display position of the virtual space image is correct and to adjust the position. When adjusting the virtual space image to the correct display position by user operation, it is necessary to display the virtual space image of the visible object during the display position adjustment, but after adjusting the display position, the virtual space image of the visible object is displayed. Is not displayed (display control means), and the visible object can be easily seen. In addition, the real space image can be changed from the live view state to the still image state with the virtual space image adjusted to the correct display position. Thereby, it is possible to eliminate the difficulty in viewing the screen due to camera shake or the like.

(Image composition device)
The image synthesizing apparatus according to the present invention has the technology described so far. Specifically, the camera 1a (image acquisition means), the monitor 1b (display screen), and measurement means (GPS, 6-axis sensor, etc.) A terminal 1 having a space model, storage means for storing the space model, image creating means for creating a virtual space model from space information of the space model and further creating a virtual space image, and displaying the virtual space image on the monitor 1b. Display means for superimposing and displaying on the real space image is provided. In addition, display control means for selecting whether to display the virtual space image on the monitor 1b or non-display can be provided, and display position adjusting means for displaying the virtual space image at a correct position (adjustment by user operation) Or automatic adjustment). Furthermore, the drawing means and attribute information related to the virtual space image displayed on the monitor 1b can be displayed by the display means, and display / non-display of these drawings and attribute information can be selected by the display control means. . In addition, although the terminal 1 in which all these means are stored can be used as the image composition apparatus of the present invention, the storage means and the image creation means may be accommodated in a casing different from the terminal 1.

(Image composition program)
The image composition program of the present invention causes a computer to execute the technology described so far. Specifically, a model reading function for retrieving and reading predetermined spatial information from a database of spatial models, GPS and six axes A measurement value reading function for reading a value measured by a measuring means such as a sensor, an image creation function for creating a virtual space model from space information and measurement values, and a virtual space image on the monitor 1b It has a display function to superimpose on the image. It is also possible to provide a display control function for selecting whether the virtual space image is displayed or not displayed on the monitor 1b, and a display position adjustment function (adjustment by user operation) that displays the virtual space image at the correct position. Or automatic adjustment).

  When the display position adjustment function is automatic adjustment, a feature part extraction function that extracts a feature part from a real space image and extracts a shape feature part from the feature part, and a virtual object of a visible object based on the shape feature part. The display position of the virtual space image so that the matching portion detection function that performs matching processing on the spatial image to detect the matching portion and the position of the shape feature portion and the matching portion on the display image match (or substantially match) A position adjusting function for adjusting the position.

  Furthermore, it is possible to search and display drawings and attribute information related to the virtual space image displayed on the monitor 1b by the display function, and to select display / non-display of these drawings and attribute information by the display control function. You can also.

  Note that it is not necessary to configure all the functions described here as a single program (of course, a single program may be used), and a form in which functions are selected and distributed to the host computer and the client computer may be used.

(Image composition system)
The image composition system according to the present invention includes the technology described so far, and specifically includes one or two or more terminals 1 and a server. All the terminals 1 are communicably connected to a server, and the communication line is wired or wireless.

  The terminal 1 includes a camera 1a (image acquisition means), a monitor 1b (display screen), measurement means such as GPS and a six-axis sensor, display means for displaying a virtual space image on the monitor 1b, and a virtual space image. Is provided with display control means for displaying / hiding, and further with communication means for communicating with the server.

  On the other hand, the server includes a space model as a database, storage means for storing the database, image creation means, and communication means for communicating with the terminal 1. Note that the storage unit and the image creation unit do not need to be provided in the same server (may be one server as a matter of course), the storage unit is provided in the database server, the image creation unit is provided in the application server, and the like. These may be provided in different servers. In addition, a diagram or / and attribute information regarding the virtual space image can be stored in a storage unit (other storage unit) that stores the space model.

  The display position adjusting means for adjusting the display position of the virtual space image can be provided in the terminal 1 or can be provided in the server. However, when adjusting by a user operation, the display position adjusting means is provided in the terminal 1 and automatically adjusted. If so, it is desirable to have a server.

A flow until the virtual space image is superimposed and displayed on the monitor 1b by the image composition system of the present invention will be described.
(1) A user photographs a scene (real space) in front of the user with the camera 1a of the terminal 1. Shooting in this case is an act of displaying the live view on the monitor 1b by holding the camera 1a in a predetermined direction, and is not necessarily limited to operating the shutter.
(2) When the user takes a picture, the position and orientation of the camera 1a are measured by the measuring means at the same time.
(3) The user requests the server to display a virtual space image using the communication means of the terminal 1. This request means can use a known technique such as operating the keyboard of the terminal 1.
(4) Information required for the server is transmitted from the terminal 1 via a communication line in response to a user request. The necessary information includes request information (to request a virtual space image) and the position coordinates and orientation (measurement value) of the camera 1a measured by the measurement means. If necessary, request information indicating that a diagram or / and attribute information regarding the virtual space image is to be transmitted is also sent.
(5) The server to which necessary information is sent from the terminal 1 calculates the drawing range to be displayed on the user's terminal 1 based on the measurement value and camera specifications (view angle, focal length, etc.). At this time, the camera specifications may be transmitted directly from the terminal 1, or various camera specifications are stored in the server, and the camera specifications according to the model of the terminal (model information is transmitted). It may be read.
(6) The server also retrieves spatial information within the drawing range from the spatial model database, and creates a virtual space model for this. Furthermore, based on this virtual space model, a virtual space image which is a perspective view displayed in perspective by a perspective method is created. This series of processing is executed by the image creation function based on the measurement value and the spatial information of the spatial model.
(7) The virtual space image created by the server is transmitted to the terminal 1 by the communication means of the server. When this is received by the communication means of the terminal 1, the virtual space image is displayed on the monitor 1b by the display means.
(8) When the display position adjusting means is provided in the terminal 1, after being displayed on the monitor 1b of the terminal 1, the user performs an operation while checking the screen, and the virtual space image becomes the correct display position. Adjust as follows. On the other hand, when the display control function is provided in the server and is automatically adjusted, a real space image is transmitted from the terminal 1, and a feature portion is obtained from the real space image by a program (feature portion extraction function). And the shape feature part are extracted, the collation part is detected from the virtual space image by the program (collation part detection function), and the display position of the virtual space image is adjusted by the program (position adjustment function). The adjusted virtual space image is transmitted to the terminal 1 and displayed on the monitor 1b.

  The image composition system of the present invention is the management of underground buried objects under roads, the management of wiring and piping contained in the walls of buildings, the confirmation of land boundaries and the locality of villages, towns and villages, unconstructed roads and apartments, etc. It can be used for explanation of residents about the planned structure, on-site confirmation of dangerous areas such as landslide disaster warning areas and designated landslide areas, and it can be applied to various applications such as car navigation, advertising, and games.

1 terminal 1a camera 1b monitor 2 center line 2R (real space image) center line 2V (virtual space image) center line 3 sewer pipe 3V (virtual space image) sewer pipe 4V (virtual space image) ground boundary 5V Site boundary surface (of virtual space image) 6 Plan view regarding sewer pipe A Optical fiber cable B Sewer pipe C (showing plane arrangement of underground objects) White line

Claims (10)

An image synthesizing apparatus that superimposes a virtual space image created based on a space model having space information on a real space image obtained by capturing a real space, and displays the image on a display screen.
Image acquisition means for acquiring the real space image;
Measuring means for measuring the position and orientation of said image acquiring means,
Storage means for storing the spatial model including the visible spatial information of visible objects and the invisible visible spatial information ;
A calculation process is performed based on the measurement value by the measurement unit and the space model, and the virtual space image of the visible object and the virtual space image of the invisible object are obtained so as to be a perspective view displayed on the display screen. Image creation means to create;
Display means for superimposing and displaying the real space image displayed in perspective and the virtual space image displayed in perspective on the display screen;
An image composition apparatus comprising: a display control unit that selectively displays or hides the virtual space image of the visible object and the virtual space image of the invisible object . The image synthesizing apparatus according to claim 1 Symbol placement,
When the display positions of the virtual space image and the real space image for the same visible visible object do not match on the display screen, the display positions of the virtual space image and the real space image match on the display screen. Alternatively, an image synthesizing apparatus comprising display position adjusting means for moving the virtual space image so as to substantially match. The image composition apparatus according to claim 2 ,
The display position adjusting means automatically extracts the shape feature based on the image feature of the real space image, and automatically detects the matching portion to be matched with the shape feature from the virtual space image of the visible object. An image synthesizing apparatus that automatically adjusts the display position of a virtual space image so that the positions of the feature part and the collation part on the display image match or substantially match. The image composition device according to any one of claims 1 to 3 ,
The display means displays a figure or / and attribute information related to the virtual space image,
An image synthesizing apparatus, wherein the display control means displays or hides the figure and / or attribute information. An image synthesis program that superimposes a virtual space image created based on a space model on a real space image obtained by capturing a real space and displays the image on a display screen,
A model reading function for acquiring this spatial model from a storage means storing a spatial model including spatial information of visible objects that are visible and spatial information of invisible objects that are not visible;
A measurement value reading function for acquiring the position and orientation of the image acquisition means obtained by measurement;
Based on the position and orientation of the image acquisition means and the space model, a virtual space image of the visible object and a virtual space image of the invisible object are created so that a perspective view is displayed on the display screen. Image creation function,
A display function for superimposing and displaying the real space image displayed in perspective and the virtual space image displayed in perspective on the display screen;
A display control function for selecting and displaying or not displaying the virtual space image of the visible object and the virtual space image of the invisible object;
When the display positions of the virtual space image and the real space image for the same visible object do not match on the display screen, the display positions of the virtual space image and the real space image match or substantially match with each other. As described above, a computer executes a display position adjustment function for moving the virtual space image. In the image composition program according to claim 5 ,
The display position adjustment function
A feature extraction function for extracting a shape feature based on a feature of the image in the real space image;
A matching part detection function for detecting a matching part to be matched with the shape feature part from a virtual space image of a visible object;
An image composition program comprising: a position adjustment function for adjusting a display position of a virtual space image so that positions of the shape feature portion and the collation portion on a display image are matched or substantially matched. In the image composition program according to claim 5 or 6 ,
The display function displays a figure or / and attribute information related to the virtual space image,
An image composition program, wherein the display control function displays or hides the figure and / or attribute information. An image synthesis system that superimposes a virtual space image created based on a space model on a real space image obtained by capturing a real space and displays the image on a display screen,
A server, and one or more terminals connected to the server via a wireless or wired communication line,
The server includes a storage unit that stores a spatial model including spatial information of a visible object that is visible and spatial information of an invisible object that is not visible, and a perspective view that is displayed in perspective on the display screen of the terminal. Image creating means for creating a virtual space image, and communication means,
The terminal includes an image acquisition unit that acquires the real space image, a measurement unit that measures the position and orientation of the image acquisition unit, and a display unit that displays the real space image and the virtual space image on a display screen. A display control unit that selects and displays or hides the virtual space image of the visible object and the virtual space image of the invisible object, and a communication unit.
When the user of the terminal requests the server, the measurement value by the measuring means of the terminal is transmitted to the server,
The server creates a virtual space image of the visible object and a virtual space image of the invisible object based on the measurement value of the terminal and the space model according to a request of the user of the terminal, Transmit these virtual space images to the terminal,
On the display screen of the terminal that has received the virtual space image, the real space image displayed in perspective and the virtual space image displayed in perspective are superimposed and displayed,
When the display positions of the virtual space image and the real space image for the same visible object do not match on the display screen, the display position adjustment means provided in the terminal unit causes the virtual space image and the real space image to be displayed. An image composition system, characterized in that the virtual space image can be moved so that the display positions coincide or substantially coincide on the display screen. An image synthesis system that superimposes a virtual space image created based on a space model on a real space image obtained by capturing a real space and displays the image on a display screen,
A server, and one or more terminals connected to the server via a wireless or wired communication line,
The server includes a storage unit that stores a spatial model including spatial information of a visible object that is visible and spatial information of an invisible object that is not visible, and a perspective view that is displayed in perspective on the display screen of the terminal. Image creating means for creating a virtual space image, and communication means,
The terminal includes an image acquisition unit that acquires the real space image, a measurement unit that measures the position and orientation of the image acquisition unit, and a display unit that displays the real space image and the virtual space image on a display screen. A display control unit that selects and displays or hides the virtual space image of the visible object and the virtual space image of the invisible object, and a communication unit.
When the user of the terminal requests the server, the measurement value by the measurement unit of the terminal and the real space image by the image acquisition unit of the terminal are transmitted to the server,
The server creates a virtual space image of the visible object and a virtual space image of the invisible object based on the measurement value of the terminal and the space model according to a request of the user of the terminal,
Furthermore, the server, when the display position of the virtual space image and the real space image for the same visible object do not match on the display screen, the shape feature based on the image feature of the real space image Part is automatically extracted, and a matching part to be matched with the shape feature is automatically detected from the virtual space image of the visible object, so that the position on the display image of the shape feature and the matching part is matched or substantially matched. After automatically adjusting the display position of the spatial image, the virtual space image of the visible object and the virtual space image of the invisible object are transmitted to the terminal,
An image composition system, wherein the real space image displayed in perspective and the virtual space image displayed in perspective are superimposed and displayed on a display screen of a terminal that has received the virtual space image. The image composition system according to claim 8 or 9 ,
The display means displays a figure or / and attribute information related to the virtual space image,
An image composition system, wherein the display control means displays or hides the figure and / or attribute information.

Images