WO2018220856A1

WO2018220856A1 - Street viewer system

Info

Publication number: WO2018220856A1
Application number: PCT/JP2017/020721
Authority: WO
Inventors: 秀敏櫻井
Original assignee: 秀敏櫻井
Priority date: 2017-06-02
Filing date: 2017-06-02
Publication date: 2018-12-06
Also published as: JPWO2018220856A1; JP6390992B1

Abstract

[Problem] To provide a smart viewer system with which a traveler can actually go to a tourist spot and view, together with additional information, landscape image information corresponding to traveler viewpoint information. [Solution] A street viewer system comprises: a landscape server device; and a portable terminal device. The landscape server device includes: a landscape image information retention unit that retains landscape image information, which is an image that shows a landscape, in association with viewpoint information comprising positional information and sight line information (to be discussed below) associated with the positional information; a viewpoint information acquisition unit; and a landscape image information output unit. The portable terminal device includes: a positional information acquisition unit; a sight line information acquisition unit that acquires sight line information, which is information indicating a sight line direction and/or viewpoint height information in the acquired positional information; a viewpoint information output unit that outputs, to the landscape server device, the viewpoint information which associates the acquired positional information and the sight line information; a landscape image information acquisition unit that acquires the landscape image information from the landscape server device according to the outputted viewpoint information; and a landscape image information output unit.

Description

Street viewer system

The present invention relates to a street viewer system, and more particularly to a street viewer system suitable for displaying a landscape image showing a current or past landscape or a cityscape (townscape).

Conventionally, when the street view function of Google map is used, it is possible to display the current scenery and the scenery showing the townscape from a personal computer browser. Recently, a “time machine function” for viewing past scenery has been added to the street view function of the Google map. When the Google map is displayed on the personal computer and the street view is started, the place where the street view can be displayed on the map is displayed, and the past street view of the place can be displayed.

Also, as a system that reproduces the past or future landscape of a point designated by a user with a mobile terminal, Japanese Patent No. 5363971 (Patent Document 1) can be cited. In this patent document 1, it is the system which reproduces the scenery in arbitrary recurrent times. This reproduction landscape system is a landscape reproduction system consisting of a user portable terminal and a landscape reproduction server that are connected to each other via a network. In the reproduction image request receiving unit 801, the position, orientation, and inclination of the user portable terminal, The reproduction image request including the instructed re-current time is received, and the reconstructed terrain extraction unit 804 and the reproduced object extraction unit 807 extract the three-dimensional shape information of the terrain and the object corresponding to the re-current time, and the two-dimensional projection image generation unit 809. Thus, a reproduction image in which the terrain object and the object object are arranged in a virtual three-dimensional space is generated, and the reproduction image is transmitted from the reproduction image transmission unit 811 to the user portable terminal.

Japanese Patent No. 5363971

In the Google Map street view function above, you can see the current or past scenery from the map displayed on the display screen using a browser on your computer, and you do not have to go to the actual sightseeing spot or place. This is a virtual experience that can be seen, and it is not enough for tourist spots and local areas to have an economic effect unless they actually come.

In the above-mentioned Patent Document 1, since the landscape is reproduced by the position, orientation and inclination of the user mobile terminal, when the child uses the user mobile terminal for a social studies tour, the viewpoint of the child and the user mobile terminal There was a problem that the viewpoint was shifted and it was difficult to use. In addition, since the landscape image is displayed with the camera of the terminal device as the line of sight orientation, the posture to always capture the landscape of the area where you want to see the reproduced image (usually the camera is on the back side of the display, so the display is placed at the eye position) If you do not look at the screen of the mobile terminal, you could not see the reproduced image.

An object of the present invention is to provide a smart viewer system that allows a traveler to actually visit a sightseeing spot and view landscape image information according to the traveler's viewpoint information together with additional information.

Another object of the present invention is to improve the operability in consideration of the landscape image to be displayed and the form of using the mobile terminal device, and go to the city for a social studies tour or the like to view the landscape information at a desired viewpoint height It is to provide a street viewer system that can do this.

In order to solve the above problems, the present invention provides the following street viewer system. That is, as a first invention, a street viewer system including a landscape server device and a mobile terminal device, wherein the landscape server device is associated with viewpoint information including position information and postscript line-of-sight information associated therewith. A landscape image information holding unit that holds landscape image information that is an image showing a landscape, a viewpoint information acquisition unit that acquires viewpoint information from a mobile terminal device, and a landscape image information associated with the acquired viewpoint information. A landscape image information output unit for transmitting to the mobile terminal device, a location information acquisition unit for acquiring location information that is information on a location where the mobile terminal device currently exists, and a location information acquisition unit A gaze information acquisition unit that acquires gaze information that is information indicating gaze direction or / and viewpoint height information, and view information that associates the acquired position information and gaze information with a landscape A viewpoint information output unit that outputs to the device, a landscape image information acquisition unit that acquires the landscape image information from the landscape server device according to the output viewpoint information, a landscape image information output unit that outputs the acquired landscape image information, and (Corresponding to claim 1).

In addition to the features described above, the landscape server apparatus includes a landscape image information holding unit by season in which the landscape image information holding unit holds landscape image information by season that is landscape image information by a plurality of past times. The time when the information output unit has time-specific landscape image information output means for outputting the landscape-specific landscape image information of the time corresponding to the acquired time information to the mobile terminal device, and is the information indicating the past time from the mobile terminal device A server-side time information acquisition unit that acquires information, and the mobile terminal device includes a mobile terminal-side time information acquisition unit that acquires time information, and a time information transmission unit that transmits the acquired time information to the landscape server. The landscape image information acquisition unit includes a season-specific landscape image information acquisition unit that acquires time-specific landscape image information according to the transmitted time information (corresponding to claim 2).

In addition to the features described above, the landscape server apparatus has a landscape image information holding unit for each event in which the landscape image information holding unit holds landscape image information for each event that is a landscape image information of a historical or / and seasonal event. The landscape image information output unit has event-specific landscape image information output means for outputting the event-specific landscape image information corresponding to the acquired event identification information to the mobile terminal device, and acquires event identification information from the mobile terminal device. The mobile terminal device has a server-side event identification information acquisition unit, the mobile terminal-side event identification information acquisition unit that acquires the event identification information, and the event identification information transmission unit that transmits the acquired event identification information to the landscape server device The landscape image information acquisition unit acquires an event-specific landscape image information according to the transmitted event identification information. Having another landscape image information acquiring means (Claim 3 corresponds).

In addition to the above features, a street viewer system in which a mobile terminal device is used instead of the mobile terminal device (corresponding to claim 4).

The landscape server device according to any one of claims 1 to 4 (corresponding to claim 5).

The portable terminal device according to any one of claims 1 to 3 (corresponding to claim 6).

The mobile-equipped terminal device according to claim 4 (corresponding to claim 7).

As a second invention, a street consisting of a landscape server device that holds landscape image information, which is an image showing a landscape in association with viewpoint information consisting of position information and postscript line-of-sight information associated therewith, and a mobile terminal device It is an operation method of the viewer system, and the operation method of the landscape server device transmits a viewpoint information acquisition step of acquiring viewpoint information from the mobile terminal device and landscape image information associated with the acquired viewpoint information to the mobile terminal device. A landscape image information output step, and the operation method of the mobile terminal device is a location information acquisition step of acquiring location information that is information on a location where the mobile terminal device currently exists, and the acquired location information. A line-of-sight information acquisition step for acquiring line-of-sight information, which is information indicating the line-of-sight direction and / or viewpoint height information, and the acquired position information and line-of-sight information. A viewpoint information output step for outputting linked viewpoint information to the landscape server device, a landscape image information acquisition step for acquiring landscape image information from the landscape server device according to the output viewpoint information, and outputting the acquired landscape image information An operation method of a street viewer system having a landscape image information output step (corresponding to claim 8).

As a third invention, a street consisting of a landscape server device that holds landscape image information that is an image showing a landscape in association with viewpoint information consisting of position information and postscript line-of-sight information associated therewith, and a mobile terminal device An operation program of the viewer system, the operation program of the landscape server device transmits a viewpoint information acquisition step of acquiring viewpoint information from the mobile terminal device, and landscape image information associated with the acquired viewpoint information to the mobile terminal device A landscape image information output step, and a landscape server device operation program recorded in a readable and executable manner on a landscape server device that is a computer. The mobile terminal device operation program is information on a position where the mobile terminal device currently exists. Position information acquisition step to acquire the position information and the line-of-sight method with the acquired position information Or / and a line-of-sight information acquisition step of acquiring line-of-sight information, which is information indicating viewpoint height information, and a viewpoint information output step of outputting viewpoint information that associates the acquired position information and line-of-sight information to the landscape server device, and output A landscape image information acquisition step for acquiring landscape image information from the landscape server device in accordance with the viewpoint information and a landscape image information output step for outputting the acquired landscape image information are recorded on the portable terminal device that is a computer so as to be executable. An operation program of the street viewer system, which is a program of the portable terminal device (corresponding to claim 9).

The operation method of the landscape server device according to claim 8 (corresponding to claim 10).

The operation method of the mobile terminal device according to claim 8 (corresponding to claim 11).

An operation program for the landscape server device according to claim 9 (corresponding to claim 12).

An operation program of the mobile terminal device according to claim 9 (corresponding to claim 13).

In addition to the above features, the line-of-sight information acquisition unit is continuously acquiring line-of-sight direction sensors that detect the direction of east, west, north, and south, and line-of-sight direction acquisition means that continuously acquires the line-of-sight directions from the line-of-sight direction sensor. A line-of-sight direction holding unit that fixes and holds the line-of-sight direction at a holding instruction acquisition timing, a line-of-sight direction holding unit that outputs a line-holding command based on an input from a user, and a line-of-sight direction holding unit Holding cancellation command output means for canceling holding of the held line-of-sight direction based on an input from the user, and the line-of-sight direction included in the line-of-sight information is configured to be the line-of-sight direction held A street viewer system according to claim 1 (corresponding to claim 14).

In addition to the features described above, the mobile terminal device includes a camera and a display, and the line-of-sight direction sensor acquires a line of sight of a scene projected on the display through the camera as a line-of-sight direction (corresponding to claim 15). .

In addition to the above features, the camera further includes a holding command shooting unit for shooting a landscape still image according to the output of the holding command of the holding command output means (corresponding to claim 16).

In addition to the above features, the camera has a zoom-in / zoom-out function that can be operated by the user, and has a magnification acquisition unit that acquires the magnification of the landscape still image at the time of shooting by the holding command shooting unit, and landscape image information The output unit further includes a landscape image output means for outputting a landscape image having the same magnification as that of the photographed still image on the display (corresponding to claim 17).

In addition to the above features, it further includes an alternate output unit capable of alternately outputting a landscape still image photographed on the display and a landscape image of the same magnification (corresponding to claim 18).

In addition to the above features, the image processing apparatus further includes a superimposition output unit that can superimpose and output a landscape still image photographed on the display and a landscape image at the same magnification (corresponding to claim 19).

As a fourth invention, a street consisting of a landscape server device that holds landscape image information that is an image showing a landscape in association with viewpoint information consisting of position information and postscript line-of-sight information associated therewith, and a mobile terminal device It is an operation method of the viewer system, and the operation method of the landscape server device transmits a viewpoint information acquisition step of acquiring viewpoint information from the mobile terminal device and landscape image information associated with the acquired viewpoint information to the mobile terminal device. A landscape image information output step, and the operation method of the mobile terminal device is a location information acquisition step of acquiring location information that is information on a location where the mobile terminal device currently exists, and the acquired location information. A line-of-sight information acquisition step for acquiring line-of-sight information, which is information indicating the line-of-sight direction and / or viewpoint height information, and the acquired position information and line-of-sight information. A viewpoint information output step for outputting linked viewpoint information to the landscape server device, a landscape image information acquisition step for acquiring landscape image information from the landscape server device according to the output viewpoint information, and outputting the acquired landscape image information A landscape image information output step, wherein the line-of-sight information acquisition step continuously detects a line-of-sight direction detected by the line-of-sight direction sensor substep and the line-of-sight direction sensor substep. A line-of-sight direction acquisition sub-step, and a line-of-sight direction holding sub-step for fixing and holding the line-of-sight direction being acquired continuously at the line-of-sight direction holding means at the acquisition timing of a holding command based on an input from the user; Holding cancellation command output sub for canceling the holding of the line-of-sight direction held by the line-of-sight holding means based on the input from the user Has a step, the configured Street viewer system method of operation as the viewing direction, which is included in the line-of-sight information is line-of-sight direction, which is held (claim 20 compatible).

As a fifth invention, a street comprising a landscape server device that holds landscape image information that is an image indicating a landscape in association with viewpoint information that includes position information and postscript line-of-sight information associated therewith, and a mobile terminal device The operation program of the viewer system, which is an operation program of the viewer system, includes a viewpoint information acquisition step of acquiring viewpoint information from the mobile terminal device, and a landscape image information associated with the acquired viewpoint information. The landscape image information output step to be transmitted to the computer is described in such a manner that it can be read and executed by the landscape server device that is a computer, and the operation program of the mobile terminal device that is the computer is information on the position where the mobile terminal device currently exists A position information acquisition step for acquiring position information, and a line-of-sight direction and / or high-viewpoint information in the acquired position information; A line-of-sight information acquisition step for acquiring line-of-sight information, a viewpoint information output step for outputting viewpoint information associating the acquired position information and line-of-sight information to the landscape server device, and a landscape according to the output viewpoint information A landscape image information acquisition step for acquiring landscape image information from the server device, and a landscape image information output step for outputting the acquired landscape image information are described in a computer-readable portable terminal device so as to be executable. The line-of-sight information acquisition step of the operation program includes a line-of-sight direction sensor sub-step for continuously detecting the direction of east, west, south, north, and south, and a line-of-sight direction acquisition sub for continuously acquiring the line-of-sight direction detected by the line-of-sight direction sensor sub-step Step and acquisition of a holding command based on the input from the user to the line-of-sight direction holding means for continuously acquiring the line-of-sight direction A line-of-sight holding sub-step that is fixed and held by imming, and a holding cancellation command output sub-step that cancels the line-of-sight holding held in the line-of-sight holding means based on an input from the user, An operation program of a street viewer system that is processed so that the line-of-sight direction included in the line-of-sight information is the line-of-sight direction held (corresponding to claim 21).

The operation program of the landscape server device according to claim 21 (corresponding to claim 22).

The operation program of the portable terminal device according to claim 21 (corresponding to claim 23).

According to the configuration described above, a traveler can actually go to a sightseeing spot and view landscape image information according to the traveler's viewpoint information together with additional information. In addition, you can visit the city for social studies tours, etc., and see the landscape information at the height of the desired viewpoint.

The figure for demonstrating the outline | summary of the smart viewer system of Embodiment 1. FIG. Functional block diagram of the smart viewer system of Embodiment 1 Functional block diagram of a configuration not including a display of the smart viewer system of the first embodiment The figure which shows the example of the display classification of Embodiment 1. The figure which shows an example (tablet) of the display in the smart viewer system of Embodiment 1. The figure which shows the other example (smart phone) of the display in the smart viewer system of Embodiment 1. The figure which shows the other example (head mounted display) in the smart viewer system of Embodiment 1. FIG. The figure which shows the other example (car navigation system) in the smart viewer system of Embodiment 1. FIG. The figure which shows an example of the acquisition means of the viewpoint information in the smart viewer system of Embodiment 1. The figure for demonstrating acquisition of the viewpoint information in the smart viewer system of Embodiment 1. The figure which shows an example of the landscape image acquisition interface screen in the smart viewer system of Embodiment 1. The figure which shows an example of the hardware constitutions of the landscape server apparatus in the smart viewer system of Embodiment 1. 2 is a diagram illustrating an example of a hardware configuration of a mobile terminal device in the smart viewer system of Embodiment 1. FIG. Process flow chart of smart viewer system of embodiment 1 Process flow chart of smart viewer system of embodiment 1 The figure for demonstrating the outline | summary of the smart viewer system of Embodiment 2. FIG. Functional block diagram of the smart viewer system of the second embodiment Functional block diagram of a configuration not including a display of the smart viewer system of the second embodiment The figure which shows an example of the landscape image acquisition interface screen in the smart viewer system of Embodiment 2. The figure which shows an example of the hardware constitutions of the landscape server apparatus in the smart viewer system of Embodiment 2. The figure which shows an example of the hardware constitutions of the portable terminal device in the smart viewer system of Embodiment 2. Process Flowchart of Smart Viewer System of Embodiment 2 Process Flowchart of Smart Viewer System of Embodiment 2 The figure for demonstrating the outline | summary of the smart viewer system of Embodiment 1. FIG. Functional block diagram of the smart viewer system of Embodiment 3 Functional block diagram of a configuration not including a display of the smart viewer system of Embodiment 3 The figure which shows an example of the hardware constitutions of the landscape server apparatus of the smart viewer system of Embodiment 3. The figure which shows an example of the hardware constitutions of the portable terminal device of the smart viewer system of Embodiment 3. Process Flowchart of Smart Viewer System of Embodiment 3 Process Flowchart of Smart Viewer System of Embodiment 3 Functional block diagram of the smart viewer system of Embodiment 4 The figure which shows an example of the hardware constitutions of the portable terminal device of the smart viewer system of Embodiment 4. Process Flowchart of Smart Viewer System of Embodiment 4 FIG. 10 is a diagram for explaining a zoom-in / zoom-out function of a camera in the smart viewer system according to the fifth embodiment. The figure which shows the example of a cap carrying the wearable camera in the smart viewer system of Embodiment 5.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the present invention is not limited to these embodiments, and can be implemented in various modes without departing from the spirit of the present invention.

Note that the functional blocks of the apparatus described below can be realized as a combination of hardware and software. Specifically, if a computer is used, a CPU (Central Processing Unit), a main memory, a system bus, or a secondary storage device (a storage medium such as a hard disk drive, a non-volatile memory, a CD or a DVD, and those media) Hardware, such as an input device used for information input, a display device, and other external peripheral devices, such as a tablet, a smartphone, a VR head-mounted display, a glasses-type display (monocular, see-through, etc.), Display with glasses-type camera (camera is built-in type, clip-on type, etc.), wearable camera (clothing / embedding, clip-on type, cap built-in, etc.) and interface for external peripheral devices, communication interface, location information interface , Gaze input in Over the face, the height input interface, a touch input interface, software keyboard interface, they and driver program for controlling the hardware other application programs, and the like for user interface applications. Then, by CPU processing according to the program developed on the main memory, the data input from the input device or other interface and held in the memory or hard disk is processed and stored, An instruction for controlling the software is generated. Alternatively, the functional blocks of this apparatus may be realized by dedicated hardware. Further, the present apparatus is not limited to being configured by a single piece of hardware or software, and may be configured by a combination of a plurality of pieces of hardware or software, or may be configured including a server apparatus via a network.

The present invention can be realized not only as an apparatus but also as a method. A part of the invention can be configured as software. Furthermore, a program used for causing a computer to execute such software and a recording medium on which the program is fixed are naturally included in the technical scope of the present invention (the same applies throughout this specification).
<Embodiment 1>
<Overview>

As shown in FIG. 1, the present embodiment 1 is a case where a traveler actually travels to a sightseeing spot or a student or student actually goes to the town for a social studies tour and is sent from a landscape server device (not shown). A landscape image indicating a landscape such as a landscape, a cityscape, etc. is received by a mobile terminal device or the like, and a street view (street) corresponding to viewpoint information composed of position information, a sight line direction of a traveler, and sight line information including viewpoint height information It is related with the street viewer system suitable for displaying a landscape) with guidance information. This street viewer system realizes a system capable of performing street view display according to a desired viewpoint (for example, an adult or a child's viewpoint) using the viewpoint height information.
In the example of FIG. 1, an example is shown in which a traveler has a portable terminal device and displays a Kyoto cityscape (street view) as a sightseeing spot with guidance information added. Or, as shown in FIG. 24, when a traveler has a mobile terminal device, visits shopping in Ginza, and points the mobile terminal device to a department store in Ginza 4-chome, the floor information of the department store is added and displayed. Is shown.
<Configuration of Embodiment 1>

As shown in FIG. 2, the street viewer system according to the first embodiment is a street viewer system including a landscape server device 0200 and a mobile terminal device 0210, and the landscape server device 0200 includes a landscape image information holding unit 0201. , A viewpoint information acquisition unit 0202, and a landscape image information output unit 0203. The mobile terminal device 0210 includes a position information acquisition unit 0211, a line-of-sight information acquisition unit 0212, a viewpoint information output unit 0213, and landscape image information. An acquisition unit 0214 and a landscape image information output unit 0215 are included.
<Description of Configuration of Embodiment 1>
<Embodiment 1 Landscape server device>

This landscape server device is, for example, arranged with a server room installed in one room of a building, or stored as big data in a data center. And it is connected with a portable terminal device via a communication network (including the Internet, a broadband network, a mobile network, etc.), acquires viewpoint information from the portable terminal device, and a landscape image corresponding to the acquired viewpoint information Information is transmitted to the mobile terminal device. The configuration of the landscape server device is as shown in FIG. 2 and is as follows.
<Embodiment 1 Landscape image information holding part>

The “landscape image information holding unit” 0201 holds landscape image information that is an image showing a landscape in association with viewpoint information including position information and postscript line-of-sight information associated therewith. Here, the retained landscape image information is the landscape image of a landscape, a cityscape, or the like, and position information and line-of-sight information (such as line-of-sight information including line-of-sight direction information and line-of-sight height information) associated with travelers and social studies visitors. ) Is stored for each piece of viewpoint information. In addition, when it is difficult to store landscape image information corresponding to all viewpoint information in the server, a method may be used in which a basic landscape image model is created and the viewpoint information is received and created instantaneously by simulation. . The landscape image information is literally the image information of the landscape, but the image includes a perspective image (composite image by computer) that sees through the interior of the building, a perspective image that sees through the underground shopping street (composite image by computer), It may include letters, symbols, codes, etc., and may include voice, music, sound effects, commentary voice, smell, smell, vibration (instruction for causing the mobile terminal to vibrate). The description may include a chronology, a map showing the current position, an old map showing the current position, a picture of a related artifact, a picture of a related cultural heritage, and the like. The landscape image may be an image synthesized by a computer or a real image. A high-definition satellite image associated with three-dimensional position information such as Google Earth (registered trademark) can be used as a mechanism for holding an image showing a landscape in association with viewpoint information. If this is used, the front and back of a building can be seen using a satellite photograph, and a landscape image associated with viewpoint information can be arbitrarily acquired. The stored landscape image information may include the current landscape and past landscapes, but for the current landscape, the landscape image is always located from a wearable camera that a person is looking at or a camera mounted on a car. It can be configured to be collected in association with information (collected from a car navigation system or the like) and constantly updated. Therefore, the collected images can be held in association with time such as date, morning, noon, evening, and night. Therefore, a landscape image can be returned to the mobile terminal device according to the time for acquiring the viewpoint information.
<Embodiment 1 Viewpoint information acquisition unit>

The “viewpoint information acquisition unit” 0202 acquires viewpoint information from the mobile terminal device 0210. As described above, the viewpoint information includes position information and line-of-sight information associated with the position information. Position information acquired from a GPS signal, a landscape image, or the like, and a line-of-sight direction of a traveler or the like associated with the position information View height information indicating the view height information is acquired from the mobile terminal device. Further, a configuration is also conceivable in which the landscape server device 0200 detects the position of the mobile terminal device 0210 and acquires the line-of-sight direction and viewpoint height information using the position information service system. The viewpoint information does not necessarily require the line-of-sight direction and the viewpoint height information, and the viewpoint height information may be selectively adopted. The viewpoint information may include a landscape image photographed by the camera of the mobile terminal device, a campus photographed image, and the like. This information can be used as reference information for determining viewpoint information by collating the viewpoint image held with the viewpoint information held in the scenery server device.
<Embodiment 1 Landscape image information output unit>

The “landscape image information output unit” 0203 transmits the landscape image information associated with the acquired viewpoint information to the mobile terminal device 0210. That is, the transmitted landscape image information is a landscape image in which a traveler or a social studies visitor actually goes out to the city and has a gaze direction and a gaze height from the actual position.
<Embodiment 1 portable terminal device>

The portable terminal device of the present embodiment includes a tablet having a communication function, a smartphone, a wearable terminal such as a glasses type or a watch type, a VR head mount system, and the like. Since a display is included in these tablets, as shown in FIG. 3, a mode in which landscape image information is externally output from a mobile terminal device and displayed on various displays can be considered.
As shown in FIG. 4, the display types include a tablet screen (see FIG. 5), a smartphone screen (see FIG. 6), a glasses built-in display, a VR head mounted display (see FIG. 7), and a mobile window glass (for example, A thin sheet on the surface of a train window such as the Shinkansen, or a head-up display on the window glass surface, a glass surface other than the windshield of an automobile), an in-car display (display device such as a car navigation system (Fig. 8), a display that can be viewed on the back seat of a car, such as a head-up display that is displayed on the windshield of a car), a display that is mounted in front of a rickshaw seat in a sightseeing spot, a display that can be viewed from a passenger seat of a bicycle taxi, There are telescope type displays and flexible sheet displays. Further, the communication function may be realized by a small device, a separate configuration may be adopted, and the communication unit and the display unit may be configured separately.

The viewpoint information acquisition means is performed by the position information acquisition unit 0211 and the line-of-sight information acquisition unit 0121.

As shown in FIG. 9, it may be realized by attaching various sensors to the display as described above, or wearing various sensors for travelers and social studies visitors, or incorporating various sensors in clothing. good. That is, as the means for acquiring viewpoint information, as shown in FIG. 9, glasses with sensors, tablet terminals with sensors, smartphones with sensors, smart watches with sensors, clothes with sensors, and the like are conceivable.

FIG. 10 shows a specific example of the viewpoint information acquisition method. That is, the line-of-sight direction, position information, and viewpoint height are shown. Hereinafter, in the description of the configuration of each unit of the mobile terminal device 0210, the description will be made at the related portions.
<Embodiment 1 Location information acquisition unit>

The “location information acquisition unit” 0211 acquires location information that is information on a location where the mobile terminal device currently exists. The position information acquisition unit 0211 acquires position information of longitude and latitude as shown in FIG. Specifically, it can be acquired from an acceleration sensor + gyro sensor, a GPS signal, road surface tag information, a landscape image, and the like. The position information can also be obtained from a system connected to a surveillance camera provided on the street. This is because the surveillance camera determines the person from the facial features, identifies the mobile terminal owned by the person from the database, identifies the position that can be grasped by the video of the surveillance camera, and identifies the mobile terminal It sends the location information to the device. If a plurality of mobile terminal devices are extracted at this time, it is possible to determine which mobile terminal device is carried by the person using the base station ID of the mobile terminal device covering the area of the monitoring camera. Mobile terminal devices can be narrowed down. The mobile terminal device can be easily specified by searching together with the use registration information of this system.
<Embodiment 1 Gaze Information Acquisition Unit>

The “line-of-sight information acquisition unit” 0212 acquires line-of-sight information, which is information indicating the line-of-sight direction and / or viewpoint height information in the acquired position information. As shown in FIG. 10, the line-of-sight direction can be acquired from a gyro sensor, a compass, a landscape image, or the like. Further, as shown in FIG. 10, the viewpoint height information can be acquired from condition setting by user input of the mobile terminal device, atmospheric pressure information, GPS signal, road surface tag information, landscape image, and the like. The line-of-sight information may be acquired from a wearable camera.
<First Embodiment Viewpoint Information Output Unit>

The “viewpoint information output unit” 0213 outputs viewpoint information in which the acquired position information and line-of-sight information are associated with each other to the landscape server device. Here, since the landscape server device 0200 and the mobile terminal device 0210 are connected via a network (including the Internet, broadband, and mobile network), information is exchanged by wireless communication. In addition, when there is a wireless WIFI access point in the town, it can be output to the landscape server device 0200 via the wireless WIFI access point. When using a mobile network of a mobile phone network, direct communication can be performed if the mobile terminal device is compatible with 3G or the mobile terminal device is compatible with LTE. In the above description, the viewpoint information is acquired based on the position where the mobile terminal actually exists. However, an option for setting a virtual viewpoint viewed from the sky may be provided. In this case, the landscape image information including the virtual appearance of the user standing at the position where the mobile terminal is located in the landscape image from the air is transmitted from the landscape server device and configured so that it can be seen. Can be considered.
Furthermore, the mobile terminal is not limited to what it has, but may be a drone. It is configured such that the line of sight of the camera mounted on the drone is acquired as viewpoint information.
<Embodiment 1 Landscape image information acquisition part>

The “landscape image information acquisition unit” 0214 acquires landscape image information from the landscape server device in accordance with the output viewpoint information. Here, examples of the landscape image acquisition interface screen of the mobile terminal device 0210 include various selection menus as shown in FIG. For example, there are selection menus such as “Building / Landmark / Scenery Commentary” and “Morning / Noon / Night”, and the user of the mobile terminal device selects “Building / Landmark / Landscape Description” as shown in FIG. Such commentary on buildings, landmarks, and landscapes (including guidance information) is displayed along with landscape image information. In addition, when the user of the mobile terminal device selects a selection menu of “morning, noon, night”, landscape image information corresponding to the time zone of the day is displayed. Further, “sunny”, “rain”, “cloudy”, “snowfall”, “storm” (accumulated in the landscape server apparatus) may be selected as the weather. It is also possible to make these selectable with a slide bar so that they change continuously from “clear” to “cloudy”, “cloudy” to “rain”, and “rain” to “storm” (continuous). The change is automatically generated in the landscape server device). By acquiring local weather information from the Internet and sending it to the landscape server device, it can be configured to acquire and display landscape image information according to the weather from the landscape server device, or in reverse weather It is also possible to configure to acquire and display the. The landscape image information includes a commentary on the landscape. As the selection menu, “child mode” (explained for children) may be set, or the selection menu may be a dial type. When “Children Mode” is selected, “Elementary School Students” (explained for elementary school students, the viewpoint height is set to the average viewpoint height of elementary school students), “Junior High School Students” (explained for junior high school students, The viewpoint height is set to the average viewpoint height of junior high school students), “High school students” (explained for high school students, the viewpoint height is set to the average viewpoint height of high school students), “University students” (for university students) Or a pull-down menu such as that the viewpoint height is set to the average viewpoint height of university students) or the user himself / herself may directly input his / her height. The mobile terminal device may automatically calculate the height of the viewpoint from its height. When one-touch input is performed using the pull-down menu, the height of the viewpoint varies from person to person, and in cases where the display screen is difficult to see, the viewpoint height can be changed using the “+” plus button or “−” minus button. May be.
<Embodiment 1 Landscape image information output unit>

The “landscape image information output unit” 0215 outputs the acquired landscape image information. Here, when the mobile terminal device 0210 includes a display, landscape image information that matches the viewpoint information of the traveler or social studies visitor is displayed on the display screen of the mobile terminal device in response to the selection of the landscape image acquisition interface. Is done. When the mobile terminal device does not include a display, the landscape image information is displayed on the display 0316 connected to the mobile terminal device 0310 as shown in FIG. In addition to this, the output destination may be a built-in display of another portable terminal device that wants to share the same image, or a display connected to another computer. Moreover, you may comprise so that the landscape image information output may be preserve | saved simultaneously with the actual landscape image currently image | photographed with the camera. This is executed by a landscape image simultaneous storage unit or the like. In this case, both images can be displayed side by side on a display such as at home later and enjoyed. This may be a moving image or a still image. In addition, in the case where the landscape image cannot be output unless the display obstructs the field of view, the landscape image information output unit determines that it is walking when there is a change in position information, and does not output it. It can also be configured. Furthermore, when the landscape server device goes out to an area that does not have a landscape image, it is configured to receive information to that effect from the landscape server device, and configured to notify that it has gone out of the landscape image output range by an alarm. It is possible to do.
This is the case where the landscape image storage range of the landscape server device is limited to, for example, a historic building and its surroundings. Furthermore, it is a case where the view range of the landscape image of the landscape server device is limited to sightseeing spots such as Kyoto.
<Embodiment 1 Action: Hardware Configuration (Landscape Server Device)>

As shown in FIG. 12, the configuration of the computer of the landscape server apparatus includes a CPU 1201, a nonvolatile memory 1202 such as an HDD and a ROM, a main memory 1203 such as a D-RAM, and an interface 1204. The nonvolatile memory 1202 stores programs such as a landscape image information holding program, a viewpoint information acquisition program, and a landscape image information output program. The data includes position information, line-of-sight information, viewpoint information, landscape image information, and the like. These programs and data are read into the holding area of the main memory 1203 and executed in the work area. The interface 1204 includes a network interface 1205, a display interface 1206, a disk interface 1207, a keyboard interface 1208, and the like.
<Embodiment 1 Operation: Hardware Configuration (Mobile Terminal Device)>

As shown in FIG. 13, the configuration of the computer of the mobile terminal device includes a CPU 1301, a nonvolatile memory 1302 such as an HDD and a ROM, a main memory 1303 such as a D-RAM, and an interface 1304. The nonvolatile memory 1302 stores programs such as a position information acquisition program, a line-of-sight information acquisition program, a viewpoint information acquisition program, a landscape image information acquisition program, and a landscape image information output program. The data includes position information, line-of-sight direction information, viewpoint height information, line-of-sight information, viewpoint information, landscape image information, and the like. These programs and data are read into the holding area of the main memory 1303 and executed in the work area. The interface 1304 includes a network interface 1305, a position information interface 1306, a line-of-sight input interface 1307, a height input interface 1308, a touch input interface 1309, a software keyboard input interface 1310, and the like.
<Embodiment 1 Process Flow Landscape Server Device>

As shown in the left diagram of FIG. 14, the operation of the computer of such a landscape server apparatus executes a viewpoint information acquisition step of acquiring viewpoint information from the mobile terminal apparatus by the viewpoint information acquisition program (step 1401). Here, the viewpoint information includes position information and line-of-sight information associated with the position information, and since details have been described above, description thereof will be omitted.

Next, a landscape image information output step of transmitting the landscape image information associated with the viewpoint information acquired by the landscape image information output program to the mobile terminal device is executed (step 1402). The viewpoint information acquisition step 1401 and the landscape image information output step 1403 described above are repeated until the viewpoint information acquisition from the portable terminal device is completed (step 1403).
<First Embodiment Flow of Processing Mobile Terminal Device>

As shown in the right diagram of FIG. 14, the operation of the computer of such a mobile terminal device includes a location information acquisition step of acquiring location information that is information on a location where the mobile terminal device currently exists by a location information acquisition program. Execute (step 1411). Here, since acquisition of position information has been described above, a description thereof will be omitted.

Next, a line-of-sight information acquisition step of acquiring line-of-sight information which is information indicating the line-of-sight direction and / or viewpoint height information in the position information acquired by the line-of-sight information acquisition program is executed (step 1412). Here, the line-of-sight information may include only information on the line-of-sight direction, or may include both the line-of-sight direction and the viewpoint height information.

Next, a viewpoint information output step of outputting the viewpoint information in which the position information acquired by the viewpoint information output program and the line-of-sight information are associated to the landscape server apparatus is executed (step 1413). Here, since the details of the viewpoint information acquisition have been described above, the description thereof will be omitted.
Next, a landscape image information acquisition step of acquiring landscape image information from the landscape server device according to the viewpoint information output by the landscape image information acquisition program is executed (step 1414).
Next, a landscape image information output step for outputting the landscape image information acquired by the landscape image information output program is executed (step 1415). Then, on the display screen of the mobile terminal device, the landscape image information that matches the viewpoint information of the traveler or social studies visitor is displayed corresponding to the selection of the landscape image acquisition interface as described above.
The processing steps 1411 to 1415 are repeated until the processing is finished (step 1416).

When the mobile terminal device does not include a display, a display display step 1516 is executed after the landscape image information output step 1515 as shown in FIG. As shown in FIG. 3, landscape image information is displayed on a display 0316 connected to the mobile terminal device 0310. Since other processes are the same as the processes of FIG. 14 of the first embodiment, the description thereof is omitted.

As described above, according to the first embodiment, a traveler can actually go to a sightseeing spot and view landscape image information according to the viewpoint information of the traveler together with additional information. In addition, it is possible to go to the town for a social studies tour or the like and view the landscape information at a desired viewpoint (for example, a child).
<Embodiment 2>
<Overview>

As shown in FIG. 16, the second embodiment is a system that can display past landscape-specific landscape image information in addition to the street viewer system of the first embodiment. In the example of FIG. 16, the example which displayed the landscape image information of the year 972 AD is shown. When these past landscape information by time is actually present as a database, realistic past landscape image information is held in the landscape server device, and the held landscape image information is displayed. If there is no landscape image information going back considerably, the content of the landscape image information is created and stored in the landscape server device. Storage in the landscape server is different according to the era, and a 3D version of the above-mentioned Google Earth computer composition is available. As described above, the viewpoint can be freely moved in the computer composition, and a landscape image can be acquired from an arbitrary viewpoint. These can also be used for historical materials.
<Configuration of Embodiment 2>

As illustrated in FIG. 17, the street viewer system according to the second embodiment is a street viewer system including a landscape server device 1700 and a mobile terminal device 1710, and the landscape server device 1700 includes a landscape image information holding unit 1701. , A viewpoint information acquisition unit 1702, and a landscape image information output unit 1704, a landscape image information holding unit 1701 has a landscape image information storage unit 1704 classified by time, and a landscape image information output unit 1705 classified by time. A server-side time information acquisition unit 1706 is also included.
The portable terminal device 1710 includes a position information acquisition unit 1711, a line-of-sight information acquisition unit 1712, a viewpoint information output unit 1713, a landscape image information acquisition unit 1714, and a landscape image information output unit 1715, and includes portable terminal side time information. An acquisition unit 1716 and a time information transmission unit 1717 are included, and the landscape image information acquisition unit 1714 includes a landscape image information acquisition unit 1718 classified by time.
FIG. 17 shows a configuration in which the mobile terminal device of the second embodiment includes a display, and FIG. 18 shows a configuration in which the mobile terminal device of the second embodiment does not include a display. Except for the display 1819 of FIG. 18, the configuration is the same as that of FIG.
<Description of Configuration of Second Embodiment>
<Embodiment 2 landscape server apparatus>

In addition to the configuration of the first embodiment, the landscape server device holds landscape image information according to time in the device, acquires time information from the mobile terminal device, and landscape according to time according to the acquired viewpoint information. It is configured to output image information. Since other details are the same as those of the first embodiment, detailed description thereof is omitted.
Hereinafter, features common to the first embodiment will be omitted, and a characteristic configuration will be described.
<Embodiment 2 Landscape image information holding means according to period>

The “landscape image information holding unit by time” 1704 is provided in the landscape image information holding unit 1701, and holds landscape image information by time that is landscape image information by a plurality of past times. For example, in the mobile terminal device, when the user of the mobile terminal device selects a selection menu such as according to time in the landscape image acquisition interface as shown in FIG. 11, “Meiji to Showa”, “Edo late”, “Edo” It holds landscape image information by multiple periods so that the periods of “mid-term”, “early Edo”, “Azuchi / Momoyama”, “Muromachi”, and “Kamakura” can be set. For landscape image information that does not currently exist, landscape image information is created by image processing technology, or when a computer graphic image is stored in another database, the image is used.

A plurality of typical landscape image components are prepared in advance according to the position information, and the prepared landscape image components may be combined by processing rules to generate and transmit a landscape image. . For example, if the location information and the time information specify the vicinity of the Kyoto Imperial Palace in the Muromachi period, the landscape image components that make up the typical noble residence, the landscape image components that make up the Imperial Palace, etc. In combination, the cityscape is reproduced. If the location information and timing information are near the Yokohama port in the Meiji period, the landscape image components that make up the Western-style building and the landscape image components that make up the ship in the Meiji period are combined into the city of Yokohama. Is transmitted as a landscape image.

In addition, you may have an old photo storage part which accumulate | stores an old photograph, and a landscape image component creation part which produces a landscape image component from the accumulated old photo. In order to create landscape image components from old photos, building image recognition means for recognizing old photo building parts, etc., and building surface configuration to determine what material the recognized building surface is composed of It may be composed of element judgment means. Then, based on the determined result, a landscape image component is created by computer graphics.
<Embodiment 2 server side time information acquisition part>

The “server-side time information acquisition unit” 1706 acquires time information that is information indicating the past time from the mobile terminal device. The period information is information by historical age of “Meiji to Showa”, “Late Edo”, “Middle Edo”, “Edo early”, “Azuchi / Momoyama”, “Muromachi”, “Kamakura” It may be the age of the first year, 1998, 2008, 2013, etc., or it may be detailed information such as every year, every two years, how many hours and what minutes. When such time information is selected from the selection menu of the portable terminal device by the user, the server-side time information acquisition unit 1706 acquires the past time information as described above via the network.
<Embodiment 2 Landscape image information output means by period>

The “landscape image information output unit by time” 1705 is provided in the landscape image information output unit 1703, and outputs the landscape image information by time corresponding to the acquired time information to the mobile terminal device 1710. The landscape image information according to time is composed of moving images for realism if the images are still images or moving images in the Meiji and Taisho eras, or if moving images remain in the Showa era. In the Edo period, the Azuchi / Momoyama period, etc., there are no moving images, so digital landscapes of digital images are created from past paintings and materials, and content videos such as animation are used. Depending on the sightseeing spot, videos of NHK taiga dramas such as historically famous greats remain, so such videos can be used as landscape image information by period. In addition, original video content may be created in the cities. When using this system for social studies visits, landscape images by era are constructed by adding era landscape image information and explanations so that they can be used as digital teaching materials.
<Embodiment 2 portable terminal device>
<Embodiment 2 portable terminal side time information acquisition part>

The “portable terminal side time information acquisition unit” 1716 acquires time information. Here, the time information is obtained by selecting a selection menu by the user of the portable terminal device or by directly inputting the date.
As shown in FIG. 19, the landscape image acquisition interface screen is displayed on the display screen of the mobile terminal device, the current position or the recording position is selected, and the past age is selected on the time setting screen. For example,
Acquire historical time reports for the Meiji-Showa, Late Edo, Mid-Edo, Early Edo, Azuchi / Momoyama, Muromachi, and Kamakura periods. Alternatively, it may be an age such as 1989, 1998, 2008, 2013, or a detailed information input screen such as every year, every two years, how many hours and what minutes. As for the time information, the selection menu of the street viewer system can be appropriately changed by a user such as a traveler or a social studies visitor.
When the “OK button” is selected by the user, it is transmitted from the time information transmission unit 1717 to the landscape server apparatus 1700. If you make a mistake, you can select the “Redo button” and start over. In the above-described example, the selection example using the selection menu is shown. However, a selection method based on a user's voice instruction through a voice input interface may be used. The time information acquisition unit may be configured to display a slide bar (may be configured to be slidable with a touch panel) on the display as an interface and acquire the time information while continuously changing with the slide bar. Is possible. In such a case, it is possible to continuously view landscape image information of different times in synchronization with the movement of the slide bar. The slide bar may be configured to be displayed together with the history chronology. In other words, it is convenient to display the slide bar adjacent to the position indicating the year of the belt-like history chronology and move it with the touch panel. Then, for example, the state of the Great Buddha in Nara can be seen continuously for repairs according to the times.
<Embodiment 2 time information transmission part>

The “time information transmission unit” 1717 transmits the acquired time information to the landscape server device 1700. Here, the above-mentioned time information acquired from the mobile terminal side time information acquisition unit 1716 is transmitted to the landscape server apparatus 1700 via a network (not shown).
<Embodiment 2 Landscape image information acquisition means according to time>

The “landscape image information acquisition unit by time” 1718 is provided in the landscape image information acquisition unit 1714, and acquires landscape image information by time according to the transmitted time information. For example, in a case where a user of a mobile terminal device uses a street viewer system while traveling in Kyoto, landscape image information according to the Kyoto era is displayed. This is not limited to Kyoto, and by displaying landscape image information suitable for the area, it is possible to convey the appeal to travelers in each tourist spot and those who live there.
<Embodiment 2 Action: Hardware Configuration (Landscape Server Device)>

As shown in FIG. 20, the configuration of the computer of the landscape server apparatus includes a CPU 2001, a nonvolatile memory 2002 such as an HDD and a ROM, a main memory 2003 such as a D-RAM, an interface 2004, a network interface 2005, and a display An interface 2006, a disk interface 2007, and a keyboard interface 2008 are included. The nonvolatile memory 2002 stores programs such as a time-specific landscape image information holding subprogram, a server-side time information acquisition program, and a time-specific landscape image information output subprogram. The data includes position information, line-of-sight information, viewpoint information, landscape image information, time information, and the like. These programs and data are read into the holding area of the main memory 2003 and executed in the work area. The other program configuration is the same as that of the first embodiment shown in FIG.
<Embodiment 2 Operation: Hardware Configuration (Mobile Terminal Device)>

As shown in FIG. 21, the configuration of the computer of the portable terminal device includes a CPU 2101, a nonvolatile memory 2102 such as an HDD and a ROM, a main memory 2103 such as a D-RAM, an interface 2104, a network interface 2105, a position The information interface 2106, the line-of-sight input interface 2107, the height input interface 2108, the touch input interface 2109, and the software keyboard input interface 2110 are configured. The nonvolatile memory 2102 stores programs such as a portable terminal side time information acquisition program, a time information transmission program, and a time-specific landscape image information acquisition program. The data includes position information, line-of-sight direction information, viewpoint height information, line-of-sight information, viewpoint information, landscape image information, time information, and the like. These programs and data are read into the holding area of the main memory 2103 and executed in the work area. The other program configuration is the same as the configuration of FIG.
<Embodiment 2 Process Flow Landscape Server Device>

As shown in the left diagram of FIG. 22, the operation of the computer of such a landscape server apparatus executes a viewpoint information acquisition step of acquiring viewpoint information from the mobile terminal apparatus by the viewpoint information acquisition program (step 2201). Here, the viewpoint information includes position information and line-of-sight information associated with the position information, and since details have been described above, description thereof will be omitted.

Next, a server-side time information acquisition step is executed for acquiring time information, which is information indicating the past time, from the mobile terminal device by the server-side time information acquisition program (step 2202). Here, since the timing information has been described above, the description thereof is omitted.

Next, a landscape image information output step of transmitting landscape image information associated with the viewpoint information acquired by the landscape image information output program to the mobile terminal device is executed (step 2203). In the landscape image information output step 2203, a landscape image information output step by time for outputting landscape image information by time corresponding to the time information acquired by the landscape image information output subprogram by time to the mobile terminal device is executed (step) 2204).

Until the viewpoint information acquisition from the portable terminal device is completed (step 2205), the viewpoint information acquisition step 2201, the server-side time information acquisition step 2202, the landscape image information output step 2203, and the landscape image information output sub-step 2204 according to time are performed. repeat.
<Embodiment 2 Processing Flow Portable Terminal Device>

As shown in the right diagram of FIG. 22, the operation of the computer of such a portable terminal device executes a position information acquisition step of acquiring position information as described above by the position information acquisition program (step 2211).

Next, a line-of-sight information acquisition step of acquiring line-of-sight information which is information indicating the line-of-sight direction and / or viewpoint height information in the position information acquired by the line-of-sight information acquisition program is executed (step 2212). Here, the line-of-sight information may include only information on the line-of-sight direction, or may include both the line-of-sight direction and the viewpoint height information.

Next, a viewpoint information output step for outputting the viewpoint information in which the position information acquired by the viewpoint information output program and the line-of-sight information are associated to the landscape server apparatus is executed (step 2213). Here, since the details of the viewpoint information acquisition have been described above, the description thereof will be omitted.

Next, a mobile terminal side time information acquisition step of acquiring time information by the mobile terminal side time information acquisition program is executed (step 2214). Here, since the acquisition of the time information has been described above, the description thereof will be omitted.

A time information transmission step of transmitting the time information acquired by the time information transmission program to the landscape server device is executed (step 2215).
Next, a landscape image information acquisition step of acquiring landscape image information from the landscape server device according to the viewpoint information output by the landscape image information acquisition program is executed (step 2216). In the landscape image information acquisition step 2216, a landscape image information acquisition step by time for acquiring landscape image information by time according to the time information transmitted by the landscape image information acquisition program by time is executed (step 2217).

Next, a landscape image information output step for outputting the landscape image information acquired by the landscape image information output program is executed (step 2218). And according to the selection of the landscape image acquisition interface as described above on the display screen of the mobile terminal device, the landscape image information according to the time corresponding to the viewpoint information of the traveler or social studies visitor according to the transmitted time information Is displayed.
The processing is repeated until the processing steps 2211 to 2218 are completed (step 2219).
When the mobile terminal device does not include a display, a display display step 2319 is executed after the landscape image information output step 2318 as shown in FIG. As shown in FIG. 18, landscape image information is displayed on a display 1819 connected to the mobile terminal device 1810.

As described above, according to the second embodiment, in addition to the first embodiment, by displaying the past landscape image information according to the time period, it is possible to convey the further attractiveness of the sightseeing spot and the area to the tourist and the social studies visitor. it can.
<Embodiment 3>
<Overview>

In the third embodiment, in addition to the street viewer system of the first or second embodiment, the event-specific landscape image information that is the landscape information of a historical or seasonal event can be displayed. A typical example of a historical event is “Honnoji Temple”. The landscape image information of such a historical event is stored in the landscape server device together with viewpoint information and event identification information. Landscape information of seasonal events is periodically collected and stored in a landscape server device. In the case of a historical event, the landscape image information related to the event is stored in the landscape server device along with viewpoint information (location information + line-of-sight information) such as historical monuments, castle ruins, and buildings that existed in the past. is doing.
<Configuration of Embodiment 3>

As shown in FIG. 25, the street viewer system according to the third embodiment is a street viewer system including a landscape server device 2500 and a mobile terminal device 2510. The landscape server device 2500 includes a landscape image information holding unit 2501 and , A viewpoint information acquisition unit 2502, and a landscape image information output unit 2504, a landscape image information holding unit 2501 has a landscape image information holding unit 2504 for each event, and a landscape image information output unit 2505 for each event. And a server-side event identification information acquisition unit 2606,
The mobile terminal device 2510 includes a position information acquisition unit 2511, a line-of-sight information acquisition unit 2512, a viewpoint information output unit 2513, a landscape image information acquisition unit 2514, and a landscape image information output unit 2515, and includes a mobile terminal side event identification. It has an information acquisition unit 2516 and an event identification information transmission unit 2517, and the landscape image information acquisition unit 2514 has an event-specific landscape image information acquisition unit 2518.

FIG. 25 illustrates a configuration in which the mobile terminal device of the third embodiment includes a display, and FIG. 26 illustrates a configuration in which the mobile terminal device of the third embodiment does not include a display. Except for the display 2619 of FIG. 26, the configuration is the same as that of FIG.
<Description of Configuration of Third Embodiment>
<Embodiment 3 Landscape server device>

In addition to the configuration of the first embodiment or the second embodiment, this landscape server device holds event-specific landscape image information in the device, acquires event identification information from the mobile terminal device, and obtains the acquired viewpoint information. It is configured to output event-specific landscape image information. Further, the landscape image for each event is configured to additionally hold characters, text, audio information, and the like. Since other details are the same as those in the first or second embodiment, detailed description thereof is omitted.
Hereinafter, features common to the first embodiment or the second embodiment are omitted, and a characteristic configuration will be described.
<Embodiment 3 Landscape image information holding means by event>

The “landscape image information holding unit by event” 2504 has in the landscape image information holding unit 2501 and holds the landscape image information by event which is the landscape image information of a historical or / and seasonal event. The event-specific landscape image displays a related landscape image in relation to the position where the user of the mobile terminal device currently exists, and does not include an image that is not related to the position in which the user currently exists. The event-specific landscape image information is collected in advance by collecting historical event data and seasonal event data, along with still images, moving images, etc., and adding audio information of commentary and narration. Or when the scenery image information relevant to others exists on the internet as needed, a scenery server apparatus may search and hold on the internet.
<Embodiment 3 Server-side event identification information acquisition unit>

The “server-side Internet identification information acquisition unit” 2506 acquires event identification information from the mobile terminal device. The event identification information is information on historical events such as monuments, castle ruins, and ancient battlefields, and information on seasonal events such as international and domestic event venues.
When the “event menu” is selected from the selection menu of the mobile terminal device as shown in FIG. 11, the server side event identification information acquisition unit 2506 acquires the event identification information as described above via the network. Moreover, when a traveler goes to a sightseeing spot and receives event identification information that has nothing to do with the sightseeing spot, the landscape server device transmits it as a user message to the portable terminal device such as an error display.
<Embodiment 3 Landscape image information output means by event>

The “landscape image information output unit by event” 2505 is provided in the landscape image information output unit 2503 and outputs event-specific landscape image information corresponding to the acquired event identification information to the mobile terminal device. As the landscape image information by event, landscape image information associated with viewpoint information such as an old battlefield ruins is displayed, or landscape image information associated with position information of a department store in the city is displayed. The historical event-specific landscape image information is displayed to social studies visitors and travelers who have actually visited the event, so that the memory of the location can be made clear. Information tools that can appeal to travelers and social studies visitors can be realized.
<Embodiment 3 portable terminal device>
<Embodiment 3 Mobile terminal side event identification information acquisition part>

The “portable terminal side event identification information acquisition unit” 2516 acquires event identification information. Here, the event identification information is performed by selection of a selection menu or direct input by the user of the mobile terminal device.
As shown in FIG. 11, the landscape image acquisition interface screen is displayed on the display screen of the mobile terminal device, the current position or the recording position is selected, and the “event” menu is selected. The selection menu of these event identification information can be appropriately changed by a user such as a traveler or a social studies visitor in the street viewer system.
When the “event menu” is selected by the user, the event identification information transmission unit 2517 transmits the event menu to the landscape server apparatus 2500. In the above-described example, the selection example using the selection menu is shown. However, a selection method based on the voice instruction of the user through the voice input interface may be used.
<Embodiment 3 Event identification information transmitter>

The “event identification information transmission unit” 2517 transmits the acquired time information to the landscape server device 2500. Here, the event identification time information as described above acquired from the mobile terminal side event identification information acquisition unit 2516 is transmitted to the landscape server apparatus 2500 via a network (not shown).
<Embodiment 3 Landscape image information acquisition means by event>

The “landscape image information acquisition unit by event” 2518 is provided in the landscape image information acquisition unit 2514, and acquires the landscape image information by event according to the transmitted event identification information. For example, when a traveler visits Honnoji Temple in Kyoto and uses the smart viewer system of the third embodiment, and a traveler turns his mobile terminal device outside Honnoji Temple, the state of Akechi Mitsuhide army of "Honnoji Temple" is shown. It is displayed as event-specific landscape image information, or when entering the Honnoji Temple, when a traveler looks at the mobile terminal device, an event-specific landscape image showing the battle between Oda Nobunaga and Akechi Mitsuhide is displayed. The event-specific landscape image information can be changed depending on the position of the user of the mobile terminal device. This is not limited to “the transformation of Honnoji”, and by displaying the landscape image information suitable for the area, it is possible to convey the appeal to travelers in each tourist spot and those who live there.
<Embodiment 3 Action: Hardware Configuration (Landscape Server Device)>

As shown in FIG. 27, the configuration of the computer of the landscape server apparatus includes a CPU 2701, a nonvolatile memory 2702 such as an HDD and a ROM, a main memory 2703 such as a D-RAM, an interface 2704, a network interface 2705, and a display. An interface 2706, a disk interface 2707, and a keyboard interface 2708 are included. Nonvolatile memory 2702 stores programs such as event-specific landscape image information holding subprograms, server-side event identification information acquisition programs, and event-specific landscape image information output subprograms. The data includes position information, line-of-sight information, viewpoint information, landscape image information, event identification information, and the like. These programs and data are read into the holding area of the main memory 2703 and executed in the work area. The other program configuration is the same as that of the first embodiment shown in FIG.
<Embodiment 3 Operation: Hardware Configuration (Mobile Terminal Device)>

As shown in FIG. 28, the configuration of the computer of the portable terminal device includes a CPU 2801, a nonvolatile memory 2802 such as an HDD and a ROM, a main memory 2803 such as a D-RAM, an interface 2804, a network interface 2805, a position The information interface 2806 includes a line-of-sight input interface 2807, a height input interface 2808, a touch input interface 2809, and a software keyboard input interface 2810. The non-volatile memory 2802 stores programs such as a portable terminal side event identification information acquisition program, an event identification information transmission program, and an event identification-specific landscape image information acquisition program. The data includes position information, line-of-sight direction information, viewpoint height information, line-of-sight information, viewpoint information, landscape image information, event identification information, and the like. These programs and data are read into the holding area of the main memory 2803 and executed in the work area. The other program configuration is the same as the configuration of FIG.
<Embodiment 3 Process Flow Landscape Server Device>

As shown in the left diagram of FIG. 29, the operation of the computer of such a landscape server apparatus executes a viewpoint information acquisition step of acquiring viewpoint information from the mobile terminal apparatus by the viewpoint information acquisition program (step 2901). Here, the viewpoint information includes position information and line-of-sight information associated with the position information, and since details have been described above, description thereof will be omitted.

Next, a server-side event identification information acquisition step of acquiring event identification information from the mobile terminal device by the server-side event identification information acquisition program is executed (step 2902). Here, since the event identification information has been described above, a description thereof will be omitted.

Next, a landscape image information output step of transmitting landscape image information associated with the viewpoint information acquired by the landscape image information output program to the mobile terminal device is executed (step 2903). In the landscape image information output step 2903, the event-specific landscape image information output step of outputting the event-specific landscape image information of the event corresponding to the event identification information acquired by the event time-specific landscape image information output subprogram to the mobile terminal device is executed. (Step 2904).

Until the viewpoint information acquisition and event identification information acquisition from the portable terminal device is completed (step 2905), the viewpoint information acquisition step 2901, the server-side event identification information acquisition step 2902, the landscape image information output step 2903, and the event-specific landscape are described above. The image information output substep 2904 is repeated.
<Embodiment 3 Processing Flow Portable Terminal Device>

As shown in the right diagram of FIG. 29, the operation of the computer of such a portable terminal device executes a position information acquisition step of acquiring position information as described above by the position information acquisition program (step 2911).

Next, a line-of-sight information acquisition step of acquiring line-of-sight information, which is information indicating the line-of-sight direction and / or viewpoint height information in the position information acquired by the line-of-sight information acquisition program, is executed (step 2912). Here, the line-of-sight information may include only information on the line-of-sight direction, or may include both the line-of-sight direction and the viewpoint height information.

Next, a viewpoint information output step of outputting the viewpoint information in which the position information acquired by the viewpoint information output program and the line-of-sight information are associated to the landscape server apparatus is executed (step 2913). Here, since the details of the viewpoint information acquisition have been described above, the description thereof will be omitted.

Next, the portable terminal side event identification information acquisition unit step of acquiring event identification information by the portable terminal side event identification information acquisition program is executed (step 2914). Here, since acquisition of the event identification information has been described above, a description thereof will be omitted.

Next, an event identification information transmission step of transmitting event identification information acquired by the event identification information transmission program to the landscape server apparatus is executed (step 2915).
Next, a landscape image information acquisition step of acquiring landscape image information from the landscape server device according to the viewpoint information output by the landscape image information acquisition program is executed (step 2916). In landscape image information acquisition step 2916, an event-specific landscape image information acquisition step of acquiring event-specific landscape image information according to the event identification information transmitted by the event-specific landscape image information acquisition program is executed (step 2917).
Next, a landscape image information output step for outputting the landscape image information acquired by the landscape image information output program is executed (step 2918). And according to the selection of the landscape image acquisition interface as described above on the display screen of the mobile terminal device, it matches the viewpoint information of the traveler or social studies visitor, and the landscape image information by event according to the transmitted event identification information Is displayed.
Processing is repeated until the processing steps 2911 to 2918 are completed (step 2919).

When the mobile terminal device does not include a display, a display display step 3019 is executed in a landscape image information output step 3018 as shown in FIG. As shown in FIG. 26, landscape image information is displayed on a display 2619 connected to the mobile terminal device 2610.

As described above, according to the third embodiment, in addition to the first or second embodiment, by displaying the event-specific landscape image information of the event, further attractions of tourist spots and regions for tourists and social studies visitors. Can be communicated.
<Embodiment 4>
<Overview>

In the fourth embodiment, in addition to the smart viewer system described in any one of the first to third embodiments, a street viewer system corresponding to the posture of a user who views a landscape image in consideration of the usage pattern of the mobile terminal device. Realized.
<Configuration of Embodiment 4>

As illustrated in FIG. 31, the street viewer system according to the fourth embodiment is a street viewer system including a landscape server device 3100 and a mobile terminal device 3110, and the landscape server device 3100 includes a landscape image information holding unit 3101. , A viewpoint information acquisition unit 3102, and a landscape image information output unit 3103. The mobile terminal device 3110 includes a position information acquisition unit 3111, a line-of-sight information acquisition unit 3112, a viewpoint information output unit 3113, and landscape image information. An acquisition unit 3114 and a landscape image information output unit 3115 are included.
Further, the line-of-sight information acquisition unit 3112 includes a line-of-sight direction sensor 3120, a line-of-sight direction acquisition unit 3121, a line-of-sight direction holding unit 3122, a holding command output unit 3123, and a holding cancellation command output unit 3124. The line-of-sight direction included in the information is configured to be a retained line-of-sight direction.
Hereinafter, the description of the common configuration described in any one of the first to third embodiments will be omitted, and the configuration characteristic to the fourth embodiment will be described.
<Description of Configuration of Fourth Embodiment>
<Embodiment 4 landscape server apparatus>

Since the landscape server apparatus of this Embodiment 4 is the same as the landscape server apparatus as described in any one of Embodiment 1 to Embodiment 3, description is abbreviate | omitted.
<Embodiment 4 portable terminal device>

The mobile terminal device according to the fourth embodiment is characterized by the function of the line-of-sight information acquisition unit 3112 in addition to the mobile terminal devices according to the first to third embodiments. Hereinafter, a characteristic configuration of the mobile terminal device according to the fourth embodiment will be described.
<Embodiment 4 Gaze direction sensor>

A “line-of-sight direction sensor” 3120 detects the direction of up, down, east, west, north, and south. Here, examples of the line-of-sight direction sensor include a digital electronic compass, and the vertical direction is detected by detecting the direction of east, west, south, and north and the level. Such a digital electronic compass is built in the portable terminal device 3110.
<Embodiment 4 Gaze direction acquisition means>

“Gaze direction acquisition means” 3121 continuously acquires the gaze direction from the gaze direction sensor 3120. When the user of the mobile terminal device 3110 is using the mobile terminal device in various directions, the line-of-sight direction is continuously acquired. As a general rule, the information of the line-of-sight direction acquired continuously is continuously included in the viewpoint information and transmitted to the landscape server device. The same applies to the position information and the viewpoint height information, and the viewpoint information is continuously generated and continuously transmitted to the landscape server device. Therefore, landscape image information is continuously received from the landscape server device according to the viewpoint information transmitted continuously. This is one operation pattern. This operation is the same in the systems of the first to third embodiments. On the other hand, as described above, the line-of-sight direction is generally the same as the imaging direction of the camera of the mobile terminal device, so in order to see the landscape image, the camera must always be directed to the desired landscape. It becomes impossible to see the actual landscape behind the shadows. In other words, it is difficult to observe the actual landscape while comparing the landscape image. Therefore, a configuration is desired in which a landscape image transmitted from the landscape server device can be seen even if the line of sight excludes the screen of a tablet terminal placed on the knee, for example.
The configuration for solving this problem is described below.
<Embodiment 4 Gaze direction holding means>

The “line-of-sight holding unit” 3122 fixes and holds the line-of-sight direction being acquired continuously at the acquisition timing of the holding command. When there is an input from the user of the portable terminal device, a holding instruction in the line-of-sight direction acquired continuously from the holding instruction output means 3123 is issued and fixed at the acquisition timing. The mode for fixing the time may include a mode for fixing the moment and a mode for fixing the predetermined time length. The former fixes the instantaneous gaze direction, and the latter fixes the gaze direction for a certain time, for example, 10 seconds. In the latter case, for example, the tablet camera is rotated 360 degrees so that the entire surrounding landscape image can be seen in 10 seconds.
<Embodiment 4 Holding Command Output Unit>

The “holding command output unit” 3123 outputs a holding command based on an input from the user to the line-of-sight direction holding unit 3122. Here, this can be realized by the user touching a “hold button icon” provided on the display screen of the mobile terminal device. Moreover, although this holding method used the touch input method by a user, the method hold | maintained by audio | voice input may be sufficient. Thus, since the line-of-sight direction included in the line-of-sight information is maintained, the user can view a desired landscape image in a free posture without worrying about the orientation of the mobile terminal device. In the landscape server device, when the fixed viewpoint information output and received by the holding command output means is received, the fixed information is fixed unless the portable terminal device notifies that the holding has been canceled in accordance with the holding cancellation command described later. Do not send anything other than time landscape images. This is so that the mobile terminal device does not have to receive an extra landscape image in the line-of-sight direction. In addition, when the fixed time is 10 seconds, the landscape server device is configured to repeatedly transmit a landscape image of 10 seconds. 10 seconds is the time length of the movement of the line of sight.
<Embodiment 4 Holding Cancel Command Output Unit>

The “holding cancellation command output unit” 3123 cancels the holding of the line-of-sight direction held by the line-of-sight direction holding unit 3122 based on the input from the user. Here, in order to cancel the fixed (fixed) state while maintaining the line-of-sight direction, the user can return to the original normal display state by touching the “hold cancellation button icon”. In this case as well, if there is no problem of environmental noise in which the user is present, a method of canceling the holding by voice input may be used.
<Embodiment 4 Landscape image information output unit>

The “landscape image information output unit” 3114 outputs the acquired landscape image information. Here, when the mobile terminal device 3110 includes a display, landscape image information that matches the viewpoint information of the traveler or social studies visitor is displayed on the display screen of the mobile terminal device corresponding to the selection of the landscape image acquisition interface. Is done. For example, if the fix is for 10 seconds, the landscape image for 10 seconds is repeatedly displayed.
<Embodiment 4 Action: Hardware Configuration (Landscape Server Device)>

Since the hardware configuration of the computer of the landscape server device of the fourth embodiment is the same as that of any one of the first to third embodiments, description thereof is omitted.
<Embodiment 4 Operation: Hardware Configuration (Mobile Terminal Device)>

As shown in FIG. 32, the configuration of the computer of the mobile terminal device includes a CPU 3201, a non-volatile memory 3202 such as an HDD and a ROM, a main memory 3203 such as a D-RAM, an interface 3204, a network interface 3205, a position The information interface 3206, the line-of-sight input interface 3207, the height input interface 3208, the touch input interface 3209, and the software keyboard input interface 3210 are configured. The nonvolatile memory 3202 stores programs such as a gaze direction sensor subprogram, a gaze direction acquisition subprogram, a gaze direction holding subprogram, a holding command output subprogram, and a holding cancellation command output subprogram. The data includes position information, line-of-sight direction information, viewpoint height information, line-of-sight information, viewpoint information, landscape image information, and the like. These programs and data are read into the holding area of the main memory 3203 and executed in the work area.
<Embodiment 4 Process Flow Landscape Server Device>

33, the operation of the computer of such a landscape server apparatus executes processing of a viewpoint information acquisition step 3301 and a landscape image information output step 3302.

The viewpoint information acquisition step 3301 and the landscape image information output step 3302 described above are repeated until the viewpoint information acquisition from the portable terminal device is completed (step 3303).
<Embodiment 4 Processing Flow Portable Terminal Device>

As shown in the right side of FIG. 33, the flow of processing of such a mobile terminal device is as follows.

First, in position information acquisition step 3311, position information of the current position of the mobile terminal device is acquired. Next, in the line-of-sight information acquisition step 3312, the initial line-of-sight direction was continuously acquired. However, upon receiving a holding command from the user, this is output to the line-of-sight direction holding means, and the line-of-sight direction holding step is executed. The line-of-sight holding sub-step 3313 is a step of holding the line-of-sight direction continuously acquired at the timing of the user's input instantaneously while stopping the time or holding it for a predetermined time length. Then, step 3315 which transmits viewpoint information to a landscape server apparatus is performed through sub-step 3314 which produces | generates viewpoint information with gaze height information. Step 3317 of displaying a landscape image on a display etc. is performed through step 3316 of receiving the landscape image from the landscape server apparatus which received the viewpoint information generated based on this held line-of-sight direction. As a result, it is possible to view a landscape image based not only on the continuously acquired line-of-sight direction but also on the fixed line-of-sight direction without worrying about the current direction of the line-of-sight direction.
<Embodiment 5A>
<Summary of Embodiment 5A>

The mobile terminal device according to the present embodiment has a camera and a display, and the line-of-sight direction sensor is characterized in that a line-of-sight viewing a landscape projected on the display through the camera is acquired as the line-of-sight direction.
<Embodiment 5A Configuration>

The configuration of Embodiment 5A is based on the configuration of Embodiment 14 with a camera and a display added. The configuration requirements common to the fourteenth embodiment are the landscape image information holding unit, the viewpoint information acquisition unit, and the landscape image information output unit for the landscape server device, and the location information acquisition unit for the mobile terminal device. The line-of-sight direction included in the line-of-sight information included in the line-of-sight information acquisition unit, the viewpoint information output unit, the landscape image information acquisition unit, the landscape image information output unit, and the line-of-sight information acquisition unit is retained. A line-of-sight sensor configured to be in the line-of-sight direction, line-of-sight direction acquisition means, line-of-sight direction holding means, holding command output means, and holding cancellation command output means, and description of common parts will be omitted.
<Description of Configuration of Embodiment 5A>
<Embodiment 5A Description of Camera>

“Camera” is provided in the mobile terminal device. A single lens or a plurality of lenses may be used. A fixed focus lens or a variable focus lens may be used. The image pickup device of the camera is generally an individual image pickup device. CMOS is better from the viewpoint of imaging speed. The center normal direction of the image captured by the image sensor is processed as the line-of-sight direction. That is, various built-in viewpoint related functions are configured to detect the center normal direction of the image sensor.

When the camera has a plurality of lenses, it can be configured to switch the line-of-sight direction by switching the lens as the line-of-sight direction. Moreover, in the case of a omnidirectional camera, it can be considered that a landscape image in the omnidirectional direction is transmitted from the landscape server device. In the case of an omnidirectional camera capable of 360-degree shooting, a landscape image of the all-sky movie may be transmitted from the landscape server device. It may be one that uses two omnidirectional cameras and can shoot similar to the omnidirectional camera.

As shown in FIG. 35, when using a wearable camera (a part of the mobile terminal device) in which the imaging center of the camera attached to the cap is the viewing direction, the viewing direction is fixed in the state of (A). Even if the person turns downward as shown in (B) afterwards (for example, by voice or an operating means at hand), the camera's line-of-sight direction may be kept fixed. By doing so, you can view the old landscape image of the landscape you saw from a tablet display with a natural posture.
<Embodiment 5A Display>

The “display” displays an image captured by the camera, or displays a landscape image returned from the landscape server device by viewpoint information including the line-of-sight direction obtained by the line-of-sight direction sensor. First, the user displays a local landscape image captured by the camera on the display, and checks the angle of view and uses the landscape image sent from the landscape server device on the display.

However, since a single display may be used instead of a single display, the local landscape may be displayed on one display and the landscape image may be displayed on another display.
Furthermore, when there is one display, a landscape image and an actual landscape may be switched by a switching button. In addition, the built-in display displays the actual landscape, and the landscape image from the landscape server device is output to a portable terminal device such as a tablet terminal by proximity wireless communication such as Bluetooth (registered trademark). You may enable it to view the transmitted landscape image.

<Other incidental elements>
About "sound", the scenery image from the scenery server device may be one to which sound is added. The sound that matches the scenery is prepared in advance. For example, the sound of a temple bell, the sound of a wind, and the voice of a merchant. For example, in the case of a bell sound, it is preferable to configure the sound so that the sound can be heard from the direction in which the bell exists in the landscape image. It can be configured using multi-channel speakers.
<Embodiment 5B>
<Overview of Embodiment 5B>

The mobile terminal device according to the present embodiment is characterized in that the camera has a holding command shooting unit and takes a landscape still image according to the output of the holding command from the holding command output means.
<Embodiment 5B Configuration>

The configuration of Embodiment 5B is based on the configuration of Embodiment 5A, with a holding instruction photographing unit added to the camera. The configuration requirements common to the fifteenth embodiment are the landscape image information holding unit, the viewpoint information acquisition unit, and the landscape image information output unit for the landscape server device, and the location information acquisition unit for the mobile terminal device. The line-of-sight direction included in the line-of-sight information included in the line-of-sight information acquisition unit, the viewpoint information output unit, the landscape image information acquisition unit, the landscape image information output unit, and the line-of-sight information acquisition unit is retained. A line-of-sight sensor configured to be in the line-of-sight direction, line-of-sight direction acquisition means, line-of-sight direction holding means, holding command output means, and holding cancellation command output means, and description of common parts will be omitted.
<Description of Configuration of Embodiment 5B>
<Embodiment 5B Shooting Unit at Holding Command>
The “holding command shooting unit” is built in the camera and shoots a landscape still image according to the output of the holding command from the holding command output means 3123. The image capturing unit at the time of holding command can capture a still image in the holding state in the line-of-sight direction. The still image may be a moving image having a predetermined time length. Furthermore, the still image can be configured to shoot a plurality of still images while switching the focus to a plurality. This is because the focus is on the front teahouse or the back gate. This focus information is preferably included in the viewpoint information and transmitted to the landscape server device. The landscape server device may be configured to transmit the landscape image after taking the focus position into account. A plurality of landscape images are transmitted to the mobile terminal device according to the focus position, and the mobile terminal device is configured to hold the landscape image and the still image in association with each other at the focus position.
<Embodiment 5C>
<Outline of Embodiment 5C>

The mobile terminal device of the present embodiment has a magnification acquisition unit in a camera having a zoom-in / zoom-out function that can be operated by a user, acquires a magnification of a landscape still image at the time of shooting by a holding command shooting unit, and displays Is characterized in that a landscape image having the same magnification as that of the photographed still image is output.
<Embodiment 5C Configuration>

The configuration of the embodiment 5C is based on the configuration of the embodiment 5B, and a magnification acquisition unit is added to a camera that performs zoom-in / zoom-out functions, and a landscape image having the same magnification as the still image photographed on the display is output to a landscape image information output unit. The same magnification landscape image output means for outputting is added. The configuration requirements common to the sixteenth embodiment are the landscape image information holding unit, the viewpoint information acquisition unit, and the landscape image information output unit for the landscape server device, and the location information acquisition unit for the mobile terminal device. The line-of-sight direction included in the line-of-sight information included in the line-of-sight information acquisition unit, the viewpoint information output unit, the landscape image information acquisition unit, the landscape image information output unit, and the line-of-sight information acquisition unit is retained. Gaze direction sensor configured to be in the gaze direction, gaze direction acquisition means, gaze direction holding means, holding command output means, holding release command output means, and magnification acquisition possessed by a camera having a zoom in / zoom out function And the same-magnification landscape image output means in the landscape image information output unit, and the description of the common parts is omitted.
<Description of Configuration of Embodiment 5C>
<Description of Embodiment 5C Camera>
The “camera” has a zoom-in / zoom-out function that can be operated by the user, and can perform magnification shooting by the user's operation.
<Embodiment 5C Magnification acquisition unit>

The “magnification acquisition unit” acquires the magnification of the landscape still image at the time of photographing by the photographing unit at the time of holding instruction. When the user holds the line-of-sight direction of the mobile terminal device using the zoom-in / zoom-out function of the camera, the magnification of the landscape still image at the time of shooting by the holding command shooting unit can be acquired. As shown in FIG. 34A, in the case where the line-of-sight direction of the mobile terminal device is held, an example of performing double zoom shooting is shown. If a building and a part of a five-storied tower are photographed behind it, the magnification is acquired from the magnification acquisition unit, and the position coordinates of the user and the position coordinates of the object to be displayed in the landscape image and the size of the object are known in advance. Therefore, if it is 2 times zoom, it can be calculated | required by calculation what scale image a landscape image should be projected on a portable terminal device.
<Embodiment 5C same magnification landscape image output means>
The “same magnification landscape image output means” is provided in the landscape image information output unit, and outputs a landscape image having the same magnification as the photographed still image on the display. As shown in FIG. 34 (B), since the coordinate position and size of the landscape image are known, a past landscape image of the year 972, for example, as shown in FIG. To do.
<Embodiment 5D>
<Outline of Embodiment 5D>

The mobile terminal device of this embodiment adds an alternate output unit capable of alternately outputting a landscape still image captured on the display and a landscape image of the same magnification, and has the same magnification as the landscape still image captured on the display. It is characterized in that landscape images are output alternately.
<Embodiment 5D Configuration>

The configuration of Embodiment 5D is obtained by adding an alternate output unit capable of alternately outputting a landscape still image photographed on the display and a landscape image of the same magnification based on the configuration of Embodiment 5C.
The configuration requirements common to the seventeenth embodiment are the landscape image information holding unit, the viewpoint information acquisition unit, and the landscape image information output unit for the landscape server device, and the location information acquisition unit for the mobile terminal device. The line-of-sight direction included in the line-of-sight information included in the line-of-sight information acquisition unit, the viewpoint information output unit, the landscape image information acquisition unit, the landscape image information output unit, and the line-of-sight information acquisition unit is retained. Gaze direction sensor configured to be in the gaze direction, gaze direction acquisition means, gaze direction holding means, holding command output means, holding release command output means, and magnification acquisition possessed by a camera having a zoom in / zoom out function And the same-magnification landscape image output means in the landscape image information output unit, and the description of the common parts is omitted.
<Description of Configuration of Embodiment 5D>

The “alternate output unit” alternately outputs a landscape still image photographed on the display and a landscape image at the same magnification. Thereby, it can watch while comparing with a landscape still image and a landscape image alternately.
<Embodiment 5E>
<Summary of Embodiment 5E>

The mobile terminal device of this embodiment adds a superimposition output unit that can superimpose and output a landscape still image captured on the display and a landscape image of the same magnification, and has the same magnification as the landscape still image captured on the display. It is characterized by the fact that the landscape image is superimposed and output.
<Embodiment 5E Configuration>

The configuration of Embodiment 5E is obtained by adding a superimposition output unit that can superimpose and output a landscape still image photographed on the display and a landscape image of the same magnification based on the configuration of Embodiment 5C or 5D. The configuration requirements common to the embodiment 5C or 5D are the landscape image information holding unit, the viewpoint information acquisition unit, and the landscape image information output unit for the landscape server device, and the location information acquisition for the mobile terminal device. The line-of-sight direction included in the line-of-sight information provided in the line-of-sight information provided in the line-of-sight information acquisition unit, the viewpoint information output unit, the landscape image information acquisition unit, the landscape image information output unit, and the line-of-sight information acquisition unit is retained. A camera having a zoom-in / zoom-out function, a gaze direction sensor configured to be a gaze direction, a gaze direction acquisition unit, a gaze direction holding unit, a holding command output unit, a holding cancellation command output unit, and The magnification acquisition unit, the same magnification landscape image output means included in the landscape image information output unit, and the alternate output unit, and the description of the common parts is omitted.
<Description of Configuration of Embodiment 5E>

The “superimposition output unit” superimposes and outputs a landscape still image captured on the display and a landscape image at the same magnification. Thereby, a landscape still image and a past landscape image are displayed on the display in a mixed form, and a new experience that combines the present and the past can be provided for travelers and social studies visitors.

As described above, according to the embodiment, the traveler can actually go to the sightseeing spot and view the landscape image information according to the viewpoint information of the traveler together with the additional information. In addition, it is possible to go to the city for a social studies tour or the like and view the landscape information at a desired viewpoint (for example, the viewpoint of a child).

Claims

A street viewer system comprising a landscape server device and a mobile terminal device,
The landscape server device
A landscape image information holding unit that holds landscape image information that is an image indicating a landscape in association with viewpoint information including position information and postscript line-of-sight information associated therewith,
A viewpoint information acquisition unit that acquires viewpoint information from the mobile terminal device;
A landscape image information output unit that transmits landscape image information associated with the acquired viewpoint information to the mobile terminal device; and
Have
The mobile terminal device
A position information acquisition unit that acquires position information that is information of a position where the mobile terminal device currently exists;
A line-of-sight information acquisition unit that acquires line-of-sight information, which is information indicating the line-of-sight direction and / or viewpoint height information in the acquired position information;
A viewpoint information output unit that outputs viewpoint information that associates the acquired position information and line-of-sight information to the landscape server device;
A landscape image information acquisition unit that acquires landscape image information from the landscape server device according to the output viewpoint information;
A landscape image information output unit for outputting the acquired landscape image information;
Having a street viewer system.
The landscape server device
The landscape image information holding unit has a landscape image information holding unit by season that holds landscape image information by season that is landscape image information by a plurality of past times,
The landscape image information output unit has a season-specific landscape image information output means for outputting the landscape-specific landscape image information corresponding to the acquired time information to the mobile terminal device,
It has a server-side time information acquisition unit that acquires time information that is information indicating the past time from the mobile terminal device,
The mobile terminal device
A mobile terminal side time information acquisition unit for acquiring time information;
A time information transmitter that transmits the acquired time information to the landscape server;
Have
The street viewer system according to claim 1, wherein the landscape image information acquisition unit includes a season-specific landscape image information acquisition unit that acquires time-based landscape image information according to the transmitted time information.
The landscape server device
The landscape image information holding unit has event-specific landscape image information holding means for holding event-specific landscape image information that is landscape image information of historical or / and seasonal events,
The landscape image information output unit has event-specific landscape image information output means for outputting the event-specific landscape image information corresponding to the acquired event identification information to the mobile terminal device,
It has a server side event identification information acquisition unit that acquires event identification information from a mobile terminal device,
The mobile terminal device
A mobile terminal side event identification information acquisition unit for acquiring event identification information;
An event identification information transmitter that transmits the acquired event identification information to the landscape server device;
Have
The street viewer system according to claim 1, wherein the landscape image information acquisition unit includes an event-specific landscape image information acquisition unit that acquires event-specific landscape image information according to the transmitted event identification information.
The street viewer system according to any one of claims 1 to 3, wherein the mobile terminal device is replaced with a mobile terminal device.
The landscape server device according to any one of claims 1 to 4.
The portable terminal device as described in any one of Claims 1-3.
The mobile body-mounted terminal device according to claim 4.
In the operation method of the street viewer system, which includes a landscape server device that holds landscape image information that is an image showing a landscape in association with viewpoint information that includes positional information and postscript line-of-sight information associated therewith, and a mobile terminal device There,
The operation method of the landscape server device is
A viewpoint information acquisition step of acquiring viewpoint information from the mobile terminal device;
A landscape image information output step of transmitting landscape image information associated with the acquired viewpoint information to the mobile terminal device;
Have
The operation method of the mobile terminal device is as follows:
A position information acquisition step of acquiring position information that is information of a position where the mobile terminal device currently exists;
A line-of-sight information acquisition step of acquiring line-of-sight information, which is information indicating the line-of-sight direction or / and viewpoint height information in the acquired position information;
A viewpoint information output step of outputting viewpoint information that associates the acquired position information and line-of-sight information to the landscape server device;
A landscape image information acquisition step of acquiring landscape image information from the landscape server device according to the output viewpoint information;
A landscape image information output step for outputting the acquired landscape image information;
A method of operating a street viewer system.
In an operation program of a street viewer system comprising a landscape server device that holds landscape image information that is an image indicating a landscape in association with viewpoint information that includes position information and postscript line-of-sight information associated therewith, and a mobile terminal device There,
The operation program of the landscape server device is
A viewpoint information acquisition step of acquiring viewpoint information from the mobile terminal device;
A landscape image information output step for transmitting the landscape image information associated with the acquired viewpoint information to the mobile terminal device, and an operation program for the landscape server device that is recorded on the landscape server device that is a computer so as to be executable.
The operation program of the mobile terminal device is
A position information acquisition step of acquiring position information that is information of a position where the mobile terminal device currently exists;
A line-of-sight information acquisition step of acquiring line-of-sight information, which is information indicating the line-of-sight direction or / and viewpoint height information in the acquired position information;
A viewpoint information output step of outputting viewpoint information that associates the acquired position information and line-of-sight information to the landscape server device;
A landscape image information acquisition step of acquiring landscape image information from the landscape server device according to the output viewpoint information;
A landscape image information output step for outputting the acquired landscape image information;
Is an operation program of a street viewer system, which is a program of a mobile terminal device that is recorded so as to be readable and executable on a mobile terminal device that is a computer.
The operation | movement method of the landscape server apparatus of Claim 8.
The operation method of the mobile terminal device according to claim 8.
The operation | movement program of the landscape server apparatus of Claim 9.
The operation program of the portable terminal device according to claim 9.
The line-of-sight information acquisition unit
A line-of-sight sensor that detects the direction of east, west, south, north, and north
Gaze direction acquisition means for continuously acquiring the gaze direction from the gaze direction sensor;
Gaze direction holding means for fixing and holding the gaze direction being acquired continuously at the acquisition timing of the holding command;
Holding command output means for outputting a holding command based on an input from the user to the line-of-sight holding means;
Holding cancellation command output means for canceling the holding of the line-of-sight direction held in the line-of-sight direction holding means based on the input from the user,
The street viewer system according to any one of claims 1 to 3, wherein the visual line direction included in the visual line information is configured to be a retained visual line direction.
The portable terminal device has a camera and a display,
The street viewer system according to claim 14, wherein the line-of-sight direction sensor acquires a line of sight of a scene projected on a display through a camera as a line-of-sight direction.
In response to the output of the holding command of the holding command output means, the camera is a holding command shooting unit for shooting a landscape still image,
The street viewer system according to claim 15, further comprising:
The camera has a zoom-in / zoom-out function that can be operated by the user,
It has a magnification acquisition unit that acquires the magnification of the landscape still image at the time of shooting in the shooting unit at the time of holding instruction,
The street viewer system according to claim 16, wherein the landscape image information output unit further includes a landscape image output means for outputting a landscape image having the same magnification as that of the photographed still image on the display.
The street viewer system according to claim 17, further comprising an alternate output unit capable of alternately outputting a landscape still image photographed on the display and a landscape image of the same magnification.
The street viewer system according to claim 17 or 18, further comprising a superimposition output unit capable of superimposing and outputting a landscape still image photographed on the display and a landscape image at the same magnification.
In the operation method of the street viewer system, which includes a landscape server device that holds landscape image information that is an image showing a landscape in association with viewpoint information that includes positional information and postscript line-of-sight information associated therewith, and a mobile terminal device There,
The operation method of the landscape server device is
A viewpoint information acquisition step of acquiring viewpoint information from the mobile terminal device;
A landscape image information output step of transmitting landscape image information associated with the acquired viewpoint information to the mobile terminal device;
Have
The operation method of the mobile terminal device is as follows:
A position information acquisition step of acquiring position information that is information of a position where the mobile terminal device currently exists;
A line-of-sight information acquisition step of acquiring line-of-sight information, which is information indicating the line-of-sight direction or / and viewpoint height information in the acquired position information;
A viewpoint information output step of outputting viewpoint information that associates the acquired position information and line-of-sight information to the landscape server device;
A landscape image information acquisition step of acquiring landscape image information from the landscape server device according to the output viewpoint information;
A landscape image information output step for outputting the acquired landscape image information;
Have
The line-of-sight information acquisition step includes
Gaze direction sensor sub-step for continuously detecting the direction of east, west, south, north, and south, and
A line-of-sight direction acquisition sub-step for continuously acquiring a line-of-sight direction detected in the line-of-sight direction sensor sub-step;
A line-of-sight direction holding sub-step for fixing and holding the line-of-sight direction being acquired continuously at the line-of-sight direction holding means at the acquisition timing of the holding instruction based on the input from the user;
Holding cancellation command output sub-step for canceling the holding of the line-of-sight direction held in the line-of-sight holding means based on the input from the user,
An operation method of a street viewer system configured such that the line-of-sight direction included in the line-of-sight information is a retained line-of-sight direction.
In an operation program of a street viewer system comprising a landscape server device that holds landscape image information that is an image indicating a landscape in association with viewpoint information that includes position information and postscript line-of-sight information associated therewith, and a mobile terminal device There,
The operation program of the landscape server device, which is a computer,
A viewpoint information acquisition step of acquiring viewpoint information from the mobile terminal device;
A landscape image information output step of transmitting landscape image information associated with the acquired viewpoint information to the mobile terminal device;
Is read and executed on the landscape server device, which is a computer,
The operation program of the portable terminal device which is a computer is
A position information acquisition step of acquiring position information that is information of a position where the mobile terminal device currently exists;
A line-of-sight information acquisition step of acquiring line-of-sight information, which is information indicating the line-of-sight direction or / and viewpoint height information in the acquired position information;
A viewpoint information output step of outputting viewpoint information that associates the acquired position information and line-of-sight information to the landscape server device;
A landscape image information acquisition step of acquiring landscape image information from the landscape server device according to the output viewpoint information;
A landscape image information output step for outputting the acquired landscape image information;
Is described in such a manner that it can be read and executed by a portable terminal device that is a computer,
The line-of-sight information acquisition step of the operation program of the mobile terminal device includes:
Gaze direction sensor sub-step for continuously detecting the direction of east, west, south, north, and south, and
A line-of-sight direction acquisition sub-step for continuously acquiring a line-of-sight direction detected in the line-of-sight direction sensor sub-step;
A line-of-sight direction holding sub-step for fixing and holding the line-of-sight direction being acquired continuously at the line-of-sight direction holding means at the acquisition timing of the holding instruction based on the input from the user;
Holding cancellation command output sub-step for canceling the holding of the line-of-sight direction held in the line-of-sight holding means based on the input from the user,
An operation program of a street viewer system that is processed so that the line-of-sight direction included in the line-of-sight information is the line-of-sight direction held.
The operation program of the landscape server device according to claim 21.
The operation program of the portable terminal device according to claim 21.