JP7378510B2 - street viewer system - Google Patents

Description

The present invention relates to a street viewer system, and particularly to a street viewer system suitable for displaying landscape images showing current or past landscapes and cityscapes (townscapes).

Conventionally, by using the street view function of Google Maps, it is possible to display current landscapes and townscapes from a computer browser. Recently, a ``time machine function'' has been added to Google Maps' Street View function that allows you to see past scenery. When you display Google Maps on your computer and start Street View, the locations where Street View can be displayed will be displayed on the map, and past Street Views of that location can be displayed.

Furthermore, Japanese Patent No. 5363971 (Patent Document 1) is cited as a system that reproduces past or future scenery at a point specified by a user with a mobile terminal. This Patent Document 1 is a system that reproduces the scenery at an arbitrary current time. This landscape reproduction system is a landscape reproduction system consisting of a user mobile terminal and a landscape reproduction server that are connected to each other via a network, and a reproduction image request receiving unit 801 receives the position, orientation, and inclination of the user mobile terminal, A reproduction image request including the instructed re-current point is received, a re-creation terrain extraction unit 804 and a re-enactment object extraction unit 807 extract three-dimensional shape information of the terrain and object corresponding to the re-present point, and a two-dimensional projection image generation unit 809 , a configuration is shown in which a reproduced image is generated in which the terrain object and the physical object are arranged in a virtual three-dimensional space, and the reproduced image is sent back to the user's mobile terminal from the reproduced image transmitter 811.

Patent No. 5363971

The street view function of Google Maps mentioned above uses a computer browser to view current or past landscapes from a map displayed on the display screen, without having to go to the actual tourist spot or location. It provides a virtual experience that allows tourists to see things, but it has not been sufficient for tourist destinations and the region, as they cannot expect economic effects unless people actually come.

In Patent Document 1, the scenery is reproduced based on the position, orientation, and inclination of the user's mobile terminal, so when a child uses the user's mobile terminal in a social studies field trip, the child's perspective and the user's mobile terminal are There was a problem in that there was a misalignment with the viewpoint, making it difficult to use. In addition, since the device is configured to display landscape images with the terminal device's camera in the line of sight, you should always take a picture of the landscape in the area you want to see the reproduced image (normally the camera is on the back of the display, so hold the display up to your eyes This was inconvenient because it was not possible to see the reproduced image unless you looked at the screen of your mobile device while in a position where you were in a position to continue.

An object of the present invention is to provide a smart viewer system that allows a traveler to actually go to a tourist spot and view landscape image information along with additional information according to the traveler's viewpoint information.

Another object of the present invention is to improve the operability by taking into consideration the landscape images to be displayed and the mode of use of the mobile terminal device, so that landscape information can be viewed at a desired viewpoint height when going to the city on a social studies field trip or the like. Our goal is to provide a street viewer system that allows you to

In order to solve the above problems, the present invention provides the following street viewer system. That is, as a first invention, there is provided a street viewer system comprising a landscape server device and a mobile terminal device, wherein the landscape server device is associated with viewpoint information consisting of position information and line-of-sight information (described below) associated therewith. a landscape image information holding unit that holds landscape image information that is an image showing a past landscape obtained by combining a current landscape that is a live-action image and a past landscape by a computer; and a landscape image information storage unit that acquires viewpoint information from a mobile terminal device. a viewpoint information acquisition unit; a server-side time information acquisition unit that acquires time information that is information indicating a past time from a mobile terminal device; and a landscape that transmits landscape image information associated with the acquired viewpoint information to the mobile terminal device. Further, the landscape image information holding section has a period-specific landscape image information holding means for holding period-specific landscape image information that is landscape image information for each of a plurality of past periods, The landscape image information output unit has a period-specific landscape image information output means for outputting period-specific landscape image information of the period corresponding to the acquired period information to the mobile terminal device, and the mobile terminal device is configured to output the period-specific landscape image information of the period corresponding to the acquired period information to the mobile terminal device, a position information acquisition unit that acquires position information, which is information about the position where the camera is located, and a line of sight that acquires line-of-sight information, which is information indicating the line-of-sight direction based on the acquired position information and/or viewpoint height information that can be changed by a change input. an information acquisition unit, a mobile terminal-side timing information acquisition unit that acquires timing information, a viewpoint information output unit that outputs viewpoint information in which the acquired position information and line of sight information are associated to the landscape server device, and a viewpoint information output unit that a landscape image information acquisition unit that has a function of inputting changes and acquires landscape image information from a landscape server device according to the output viewpoint information; a landscape image information output unit that outputs the acquired landscape image information; and a landscape image information output unit that outputs the acquired landscape image information. a period information transmitting section that transmits the period information to the landscape server device; ( corresponds to claim 1).

In addition to the above features, the landscape server device has a landscape image information holding unit in which the landscape image information holding unit has period-specific landscape image information holding means for holding period-specific landscape image information that is landscape image information for each of a plurality of past periods, The information output unit has a period-specific landscape image information output means for outputting period-specific landscape image information of a period corresponding to the acquired period information to the mobile terminal device, and also outputs period-specific landscape image information from the mobile terminal device that is information indicating a past period. The mobile terminal device has a server-side timing information acquisition unit that acquires information, and the mobile terminal device includes a mobile terminal-side timing information acquisition unit that acquires the timing information, and a timing information transmission unit that transmits the acquired timing information to the landscape server. The landscape image information acquisition unit includes a period-specific landscape image information acquisition means for acquiring period-specific landscape image information according to the transmitted period information (corresponding to claim 2).

In addition to the above characteristics, the landscape server device has an event-specific landscape image information holding unit in which the landscape image information holding unit holds event-specific landscape image information that is landscape image information of historical and/or seasonal events. The landscape image information output unit includes event-specific landscape image information output means for outputting event-specific landscape image information corresponding to the acquired event identification information to the mobile terminal device, and acquires the event identification information from the mobile terminal device. The mobile terminal device has a server-side event identification information acquisition unit, and the mobile terminal device includes a mobile terminal-side event identification information acquisition unit that acquires event identification information, and an event identification information transmission unit that transmits the acquired event identification information to the landscape server device. The landscape image information acquisition unit includes event-specific landscape image information acquisition means for acquiring event-specific landscape image information according to the transmitted event identification information (corresponding to claim 3).

In addition to the above-mentioned features, a street viewer system (corresponding to claim 4) in which a terminal device mounted on a mobile object is used instead of a mobile terminal device.

Landscape server device according to any one of claims 1 to 4 (corresponding to claim 5)

A mobile terminal device according to any one of claims 1 to 3 (corresponding to claim 6).

The mobile body-mounted terminal device according to claim 4 (corresponding to claim 7).

As a second invention, there is provided a street comprising a landscape server device that holds landscape image information, which is an image showing a landscape in association with viewpoint information consisting of position information and line-of-sight information related thereto, and a mobile terminal device. The operating method of the viewer system and the operating method of the landscape server device include a viewpoint information acquisition step of acquiring viewpoint information from a mobile terminal device, and transmitting landscape image information associated with the acquired viewpoint information to the mobile terminal device. A landscape image information output step, and the operating method of the mobile terminal device includes a location information acquisition step of acquiring location information that is information on the current location of the mobile terminal device, and a location information acquisition step of acquiring location information that is information on the location where the mobile terminal device currently exists. a line-of-sight information acquisition step of acquiring line-of-sight information that is information indicating the line-of-sight direction and/or viewpoint height information; a viewpoint information output step of outputting viewpoint information in which the acquired position information and line-of-sight information are associated to the landscape server device; A method of operating a street viewer system comprising: a landscape image information acquisition step of acquiring landscape image information from a landscape server device according to output viewpoint information; and a landscape image information output step of outputting the acquired landscape image information 8 compatible).

As a third invention, there is provided a street comprising a landscape server device that holds landscape image information, which is an image showing a landscape in association with viewpoint information consisting of position information and line-of-sight information related thereto, and a mobile terminal device. The operation program of the viewer system and the operation program of the landscape server device include a viewpoint information acquisition step of acquiring viewpoint information from a mobile terminal device, and a step of transmitting landscape image information associated with the acquired viewpoint information to the mobile terminal device. The landscape image information output step is an operation program of the landscape server device which is readable and executable recorded on the landscape server device which is a computer, and the operation program of the mobile terminal device is an operation program of the landscape image information output step, and the operation program of the mobile terminal device is an operation program of the landscape image information output step. a position information acquisition step of acquiring position information, a line-of-sight information acquisition step of acquiring line-of-sight information, which is information indicating the line-of-sight direction and/or viewpoint height information in the acquired position information, and the acquired position information and line-of-sight information. a viewpoint information output step of outputting viewpoint information associated with the above to the landscape server device; a landscape image information acquisition step of acquiring landscape image information from the landscape server device according to the output viewpoint information; and outputting the acquired landscape image information. An operation program for a street viewer system, which is a program for a mobile terminal device, in which a step of outputting landscape image information is readable and executable on a mobile terminal device, which is a computer.

An operating method for the landscape server device according to claim 8 (corresponding to claim 10).

A method for operating a mobile terminal device according to claim 8 (corresponding to claim 11).

An operating program for a landscape server device according to claim 9 (corresponding to claim 12).

An operation program for a mobile terminal device according to claim 9 (corresponding to claim 13).

In addition to the above-mentioned characteristics, the line-of-sight information acquisition unit includes a line-of-sight direction sensor that detects directions of east, west, north, south, up and down, a line-of-sight direction acquisition means that continuously acquires the line-of-sight direction from the line-of-sight direction sensor, and a line-of-sight direction acquisition unit that continuously acquires the line-of-sight direction from the line-of-sight direction sensor. A line-of-sight direction holding means for fixing and holding the line-of-sight direction at the acquisition timing of the holding command; a holding command output means for outputting a holding command based on an input from a user to the line-of-sight direction holding means; a retention cancellation command output means for dissolving the retention of the gaze direction based on input from the user, and configured such that the gaze direction included in the gaze information is the retained gaze direction. The street viewer system according to any one of the above (corresponding to claim 14).

In addition to the above characteristics, the mobile terminal device has a camera and a display, and the line of sight direction sensor acquires the line of sight of the scenery displayed on the display through the camera as the line of sight direction (corresponding to claim 15). .

In addition to the above-mentioned features, the camera further includes a holding command photography unit that photographs a landscape still image in response to the holding command output from the holding command output means (corresponding to claim 16).

In addition to the above-mentioned features, the camera has a zoom-in/zoom-out function that can be operated by the user, has a magnification acquisition unit that acquires the magnification of a landscape still image at the time of photography by the photography unit when a hold command is issued, and landscape image information. The output unit further includes a same magnification landscape image output means for outputting a landscape image of the same magnification as the photographed still image to the display (corresponding to claim 17).

In addition to the above characteristics, the apparatus further includes an alternating output section capable of alternately outputting a landscape still image photographed on the display and a landscape image of the same magnification (corresponding to claim 18).

In addition to the above features, the apparatus further includes a superimposition output section capable of superimposing and outputting a landscape still image photographed on the display and a landscape image of the same magnification (corresponding to claim 19).

As a fourth invention, a street comprising a landscape server device that holds landscape image information, which is an image showing a landscape in association with viewpoint information consisting of position information and line-of-sight information related to the latter, and a mobile terminal device. The operating method of the viewer system and the operating method of the landscape server device include a viewpoint information acquisition step of acquiring viewpoint information from a mobile terminal device, and transmitting landscape image information associated with the acquired viewpoint information to the mobile terminal device. A landscape image information output step, and the operating method of the mobile terminal device includes a location information acquisition step of acquiring location information that is information on the current location of the mobile terminal device, and a location information acquisition step of acquiring location information that is information on the location where the mobile terminal device currently exists. a line-of-sight information acquisition step of acquiring line-of-sight information that is information indicating the line-of-sight direction and/or viewpoint height information; a viewpoint information output step of outputting viewpoint information in which the acquired position information and line-of-sight information are associated to the landscape server device; The method includes a landscape image information acquisition step of acquiring landscape image information from a landscape server device according to the output viewpoint information, and a landscape image information output step of outputting the acquired landscape image information, and the line of sight information acquisition step includes: A line-of-sight direction sensor substep that continuously detects the direction of east, west, north, south, up, and down; a line-of-sight direction acquisition substep that continuously acquires the line-of-sight direction detected by the line-of-sight direction sensor substep; A gaze direction holding substep in which the direction is fixed and held in the gaze direction holding means at the acquisition timing of a holding command based on an input from the user, and an input from the user to maintain the gaze direction held in the gaze direction holding means. and a substep of outputting a holding cancellation command for canceling the hold based on the following: A method for operating a street viewer system configured such that the line-of-sight direction included in the line-of-sight information is the held line-of-sight direction (Claim 20) correspondence).

As a fifth invention, a street consisting of a landscape server device that holds landscape image information, which is an image indicating a landscape in association with viewpoint information consisting of position information and line-of-sight information related to the latter, and a mobile terminal device. The operation program of the viewer system and the operation program of the landscape server device, which is a computer, includes a viewpoint information acquisition step of acquiring viewpoint information from a mobile terminal device, and a viewpoint information acquisition step of acquiring viewpoint information from a mobile terminal device. The step of outputting landscape image information to be transmitted to the landscape server device, which is a computer, is readable and executable, and the operation program of the mobile terminal device, which is a computer, is information on the current location of the mobile terminal device. a position information acquisition step for acquiring position information; a line-of-sight information acquisition step for acquiring line-of-sight information, which is information indicating the line-of-sight direction and/or viewpoint height information in the acquired position information; and a line-of-sight information acquisition step for acquiring position information and line-of-sight information. a viewpoint information output step of outputting associated viewpoint information to a landscape server device; a landscape image information acquisition step of acquiring landscape image information from the landscape server device according to the output viewpoint information; and a landscape image information outputting step of outputting the acquired landscape image information. The image information output step is written to be readable and executable by a mobile terminal device, which is a computer, and the line of sight information acquisition step of the operation program of the mobile terminal device includes a line of sight direction sensor sub that continuously detects directions of east, west, north, south, up, and down. step, a line-of-sight direction acquisition substep of continuously acquiring the line-of-sight direction detected by the line-of-sight direction sensor substep, and a command to hold the line-of-sight direction being continuously acquired to the line-of-sight direction holding means based on input from the user. a gaze direction holding substep that fixes and holds the gaze direction at the acquisition timing; and a holding cancellation command output substep that releases the gaze direction held in the gaze direction holding means based on an input from the user. An operating program for a street viewer system (corresponding to claim 21), wherein the line-of-sight direction included in the line-of-sight information is processed to be the maintained line-of-sight direction.

An operating program for a landscape server device according to claim 21 (corresponding to claim 22).

An operating program for a mobile terminal device according to claim 21 (corresponding to claim 23).

According to the above-described configuration, a traveler can actually go to a tourist spot and view landscape image information according to the traveler's viewpoint information together with additional information. In addition, users can go to the city for social studies field trips and view landscape information from a desired viewpoint height.

Diagram for explaining an overview of the smart viewer system of Embodiment 1 Functional block diagram of smart viewer system of Embodiment 1 Functional block diagram of a configuration that does not include a display of the smart viewer system of Embodiment 1 A diagram showing an example of display types in Embodiment 1. A diagram showing an example of a display (tablet) in the smart viewer system of Embodiment 1. A diagram showing another example of the display (smartphone) in the smart viewer system of Embodiment 1. A diagram showing another example (head mounted display) of the smart viewer system of Embodiment 1. A diagram showing another example (car navigation system) of the smart viewer system of Embodiment 1. A diagram showing an example of viewpoint information acquisition means in the smart viewer system of Embodiment 1 Diagram for explaining acquisition of viewpoint information in the smart viewer system of Embodiment 1 A diagram showing an example of a landscape image acquisition interface screen in the smart viewer system of Embodiment 1. A diagram showing an example of the hardware configuration of a landscape server device in the smart viewer system of Embodiment 1. A diagram showing an example of the hardware configuration of a mobile terminal device in the smart viewer system of Embodiment 1. Processing flowchart of the smart viewer system of Embodiment 1 Processing flowchart of the smart viewer system of Embodiment 1 Diagram for explaining an overview of the smart viewer system of Embodiment 2 Functional block diagram of smart viewer system of Embodiment 2 Functional block diagram of a configuration that does not include a display of the smart viewer system of Embodiment 2 A diagram showing an example of a landscape image acquisition interface screen in the smart viewer system of Embodiment 2. A diagram showing an example of the hardware configuration of the landscape server device in the smart viewer system of Embodiment 2. A diagram showing an example of the hardware configuration of a mobile terminal device in the smart viewer system of Embodiment 2. Processing flowchart of the smart viewer system of Embodiment 2 Processing flowchart of the smart viewer system of Embodiment 2 Diagram for explaining an overview of the smart viewer system of Embodiment 1 Functional block diagram of smart viewer system of Embodiment 3 Functional block diagram of a configuration that does not include a display of the smart viewer system of Embodiment 3 A diagram showing an example of the hardware configuration of the landscape server device of the smart viewer system of Embodiment 3. A diagram showing an example of the hardware configuration of a mobile terminal device of the smart viewer system of Embodiment 3. Processing flowchart of the smart viewer system of Embodiment 3 Processing flowchart of the smart viewer system of Embodiment 3 Functional block diagram of smart viewer system of Embodiment 4 A diagram showing an example of the hardware configuration of a mobile terminal device of the smart viewer system of Embodiment 4. Processing flowchart of smart viewer system of embodiment 4 Diagram for explaining the zoom-in/zoom-out function of the camera in the smart viewer system of Embodiment 5 A diagram showing an example of a cap equipped with a wearable camera in the smart viewer system of Embodiment 5.

Embodiments of the present invention will be described below with reference to the drawings. Note that the present invention is not limited to these embodiments in any way, and may be implemented in various forms without departing from the spirit thereof.

Note that the functional blocks of the present device described below can be realized as a combination of hardware and software. Specifically, if a computer is used, the CPU (central processing unit), main memory, system bus, or secondary storage device (hard disk drive, non-volatile memory, storage media such as CDs and DVDs, and such media) hardware components such as input devices used for information input, display devices, and other external peripheral devices, such as tablets, smartphones, VR head-mounted displays, glasses-type displays (monocular, see-through, etc.), Displays with eyeglass-type cameras (cameras are built-in types, clip-on types, etc.), wearable cameras (attached to clothing/embedded, clip-on types, built-in caps, etc.), interfaces for external peripheral devices, communication interfaces, location information interfaces , line of sight input interface, height input interface, touch input interface, software keyboard interface, driver programs and other application programs for controlling these hardware, and user interface applications. Then, through the arithmetic processing of the CPU according to the program developed on the main memory, the data input from input devices and other interfaces, etc. and held on the memory and hard disk are processed and accumulated, and the data etc. Instructions for controlling software are generated. Alternatively, the functional blocks of this device may be realized by dedicated hardware. Furthermore, the present device is not limited to being configured by one piece of hardware or software, but may be configured by a combination of multiple pieces of hardware or software, or may include a server device via a network.

Further, the present invention can be realized not only as an apparatus but also as a method. Moreover, a part of such an invention can be configured as software. Furthermore, a program used to cause a computer to execute such software and a recording medium on which the program is fixed are naturally included in the technical scope of the present invention (the same applies throughout this specification).
<Embodiment 1>
<Summary>

As shown in FIG. 1, in the first embodiment, when a traveler actually travels to a tourist spot or a student actually goes to a town for a social studies tour, data is sent from a landscape server device (not shown). A street view (street view) is created based on viewpoint information, which is composed of location information, gaze direction of the traveler, and viewpoint height information. The present invention relates to a street viewer system that is suitable for displaying landscapes (scenery) along with guide information. This street viewer system realizes a system that can display a street view according to a desired viewpoint (for example, the viewpoint of an adult or a child) using viewpoint height information.
In the example of FIG. 1, a traveler carries a mobile terminal device and displays the cityscape (street view) of Kyoto, a tourist destination, with guidance information added. Alternatively, as shown in Fig. 24, when a traveler carries a mobile terminal device, goes shopping in Ginza, and points the mobile terminal device at a department store on Ginza 4-chome, the floor information of that department store is displayed with added information. It shows.
<Embodiment 1 Configuration>

As shown in FIG. 2, the street viewer system of the first embodiment is a street viewer system that includes a landscape server device 0200 and a mobile terminal device 0210, and the landscape server device 0200 includes a landscape image information storage unit 0201 and a landscape image information storage unit 0201. , a viewpoint information acquisition unit 0202, and a landscape image information output unit 0203, and the mobile terminal device 0210 has a position information acquisition unit 0211, a line of sight information acquisition unit 0212, a viewpoint information output unit 0213, and a landscape image information output unit 0210. It has an acquisition section 0214 and a landscape image information output section 0215.
<Embodiment 1 Description of configuration>
<Embodiment 1 Landscape server device>

This landscape server device is, for example, installed in a server room or the like in a room in a building, or is stored as big data in a data center. It is connected to a mobile terminal device via a communication network (including the Internet, broadband network, mobile network, etc.), acquires viewpoint information from the mobile terminal device, and images a landscape image corresponding to the acquired viewpoint information. Send the information to the mobile terminal device. The configuration of the landscape server device is as shown in FIG. 2, and is as follows.
<Embodiment 1 Landscape image information holding unit>

The "scenery image information holding unit" 0201 holds scenery image information, which is an image showing a scenery in association with viewpoint information consisting of position information and line-of-sight information related thereto, which will be described later. Here, the landscape image information held includes landscape images such as landscapes and cityscapes, location information, and associated line-of-sight information (line-of-sight information including line-of-sight direction information and line-of-sight height information) of tourists, social studies visitors, etc. ) is saved for each viewpoint information. Additionally, if it is difficult to save landscape image information corresponding to all viewpoint information in the server, it is also possible to create a basic landscape image model and instantaneously create it by simulation based on the viewpoint information. . Landscape image information is literally image information of a landscape, but the images include perspective images that look through the inside of buildings (computer-generated images), perspective images that see through underground malls (computer-generated images), and explanation images. It may include characters, symbols, codes, etc., and may also include audio, music, sound effects, explanatory audio, and vibrations (commands to make the mobile terminal vibrate). The explanation may include a chronology, a map showing the current location, an old map showing the current location, photos of related artifacts, photos of related cultural heritage, etc. The landscape image may be an image synthesized by a computer or a real image. A high-definition satellite image associated with three-dimensional position information, such as Google Earth (registered trademark), can be used as a mechanism for holding an image showing a landscape in association with viewpoint information. By using this, you can see both the front and back sides of buildings using satellite photographs, and you can arbitrarily obtain landscape images associated with viewpoint information. In addition, the landscape image information that is maintained can include current landscapes and past landscapes, but regarding the current landscape, landscape images are constantly located from wearable cameras that people keep around, cameras installed in cars, etc. It can be configured to be collected in association with information (collected from car navigation systems, etc.) and constantly updated. Therefore, the collected images can be held in association with time, such as year, month, day, morning, noon, evening, and night. Therefore, the landscape image can be sent back to the mobile terminal device according to the time at which viewpoint information is acquired.
<Embodiment 1 Viewpoint information acquisition unit>

The "viewpoint information acquisition unit" 0202 acquires viewpoint information from the mobile terminal device 0210. As mentioned above, viewpoint information consists of position information and line-of-sight information associated with this information, and includes position information obtained from GPS signals, landscape images, etc., and the line-of-sight direction of travelers, etc. associated with this position information. Obtain viewpoint height information indicating viewpoint height information from a mobile terminal device. Furthermore, a configuration in which the landscape server device 0200 detects the position of the mobile terminal device 0210 and acquires the line-of-sight direction and viewpoint height information using a location information service system is also conceivable. The viewpoint information does not necessarily include viewing direction and viewpoint height information, and viewpoint height information may be selectively employed. The viewpoint information may include a landscape image photographed by a camera of a mobile terminal device, an in-plant photographed image, and the like. This information can be used as reference information for determining viewpoint information by comparing this information with the scenery image associated with the viewpoint information held in the scenery server device.
<Embodiment 1 Landscape image information output unit>

The “landscape image information output unit” 0203 transmits landscape image information associated with the acquired viewpoint information to the mobile terminal device 0210. In other words, the landscape image information to be transmitted is a landscape image of the direction and height of the line of sight from the actual position of the tourist or social studies student when he or she actually goes out into the city.
<Embodiment 1 Mobile terminal device>

The mobile terminal device of this embodiment includes a tablet with a communication function, a smartphone, a wearable terminal such as a glasses type or a wristwatch type, a VR head mount system, and the like. Since these tablets include a display, it is possible to output landscape image information from a mobile terminal device to the outside and display it on various displays, as shown in FIG.
As shown in Figure 4, the types of displays include a tablet screen (see Figure 5), a smartphone screen (see Figure 6), a display with built-in glasses, a VR head-mounted display (see Figure 7), and a mobile window glass (for example, , thin sheets attached to the window glass surface of trains such as Shinkansen trains, or displays displayed on the window glass surface with a head-up display), in-mobile displays (display devices such as car navigation systems (see Figure 8), and car windshield surfaces) head-up displays, etc.), telescope-shaped displays, and flexible sheet displays. Alternatively, the communication function may be realized by a small device, a separate configuration may be adopted, and the communication section and the display section may be configured separately.

The viewpoint information acquisition unit is performed by a position information acquisition unit 0211 and a line of sight information acquisition unit 0121.

As shown in Figure 9, it may be realized by attaching various sensors to the display as described above, or by having tourists and social studies visitors wear various sensors, or by incorporating various sensors into clothing. good. That is, as shown in FIG. 9, the viewpoint information acquisition means may include sensor-equipped glasses, sensor-equipped tablet terminals, sensor-equipped smartphones, sensor-equipped smart watches, sensor-equipped clothing, and the like.

Further, FIG. 10 shows a specific example of a method for acquiring viewpoint information. That is, it shows the form of the line of sight direction, position information, and height of the viewpoint. Below, in explaining the configuration of each part of the mobile terminal device 0210, relevant parts will be explained.
<Embodiment 1 Location information acquisition unit>

The “location information acquisition unit” 0211 acquires location information that is information on the current location of the mobile terminal device. The location information acquisition unit 0211 acquires location information in terms of longitude and latitude, as shown in FIG. Specifically, it can be acquired from sensors such as an acceleration sensor and a gyro sensor, GPS signals, road surface tag information, landscape images, etc. It is also possible to obtain location information from a system connected to surveillance cameras installed on the street. This is done by using a surveillance camera to identify a person based on their facial features, then extracting and identifying the mobile device owned by that person from a database, determining the location that can be grasped from the surveillance camera image, and identifying the mobile device. It sends location information to the device. If multiple mobile terminal devices are extracted at this time, it is possible to determine which mobile terminal device is being carried by the person using the base station ID of the mobile terminal device that covers the area of the surveillance camera. Mobile terminal devices can be narrowed down. It is also easier to identify the mobile terminal device by searching for the usage registration information of this system.
<Embodiment 1 Gaze information acquisition unit>

The “line-of-sight information acquisition unit” 0212 acquires line-of-sight information, which is information indicating the line-of-sight direction and/or viewpoint height information at the acquired position information. The viewing direction can be obtained from a gyro sensor, a compass, a landscape image, etc., as shown in FIG. Furthermore, as shown in FIG. 10, the viewpoint height information can be obtained from condition settings input by the user of the mobile terminal device, atmospheric pressure information, GPS signals, road surface tag information, landscape images, and the like. The line of sight information may be acquired from a wearable camera.
<Embodiment 1 Viewpoint information output unit>

The "viewpoint information output unit" 0213 outputs viewpoint information in which the acquired position information and line of sight information are associated with each other to the landscape server device. Here, since the scenery server device 0200 and the mobile terminal device 0210 are connected via a network (including the Internet, broadband, and mobile network), information is exchanged by wireless communication. Furthermore, if there is a wireless WIFI access point in the city, the information can be output to the landscape server device 0200 via the wireless WIFI access point. When using a mobile network such as a mobile phone network, direct communication can be performed with a 3G-compatible mobile terminal device or an LTE-compatible mobile terminal device. Note that although the viewpoint information has been acquired above based on the actual location of the mobile terminal, an option may be provided to set a virtual viewpoint viewed from above. In this case, landscape image information including a virtual figure of the user standing at the location of the mobile terminal in the landscape image from the air is transmitted from the landscape server device, and the configuration is such that this can be viewed. is possible.
Furthermore, the mobile terminal is not limited to something that the user carries, but may be something like a drone. The configuration is configured so that the line of sight of the camera mounted on the drone is acquired as viewpoint information.
<Embodiment 1 Landscape image information acquisition unit>

The “landscape image information acquisition unit” 0214 acquires landscape image information from the landscape server device according to the output viewpoint information. Here, the landscape image acquisition interface screen of the mobile terminal device 0210 includes various selection menus as shown in FIG. For example, there are selection menus such as ``Explanation of buildings, landmarks, and scenery'' and ``by morning, noon, and night,'' and when the user of the mobile terminal device selects ``Explanation of buildings, landmarks, and scenery,'' as shown in Figure 1. Explanations (including guide information) of buildings, landmarks, and landscapes are displayed together with landscape image information. Furthermore, when the user of the mobile terminal device selects the selection menu of "Morning, Noon, and Night", landscape image information corresponding to the time of day will be displayed. Further, as the weather, it may be possible to select "sunny", "rainy", "cloudy", "snow", "storm" (accumulated in the landscape server device according to these), etc. It is also conceivable that these can be selected with a slide bar, so that they can be selected continuously from "sunny" to "cloudy", from "cloudy" to "rainy", and from "rainy" to "stormy" (continuous). changes are automatically generated within the landscape server device). By acquiring local weather information from the Internet etc. and sending it to the landscape server device, the landscape server device can be configured to acquire and display landscape image information tailored to the weather, or to display the opposite weather condition. It is also conceivable to configure and display the information. The scenery image information is configured to include an explanation of the scenery. As the selection menu, a "child mode" (with explanations for children) may be set, or the selection menu may be in the form of a dial. If you select "Children mode", "Elementary school student" (Explanation for elementary school students is provided, and the viewpoint height is set to the average viewpoint height of elementary school students), "Junior high school student" (Explanation for junior high school students is provided, and the viewpoint height is set to the average viewpoint height of elementary school students. (The viewpoint height is set to the average viewpoint height of junior high school students), "High school student" (Explanation for high school students is provided, and the viewpoint height is set to the average viewpoint height of high school students), "College student" (For university students) A pull-down menu may be displayed, such as "The viewpoint height is set to the average viewpoint height of university students", or the user may directly enter his or her own height. The mobile terminal device may automatically calculate the height of the viewpoint based on the user's height. When performing one-touch input using the pull-down menu, if the height of the viewpoint varies from person to person and the display screen is difficult to see, change the height of the viewpoint using the "+" plus button or the "-" minus button. It's okay.
<Embodiment 1 Landscape image information output unit>

“Landscape image information output unit” 0215 outputs the acquired landscape image information. Here, if the mobile terminal device 0210 includes a display, landscape image information that matches the viewpoint information of the traveler or social studies visitor is displayed on the display screen of the mobile terminal device in response to the selection of the landscape image acquisition interface. be done. Furthermore, if the mobile terminal device does not include a display, landscape image information is displayed on a display 0316 connected to the mobile terminal device 0310, as shown in FIG. In addition to this, the output destination may be a built-in display of another mobile terminal device with which you want to share the same image, or a display connected to another computer. Further, the landscape image information to be output may be configured to be saved simultaneously with the actual landscape image being photographed by the camera. This is executed by the landscape image simultaneous storage unit or the like. In this case, it becomes possible to display and enjoy both images side by side on a display at home or the like at a later date. This may be a moving image or a still image. In addition, if the display is in a format that cannot output a landscape image unless it obstructs the view, the landscape image information output section will determine that the user is walking if there is a change in position information, and will not output it. It is also possible to configure Furthermore, the configuration is configured to receive information to that effect from the landscape server device when the landscape server device goes into an area that does not have landscape images, and an alarm is configured to notify that the landscape server device has gone outside the landscape image output range. It is possible to do so.
This is the case when the landscape image storage range of the landscape server device is limited to, for example, historical buildings and their surroundings. Furthermore, there is a case where the landscape image storage range of the landscape server device is limited to tourist spots such as Kyoto.
<Embodiment 1 Effect: Hardware configuration (landscape server device)>

As shown in FIG. 12, the computer configuration of the landscape server device includes a CPU 1201, a nonvolatile memory 1202 such as an HDD or ROM, a main memory 1203 such as a D-RAM, and an interface 1204. The nonvolatile memory 1202 stores programs such as a landscape image information holding program, a viewpoint information acquisition program, and a landscape image information output program. The data includes position information, line of sight information, viewpoint information, landscape image information, etc. These programs and data are read into the holding area of the main memory 1203 and executed in the work area. Further, the interface 1204 includes a network interface 1205, a display interface 1206, a disk interface 1207, a keyboard interface 1208, and the like.
<Embodiment 1 Effect: Hardware configuration (mobile terminal device)>

As shown in FIG. 13, the computer configuration of the mobile terminal device includes a CPU 1301, a nonvolatile memory 1302 such as an HDD or ROM, a main memory 1303 such as a D-RAM, and an interface 1304. The nonvolatile memory 1302 stores programs such as a position information acquisition program, a line of sight information acquisition program, a viewpoint information acquisition program, a landscape image information acquisition program, and a landscape image information output program. The data includes position information, line-of-sight direction information, viewpoint height information, line-of-sight information, viewpoint information, landscape image information, and the like. These programs and data are read into the holding area of the main memory 1303 and executed in the work area. Further, the interface 1304 includes a network interface 1305, a position information interface 1306, a line of sight input interface 1307, a height input interface 1308, a touch input interface 1309, a software keyboard input interface 1310, and the like.
<Embodiment 1 Process flow Landscape server device>

As shown in the left diagram of FIG. 14, the operation of the computer of such a landscape server device executes a viewpoint information acquisition step of acquiring viewpoint information from a mobile terminal device using a viewpoint information acquisition program (step 1401). Here, the viewpoint information includes position information and line-of-sight information associated therewith, and since the details have been described above, the explanation will be omitted.

Next, a landscape image information output step of transmitting landscape image information associated with the viewpoint information acquired by the landscape image information output program to the mobile terminal device is executed (step 1402). The above-described viewpoint information acquisition step 1401 and scenery image information output step 1403 are repeated until the viewpoint information acquisition from the mobile terminal device is completed (step 1403).
<Embodiment 1 Processing flow Mobile terminal device>

As shown in the right diagram of FIG. 14, the operation of the computer of such a mobile terminal device includes a location information acquisition step of acquiring location information, which is information on the current location of the mobile terminal device, using a location information acquisition program. Execute (step 1411). Here, since the acquisition of position information has been described above, a description thereof will be omitted.

Next, a line-of-sight information acquisition step is executed to acquire line-of-sight information, which is information indicating the line-of-sight direction and/or viewpoint height information based on the position information acquired by the line-of-sight information acquisition program (step 1412). Here, the line-of-sight information may include only information on the line-of-sight direction, or may include both line-of-sight direction and viewpoint height information.

Next, a viewpoint information output step is executed to output viewpoint information in which the position information and line of sight information obtained by the viewpoint information output program are associated with each other to the landscape server device (step 1413). Here, since the details of acquiring the viewpoint information have been described above, the explanation will be omitted.
Next, a landscape image information acquisition step is executed to acquire landscape image information from the landscape server device according to the viewpoint information output by the landscape image information acquisition program (step 1414).
Next, a landscape image information output step is executed to output the landscape image information acquired by the landscape image information output program (step 1415). Then, landscape image information that matches the viewpoint information of the traveler or social studies visitor is displayed on the display screen of the mobile terminal device in response to the selection of the landscape image acquisition interface as described above.
The processing from processing steps 1411 to 1415 is repeated until the processing is completed (step 1416).

If the mobile terminal device does not include a display, as shown in FIG. 15, a display step 1516 is executed after the landscape image information output step 1515. As shown in FIG. 3, landscape image information is displayed on a display 0316 connected to a mobile terminal device 0310. Other processing is similar to the processing in FIG. 14 of the first embodiment, so the explanation will be omitted.

In this way, according to the first embodiment, a traveler can actually go to a tourist spot and view landscape image information according to the traveler's viewpoint information together with additional information. In addition, when the user goes to a town on a social studies field trip or the like, the user can view landscape information at the height of a desired viewpoint (for example, a child's viewpoint).
<Embodiment 2>
<Summary>

As shown in FIG. 16, the second embodiment is a system that can display past seasonal landscape image information in addition to the street viewer system of the first embodiment. The example in FIG. 16 shows an example in which landscape image information from 972 AD is displayed. If this past period-specific landscape information actually exists as a database, realistic past landscape image information is held in the landscape server device, and the held landscape image information is displayed. If landscape image information does not exist much back in time, content of landscape image information is created and stored in the landscape server device. Landscapes are stored on the landscape server by era, and a three-dimensional version of the aforementioned Google Earth computer composition is available. As mentioned above, the system is configured so that the viewpoint can be freely moved within the computer composition, allowing landscape images to be acquired from any viewpoint. These can also be used as history teaching materials.
<Embodiment 2 Configuration>

As shown in FIG. 17, the street viewer system of the second embodiment is a street viewer system that includes a landscape server device 1700 and a mobile terminal device 1710, and the landscape server device 1700 includes a landscape image information storage unit 1701 and a landscape image information storage unit 1701. , a viewpoint information acquisition section 1702 and a landscape image information output section 1704, the landscape image information storage section 1701 has a period-specific landscape image information storage section 1704, and a period-specific landscape image information output section 1705. It also has a server-side timing information acquisition unit 1706,
The mobile terminal device 1710 includes a position information acquisition unit 1711, a line of sight information acquisition unit 1712, a viewpoint information output unit 1713, a landscape image information acquisition unit 1714, and a landscape image information output unit 1715, and includes a mobile terminal side time information The landscape image information acquisition section 1714 includes an acquisition section 1716 and a period information transmission section 1717, and the landscape image information acquisition section 1714 has a period-specific landscape image information acquisition means 1718.
FIG. 17 shows a configuration in which the mobile terminal device according to the second embodiment includes a display, and FIG. 18 shows a configuration in which the mobile terminal device according to the second embodiment does not include a display. The configuration other than the display 1819 in FIG. 18 is the same as the configuration in FIG. 17, so a description thereof will be omitted.
<Embodiment 2 Description of configuration>
<Embodiment 2 Landscape server device>

In addition to the configuration of Embodiment 1, this landscape server device holds landscape image information by period in the device, acquires period information from a mobile terminal device, and acquires landscape images by period according to the acquired viewpoint information. It is configured to output image information. Other details are the same as in Embodiment 1, so detailed explanation will be omitted.
Hereinafter, features common to Embodiment 1 will be omitted, and the characteristic configuration will be described.
<Embodiment 2 Era-specific landscape image information storage means>

A "period-specific landscape image information holding unit" 1704 is included in the landscape image information storage unit 1701 and holds period-specific landscape image information that is landscape image information for a plurality of past periods. For example, in a mobile terminal device, if the user of the mobile terminal device selects a selection menu such as "by period" on the landscape image acquisition interface as shown in FIG. It maintains landscape image information for multiple periods so that the following periods can be set: "Middle Period," "Early Edo Period," "Azuchi/Momoyama," "Muromachi," and "Kamakura." For landscape image information that does not currently exist, landscape image information is created using image processing technology, or if computer graphic images are stored in another database, that image is used.

Alternatively, a plurality of typical landscape image components may be prepared in advance according to location information, and the landscape image may be created and transmitted by combining the prepared landscape image components according to processing rules. . For example, if the area identified by location information and time information is the vicinity of the Kyoto Imperial Palace in the Muromachi period, landscape image components that make up a typical aristocratic mansion, landscape image components that make up the walls of the imperial palace, etc. They are combined to recreate the cityscape. In addition, if the object identified by the location information and time information is near Yokohama Port in the Meiji period, the landscape image components of Western-style buildings and the landscape image components of ships in the Meiji period are combined to The images are then sent as landscape images.

Note that it may include an old photo storage unit that stores old photos, and a landscape image component creation unit that creates landscape image components from the stored old photos. In order to create landscape image constituent elements from old photographs, we need a building image recognition means that recognizes buildings, etc. in old photographs, and a building surface composition that determines what kind of material the recognized building surfaces are made of. It is preferable that it consists of element judgment means, etc. Then, based on the determined results, landscape image components are created using computer graphics.
<Embodiment 2 Server-side timing information acquisition unit>

The "server-side timing information acquisition unit" 1706 acquires timing information, which is information indicating past timing, from the mobile terminal device. Period information is information by historical period such as "Meiji to Showa", "Late Edo", "Middle Edo", "Early Edo", "Azuchi/Momoyama", "Muromachi", "Kamakura", and Heisei era. It may be the year 1998, 1998, 2008, 2013, etc., or detailed information such as every year, every two years, what year, hour, minute, etc. When the user selects such timing information from the selection menu of the mobile terminal device, the server-side timing information acquisition unit 1706 acquires past timing information as described above via the network.
<Embodiment 2 Era-specific landscape image information output means>

The “seasonal landscape image information output unit” 1705 is included in the landscape image information output unit 1703 and outputs seasonal landscape image information of the period corresponding to the acquired period information to the mobile terminal device 1710. The period-specific landscape image information includes still images and video images from the Meiji and Taisho eras, and from the Showa era, if moving images remain, it is composed of moving images for realism. Furthermore, since there are no moving images from the Edo period, Azuchi and Momoyama periods, etc., digital period-specific landscape images are created from past paintings and materials, and content videos such as anime are used. Some tourist spots still have footage from NHK taiga dramas depicting historically famous figures, so such footage can be used as period-specific landscape image information. Additionally, original video content may be created in each city. When this system is used for social studies field trips, the period-specific landscape images are configured with period-specific landscape image information and explanations so that they can be used as digital teaching materials.
<Embodiment 2 Mobile terminal device>
<Embodiment 2 Mobile terminal side timing information acquisition unit>

The “mobile terminal side timing information acquisition unit” 1716 acquires timing information. Here, the timing information is obtained by selecting a selection menu by the user of the mobile terminal device or by directly inputting the year, month, and day.
As shown in FIG. 19, the landscape image acquisition interface screen is displayed on the display screen of the mobile terminal device, the current location or the recording location is selected, and the past year is selected on the time setting screen. for example,
Obtain time information for the historical periods of ``Meiji to Showa'', ``Late Edo'', ``Middle Edo'', ``Early Edo'', ``Azuchi/Momoyama'', ``Muromachi'', and ``Kamakura''. Alternatively, it may be a screen for inputting detailed information such as years such as 1989, 1998, 2008, 2013, every year, every two years, what year, hour and minute, etc. The selection menu of these timing information can be changed as appropriate by the users of the street viewer system, such as tourists and social studies visitors.
When the “OK button” is selected by the user, the timing information is transmitted from the time information transmitting unit 1717 to the scenery server device 1700. If you make a mistake in your selection, you can select the "Redo button" to start over. In the above example, an example of selection using a selection menu was shown, but a selection method based on a user's voice instruction using a voice input interface may also be used. The timing information acquisition unit may be configured to display a slide bar (which may be configured to be slidable on a touch panel) on the display as an interface, and acquire the timing information while continuously changing the slide bar. It is possible. In such a case, landscape image information from different eras can be viewed continuously in synchronization with the movement of the slide bar. Further, the slide bar may be configured to be displayed together with the historical chronology. In other words, it is convenient to display a slide bar adjacent to the position indicating the year in the band-shaped historical chronology and configure it so that it can be moved using a touch panel. In this way, for example, you can see the state of the Great Buddha in Nara as it has been restored over time.
<Embodiment 2 Timing information transmitter>

The “time information transmitting unit” 1717 transmits the acquired time information to the scenery server device 1700. Here, the above-mentioned timing information acquired from the mobile terminal side timing information acquisition unit 1716 is transmitted to the landscape server device 1700 via a network (not shown).
<Embodiment 2 Time-based landscape image information acquisition means>

A "seasonal landscape image information acquisition unit" 1718 is included in the landscape image information acquisition unit 1714 and acquires seasonal landscape image information according to the transmitted period information. For example, in a case where a user of a mobile terminal device uses a street viewer system while traveling in Kyoto, landscape image information of Kyoto by era will be displayed. This is not limited to Kyoto, but by displaying landscape image information suitable for the region, it is possible to convey the appeal of each tourist destination to tourists and people who live there.
<Embodiment 2 Effect: Hardware configuration (landscape server device)>

As shown in FIG. 20, the computer configuration of the landscape server device includes a CPU 2001, a nonvolatile memory 2002 such as an HDD or ROM, a main memory 2003 such as a D-RAM, an interface 2004, a network interface 2005, and a display. It is composed of an interface 2006, a disk interface 2007, and a keyboard interface 2008. The nonvolatile memory 2002 stores programs such as a seasonal landscape image information holding subprogram, a server-side seasonal information acquisition program, and a seasonal landscape image information output subprogram. The data includes location information, line of sight information, viewpoint information, landscape image information, time information, etc. These programs and data are read into the holding area of the main memory 2003 and executed in the work area. The other program configurations are the same as the configuration of the first embodiment shown in FIG. 12, so the description thereof will be omitted.
<Embodiment 2 Effect: Hardware configuration (mobile terminal device)>

As shown in FIG. 21, the configuration of the computer of the mobile terminal device includes a CPU 2101, a non-volatile memory 2102 such as an HDD or ROM, a main memory 2103 such as a D-RAM, an interface 2104, a network interface 2105, and a It is composed of an information interface 2106, a line of sight input interface 2107, a height input interface 2108, a touch input interface 2109, and a software keyboard input interface 2110. The nonvolatile memory 2102 stores programs such as a mobile terminal side period information acquisition program, a period information transmission program, and a period-specific landscape image information acquisition program. The data includes position information, line-of-sight direction information, viewpoint height information, line-of-sight information, viewpoint information, landscape image information, time information, and the like. These programs and data are read into the holding area of the main memory 2103 and executed in the work area. The other program configurations are the same as the configuration of the first embodiment shown in FIG. 13, so the description thereof will be omitted.
<Embodiment 2 Process flow Landscape server device>

As shown in the left diagram of FIG. 22, the computer of such a landscape server device operates by executing a viewpoint information acquisition step of acquiring viewpoint information from a mobile terminal device using a viewpoint information acquisition program (step 2201). Here, the viewpoint information includes position information and line-of-sight information associated therewith, and since the details have been described above, the explanation will be omitted.

Next, the server-side timing information acquisition program executes a server-side timing information acquisition step of acquiring timing information, which is information indicating past periods, from the mobile terminal device (step 2202). Here, since the timing information has been described above, the explanation will be omitted.

Next, a landscape image information output step of transmitting landscape image information associated with the viewpoint information acquired by the landscape image information output program to the mobile terminal device is executed (step 2203). In the landscape image information output step 2203, a period-specific landscape image information output step is executed to output period-specific landscape image information of the period corresponding to the period information acquired by the period-specific landscape image information output subprogram to the mobile terminal device (step 2204).

Until the viewpoint information acquisition from the mobile terminal device is completed (step 2205), the above-mentioned viewpoint information acquisition step 2201, server side time information acquisition step 2202, landscape image information output step 2203, and seasonal landscape image information output sub-step 2204 are performed. repeat.
<Embodiment 2 Processing flow Mobile terminal device>

As shown in the right diagram of FIG. 22, the operation of the computer of such a mobile terminal device executes a position information acquisition step in which the above-described position information is acquired by the position information acquisition program (step 2211).

Next, a line-of-sight information acquisition step is executed to acquire line-of-sight information, which is information indicating the line-of-sight direction and/or viewpoint height information based on the position information acquired by the line-of-sight information acquisition program (step 2212). Here, the line-of-sight information may include only information on the line-of-sight direction, or may include both line-of-sight direction and viewpoint height information.

Next, a viewpoint information output step is executed to output viewpoint information in which the position information and line of sight information obtained by the viewpoint information output program are associated with each other to the landscape server device (step 2213). Here, since the details of acquiring the viewpoint information have been described above, the explanation will be omitted.

Next, a mobile terminal side timing information acquisition step for acquiring timing information by the mobile terminal side timing information acquisition program is executed (step 2214). Here, since the acquisition of timing information has been described above, the explanation will be omitted.

A timing information sending step is executed to send the timing information acquired by the timing information sending program to the landscape server device (step 2215).
Next, a landscape image information acquisition step is executed to acquire landscape image information from the landscape server device according to the viewpoint information output by the landscape image information acquisition program (step 2216). In the landscape image information acquisition step 2216, a seasonal landscape image information acquisition step is executed to acquire seasonal landscape image information according to the time information transmitted by the seasonal landscape image information acquisition program (step 2217).

Next, a landscape image information output step is executed to output the landscape image information acquired by the landscape image information output program (step 2218). Then, on the display screen of the mobile terminal device, seasonal landscape image information that matches the viewpoint information of the traveler or social studies visitor and corresponds to the transmitted period information is displayed in accordance with the selection of the landscape image acquisition interface as described above. Is displayed.
The processing from processing steps 2211 to 2218 is repeated until the processing is completed (step 2219).
If the mobile terminal device does not include a display, as shown in FIG. 23, a display display step 2319 is executed after the landscape image information output step 2318. As shown in FIG. 18, landscape image information is displayed on a display 1819 connected to a mobile terminal device 1810.

In this way, according to the second embodiment, in addition to the first embodiment, by displaying past period-specific landscape image information, it is possible to convey even more attractiveness of tourist spots and regions to tourists and social studies visitors. can.
<Embodiment 3>
<Summary>

In addition to the street viewer system of Embodiment 1 or 2, Embodiment 3 is a system that can display event-specific landscape image information, which is landscape information of historical or/seasonal events. A typical example of a historical event is the ``Honnoji Incident.'' The landscape image information of such historical events is stored in the landscape server device along with viewpoint information and event identification information. Landscape information of seasonal events is collected periodically and stored in the landscape server device. In the case of historical events, the content of landscape image information related to the event is stored in the landscape server device along with viewpoint information (location information + line of sight information) of historical monuments, castle ruins, buildings that existed in the past, etc. are doing.
<Embodiment 3 Configuration>

As shown in FIG. 25, the street viewer system according to the third embodiment is a street viewer system that includes a landscape server device 2500 and a mobile terminal device 2510, and the landscape server device 2500 includes a landscape image information storage unit 2501 and a landscape image information storage unit 2501. , a viewpoint information acquisition unit 2502 and a landscape image information output unit 2504, the landscape image information storage unit 2501 has an event-specific landscape image information storage unit 2504, and an event-specific landscape image information output unit 2505. It also has a server-side event identification information acquisition unit 2606,
The mobile terminal device 2510 includes a position information acquisition unit 2511, a line of sight information acquisition unit 2512, a viewpoint information output unit 2513, a landscape image information acquisition unit 2514, and a landscape image information output unit 2515, and includes a mobile terminal side event identification unit. The landscape image information acquisition unit 2514 includes an information acquisition unit 2516 and an event identification information transmission unit 2517, and the landscape image information acquisition unit 2514 includes an event-specific landscape image information acquisition unit 2518.

FIG. 25 shows a configuration in which the mobile terminal device according to the third embodiment includes a display, and FIG. 26 shows a configuration in which the mobile terminal device according to the third embodiment does not include a display. The configuration other than the display 2619 in FIG. 26 is the same as the configuration in FIG. 25, so a description thereof will be omitted.
<Embodiment 3 Description of configuration>
<Embodiment 3 Landscape server device>

In addition to the configuration of Embodiment 1 or 2, this landscape server device holds landscape image information by event in the device, acquires event identification information from a mobile terminal device, and uses the acquired viewpoint information. The system is configured to output landscape image information for each event according to the event. Additionally, the event-specific landscape images are configured to additionally hold character, text, audio information, and the like. Other details are the same as those in Embodiment 1 or Embodiment 2, so detailed explanation will be omitted.
Hereinafter, features common to Embodiment 1 or Embodiment 2 will be omitted, and characteristic configurations will be described.
<Embodiment 3: Landscape image information holding means by event>

The “event-specific landscape image information storage unit” 2504 is included in the landscape image information storage unit 2501 and holds event-specific landscape image information that is landscape image information of historical and/or seasonal events. The event-specific landscape images display landscape images that are related to the current location of the user of the mobile terminal device, and do not include landscape images that are not related to the current location of the user. For event-specific landscape image information, data on historical events and seasonal events is collected in advance and stored along with still images, videos, etc., along with commentary and narration audio information. Alternatively, if other related landscape image information exists on the Internet, the landscape server device may search for it on the Internet and hold it.
<Embodiment 3 Server-side event identification information acquisition unit>

The “server-side Internet identification information acquisition unit” 2506 acquires event identification information from the mobile terminal device. Event identification information is information on historical events such as monuments, castle ruins, and ancient battlefields, and information on seasonal events such as international and domestic event venues.
When the user selects the "event menu" from the selection menu of the mobile terminal device as shown in FIG. 11, the server-side event identification information acquisition unit 2506 acquires the event identification information as described above via the network. Further, when the tourist goes to a tourist spot and receives event identification information that is completely unrelated to the tourist spot, the landscape server device sends an error message to the mobile terminal device as a user message.
<Embodiment 3: Event-specific landscape image information output means>

"Event-specific landscape image information output means" 2505 is included in the landscape image information output unit 2503 and outputs event-specific landscape image information corresponding to the acquired event identification information to the mobile terminal device. As the event-specific landscape image information, landscape image information associated with viewpoint information such as ancient battlefield sites is displayed, or landscape image information associated with position information of department stores in the city is displayed. Landscape image information classified by historical event is displayed to social studies visitors and tourists who have actually visited the place, so they can sharpen their memories of the place. It is possible to create an information tool that can strongly appeal to tourists and social studies visitors.
<Embodiment 3 Mobile terminal device>
<Embodiment 3 Mobile terminal side event identification information acquisition unit>

“Mobile terminal side event identification information acquisition unit” 2516 acquires event identification information. Here, the event identification information is provided by selection from a selection menu or direct input by the user of the mobile terminal device.
As shown in FIG. 11, the landscape image acquisition interface screen is displayed on the display screen of the mobile terminal device, the current position or the recording position is selected, and the "event" menu is selected. The selection menu of these event identification information can be changed as appropriate by the user of the street viewer system, such as a tourist or a social studies visitor.
When the “event menu” is selected by the user, the event identification information is transmitted from the event identification information transmitting unit 2517 to the scenery server device 2500. In the above example, an example of selection using a selection menu was shown, but a selection method using a voice input interface by the user may also be used.
<Embodiment 3 Event identification information transmitter>

The “event identification information transmitting unit” 2517 transmits the acquired timing information to the scenery server device 2500. Here, the above-described event identification timing information acquired from the mobile terminal side event identification information acquisition unit 2516 is transmitted to the scenery server device 2500 via a network (not shown).
<Embodiment 3: Landscape image information acquisition means by event>

"Event-specific landscape image information acquisition means" 2518 is included in the landscape image information acquisition unit 2514 and acquires event-specific landscape image information according to the transmitted event identification information. For example, when a traveler visits Honnoji Temple in Kyoto and uses the smart viewer system of Embodiment 3, when the traveler points the mobile terminal device outside Honnoji Temple, it shows the state of Akechi Mitsuhide's army during the ``Honnoji Incident.'' It may be displayed as event-specific landscape image information, or when a traveler enters Honnoji Temple and looks at his mobile terminal, an event-specific landscape image showing the battle between Oda Nobunaga and Akechi Mitsuhide's army is displayed. The content of the event-specific landscape image information can be changed depending on the location of the user of the mobile terminal device. This is not limited to the ``Honnoji Incident,'' but by displaying landscape image information appropriate to the area, it is possible to convey the appeal of each tourist destination to tourists and people who live there.
<Embodiment 3 Effect: Hardware configuration (landscape server device)>

As shown in FIG. 27, the computer configuration of the landscape server device includes a CPU 2701, a nonvolatile memory 2702 such as an HDD or ROM, a main memory 2703 such as a D-RAM, an interface 2704, a network interface 2705, and a display. It consists of an interface 2706, a disk interface 2707, and a keyboard interface 2708. The non-volatile memory 2702 stores programs such as an event-specific scenic image information holding subprogram, a server-side event identification information acquisition program, and an event-specific scenic image information output subprogram. The data includes position information, line of sight information, viewpoint information, landscape image information, event identification information, and the like. These programs and data are read into the holding area of the main memory 2703 and executed in the work area. The other program configurations are the same as the configuration of the first embodiment shown in FIG. 12, so the description thereof will be omitted.
<Embodiment 3 Effect: Hardware configuration (mobile terminal device)>

As shown in FIG. 28, the configuration of the computer of the mobile terminal device includes a CPU 2801, a nonvolatile memory 2802 such as an HDD or ROM, a main memory 2803 such as a D-RAM, an interface 2804, a network interface 2805, and a It is composed of an information interface 2806, a line of sight input interface 2807, a height input interface 2808, a touch input interface 2809, and a software keyboard input interface 2810. The nonvolatile memory 2802 stores programs such as a mobile terminal side event identification information acquisition program, an event identification information transmission program, and a landscape image information acquisition program by event identification. The data includes position information, line-of-sight direction information, viewpoint height information, line-of-sight information, viewpoint information, landscape image information, event identification information, and the like. These programs and data are read into the holding area of the main memory 2803 and executed in the work area. The other program configurations are the same as the configuration of the first embodiment shown in FIG. 13, so the description thereof will be omitted.
<Embodiment 3 Processing flow Landscape server device>

As shown in the left diagram of FIG. 29, the computer of such a landscape server device operates by executing a viewpoint information acquisition step of acquiring viewpoint information from a mobile terminal device using a viewpoint information acquisition program (step 2901). Here, the viewpoint information includes position information and line-of-sight information associated therewith, and since the details have been described above, the explanation will be omitted.

Next, the server-side event identification information acquisition program executes a server-side event identification information acquisition step of acquiring event identification information from the mobile terminal device (step 2902). Here, since the event identification information has been described above, a description thereof will be omitted.

Next, a landscape image information output step of transmitting landscape image information associated with the viewpoint information acquired by the landscape image information output program to the mobile terminal device is executed (step 2903). In the landscape image information output step 2903, an event-specific landscape image information output step is executed to output event-specific landscape image information of the event corresponding to the event identification information acquired by the event period-specific landscape image information output subprogram to the mobile terminal device. (Step 2904).

Until the acquisition of viewpoint information and event identification information from the mobile terminal device is completed (step 2905), the above-mentioned viewpoint information acquisition step 2901, server side event identification information acquisition step 2902, landscape image information output step 2903, and event-specific landscape are performed. Image information output sub-step 2904 is repeated.
<Embodiment 3 Processing flow Mobile terminal device>

As shown in the right diagram of FIG. 29, the operation of the computer of such a mobile terminal device executes a position information acquisition step of acquiring position information as described above using a position information acquisition program (step 2911).

Next, a line-of-sight information acquisition step is executed to acquire line-of-sight information, which is information indicating the line-of-sight direction and/or viewpoint height information based on the position information acquired by the line-of-sight information acquisition program (step 2912). Here, the line-of-sight information may include only information on the line-of-sight direction, or may include both line-of-sight direction and viewpoint height information.

Next, a viewpoint information output step is executed to output viewpoint information in which the position information and line of sight information acquired by the viewpoint information output program are associated to the scenery server device (step 2913). Here, since the details of acquiring the viewpoint information have been described above, the explanation will be omitted.

Next, a mobile terminal event identification information acquisition unit step for acquiring event identification information by the mobile terminal event identification information acquisition program is executed (step 2914). Here, since the acquisition of event identification information has been described above, the explanation will be omitted.

Next, an event identification information sending step is executed to send the event identification information acquired by the event identification information sending program to the landscape server device (step 2915).
Next, a landscape image information acquisition step is executed to acquire landscape image information from the landscape server device according to the viewpoint information output by the landscape image information acquisition program (step 2916). In the landscape image information acquisition step 2916, an event-specific landscape image information acquisition step is executed to acquire event-specific landscape image information according to the event identification information transmitted by the event-specific landscape image information acquisition program (step 2917).
Next, a landscape image information output step is executed to output the landscape image information acquired by the landscape image information output program (step 2918). Then, event-specific landscape image information is displayed on the display screen of the mobile terminal device in accordance with the viewpoint information of the traveler or social studies visitor and according to the transmitted event identification information in accordance with the selection of the landscape image acquisition interface as described above. is displayed.
The processing from processing steps 2911 to 2918 is repeated until the processing is completed (step 2919).

If the mobile terminal device does not include a display, as shown in FIG. 30, a display display step 3019 is executed in the landscape image information output step 3018. As shown in FIG. 26, landscape image information is displayed on a display 2619 connected to a mobile terminal device 2610.

In this way, according to the third embodiment, in addition to the first or second embodiment, by displaying event-specific landscape image information of events, tourist spots and regions can be made more attractive to tourists and social studies visitors. can convey.
<Embodiment 4>
<Summary>

In addition to the smart viewer system described in any one of Embodiments 1 to 3, Embodiment 4 provides a street viewer system that takes into account the usage pattern of a mobile terminal device and supports the posture of a user viewing landscape images. It has been realized.
<Embodiment 4 Configuration>

As shown in FIG. 31, the street viewer system of the fourth embodiment is a street viewer system that includes a landscape server device 3100 and a mobile terminal device 3110, and the landscape server device 3100 includes a landscape image information storage unit 3101 and a landscape image information storage unit 3101. , a viewpoint information acquisition section 3102, and a landscape image information output section 3103, and the mobile terminal device 3110 has a position information acquisition section 3111, a line of sight information acquisition section 3112, a viewpoint information output section 3113, and a landscape image information output section 3110. It has an acquisition section 3114 and a landscape image information output section 3115.
Furthermore, the line-of-sight information acquisition unit 3112 includes a line-of-sight direction sensor 3120, a line-of-sight direction acquisition means 3121, a line-of-sight direction holding means 3122, a holding command outputting means 3123, a holding cancellation command outputting means 3124, and The viewing direction included in the information is configured to be the maintained viewing direction.
In the following, explanations of common configurations described in any one of Embodiments 1 to 3 will be omitted, and only configurations characteristic of Embodiment 4 will be described.
<Embodiment 4 Description of configuration>
<Embodiment 4 Landscape server device>

The landscape server device according to the fourth embodiment is the same as the landscape server device described in any one of the first to third embodiments, so the description thereof will be omitted.
<Embodiment 4 Mobile terminal device>

In addition to the mobile terminal devices of Embodiments 1 to 3, the mobile terminal device of the fourth embodiment has a feature in the function of the line-of-sight information acquisition unit 3112. The characteristic configuration of the mobile terminal device according to the fourth embodiment will be described below.
<Embodiment 4 Gaze direction sensor>

The "line-of-sight direction sensor" 3120 detects east, west, north, south, up, and down directions. Here, an example of the line-of-sight direction sensor is a digital electronic compass, and the vertical direction is detected by detecting the north, south, east, west, and horizontal directions. Such a digital electronic compass is built into the mobile terminal device 3110.
<Embodiment 4 Gaze direction acquisition means>

The "line-of-sight direction acquisition means" 3121 continuously acquires the line-of-sight direction from the line-of-sight direction sensor 3120. When the user of the mobile terminal device 3110 uses the mobile terminal device in various orientations, the viewing direction is continuously acquired. In principle, the information on the line of sight direction that is continuously acquired is continuously transmitted to the landscape server device while being included in the viewpoint information. The same applies to position information and viewpoint height information, and viewpoint information is continuously generated and continuously transmitted to the landscape server device. Therefore, landscape image information is continuously received from the landscape server device in accordance with continuously transmitted viewpoint information. This is one operation pattern. This operation is similar in the systems of the first to third embodiments. On the other hand, as mentioned above, the viewing direction is generally the same as the imaging direction of the camera of the mobile terminal device, so in order to view landscape images, the camera must always be pointed at the desired landscape, and from the naked eye, the direction of the camera of the mobile terminal device is the same. You can't see the actual scenery because it's hidden in the shadows. In other words, it becomes difficult to compare and observe the actual scenery and the scenery image. Therefore, it is desirable to have a configuration in which the landscape image transmitted from the landscape server device can be viewed even if the viewing direction is such that the screen of the tablet terminal placed on the lap is excluded.
The configuration for solving this problem is described below.
<Embodiment 4 Line-of-sight direction holding means>

The "line-of-sight direction holding means" 3122 fixes and holds the line-of-sight direction that is being continuously acquired at the acquisition timing of the holding command. When there is an input from the user of the mobile terminal device, the holding command output means 3123 issues a holding command for the consecutively acquired line-of-sight directions, and fixes them at the acquisition timing. Further, among the modes for fixing time, there may be a mode for fixing an instant and a mode for fixing a predetermined length of time. The former fixes the instantaneous viewing direction, and the latter fixes the viewing direction for a certain period of time, for example 10 seconds. In the latter case, for example, the tablet camera is configured to rotate 360 degrees so that a landscape image of the entire surrounding area can be viewed in 10 seconds.
<Embodiment 4 Hold command output means>

The "holding command output means" 3123 outputs a holding command to the line-of-sight direction holding means 3122 based on the input from the user. Here, this can be achieved by the user touching a "hold button icon" provided on the display screen of the mobile terminal device. Further, although this holding method uses a touch input method by the user, it may also be held using voice input. Since the line-of-sight direction included in the line-of-sight information is thus maintained, the user can view a desired landscape image in a free posture without worrying about the orientation of the mobile terminal device. When the landscape server device receives fixed viewpoint information output and received by the retention command output means, the fixed viewpoint information is not fixed unless the mobile terminal device notifies that the retention has been canceled in accordance with the retention cancellation command described later. Do not send anything other than the temporal landscape image. This is to prevent the mobile terminal device from receiving unnecessary landscape images in the viewing direction. Further, when the fixed time is 10 seconds, the landscape server device is configured to repeatedly transmit landscape images for 10 seconds. 10 seconds is the length of time for eye movement.
<Embodiment 4 Hold cancellation command output means>

The "holding cancellation command outputting means" 3123 cancels the holding of the sight line direction held in the sight line direction holding means 3122 based on the input from the user. Here, in order to cancel the fixed state in which the viewing direction is held, the user can return to the original normal display state by touching the "holding release button icon". In this case as well, if there is no problem with the environmental noise in which the user is located, a method of holding and canceling by voice input may be used.
<Embodiment 4 Landscape image information output unit>

"Landscape image information output unit" 3114 outputs the acquired landscape image information. Here, if the mobile terminal device 3110 includes a display, landscape image information that matches the viewpoint information of the traveler or social studies visitor is displayed on the display screen of the mobile terminal device in response to the selection of the landscape image acquisition interface. be done. For example, if the fix is for 10 seconds, the landscape image for 10 seconds is displayed repeatedly.
<Embodiment 4 Effect: Hardware configuration (landscape server device)>

The hardware configuration of the computer of the landscape server device of Embodiment 4 is the same as that of the landscape server device described in any one of Embodiments 1 to 3, so a description thereof will be omitted.
<Embodiment 4 Effect: Hardware configuration (mobile terminal device)>

As shown in FIG. 32, the configuration of the computer of the mobile terminal device includes a CPU 3201, a nonvolatile memory 3202 such as an HDD or ROM, a main memory 3203 such as a D-RAM, an interface 3204, a network interface 3205, and a It is composed of an information interface 3206, a line of sight input interface 3207, a height input interface 3208, a touch input interface 3209, and a software keyboard input interface 3210. The nonvolatile memory 3202 stores programs such as a line-of-sight direction sensor subprogram, a line-of-sight direction acquisition subprogram, a line-of-sight direction holding subprogram, a hold command output subprogram, and a hold cancellation command output subprogram. The data includes position information, line-of-sight direction information, viewpoint height information, line-of-sight information, viewpoint information, landscape image information, and the like. These programs and data are read into the holding area of the main memory 3203 and executed in the work area.
<Embodiment 4 Processing flow Landscape server device>

As shown in the left diagram of FIG. 33, the computer of the landscape server device operates as follows: viewpoint information acquisition step 3301 and landscape image information output step 3302.

The above-described viewpoint information acquisition step 3301 and landscape image information output step 3302 are repeated until the viewpoint information acquisition from the mobile terminal device is completed (step 3303).
<Embodiment 4 Processing flow Mobile terminal device>

As shown in the right diagram of FIG. 33, the flow of processing of such a mobile terminal device is as follows.

First, in a location information acquisition step 3311, location information of the current location of the mobile terminal device is acquired. Next, in the line-of-sight information acquisition step 3312, the line-of-sight direction is initially acquired continuously, but upon receiving a holding command from the user, the line-of-sight direction is output to the line-of-sight direction holding means, and the line-of-sight direction holding step is executed. The line-of-sight direction holding sub-step 3313 is a step in which the line-of-sight direction, which has been continuously acquired at the timing of the user's input, is momentarily stopped and held, or held for a predetermined length of time. Thereafter, a sub-step 3314 of generating viewpoint information together with line-of-sight height information is performed, and then step 3315 of transmitting the viewpoint information to the scenery server device is executed. After step 3316 of receiving the landscape image from the landscape server device that has received the viewpoint information generated based on the maintained line-of-sight direction, step 3317 of displaying the landscape image on a display or the like is executed. This makes it possible to appreciate a landscape image based not on the continuously acquired line-of-sight direction but also on a fixed line-of-sight direction without worrying about the current direction of the line-of-sight direction.
<Embodiment 5A>
<Embodiment 5A Overview>

The mobile terminal device of this embodiment is characterized in that it has a camera and a display, and the line-of-sight direction sensor obtains the line-of-sight that looks at the scenery displayed on the display through the camera as the line-of-sight direction.
<Embodiment 5A configuration>

The configuration of Embodiment 5A is based on the configuration of Embodiment 14 with the addition of a camera and a display. The common components with the fourteenth embodiment are a landscape image information holding unit, a viewpoint information acquisition unit, and a landscape image information output unit for the landscape server device, and a position information acquisition unit and a position information acquisition unit for the mobile terminal device. , a line-of-sight information acquisition unit, a viewpoint information output unit, a landscape image information acquisition unit, a landscape image information output unit, and the line-of-sight direction included in the line-of-sight information provided in the line-of-sight information acquisition unit is retained. These are a line-of-sight direction sensor configured to be the line-of-sight direction, a line-of-sight direction acquisition means, a line-of-sight direction holding means, a holding command outputting means, and a holding cancellation command outputting means, and a description of common parts will be omitted.
<Explanation of Embodiment 5A Configuration>
<Explanation of Embodiment 5A Camera>

A "camera" is provided in a mobile terminal device. The number of lenses may be singular or plural. Further, a fixed focus lens or a variable focus lens may be used. The image sensor of a camera is generally a solid state image sensor. CMOS is even better from the viewpoint of imaging speed. It is configured to process the center normal direction of the image captured by the image sensor as the viewing direction. In other words, the various built-in viewpoint-related functions are configured to detect the center normal direction of the image sensor.

If the camera is equipped with a plurality of lenses, the camera can be configured to switch the line-of-sight direction by switching the lens used as the line-of-sight direction. Further, in the case of an all-sky camera, it is conceivable to configure it so that landscape images in all-sky directions are transmitted from a landscape server device. In the case of an omnidirectional camera capable of 360-degree photography, the configuration may be such that the scenery image of an all-sky video is transmitted from the scenery server device. It may also be possible to use two omnidirectional cameras to take pictures similar to the omnidirectional camera.

As shown in Figure 35, when using a wearable camera (part of a mobile terminal device) whose viewing direction is the imaging center of the camera attached to the cap, the viewing direction must be fixed as shown in (A). (For example, by voice, operating means at hand, etc.) The camera may be configured so that the line of sight direction of the camera remains fixed even if the person then looks down as shown in (B). By doing so, you can view old landscape images of the landscape you saw in a natural posture on a tablet display.
<Embodiment 5A Display>

The "display" displays an image captured by a camera, or a landscape image returned from the landscape server device based on viewpoint information including the viewing direction obtained by the viewing direction sensor. First, a user displays a local landscape image captured by a camera on a display, checks its angle of view, and then displays the landscape image sent from the landscape server device on the display.

However, since there may be a plurality of displays instead of a single display, the local scenery may be displayed on one display and the scenery image may be displayed on the other display.
Furthermore, when there is only one display, the landscape image and the actual landscape may be switched by a switching button. In addition, the built-in display displays a landscape image of the actual landscape, and the landscape image from the landscape server device can be output to a mobile terminal device such as another tablet terminal using close proximity wireless communication such as Bluetooth (registered trademark), and can be used there. The landscape image that has been sent may be made viewable.

<Other incidental elements>
Regarding "audio", the landscape image from the landscape server device may have audio added to it. Prepare in advance the sounds that match the scenery. Examples include the sound of a temple bell, the sound of the wind, and the voice of a merchant calling. For example, in the case of the sound of a bell, it is preferable to configure the sound so that the sound can be heard from the direction of the bell in the landscape image. It can be configured using multi-channel speakers, etc.
<Embodiment 5B>
<Embodiment 5B Overview>

The mobile terminal device of the present embodiment is characterized in that the camera has a holding command photographing unit, and photographs a landscape still image in response to the holding command output from the holding command output means.
<Embodiment 5B Configuration>

The configuration of Embodiment 5B is based on the configuration of Embodiment 5A, with the addition of a holding command photographing unit to the camera. The common structural requirements with the fifteenth embodiment are a landscape image information holding unit, a viewpoint information acquisition unit, and a landscape image information output unit for the landscape server device, and a position information acquisition unit and a position information acquisition unit for the mobile terminal device. , a line-of-sight information acquisition unit, a viewpoint information output unit, a landscape image information acquisition unit, a landscape image information output unit, and the line-of-sight direction included in the line-of-sight information provided in the line-of-sight information acquisition unit is retained. These are a line-of-sight direction sensor configured to be the line-of-sight direction, a line-of-sight direction acquisition means, a line-of-sight direction holding means, a holding command outputting means, and a holding cancellation command outputting means, and a description of common parts will be omitted.
<Embodiment 5B Description of configuration>
<Embodiment 5B Photographing unit at the time of holding command>
The "holding command photography unit" is built into the camera and photographs a landscape still image in response to the holding command output from the holding command output means 3123. The photographing unit upon holding command can photograph a still image while the line of sight direction is held. Note that the still image may be a moving image with a predetermined length of time. Furthermore, the still image can be configured to shoot a plurality of still images while switching the focus to a plurality of positions. This is because there is a difference in whether the focus is on the teahouse in the foreground or the Sanmon gate in the back. This focus information is preferably included in the viewpoint information and transmitted to the landscape server device, and the landscape server device is preferably configured to take the focus position into consideration before transmitting the landscape image. A plurality of landscape images are transmitted to the mobile terminal device according to the focus position, and the mobile terminal device is configured to hold the landscape image and the still image in association with each other based on the focus position.
<Embodiment 5C>
<Embodiment 5C Overview>

The mobile terminal device of this embodiment has a magnification acquisition unit in a camera having a zoom-in/zoom-out function that can be operated by the user, acquires the magnification of a landscape still image when shooting with the photography unit at the time of a hold command, and displays the image. The feature is that a landscape image having the same magnification as the photographed still image is output.
<Embodiment 5C configuration>

The configuration of Embodiment 5C is based on the configuration of Embodiment 5B, and a magnification acquisition unit is added to the camera that functions to zoom in and zoom out, and a landscape image information output unit displays a landscape image on the display with the same magnification as the still image taken. The same magnification landscape image output means is added. The common structural requirements with the sixteenth embodiment are a landscape image information holding unit, a viewpoint information acquisition unit, and a landscape image information output unit for the landscape server device, and a position information acquisition unit and a position information acquisition unit for the mobile terminal device. , a line-of-sight information acquisition unit, a viewpoint information output unit, a landscape image information acquisition unit, a landscape image information output unit, and the line-of-sight direction included in the line-of-sight information provided in the line-of-sight information acquisition unit is retained. A line-of-sight direction sensor configured to have a line-of-sight direction, a line-of-sight direction acquisition means, a line-of-sight direction holding means, a holding command outputting means, a holding cancellation command outputting means, and a magnification acquisition included in a camera having a zoom-in/zoom-out function. and a landscape image output means with the same magnification included in the landscape image information output section, and a description of the common parts will be omitted.
<Explanation of Embodiment 5C Configuration>
<Explanation of Embodiment 5C Camera>
The "camera" has a zoom-in/zoom-out function that can be operated by the user, and can perform magnification photography by the user's operation.
<Embodiment 5C Magnification acquisition unit>

The "magnification acquisition unit" acquires the magnification of the landscape still image at the time of photography by the photography unit at the time of the hold command. When the user uses the zoom-in/zoom-out function of the camera to maintain the viewing direction of the mobile terminal device, it is possible to obtain the magnification of the landscape still image at the time of photography by the photography unit at the time of the retention command. As shown in FIG. 34(A), an example is shown in which double zoom photography is performed when the viewing direction of the mobile terminal device is maintained. Assuming that a building and part of a five-storied pagoda are photographed behind it, the magnification is acquired from the magnification acquisition unit, and the user's position coordinates, the position coordinates of the object to be displayed in the landscape image, and the size of the object are known in advance. Therefore, with 2x zoom, it is possible to calculate at what scale the landscape image should be displayed on the mobile terminal device.
<Embodiment 5C Same magnification landscape image output means>
The "same magnification landscape image output means" is included in the landscape image information output section and outputs a landscape image having the same magnification as the photographed still image to the display. As shown in FIG. 34(B), since the coordinate position and size of the landscape image are known, a past landscape image of 972 A.D., for example, as shown in FIG. 16, with the same magnification as the photographed landscape still image is output. do.
<Embodiment 5D>
<Embodiment 5D Overview>

The mobile terminal device of the present embodiment includes an alternating output unit capable of alternately outputting landscape still images photographed on the display and landscape images of the same magnification. The feature is that landscape images are output alternately.
<Embodiment 5D Configuration>

The configuration of Embodiment 5D is based on the configuration of Embodiment 5C, with the addition of an alternate output unit that can alternately output landscape still images photographed on the display and landscape images of the same magnification.
The common structural requirements with the seventeenth embodiment are a landscape image information holding unit, a viewpoint information acquisition unit, and a landscape image information output unit for the landscape server device, and a position information acquisition unit and a position information acquisition unit for the mobile terminal device. , a line-of-sight information acquisition unit, a viewpoint information output unit, a landscape image information acquisition unit, a landscape image information output unit, and the line-of-sight direction included in the line-of-sight information provided in the line-of-sight information acquisition unit is retained. A line-of-sight direction sensor configured to have a line-of-sight direction, a line-of-sight direction acquisition means, a line-of-sight direction holding means, a holding command outputting means, a holding cancellation command outputting means, and a magnification acquisition included in a camera having a zoom-in/zoom-out function. and a landscape image output means with the same magnification included in the landscape image information output section, and a description of the common parts will be omitted.
<Explanation of Embodiment 5D Configuration>

The "alternate output unit" alternately outputs the photographed scenery still image and the scenery image of the same magnification to the display. Thereby, landscape still images and landscape images can be viewed while being compared alternately.
<Embodiment 5E>
<Embodiment 5E Overview>

The mobile terminal device of the present embodiment includes a superimposition output unit capable of superimposing and outputting a landscape still image photographed on the display and a landscape image of the same magnification, and a landscape still image photographed on the display and a landscape image of the same magnification are added. The feature is that the output is superimposed with the landscape image.
<Embodiment 5E Configuration>

The configuration of Embodiment 5E is based on the configuration of Embodiment 5C or 5D, with the addition of a superimposition output unit that can superimpose and output a landscape still image photographed on the display and a landscape image of the same magnification. The common components with Embodiment 5C or 5D are a landscape image information holding unit, a viewpoint information acquisition unit, and a landscape image information output unit for the landscape server device, and a position information acquisition unit for the mobile terminal device. The line of sight information acquisition unit, the line of sight information output unit, the landscape image information acquisition unit, the landscape image information output unit, and the line of sight direction included in the line of sight information provided in the line of sight information acquisition unit are retained. a camera having a zoom-in/zoom-out function; a gaze direction sensor configured to have a gaze direction in which the gaze direction is set; They are a magnification acquisition section, a same magnification landscape image output means included in the landscape image information output section, and an alternate output section, and the explanation of the common parts will be omitted.
<Embodiment 5E Configuration Description>

The "superimposition output section" superimposes and outputs a scenery still image photographed on the display and a scenery image of the same magnification. This allows a mix of still landscape images and past landscape images to be displayed on the display, providing travelers and social studies visitors with a new experience that combines the present and the past.

In this manner, according to the embodiment, a traveler can actually go to a tourist spot and view landscape image information according to the traveler's viewpoint information together with additional information. In addition, the user can go to a town on a social studies field trip or the like and view landscape information from a desired viewpoint (for example, a child's viewpoint).

Claims (1)

A street viewer system comprising a landscape server device and a mobile terminal device,
The landscape server device is
Landscape image information, which is an image showing a past landscape obtained by combining a current landscape, which is a live-action image, and a past landscape, by a computer, in association with viewpoint information consisting of position information and line-of-sight information, which will be described later, is associated with the position information. a landscape image information holding unit that holds;
a viewpoint information acquisition unit that acquires viewpoint information from a mobile terminal device;
a server-side timing information acquisition unit that acquires timing information, which is information indicating a past period, from the mobile terminal device;
a landscape image information output unit that transmits landscape image information associated with the acquired viewpoint information to a mobile terminal device;
moreover,
The landscape image information holding unit has a period-specific landscape image information holding means that holds period-specific landscape image information that is landscape image information for each of a plurality of past periods;
The landscape image information output unit includes a period-specific landscape image information output means for outputting period-specific landscape image information of a period corresponding to the acquired period information to the mobile terminal device,
The mobile terminal device is
a location information acquisition unit that acquires location information that is information about the current location of the mobile terminal device;
a line-of-sight information acquisition unit that acquires line-of-sight information that is information indicating a line-of-sight direction in the acquired position information and/or viewpoint height information that can be changed by a change input;
a period information acquisition unit on the mobile terminal side that acquires period information using a slide bar displayed together with the historical chronology ;
a viewpoint information output unit that outputs viewpoint information in which the acquired position information and line of sight information are associated to the landscape server device;
a landscape image information acquisition unit that has a function of inputting a change in the height of a viewpoint and acquires landscape image information from a landscape server device according to the output viewpoint information;
a landscape image information output unit that outputs the acquired landscape image information;
and a timing information transmission unit that transmits the acquired timing information to the landscape server device,
moreover,
A street viewer system comprising a period-specific landscape image information acquisition means for acquiring period-specific landscape image information according to period information transmitted to a landscape image information acquisition section.

Images