JP2011039130A - Linked display device, linked display method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a linked display device where the operation of the other can be automatically carried out in accordance with user's operation of one of a map and a scene. <P>SOLUTION: The linked display device includes: an operation receiving section 11 receiving user's operation; a map image acquisition section 13 acquiring a map image according to the operation of a map by using map information; a scene image acquisition section 15 acquiring a scene image according to the operation of a scene by using scene information; a display section 16 displaying the acquired map image and the scene image; an operation history accumulation section 17 accumulating the history of operation; a correspondence information storage section 18 storing correspondence information where operation pattern information indicating patterns of two or more operations of one of the map and the scene are made to correspond to the operation information indicating the operation of the other; and an operation information acquisition section 19 acquiring the operation information corresponding to the operation pattern information when the accumulated history of operation coincides with the operation pattern information. Thus, the operation according to the acquired operation information is performed. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a cooperative display device that displays a map and a landscape.

  Conventionally, a two-dimensional map can be displayed on an online map, a car navigation system, or the like. In addition, a technique capable of displaying a three-dimensional landscape is also known. The three-dimensional landscape may be, for example, an image obtained by photographing a real space with a camera. As a system for displaying a three-dimensional landscape, for example, street view, location view, Patent Document 1 and the like are known.

JP 2007-226580 A

  However, in the conventional cooperative display device, even when a two-dimensional map and a three-dimensional landscape are displayed at the same time, the two cannot be appropriately coordinated. For example, even if you are paying attention to a certain point on a two-dimensional map, the three-dimensional landscape does not change accordingly. There wasn't.

  Generally speaking, when displaying a two-dimensional map and a three-dimensional landscape, each of them must be operated separately, and complicated operations are imposed on the user.

  The present invention has been made in order to solve the above-described problems, and is a cooperative display that can automatically operate the other in response to a user operation on one of a two-dimensional map and a three-dimensional landscape. An object is to provide a device or the like.

  In order to achieve the above object, a cooperative display device according to the present invention uses an operation accepting unit that accepts a user's operation and map information that is information about the map, and a map image corresponding to an operation on the map accepted by the operation accepting unit. The map image acquisition unit that acquires the image, the landscape image acquisition unit that acquires the landscape image corresponding to the operation received by the operation reception unit using the landscape information that is information about the landscape, and the map image acquisition unit acquired Two or more for one of the display unit that displays the map image and the landscape image acquired by the landscape image acquisition unit, the operation history storage unit that stores two or more history of operations received by the operation reception unit, and the map and landscape Correspondence information is stored that associates operation pattern information that is information indicating the operation pattern of the operation information with operation information that is information indicating the operation on the other. And an operation information acquisition unit that acquires operation information corresponding to the operation pattern information when the operation history accumulated by the operation history accumulation unit matches the operation pattern information. The operation according to the acquired operation information is executed.

  With such a configuration, when an operation indicated by the operation pattern information is performed on one of the map and the landscape, an operation indicated by the operation information corresponding to the operation pattern information is performed on the other. become. Therefore, by appropriately configuring the correspondence information, an appropriate operation corresponding to the operation performed on one of the map and the landscape is automatically performed on the other, and the user However, the user's load is reduced as compared with the case where the operation is manually performed on both.

In the cooperative display device according to the present invention, the operation history storage unit stores the history of operations on the map, and the correspondence information storage unit stores at least correspondence information that associates the operation pattern information on the map with the operation information on the scenery. The landscape image acquisition unit may acquire a landscape image according to the operation information for the landscape acquired by the operation information acquisition unit.
With such a configuration, it is possible to automatically execute an appropriate operation on the landscape according to the operation performed on the map.

In the cooperative display device according to the present invention, the operation on the map accepted by the operation accepting unit may be at least one of position movement, zoom-in, and zoom-out.
With such a configuration, for example, when two or more operations of moving a position, zooming in, and zooming out are performed on a map, an appropriate operation corresponding to the operation is automatically performed on the scenery. Will be.

In the cooperative display device according to the present invention, the operation information may include an operation for changing the viewing direction.
With such a configuration, for example, the line-of-sight direction of the scenery is automatically changed according to an operation on the map.

  In the cooperative display device according to the present invention, in the correspondence information storage unit, (1) operation pattern information indicating one or more movements with respect to the map, one or more subsequent zoom-ins, and subsequent zero or more movements; Corresponding information for associating operation information for changing the line-of-sight direction to a zoom-in point with respect to the scenery, (2) one or more movements with respect to the map, one or more subsequent zoom-ins, one or more subsequent zoom-outs, and then Corresponding information for associating operation pattern information indicating zero or more movements with operation information for changing the line-of-sight direction to the reverse direction of the zoom-in point with respect to the scenery, (3) one or more zoom-in with respect to the map, and thereafter Operation pattern information indicating that the map image is no longer included in the map image due to the last movement in one or more movements. If, for views, at least one or more association information for associating the operation information to change the viewing direction in the direction of the last movement may be stored.

In the cooperative display device according to the present invention, the operation history storage unit stores the history of operations on the scenery, and the correspondence information storage unit stores at least correspondence information that associates the operation pattern information on the scenery with the operation information on the map. Then, the map image acquisition unit may acquire a map image according to the operation information for the map acquired by the operation information acquisition unit.
With such a configuration, it is possible to automatically execute an appropriate operation corresponding to the operation performed on the scenery on the map.

In the cooperative display device according to the present invention, the operation on the scenery received by the operation receiving unit may be at least one of position movement, zoom-in, zoom-out, and change of the line-of-sight direction.
With such a configuration, for example, when two or more operations are performed on the scenery, such as moving the position, zooming in, zooming out, or changing the line-of-sight direction, an appropriate action corresponding to the operation is performed on the map. Will be executed automatically.

  In the cooperative display device according to the present invention, in the correspondence information storage unit, (1) operation pattern information indicating one or more changes to the upward direction of the line of sight with respect to the scenery, and one or more subsequent zoom-ins, Corresponding information for associating the operation information to be zoomed out with respect to the map, (2) One or more changes in the gaze direction to the right, left, or down with respect to the scenery, and one or more subsequent zoom-ins. Correspondence information for associating operation pattern information with operation information for zooming in on a map, (3) two or more operation pattern information for a landscape, indicating a change in the right or left direction of the line of sight, and for a map, At least one or more correspondence information that associates two or more pieces of operation information that moves in the line-of-sight direction after the last change in the line-of-sight direction to the right or left direction There may be stored.

Further, in the cooperative display device according to the present invention, the map image acquisition unit acquires a map image by accessing a server holding map information, and the landscape image acquisition unit is connected to a server holding landscape information. A landscape image may be acquired by accessing.
With such a configuration, for example, a map image or a landscape image can be acquired via the Internet or the like.

The cooperative display device according to the present invention further includes a map information storage unit that stores map information and a landscape information storage unit that stores landscape information, and the map image acquisition unit accesses the map information storage unit. The landscape image acquisition unit may acquire the landscape image by accessing the landscape information storage unit.
With such a configuration, for example, as in a car navigation system, a map image and a landscape image can be acquired using map information and the like that the system has.

In the cooperative display device according to the present invention, the scenery may be a real space photographed with a camera.
With such a configuration, it is possible to display a map image and a live-action three-dimensional landscape image (for example, the aforementioned street view or location view image) in cooperation with each other.

  Further, the cooperative display method according to the present invention includes two or more operations for one of the operation reception unit, the map image acquisition unit, the landscape image acquisition unit, the display unit, the operation history storage unit, the map and the landscape. Processing is performed using a correspondence information storage unit that stores correspondence information that is information for associating operation pattern information that is information indicating a pattern and operation information that is information indicating an operation on the other, and an operation information acquisition unit. According to the linked display method, the operation reception unit receives an operation of the user by the operation reception step, and the map image acquisition unit uses the map information that is information about the map, according to the operation on the map received in the operation reception step. The map image acquisition step for acquiring the map image and the landscape image acquisition unit for the landscape received in the operation reception step using the landscape information that is information about the landscape. A landscape image acquisition step for acquiring a landscape image corresponding to the operation to be performed, a display step for displaying the map image acquired in the map image acquisition step and the landscape image acquired in the landscape image acquisition step, and an operation history storage The operation history storage step in which the operation history storage step stores two or more operation histories received in the operation reception step, and the operation information acquisition unit stores the operation histories stored in the operation history storage step. An operation information acquisition step for acquiring operation information corresponding to the operation pattern information, and an operation according to the operation information acquired in the operation information acquisition step is executed.

  According to the cooperation display apparatus etc. by this invention, the other can be automatically operated according to the user's operation with respect to one of a two-dimensional map and a three-dimensional landscape.

The block diagram which shows the structure of the cooperation display apparatus by Embodiment 1 of this invention. The flowchart which shows operation | movement of the cooperation display apparatus by the embodiment The flowchart which shows operation | movement of the cooperation display apparatus by the embodiment The figure which shows an example of the map information in the embodiment The figure which shows an example of the landscape information in the embodiment The figure which shows an example of the correspondence information in the embodiment The figure which shows an example of the map operation history information in the embodiment The figure which shows an example of a display of the map image and landscape image in the embodiment The figure which shows an example of a display of the map image and landscape image in the embodiment The figure which shows an example of a display of the map image and landscape image in the embodiment The figure which shows an example of a display of the map image and landscape image in the embodiment The figure which shows an example of a display of the map image and landscape image in the embodiment The figure which shows an example of the scenery operation history information in the embodiment The figure which shows an example of a display of the map image and landscape image in the embodiment The figure which shows an example of a display of the map image and landscape image in the embodiment The figure which shows an example of a display of the map image and landscape image in the embodiment The figure which shows an example of a display of the map image and landscape image in the embodiment Schematic diagram showing an example of the appearance of the computer system in the embodiment The figure which shows an example of a structure of the computer system in the embodiment

  Hereinafter, a cooperative display device according to the present invention will be described using embodiments. In the following embodiments, components and steps denoted by the same reference numerals are the same or equivalent, and repetitive description may be omitted.

(Embodiment 1)
A cooperative display apparatus according to Embodiment 1 of the present invention will be described with reference to the drawings. The cooperative display apparatus according to the present embodiment automatically operates the other according to the user's operation on one of the map and the scenery.

  FIG. 1 is a block diagram showing a configuration of a cooperative display device 1 according to the present embodiment. The cooperative display device 1 according to the present embodiment includes an operation reception unit 11, a map information storage unit 12, a map image acquisition unit 13, a landscape information storage unit 14, a landscape image acquisition unit 15, a display unit 16, An operation history storage unit 17, a correspondence information storage unit 18, and an operation information acquisition unit 19 are provided.

  The operation reception unit 11 receives a user operation. This operation may be an operation on a map or an operation on a landscape. A landscape is a two-dimensional display of objects such as landscapes and objects in a three-dimensional space. For example, the scenery may be a photograph of a subject in a three-dimensional real space with an actual camera, and may be on a three-dimensional virtual space (for example, a space constituted by three-dimensional computer graphics). A two-dimensional image generated (rendered) from an arranged model or the like may be used. When the operation receiving unit 11 receives an operation, it is preferable that it is clear whether the operation is for a map or a landscape. The operation on the map is, for example, movement of the map position, zoom-in, zoom-out, or the like. The movement of the position may be, for example, a movement of a relative position (for example, movement to the right, left, up, or down with respect to the displayed map), and movement by designation of an absolute position (for example, It may be a movement specifying the coordinates after the movement). The movement of the position may include an operation of changing a desired position to the center of the display (this operation may be referred to as “centering”). The operations on the scenery include, for example, movement of the scenery position, zooming in, zooming out, and changing the line of sight. The movement of the position may be, for example, a movement of a relative position (for example, movement to an adjacent landscape with respect to the displayed map), or movement by designation of an absolute position (for example, coordinates after movement). It may be a movement specified by. The gaze direction change operation may be divided into, for example, a gaze direction upward change operation, a downward change operation, a right change operation, and a left change operation. It may be an operation by inputting an azimuth angle and a zenith angle (or may be an elevation angle), or may be another operation. The azimuth is an angle indicating a direction (azimuth) in the XY coordinate plane in an XYZ orthogonal coordinate system in which the map plane is the XY coordinate plane and the Z axis is the zenith direction. The azimuth angle takes a value of 0 ° or more and less than 360 °. That is, the azimuth is a remainder of 360 °. In the present embodiment, when the azimuth is used, 0 ° is north, and the azimuth is increased clockwise. That is, the azimuth angle is 90 ° in the east direction and the azimuth angle is 180 ° in the south direction. The zenith angle is an angle with the Z axis, and takes a value of 0 ° or more and 180 ° or less. If the zenith angle is 0 °, the orientation is directly above (the direction of the zenith), if the zenith angle is 90 °, the orientation is horizontal, and if the zenith angle is 180 °, the orientation is the center of the earth. . In addition, for operations on the map and scenery, for example, using a GUI (Graphical User Interface) such as clicking a displayed button, dragging a slider, or clicking or dragging a displayed map image or scenery image. It may also be performed, or may be performed by pressing up / down / left / right buttons such as a keyboard or inputting a command corresponding to an operation. These operations are known as operations of a two-dimensional online map and an electronic map, and operations of a system that displays a three-dimensional landscape (for example, the above-described street view and location view), and will not be described in detail.

  For example, the operation receiving unit 11 may receive an operation input from an input device (for example, a keyboard, a mouse, a touch panel, etc.), or may receive an operation transmitted via a wired or wireless communication line. . Note that the operation reception unit 11 may or may not include a device (for example, a modem or a network card) for reception. In addition, the operation reception unit 11 may be realized by hardware, or may be realized by software such as a driver that drives a predetermined device.

  The map information storage unit 12 stores map information. Map information is information relating to a map. Further, a map image is acquired by the map image acquisition unit 13 described later using this map information. Therefore, as long as the map image according to the operation received by the operation receiving unit 11 can be acquired from the map information, the content of the map information does not matter.

  The map information may be map image information, for example. This image information may be raster data (bitmap data) or vector data, for example. When the image information is raster data, the map information may include image information corresponding to a plurality of scales. For example, map information may include large-scale image information, medium-scale image information, and small-scale image information for the same region. Moreover, map information shows the map divided | segmented into the tile shape, The map of various area | regions can be displayed now by combining suitably. Here, the “map” may be a topographic map, a topographic map, a geological map, a land use map, a house map, a route map, a road map, a guide map, or the like. In addition, the “map” may be an aerial photograph or satellite photograph in which the topography, roads, etc. can be grasped, or those in which symbols or characters are entered. In the map information, it is preferable that the correspondence between each point on the map and the coordinates can be known. The coordinate may be a coordinate with a reference point as the origin (this coordinate may be a distance, for example), may be latitude / longitude, or may be information that can identify other positions. . The coordinates themselves may be included in the map information. The map information is known as a two-dimensional online map, an electronic map, etc., and detailed description thereof is omitted.

  The process in which map information is memorize | stored in the map information storage part 12 is not ask | required. For example, map information may be stored in the map information storage unit 12 via a recording medium, or map information transmitted via a communication line or the like may be stored in the map information storage unit 12. It may be. The storage in the map information storage unit 12 may be temporary storage in a RAM or the like, or may be long-term storage. The map information storage unit 12 can be realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, etc.).

  The map image acquisition unit 13 uses the map information stored in the map information storage unit 12 to acquire a map image corresponding to an operation on the map. The operation for this map is accepted by the operation accepting unit 11. For example, when the operation reception unit 11 receives an operation for displaying a map of a certain area, the map image acquisition unit 13 acquires a map image of the area in response to the operation. For example, when the map information is raster data, the map image acquisition unit 13 may acquire the map information by cutting out the map image of the area. At that time, map information corresponding to the instructed scale may be used. Specifically, the map image acquisition unit 13 determines the size of the displayed map image (for example, vertical length, horizontal length, etc.) using the scale of the map image to be displayed on the map. It may be obtained by converting into a length and cutting out a specified area using the length. Further, for example, when the map information is vector data, the map image acquisition unit 13 enlarges / reduces the map image according to the specified scale, and cuts out the map image of the specified area. May be. Further, for example, when the operation accepting unit 11 accepts an operation of moving a map, the map image acquiring unit 13 responds to the operation, and the map image obtaining unit 13 performs a map corresponding to the moved area in the same manner as described above. An image may be acquired. In the movement operation, the movement in a certain direction (for example, right or north) is to acquire a map image of a point moved in that direction from the currently displayed map image. For example, the movement to the right is to acquire a map image whose center position is a position on the right side of the center position of the currently displayed map image. Further, for example, when the operation accepting unit 11 accepts an operation of zooming in or zooming out, the map image acquiring unit 13 responds to the operation by zooming in or zooming in the same region in the same manner as the above-described processing. You may acquire the map image after out. Note that a position of interest, which is a position that does not change during zoom-in or zoom-out, may be received. For example, if you can zoom in and out using the mouse wheel, the position of the mouse pointer on the map image is accepted as the position of interest, and zoom in and zoom according to the amount of change in the mouse wheel You may accept an operation of out. In that case, the position of the target position is not changed before and after zooming. These processes are known as processes for acquiring a map image in a two-dimensional online map, an electronic map, etc., and detailed description thereof is omitted. The map image acquisition unit 13 also performs a process of acquiring a map image according to operation information described later for the map acquired by the operation information acquisition unit 19 described later. The operation indicated by this operation information is the same as the operation received by the operation receiving unit 11.

  In the present embodiment, the case where the map image acquisition unit 13 acquires a map image by accessing the map information storage unit 12 will be described, but this need not be the case. For example, the map image acquisition unit 13 may acquire a map image by accessing a server (not shown) that holds map information. In that case, for example, the map image acquisition unit 13 may transmit a region or scale of a desired map image to the server and receive the map image accordingly. When the map information is vector data, the map image acquisition unit 13 receives map information corresponding to the transmitted area from the server, and uses the received map information to map corresponding to a desired scale. An image may be generated. These processes are known as processes for acquiring a map image in a two-dimensional online map, and detailed description thereof is omitted. Further, when the map image acquisition unit 13 acquires a map image by accessing the server, the cooperative display device 1 may not include the map information storage unit 12, and the map image acquisition unit 13 may May be equipped with a device.

  In the scenery information storage unit 14, scenery information is stored. The scenery information is information regarding scenery. In addition, a landscape image is acquired by the landscape image acquisition unit 15 described later using this landscape information. Therefore, as long as the landscape image corresponding to the operation accepted by the operation accepting unit 11 can be acquired from the landscape information, the content of the landscape information is not limited.

  The scenery information may be, for example, scenery image information associated with discrete coordinates. In this embodiment, this case will be described. For example, street view, location view, and Patent Document 1 are as described above. This image information may be raster data or vector data, for example. When the scenery is a real space photographed with a camera, this image information is raster data. The image information may be, for example, panoramic image information corresponding to all azimuth angles, hemispherical image information disclosed in Patent Document 1, or full spherical image information. . When the image information is raster data, the landscape information may include image information corresponding to a plurality of zoom levels. For example, for the same position, zoomed-in image information, standard image information, and zoomed-out image information may be included in the landscape information. The discrete coordinates may be coordinates with a certain reference point as the origin, may be latitude / longitude, or may be information that can identify other positions. The coordinates itself may be included in the scenery information. The scenery information is known in the above-described street view and location view, Patent Document 1, and the like, and detailed description thereof is omitted.

  The scenery information may be, for example, information on three-dimensional (3D) graphics. The scene information includes shape information (model information) indicating a plurality of shapes composed of polygons, texture information indicating a surface image of the shape indicated by the shape information, or other 3D graphics information. May be included.

  The process in which the scenery information is stored in the scenery information storage unit 14 does not matter. For example, the scenery information may be stored in the scenery information storage unit 14 via a recording medium, or the scenery information transmitted via a communication line or the like may be stored in the scenery information storage unit 14. It may be. Storage in the scenery information storage unit 14 may be temporary storage in a RAM or the like, or may be long-term storage. The scenery information storage unit 14 can be realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, etc.).

  The landscape image acquisition unit 15 acquires a landscape image corresponding to the operation on the landscape, using the landscape information that is information about the landscape. The scenery image acquired by the scenery image acquisition unit 15 becomes a scenery image for display. The operation on the scenery is received by the operation receiving unit 11. For example, when the operation receiving unit 11 receives an operation for displaying a landscape in a certain direction at a certain position, the landscape image acquiring unit 15 responds to the operation by the landscape image acquiring unit 15 according to the direction in that position. To get. For example, when the landscape information is image information of a landscape associated with discrete coordinates, the landscape image acquisition unit 15 performs image information corresponding to the coordinates according to the position (for example, panoramic image information or , Hemispherical and full spherical image information) may be obtained by cutting out an image corresponding to that direction. At this time, if the image information is raster data, image information corresponding to the degree of zooming may be used. Further, when the image is cut out, deformation for removing distortion of the image may be performed. Further, when the image information is vector data, the image information may be enlarged or reduced according to the degree of zooming. Further, for example, when the operation accepting unit 11 accepts an operation of moving the position, the landscape image acquiring unit 15 uses the image information corresponding to the position after the movement in response to the operation. It may be obtained by cutting out a landscape image that is an image corresponding to the direction. Further, for example, when the operation accepting unit 11 accepts an operation of zooming in or zooming out, in response to the operation, the landscape image acquiring unit 15 performs image information corresponding to the same position in the same manner as the above-described processing. May be used to acquire a landscape image after zooming in or zooming out. Further, for example, when the operation receiving unit 11 receives an operation for changing the line-of-sight direction, the landscape image acquisition unit 15 responds to the operation by performing image information corresponding to the same position in the same manner as the above-described processing. May be used to acquire a landscape image corresponding to the changed line-of-sight direction. These processes are known as a street view, a location view, and a process for acquiring a landscape image in Patent Document 1, and detailed description thereof is omitted.

  Further, for example, when the scenery information is 3D graphics information, the scenery image acquisition unit 15 determines the viewpoint and the line-of-sight direction using the designated position and the designated direction, and uses them. The scenery image may be acquired by performing rendering using the image. Each time the operation reception unit 11 receives an operation on a landscape, the landscape image acquisition unit 15 performs rendering according to the line of sight and the line of sight after the operation, and acquires a landscape image. This rendering is known as 3D graphics processing, and a detailed description thereof will be omitted.

  The scenery image acquisition unit 15 also performs processing for acquiring a scenery image according to operation information for the scenery acquired by the operation information acquisition unit 19 described later. The operation indicated by this operation information is the same as the operation received by the operation receiving unit 11.

  In this embodiment, the case where the landscape image acquisition unit 15 acquires the landscape image by accessing the landscape information storage unit 14 will be described, but this need not be the case. For example, the scenery image acquisition unit 15 may acquire a scenery image by accessing a server (not shown) that holds scenery information. In that case, for example, the landscape image acquisition unit 15 may transmit the position, line of sight, and zoom level of the desired landscape image to the server, and acquire the landscape image accordingly. When the image information included in the landscape information is vector data, the landscape image acquisition unit 15 receives the landscape information corresponding to the transmitted position and line of sight from the server, and uses the received landscape information, A landscape image corresponding to a desired degree of zoom may be generated. These processes are known as a street view, a location view, and a process for acquiring a landscape image in Patent Document 1, and detailed description thereof is omitted. When the landscape image acquisition unit 15 acquires a landscape image by accessing the server, the cooperative display device 1 may not include the landscape information storage unit 14, and the landscape information storage unit 14 May be equipped with a device.

  The display unit 16 displays the map image acquired by the map image acquisition unit 13 and the landscape image acquired by the landscape image acquisition unit 15. The display unit 16 preferably displays both images together. Displaying together may be, for example, displaying side by side in the horizontal, vertical, or diagonal directions, or may include displaying the other image inside one image. In the latter case, the image displayed inside is assumed to be smaller than the other image. In addition, the display of a map image or the like may be performed at a place away from the cooperative display device 1. In that case, the display unit 16 may transmit a map image and a landscape image for display to the outside of the cooperative display device 1. Further, the display unit 16 may or may not include a display device (for example, a CRT, a liquid crystal display, or a projector) that performs the display. The display unit 16 may be realized by hardware, or may be realized by software such as a driver that drives a display device.

  The operation history storage unit 17 stores two or more operation histories received by the operation reception unit 11 in a predetermined recording medium. The recording medium is, for example, a semiconductor memory, an optical disk, a magnetic disk, or the like, and may be included in the operation history storage unit 17 or may exist outside the operation history storage unit 17. Further, this recording medium may or may not temporarily store the operation history.

  The operation history storage unit 17 stores the history of operations on the map and the history of operations on the scenery in a distinguishable manner. For example, the operation history storage unit 17 may separately store map operation history information that is information indicating a history of operations on a map and landscape operation history information that is information indicating a history of operations on a landscape. The map operation history and the landscape operation history may be distinguished from each other by setting a flag or the like when accumulating. In the present embodiment, the former case will be described. Here, in the history of operations on the map, the operation ID for identifying the operation, the center position of the map image displayed after the operation, and the zoom-in and zoom-out are unchanged when the zoom-in and zoom-out are performed. The attention position that is the position to be and the scale of the map image displayed after the operation may be accumulated. The operation ID may identify, for example, movement, zoom in, and zoom out. The center position and the target position may be indicated by coordinates such as latitude and longitude. For example, when an operation of dragging on the map image or an operation of clicking the center position of the moved map image is accepted, the operation history storage unit 17 may store an operation ID indicating movement. Further, for example, when a zoom-in or zoom-out operation specifying a position of interest for a map image is accepted, the operation history accumulation unit 17 accumulates the position of interest and an operation ID indicating zoom-in or zoom-out. May be. For example, when a move button for a map image is clicked, the operation history accumulation unit 17 may accumulate an operation ID indicating movement or the like. Further, in the history of operations on the scenery, an operation ID for identifying the operation, a position associated with image information corresponding to the landscape image displayed after the operation, and an angle indicating the line-of-sight direction after the operation, The degree of zoom may be accumulated. The operation ID may identify, for example, movement, zoom-in, zoom-out, upward line-of-sight change, downward line-of-sight change, left line-of-sight change, and right line-of-sight change. The position may be indicated by coordinates such as latitude and longitude. The angle may be indicated by, for example, an azimuth angle and a zenith angle (or elevation angle). For example, when a move button corresponding to a landscape image is clicked, the operation history accumulation unit 17 may accumulate an operation ID indicating movement or the like. For example, when a drag operation on a map image is accepted, the operation history storage unit 17 may store an operation ID indicating a change in the line-of-sight direction that has changed in accordance with the drag.

  In addition, when the operation reception unit 11 receives the operation ID itself, the operation history storage unit 17 stores the operation ID accordingly. On the other hand, when the operation reception unit 11 receives the coordinates of the changed center position and the changed angle, the operation receiving unit 11 determines what operation has been performed in accordance with the change and acquires the operation ID. To do. For example, when the scale is changed in the operation on the map and the zoom is changed in the zoom-in direction, the operation history accumulation unit 17 determines that the zoom-in (zoom-out) has been performed. . For example, when the center position is changed in the operation on the map, the operation history accumulation unit 17 determines that the movement has been performed. Further, for example, when the zoom level is changed in the operation on the scenery and the zoom level is changed in the zoom-in (zoom-out) direction, the operation history storage unit 17 performs zoom-in (zoom-out). Judge that it was broken. Further, for example, when the position is changed in the operation on the scenery, the operation history accumulation unit 17 determines that the movement has been performed. Further, for example, when the azimuth angle is set to increase clockwise in the operation on the landscape and the azimuth angle is changed to increase (decrease), the operation history storage unit 17 It is determined that the line of sight has been changed to the right (left). Further, for example, when the zenith angle is changed to increase (decrease) in the operation on the scenery, the operation history accumulation unit 17 determines that the line of sight is changed downward (up). In these cases, the operation history storage unit 17 uses the information that associates the operation content received by the operation reception unit 11 and the operation ID, and changes the operation content (for example, change in zenith angle). You may convert into ID.

  The correspondence information storage unit 18 stores correspondence information. The correspondence information is information for associating operation pattern information, which is information indicating two or more operation patterns for one map, and operation information, which is information indicating an operation for the other, with respect to a map and a landscape. The two or more operation patterns in the operation pattern information are usually patterns including the sequence (sequence) of the operations. For example, the operation pattern information may be a sequence of a plurality of operation IDs. Further, the operation pattern information may be described using a regular expression. Further, the operation pattern information may include conditions relating to a target position other than the operation ID. Further, the operation information may indicate, for example, movement, zoom-in, zoom-out, and change of the line-of-sight direction. The operation information may be, for example, one operation ID, and may include a parameter indicating how much the operation identified by the operation ID is performed in addition to the operation ID. It may directly indicate the coordinates and the angle of the line-of-sight direction.

  Here, the correspondence information will be further described. The operation pattern information is information indicating an operation pattern that allows the user's intention according to the operation to be known when an operation is performed on the map or the scenery. Corresponding information is information in which operation information indicating an operation according to the user's intention is associated with the operation pattern information. Therefore, the operation pattern information and the operation information are associated with each other through the user's intention according to the operation.

  For example, the following can be considered as correspondence information in which operation pattern information indicating a pattern of an operation on a map is associated with operation information on a landscape.

(1) Operation pattern information indicating one or more movements with respect to the map, one or more subsequent zoom-ins, and subsequent zero or more movements, and operation information for changing the line-of-sight direction to a zoom-in point with respect to the scenery Corresponding correspondence information In this case, since a specific position is zoomed in after moving the map, it can be assumed that the user is interested in the zoomed-in position. Therefore, it is considered that an appropriate operation is to turn the line-of-sight direction to a position of interest of the user even in the scenery.

(2) Operation pattern information indicating one or more movements with respect to the map, one or more subsequent zoom-ins, one or more subsequent zoom-outs, and subsequent zero or more movements, and zoom in on the gaze direction with respect to the scenery Corresponding information that associates the operation information that changes in the opposite direction of the point in this case In this case, because you are zoomed in and out after moving the map, you are interested in the zoomed in position, but actually zoomed in As a result, it was not the place that I had expected, so I can assume that I decided to zoom out and search for a new position. Therefore, it is considered that an appropriate operation in the scenery is to turn the line-of-sight direction in a direction other than the position where the user loses interest, particularly in the opposite direction (opposite direction) to the position.

(3) Operation pattern information indicating that the map image is no longer included in the map image due to one or more zoom-ins to the map and one or more subsequent movements, and the last movement in the one or more movements. Corresponding information that associates the operation information for changing the line-of-sight direction to the direction of the last movement with respect to the scenery. In this case, the map image does not include the zoomed-in position, that is, the position that the user is considered interested Therefore, it can be assumed that the user is searching for a new position. Therefore, it is considered that an appropriate operation is to turn the line-of-sight direction in the direction in which the user is searching, that is, the last movement direction.

  Needless to say, the correspondence information in which the operation pattern information indicating the operation pattern for the map is associated with the operation information for the scenery may include other information.

  Next, for example, the following can be considered as correspondence information in which operation pattern information indicating a pattern of an operation on a landscape is associated with operation information on a map.

(1) Correspondence information associating operation pattern information indicating one or more changes in the line-of-sight direction with respect to the landscape and one or more subsequent zoom-in operations and operation information for zooming out the map in this case Can be assumed that the user is interested in a distant view. Therefore, in order to display a map including the distant view, it is considered that zooming out the map is an appropriate operation.

(2) Operation pattern information indicating one or more changes in the gaze direction to the right, left, or downward with respect to the scenery, and subsequent one or more zoom-ins, and operation information for zooming in on the map Corresponding Corresponding Information In this case, it can be assumed that the user is interested in the near view, particularly in the vicinity of the position corresponding to the scenery. Therefore, it is considered that an appropriate operation is to zoom in on the map in order to display detailed information of the near view. Note that “one or more changes to the right direction, left direction, or downward direction of the line-of-sight direction” means that the line-of-sight direction changes to the right direction, the line-of-sight direction changes to the left direction, or the line-of-sight direction downwards. Means any combination of one or more of the changes.

(3) In the operation pattern information indicating a change of two or more gaze directions to the right or left with respect to the scenery, and to the map, the two or more gaze direction changes to the right or left Corresponding information that associates the operation information that moves in the line-of-sight direction after the last change In this case, it can be assumed that the user is searching for the moving direction by checking the surrounding scenery. Therefore, it is considered that an appropriate operation is to move the map so that the range of the last line-of-sight direction is displayed more, that is, to move the map in the last line-of-sight direction. Note that “two or more changes to the right or left direction of the line of sight” means any combination of two or more of the change of the line of sight to the right and the change of the line of sight to the left. It is. For example, it may be a change in the line-of-sight direction in two or more consecutive right directions, may be a change in the line-of-sight direction in two or more consecutive left directions, a change in the line-of-sight direction in the left direction, A combination with a change in the line-of-sight direction to the right may be used.

  The process in which the correspondence information is stored in the correspondence information storage unit 18 does not matter. For example, correspondence information may be stored in the correspondence information storage unit 18 via a recording medium, and correspondence information transmitted via a communication line or the like is stored in the correspondence information storage unit 18. Alternatively, the correspondence information input via the input device may be stored in the correspondence information storage unit 18. The storage in the correspondence information storage unit 18 may be temporary storage in a RAM or the like, or may be long-term storage. The correspondence information storage unit 18 can be realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, etc.).

  Here, “associating operation pattern information and operation information” means that it is only necessary to obtain operation information from the operation pattern information. Therefore, the correspondence information may be information including the operation pattern information and the operation information as a set, or may be information that links the operation pattern information and the operation information. In the latter case, the correspondence information may be, for example, information that associates the operation pattern information with a pointer or address indicating the position where the operation information is stored. In the present embodiment, the former case will be described. Further, the operation pattern information and the operation information may not be directly associated with each other. For example, the third information may correspond to the operation pattern information, and the operation information may correspond to the third information.

  The operation information acquisition unit 19 acquires operation information corresponding to the operation pattern information when the operation history accumulated by the operation history accumulation unit 17 matches the operation pattern information. For example, when the latest operation history accumulated by the operation history accumulation unit 17 matches any of the operation pattern information, the operation information acquisition unit 19 performs an operation associated with the operation pattern information by the correspondence information. Information may be acquired. When the operation information acquired by the operation information acquisition unit 19 indicates an operation on a map, a map image corresponding to the operation information is acquired and displayed. On the other hand, when the operation information acquired by the operation information acquisition unit 19 indicates an operation on a landscape, a landscape image corresponding to the operation information is acquired and displayed. The operation history that is a target for the operation information acquisition unit 19 to determine whether or not it matches the operation pattern information is usually a history of two or more operations.

  Note that any two or more storage units or recording media of the map information storage unit 12, the scenery information storage unit 14, the recording medium in which the operation history storage unit 17 stores the operation history, and the correspondence information storage unit 18 These may be realized by the same recording medium, or may be realized by separate recording media. In the former case, for example, an area storing map information is the map information storage unit 12, and an area storing landscape information is the landscape information storage unit 14.

Next, operation | movement of the cooperation display apparatus 1 by this Embodiment is demonstrated using the flowchart of FIG.
(Step S101) The display unit 16 determines whether to display a map image or a landscape image. And when displaying a map image etc., it progresses to step S102, and when that is not right, the process of step S101 is repeated. For example, the display unit 16 may determine to display a map image or the like when the operation reception unit 11 receives an instruction to display a map image or the like.

  (Step S102) The map image acquisition unit 13 acquires a map image, and the landscape image acquisition unit 15 acquires a landscape image. For example, when information indicating the position of a map image or the like to be displayed is received, a map image or the like corresponding to the information may be acquired. Further, for example, when information indicating a position such as a map image to be displayed is not received, a map image or the like corresponding to a default position may be acquired. The default position information may be stored in a recording medium (not shown).

  (Step S103) The display unit 16 displays the map image acquired by the map image acquisition unit 13 and the landscape image acquired by the landscape image acquisition unit 15 together. For example, the display unit 16 may display the map image and the landscape image side by side in the same window.

  (Step S104) The operation reception unit 11 determines whether an operation for a map has been received. If an operation for the map is accepted, the process proceeds to step S105, and if not, the process proceeds to step S111.

  (Step S105) The map image acquisition unit 13 acquires a map image according to the operation received by the operation reception unit 11 in step S104. The map image may be temporarily stored on a recording medium (not shown).

  (Step S106) The operation history storage unit 17 stores a history according to the operation received by the operation reception unit 11 in step S104. When information such as the center position and scale is necessary when the history is accumulated, the information is acquired using the map image acquisition unit 13 or the map image acquired by the map image acquisition unit 13. Also good.

  (Step S107) The operation information acquisition unit 19 determines whether there is operation pattern information that matches the latest operation history for the map. And when it exists, it progresses to step S108, and when that is not right, it progresses to step S110.

  (Step S108) The operation information acquisition unit 19 acquires operation information associated with operation pattern information that matches the latest operation history for the map.

  (Step S109) The landscape image acquisition unit 15 acquires a landscape image according to the operation indicated by the operation information acquired in step S108. The scenery image may be temporarily stored on a recording medium (not shown).

  (Step S110) The display unit 16 displays the map image acquired by the map image acquisition unit 13. In addition, when the scenery image is acquired in step S109, the display unit 16 also displays the scenery image. Then, the process returns to step S104.

  (Step S111) The operation reception unit 11 determines whether an operation for a landscape has been received. If an operation for a landscape is accepted, the process proceeds to step S112. If not, the process proceeds to step S118.

  (Step S112) The landscape image acquisition unit 15 acquires a landscape image in accordance with the operation received by the operation reception unit 11 in step S111. The scenery image may be temporarily stored on a recording medium (not shown).

  (Step S113) The operation history storage unit 17 stores a history according to the operation received by the operation reception unit 11 in step S111. If information such as position and angle is necessary when the history is accumulated, such information may be acquired using the landscape image acquisition unit 15 or the landscape image acquired by the landscape image acquisition unit 15. Good.

  (Step S114) The operation information acquisition unit 19 determines whether there is operation pattern information that matches the latest operation history for the scenery. And when it exists, it progresses to step S115, and when that is not right, it progresses to step S117.

  (Step S115) The operation information acquisition unit 19 acquires operation information associated with operation pattern information that matches the latest operation history with respect to the scenery.

  (Step S116) The map image acquisition unit 13 acquires a map image according to the operation indicated by the operation information acquired in Step S115. The map image may be temporarily stored on a recording medium (not shown).

  (Step S117) The display unit 16 displays the landscape image acquired by the landscape image acquisition unit 15. Moreover, the display part 16 also displays the map image, when acquisition of the map image in step S116 was performed. Then, the process returns to step S104.

(Step S118) The display unit 16 determines whether or not to end the display of the map image and the landscape image. If the process is to be ended, the display of the map image or the like is ended and the process returns to step S101. If the process is not ended, the process returns to step S104. For example, the display unit 16 determines to end the display of the map image or the like when an instruction to end the display of the map image or the like (for example, an instruction to close the window displaying the map image or the like) is received. May be.
In the flowchart of FIG. 2, the process is terminated by powering off or a process termination interrupt.

The flowchart in FIG. 3 is a flowchart showing details of the matching determination process in step S107 or step S114 in the flowchart in FIG.
(Step S201) The operation information acquisition unit 19 sets the counter i to 1.

  (Step S202) The operation information acquisition unit 19 determines whether or not the i-th operation pattern information matches the latest history. If the flowchart of FIG. 3 corresponds to the process of step S107, the operation information acquisition unit 19 may make a determination using the operation pattern information about the map and the history about the map. If the flowchart of FIG. 3 corresponds to the process of step S114, the operation information acquisition unit 19 may make a determination using the operation pattern information about the scenery and the history about the scenery.

  (Step S203) The operation information acquisition unit 19 increments the counter i by 1.

  (Step S <b> 204) The operation information acquisition unit 19 determines whether or not correspondence information including the i-th operation pattern information exists in the correspondence information storage unit 18. And when it exists, it returns to step S202, and when that is not right, it progresses to step S205.

  (Step S205) The operation information acquisition unit 19 determines that there is no operation pattern information that matches the latest operation history, and returns to the flowchart of FIG. If the flowchart of FIG. 3 corresponds to the process of step S107, the process proceeds to step S110. If the flowchart of FIG. 3 corresponds to the process of step S114, the process proceeds to step S117.

  (Step S206) The operation information acquisition unit 19 determines that there is operation pattern information that matches the latest operation history, and returns to the flowchart of FIG. If the flowchart of FIG. 3 corresponds to the process of step S107, the process proceeds to step S108. If the flowchart of FIG. 3 corresponds to the process of step S114, the process proceeds to step S115. Thereafter, operation information corresponding to the i-th operation pattern information is acquired.

Next, the operation of the cooperative display device 1 according to the present embodiment will be described using a specific example.
In this specific example, it is assumed that the map information storage unit 12 stores the map information shown in FIG. In the map information of FIG. 4, the map image and the scale are associated with each other. The map image is an image of raster data indicating a map of the entire area to be displayed. As described above, it is assumed that latitude and longitude are associated with this map image. Therefore, the map image acquisition part 13 can acquire by cutting out the map image of a desired area | region. Each map image is displayed at a corresponding scale. For example, the map image “map001” is a map image used when displaying at the scale “SC001”.

  In this specific example, it is assumed that the scenery information storage unit 14 stores the scenery information shown in FIG. In the landscape information of FIG. 5, position information, a landscape image, and movable position information are associated with each other. The position information is information indicating latitude / longitude corresponding to the landscape image. The scenery image in this specific example is a live-action image photographed by a camera on a road, and the photographing position is indicated by position information. XA1 and the like are longitudes, and YA1 and the like are latitudes. The scenery image is a panoramic image. Further, there are wide-area landscape images, standard landscape images, and detailed landscape images depending on the degree of zoom. When a wide-area landscape image is zoomed in, it becomes a standard landscape image, and when a standard landscape image is zoomed in, it becomes a detailed landscape image. Further, since the landscape image is taken on the road, when moving from a landscape image at a certain position to a landscape image at another position, the landscape images that can move without a sense of incongruity are limited. The information indicating the position information of the adjacent scenery image that can move without a sense of incongruity is the movable position information. For example, since the movable position information of the scenery image corresponding to the second record in FIG. 5 is (XA1, YA1) (XA3, YA3), the first and third can be moved from the scenery information. It turns out that it is the scenery information of a record.

  In this specific example, the correspondence information storage unit 18 stores the correspondence information shown in FIG. In FIG. 6, the type flag, the operation pattern information, and the operation information are associated with each other. A record whose type flag is “0” has operation pattern information for a map and operation information for a landscape. A record whose type flag is “1” has operation pattern information for a landscape and operation information for a map. In the operation pattern information, an operation pattern is shown using a regular expression. For example, “+” indicates one or more repetitions of the immediately preceding operation. “*” Indicates zero or more repetitions of the previous operation. {N,} indicates repetition of the previous operation N times or more. [] Indicates an arbitrary operation included in the parentheses. In the operation on the map, “m” indicates movement, “i” indicates zoom-in, and “o” indicates zoom-out. In the operation on the scenery, “M” indicates movement, “I” indicates zoom-in, “O” indicates zoom-out, “L” indicates a leftward change of the line-of-sight direction, and “R” indicates the line-of-sight. The direction changes to the right direction, “U” indicates the upward change in the viewing direction, and “D” indicates the downward change in the viewing direction. Further, in the correspondence information, all items indicating the position such as the attention position and the center position indicate the latest attention position and the center position. In the operation pattern information, “attention position = out” indicates that the latest attention position is not included in the range of the displayed map image. Here, whether or not a certain position is included in the range of the map image is obtained, for example, by obtaining the maximum latitude / longitude and the minimum latitude / longitude of the displayed map image, and the latitude / longitude of the position. May be determined by determining from the minimum latitude / longitude whether it falls within the range of the maximum latitude / longitude. If the latitude / longitude of the location is included in the range of the minimum latitude / longitude to the maximum latitude / longitude, the location will be included in the map image range. If the longitude is not included in the range from the minimum latitude / longitude to the maximum latitude / longitude, the position is not included in the range of the map image. In the operation information, Angle (α, β) is a function that calculates the azimuth angle of a vector having the position of α as the start point and the position of β as the end point. For example, in the case of Angle (position, attention position), it is the azimuth angle of a vector extending from the latest position in the landscape to the direction of the latest attention position in the map. Θ ← α is an operation for setting the azimuth angle θ to the angle α. M (α) is an operation for moving the scenery to the position of α. If the position information indicating the position of α does not exist in the landscape information, M (α) is an operation of moving to the position of the position information indicating the position closest to the position of α. For example, M (center position) is an operation of moving to a position of position information indicating a position closest to the latest center position. Further, o (α) is an operation for zooming out the map with the position of α as a target position. Further, i (α) is an operation for zooming in on the position of α as the position of interest. Further, m (center position ← α) is an operation of moving the map so that the center position becomes the position of α. Place (α, β, γ) is a function indicating a position moved from the position α by a distance γ in the direction of the azimuth angle β. That is, it is a function indicating the position of the end point of a vector having the position of α as the viewpoint, the azimuth angle β and the length of γ. Switch (scale, α, β, γ, δ,...) Is β if the scale of the map image being displayed is α, δ if the scale of the map image being displayed is γ, and thereafter Is the same function.

  The first record in FIG. 6 includes one or more movements with respect to the map, one or more subsequent zoom-ins, one or more subsequent zoom-outs, and operation pattern information indicating subsequent zero or more movements, and scenery. Is the correspondence information that correlates the operation information for changing the line-of-sight direction to the reverse direction of the latest zoom-in point (that is, the latest zoom-in target position).

  The second record in FIG. 6 includes operation pattern information indicating one or more movements with respect to the map, one or more subsequent zoom-ins, and subsequent zero or more movements, and a point of zoom-in with respect to the scenery. That is, it is correspondence information that associates operation information to be changed to the latest attention position.

  The third record in FIG. 6 is one or more zoom-ins with respect to the map and one or more subsequent movements, and the latest attention position that is the zoom-in point by the last movement in the one or more movements is the map image. Change the line-of-sight direction to the last movement direction (that is, the direction from the previous center position to the latest center position) and the center position of the map image. This is correspondence information that associates the operation information to be moved to.

  The fourth record in FIG. 6 is one movement with respect to the map, and the operation pattern information indicating that the latest attention position is not included in the map image by the one movement, and the map image with respect to the scenery This is correspondence information that associates the operation information that moves to the latest center position.

  The first record whose type flag is “1” in FIG. 6 includes operation pattern information indicating one or more upward changes in the line-of-sight direction and subsequent one or more zoom-in to the scenery, and the map. The correspondence information associates the operation information for zooming out with the latest position corresponding to the scenery image as the attention position.

  The second record whose type flag is “1” in FIG. 6 indicates one or more changes to the view in the right direction, left direction, or downward direction, and one or more subsequent zoom-ins. This is correspondence information that associates the operation pattern information with the operation information for zooming in on the map with the latest position corresponding to the scenery image as the target position.

  The third record whose type flag is “1” in FIG. 6 includes two or more operation pattern information indicating the change of the gaze direction to the right or left direction with respect to the scenery, and the two or more of the map with respect to the map. From the latest position corresponding to the landscape image in the direction of the gaze direction after the last change in the change to the right direction or the left direction of the gaze direction, that is, the direction of the azimuth angle after the last change was determined for each scale. This is correspondence information for associating operation information for movement centered on a position separated by a distance. The distance determined for each scale is, for example, a distance shorter than the distance from the center of the map image to the outer edge closest to the center (for example, the distance from the center of the map image to the outer edge closest to the center is 0. Or a distance multiplied by 8).

  The fourth record whose type flag is “1” in FIG. 6 is a map with the operation pattern information indicating movement with respect to the scenery and the position after the movement (that is, the latest position corresponding to the scenery image) with respect to the map. This is correspondence information that associates operation information for moving the center position of an image.

  First, it is assumed that the user operates the cooperative display device 1 and inputs an instruction to display a map and scenery near Kyoto Station in Japan. The instruction is received by the operation receiving unit 11 and passed to the display unit 16. The display unit 16 determines to display a map image or the like according to the instruction (step S101). Then, the display unit 16 accesses a recording medium (not shown) in which information that associates a place name, a station name, and the like with latitude / longitude is stored, and acquires latitude / longitude corresponding to Kyoto station. Then, the latitude / longitude and an instruction to acquire a map image corresponding to the latitude / longitude are passed to the map image acquisition unit 13, and the latitude / longitude and an instruction to acquire a landscape image corresponding to the latitude / longitude are acquired. Pass to part 15.

  The map image acquisition unit 13 accesses the map information shown in FIG. 4 and cuts out the map image corresponding to the latitude / longitude received from the display unit 16 from the map image “map002” corresponding to the scale “SC002”. It is assumed that when acquiring a map image for the first time, it is set in advance to acquire a map image of a scale “SC002”. The landscape image acquisition unit 15 accesses the landscape information shown in FIG. 5 and is a wide-area landscape image corresponding to the received position information closest to the latitude / longitude, and the azimuth and zenith angles are (90 °, 90 °). °) is acquired (step S102). Note that when a landscape image is acquired for the first time, it is set in advance to acquire a landscape image that is a wide-area landscape image and has an azimuth angle and a zenith angle of (90 °, 90 °). . Then, the display unit 16 displays the map image near the Kyoto station acquired by the map image acquisition unit 13 and the landscape image near the Kyoto station acquired by the landscape image acquisition unit 15 on a display (not shown) (step S103).

  Next, it is assumed that the user performs a drag operation for moving the displayed map image slightly to the north side by operating the mouse. Then, since the operation is received by the operation receiving unit 11 and is an operation in the area where the map image is displayed, it is determined that the operation is an operation on the map (step S104). Then, the operation reception unit 11 passes information indicating the distance and direction according to the dragging with the mouse to the map image acquisition unit 13. The map image acquisition unit 13 calculates the center position of a new map image using the center position of the map image being displayed and information indicating the distance and direction of the drag. For example, when the information indicating the distance and direction of the drag is information indicating the position α of the drag start point and the position β of the drag end point, the vector portion where the position β is the start point and the position α is the end point. The center position of a new map image that is a position obtained by moving the center position of the currently displayed map image is calculated. In that case, you may perform the process which converts the distance on a screen coordinate into the distance on a map image. In the process, the scale of the map image being displayed may be used. Note that when the map is dragged southward, the direction of dragging and the moving direction of the center position are reversed, as can be seen from the fact that the center of the map moves northward. Assume that the calculated center position is (X1, Y1). Then, the map image acquisition unit 13 cuts out and acquires the map image at the center position (X1, Y1) from the map image “map002” corresponding to the scale “SC002” of the map image being displayed (step S105). In addition, the map image acquisition part 13 shall hold | maintain information, such as a center position and a scale corresponding to the acquired newest map image, with the recording medium which is not shown in figure. The operation history storage unit 17 stores map operation history information that associates the operation ID “m”, the center position (X1, Y1), and the scale “SC002” because the drag operation is performed on the map ( Step S106). The operation history storage unit 17 can receive the center position and scale from the map image acquisition unit 13. The first record in FIG. 7 is the map operation history information accumulated as described above.

  After that, the operation information acquisition unit 19 determines whether there is operation pattern information that matches the latest operation history for the map (step S107). In this case, it is assumed that there is no operation pattern information that matches the first record in FIG. 7 and whose correspondence information type flag shown in FIG. 6 is “0” (steps S201 to S205). Then, the display part 16 updates a map image by displaying the map image newly acquired by the map image acquisition part 13 on a display (step S110). FIG. 8 is a diagram illustrating an example of a map image and a landscape image displayed in this manner. In FIG. 8, the left square image is a map image, and the right square image is a landscape image. Further, in the map image of FIG. 8, a viewpoint graphic 9 which is a graphic indicating the position of the landscape image and the azimuth angle in the line-of-sight direction is displayed. It is assumed that the black circle of the viewpoint graphic 9 is set to a position indicated by position information corresponding to the scenery image being displayed. Further, the direction from the black circle of the viewpoint graphic 9 to the midpoint of the two end points opposite to the black circles of the two line segments of the viewpoint graphic 9 is the azimuth angle of the visual line direction corresponding to the displayed scenery image. It is assumed that it is set. This viewpoint graphic 9 may be generated by any component. For example, it may be generated by the landscape image acquisition unit 15, may be generated by the map image acquisition unit 13, or may be generated by other components. The position and direction of the viewpoint graphic 9 also change according to changes in the position and azimuth of the scenery image being displayed. Further, in FIG. 8, two arrows (triangles) are displayed on the scenery image, which corresponds to the moving direction of the scenery image. The scenery image acquisition unit 15 acquires movable position information corresponding to the displayed scenery image, calculates which direction the position indicated by the movable position information is in the displayed scenery image, and Depending on the calculation result, this arrow is displayed. In addition, the scenery image is moved in response to the arrow being clicked. The display of the viewpoint graphic 9, the display of the arrow for movement, and the movement of the scenery image using the arrow are already known in the street view.

  In FIG. 8, on the lower side of the map image, a zoom-in button 51, a zoom-out button 52, a move button 53 to the north side (upper side), a move button 54 to the east side (right side), and a south side (lower side). ) Move button 55 and a west (left) move button 56 are displayed. When the zoom-in button 51 is clicked, the map is zoomed in with the center position of the currently displayed map image as the target position. When the zoom-out button 52 is clicked, the map is zoomed out with the center position of the map image being displayed as the position of interest. When the North, East, South, and West move buttons 53, 54, 55, and 56 are clicked, the positions of the north, east, south, and west sides are centered by a predetermined distance from the center position of the displayed map image. It shall be moved to the map. Further, in FIG. 8, on the lower side of the landscape image, a zoom-in button 61, a zoom-out button 62, an upward gaze direction change button 63, a right gaze direction change button 64, and a lower side image. A line-of-sight direction change button 65 and a line-of-sight direction change button 66 to the left are displayed. When the zoom-in button 61 is clicked, the scenery image being displayed is zoomed in. When the zoom-out button 62 is clicked, the scenery image being displayed is zoomed out. When the change button 63 for the line-of-sight direction to the upper side is clicked, the zenith angle in the line-of-sight direction decreases by a predetermined angle. When the change button 64 of the gaze direction to the right is clicked, the azimuth angle in the gaze direction is increased by a predetermined angle. When the change button 65 for the line-of-sight direction to the lower side is clicked, the zenith angle in the line-of-sight direction increases by a predetermined angle. When the change button 66 of the gaze direction to the left is clicked, it is assumed that the azimuth angle in the gaze direction decreases by a predetermined angle.

  In the display of FIG. 8, it is assumed that the user clicks the zoom-in button 51 by operating the mouse. Then, the operation is accepted (step S104), and a map image having the same center position as the center position of the map image being displayed and having a scale that is one step larger than the scale of the map image being displayed is acquired (step S104). S105). Specifically, a map image whose center position is (X1, Y1) and whose scale is “SC003” is acquired. Further, according to the operation, the operation history storage unit 17 has a map operation history having an operation ID “i”, a center position (X1, Y1), an attention position (X1, Y1), and a scale “SC003”. Information is accumulated (step S106). The second record in FIG. 7 is accumulated in this way. In this operation, since the position of interest is not set, the position of interest is the same as the center position. Further, the center position and scale can be received from the map image acquisition unit 13.

  After that, the operation information acquisition unit 19 determines whether there is operation pattern information that matches the latest operation history for the map (step S107). In this case, since the second operation pattern information in FIG. 6 matches the first and second records in FIG. 7, the operation information acquisition unit 19 determines that the second operation pattern information matches the latest history. Determination is made (steps S201 to S204, S202). Then, the operation information acquisition unit 19 determines that there is matching operation pattern information, acquires the operation information “θ ← Angle (position, attention position)” from the second record in FIG. 15 (step S108). When receiving the operation information, the scenery image acquisition unit 15 acquires the position indicated by the position information corresponding to the acquired latest scenery image and the latest attention position (X1, Y1) of the map operation history information, Using these as arguments, the value of the function Angle (position, attention position) is calculated. In this case, it is assumed that the value of the function is “0 °”. Then, the landscape image acquisition unit 15 acquires a landscape image corresponding to the same position and zoom level as the displayed landscape image and having an azimuth angle of 0 °, and passes it to the display unit 16. (Step S109). The display unit 16 displays the zoomed-in map image and the landscape image with the azimuth angle changed (step S110). FIG. 9 is a diagram illustrating an example of a map image and a landscape image displayed in this manner. Further, the orientation of the viewpoint graphic 9 also changes according to the change of the azimuth angle.

  In the display of FIG. 9, it is assumed that the user clicks the zoom-out button 52 by operating the mouse. Then, the operation is accepted (step S104), and a map image having the same center position as the center position of the map image being displayed and having a scale one step smaller than the scale of the map image being displayed is acquired (step S104). S105). Specifically, a map image whose center position is (X1, Y1) and whose scale is “SC002” is acquired. In response to the operation, the operation history storage unit 17 has a map operation history having an operation ID “o”, a center position (X1, Y1), a target position (X1, Y1), and a scale “SC002”. Information is accumulated (step S106). The third record in FIG. 7 is accumulated in this way. In this operation, since the position of interest is not set, the position of interest is the same as the center position. Further, the center position and scale can be received from the map image acquisition unit 13.

  After that, the operation information acquisition unit 19 determines whether there is operation pattern information that matches the latest operation history for the map (step S107). In this case, since the first operation pattern information in FIG. 6 matches the first to third records in FIG. 7, the operation information acquisition unit 19 determines that the first operation pattern information matches the latest history. Judgment is made (steps S201 and S202). Then, the operation information acquisition unit 19 determines that there is matching operation pattern information, acquires operation information “θ ← Angle (position, focus position for zooming in) + 180 °” from the first record in FIG. The scene image acquisition unit 15 is handed over (step S108). When receiving the operation information, the scenery image acquisition unit 15 receives the position indicated by the position information corresponding to the scenery image displayed at that time, the latest zoom-in attention position (X1, Y1) of the map operation history information, and And using these as arguments, the value of the function Angle (position, position of interest) is calculated. In this case, it is assumed that the value of the function is “0 °”, and as a result, the operation information is “θ ← 180 °”. Then, the landscape image acquisition unit 15 acquires a landscape image corresponding to the same position and zoom level as the display and having an azimuth angle of 180 °, and passes it to the display unit 16 (step S109). ). The display unit 16 displays the zoomed-out map image and the landscape image with the azimuth angle changed (step S110). FIG. 10 is a diagram illustrating an example of the map image and the scenery image displayed as described above. Further, the orientation of the viewpoint graphic 9 also changes according to the change of the azimuth angle.

  In the display of FIG. 10, it is assumed that the user clicks the zoom-out button 52 again by operating the mouse. In this case as well, the map image is zoomed out and the operation information of the first record in FIG. There is no change, and only the map image is changed (steps S104 to S110). By this series of operations, the fourth record in FIG. 7 is accumulated.

  After that, it is assumed that the user clicks the zoom-in button 51 by operating the mouse. In this case, the map image is zoomed in, but operation information is not acquired because there is no operation pattern information corresponding to the latest history. And display of a map image etc. becomes the same as that of FIG. 10 (steps S104-S07, S110). By this series of operations, the fifth record in FIG. 7 is accumulated.

  Thereafter, the user operates the mouse and clicks the south movement button 55. Then, the operation is accepted (step S104), and a position on the south side by a predetermined distance from the center position of the map image being displayed becomes the center position, and a map image having the same scale as the map image being displayed is acquired ( Step S105). Specifically, a map image whose center position is (X2, Y2) and whose scale is “SC002” is acquired. In response to the operation, the operation history storage unit 17 stores map operation history information having the operation ID “m”, the center position (X2, Y2), and the scale “SC002” (step S106). The sixth record in FIG. 7 is accumulated in this way. In this case, it is assumed that the latest attention position (X1, Y1) is included in the range of the map image. Then, since there is no operation pattern information that matches the latest map operation history information, the operation information is not acquired (step S107), and only the map image is updated (step S110).

  Further, after that, it is assumed that the user operates the mouse and clicks the south movement button 55 again. Then, the operation is accepted (step S104), and a position on the south side by a predetermined distance from the center position of the map image being displayed becomes the center position, and a map image having the same scale as the map image being displayed is acquired ( Step S105). Specifically, a map image whose center position is (X3, Y3) and whose scale is “SC002” is acquired. In response to the operation, the operation history storage unit 17 stores map operation history information having the operation ID “m”, the center position (X3, Y3), and the scale “SC002” (step S106). The seventh record in FIG. 7 is accumulated in this way. In this case, it is assumed that the latest attention position (X1, Y1) is not included in the range of the map image. Then, it is determined that the third operation pattern information in FIG. 6 matches the latest map operation history information (step S107), and the operation information “θ ← Angle (center immediately before) corresponding to the third operation pattern information in FIG. (Position, center position) and M (center position) "are acquired and passed to the landscape image acquisition unit 15 (step S108). The scenery image acquisition unit 15 refers to the map operation history information in FIG. 7 and acquires the latest center position (X3, Y3) and the immediately preceding center position (X2, Y2). Then, Angle (immediately before center position, center position) = 180 ° is calculated. As a result, the azimuth angle does not change from 180 °. The scenery image acquisition unit 15 specifies position information in the scenery information of FIG. 5 that is closest to the center position (X3, Y3). Then, a landscape image having an azimuth angle of 180 ° and a zenith angle of 90 ° is acquired from a wide-area landscape image corresponding to the position information, and is passed to the display unit 16 (step S109). The display unit 16 displays the moved map image and the moved landscape image (step S110). FIG. 11 is a diagram illustrating an example of a map image and a landscape image displayed in this manner. Further, the position of the viewpoint graphic 9 also changes according to the change in the position of the scenery image.

  Thereafter, it is assumed that the map image is repeatedly moved, zoomed in, zoomed out, etc., and the display shown in FIG. 12 is obtained. In the display, it is assumed that the user starts an operation on a landscape image. First, it is assumed that the user clicks the button 63 for changing the line of sight upward by operating the mouse. Then, the operation is accepted (step S111), and the fact that the button 63 for changing the line of sight to the upper side is clicked is passed to the landscape image acquisition unit 15. The scenery image acquisition unit 15 decreases the zenith angle by 15 ° with respect to the viewing direction of the scenery image being displayed. As a result, the zenith angle is assumed to be 75 °. Then, a landscape image in which only the zenith angle in the line-of-sight direction is changed while the position of the landscape image and the zoom level remain the same is acquired (step S112). In addition, since the operation of changing the line-of-sight direction has been performed, the operation history accumulation unit 17 has an operation ID “U”, a position (XA10, YA10) corresponding to the displayed scenery image, an azimuth indicating the line-of-sight direction, Landscape operation history information that associates the zenith angle (180 °, 75 °) with the zoom level “wide area” is stored (step S113). The scenery image acquisition unit 15 holds the position corresponding to the latest scenery image acquired, the information on the line-of-sight direction, the degree of zooming, and the like on a recording medium (not shown), as with the map image acquisition unit 13. And The operation history storage unit 17 can receive information on the position and line-of-sight direction, the degree of zooming, and the like from the landscape image acquisition unit 15. The first record in FIG. 13 is scenery operation history information accumulated in this way.

  Thereafter, the operation information acquisition unit 19 determines whether or not there is operation pattern information that matches the latest operation history for the scenery (step S114). In this case, it is assumed that there is no matching operation pattern information (steps S201 to S205). Then, the display unit 16 updates the scenery image by displaying on the display the scenery image that has been newly acquired by the scenery image acquisition unit 15 and whose line-of-sight direction has been changed (step S117).

  Next, it is assumed that the user clicks the zoom-in button 61 by operating the mouse again. Then, the operation is accepted (step S111), the same position as the position of the scenery image being displayed, the line-of-sight direction, and the zoom-in side (tele side) is closer to the zoom-in side than the degree of zooming of the scenery image being displayed. A landscape image having a degree of “standard” is acquired (step S112). Specifically, a landscape image whose position is (XA10, YA10), the angle of the line-of-sight direction is (180 °, 75 °), and the zoom level is “standard” is acquired. In response to the operation, the operation history storage unit 17 sets the operation ID “I”, the position (XA10, YA10), the angle of the line of sight (180 °, 75 °), and the degree of zoom “standard”. Is stored (step S113). The second record in FIG. 13 is accumulated in this way.

  Thereafter, the operation information acquisition unit 19 determines whether or not there is operation pattern information that matches the latest operation history for the scenery (step S114). In this case, since the operation pattern information of the first record whose type flag in FIG. 6 is “1” matches the first and second records of FIG. It is determined that the information matches the latest history (steps S201 and S202). And the operation information acquisition part 19 acquires operation information "o (position)" from the record in which the matching operation pattern information exists, and passes it to the map image acquisition part 13 (step S115). When the map image acquisition unit 13 receives the operation information, the map image acquisition unit 13 acquires a position (XA10, YA10) indicated by the position information corresponding to the acquired latest landscape image, and zooms out the map image with the position as a target position. Obtain (step S116). Note that the map image acquisition unit 13 may acquire the latest position (XA10, YA10) corresponding to the displayed scenery image from the scenery operation history information, or may send the scenery image acquisition part 15 through a route not shown. You may access and get it. The display unit 16 displays the zoomed-out map image and the zoomed-in scenery image (step S117). FIG. 14 is a diagram illustrating an example of a map image and a landscape image displayed in this manner. When the user changes the line-of-sight direction upward in the landscape image and zooms in, it is considered that he / she is interested in the distant view, but the map image is also zoomed out to include the distant view. Therefore, it is possible to realize a collaborative operation that improves user convenience.

  Next, the user operates the mouse, clicks the zoom-out button 62, clicks the downward gaze direction change button 65, clicks the right gaze direction change button 64, and then zooms in It is assumed that the button 61 is clicked. Then, those operations are sequentially accepted (step S111), and scenery images corresponding to those operations are acquired (step S112), and the respective histories are shown by the third to sixth records in FIG. Is accumulated (step S113). When the change button 65 for the gaze direction to the lower side is clicked, the zenith angle is increased by 15 °, and when the change button 64 (66) for the gaze direction to the right side (left side) is clicked. Is set so that the azimuth is increased (decreased) by 90 °.

  Thereafter, the operation information acquisition unit 19 determines whether or not there is operation pattern information that matches the latest operation history for the scenery (step S114). In this case, the operation pattern information of the second record whose type flag is “1” in FIG. 6 matches the fourth to sixth records in FIG. Operation information “i (position)” corresponding to the information is acquired and passed to the map image acquisition unit 13 (steps S114 and S115). Upon receiving the operation information, the map image acquisition unit 13 acquires a position (XA10, YA10) corresponding to the acquired latest landscape image, and acquires a zoomed-in map image with the position as a target position (step S116). ). The display unit 16 displays the zoomed-in map image and the zoomed-in scenery image (step S117). FIG. 15 is a diagram illustrating an example of a map image and a landscape image displayed in this manner. When a user changes the line-of-sight direction to the right, left, or lower side in a landscape image and zooms in, it is considered that he / she is interested in the near view. Thus, it is possible to zoom in, and to realize a collaborative operation that improves user convenience.

  After that, it is assumed that an operation is performed on the map image and the landscape image, as shown in FIG. Thereafter, it is assumed that the user operates the mouse and clicks the change button 64 of the gaze direction to the right with respect to the scenery twice. Then, those operations are sequentially accepted (step S111), and scenery images corresponding to those operations are acquired (step S112), and the respective histories are the first and second records from the bottom of FIG. Accumulated as shown (step S113).

  The operation information acquisition unit 19 determines whether there is operation pattern information that matches the latest operation history with respect to the scenery (step S114). In this case, the operation pattern information of the third record whose type flag is “1” in FIG. 6 matches the first and second records from the bottom of FIG. Operation information “m (center position ← Place (position, θ, Switch (scale, SC001, L1, SC002, L2,...))) Corresponding to the operation pattern information is acquired and passed to the map image acquisition unit 13 (step S114, S115). When receiving the operation information, the map image acquisition unit 13 acquires the position (XA25, YA25) and the azimuth angle θ “180 °” corresponding to the acquired latest landscape image, and the acquired latest map image The corresponding scale “SC002” is acquired. Then, the value “L2” of the function Switch is calculated, and the function Place ((XA25, YA25), 180 °, L2) is calculated. As a result of the calculation, the position that is the value of the function Place is a position that is separated by L2 in the azimuth direction from the position corresponding to the latest acquired landscape image. As a result, a map image centered on the position that is the value of the function Place is acquired and passed to the display unit 16 (step S116). As shown in FIG. 17, the display unit 16 displays the moved map image and the landscape image with the line-of-sight direction changed (step S <b> 117). When the user repeatedly changes the line-of-sight direction to the right or left side in the landscape image, it is considered that the user is interested in the direction of the line-of-sight direction. It is possible to move so as to be displayed widely, and it is possible to realize a collaborative operation that enhances user convenience.

  In this specific example, the map and scenery near Kyoto Station have been described. However, these are merely examples, and the map and scenery are not necessarily accurate.

  Also, in this specific example, a case has been described in which there is correspondence information that associates operation pattern information corresponding to one operation and operation information, but as described above, operation pattern information includes two or more operations. Information indicating a pattern may be used. In that case, for example, the process of moving the other according to the movement process for one of the map and the scenery may be realized without using the correspondence information. Moving the map according to the movement in the scenery is realized in Street View and is well known. In addition, a method for moving the position of the scenery according to the movement on the map can be easily realized. For example, the scenery image may be changed so that the scenery image closest to the center position in the map image is displayed.

  Further, in this specific example, a case has been described in which it is determined whether or not the operation pattern information matches the latest history. When the time has passed, it is not necessary to determine that the operation pattern information matches the operation sequence. For example, even if the map is moved and zoomed in after about 10 minutes have elapsed, it may not be determined that the map matches the operation pattern information of the second record in FIG. Therefore, for example, when accumulating the operation history, the current date and time may be associated and accumulated. Then, the operation information may be acquired using only the operation history accumulated within a predetermined time (for example, 5 minutes, 10 minutes, etc.) from the time of comparison with the operation pattern information. Alternatively, for example, when accumulating the operation history, the current date and time may be associated and accumulated, and the history after a predetermined time may be deleted.

  In this specific example, the case where the viewpoint graphic 9 is displayed has been described, but this need not be the case. In this specific example, the case where an arrow (triangle) in a movable direction is displayed in the scenery image has been described, but this need not be the case. A button for moving is displayed on the lower side of the scenery image, and the button may be moved by clicking on it.

  Also, in this specific example, the case where the information indicating the position such as the center position of the map image and the information indicating the position of the landscape image are both the same type of information, that is, the case where it is latitude / longitude. However, if the two are different, the information from one position may be appropriately converted to the information at the other position using a conversion formula or conversion table for converting from one to the other.

  As described above, according to the cooperative display device 1 according to the present embodiment, the other can be automatically operated according to the user's operation on one of the two-dimensional map and the three-dimensional landscape. Therefore, it is not necessary for the user to perform complicated operations on both, the user's load is reduced, and the convenience for the user is improved. In particular, by using the correspondence information to associate the operation pattern information corresponding to a certain intention with the operation information corresponding to the intention, an operation corresponding to the intention of the user's operation for one of the map and the scenery is made to the other. Will be able to do it. In addition, unless the latest history matches the operation pattern information, an automatic operation is not performed on an image that is not an operation target, so that an inadvertent operation on the image does not occur.

  Although the correspondence information shown in FIG. 6 has been described in the present embodiment, it is needless to say that other correspondence information may be used. For example, using correspondence information that associates operation pattern information indicating that two or more movements with respect to the map have been repeated and operation information with respect to a landscape to change the line-of-sight direction to the direction of the last movement of the two or more movements. May be. Further, in the operation pattern information of the correspondence information, the case where the operation pattern is mainly indicated using the operation ID has been described, but the operation pattern may be indicated using an operation parameter or the like. For example, an operation pattern may be indicated using an azimuth angle or a zenith angle in the line-of-sight direction, or an operation pattern may be indicated using a center position, a coordinate value of the position, or the like. In this case, the operation history may also be described using, for example, the center position and the coordinate value of the position.

  Further, in the present embodiment, a case has been described in which an operation is performed on either the map or the landscape, but an operation is performed on the other, but this need not be the case. For example, when the operation on the map is performed, the cooperative display device 1 may automatically perform the operation on the landscape according to the operation, or when the operation on the landscape is performed. The operation on the map may be automatically performed according to the operation. In the former case, for example, the operation history storage unit 17 may store only the history of operations related to the map, and the correspondence information storage unit 18 associates the operation pattern information for the map with the operation information for the scenery. Only the information may be stored, and the operation information acquisition unit 19 may compare whether or not the operation pattern information matches only the operation history related to the map. In the latter case, for example, the operation history storage unit 17 may store only the history of operations related to scenery, and the correspondence information storage unit 18 stores operation pattern information for scenery and operation information for maps. Only the correspondence information to be associated may be stored, and the operation information acquisition unit 19 may compare whether or not the operation history information only matches the operation pattern information.

  Further, in the present embodiment, the case where the operation on the map is at least one of position movement, zoom-in, and zoom-out has been described, but there may be other operations. Moreover, although the case where the operation on the scenery is at least one of position movement, zoom-in, zoom-out, and change of the line-of-sight direction has been described, other operations may be performed.

  In the present embodiment, the case where the operation ID, the center position, the scale, and the like are included in the map operation history has been described, but this need not be the case. For example, the operation history of the map may include only the operation ID, may include only the operation ID and the center position, may include only the operation ID and the scale, or other combinations It may be.

  In this embodiment, the case where the operation ID, the position, the angle of the line-of-sight direction, the zoom level, and the like are included in the history operation history has been described, but this need not be the case. For example, the history operation history may include only the operation ID, may include only the operation ID and the angle of the line of sight, may include only the operation ID and the position, It may be a combination.

  In the present embodiment, the map image acquisition unit 13 may acquire a map image by accessing the server, or the landscape image acquisition unit 15 may acquire a landscape image by accessing the server. Is as described above. At that time, both the map image acquisition unit 13 and the landscape image acquisition unit 15 may acquire a map image or the like from the server, or only the map image acquisition unit 13 accesses the server to obtain the map image. It may be acquired, or only the landscape image acquisition unit 15 may acquire the landscape image by accessing the server.

  Moreover, although the case where the cooperation display apparatus 1 was stand-alone was demonstrated in the said embodiment, the cooperation display apparatus 1 may be a stand-alone apparatus, and may be a server apparatus in a server client system. In the latter case, the display unit or the reception unit may receive an input or display information via a communication line.

  In the above embodiment, each process or each function may be realized by centralized processing by a single device or a single system, or may be distributedly processed by a plurality of devices or a plurality of systems. It may be realized by doing.

  In the above embodiment, information related to processing executed by each component, for example, information received, acquired, selected, generated, transmitted, or received by each component In addition, information such as threshold values, mathematical formulas, addresses, etc. used by each component in processing is retained temporarily or over a long period of time on a recording medium (not shown) even when not explicitly stated in the above description. It may be. Further, the storage of information in the recording medium (not shown) may be performed by each component or a storage unit (not shown). Further, reading of information from the recording medium (not shown) may be performed by each component or a reading unit (not shown).

  In the above embodiment, when information used by each component, for example, information such as a threshold value, an address, and various setting values used by each component may be changed by the user Even if it is not specified in the above description, the user may be able to change the information as appropriate, or it may not be. If the information can be changed by the user, the change is realized by, for example, a not-shown receiving unit that receives a change instruction from the user and a changing unit (not shown) that changes the information in accordance with the change instruction. May be. The change instruction received by the receiving unit (not shown) may be received from an input device, information received via a communication line, or information read from a predetermined recording medium, for example. .

  Moreover, in the said embodiment, when two or more components included in the cooperation display apparatus 1 have a communication device, an input device, etc., two or more components may have a physically single device. Or you may have separate devices.

  In the above embodiment, each component may be configured by dedicated hardware, or a component that can be realized by software may be realized by executing a program. For example, each component can be realized by a program execution unit such as a CPU reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. In addition, the software which implement | achieves the cooperation display apparatus 1 in the said embodiment is the following programs. In other words, this program uses a computer to receive a user operation, a map image acquisition unit that acquires a map image according to an operation on a map received by the operation reception unit, using map information that is information about the map. A landscape image acquisition unit that acquires a landscape image according to an operation on a landscape received by the operation reception unit using a landscape information that is information related to a landscape, a map image acquired by the map image acquisition unit, and a landscape image acquisition unit A display unit for displaying a landscape image, an operation history storage unit for storing two or more histories of operations received by the operation reception unit, and an operation history stored by the operation history storage unit for a map and a landscape. Correspondence that is information associating operation pattern information that is information indicating the above operation pattern with operation information that is information indicating an operation on the other A program that functions as an operation information acquisition unit that acquires operation information corresponding to the operation pattern information when the operation pattern information matches the operation pattern information, and performs an operation according to the operation information acquired by the operation information acquisition unit It is.

  In the program, the functions realized by the program do not include functions that can be realized only by hardware. For example, functions that can be realized only by hardware such as a modem or an interface card in an acquisition unit that acquires information, a display unit that displays information, and the like are not included in at least the functions realized by the program.

  Further, this program may be executed by being downloaded from a server or the like, and a program recorded on a predetermined recording medium (for example, an optical disk such as a CD-ROM, a magnetic disk, a semiconductor memory, or the like) is read out. May be executed by Further, this program may be used as a program constituting a program product.

  Further, the computer that executes this program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

  FIG. 18 is a schematic diagram illustrating an example of the external appearance of a computer that executes the program and realizes the cooperative display device 1 according to the embodiment. The above-described embodiment can be realized by computer hardware and a computer program executed on the computer hardware.

  In FIG. 18, a computer system 900 includes a computer 901 including a CD-ROM (Compact Disk Read Only Memory) drive 905, an FD (Floppy (registered trademark) Disk) drive 906, a keyboard 902, a mouse 903, a monitor 904, and the like. Is provided.

  FIG. 19 is a diagram showing an internal configuration of the computer system 900. In FIG. 19, in addition to the CD-ROM drive 905 and the FD drive 906, a computer 901 is connected to an MPU (Micro Processing Unit) 911, a ROM 912 for storing a program such as a bootup program, and the MPU 911. A RAM (Random Access Memory) 913 that temporarily stores program instructions and provides a temporary storage space, a hard disk 914 that stores application programs, system programs, and data, and an MPU 911 and a ROM 912 are interconnected. And a bus 915. The computer 901 may include a network card (not shown) that provides connection to the LAN.

  A program for causing the computer system 900 to execute the function of the cooperative display device 1 according to the above-described embodiment is stored in the CD-ROM 921 or FD 922, inserted into the CD-ROM drive 905 or FD drive 906, and stored in the hard disk 914. May be forwarded. Instead, the program may be transmitted to the computer 901 via a network (not shown) and stored in the hard disk 914. The program is loaded into the RAM 913 when executed. The program may be loaded directly from the CD-ROM 921, the FD 922, or the network.

  The program does not necessarily include an operating system (OS), a third-party program, or the like that causes the computer 901 to execute the function of the cooperative display device 1 according to the above-described embodiment. The program may include only a part of an instruction that calls an appropriate function (module) in a controlled manner and obtains a desired result. How the computer system 900 operates is well known and will not be described in detail.

  Further, the present invention is not limited to the above-described embodiment, and various modifications are possible, and it goes without saying that these are also included in the scope of the present invention.

  As mentioned above, according to the cooperation display apparatus by this invention, etc., according to the user's operation with respect to one of a two-dimensional map and a three-dimensional landscape, the effect that the other can be operated automatically is acquired, This is useful as a device or system for displaying a map and scenery.

DESCRIPTION OF SYMBOLS 1 Cooperation display apparatus 11 Operation reception part 12 Map information storage part 13 Map image acquisition part 14 Landscape information storage part 15 Landscape image acquisition part 16 Display part 17 Operation history storage part 18 Corresponding information storage part 19 Operation information acquisition part

Claims (13)

An operation reception unit for receiving user operations;
A map image acquisition unit that acquires a map image according to an operation on the map received by the operation reception unit, using map information that is information about the map;
A landscape image acquisition unit that acquires a landscape image according to an operation on the landscape received by the operation reception unit, using landscape information that is information about the landscape;
A display unit for displaying the map image acquired by the map image acquisition unit and the landscape image acquired by the landscape image acquisition unit;
An operation history accumulating unit that accumulates two or more histories of operations accepted by the operation accepting unit;
Correspondence information storage unit for storing correspondence information as information for associating operation pattern information, which is information indicating two or more operation patterns for one of the map and scenery, with operation information, which is information indicating the operation for the other When,
An operation information acquisition unit that acquires operation information corresponding to the operation pattern information when the operation history accumulated by the operation history accumulation unit matches the operation pattern information;
A cooperative display device in which an operation according to the operation information acquired by the operation information acquisition unit is executed. The operation history storage unit stores a history of operations on the map,
The correspondence information storage unit stores at least correspondence information for associating operation pattern information for a map with operation information for a landscape,
The cooperative display device according to claim 1, wherein the landscape image acquisition unit acquires a landscape image according to operation information for the landscape acquired by the operation information acquisition unit. The cooperative display apparatus according to claim 1, wherein the operation on the map received by the operation receiving unit is at least one of position movement, zoom-in, and zoom-out. The cooperative display device according to claim 2, wherein the operation information includes an operation for changing a line-of-sight direction. In the correspondence information storage unit, (1) the operation pattern information indicating one or more movements with respect to the map, the subsequent one or more zoom-ins, and the subsequent zero or more movements; (2) Operation indicating one or more movements, one or more subsequent zoom-ins, one or more subsequent zoom-outs, and subsequent zero or more movements. Correspondence information for associating pattern information with operation information for changing the line-of-sight direction to the reverse direction of the zoom-in point with respect to the scenery, (3) one or more zoom-ins to the map, and one or more subsequent movements, Operation pattern information indicating that the zoom-in point is no longer included in the map image due to the last movement of one or more movements, and the gaze direction with respect to the scenery Wherein at least one or more is stored in the correspondence information for associating the operation information to change the direction of the last movement, coordination display device according to any one of claims 2 4. The operation history storage unit stores a history of operations on the scenery,
The correspondence information storage unit stores at least correspondence information for associating operation pattern information for a landscape with operation information for a map,
The cooperative display device according to claim 1, wherein the map image acquisition unit acquires a map image according to operation information for the map acquired by the operation information acquisition unit. The cooperative display device according to claim 6, wherein the operation on the scenery received by the operation receiving unit is at least one of position movement, zoom-in, zoom-out, and change of the line-of-sight direction. In the correspondence information storage unit, (1) operation pattern information indicating one or more upward changes in the line-of-sight direction with respect to the landscape and subsequent one or more zoom-ins, operation information for zooming out the map, (2) One or more changes in the gaze direction to the right, left or down, and then one or more zoom-in operations, and zoom-in on the map Corresponding information for associating the operation information to be performed, (3) Two or more operation pattern information indicating the change of the gaze direction to the right or left direction with respect to the landscape, and the two or more gaze direction right to the map The correspondence information for associating the operation information moving in the line-of-sight direction after the last change in the change to the direction or the left direction is stored. Cooperation display device according to claim 7. The map image acquisition unit acquires a map image by accessing a server holding the map information,
The cooperative display device according to claim 1, wherein the scenery image acquisition unit acquires a scenery image by accessing a server that holds the scenery information. A map information storage unit for storing the map information;
A landscape information storage unit for storing the landscape information;
The map image acquisition unit acquires a map image by accessing the map information storage unit,
The cooperative display device according to claim 1, wherein the scenery image acquisition unit acquires a scenery image by accessing the scenery information storage unit. The cooperative display device according to claim 1, wherein the scenery is a real space photographed by a camera. Operation pattern information, which is information indicating two or more operation patterns for one of the operation reception unit, the map image acquisition unit, the landscape image acquisition unit, the display unit, the operation history storage unit, and the map and landscape. A cooperative display method that is processed using a correspondence information storage unit that stores correspondence information that is information that correlates operation information that is information indicating an operation on the other, and an operation information acquisition unit,
An operation receiving step in which the operation receiving unit receives a user operation;
The map image acquisition unit acquires a map image according to an operation on the map received in the operation reception step, using map information that is information about a map;
The landscape image acquisition unit acquires a landscape image according to the operation for the landscape received in the operation reception step using the landscape information that is information about the landscape; and
A display step in which the display unit displays the map image acquired in the map image acquisition step and the landscape image acquired in the landscape image acquisition step;
An operation history storage step in which the operation history storage unit stores two or more operation histories received in the operation reception step;
The operation information acquisition unit includes an operation information acquisition step of acquiring operation information corresponding to the operation pattern information when the operation history stored in the operation history storage step matches the operation pattern information. ,
A cooperative display method in which an operation according to the operation information acquired in the operation information acquisition step is executed. Computer
An operation receiving unit for receiving user operations,
A map image acquisition unit that acquires a map image according to an operation on the map received by the operation reception unit, using map information that is information about the map;
A landscape image acquisition unit that acquires a landscape image according to an operation on the landscape received by the operation reception unit, using landscape information that is information about the landscape;
A display unit for displaying the map image acquired by the map image acquisition unit and the landscape image acquired by the landscape image acquisition unit;
An operation history accumulating unit that accumulates two or more histories of operations accepted by the operation accepting unit;
The operation history accumulated by the operation history accumulation unit associates, with respect to a map and a landscape, operation pattern information that is information indicating two or more operation patterns for one and operation information that is information indicating an operation for the other. When it matches the operation pattern information in the correspondence information that is information, it functions as an operation information acquisition unit that acquires operation information corresponding to the operation pattern information,
A program in which an operation according to operation information acquired by the operation information acquisition unit is executed.

Images