JP2011215412A - Map image processing device, map image processing method, and map image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a system for providing a user with surrounding conditions of the user together with additional information wherein services using a three-dimensional map suitable for a portable terminal are provided.SOLUTION: A map image processing device includes: a tag associating means which specifies a building having coordinate information within a prescribed map region from a building information storage means and associates the specified building with a tag; a tag combining means which combines the information of the tag associated with the building at the position of the specified building on a three-dimensional map image on the basis of the coordinate information of the viewpoint position of the three-dimensional map image, a line of sight direction, and three-dimensional coordinate of the specified building; a store information associating means which specifies store information of a store in the building associated with the tag from the building information storage means, and associates the specified store information with the three-dimensional map image combined with the information of the tag; and a storage means which stores the three-dimensional map image combined with the information of the tag in a tagged three-dimensional map storage means.

Description

  The present invention relates to a technique for processing a map image, and particularly to a technique for processing an image of a two-dimensional map and a three-dimensional map.

  Services that use augmented reality (AR), which adds information to computers in the real environment, have appeared. As a service in which such augmented reality is applied to a mobile terminal, for example, when a scene in front of the user is displayed on a display by a digital camera provided in the mobile terminal, additional information related to the location and the object (building, signboard, etc.) There are known services that display (text, images, etc.) superimposed on video.

  Various services using digital maps are also provided. Patent Document 1 describes a navigation device mounted on a vehicle as an example of such a service. The navigation device displays a 2D map in which the first intersection and the second intersection fit on the display screen when turning at the first intersection on the route, and when turning around the second intersection, the 2D map Another 2D map with a higher scale is displayed on the display screen, and then a 3D map based on the vehicle is displayed on the display screen.

Japanese Patent No. 4077304

  By the way, in the route search service that can be used from the mobile terminal, when the user inputs a destination, the route from the current position to the destination is searched and displayed superimposed on the 2D map. Therefore, the user can proceed to the destination while checking the route on the map, but it may be difficult to grasp the surrounding situation on the 2D map, and particularly when the route to the destination is complicated. In some cases, it may be desirable to confirm the position of the user using the augmented reality described above in the middle of the route.

  However, in augmented reality, additional information is superimposed on the video captured by the camera included in the mobile terminal as in the above-described prior art, and since the video is a real landscape, for example, at night or in the rain When the service is used, it is difficult to recognize the image as a result of the image becoming dark or cloudy. Also, even when used in the daytime in fine weather, if an obstacle such as a large car or a passerby is interposed between the camera and the building that the user is trying to image, the image of the building intended by the user Cannot be obtained, and the additional information is superimposed and displayed on the video of a large automobile or the like.

  On the other hand, with the rapid spread of mobile terminals, the Internet, navigation systems, etc., the use of digital maps has expanded in various scenes. As a result, there is a need among users to improve the convenience of services using digital maps. Is growing more and more. However, in a portable terminal, there are restrictions on the physical size, processing capability, cost, etc. of the device, so the technology described in Patent Document 1, for example, map information read from a CD-ROM is stored in the device. In addition, it is difficult to directly apply a technology including a navigation controller to a mobile terminal. Since a 3D map has a larger amount of data and a longer processing time than a 2D map, when handling a 3D map on a mobile terminal, it is necessary to consider problems such as processing capacity of the mobile terminal and network communication load.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a map image processing apparatus that provides a user with a situation using the 3D map suitable for a mobile terminal, and is a mechanism that can provide the user with the surrounding information along with additional information. There is.

  The map image processing apparatus includes a three-dimensional map storage unit that stores a three-dimensional map image, coordinate information of a viewpoint position of the three-dimensional map image, and a line-of-sight direction of the viewpoint position, and a three-dimensional map of the building. The building information storage means for storing the coordinate information and the store information of the store accommodated in the building in association with each other, and the building having the coordinate information within a predetermined map area is specified from the building information storage means, and the specified Tag linking means for linking a tag to a building, the position of the identified building on the three-dimensional map image based on the coordinate information and the line-of-sight direction of the three-dimensional map image and the three-dimensional coordinates of the identified building A tag composing unit that synthesizes information of a tag associated with the building, and store information of a store accommodated in the building associated with the tag from the building information storage unit, and the identified store Affection Store information linking means for linking the tag information with the synthesized 3D map image, storage means for storing the tagged 3D map image in the tagged 3D map storage means, It is characterized by providing. Further, the store information linking means can link the specified store information with the information of the tag combined with the three-dimensional map image.

  Further, the apparatus further comprises a two-dimensional map storage means for storing the two-dimensional map image and the coordinate information in association with each other, and the tag composition means is based on the coordinate information of the two-dimensional map image and the coordinate information of the specified building. The information of the tag linked to the building is synthesized at the position of the specified building on the two-dimensional map image, and the store information combining means combines the information of the tag with the specified store information. In association with a two-dimensional map image, the storage means stores the two-dimensional map image synthesized with the tag information in the tagged two-dimensional map storage means.

  Further, a specifying means for specifying a 3D map display position existing on a route from the current position of the user to the destination, and a tagged 3D map image corresponding to the specified 3D map display position are the tag Judgment means for judging whether or not it is stored in the attached 3D map storage means, and if the judgment result is NO, a tagged 3D map image corresponding to the 3D map display position is generated. The tag setting means, the tag synthesizing means and the control means for controlling the store information synthesizing means, and the current position updated according to the movement of the user substantially coincides with the 3D map display position on the route, The tagged 3D map image stored in the tagged 3D map storage means or the tagged 3D map image generated as a result of the control by the control means is stored in the user's mobile phone. And transmits to the end.

  Furthermore, a specifying means for specifying a 3D map display position existing on a route from the current position of the user to the destination, and a tagged 3D map image corresponding to the specified 3D map display position are the tag. Judgment means for judging whether or not it is stored in the attached 3D map storage means, and if the judgment result is NO, a tagged 3D map image corresponding to the 3D map display position is generated. The tag setting means, the control means for controlling the tag composition means and the store information composition means, and when receiving an acquisition request for a tagged 3D map image corresponding to the 3D map display position from a portable terminal, The tagged 3D map image stored in the tagged 3D map storage means, or the tagged 3D map image generated as a result of the control by the control means, Characterized in that it sent to The portable terminal.

  In addition, the map image processing apparatus includes a reception unit that receives a captured image transmitted from a terminal device, coordinate information of a viewpoint of the captured image, and a gaze direction of the viewpoint, three-dimensional coordinate information of a building, and the building The building information storage means for storing the store information of the stores housed in the store in association with each other, the building having the coordinate information within a predetermined map area is identified from the building information storage means, and a tag is linked to the identified building. A tag associated with the building at the position of the identified building on the captured image based on the coordinate information and the line-of-sight direction of the captured image and the three-dimensional coordinates of the identified building Tag composition means for synthesizing the information of the store, store information of the store accommodated in the building associated with the tag from the building information storage means, and the information of the tag matches the identified store information. And store information string pickled means characterizing captured images and cord that is, the information of the tag is characterized a transmitting means for transmitting the captured image combined into the terminal device, in that it comprises.

  The map image processing method is a map image processing method in a map image processing apparatus, and the map image processing apparatus includes a three-dimensional map image, coordinate information of a viewpoint position of the three-dimensional map image, and the viewpoint. A three-dimensional map storage device that stores the line-of-sight direction of the position in association with each other, and a building information storage device that stores the three-dimensional coordinate information of the building in association with the store information of the store accommodated in the building. The method includes identifying a building having coordinate information within a predetermined map area from the building information storage device, linking a tag to the identified building, and coordinate information of the three-dimensional map image And a tag synthesis step for synthesizing tag information linked to the identified building on the 3D map image based on the line-of-sight direction and the three-dimensional coordinates of the identified building, Store information associated with the store information of the store accommodated in the building associated with the tag is identified from the building information storage device, and the identified store information is associated with the 3D map image in which the information of the tag is synthesized. And a storing step of storing a 3D map image in which the tag information is combined in a tagged 3D map storage device.

  Further, the present invention is also realized as a program for causing a computer to execute the above steps. This program can be installed or loaded on a computer through various recording media such as an optical disk such as a CD-ROM, a magnetic disk, or a semiconductor memory, or via a communication network.

  Further, in this specification and the like, the term “means” does not simply mean a physical means, but includes a case where the functions of the means are realized by software. Further, the function of one means may be realized by two or more physical means, or the functions of two or more means may be realized by one physical means.

  ADVANTAGE OF THE INVENTION According to this invention, it is a structure which can provide a user's surrounding condition with additional information to a user, Comprising: It becomes possible to provide the service using a 3D map suitable for a portable terminal.

It is a block diagram which shows schematic structure of a map image processing system. It is a block diagram which shows schematic structure of a portable terminal. It is a figure which shows an example of the data structure of 2D map DB. It is a figure which shows an example of the data structure of 3D map DB. It is a figure which shows an example of the data structure of building information DB. It is a figure for demonstrating switching between AR screen, 2D screen, and 3D screen. It is a flowchart which shows an example of the flow of map image generation processing with a bill tag. It is a figure which shows an example of a map image with a bill tag. It is a flowchart which shows an example of the flow of a 1st rally map control process. It is a figure which shows an example of a rally map map image. It is a flowchart which shows an example of the flow of a 2nd rally map control process. It is a flowchart which shows an example of the flow of a 3rd rally map control process. It is a figure which shows an example of a rally map map image.

  Hereinafter, in the embodiment of the present invention, as an example, a map service using a two-dimensional map (hereinafter referred to as “2D map”) and a three-dimensional map (hereinafter referred to as “3D map”) is provided as a mobile phone or the like. A map image processing apparatus suitable for providing to a terminal will be described. Features of the map image processing apparatus according to the present embodiment will be described in more detail with reference to the drawings. In each description, the same elements are denoted by the same reference numerals, and redundant description is omitted.

[Configuration of map image processing system]
FIG. 1 is a block diagram showing a schematic configuration of a map image processing system 100 to which a map image processing apparatus according to the present embodiment is applied. The map image processing system 100 includes a map image processing device 10 and a mobile terminal 20, and is configured by connecting them to each other via a network N.

  The map image processing apparatus 10 includes a main control unit 101, a storage unit 102, a communication unit 103, a bill tag linking unit 104, a 2D map generation unit 105 with a bill tag, a 3D map generation unit 106 with a bill tag, and a route search unit 107 with a rally map function. , Mainly including various function implementing means such as a switching means 108 and an AR (Augmented Reality) processing means 109.

  Although not shown in the figure, the map image processing apparatus 10 has a CPU, a memory such as a ROM and a RAM, an external storage device for storing various information, an input interface, an output interface, a communication interface, and a bus connecting these. Alternatively, a general-purpose server computer can be applied. For example, when the CPU executes a predetermined program stored in a ROM or the like, it functions as each of the function realizing means. The map image processing apparatus 10 may be composed of a single computer or may be composed of a plurality of computers distributed on a network.

  The main control unit 101 controls the entire operation of the map image processing apparatus 10 and the operation of each unit described above. The storage means 102 includes a 2D map image for displaying a 2D map, a 3D map image for displaying a 3D map, building information (building information) regarding a building in the map area, and a building tag in which a building tag is synthesized. A storage device that stores a 2D map image and a 3D map image with a bill tag, and has a function as a database. The storage means 102 is also referred to as a 2D map DB, a 3D map DB, a building information DB, a 2D map DB with a bill tag, and a 3D map DB with a bill tag according to the stored information. Hereinafter, the data structure of the data stored in the storage unit 102 will be described with reference to FIGS. 3 to 5, but the data structure is not limited to the following, and can be changed as appropriate according to the design.

  FIG. 3 shows an example of the data structure of the 2D map DB. In the 2D map DB, for example, a map ID of a 2D map, map coordinate information (latitude and longitude), and an image of the map (2D map image data) are associated with each other. A 2D map is, for example, a mesh map, but is not limited thereto.

  FIG. 4 shows an example of the data structure of the 3D map DB. The 3D map is, for example, 3D map image data such as a photographed image (still image) of a landscape on a map photographed by a camera or a CG image created using a virtual camera, a map ID of the 3D map, and the map The coordinate information of the viewpoint, the direction, the line-of-sight direction of the camera (viewpoint), the focal length, the imaging method (vertical or horizontal), the image aspect ratio, and the like are associated with each other. Information such as the focal length, shooting method, and image aspect ratio can be acquired from, for example, Exif information when a landscape shot by a camera is saved in JPEG format. In this embodiment, a 3D map is prepared for a landscape in a predetermined line-of-sight direction at a predetermined viewpoint position in the map area. However, the predetermined viewpoint position and line-of-sight direction are not particularly limited. This applies to places and intersections where buildings (such as buildings, pedestrian bridges, railway guards, railroad crossings, and landmarks) that are likely to be located are present.

  FIG. 5 shows an example of the data structure of the building information DB. The building information is information regarding a predetermined building included in the map area. For example, identification information (building ID) for uniquely identifying a building and building attribute information are associated with each other. The attribute information of the building is, for example, the building name of the building, the three-dimensional coordinate information (latitude, longitude, altitude) of the representative point, the four-dimensional coordinates of the four corners of the building, the store information regarding the stores housed in the building, . Although not shown in the figure, the store information includes a store name and other detailed information (address, telephone number, etc.).

  Returning to FIG. 1, description of each function realizing means will be continued. The communication means 103 is an interface for transmitting and receiving various types of information by communicating with the mobile terminal 20 via the network N. The communication unit 103 also functions as a reception unit that receives the current position and the destination specifying information transmitted from the mobile terminal 20.

  The building tag linking unit 104 links a building tag to a building on the map. The building tag (also referred to as “tag”) is associated with a building (building) as a unit, and is identification information that uniquely identifies the identified building within a predetermined map area. For example, the numbers 1 to N are linked in order according to the number of buildings. Specifically, the building tag associating means identifies a predetermined building having coordinate information within a predetermined map area from the building information DB, and associates a building tag with the building ID of the identified building (1 to N). To do.

  The building tag-added 2D map generation means 105 generates a building tag-added 2D map image by synthesizing the corresponding building tag with the 2D map image. Specifically, a 2D map image having coordinate information within a predetermined map area is acquired from the 2D map DB, and the position of the corresponding building on the 2D map image is based on the coordinate information of the building associated with the building tag. The information representing the bill tag (hereinafter referred to as “building tag object”) is synthesized.

  The 2D map generating unit 105 with the building tag further associates the store information of the building with the 2D map image. For example, store information housed in a building associated with a building tag is acquired from the building information DB, and the acquired store information is synthesized so as to be displayed adjacent to the 2D map image. Moreover, you may link the shop information of a building with the building tag object synthesize | combined with 2D map image. Thereby, when the user clicks on the bill tag object displayed on the 2D map image, the store information associated with the bill tag object is displayed. Note that the association of the store information with the building tag object can be realized by applying a conventional technique such as setting a link to the store information in the coordinate area of the building tag object on the 2D map image. The building tag-attached 2D map generating unit 105 stores the generated 2D map image with the building tag in the 2D map DB with the building tag.

  Note that the 2D map generating unit with bill tag 105 may combine other information as appropriate in addition to the bill tag and the store information. For example, at a predetermined point on the 2D map, the user can use a 3D map image with a bill tag, which will be described later. Therefore, information (hereinafter referred to as “3D map object”) representing a point where the 3D map image with the bill tag is displayed (hereinafter referred to as “3D map image display point”) may be synthesized. In addition, route guidance sign information (for example, a sign described as “to OO”), arrow information indicating the traveling direction / line-of-sight direction, and the like can be combined with the 2D map image.

  The 3D map generation means 106 identifies the building tag of the building included in the target 3D map area from the buildings included in the predetermined map area, and combines the identified building tag with the 3D map image to add the building tag. A 3D map image is generated. Specifically, a 3D map image having coordinate information within a predetermined map area is acquired from the 3D map DB, and the coordinate information of the 3D map image, the camera line-of-sight direction and other attached information at the time of shooting (azimuth, focal length, shooting method) (Vertical / horizontal) etc.) and the coordinate information of the building to which the building tag is linked (three-dimensional coordinates of representative points, four-dimensional two-dimensional coordinates, etc.) A building tag object is synthesized with the position and height of the identified building on the 3D map. The 3D map generation unit 105 stores the generated 3D map image with the bill tag in the 3D map DB with the bill tag.

  Note that the 3D map generation unit 105 may associate the store information of the building with a 3D map image or a building tag object synthesized with the 3D map image. Since the association is the same as the association of the store information to the 2D map image or the building tag object synthesized with the 2D map image by the 2D map generation unit 105, the description is omitted.

  Further, the 3D map generation unit 105 may appropriately combine other information in addition to the building tag and the store information. For example, sign information for route guidance (for example, a sign described as “to ○○”) and traveling direction information (for example, an arrow extending in the traveling direction) indicating the traveling direction of the route can be synthesized. In this case, in the same way as the building tag synthesis, the coordinate position of the 3D map image, the line-of-sight direction and other attached information at the time of shooting (direction, focal length, shooting method (vertical / horizontal), etc.) are taken into consideration Determine the direction. Further, the same sign information is synthesized so as to be superimposed and displayed in the same manner in both the 3D map image and the 2D map image.

  The route search unit with a rally map function 107 specifies a route from the current position received by the communication unit 103 to the destination, and generates predetermined route guidance information. The route guidance information corresponds to, for example, various types of information displayed on the map (an arrow indicating the traveling direction, a line indicating the entire route, a distance to the destination), voice information for guiding the user, and the like. Then, route search support is performed using the 2D map image with the bill tag and the 3D map image with the bill tag. Specifically, a 2D map image with a bill tag on which route guidance information from the current position to the destination is superimposed is provided to the user, while a 3D map image with a bill tag is provided at a predetermined position (for example, a corner) on the route. I will provide a. There is no particular limitation on the form of providing the map image, but in the present embodiment, the following three forms will be described.

  The first form of providing a 3D map image with a bill tag appropriately specifies the current position of the user on the route based on the current position information transmitted from the mobile terminal, and the user is in the vicinity of the 3D map image display point (predetermined) If it is determined that it is within the distance, the corresponding 3D map image with the bill tag is transmitted to the mobile terminal 20. The second form of provision assumes a case where the user's mobile terminal does not have a GPS, or is in a situation where the current position cannot be transmitted due to radio waves, based on a map acquisition request from the user. Send 3D map image with bill tag. In the third provision form, when a photographed image photographed by the user at an arbitrary point on the route is sent, tag information corresponding to the photographed image is attached and sent back to the portable terminal.

  Based on the instruction transmitted from the mobile terminal 20, the switching unit 108 performs control so that the 2D map screen with the bill tag, the 3D map screen with the bill tag, and the AR screen are appropriately switched. FIG. 6 shows how the AR screen 401 is switched to the 2D map screen 402 or the 3D map screen 403. According to the figure, the AR screen 401 includes the video 401a, the building tag 401b, the store information 401c, and the route guidance information 401d. Therefore, it is difficult to recognize the background video 401a itself. . Therefore, for example, when a user transmits a 3D map screen request to the map image generation device 10, a 3D map image with a bill tag is transmitted. Therefore, the user can easily grasp peripheral information from a 3D image imitating an actual video. Can do.

  The AR (Augmented Reality) processing means 109 exists in the direction of the camera at the time of photographing when the image of the subject photographed by the photographing means (camera) 206 attached to the portable terminal 20 is displayed on the display. Control is performed so that additional information representing a building or the like to be displayed is synthesized and displayed on the video displayed on the display. The additional information is not particularly limited, and is, for example, a building tag or store information.

[Configuration of mobile terminal]
The mobile terminal 20 is a mobile phone, PDA, personal computer, portable navigation device, or other terminal device that can display map data. As shown in FIG. 2, the portable terminal 20 includes various function realizing means such as a main control means 201, a storage means 202, an input means 203, a display means 204, a communication means 205, a photographing means 206, and a position / orientation measuring means 207. Prepare.

  Although not shown, the portable terminal 20 includes a CPU that controls processing and operations of the portable terminal, a memory such as a ROM and a RAM, an external storage device that stores various information, an input interface, an output interface, a communication interface, a camera A dedicated or general-purpose computer including a GPS receiver, an electronic compass, and a bus connecting them can be applied. In addition, although the one portable terminal 20 is described in FIG. 1, the some portable terminal 20 can also be connected with the map image processing apparatus 10 according to a utilization form. When the mobile terminal 20 receives information input from the user by executing a predetermined program stored in a memory or an external storage device, the mobile terminal 20 receives a map display request or the like based on the received information. When transmitting to the device 10 and receiving map data or the like from the map image processing device 10, a function of displaying a map image on the display is realized based on this.

  The network N is a communication line for transmitting and receiving information between the map image processing apparatus 10 and the mobile terminal 20. For example, it may be any of the Internet, a LAN, a dedicated line, a packet communication network, a telephone line, a corporate network, other communication lines, combinations thereof, and the like, regardless of whether they are wired or wireless.

[Generation of map image with building tag]
Next, the outline | summary of operation | movement of the map image processing system 100 comprised as mentioned above is demonstrated using FIGS. In addition, each processing step to be described later can be executed in any order or in parallel as long as there is no contradiction in processing contents, and other steps may be added between the processing steps. . Further, a step described as one step for convenience can be executed by being divided into a plurality of steps, while a step described as being divided into a plurality of steps for convenience can be grasped as one step.

  FIG. 7 is a flowchart illustrating an example of the flow of a bill tag-added map image generation process. The building tag linking means 104 extracts a building included in the predetermined map area from the building information DB based on the coordinate information that defines the predetermined map area, and links the building tag to the extracted building (S101). . Specifically, the building ID of the extracted building is associated with each building tag. Then, by referring to the building information DB based on the building ID of the extracted building, store information of the building is acquired and a store list is generated (S102).

  Next, the 3D map generation unit 106 extracts a 3D map image corresponding to the predetermined map area from the 3D map DB based on the coordinate information defining the predetermined map area (S103). Next, processing for generating a 3D map image with a bill tag is executed for the extracted 3D map image (S104 to S108). That is, a building tag synthesis position on the 3D map image is identified based on the three-dimensional coordinates of the building to which the tag is linked, the coordinates of the 3D map image, and the line-of-sight direction, and a building tag object is synthesized at the identified building tag synthesis position. (S105). Further, the store list is associated with a 3D map image or a bill tag object (S106). Note that the 3D map generation unit 106 may synthesize other various information (for example, a marker sign such as “Too”).

  The 3D map image combined with the bill tag object is stored in the 3D map DB with a bill tag in association with the coordinate information and the line-of-sight direction (S107). The 3D map generation unit 106 repeatedly executes the generation process for the extracted 3D map image until a 3D map image with a bill tag is generated.

  Next, the 2D map generating unit 105 with the bill tag extracts a 2D map image corresponding to the predetermined map area from the 2D map DB based on the coordinate information defining the predetermined map area (S109). Next, based on the coordinates of the building to which the tag is linked and the coordinates of the 2D map image, a building tag synthesis position on the 2D map image is specified, and a building tag object is combined with the specified building tag synthesis position (S110). . Then, the store list is associated with the 2D map image and the bill tag object (S111). The 2D map image combined with the bill tag object is stored in the 2D map DB with the bill tag in association with the coordinate information (S112).

  As described above, the 3D map image with the bill tag and the 2D map image with the bill tag are generated for each predetermined map area and stored in the respective storage means. FIG. 8 shows an example of the generated 3D map image with a bill tag and 2D map image with a bill tag. 403 in FIG. 8 shows a state in which the bill tag objects (1 to 9) and the 3D map objects (A to F) are superimposed and displayed on the 2D map image. 404 of FIG. 8 represents the 3D map image with a bill tag displayed at each 3D map image display point (AF).

[Rally map control processing]
[First Rally Map Control Process]
FIG. 9 is a flowchart showing an example of the flow of the first rally map control process.
When the route search unit 107 with the rally map function receives a route search request (including the destination) from the mobile terminal 20 (S201; YES), the route search unit 107 with the rally map function acquires a current position periodically transmitted from the mobile terminal 20 (S202). . Then, the route from the current position to the destination is determined by referring to the route search DB (not shown) (S203), and the 3D image display point (coordinate information) existing on the route is determined from the 3D map DB. Specify (S204).

  Then, the 2D map image with the bill tag at the current position is extracted from the 2D map DB with the bill tag, and the route guidance information is synthesized on the extracted 2D map image with the bill tag (see 405a in FIG. 10) on the 3D map image display point. Then, the 3D map object is synthesized and transmitted to the mobile terminal 20 (S205) (see 405 in FIG. 10). In addition, when the 2D map image with a bill tag is not stored, you may generate by controlling the 2D map generation means 105 with a bill tag.

  The route search unit with rally map function 107 determines whether or not a 3D map image display point exists within a predetermined distance from the current position (S206). If the determination result is correct, it is determined whether the 3D map image with the bill tag at the 3D map image display point is stored in the 3D map DB with the bill tag (S207).

  If the determination result is right (S207; YES), the corresponding 3D map image with the bill tag is read from the 3D map DB with the bill tag and transmitted to the mobile terminal 20 (S209). On the other hand, if the determination result is negative (S207; NO), the 3D map generation means 106 is controlled to generate a 3D map image with a bill tag (S208). Note that generation of a 3D map image with a bill tag has already been described with reference to FIG. 7, and thus description thereof is omitted here. Then, the generated 3D map image with the bill tag is transmitted to the mobile terminal 20 (S208) (see 406 in FIG. 10). The generated 3D map image with the bill tag may be stored in the 3D map DB with the bill tag.

  The route search unit 107 with the rally map function determines whether or not the current position appropriately transmitted from the mobile terminal 20 has passed the 3D map display point (S210). If it has passed (S210; YES), it is determined whether or not there is a 3D map image to be displayed next (S211; YES). If the determination result is positive, the next 3D map image is displayed. Processing is executed (S206 to S209). On the other hand, if the determination result is NO, there is no 3D map image to be displayed next, and the process is terminated (S211; NO).

  According to the above, when a route search result is provided in the form of a rally map, a 3D map image with a bill tag corresponding to the current position of the user is automatically generated and transmitted, or read from a 3D map DB with a bill tag. Can be sent.

[Second Rally Map Control Process]
Next, the second rally map control process will be described. FIG. 11 is a flowchart showing an example of the flow of the second rally map control process. In the second rally map control process, as described above, a 3D map image with a bill tag is transmitted to the mobile terminal 20 in response to a user request.

  In the second rally map control process, the first step 310 is the same as the processes in steps S201 to S205 in FIG.

  The route search unit with rally map function 107 determines whether or not a 3D map image request has been received from the mobile terminal 20 (S311). If the determination result is correct, it is determined whether or not the 3D map image with the bill tag at the point designated by the user is stored in the 3D map DB with the bill tag (S312). If the determination result is right (S312; YES), the corresponding 3D map image with the bill tag is read from the 3D map DB with the bill tag and transmitted to the mobile terminal 20 (S314).

  On the other hand, if the determination result is negative (S312; NO), the 3D map generation means 106 is controlled to generate a 3D map image with a bill tag (S313). Note that generation of a 3D map image with a bill tag has already been described with reference to FIG. 7, and thus description thereof is omitted here. Then, the generated 3D map image with the bill tag is transmitted to the mobile terminal 20 (S313). The generated 3D map image with the bill tag may be stored in the 3D map DB with the bill tag.

  The route search means with rally map function 107 determines whether there is a 3D map image to be displayed next (S315; YES). If the determination result is YES, the next 3D map image is displayed. Processing is executed (S311 to S314). On the other hand, if the determination result is NO, there is no 3D map image to be displayed next, and the process is terminated (S315; NO).

  According to the above, when a route search result is provided in the form of a rally map, even if the current position cannot be obtained from the user's mobile terminal, the corresponding 3D map image with the bill tag is obtained according to the user's acquisition request. It can be automatically generated or read from a 3D map DB with a bill tag and transmitted.

[Third rally map control process]
Next, the third rally map control process will be described. FIG. 12 is a flowchart illustrating an example of the flow of the third rally map control process. In the third rally map control process, as described above, a 3D map image with a bill tag is generated (AR bill tag generation) based on the captured image captured by the user.

  When the route search unit 107 with the rally map function receives a route search request (including the destination) from the mobile terminal 20 (S401: YES), the route search unit 107 with the rally map function acquires the current position periodically transmitted from the mobile terminal 20, The route from the position to the destination is determined by referring to a route search DB (not shown) (S402). Then, the 2D map image with the bill tag at the current position is extracted from the 2D map DB with the bill tag, and the route guidance information (407a in FIG. 13) is synthesized on the extracted 2D map image with the bill tag and transmitted to the mobile terminal 20. (S403) (see 407 in FIG. 13).

  The route search means with rally map function 107 determines whether or not a composite request for a shot still image has been received (S404). If the determination result is correct (S404; YES), the current position (viewpoint position) and the camera direction are determined. A wire frame that matches (line-of-sight direction) is identified from the 3D map DB, and the identified wire frame is transmitted to the mobile terminal 20 (S405). The user displays the received wire frame on the display, operates the mobile terminal 20 so that the subject matches the wire frame, and images the subject at a timing when the two substantially match. The user transmits the captured still image to the map image processing apparatus 10.

  When the route search means 107 with the rally map function receives the captured still image captured in accordance with the wire frame (S406), the bill tag information is combined with the captured still image based on the current position and the camera direction (S407). The photographed still image with the bill tag combined with the bill tag is transmitted to the portable terminal 20 (see 408 in FIG. 13).

  According to the above, when a route search result is provided in the form of a rally map, a bill tag can be assigned to the video captured by the user.

  As described above, in the present embodiment, since the building tag is provided to the user by attaching the building tag to the three-dimensional map image imitating a real landscape, for example, even when the service is used at night or in rainy weather It is possible to provide both the corresponding video and additional information in a form that is visible to the user. Therefore, the user can surely check the surrounding video as compared with the case of using augmented reality.

  In this embodiment, since the map image processing device generates a 3D map image or a 2D map image and transmits the generated data to the mobile terminal, the load on the mobile terminal is reduced. It is possible to provide a service that takes into account problems such as processing capacity and network communication load.

  Further, in this embodiment, since the attached information is displayed in units of buildings, the overlapping of building tags is reduced, and the user can easily grasp the relationship between the object (building) in the video and the attached information. As a result, the convenience of the route search service is improved.

  Moreover, in this embodiment, since both the 2D map image and the 3D map image are provided in the rally map, the user can confirm the route to the vicinity of the current position and the destination by the 2D map image. At the same time, the scenery at a predetermined position on the route can be confirmed from the 3D map image, so that the user can smoothly move toward the destination on the route.

[Other Embodiments]
The present invention is not limited to the above-described embodiment, and can be implemented in various other forms without departing from the gist of the present invention. The above-described embodiment is merely an example in all respects, and is not construed as limiting.

DESCRIPTION OF SYMBOLS 10 ... Map image processing apparatus, 20 ... Portable terminal, 100 ... Map image processing system, 101 ... Main control means, 102 ... Memory | storage means, 103 ... Communication means, 104 ... Building tag linking means, 105 ... 2D map generation means with bill tag, 106 ... 3D map generation means with bill tag, 107 ... Route search means, 108 ... Switching means, 109 ... AR processing means , N ... Network

Claims (8)

3D map storage means for storing a 3D map image, coordinate information of a viewpoint position of the 3D map image, and a line-of-sight direction of the viewpoint position in association with each other;
Building information storage means for storing the three-dimensional coordinate information of the building and store information of the store accommodated in the building in association with each other;
A tag linking unit for identifying a building having coordinate information within a predetermined map area from the building information storage unit, and linking a tag to the identified building;
A tag linked to the specified building position on the three-dimensional map image based on the coordinate information and the line-of-sight direction of the viewpoint position of the three-dimensional map image and the three-dimensional coordinate information of the specified building Tag synthesis means for synthesizing the information of
Store information that identifies store information of a store accommodated in a building associated with the tag from the building information storage unit, and associates the identified store information with a three-dimensional map image in which the information of the tag is combined. Linking means,
Storage means for storing in the tagged 3D map storage means a 3D map image in which the tag information is synthesized;
A map image processing apparatus comprising: The store information linking means is:
The map image processing apparatus according to claim 1, wherein the specified store information is associated with information of the tag combined with the three-dimensional map image. 2D map storage means for storing a 2D map image and coordinate information in association with each other;
The tag synthesizing means includes
Based on the coordinate information of the two-dimensional map image and the coordinate information of the identified building, the information of the tag associated with the building is synthesized at the position of the identified building on the two-dimensional map image,
The store information synthesizing means is
The identified store information is linked to the two-dimensional map image in which the information of the tag is synthesized,
The storage means includes
The map image processing apparatus according to claim 1 or 2, wherein the two-dimensional map image synthesized with the tag information is stored in a tagged two-dimensional map storage means. A specifying means for specifying a 3D map display position existing on a route from the current position of the user to the destination;
Determining means for determining whether or not a tagged 3D map image corresponding to the specified 3D map display position is stored in the tagged 3D map storage means;
Control that controls the tag setting means, the tag composition means, and the store information composition means so that a tagged three-dimensional map image corresponding to the three-dimensional map display position is generated when the determination result is negative Means,
When the current position updated according to the movement of the user substantially matches the 3D map display position on the route, the tagged 3D map image stored in the tagged 3D map storage means, or The map image processing apparatus according to any one of claims 1 to 3, wherein a tagged three-dimensional map image generated as a result of control by the control unit is transmitted to the mobile terminal of the user. A specifying means for specifying a 3D map display position existing on a route from the current position of the user to the destination;
Determining means for determining whether a tagged 3D map image corresponding to the identified 3D map display position is stored in the tagged 3D map storage means;
Control that controls the tag setting means, the tag composition means, and the store information composition means so that a tagged three-dimensional map image corresponding to the three-dimensional map display position is generated when the determination result is negative Means,
When an acquisition request for a tagged 3D map image corresponding to the 3D map display position is received from a mobile terminal, the tagged 3D map image stored in the tagged 3D map storage means, or the control 5. The map image processing apparatus according to claim 1, wherein the tagged three-dimensional map image generated as a result of control by the means is transmitted to the mobile terminal of the user. Receiving means for receiving a captured image transmitted from the terminal device, coordinate information of a viewpoint of the captured image, and a line-of-sight direction of the viewpoint;
Building information storage means for storing the three-dimensional coordinate information of the building and store information of the store accommodated in the building in association with each other;
A tag linking unit for identifying a building having coordinate information within a predetermined map area from the building information storage unit, and linking a tag to the identified building;
A tag that synthesizes information of a tag linked to the specified building position on the captured image based on the coordinate information of the viewpoint of the captured image, the line-of-sight direction, and the three-dimensional coordinates of the identified building Combining means;
Store information associated with the store information stored in the building associated with the tag is identified from the building information storage means, and the identified store information is associated with the captured image obtained by combining the tag information. Means,
Transmitting means for transmitting the captured image in which the tag information is combined to the terminal device;
A map image processing apparatus comprising: A map image processing method in a map image processing device, wherein the map image processing device associates a three-dimensional map image, coordinate information of a viewpoint position of the three-dimensional map image, and a gaze direction of the viewpoint position. A three-dimensional map storage device for storing, and a building information storage device for storing the three-dimensional coordinate information of the building and store information of the store accommodated in the building in association with each other.
A tag linking step of identifying a building having coordinate information within a predetermined map area from the building information storage device, and linking a tag to the identified building;
Based on the coordinate information of the viewpoint position of the three-dimensional map image, the line-of-sight direction, and the three-dimensional coordinates of the specified building, information on the tag linked to the specified building position on the 3D map image A tag synthesis step for synthesizing
Store information string for identifying store information of a store housed in the building associated with the tag from the building information storage device, and associating the identified store information with a 3D map image in which the information of the tag is synthesized. Step,
A storage step of storing in the tagged 3D map storage device a 3D map image in which the tag information is synthesized;
A map image processing method comprising:   The program for making a computer perform each step of Claim 7.