JP5413170B2 - Annotation display system, method and server apparatus - Google Patents

Description

  The present invention relates to an augmented reality technology in which information obtained from a real space is superimposed with information artificially created using computer technology.

  There is a service that uses augmented reality (Augmented Reality) that superimposes information (eg, computer graphics) created artificially using computer technology on information (eg, images taken with a digital camera) obtained from real space. Has attracted attention.

  As a service using augmented reality, as shown in Non-Patent Document 1, three-dimensional computer graphics corresponding to a marker included in an image photographed by a digital camera can be displayed on the image according to the direction of the marker. Although being considered, augmented reality services that display annotation information related to objects (such as buildings) captured by a mobile terminal on an image captured by a digital camera built in the mobile terminal are also available Developed for some handset models.

  Although it may not be directly related to augmented reality, Patent Document 1 treats a mobile terminal like a pointing device and provides information related to an object (such as a building) pointed to by the mobile terminal. A system that can be provided is disclosed.

  The annotation information displayed on the screen of the mobile terminal may be created by the augmented reality service provider, or may be posted by the augmented reality service user. For example, in an augmented reality application used for subway station guidance, an object created by an augmented reality service provider is used as an annotation to display an exit guide or a store signboard.

  In addition, in augmented reality services used in social services, post information posted by a user using the augmented reality service, such as characters, images, and sounds, is used as annotation information.

  When posting information posted by a user who uses an augmented reality application is used as annotation information, if the number of posted information posted by the user increases, too much annotation information is displayed on the screen of the mobile terminal. Because it is difficult to see the screen, some augmented reality services used in social services include annotation information depending on the time the user posted the posted information, the distance from the location of the mobile terminal to the location where the posted information was posted, etc. Some applications can filter

  On the other hand, focusing on mobile terminals that can use augmented reality applications, in-vehicle kits for mobile terminals are also available on the market, and the range of movement speed when users use mobile terminals is large, and users use mobile terminals. It is desired that the amount of annotation information to be displayed on the mobile terminal can be changed according to the moving speed at the time.

  For example, in situations where the moving speed is fast, such as when the user uses a mobile terminal in a car, the amount of annotation information displayed on the mobile terminal is reduced, and the moving speed is low when the user uses the mobile terminal while walking. In a slow situation, it is desirable from the viewpoint of user safety and convenience to increase the amount of annotation information displayed on the mobile terminal.

  As an invention for changing the display contents according to the moving speed of the user, Patent Document 2 has already disclosed a system for automatically changing the direction name and the like according to the scrolling speed when the map is scrolled. However, since the system disclosed in Patent Document 2 targets place names registered in advance in map information, it cannot be applied to post information posted every day.

JP-T-2004-531791 JP 2005-292621 A

Kato, H., Billinghurst, M. (1999) Marker Tracking and HMD Calibration for a video-based Augmented Reality Conferencing System. In Proceedings of the 2nd International Workshop on Augmented Reality (IWAR 99). October, San Francisco, USA.

  Therefore, an object of the present invention is to provide an annotation display system and method that can change the amount of annotation information displayed on a mobile terminal in accordance with the moving speed of the mobile terminal.

  The first invention that solves the above-described problem includes at least an annotation information based on a posting point on an augmented reality service map that includes at least a server device that receives an annotation information transmission request and a portable terminal that makes an annotation information transmission request. The portable terminal includes a camera unit that captures an image, a position measuring unit that measures position information of the portable terminal, and position information at this time obtained from the position measuring unit. An annotation display means for making a transmission request for annotation information to the server apparatus by transmitting to at least the server apparatus, and combining the annotation information received from the server apparatus with a video photographed by the camera means for display. The server device includes an augmented reality service. A plurality of grids each having a different grid size are set to divide the map, and in association with the cell number assigned to each cell surrounded by the grid, the posting point is within the cell area specified by the cell number. Each grid has a cell database that stores information that aggregates all posted information including location information as cell information, and further receives a transmission request from the mobile terminal when the transmission request for the annotation information is received from the portable terminal. At least using the position information of the mobile terminal at the time, obtaining an annotation area for displaying the annotation information, and then specifying the cell corresponding to the annotation area using the grid corresponding to the moving speed of the mobile terminal The cell information associated with the cell number of the identified cell Annotation transmission that is obtained from the cell database of the lid, generates the number of the cells for which the annotation information is specified using a part or all of the acquired cell information, and transmits the generated annotation information to the mobile terminal An annotation display system characterized by comprising means.

  According to the first invention, for example, the grid size of the grid used when the moving speed of the mobile terminal is high is increased, and the grid size of the grid used when the moving speed of the mobile terminal is low is reduced. As described above, if a plurality of grids having different grid sizes are set in the server device, the number of cells corresponding to the annotation area decreases when the moving speed of the mobile terminal is high, and the mobile The number of annotation information to be transmitted to the portable terminal increases when the moving speed of the terminal is small, and the number of annotation information decreases according to the moving speed of the portable terminal. The amount of annotation information displayed on the screen can be changed.

  Further, according to a second aspect of the present invention, the server device has a predetermined number of the keywords in descending order of importance from keywords obtained by natural language analysis of the posting information including a posting point in the cell area. Is stored as the cell information. An annotation display system according to the first invention.

  The cell information that is a summary of all the posted information in which the location information of the posting point is included in the cell area may be a summary sentence in which the posted information is aggregated. Then, the user of the portable terminal can easily guess the contents of the annotation information only by looking at the annotation information.

  Further, the third invention is characterized in that the annotation transmission means of the server device changes the number of the keywords used as the annotation information according to the moving speed of the portable terminal. An annotation display system described in 1.

  As in the third aspect of the invention, the number of keywords used as the annotation information can be changed, and the number of keywords used as the annotation information can be reduced when the moving speed of the portable terminal is fast. It is desirable to make it.

  Further, according to a fourth aspect of the present invention, the server device includes a posting information database that stores the posting information received from the portable terminal in association with the position information of the posting point, and the annotation transmission unit includes: When the moving speed is stationary, the posting information in which the position information of the posting point is included in the cell area corresponding to the annotation area is acquired from the posting information database, and the annotation information is acquired using the acquired posting information. The annotation display system according to any one of the first to third inventions, characterized in that information is generated.

  When the moving speed of the mobile terminal is stationary, it can be considered that the user of the mobile terminal is looking for more detailed information. Therefore, it is desirable to increase the information amount of the annotation information as in the fourth invention. .

  Further, according to a fifth aspect of the present invention, the portable terminal includes an azimuth angle measuring unit that measures an azimuth angle, and the annotation display unit transmits the annotation information to the server device when the azimuth angle measuring unit is requested. The annotation transmission means of the server device is a means for obtaining a photographing azimuth angle that the camera means is directed from an azimuth angle obtained from the above and transmitting the photographing azimuth angle together with the position information of the mobile phone to the server device. When an annotation information transmission request is received from the portable terminal, an annotation area is determined using position information and a shooting azimuth angle received from the portable terminal. The annotation display system according to any one of the above.

  According to the fifth invention, the annotation information related to the posted information posted in the shooting direction of the camera means of the portable terminal can be displayed.

  Further, the sixth invention is characterized in that the mobile terminal includes post information transmitting means for transmitting the post information set by a user to the server device and requesting the server device to register the post information, When receiving a registration request for the posting information from the mobile terminal, the apparatus stores the posting information in the posting information database in association with the location information of the posting point of the posting information for which the registration request has been received. The cell number of the cell that includes the location information of the mobile terminal when receiving the request is specified for each grid, and the cell information stored in the cell database in association with the specified cell number, An annotation according to the fourth or fifth aspect, comprising post information registration means for updating using post information from the portable terminal. It is a display system.

  The information stored in the database provided in the server device can be generated by a device other than the server device. However, according to the sixth invention, the server device is provided with the server device. The contents of the database can be updated.

  Furthermore, the seventh invention is a server device that transmits the annotation information to a mobile terminal that has requested transmission of annotation information, and a plurality of grids each having a different grid size for partitioning an augmented reality service map are set. Cell information is obtained by associating all the pieces of posted information including the location information of the posting point in the cell area specified by the cell number in association with the cell number assigned to each cell surrounded by the grid. For each grid, and when receiving a transmission request for the annotation information from the portable terminal, at least using the position information of the portable terminal when the transmission request is received, the annotation information After obtaining the annotation area to display The cell corresponding to the annotation area is specified using a grid, the cell information associated with the cell number of the specified cell is acquired from the cell database of the grid, and a part of the acquired cell information Alternatively, the server apparatus is characterized by comprising annotation transmission means for generating the same number of the specified annotation information as the number of the cells and transmitting the generated annotation information to the portable terminal.

  Further, according to an eighth aspect of the invention, the server device has a predetermined number of the keywords in descending order of importance from keywords obtained by natural language analysis of the posting information in which the posting point is included in the cell area. Is stored as the cell information. A server device according to a seventh aspect of the invention.

  Furthermore, the ninth invention is characterized in that the annotation transmission means of the server device changes the number of keywords used as the annotation information in accordance with the moving speed of the portable terminal. It is a server apparatus as described in.

  Furthermore, the tenth invention includes a posting information database that stores the posting information received from the portable terminal and the position information of the posting point in association with each other, and the annotation transmission means is configured such that the moving speed of the portable terminal is stationary. The post information in which the position information of the posting point is included in the cell area corresponding to the annotation area is acquired from the post information database, and the annotation information is generated using the acquired post information. A server device according to any one of the seventh to ninth inventions.

  Further, according to an eleventh aspect, when the annotation transmission means receives a transmission request for the annotation information from the mobile terminal, the annotation transmission means determines an annotation area using the position information and the shooting azimuth angle received from the mobile terminal. The server device according to any one of the seventh to tenth inventions.

  Furthermore, the twelfth aspect of the invention is to store the posted information in the posted information database when receiving a registration request for the posted information from the portable terminal, and to store the position information of the portable terminal when the registered request is received. Post that identifies the cell number of the cell included in the area for each grid and updates the cell information associated with the identified cell number and stored in the cell database using post information from the mobile terminal The server device according to the tenth invention or the eleventh invention, characterized by comprising information registration means.

  The seventh to twelfth inventions relate to a server apparatus necessary for realizing the annotation display system according to the first to sixth inventions.

  Further, the thirteenth invention is an annotation display method for displaying annotation information based on a posting point on an augmented reality service map on a mobile terminal, wherein a plurality of grids having different grid sizes separating the augmented reality service map are provided. Information that aggregates all posting information that is set and associated with the cell number assigned to each cell surrounded by the grid and that includes the location information of the posting point within the area of the cell specified by the cell number When the server apparatus having the cell database storing cell information for each grid receives a transmission request for annotation information from the mobile terminal, the position information of the mobile terminal when the transmission request is received Use at least the annotation area to display the annotation information Step a, the server device identifies the cell corresponding to the annotation area using the grid corresponding to the moving speed of the mobile terminal, and the cell information associated with the cell number of the identified cell Steps of acquiring from the cell database of the grid, generating the number of cells that specify the annotation information using a part or all of the acquired cell information, and transmitting the generated annotation information to the mobile terminal This is an annotation display method characterized in that b is executed.

  Furthermore, in a fourteenth aspect of the invention, the server device has a predetermined number of the keywords in descending order of importance from keywords obtained by natural language analysis of the posting information including a posting point in the cell area. Is stored as the cell information. An annotation display method according to the thirteenth aspect of the invention.

  Furthermore, the fifteenth invention is characterized in that, in the step b, the server device changes the number of keywords used as the annotation information in accordance with the moving speed of the portable terminal. The annotation display method described in 1.

  Further, in a sixteenth aspect of the present invention, in the step a, when the server device receives a transmission request for the annotation information from the portable terminal, the server apparatus uses the positional information and the shooting azimuth angle received from the portable terminal to perform the annotation. The annotation display method according to any one of the thirteenth to fifteenth inventions, wherein an area is determined.

  Further, in a seventeenth aspect of the present invention, in the step b, the server device is configured such that the annotation transmitting unit is configured to determine the posting information received from the portable terminal and the posting point when the moving speed of the portable terminal is stationary. Refers to a posting information database that stores positional information in association with each other, acquires the posting information in which the location information of the posting point is included in the cell area corresponding to the annotation area from the posting information database, and acquires the posting The annotation display method according to any one of the thirteenth to sixteenth aspects, wherein the annotation information is generated using information.

  The thirteenth to seventeenth inventions relate to the method, and can achieve the same effects as the first to fifth inventions.

  Thus, according to the present invention, it is possible to provide an annotation display system and method that can change the amount of annotation information displayed on a mobile terminal in accordance with the moving speed of the mobile terminal.

The figure explaining the annotation display system in this embodiment. The figure explaining the hardware with which the portable terminal was equipped. The figure explaining the function with which the portable terminal was equipped. The flowchart explaining the process which the annotation display means of a portable terminal performs. The flowchart explaining the process which the posting means of a portable terminal performs. The figure which showed an example of the icon which starts the contribution means of a portable terminal. The figure explaining the hardware with which the server apparatus was equipped. The figure explaining the function with which the server apparatus was equipped. The figure explaining contribution information DB with which the server apparatus was equipped. The figure explaining cell DB with which the server apparatus was equipped. The flowchart explaining the process which the posting information registration means of a server apparatus performs. The flowchart explaining the process which the annotation transmission means of a server apparatus performs. The figure explaining an example of the rule which defined the content of the classification of moving speed. The figure explaining the annotation area in this embodiment. The figure explaining that the number of cells contained in an annotation area changes with change of a grid size. FIG. 1 is a first diagram illustrating annotation information displayed on a mobile terminal. FIG. 2 is a second diagram illustrating annotation information displayed on a mobile terminal.

  From this point, an embodiment of the present invention will be described to the extent that a person skilled in the art of the present invention can understand the contents of the present invention and implement the present invention.

  FIG. 1 is a diagram for explaining an annotation display system 1 in the present embodiment. The annotation display system 1 illustrated in FIG. 1 is a system developed for a social service that uses augmented reality. The annotation display system 1 illustrated in FIG. 1 operates a social service that uses augmented reality. A server device 3 and a mobile terminal 2 (for example, a smartphone or a mobile phone) possessed by the user of the annotation display system 1 and accessing the server device 3 are included as devices, and a public wireless network 4b for the mobile terminal 2 and the Internet 4a is included as a network. In FIG. 1, the mobile terminal 2 is described as one unit, but actually, the mobile device 2 is configured to be accessible from many mobile terminals 2.

  The portable terminal 2 possessed by the user has a function capable of connecting to the Internet 4a via the public wireless network 4b, and an augmented reality application for using the annotation display system 1 shown in FIG. 1 is installed.

  When the augmented reality application is activated on the portable terminal 2, the augmented reality application executes a process of displaying a camera image photographed by a camera provided in the portable terminal 2, and at the predetermined time interval of the portable terminal 2. By sending the position information (latitude / longitude) and the shooting azimuth, which is the azimuth angle taken by the camera, to the server device 3, the server device 3 is requested to send annotation information and received from the server device 3. A process of combining the displayed annotation information on the camera image and displaying it is executed.

  In the server device 3, a plurality of grids having different grid sizes for dividing the map of the augmented reality service are set, and a grid corresponding to the movement speed of the mobile terminal 2 is determined for each movement speed category of the mobile terminal 2. It has been.

  When the server apparatus 3 receives the annotation information transmission request from the portable terminal 2, the server apparatus 3 determines a grid to be used according to the moving speed of the portable terminal 2, and adds the annotation information according to the position information and the shooting azimuth angle transmitted from the portable terminal 2. After the annotation area for displaying information is determined, a cell corresponding to the annotation area is specified, annotation information is generated for each specified cell, and the generated annotation information is transmitted to the mobile terminal 2.

  Increase the grid size of the grid used when the mobile terminal 2 has a high moving speed and reduce the grid size of the grid used when the mobile terminal 2 has a low moving speed. If it is changed according to the magnitude of the moving speed, the number of cells corresponding to the annotation area, that is, the number of annotations to be displayed on the portable terminal 2 can be changed according to the classification of the moving speed of the portable terminal 2. it can.

  In addition, for each cell on the grid, several (three in this case) keywords extracted from all posted information whose position information is included in the cell area, and all whose position information is included in the cell area Is stored as aggregated cell information, and part or all of the cell information is used as annotation information, thereby reducing the amount of annotation information itself.

  From here, the portable terminal 2 which comprises the annotation display system 1 which illustrated FIG. 1 is demonstrated in detail. FIG. 2 is a diagram illustrating hardware provided in the mobile terminal 2, and FIG. 3 is a diagram illustrating functions provided in the mobile terminal 2.

  As shown in FIG. 2, the portable terminal 2 is responsible for a wireless communication antenna 2 i used for wireless communication, a wireless communication circuit 2 a that handles high-frequency processing related to wireless communication, and various processes related to the portable terminal 2. The baseband circuit 2b, the touch panel 2g having the functions of a display device and an operation panel, the touch panel control circuit 2f for controlling the touch panel 2g, and the strength of the radio wave received from the relay point of the public wireless network 4b A GPS circuit 2c (GPS: Global Positioning System) that measures position information (latitude / longitude), an electronic compass circuit 2d that has front, rear, left, and right magnetic sensors and measures an azimuth angle at which the reference position of the mobile terminal 2 faces In addition to an optical system such as a lens, a camera circuit 2e having an image element and a clock circuit 2h for measuring time are provided.

  The baseband circuit 2b mounted on the portable terminal 2 is a circuit on which an arithmetic processor, a ROM, a RAM, and the like are mounted, and an augmented reality application is mounted as one of the applications that operate on the baseband circuit 2b. .

  Each means illustrated in FIG. 3 indicates a function provided in the mobile terminal 2 by the augmented reality application installed in the mobile terminal 2. When the augmented reality application is activated on the mobile terminal 2, , A position measuring means 20 for measuring position information of a point where the mobile terminal 2 exists, an orientation measuring means 21 for measuring an azimuth angle at which the reference position of the mobile terminal 2 is directed, and a transmission request for annotation information to the server device 3 Annotation display means 22 for combining and displaying the annotation information received from the server device 3 with the video imaged by the camera circuit, and a posting means 23 used when the user registers the posting information in the server device 3 Provided.

  The position measuring means 20 provided in the portable terminal 2 is means for measuring position information (latitude / longitude) at this point using the GPS circuit 2 c of the portable terminal 2. As a method for measuring the position information of the mobile terminal 2, there are a method for measuring the position information inside the mobile terminal 2 according to a communication time with a relay point at which the mobile terminal 2 can perform wireless communication, and a relay that allows the mobile terminal 2 to perform wireless communication. There are two methods of measuring the position information of the portable terminal 2 by the central device arranged in the public wireless network 4b according to the communication time with the point, and either method may be used.

  The azimuth measuring means 21 for measuring the azimuth angle of the portable terminal 2 is a means for acquiring the azimuth angle at which the reference position of the portable terminal 2 is directed from the electronic compass circuit 2 d of the portable terminal 2.

  The annotation display means 22 provided in the portable terminal 2 activates the camera circuit 2e and executes a process for displaying the video imaged by the camera circuit 2e on the touch panel 2g in real time, and then a predetermined time interval elapses. This is a means for making a request to send annotation information to the server device 3 every time and combining the annotation information received from the server device 3 with the video imaged by the camera circuit 2e and displaying it.

  FIG. 4 is a flowchart for explaining processing executed by the annotation display unit 22 of the mobile terminal 2. When the augmented reality application is activated on the portable terminal 2, the annotation display means 22 of the portable terminal 2 is automatically activated (S1).

  When the annotation display means 22 of the portable terminal 2 is activated, the annotation display means 22 of the portable terminal 2 starts to shoot video by operating the camera circuit 2e provided in the portable terminal 2, and the camera circuit 2e takes a picture. The processing for displaying the camera image on the touch panel 2g of the portable terminal 2 in real time is started (S2).

  When the process of displaying the camera image captured by the camera circuit 2e on the touch panel 2g of the portable terminal 2 in real time is executed, the annotation display means 22 of the portable terminal 2 is moved from the position measuring means 20 and the azimuth measuring means 21 at this time. After obtaining the position information and azimuth angle of the portable terminal 2 and obtaining the shooting azimuth angle using the difference between the reference position of the portable terminal 2 and the shooting direction of the camera circuit 2e, the position information and shooting direction of the portable terminal 2 are obtained. By sending a message including a corner to the server device 3, a request for sending annotation information is made to the server device 3 (S3).

  When the annotation display means 22 of the portable terminal 2 receives the annotation information from the server apparatus 3 after requesting the transmission of the annotation information, the annotation display means 22 executes a process of combining each annotation information received from the server apparatus 3 with the camera video and displaying it. (S4).

  Note that the annotation information transmitted from the server device 3 includes position information on the screen on which the annotation information is displayed in addition to the annotation information in the text format, and the annotation display means 22 of the portable terminal 2 displays the annotation information. Each annotation information is displayed on the touch panel 2g in a predetermined format (for example, a balloon) according to the position information on the screen to be displayed.

  When the annotation display means 22 of the portable terminal 2 displays the annotation information transmitted from the server device 3, the annotation display means 22 measures a predetermined time interval using the clock circuit 2h, and each time the predetermined time interval elapses, An information transmission request is made to the server device 3 (S5), and the process of displaying the annotation information received from the server device 3 is repeatedly executed until the augmented reality application is terminated on the portable terminal 2 (S6).

  The posting means 23 provided in the portable terminal 2 is means for transmitting information input to or stored in the portable terminal 2 to the server device 3 as posted information and requesting the server device 3 to register the posted information. .

  FIG. 5 is a flowchart for explaining processing executed by the posting means 23 of the mobile terminal 2. When the occurrence of an event for starting the process of sending the posted information to the server device 3 is detected, the annotation display means 22 activates the posting means 23, and the posting means 23 starts the process of sending the posted information to the server apparatus 3 ( S10).

  FIG. 6 is a diagram showing an example of an icon that activates the posting means 23 of the mobile terminal 2. In FIG. 6, the annotation information 50 transmitted from the server device 3 is displayed on the touch panel 2g of the portable terminal 2 by being synthesized with the camera video 5 photographed by the camera circuit 2e, and the posting information is displayed on the server device 3 as well. An icon 51 for inputting text is displayed as an icon for starting the process of transmitting data to.

  The event for starting the process of transmitting the posted information to the server device 3 is executed by the posting means 23 when the icon 51 illustrated in FIG. 6 is pressed. When the icon 51 illustrated in FIG. 6 is pressed, the annotation display unit 22 activates the posting unit 23, and the posting unit 23 starts a process of transmitting post information to the server device 3.

  When the posting means 23 of the portable terminal 2 is activated, it executes a process of acquiring text-format posting information (S11), then acquires the position information at this time from the position measuring means 20 (S12), and posts information together with the position information. Is sent to the server apparatus 3 and a posting information registration request is made to the server apparatus 3 (S13), and the flow of FIG. 5 ends.

  In FIG. 6, when the icon 51 shown in FIG. 6 is pressed, the posting means 23 of the mobile terminal 2 displays a software keyboard on the touch panel 2g, and text input by the user using the software keyboard is used as posting information. Acquired and transmitted to the server apparatus 3 together with the position information acquired from the position measuring means 20.

  Next, the server device 3 constituting the annotation display system 1 illustrated in FIG. 1 will be described. FIG. 7 is a diagram illustrating hardware provided in the server device 3, and FIG. 8 is a diagram illustrating functions provided in the server device 3.

  The server apparatus 3 is realized by a general-purpose web server. As shown in FIG. 7, the server apparatus 3 includes a CPU 3a (CPU: Central Processing Unit) and a ROM 3b (ROM: Read) in which a BIOS is mounted as hardware. -Only Memory), RAM 3c (RAM: Random Access Memory) which is the main memory of the computer, a large-capacity data storage device 3d (for example, a hard disk) as an external storage device, and input / output for data communication with the external device An interface 3e, a network interface 3f for network communication, a display 3g, a character input device 3h, a pointing device 3i, a clock circuit 3j for measuring time, and the like are provided. The data storage device 3d includes a server device 3 A computer program for operating the computer, and a plurality of Database (DB: DataBase) is provided.

  Each means illustrated in FIG. 8 indicates functions provided in the server device 3 by a computer program installed in the server device 3. When the server device 3 receives a request for transmission of annotation information from the portable terminal 2, An annotation transmitting means 30 for transmitting annotation information corresponding to an information transmission request to the portable terminal 2 and a posting information registering means 31 for registering the posting information received from the portable terminal 2 in the server device 3 are further provided. A posting information DB 32 for storing the registered posting information and a cell DB 33 are provided for each grid set on the social service map.

  First, each DB provided in the server device 3 will be described. FIG. 9 is a diagram for explaining the posting information DB 32 provided in the server device 3. FIG. 9A is an example of a storage format of the posting information DB 32, and FIG. 9B is stored in the posting information DB 32. It is a content example.

  The posted information DB 32 is a DB in which all posted information received from the mobile terminal 2 is stored. As illustrated in FIG. 9A, the posted information DB 32 includes a posted number for identifying posted information, When the posting information is received, the posting date and time, the location information of the posting point where the posting information is transmitted, the cell number corresponding to the point where the posting information is posted, and the keyword extracted from the posting information are given, and the mobile terminal 2 The post information received from is stored.

  The posting number is information given to the posting information on the server device 3 side in order for the server device 3 to manage the posting information. In FIG. 9B, the posting number is described as a serial number. It may be a symbol.

  The posting date / time is the date / time acquired from the clock circuit 3j of the server device 3 when the server device 3 receives the posting information from the portable terminal 2. As shown in FIG. Includes the date when the posting information was received from the portable terminal 2 and the time when the server device 3 received the posting information from the portable terminal 2.

  The position information is the position information transmitted from the portable terminal 2 together with the posted information when the server device 3 receives the posted information from the portable terminal 2. As illustrated in FIG. The latitude and longitude of the posting point that sent the information is included.

  The cell number is specified from the reference point in the service area of the social service and the latitude / longitude of the position information received from the mobile terminal 2 and the grid size of the grid set in the server device 3, and the position received from the mobile terminal 2. This is the cell number of the cell containing the information. In FIG. 9B, the cell number is described as a two-dimensional coordinate value, but it may be a numerical value of a serial number for each cell.

  In addition, the cell number given to contribution information is not one, but the cell number corresponding to the point where contribution information was contributed is provided to contribution information by the number of grids set to the server apparatus 3. In FIG. 9A, the grid size is 10 m × 10 m, the grid ID for identifying the grid is “Grid 1”, the grid size is 20 m × 20 m, and the grid ID for identifying the grid is “Grid 2”. , The grid size is 30 m × 30 m, the grid ID for identifying the grid is “grid 3”, and the grid size is 50 m × 50 m, and the grid ID for identifying the grid is “grid 4”. The cell number in each grid is assigned to the posting information.

  The keyword is extracted from the posted information received from the portable terminal 2 by the posted information registration means 31 of the server device 3 performing natural language analysis on the posted information received from the portable terminal 2, and is important as an index of TF / IDF. In FIG. 9B, there are three keywords whose degrees are equal to or higher than the threshold, but the number of keywords extracted from the posted information may be arbitrary.

  The posted information is the posted information itself received from the mobile terminal 2 by the server device 3, and the posted information is in a text format as illustrated in FIG. 9B.

  Next, the cell DB 33 provided in the server device 3 and provided for each grid set in the server device 3 will be described. 10A and 10B are diagrams for explaining the cell DB 33 provided in the server device 3. FIG. 10A is an example of the storage format of the cell DB 33, and FIG. 10B is an example of contents stored in the cell DB 33. is there.

  The cell DB 33 has a two-dimensional data table structure. For each grid set in the server device 3, the cell DB 33 corresponding to the grid is provided in the server device 3, and as shown in FIG. In one record, cell information (here, cell number and position information for identifying the cell is extracted from the posted information included in the cell for each cell divided by the grid set in the server device 3). , Keyword), and the posting number of the posting information whose position information is included in the cell is stored.

  Since the cell number has been described above, a detailed description is omitted here. As shown in FIG. 10B, the cell information is a keyword extracted from the keyword of the posting information in which the posting number is stored in the record specified by the cell number, and the posting information is the posting information DB 32. Is registered, the cell information of the cell record corresponding to the posting point is updated using the keyword extracted from the registered posting information. In FIG. 10B, the number of keywords included in the cell information is three, but the number of keywords included in the cell information may be arbitrary.

  The post number stored in the record specified by the cell number is the post number of all post information whose position information is included in the cell area specified by the cell number, and the post information is registered in the post information DB 32. Then, the posting number of the registered posting information is added to the record of the cell corresponding to the posting point of the registered posting information.

  First, the posting information registration means 31 provided in the server device 3 and for registering the posting information received from the mobile terminal 2 in the server device 3 will be described. FIG. 11 is a flowchart for explaining processing executed by the posted information registration unit 31 of the server device 3.

  Upon receiving a posting information registration request from the portable terminal 2, the posting information registration means 31 of the server device 3 is activated, and the posting information registration means 31 starts processing for registering the posting information that has received the registration request in the server device 3. (S20).

  When the process of registering the posting information that has received the registration request to the server device 3 is started, first, the posting information registration means 31 of the server device 3 generates a posting number of the posting information received from the mobile terminal 2 (S21). For example, a counter used for generating a posting number is provided, and the posting number of the posting information received from the mobile terminal 2 is generated by incrementing the counter value by one.

  Next, the posting information registration means 31 of the server device 3 acquires the date / time information at this time as the posting date / time from the clock circuit provided in the server device 3 (S22).

  Then, the loop processing L1 executed for each grid set in the server device 3 is executed, and the cell number corresponding to the posting point where the posting information is posted is acquired for each grid set in the server device 3 (S23). ). In this loop processing 1, the posting information registration means 31 of the server device 3 includes the reference point in the service area of the social service and the latitude / longitude of the position information received from the mobile terminal 2, and the grid that is the target of the loop processing L1. From the grid size, the cell number corresponding to the posting point where the posting information is posted in the grid that is the target of the loop processing 1 is obtained.

   Next, the posting information registration unit 31 of the server device 3 executes a process of extracting a keyword from the posting information received from the mobile terminal 2 according to a predetermined rule (S24).

  The posting information registration means 31 of the server device 3 calculates the importance such as an index of TF / IDF for each word extracted by performing morphological analysis on the posting information transmitted from the mobile terminal 2, and the importance is equal to or greater than a threshold value. Is extracted as a keyword of posting information.

  When keywords are extracted from the posted information received from the portable terminal 2, the posted information registration means 31 of the server device 3 outputs the posting number generated in S21, the posting date and time acquired in S22, the positional information received from the portable terminal 2, and loop processing. The plurality of cell numbers obtained in L1 and the keyword extracted in S24 are assigned, and the posting information received from the portable terminal 2 is stored in the posting information DB 32 (S25).

  When the posting information received from the portable terminal 2 is stored in the posting information DB 32, the posting information registration unit 31 of the server device 3 updates the contents of the cell DB 33 provided for each grid set in the server device 3. The loop process L2 executed for each grid set in the server apparatus 3 is executed, and the procedure of FIG.

  In the loop process L2, the posting information registration unit 31 of the server device 3 identifies the record corresponding to the cell number identified in the loop process L1 from the cell DB 33 corresponding to the grid that is the target of the loop process L2, All posting information corresponding to the posting number stored in the record is searched from the posting information DB 32, and keywords assigned to all the searched posting information are acquired (S26).

  Then, the posting information registration unit 31 of the server device 3 obtains the appearance frequency of the keywords for all the acquired keywords, and selects the top three keywords in the descending order of the appearance frequency with the cell numbers obtained in the loop process L1. By storing it as the cell information of the specified record (S27), the contents of the cell DB 33 of the grid that is the target of the loop process 2 are updated.

  Next, the annotation transmission means 30 provided in the server device 3 will be described. FIG. 12 is a flowchart for explaining processing executed by the annotation transmission unit 30 of the server device 3.

  When the annotation transmission unit 30 provided in the server device 3 receives the transmission request for the annotation information received from the mobile terminal 2, the annotation transmission unit 30 starts processing to transmit the annotation information to the mobile terminal 2 (S30).

  When the annotation transmission means 30 provided in the server device 3 starts processing to transmit annotation information to the mobile terminal 2, the annotation transmission means 30 calculates the moving speed of the mobile terminal 2 that has requested transmission of annotation information (S31).

  The annotation transmission means 30 of the server device 3 calculates the moving speed of the mobile terminal 2 that has requested the transmission of annotation information, and the unique number (for example, the telephone number of the mobile phone) of the mobile terminal 2 that has requested the transmission of annotation information. ) And the position information received from the portable terminal 2 is stored, and the moving speed of the portable terminal 2 is determined using the difference between the current time and the previous time and the position information. calculate.

  When the annotation transmission means 30 of the server device 3 calculates the movement speed of the mobile terminal 2, the annotation transmission means 30 executes a process of determining a classification of the movement speed to which the calculated movement speed belongs (S32).

  In order to determine the classification of the movement speed to which the calculated movement speed belongs, the annotation transmission means 30 of the server device 3 includes, for each movement speed classification, a range of movement speed, a grid ID used in the movement speed classification, a grid Rules that define information to be used as size and annotation information are set.

  FIG. 13 is a diagram for explaining an example of a rule that defines the contents of the moving speed classification. In FIG. 13, the mobile terminal 2 is classified into five moving speed categories. For each moving speed category, a classification ID for identifying the moving speed classification, a moving speed range, and a grid used for the moving speed classification are shown. Information used as grid ID, grid size, and annotation information is defined.

  The range of the moving speed of the section with the section ID “01” is “over 12 km / h”, and the section with the section ID “01” is a section where the user is on the vehicle. In the category with the category ID “01”, the grid with the grid ID “grid 4” is used, the grid size of this grid is 50 m × 50 m, and all cell information (here, three keywords) is used as annotation information. Use is defined.

  The range of the moving speed of the section with the section ID “02” is “over 7 km / h and less than or equal to 12 km / h”, and the section with the section ID “02” is the section where the user is running. In the category with the category ID “02”, a grid with the grid ID “grid 3” is used, and the grid size of this grid is 30 m × 30 m, and it is defined that all cell information is used as annotation information.

  The range of the moving speed of the section with the section ID “03” is “over 4 km / h and less than 7 km / h”, and the section with the section ID “03” is a section in which the user is walking fast. In the section with the section ID “03”, a grid with the grid ID “grid 2” is used, and the grid size of this grid is 20 m × 20 m, and it is defined that all cell information is used as annotation information.

  The range of the moving speed of the section with the section ID “04” is “over 0 km / h and less than 4 km / h”, and the section with the section ID “04” is a section in which the user is walking. In the section with the section ID “04”, a grid with the grid ID “grid 1” is used, and the grid size of this grid is 10 m × 10 m, and it is defined that all cell information is used as annotation information.

  The range of the moving speed of the section with the section ID “05” is “0 km / h”, and the section with the section ID “05” is a section where the user is stationary. In the category with the category ID “05”, the grid with the grid ID “grid 1” is used, and the grid size of this grid is 10 m × 10 m, and it is defined that the posting information itself is used as annotation information.

  According to the rule illustrated in FIG. 13, for example, when the calculated moving speed of the mobile terminal 2 corresponds to “exceeds 12 km / h”, the annotation transmission unit 30 of the server device 3 determines that the classification ID is “01”. Select. In addition, when the calculated moving speed of the mobile terminal 2 corresponds to “over 0 km / h and 4 km / h or less”, the annotation transmission unit 30 of the server device 3 selects the category with the category ID “04”.

  In FIG. 13, it is defined that all cell information is used as annotation information in the classifications with classification IDs “01” to “04”. It can also be defined to use part of the cell information as information.

  For example, in FIG. 13, when the mobile terminal 2 has the highest moving speed and the classification ID is “01”, a part of cell information (one keyword having the highest importance) is used as annotation information. It is possible to define that a part of the cell information (two keywords with the highest importance) is used as the annotation information in the section with the second largest moving speed and the section ID “02”.

  When the annotation transmission means 30 of the server device 3 executes the process of determining the moving speed classification to which the moving speed of the portable terminal 2 belongs, when the annotation information transmission request is received from the portable terminal 2, it is received from the portable terminal 2. After determining the annotation area in which the annotation information is displayed using the position information and the shooting azimuth angle (S33), the cell number of the cell corresponding to the annotation area is specified when the grid of the moving speed is used (S33). S34).

  FIG. 14 is a diagram for explaining the annotation area in the present embodiment. In FIG. 14, the service area 6 of the social service operated by the server device 3 is schematically square, the service area 6 is divided into a matrix by the grid 60a, and the point that received the transmission request for the annotation information is At the point A, the posting point of the posting information is illustrated by a point X.

  In the present embodiment, the shape of the annotation area 61 is a fan shape having a radius R (for example, 300 m) centered on the point A. The direction of the fan-shaped center line is the direction of the shooting azimuth angle θ2 received from the mobile terminal 2, and the fan-shaped angle θ1 may use a predetermined angle stored in the server device 3 in advance. When the viewing angle of the camera of the portable terminal 2 is stored in the server device 3 in association with the model number of the portable terminal 2, the viewing angle specified from the model number obtained from the portable terminal 2 is defined as a fan-shaped angle θ1. You can also

  In FIG. 14, cells (shaded cells) that include all or part of the annotation area 61 are identified as cells corresponding to the annotation area 61.

  Note that the annotation area 60 illustrated in FIG. 14 is merely an example, and the shape and size of the annotation area 61 can be arbitrarily set in the server device 3.

  FIG. 15 is a diagram for explaining that the number of cells included in the annotation area is changed as the grid size is changed. The service area 6 of the social service operated by the server device 3 is divided into a matrix by a grid 60b having a grid size larger than the grid 60a of FIG.

  As can be seen from FIG. 15, the number of cells corresponding to the annotation area 61 illustrated in FIG. 14 is 14, but in FIG. 15 in which the grid size is larger than that of FIG. 14, the cell corresponds to the annotation area 61. The number of cells has been reduced to four.

  As will be described later, since the number of annotation information to be transmitted to the mobile terminal 2 is the number of cells corresponding to the annotation area 61, the mobile terminal 2 increases the grid size as the moving speed of the mobile terminal 2 increases. As the moving speed of 2 increases, the number of annotation information displayed on the mobile terminal 2 can be reduced.

  FIG. 16 is a first diagram illustrating annotation information displayed on the mobile terminal 2, and FIG. 17 is a second diagram illustrating annotation information displayed on the mobile terminal 2.

  In FIG. 14, the number of cells including the posting point of the posting information is four, so that the number of annotation information 7a displayed on the mobile phone 2 is four as illustrated in FIG. In FIG. 15, the number of cells including the posting point of the posting information is three. Therefore, as illustrated in FIG. 17, the number of annotation information 7 b displayed on the mobile phone 2 is three. Also, since the upper left cell shaded in FIG. 15 includes two pieces of post information, the annotation information corresponding to this cell is the keyword extracted from the two pieces of post information (here, pan, unpan, mobile phone). Phone).

  When the annotation transmission means 30 of the server device 3 specifies the cell number of the cell corresponding to the annotation area, the annotation transmission means 30 uses the grid cell DB 33 corresponding to the classification of the moving speed determined in S31 of FIG. 12, and in S31 of FIG. According to the determined moving speed classification, a process of acquiring information used for annotation information for each cell number identified in S34 of FIG. 12 is executed (S35).

  The information used for the annotation information differs depending on the contents set in the movement speed category determined in S31 of FIG. For example, according to the rule described with reference to FIG. 13, when any of the classification IDs “01” to “04” is selected, it is stored in the record corresponding to the cell number of the cell corresponding to the annotation area. Cell information is acquired as information used for annotation information.

  Further, for example, according to the rule described with reference to FIG. 13, when the category having the category ID “05” is selected, the post number stored in the record corresponding to the cell number of the cell corresponding to the annotation area is set. Corresponding posting information is searched from the posting information DB 32, and the searched posting information is acquired as information used as annotation information.

  When the annotation transmission unit 30 of the server device 3 acquires information to be used as annotation information, the annotation transmission unit 30 determines position information on the touch panel 2g for displaying the annotation information for each acquired information, and displays the annotation information on the touch panel 2g. Annotation information to which position information is added is generated, and the generated annotation information is transmitted to the mobile terminal 2 (S36).

  The position information on the touch panel 2g for displaying the annotation information is determined based on the relative position of each cell corresponding to the annotation area with reference to the cell corresponding to the point where the posted information is posted when the annotation information is acquired. It is good to keep. For example, with respect to the direction of the azimuth angle received from the mobile terminal 2, a cell on the right side (left side) of the cell corresponding to the point where the posted information is posted is displayed on the right side (left side) of the touch panel 2g of the mobile terminal 2. In addition, the distance from the cell corresponding to the point where the posted information is posted may be displayed on the upper side of the touch panel 2g of the portable terminal 2.

DESCRIPTION OF SYMBOLS 1 Annotation display system 2 Portable terminal 20 Position measurement means 21 Direction measurement means 22 Annotation display means 23 Posting means 3 Server apparatus 30 Annotation transmission means 31 Posting information registration means 32 Posting information DB
33 Cell DB
4a Public wireless network 4b Internet

Claims (17)

An annotation display system comprising at least a server device that receives an annotation information transmission request and a mobile terminal that makes an annotation information transmission request, and displays annotation information based on a posting point on an augmented reality service map,
The portable terminal has an annotation by transmitting to the server device at least the position information obtained from the position measuring means for measuring the position information of the portable terminal, the camera measuring means for capturing the image, and the position measuring means at this time. An annotation display means for performing an information transmission request to the server apparatus, and combining the annotation information received from the server apparatus with a video image captured by the camera means;
In the server device, a plurality of grids each having a different grid size for setting a map of the augmented reality service is set, and the cell number is specified in association with the cell number assigned to each cell surrounded by the grid. A cell database is provided for each grid, which stores, as cell information, information in which all pieces of posted information including the position information of the posting point are included in a cell area, and further, a transmission request for the annotation information is sent to the mobile phone. When received from the terminal, at least using the position information of the mobile terminal when receiving the transmission request, after obtaining the annotation area for displaying the annotation information, using the grid corresponding to the moving speed of the mobile terminal The cell corresponding to the annotation area is identified, and the cell number of the identified cell The cell information associated with the grid information is acquired from the cell database of the grid, and the annotation information is generated by the number of the cells for which the annotation information is specified using a part or all of the acquired cell information. An annotation transmission means for transmitting the message to the portable terminal,
An annotation display system characterized by that.   The server device stores a predetermined number of the keywords as the cell information in descending order of importance from keywords obtained by natural language analysis of the posting information including a posting point within the cell area. The annotation display system according to claim 1, wherein:   The annotation display system according to claim 2, wherein the annotation transmission unit of the server device changes the number of keywords used as the annotation information according to a moving speed of the mobile terminal.   The server device includes a posting information database that stores the posting information received from the portable terminal in association with the position information of the posting point, and the annotation transmission unit is configured such that the moving speed of the portable terminal is stationary. , Acquiring the posting information including position information of a posting point in the cell area corresponding to the annotation area from the posting information database, and generating the annotation information using the acquired posting information, The annotation display system according to any one of claims 1 to 3.   The portable terminal includes azimuth angle measuring means for measuring an azimuth angle, and the annotation display means uses the azimuth angle obtained from the azimuth angle measuring means when requesting the server device to transmit the annotation information. Means for obtaining a shooting azimuth angle facing the means, and means for transmitting the shooting azimuth angle together with the position information of the mobile phone to the server device, wherein the annotation transmission means of the server device sends a transmission request for the annotation information The annotation according to any one of claims 1 to 4, wherein when received from a portable terminal, an annotation area is determined using position information and a shooting azimuth angle received from the portable terminal. Display system. The mobile terminal includes post information transmission means for transmitting the post information set by a user to the server device, and requesting the server device to register the post information,
When the server device receives a registration request for the posting information from the mobile terminal, the server device stores the posting information in the posting information database in association with the position information of the posting point of the posting information that has received the registration request. The cell information of the cell including the location information of the mobile terminal when receiving the registration request is specified for each grid, and the cell information stored in the cell database in association with the specified cell number 6. An annotation display system according to claim 4, further comprising post information registration means for updating the post information using post information from the portable terminal.   A server device that transmits the annotation information to a mobile terminal that has requested transmission of annotation information, and a plurality of grids each having a different grid size for partitioning an augmented reality service map are set, and each cell surrounded by the grid A cell database that stores, as cell information, information obtained by aggregating all posted information including position information of the posting point within the cell area specified by the cell number in association with the cell number assigned to Provided for each grid, and when receiving a transmission request for the annotation information from the portable terminal, an annotation area for displaying the annotation information is obtained using at least the position information of the portable terminal when the transmission request is received After, using the grid corresponding to the moving speed of the mobile terminal The cell corresponding to the annotation area is specified, the cell information associated with the cell number of the specified cell is acquired from the cell database of the grid, and a part or all of the acquired cell information is used. A server apparatus comprising: annotation transmission means for generating the annotation information by the number of the specified cells and transmitting the generated annotation information to the mobile terminal.   The server device stores a predetermined number of the keywords as the cell information in descending order of importance from keywords obtained by natural language analysis of the posting information including a posting point within the cell area. The server device according to claim 7, wherein   9. The server apparatus according to claim 8, wherein the annotation transmission unit of the server apparatus changes the number of keywords used as the annotation information according to a moving speed of the mobile terminal.   A posting information database for storing the posting information received from the portable terminal and the position information of the posting point in association with each other, and the annotation sending means is located in the annotation area when the moving speed of the portable terminal is stationary; 8. The posting information in which position information of a posting point is included in the corresponding cell area is acquired from the posting information database, and the annotation information is generated using the acquired posting information. The server device according to claim 9.   The annotation transmission means determines an annotation area using position information and a shooting azimuth angle received from the portable terminal when receiving a transmission request for the annotation information from the portable terminal. The server device according to any one of claims 7 to 10.   When receiving the posting information registration request from the mobile terminal, the posting information is stored in the posting information database, and the location information of the mobile terminal at the time of receiving the registration request is included in the cell. A post information registration unit is provided that specifies a number for each grid and updates the cell information associated with the specified cell number and stored in the cell database using post information from the mobile terminal. The server device according to claim 10 or 11, characterized in that: An annotation display method for displaying annotation information based on a posting point on an augmented reality service map on a mobile terminal,
A plurality of grids having different grid sizes for setting the map of the augmented reality service are set, and the cell number assigned to each cell surrounded by the grid is associated with the cell number specified by the cell number. A server device that has a cell database storing cell-by-cell information in which all post-information including post-location location information is stored as cell information receives a request to send annotation information from the mobile terminal And a step of obtaining an annotation area for displaying the annotation information using at least the position information of the portable terminal when receiving the transmission request,
The server device identifies the cell corresponding to the annotation area using the grid corresponding to the moving speed of the mobile terminal, and displays the cell information associated with the cell number of the identified cell. Step b, which is obtained from the cell database, generates the number of the cells specifying the annotation information using a part or all of the obtained cell information, and transmits the generated annotation information to the mobile terminal.
An annotation display method characterized in that is executed.   The server device stores a predetermined number of the keywords as the cell information in descending order of importance from keywords obtained by natural language analysis of the posting information including a posting point within the cell area. The annotation display method according to claim 13, wherein:   The annotation display method according to claim 14, wherein in the step b, the server device changes the number of keywords used as the annotation information according to a moving speed of the mobile terminal.   In the step a, when the server device receives a transmission request for the annotation information from the mobile terminal, the server device determines an annotation area using the position information and the shooting azimuth angle received from the mobile terminal. The annotation display method according to any one of claims 13 to 15. In the step b, when the moving speed of the mobile terminal is stationary, the server device stores the post information received from the mobile terminal and the position information of the post location in association with each other. The posting information database is referred to, the posting information in which the location information of the posting point is included in the cell area corresponding to the annotation area is acquired from the posting information database, and the annotation information is obtained using the acquired posting information. The annotation display method according to any one of claims 13 to 16, wherein the annotation display method is generated.

Images