JP5651372B2 - Post information control apparatus, post information control system, and post information control method - Google Patents

Description

  The present invention relates to a posted information control apparatus, a posted information control system, and a posted information control method.

  2. Description of the Related Art An augmented reality (AR) system that adds information to a real environment using a computer is known (see, for example, Patent Document 1). The apparatus described in Patent Document 1 is to present geographical information on a computer and each part in a landscape image of an actual scene in association with each other.

  In recent years, AR systems for mobile phones have attracted attention as a new form of information service platform. In a typical example of an AR system for a mobile phone, a mobile phone with a camera function transmits position information to a server, and the server transmits POI (Point of Interest) information around the current location of the mobile phone to the mobile phone. A telephone is displayed by superimposing an icon (graphics object) of POI information at a position corresponding to the POI of a live image acquired by a camera.

Japanese Patent Laid-Open No. 10-267671

  In order for the AR system to develop as a new information service platform, it is desired to create an environment where users can transmit information. For example, it is preferable that the user can add a comment to POI information such as a store, or the store side can add information related to a sale to POI information such as a store.

  In view of the above circumstances, an object of the present invention is to provide a posted information control apparatus, a posted information control system, and a posted information control method that can display information posted by a user on a terminal.

In addition, the posted information control system according to the present invention includes a terminal that superimposes and displays an object on a live video or a live image obtained by photographing a real space with a camera unit , and object information related to the object configured to be able to communicate with the terminal. A posting information control system that distributes to the terminal, wherein the terminal of the posting source is additional information to be added to the object, and is a text, an image, a moving image, a sound, or a combination thereof Terminal-side transmission means for transmitting the additional information consisting of: to the posted information control device, wherein the posted information control device records the additional information transmitted by the posting source terminal in association with the object. When the object information is extracted based on the location information of the recording means and the terminal of the distribution destination Moni, an information extracting means for extracting said additional information associated with the object corresponding to the object information, to the delivery destination of the terminal, said information extracting means and the object information and the information extracted by the extraction Information distribution means for distributing an entry indicating the additional information extracted by the means, and the terminal as a distribution destination includes the additional information on the object corresponding to the object information distributed by the posted information control device. The image display control means for graphically displaying the relationship between the object and the entry indicating the additional information , and the entry indicating the additional information displayed by the image display control means are selected. If additional information is acquired, the additional information is acquired based on the entry. Further we comprising an acquiring unit, a configured as characterized.

In the posted information control system according to the present invention, the additional information added to the object by the terminal side transmission means of the posting source terminal, the additional information consisting of text, an image, a moving image, audio, or a combination thereof is posted information control. The additional information transmitted by the posting information control device is recorded in association with the object by the additional information recording unit of the posting information control device, and is recorded by the information extracting unit based on the location information of the distribution destination terminal. The object information is extracted, and additional information associated with the object corresponding to the object information is extracted, and the information distribution unit extracts the object information and information extracted by the information extraction unit from the distribution destination terminal. It delivered an entry indicating the additional information extracted by the means, the distribution destination terminal of the image The示制control means, together with the entry indicating additional information in the object corresponding to the object information distributed is displayed is added by post information control device, association with the entry indicating the object and additional information is displayed graphically, added by the information acquisition means, when the entry indicating the additional information displayed by the image display control means is selected Ru acquired additional information based on the entry. In this way, since the posting information control device can record the additional information posted from the posting source terminal in association with the object, the posting information control device distributes the object information to the distribution destination terminal. Additional information can be transmitted at this time. Therefore, the delivery destination terminal can display the additional information together with the object. Therefore, the information posted by the user can be displayed on the terminal. Furthermore, since the relationship between the object and the additional information can be displayed graphically, the user can intuitively understand which object is the posted information. Further, since only the information of the selected additional information can be acquired from the additional information, it is possible to reduce the communication load and increase the resource efficiency.

  Here, the image display control means may graphically display a relationship between the object and the additional information by displaying an icon of the additional information in a shape ballooned out from the object. Alternatively, the image display control means may graphically display the association between the object and the additional information by connecting the object and the icon of the additional information with a line.

  Further, the image display control means displays a plurality of the additional information as an aggregation icon when there are a plurality of the additional information associated with the object, and aggregates when the aggregation icon is selected. A list screen on which the additional information is posted may be displayed. With this configuration, posted information can be displayed so that the screen is not complicated.

Further, the posted information control method according to the present invention includes a terminal that superimposes and displays an object on a live image or a live image obtained by photographing a real space with a camera unit , and object information related to the object configured to be communicable with the terminal. A posting information control method for a posting information control system comprising: a posting information control device that distributes to the terminal, wherein the posting source terminal adds additional information to the object, and is a text, image, video or audio Alternatively, the additional information transmission step of transmitting the additional information composed of a combination thereof to the posting information control device, and the posting information control device records the additional information transmitted by the terminal that is the posting source in association with the object. The additional information recording step and the posted information control device include the location information of the terminal of the distribution destination. And extracting the object information and extracting the additional information associated with the object corresponding to the object information, and the posting information control device An information distribution step of distributing the object information extracted by the information extraction step and an entry indicating the additional information extracted by the information extraction step; and the terminal of the distribution destination is distributed by the post information control device an image display control step of displaying the relationship between the entry indicating the additional information and the object in the graphical and displays added an entry indicating the additional information to the object corresponding to the object information, the terminal of the destination is Displayed by the image display control step. And when the entry indicating the additional information is selected constructed and an additional information acquisition step of acquiring the additional information based on the entry.

  According to the posted information control method of the present invention, the same effects as those of the posted information control system described above can be obtained.

  According to the present invention, information posted by a user can be displayed on a terminal.

It is the schematic which shows the structure of the contribution information control system which concerns on embodiment. It is a functional block diagram of the terminal device in FIG. It is a functional block diagram of the AR server in FIG. It is an example of the data definition of POI information. It is a flowchart which shows operation | movement of the contribution information control system which concerns on embodiment. It is a flowchart which shows operation | movement of the contribution information control system which concerns on embodiment. It is an example of the screen figure which displays a comment. It is an example of the screen figure which displays a plurality of comments.

  Hereinafter, a posting information control apparatus, a posting information control system, and a posting information control method according to embodiments of the present invention will be described with reference to the accompanying drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant description is omitted.

  The posted information control device, the posted information control system, and the posted information control method according to the present embodiment are suitably employed for augmented reality services for mobile phones, for example.

  FIG. 1 is a diagram schematically showing the configuration of an augmented reality system (AR system, posted information control system) 1 according to an embodiment of the present invention. As shown in FIG. 1, the AR system 1 includes a terminal device 100 and an augmented reality server (AR server, posting information control device) 300 that provides augmented reality information to the terminal device 100.

  The terminal device 100 is a mobile terminal such as a mobile phone that communicates by connecting to the communication network 150, and includes a camera unit and a display unit having an image acquisition function as described later. The AR server 300 is a server computer system connected to the network 200 connected to the communication network 150, and can communicate with the content server 371 and the terminal device 100, which are external servers connected to the network 200. ing. The AR server 300 may be configured to be able to communicate with a map information server (not shown) and a route information server (not shown) that are external servers connected to the network 200. The AR system 1 having such a configuration provides a service that gives the user augmented reality by superimposing information provided by the AR server 300 on an image acquired by the user of the terminal device 100 with a camera directed to a real space. It is to provide. Hereinafter, the terminal device 100 and the AR server 300 will be described in detail.

  First, the terminal device 100 will be described. FIG. 2 is a functional block diagram of the terminal device 100 in the present embodiment. The terminal device 100 mainly includes a position information acquisition unit 101 that acquires terminal position information, a display unit 102 that displays a screen, an input unit 103 that receives input by a user's key operation, and a direction in which the terminal is facing. Azimuth sensor 104 for detecting the image, communication unit (terminal-side transmission means) 105 connected to communication network 150 for communication, camera unit 106 for acquiring real-space video, and storage for storing predetermined programs and data And an AR client processing unit (image display control means) 108 that controls the operation of each unit and implements predetermined information processing.

  The position information acquisition unit 101 is configured to use, for example, a GPS module that receives radio waves from a plurality of GPS (Global Positioning System) satellites and acquires current terminal position information from their intensities. In addition to the GPS system, an AGPS (Assisted Global Positioning System) system that acquires information on satellites that exist near the terminal as assist data by communicating with the communication network 150 via the communication unit 105 and performs positioning. The terminal location information may be acquired more quickly and accurately using a module corresponding to the above. However, the position information acquisition system is not limited to the one using GPS or AGPS, and can be configured to acquire terminal position information without depending on GPS. For example, a method may be used in which a plurality of wireless communication base stations such as WiFi and Bluetooth (registered trademark) are arranged in the space, and terminal location information is acquired based on the radio field intensity therefrom. It is also possible to use a method used in combination with GPS or AGPS.

  The display unit 102 includes an LCD module and an LCD controller for performing screen display, and a sensor module and a position detection unit that detect a contact position of the user in response to an operation of touching the screen by the user. It realizes the touch panel function. In other words, the display unit 102 receives data and control signals from the AR client processing unit 108 by the LCD controller and displays a predetermined screen on the LCD module, and receives a user operation for touching the screen by the sensor module to detect the position. The contact position is detected by the unit, and a position signal related to the contact position is notified to the AR client processing unit 108 as an input. The terminal device 100 provides the user with augmented reality by adding and displaying the information acquired from the AR server 300 to the real space video acquired by the camera unit 106 according to the operation described below, and also provides a display function and a touch panel. Provides a user interface by function.

  The input unit 103 is an input device such as a keypad or other buttons provided in the casing of the terminal device 100, and a key event that is a signal corresponding to each key or button when the user operates is processed by the AR client process. This is notified to the unit 108 and used for various operations or control. For example, an icon (object) of augmented reality information (object information) being displayed by the display unit 102 can be selected and a comment can be input, and the input comment is associated with the augmented reality information. And output to the AR client processing unit 108. Note that the input unit 103 may input text, an image, a moving image, audio, or a combination thereof as a comment.

  The azimuth sensor 104 is a module that detects a posture / angle in a direction in which the azimuth sensor is facing and outputs a corresponding signal to the AR client processing unit 108. The AR client processing unit 108 can recognize the posture / angle at which the user holds the terminal device 100 from the detection value of the direction sensor 104. A combination of a plurality of geomagnetic sensors can be used as the orientation sensor 104, and a motion sensor that combines a plurality of acceleration sensors in addition to the geomagnetic sensor may be used. A six-axis sensor that combines a triaxial geomagnetic sensor and a triaxial acceleration sensor, a five-axis sensor that combines a triaxial geomagnetic sensor and a biaxial acceleration sensor, and a gyro instead of an acceleration sensor Can be used for the direction sensor 104 as a motion sensor. When the motion sensor is used in this way, it is possible to detect not only information on the attitude and angle of the terminal device 100 but also the direction and speed of the operation in which the user moves or tilts the terminal device 100. .

  The communication unit 105 is a module for connecting to the communication network 150 and communicating with other terminal devices and server systems. Examples of the communication here include circuit-switched communication performed by communication connection with other terminal devices, packet-switched communication performed with a server system connected to the network 200 via the communication network 150, and the like. . Further, when acquiring the terminal location information by the AGPS method, the communication unit 105 cooperates with the location information acquisition unit 101 to assist the acquisition of the terminal location information.

  The camera unit 106 is a module that has a predetermined optical system and an image receiving element and provides a function of acquiring a digital image. When the terminal device 100 captures a digital image, the activated camera unit 106 normally repeats the process of acquiring data by the image receiving element at a predetermined exposure time from the subject image acquired by the optical system at a predetermined time interval. The live view function is realized by displaying the data on the display unit 102. In this state, when the user further performs a predetermined operation with the input unit 103, the camera unit 106 generates image data with the shooting conditions set from the subject image acquired by the optical system, and the generated image data is Saved in the storage unit 107. The AR system 1 displays information that brings augmented reality to the real space video displayed on the display unit 102 by the above-described live view function.

  The storage unit 107 is a memory for storing predetermined information under the control of the AR client processing unit 108 and providing the stored information to the AR client processing unit 108. The storage unit 107 stores various programs executed by the AR client processing unit 108, and the AR client processing unit 108 reads and executes them as appropriate. The storage unit 107 does not need to have a uniform configuration, and can be configured by appropriately combining various memories such as ROM, RAM, and Flash ROM, and further combined with a secondary storage device such as an HDD. It doesn't matter. The storage unit 107 stores various augmented reality information provided from the AR server 300 described later. The augmented reality information (object information) stored in the storage unit 107 includes POI information. The augmented reality information is returned as a response from the AR server 300 in response to a request from the terminal device 100.

  The AR client processing unit 108 is a functional block that is virtually configured by executing a program stored in the storage unit 107 on a CPU (not shown), and includes a position information acquisition unit 101 and a display unit of the terminal device 100. 102, the input unit 103, the orientation sensor 104, the communication unit 105, the camera unit 106, and the storage unit 107 exchange various data and control signals to realize various functions of the terminal device 100. is there. Further, the AR client processing unit 108 transmits a comment (additional information) or the like input by the user via the input unit 103 to the AR server 300 via the communication unit 105. The transmitted comment is transmitted to the AR server 300 in a state associated with the augmented reality information. Further, the AR client processing unit 108 transmits the request for the terminal location information acquired by the location information acquisition unit 101 to the AR server 300, thereby sending a response of augmented reality information corresponding to the terminal location information from the AR server 300. In response, it provides the user with augmented reality. For example, the AR client processing unit 108 displays a real space image captured by the camera unit 106 on the display unit 102, determines a display position on which the augmented reality information is superimposed on the real space image, and sets the display position to the display position. A process of superimposing and drawing a graphics object corresponding to the augmented reality information is performed. At this time, when a posted comment related to the augmented reality information is received from the AR server 300, a graphics object related to the posted comment is also superimposed and drawn. Hereinafter, such processing will be described in detail.

  First, a process of determining the display position of augmented reality information based on the attitude and angle of the camera unit 106 and superimposing it on an image in the real space to be displayed on the display unit will be described. In this process, first, the posture / direction of the camera is derived from the detection value of the azimuth sensor 104, and the augmented reality stored in the storage unit 107 is based on the horizontal viewing angle and the vertical viewing angle of the camera. It is determined whether the sensation information falls within the field of view of the camera unit 106, and the result is recorded in the storage unit 107. Subsequently, it is assumed that the augmented reality information determined to be within the field of view is arranged in the coordinate space with the current terminal position of the terminal device 100 as the origin, and the horizontal position of the camera with the predetermined height of the origin as the center point. Based on the directional viewing angle, the vertical viewing angle, and the video size, a process of transmitting and projecting each augmented reality information onto a rectangular plane determined at a predetermined distance from the origin is performed, and the display position is determined based on the position of the transmissive projection. Here, it is preferable to set the height of the center point to be transparently projected to a height of about 1.5 m in order to reproduce the situation in which the user uses the terminal device 100 while standing.

  After the above process, a process is performed in which each augmented reality information is drawn at the determined display position so as to be superimposed on the real space image. When the comment information posted by the user is also received, the comment information related to the posted comment is also displayed superimposed on the real space video. Although there is no particular limitation on such processing, each augmented reality information or comment information is drawn at a predetermined display position to create a transparent screen layer, and this screen layer is displayed overlaid on the real space video. The information can be displayed on the unit 102, and the augmented reality information or the comment information is displayed at an appropriate position in the real space image. At this time, augmented reality information and comment information are displayed on the screen as graphics objects such as icons, information balloons, and sub-windows. The terminal device 100 provides an excellent augmented reality by an interface with excellent visual and recognizability with physical operation using a touch panel function.

  Next, the AR server 300 will be described. The AR server 300 is a server system in which, for example, a Web server is resident. The AR server 300 receives a request from the terminal device 100 via the communication network 150 and the network 200, and receives data from the content server 371 connected to the network 200 as necessary. Is acquired, and a process of responding to the terminal device 100 as augmented reality information is performed. More specifically, the AR server 300 receives the location information of the terminal device 100 transmitted by the terminal device 100 via the communication network 150 and the network 200, and collects augmented reality information from the external content server 371. Then, POI information within a predetermined range from the current position of the terminal device 100 is transmitted to the terminal device 100. The AR server 300 receives and records the comment information transmitted from the terminal device 100 via the communication network 150 and the network 200. Then, when the comment information is added to the augmented reality information transmitted to the terminal device 100, the AR server 300 also transmits the comment information to the terminal device 100.

  The configuration of the AR server 300 will be described. FIG. 3 is a functional block diagram of the AR server 300 in the present embodiment. The AR server 300 is physically configured as a computer system including a CPU, a ROM, a RAM, and the like. Each component to be described later is realized by the CPU reading predetermined software on the ROM or RAM and reading / writing data from / to the ROM or RAM.

  As shown in FIG. 3, the AR server 300 includes a communication unit (additional information acquisition unit, information distribution unit) 301, a POI information collection unit 302, a POI information / comment extraction unit (information distribution unit) 303, a comment input unit (additional unit). Information recording means) 304 and a POI data storage unit 316.

  The communication unit 301 is a network module that connects to the network 200 and communicates with the terminal device 100 and an external server such as the content server 371, and a publicly known one can be used.

  The POI information collection unit 302 has a function of acquiring POI information that is an example of augmented reality information from the content server 371. Here, the POI is an example of an object arranged in the real space, and is information about a specific place that is the object of someone's interest. That is, the POI information means general information associated with the position information. For example, information on facilities and stores in a specific location, including at least location information (latitude and longitude information), information such as altitude, facility / store name, telephone number, address, URL or email address Is added. The POI information is not limited to this, and may be a photo, a moving image, a voice, a message, etc. released with position information. Also, information associated with POI information including position information can be handled as POI information. For example, information regarding a tenant shop associated with information (POI information) on a shopping mall at a specific place can also be handled as POI information. POI information regarding such facilities / stores is collected from the content server 371 by the POI information collection unit 302. For example, it can be acquired using WebAPI. The POI information collection unit 302 accumulates the collected POI information in the POI data accumulation unit 316.

  The comment input unit 304 receives an input for adding a comment to the POI and registers it in the POI data storage unit 316. For example, when the communication unit 301 receives comment information transmitted from the terminal device 100, the comment input unit 304 registers the received comment information in the POI data storage unit 316.

The POI data storage unit 316 stores POI information and comment information registered by the POI information collection unit 302 and the comment input unit 304. FIG. 4 is an example of data definition of POI information stored in the POI data storage unit 316. As shown in FIG. 4, POI information includes, for example, poi_id indicating an identification number, name (title) indicating a name, lat (latitude) indicating position information, lon (longitude), altitude (altitude) indicating altitude information, Tel (phone number) indicating the contents, address (address), url (URL of the Web page), email (email address), related_poi_id indicating the POI linked to the comment information, txt indicating the content text of the comment information, comment information It is defined by resource_url indicating the data storage destination. Here, the poi_id is automatically assigned by the AR server 300, and the name (title), lat, lon, altitude, tel, address, url, and email are acquired from the content server 371 by the POI information collection unit 302. . Further, related_poi_id, txt, and resource_url are received from the terminal device 100. As described above, the POI data storage unit 316 stores the comment information posted by the user in association with the POI. Note that the poi_id is automatically assigned to the comment information stored in the POI data storage unit 316, and the name (title), lat, lon, altitude, tel, address, url, and email items are registered blank. The Thus, by additionally registering in the same format (table format) as the previously held POI information, the comment information can be handled equivalently to the POI information. Incidentally, the comment input unit 304, a comment, in advance by associating a held been related_poi_id showing the relationship between POI, may be separately registered comment or managed and POI information. By configuring in this way, POI information and comments can be managed separately, so that reliability, maintainability, etc. can be improved.

  In the case of a text comment, the content text of the comment is recorded in the txt item. On the other hand, when an image, moving image, or sound is posted as a comment, a control code corresponding to the content is recorded in the txt item, and the storage location of the resource data is recorded in the resource_url item. Resource data is automatically recorded in a storage location set in the AR server 300.

  The POI information / comment extraction unit 303 has a function of extracting POI information existing within a predetermined distance centered on the terminal position information of the terminal device 100 from the POI data storage unit 316 based on the position information of the terminal device 100. doing. The POI information / comment extraction unit 303 has a function of extracting a comment associated with the POI corresponding to the extracted POI information from the POI data storage unit 316.

  Next, operations of the terminal device 100 and the AR server 300 in the AR system 1 configured as described above will be described with reference to flowcharts. First, an operation of posting comment information from the terminal device 100 to the AR server 300 will be described. FIG. 5 is a flowchart showing the operation of the AR system 1 in the present embodiment.

  As shown in FIG. 5, processing is started from the terminal device 100. The control process illustrated in FIG. 5 is started when the input unit 103 of the terminal device 100 receives a user operation and starts an augmented reality program. First, the terminal device 100 starts live view display. For example, the camera unit 106 and various sensors (position information acquisition unit 101 and direction sensor 104) are activated, and live view display is started as an initial screen (S10). Live view display is a process in which video data is generated by an image receiving element with a predetermined exposure time from a subject image acquired by the optical system of the activated camera unit 106 and displayed on the display unit 102 according to a conventional method. Is repeated at predetermined time intervals. That is, in live view display, a captured image captured at an imaging point is displayed on the display unit 102.

  Subsequently, the position information acquisition unit 101 acquires the current terminal position information of the terminal device 100. Then, the terminal location information acquired by the terminal device 100 is transmitted to the AR server 300 to request augmented reality information (S12).

  The communication unit 301 of the AR server 300 receives the terminal location information from the terminal device 100, and outputs the terminal location information to the POI information / comment extraction unit 303 (S30). The POI information / comment extraction unit 303 extracts POI information related within a predetermined distance centered on the input terminal position information from the POI data storage unit 316 (S32). And the communication part 301 transmits the POI information extracted by the process of S32 to the terminal device 100 (S34).

  The communication unit 105 of the terminal device 100 receives the POI information around the terminal device 100 responded by the AR server 300 and records it in the storage unit 107 (S14). Then, the direction sensor 104 detects the direction (posture and angle) that the camera unit 106 faces (S16). Subsequently, the AR client processing unit 108 determines the POI information received in S14 based on the attitude and angle information of the camera unit 106 detected in S16 and the terminal position information acquired in S12. It is individually determined whether or not the associated point enters the field of view of the camera unit 106, and POI information that enters the field of view of the camera unit 106 is extracted (S18).

  Next, the terminal device 100 superimposes the POI information icon on the live view screen. The AR client processing unit 108 superimposes the POI information icon on the live view screen according to the position information and the direction information (S20). For example, assuming that the extracted POI information icon is arranged in a coordinate space with the current terminal position as the origin, the AR client processing unit 108 assumes the predetermined height of the origin as the center point and the horizontal viewing angle of the camera. The display position of each piece of information is calculated by performing a process of transparently projecting the icon of each piece of POI information on a rectangular plane determined at a predetermined distance from the origin based on the vertical viewing angle and the video size. Subsequently, based on the calculated display position of each information, an icon of each POI information is superimposed on the live view screen.

  As described above, the operation of superimposing the augmented reality information on the video in the real space is finished and the operation waits for the user's operation. The AR client processing unit 108 determines whether or not an instruction for inputting a comment is given by the user via the input unit 103 (S22). For example, the AR client processing unit 108 determines whether or not the user has selected an icon indicating POI information that is a comment input target. In the process of S22, when it is determined that the input instruction from the user is not accepted, the determination process of S22 is repeatedly executed until the user selects a predetermined icon. On the other hand, if it is determined in the process of S22 that an input instruction has been received, the AR client processing unit 108 displays a comment input screen on the display unit 102 and receives an input of a comment (S24).

  Thereafter, the AR client processing unit 108 transmits to the AR server 300 the poi_id of the POI information selected for comment input and the comment received in the processing of S24 (S26, additional information transmission step).

  The communication unit 301 of the AR server 300 receives the comment and the related poi_id from the terminal device 100, and outputs it to the comment input unit 304 (S36). The comment input unit 304 stores the input comment and the related poi_id in the POI data storage unit 316 (S38, additional information recording step). Thus, the comment information posted by the user and the POI are stored in an associated state. Then, the AR server 300 notifies the terminal device 100 of the completion of comment registration (S40) and ends the operation. On the other hand, the terminal device 100 displays the registration completion notification received from the AR server 300 on the screen (S28), and ends the operation.

  This is the end of the control process shown in FIG. By executing the control process shown in FIG. 5, the comment input to the terminal device 100 is transmitted to the AR server 300 together with the link destination poi_id and registered as new data in the POI data storage unit 316. Also, by executing the control process shown in FIG. 5, the user can input a comment in a state where the POI information is presented, so that a comment can be easily posted.

  Next, an operation in which the terminal device 100 displays comment information will be described. FIG. 6 is a flowchart showing the operation of the AR system 1 in the present embodiment.

  As illustrated in FIG. 6, the process is started from the terminal device 100. The control process shown in FIG. 6 is started when the input unit 103 of the terminal device 100 accepts a user operation and starts an augmented reality program (S50). This process is the same as the process of S10 in FIG.

  Subsequently, the location information acquisition unit 101 acquires the current terminal location information of the terminal device 100 and transmits it to the AR server 300 to request augmented reality information. This process is the same as the process of S12 of FIG.

  The communication unit 301 of the AR server 300 receives the terminal location information from the terminal device 100, and outputs the terminal location information to the POI information / comment extraction unit 303 (S70). The POI information / comment extraction unit 303 extracts POI information related within a predetermined distance centered on the input terminal position information from the POI data storage unit 316 (S72). The processes of S70 and S72 are the same as the processes of S30 and S32 of FIG. Then, the POI information / comment extraction unit 303 extracts a comment linked to the POI corresponding to the POI information extracted in the process of S72 (S74). And the communication part 301 transmits the POI information extracted by the process of S72, and the comment extracted by the process of S74 to the terminal device 100 (S76, information delivery step), and complete | finishes operation | movement.

  The communication unit 105 of the terminal device 100 receives the POI information and comments around the terminal device 100 responded by the AR server 300, and records them in the storage unit 107 (S54). The AR client processing unit 108 creates an index when a comment is recorded in the storage unit 107. Then, the direction sensor 104 detects the direction (posture and angle) in which the camera unit 106 is facing (S56). Subsequently, the AR client processing unit 108 extracts POI information that enters the field of view of the camera unit 106 (S58). The processes of S56 and S58 are the same as the processes of S16 and S18 of FIG.

  Next, the terminal device 100 renders an icon of POI information (S60). For example, the AR client processing unit 108 determines the display position based on the position information and direction information of the terminal device 100. The method for determining the display position is the same as the process of S20 in FIG.

  Next, the terminal device 100 renders the comment icon (S62). For example, the AR client processing unit 108 determines the icon display position of the comment near the POI information icon determined in the process of S60. The AR client processing unit 108 determines that the distance between them is within a predetermined range in consideration of, for example, the screen size and the size of each icon so that the comment icon is adjacent to the POI information icon.

  Then, the terminal device 100 displays the POI information and the comment icon superimposed on the live view screen (S64, image display control step). The AR client processing unit 108 superimposes each icon on the live view screen based on the display position of each icon determined in S60 and S62. FIG. 7 shows an example of a screen diagram displaying comments. In FIG. 7, the POI information is “Restaurant AAA”, and the graphic object of the POI information is a round icon. For example, as shown in FIG. 7A, the AR client processing unit 108 connects the “restaurant AAA” icon and the comment icon with a line, thereby associating the POI of the posting destination with the comment. Display graphically. In addition, the AR client processing unit 108 displays the comment icon in a balloon shape from the “Restaurant AAA” icon, for example, as shown in FIG. Is displayed graphically. If there are multiple comments for the same POI, it is displayed using an aggregate icon to avoid the complexity of the screen and make the user understand that the multiple comments are intuitively related. May be. FIG. 8A shows an example of a screen diagram displaying a plurality of comments. FIG. 8A shows a case where a plurality of comments exist for the same POI in the scene of FIG. The aggregation icon is an icon indicating that a plurality of comments exist, and for example, an icon that visually displays an overlap of a plurality of icons is used.

  As described above, the operation of superimposing the augmented reality information on the video in the real space is finished and the operation waits for the user's operation. The AR client processing unit 108 determines whether or not a comment has been selected by the user via the input unit 103 (S66). In the process of S66, when it is determined that the selection by the user is not accepted, the determination process of S66 is repeatedly executed until the user selects a comment icon. On the other hand, if it is determined in step S66 that the selection has been accepted, the process proceeds to a list screen display process.

  In the list screen display process, the AR client processing unit 108 refers to the comment index recorded in the storage unit 107, and has a comment having the same related_id as the comment selected in the process of S66, that is, a post having the same POI. Extract and display comments. If no comment with the same related_id exists, only one comment is displayed. On the other hand, if there are a plurality of posted comments with the same POI, a list screen is displayed. FIG. 8B shows an example of a screen diagram displaying a plurality of comments. FIG. 8B is an example of a screen diagram that transitions when the comment shown in FIG. 8A is selected. When the comment shown in (a) of FIG. 8 is selected, the AR client processing unit 108, as shown in (b) of FIG. 8, a plurality of comments (for example, word of mouth, etc.) for “Restaurant AAA” that is POI information ). When the process of S68 ends, the operation of the terminal device 100 ends.

  Thus, the control process shown in FIG. 6 is finished. By executing the control process shown in FIG. 6, the POI and the posted comment are displayed on the live view screen of the terminal device 100. In addition, since the number of icons to be displayed on the screen can be reduced even when a plurality of comments are posted, it is possible to avoid the screen from becoming complicated and to be easily viewed and selected as an operation target. can do.

  In the control processing shown in FIG. 6, a comment such as a word of mouth is added to a predetermined POI, that is, text information is added as additional information. However, an image, a movie, a sound, or a combination thereof may be used. Good. In this case, the image, video or audio that is the substance of the additional information is stored in the location specified by the resource_url of the comment information stored in the POI data storage unit 316 of the AR server 300, and this is to be displayed or reproduced. Will be acquired by the terminal.

  As described above, according to the AR server 300 according to the present embodiment, the communication unit 301 acquires a comment to be added to the POI from the posting source terminal device 100, and the comment input unit 304 associates the comment with the POI. The POI information / comment extraction unit 303 and the communication unit 301 associate the terminal device 100 with the POI information or the POI corresponding to the POI information based on the location information of the terminal device 100 as the distribution destination. Comments are delivered. Thus, since the comment posted from the terminal device 100 as the posting source can be recorded in association with the POI, for example, when the POI information is distributed to the terminal device 100 as the distribution destination, the comment is transmitted. Is possible. For this reason, for example, since the terminal device 100 of the distribution destination can display the comment together with the POI, it is possible to display the information posted by the user on the terminal device 100.

  In addition, according to the AR system 1 or the posted information control method according to the present embodiment, a comment to be added to the POI is transmitted to the AR server 300 by the communication unit 105 of the posting source terminal device 100, and the comment input of the AR server 300 is performed. The section 304 records the comment transmitted by the posting terminal device 100 in association with the POI, and the POI information / comment extraction unit 303 and the communication unit 301 perform the processing based on the location information of the terminal device 100 as the distribution destination. The POI information and a comment associated with the POI corresponding to the POI information are distributed to the terminal device 100, and the AR client processing unit 108 of the distribution destination terminal device 100 receives a comment on the POI distributed by the AR server 300. In addition to being displayed, the association between POI and comments is It is displayed in the Fikaru. In this way, since the AR server 300 can record the comment posted from the terminal device 100 as the posting source in association with the POI, the AR server 300 distributes the POI information to the terminal device 100 as the distribution destination. Comments can be sent. For this reason, the terminal apparatus 100 of a delivery destination can display a comment with POI. Therefore, the information posted by the user can be displayed on the terminal device 100. Furthermore, since the relationship between POIs and comments can be displayed graphically, the user can intuitively understand which POI is the posted information.

  The posted information control apparatus, posted information control system, and posted information control method of the present invention are not limited to the above-described embodiment, and various modifications can be made. For example, in the above embodiment, the case where the graphics object is a round icon has been described. However, the present invention is not limited to this, and a square icon or an information balloon may be used. I do not care.

  In the above-described embodiment, the example in which the AR client processing unit 108 of the terminal device 100 performs the process of determining the display position and the like of the augmented reality information has been described. However, the server control unit 310 of the AR server 300 performs the augmented reality information. It is also possible to perform a process for determining the display position and the like and return only the result to the terminal device 100.

  In the above-described embodiment, an example of distributing actual comment data has been described. For example, only a comment entry is distributed, and the terminal device 100 as a distribution destination includes: an AR client processing unit 108 (comment acquisition unit) When the displayed comment is selected, the comment may be acquired from the AR server 300 based on the entry. By configuring in this way, it is possible to acquire only the information of the comment selected from the comments, so it is possible to reduce the communication load and increase the resource efficiency.

  In the above embodiment, when calling a server function, the function is constructed by CGI and the call is an API, but the method is configured by any communication means between applications such as other protocols and distributed objects. Also good.

  In the above-described embodiment, an example in which the display unit 102 includes a touch panel function has been described. However, the present invention is not limited to this, and the terminal device 100 may have a function of detecting a user operation. For example, instead of the touch panel function, a screen operation by a hardware button such as a pointing device, a cross key, and a numeric keypad may be detected.

  In the above embodiment, the case where the link destination of the additional information is the POI is shown. However, the link destination of the additional information is not limited to the POI, and additional information linked to the additional information can be registered. The use of the additional information added to the POI is not limited to postings such as word-of-mouth, but can also be used as a means for presenting information to the user of the AR system from the facility or the store side.

  Further, if the definition of POI data is expanded, the addition of additional information to the POI can be prohibited.

  Furthermore, in the above-described embodiment, an example in which a comment is input from the terminal device 100 that displays the POI information icon has been described. However, a comment may be input from the terminal device 100 that does not display the POI information icon. In other words, the terminal device 100 as a posting source only needs to have a posting function, and does not have to be a distribution target of POI information.

  DESCRIPTION OF SYMBOLS 1 ... Augmented reality system (AR system, contribution information control system), 100 ... Terminal device, 101 ... Position information acquisition part, 102 ... Display part, 103 ... Input part 103, 104 ... Direction sensor, 105 ... Communication part (terminal) Side transmission unit), 106 ... camera unit, 107 ... storage unit, 108 ... AR client control unit (image display control means), 300 ... augmented reality server (AR server, posted information control device), 301 ... communication unit (additional) Information acquisition unit, information distribution unit) 302... POI information collection unit 303... POI information / comment extraction unit (information distribution unit) 304. Comment input unit (additional information recording unit) 316.

Claims (5)

A terminal that superimposes and displays an object on a live image or a live image obtained by photographing a real space with a camera unit , and a posting information control device configured to be communicable with the terminal and to distribute object information about the object to the terminal. A posted information control system comprising:
The terminal of the posting source is additional information to be added to the object, and includes terminal-side transmission means for transmitting the additional information consisting of text, an image, a moving image, audio, or a combination thereof to the posting information control apparatus,
The posted information control device includes:
Additional information recording means for recording the additional information transmitted by the terminal of the posting source in association with the object;
Information extracting means for extracting the object information based on position information of the terminal of the distribution destination and extracting the additional information associated with the object corresponding to the object information;
Information distribution means for distributing the object information extracted by the information extraction means and the entry indicating the additional information extracted by the information extraction means to the terminal of the distribution destination;
With
The terminal of the delivery destination,
Picture to see the context of the entry indicating the additional information and the object and displays added an entry indicating the additional information to the object corresponding to the object information distributed by the previous SL post information controller graphically Display control means ;
When an entry indicating the additional information displayed by the image display control means is selected, additional information acquisition means for acquiring the additional information based on the entry;
Providing
Posted information control system. 2. The posting according to claim 1 , wherein the image display control means graphically displays a relationship between the object and the additional information by displaying an icon of the additional information in a shape ballooned out from the object. Information control system. 2. The posting according to claim 1 , wherein the image display control unit graphically displays a relationship between the object and the additional information by connecting the object and the icon of the additional information with a line. Information control system. The image display control means displays a plurality of the additional information as an aggregation icon when there are a plurality of the additional information associated with the object, and is aggregated when the aggregation icon is selected. The posting information control system according to any one of claims 1 to 3 , wherein a list screen on which the additional information is posted is displayed. A terminal that superimposes and displays an object on a live image or a live image obtained by photographing a real space with a camera unit , and a posting information control device configured to be communicable with the terminal and to distribute object information about the object to the terminal. A posted information control method for a posted information control system comprising:
Additional information transmitting step for transmitting the additional information consisting of text, an image, a moving image, audio, or a combination thereof to the posted information control device , which is additional information added to the object by the terminal of the posting source;
An additional information recording step in which the post information control device records the additional information transmitted by the terminal of the posting source in association with the object;
The post information control device extracts the object information based on the location information of the terminal of the distribution destination, and an information extraction step of extracting the additional information associated with the object corresponding to the object information;
An information distribution step in which the posted information control device distributes the entry indicating the object information extracted in the information extraction step and the additional information extracted in the information extraction step to the terminal of the distribution destination;
The distribution destination terminal adds and displays an entry indicating the additional information to the object corresponding to the object information distributed by the posted information control apparatus, and the association between the object and the entry indicating the additional information Image display control step for graphically displaying,
When the delivery destination terminal selects an entry indicating the additional information displayed in the image display control step, the additional information acquisition step acquires the additional information based on the entry;
A post information control method comprising:

Images