JP6059298B1 - Information processing system, information processing device, information processing program, portable terminal device and control program thereof - Google Patents

Information processing system, information processing device, information processing program, portable terminal device and control program thereof Download PDF

Info

Publication number
JP6059298B1
JP6059298B1 JP2015125180A JP2015125180A JP6059298B1 JP 6059298 B1 JP6059298 B1 JP 6059298B1 JP 2015125180 A JP2015125180 A JP 2015125180A JP 2015125180 A JP2015125180 A JP 2015125180A JP 6059298 B1 JP6059298 B1 JP 6059298B1
Authority
JP
Japan
Prior art keywords
information
content
terminal device
information processing
current position
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2015125180A
Other languages
Japanese (ja)
Other versions
JP2017010284A (en
Inventor
浩吾 森
浩吾 森
Original Assignee
オカムラ印刷株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Family has litigation
Application filed by オカムラ印刷株式会社 filed Critical オカムラ印刷株式会社
Priority to JP2015125180A priority Critical patent/JP6059298B1/en
Application granted granted Critical
Publication of JP6059298B1 publication Critical patent/JP6059298B1/en
Publication of JP2017010284A publication Critical patent/JP2017010284A/en
First worldwide family litigation filed litigation Critical https://patents.darts-ip.com/?family=57756291&utm_source=google_patent&utm_medium=platform_link&utm_campaign=public_patent_search&patent=JP6059298(B1) "Global patent litigation dataset” by Darts-ip is licensed under a Creative Commons Attribution 4.0 International License.
Application status is Active legal-status Critical

Links

Images

Abstract

An object of the present invention is to efficiently guide a user who carries a mobile terminal device. A content holding unit that holds, in association with, first activation information for activating content providing processing, at least one second activation information for activating content providing processing, and content, and a portable terminal An information determination unit that determines that the first activation information and the second activation information are received from the apparatus, and a content when the information determination unit determines that the first activation information and the second activation information are received. Content providing means for providing the content held in the holding unit to the mobile terminal device. [Selection] Figure 1

Description

  The present invention relates to a technique for providing content to a mobile terminal device.

  In the above technical field, in Patent Document 1, when one of two dogs of a printed material is photographed with a camera of a mobile terminal and transmitted to the server, the server corresponds to a reference image similar to the photographed image of the dog. A technique for distributing specific content to a mobile terminal is disclosed.

JP 2014-238815 A

  However, in the technique described in the above-mentioned document, since the content to be provided is selected using a captured image captured by the camera of the mobile terminal as a trigger for content distribution, it is possible to efficiently guide the user who carries the mobile terminal device. could not.

  The objective of this invention is providing the technique which solves the above-mentioned subject.

In order to achieve the above object, an information processing apparatus according to the present invention provides:
First activation information related to the current position calculated by the current position calculation means of the mobile terminal device for starting the content providing process, at least one second activation information for starting the content providing process, content, Content holding means for holding
The mobile terminal device or et previous SL receives the first activation information, the first information determining means determines that the current position is within a predetermined distance from at least one target location,
Second information setting means for setting the second activation information when the first information determination means determines that the distance is within the predetermined distance;
Second information determination means for determining that the second activation information has been received from the portable terminal device;
If it is determined that the second information determining unit receives the previous SL second activation information, and content providing means for providing the content held in the content holding unit to the portable terminal device,
Is provided.

In order to achieve the above object, an information processing program according to the present invention provides:
From the portable terminal device, for starting the content providing process, currently receiving the first start information about the current position calculated by the position calculating means, a predetermined distance from the current position of at least one target location of the mobile terminal device A first information determination step for determining that the
A second information setting step for setting second activation information for activating the content providing process when it is determined that the distance is within the predetermined distance in the first information determining step;
A second information determination step for determining that the second activation information has been received from the portable terminal device;
If it is determined that it has received the previous SL second activation information Te said second information determining step odor, said been content held in the content holding unit in association with the first start information and the second activation information A content providing step to be provided to the mobile terminal device;
Is executed on the computer.

In order to achieve the above object, a mobile terminal device according to the present invention includes:
An application execution means for downloading and executing an application for executing content providing processing;
During execution of the application, said to start the content providing process, now obtains the first boot information about the current position calculated by the position calculation means of the mobile terminal device, an information processing apparatus that provides the content First information transmitting means for transmitting to
When it is determined that the current position is within a predetermined distance from at least one target location, second activation information set by the information processing apparatus for activating the content providing process is acquired, Second information transmitting means for transmitting to the information processing device;
A content reproducing means for reproducing the contents provided from the information processing apparatus in response to the transmission of the previous SL second activation information,
Is provided.

In order to achieve the above object, a control program for a mobile terminal device according to the present invention includes:
An application execution step for downloading and executing an application for executing the content providing process;
During execution of the application, said to start the content providing process, now obtains the first boot information about the current position calculated by the position calculation means of the mobile terminal device, an information processing apparatus that provides the content A first information transmission step of transmitting to
When it is determined that the current position is within a predetermined distance from at least one target location, second activation information set by the information processing apparatus for activating the content providing process is acquired, A second information transmission step of transmitting to the information processing device;
A content reproduction step of reproducing the content provided from the information processing apparatus in response to the transmission of the previous SL second activation information,
Is executed on the computer.

In order to achieve the above object, an information processing system according to the present invention provides:
First activation information related to the current position calculated by the current position calculation means of the mobile terminal device for starting the content providing process, at least one second activation information for starting the content providing process, content, Content holding means for holding
In the first activation information acquired by the mobile terminal device, first information determination means for determining that the current position is within a predetermined distance from at least one target location;
Second information setting means for setting the second activation information when the first information determination means determines that the distance is within the predetermined distance;
Second information determination means for determining that the mobile terminal device has acquired the second activation information;
Content providing means for providing the content held in the content holding means from the information processing device to the portable terminal device when it is determined that the portable terminal device has acquired the second activation information ;
Is provided.

  ADVANTAGE OF THE INVENTION According to this invention, the user who carries a portable terminal device can be guide | induced efficiently.

It is a block diagram which shows the structure of the information processing apparatus which concerns on 1st Embodiment of this invention. It is a conceptual diagram which shows operation | movement of the information processing system containing the information processing apparatus which concerns on 2nd Embodiment of this invention. It is a figure which shows operation | movement of the smart phone as a portable terminal device in the information processing system which concerns on 2nd Embodiment of this invention. It is a figure which shows the structure of the information processing system which concerns on 2nd Embodiment of this invention. It is a figure which shows the registration information in the information processing system which concerns on 2nd Embodiment of this invention. It is a sequence diagram which shows the procedure of the content registration process in the information processing system which concerns on 2nd Embodiment of this invention. It is a sequence diagram which shows the procedure of the content provision process in the information processing system which concerns on 2nd Embodiment of this invention. It is a block diagram which shows the function structure of the content provision server as an information processing apparatus which concerns on 2nd Embodiment of this invention. It is a block diagram which shows the function structure of the image feature collation part which concerns on 2nd Embodiment of this invention. It is a block diagram which shows the function structure of the present position determination part which concerns on 2nd Embodiment of this invention. It is a figure which shows the structure of the provision content database which concerns on 2nd Embodiment of this invention. It is a block diagram which shows the function structure of the smart phone as a portable terminal device which concerns on 2nd Embodiment of this invention. It is a block diagram which shows the hardware constitutions of the content provision server which concerns on 2nd Embodiment of this invention. It is a flowchart which shows the process sequence of the content provision server which concerns on 2nd Embodiment of this invention. It is a flowchart which shows the procedure of the content registration process which concerns on 2nd Embodiment of this invention. It is a flowchart which shows the procedure of the content provision process which concerns on 2nd Embodiment of this invention. It is a block diagram which shows the hardware constitutions of the smart phone which concerns on 2nd Embodiment of this invention. It is a flowchart which shows the process sequence of the smart phone which concerns on 2nd Embodiment of this invention. It is a flowchart which shows the procedure of the application execution process which concerns on 2nd Embodiment of this invention. It is a conceptual diagram which shows operation | movement of the information processing system containing the information processing apparatus which concerns on 3rd Embodiment of this invention. It is a figure which shows operation | movement of the smart phone as a portable terminal device in the information processing system which concerns on 3rd Embodiment of this invention. It is a block diagram which shows the function structure of the content provision server as an information processing apparatus which concerns on 3rd Embodiment of this invention. It is a block diagram which shows the function structure of the present position determination part which concerns on 3rd Embodiment of this invention. It is a block diagram which shows the function structure of the image feature collation part which concerns on 3rd Embodiment of this invention. It is a figure which shows the structure of the provision content database which concerns on 3rd Embodiment of this invention. It is a flowchart which shows the procedure of the content provision process which concerns on 3rd Embodiment of this invention. It is a conceptual diagram which shows operation | movement of the information processing system containing the information processing apparatus which concerns on 4th Embodiment of this invention. It is a figure which shows operation | movement of the smart phone as a portable terminal device in the information processing system which concerns on 4th Embodiment of this invention. It is a block diagram which shows the function structure of the content provision server as an information processing apparatus which concerns on 4th Embodiment of this invention. It is a block diagram which shows the function structure of the present position determination part which concerns on 4th Embodiment of this invention. It is a block diagram which shows the function structure of the present date determination part which concerns on 4th Embodiment of this invention. It is a figure which shows the structure of the provision content database which concerns on 4th Embodiment of this invention. It is a figure which shows the structure of the provision content database which concerns on 5th Embodiment of this invention. It is a figure which shows the registration information in the information processing system which concerns on 5th Embodiment of this invention. It is a figure which shows the structure of the provision content database which concerns on 6th Embodiment of this invention. It is a figure which shows operation | movement of the smart phone as a portable terminal device in the information processing system which concerns on 7th Embodiment of this invention. It is a figure which shows other operation | movement of the smart phone as a portable terminal device in the information processing system which concerns on 7th Embodiment of this invention.

  Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. However, the constituent elements described in the following embodiments are merely examples, and are not intended to limit the technical scope of the present invention only to them.

[First Embodiment]
An information processing apparatus 100 as a first embodiment of the present invention will be described with reference to FIG. The information processing apparatus 100 is an apparatus that provides content to a mobile terminal device.

  As illustrated in FIG. 1, the information processing apparatus 100 includes a content holding unit 101, an information determination unit 102, and a content providing unit 103. The content holding unit 101 holds the first activation information 111 that activates the content providing process, at least one second activation information 112 that activates the content providing process, and the content 113 in association with each other. The information determination unit 102 determines that the first activation information 111 and the second activation information 112 are received from the mobile terminal device 120. The content providing unit 103 provides the content 113 to the mobile terminal device 120 when the information determining unit 102 determines that the first information 111 and the second activation 7 information 112 are received.

  According to the present embodiment, since content is provided to the mobile terminal device in consideration of at least one second startup information in addition to the first startup information, a user carrying the mobile terminal device can be efficiently guided. it can.

[Second Embodiment]
Next, an information processing system including an information processing apparatus according to the second embodiment of the present invention will be described. In the information processing system according to the present embodiment, it is determined that a video or an image captured by a camera that is an imaging unit of the mobile terminal device corresponds to an image registered in advance. Furthermore, the distribution processing of the associated content is started using the current position calculated by the current position calculation unit of the mobile terminal device as a condition. The combination of the captured image and the current position and the content to be distributed are selected and registered in advance according to a scenario along a flow line for guiding the user to a place related to the registered image. In this embodiment, the case where the video or image captured by the camera is a building or a landscape that can be captured from the position of the user will be described. However, it is an image obtained by capturing a photograph or printed matter on a medium such as a brochure or magazine. May be.

  For example, navigation from a current position to a destination (hereinafter also referred to as a target location) related to the building or landscape is performed from the building or landscape projected by the camera of the mobile terminal device. The user is more strongly guided to the destination by navigation from the current position. According to this example, if the destination is a known destination, the user is guided to the destination without hesitation by the route guidance of the mobile terminal device, and if the destination is an unknown destination, the user can be collected by a stronger guidance.

  Note that the content is not limited to guidance content such as navigation. In other words, according to the scenario along the flow line for guiding the user based on the captured image or video and the current position, the business owner who is also the client of the information processing system of the present embodiment creates and registers the marketing. Become a powerful weapon.

《Information processing system》
Hereinafter, with reference to FIG. 2A thru | or FIG. 4B, the structure and operation | movement of an information processing system including the information processing apparatus of this embodiment are demonstrated. In the present embodiment, a description will be given of a configuration in which an image matching process and a process based on the current position are executed in an information processing apparatus as a content providing server. However, the configuration may be such that a matching process or a process based on the current position is performed in a portable terminal apparatus capable of high-speed processing with high functionality, and the result is notified to the information processing apparatus.

(Operation concept)
FIG. 2A is a conceptual diagram showing an operation of the information processing system 200 including the information processing apparatus according to the present embodiment.

  In the information processing system 200, the content providing server 230 serving as an information processing device is provided from a client PC (Personal Computer) 220 possessed by a business owner client to a smartphone 210 serving as a mobile terminal device carried by the end user. The service to be registered is registered. In the present embodiment, the target image that is the first activation information that is the start of the content providing process, the current position and the target position that are the second activation information that is the start of the content providing process, and the content that provides the service be registered. If the target position can be acquired from the target image, the client need not register from the client PC 220. Alternatively, the configuration may be such that the client is set to use the current position as the second activation information. The first activation information in the present embodiment is an image captured from the current position. In addition, even if it is not an image showing a direct guidance target, an opportunity to attract the user is expanded by registering it as a related image.

  The acquisition of the current position is not limited to calculation from GPS (Global Positioning System) or the like. For example, a captured image representing the current position may be received, information representing the position may be received from a device disposed at a transportation station, or the position may be received from a device disposed at a gate on a road. May be received. Even if it is not the current position indicating the direct guidance target, registering it as a related position widens the opportunity to attract the user.

  While the application is activated, an image seen from the current position as the first activation information is captured by an imaging unit (camera) included in the smartphone 210 as a mobile terminal device carried by the end user. If the smartphone 210 has a high function, the captured image is collated with the target image stored in advance in association with the content in the content providing server 230 if the smartphone 210 does not have the high function. If the captured image and the target image match within a predetermined range, the target position corresponding to the target image is acquired. Furthermore, if the current position transmitted as the second activation information from the smartphone 210 is within a predetermined radius from the target position, the content stored in association with the first activation information and the second activation information is transmitted from the content providing server 230. Provided to the smartphone 210. In addition, a portable terminal device is not limited to a smart phone, Other devices, such as a tablet and a mobile phone, may be sufficient.

  The content to be provided is created along the marketing flow to guide the user to business purposes, and the target image and selection information are also selected and registered in consideration of the expansion of attraction opportunities and attraction. Is done.

  In the present embodiment, an example is shown in which a feature amount of color information is extracted as feature information of an image or video, and image or video is collated. However, the feature amount of the image or video is not limited to the example of the present embodiment as long as a feature amount with a low collation processing load and high reliability of the collation result is used. Further, in the present embodiment, the description will be made as a configuration in which the provided content is received and reproduced by the smartphone 210, but the content providing server 230 or the content execution server is separately provided, and the content is executed on the server side. The structure which passes a result to the smart phone 210 may be sufficient. In addition, a configuration may be provided in which the client PC 220 is notified of the reliability level of the registered image serving as the target image based on the reliability of the collation result. The client PC 220 may be a mobile terminal device such as a smartphone or a tablet. Further, although the content providing server 230 is described as one device, the content providing server 230 may be realized by a network connection of a plurality of servers. For example, a content registration server or a Web server may be provided separately.

  FIG. 2B is a diagram illustrating an operation of the smartphone 210 as a mobile terminal device in the information processing system 200 according to the present embodiment.

  An application for providing content when the imaging information of the present embodiment satisfies the content provision conditions is downloaded and installed on the smartphone 210. Then, the application (“BR bonus” in FIG. 2B) 211 of the present embodiment is activated.

  For example, an image 212 including a sky tree is captured by the camera of the smartphone 210 while the application of the present embodiment is activated. The smartphone 210 or the content providing server 230 determines the sky tree in the image 212 based on the feature amount, and sets the sky tree as first activation information for activating the application process. Then, the position of the sky tree is acquired as the target location 213. At that time, the smartphone 210 calculates the current position using GPS, and sets a condition that the current position is within a predetermined distance 214 (200 m in the figure) from the target location 213 as the second activation information. In association with the first activation information and the second activation information, the content providing server 230 executes video content indicating the navigation 215 to the sky tree that is the guidance destination. In that case, navigation content such as navigation provides an area including the sky tree “●” that is the guidance destination and the current position “O” in dimensions that can be displayed on the smartphone 210, and the content from the current position to the sky tree is provided. Continue navigation. Therefore, guidance by accurate navigation based on the current position and the target position is possible without requiring the user's position determination.

(Constitution)
FIG. 3A is a diagram illustrating a configuration of the information processing system 200 according to the present embodiment.

  A client PC 220 of company A, which is a client, connects to a “BR Bonus server”, which is a content providing server 230 of company B, via the Internet 340, and uploads a target image and content, if necessary, a target location. Existing images may be used for the target image and the content, but it is desirable to use an original suitable for the present embodiment, which is photographed and produced by Company A. As content, for example, a URL address or a navigation system may be embedded and automatically transferred. In this case, the content from the URL address destination or the content of the navigation system may be provided from the external provider 350.

  In the content providing server 230 of company B, the target image uploaded by the client PC 220, the target location, and the content are associated and registered. At this time, the color feature amount corresponding to the present embodiment is extracted from the target image, and is used as a collation target and a search key. In addition, the company B not only registers the target image and content uploaded by the client PC 220, but also produces the target image and content in the company B based on the marketing information of the company A, and provides it to the company A. Also good. In order to respond to changes in the feature amount of the target image, such as an actual building or landscape, for example, changes due to differences in direction, season, time, and weather, a plurality of matching information may be prepared, The target image may be collated with a simplified shape or drawing.

  In the end user's smartphone 210, a dedicated application provided in advance is installed. Then, the application is started. When the application is activated and the smartphone 210 is picked up by the camera of the smartphone 210 toward the target image, the captured image is transmitted to the content providing server 230 of the B company. In addition, when the smartphone 210 can extract and compare image feature amounts with high functionality, the feature amount of the target image around the current position is distributed, and only the comparison result is transmitted to the content providing server 230 of the B company. It may be. Further, the smartphone 210 calculates the current position by GPS or the like and transmits it to the content providing server 230.

  The content providing server 230 of the company B extracts the feature amount of the received captured image, collates it with the feature amount of the target image stored in advance, and acquires a target location corresponding to the target image if they match within a predetermined range. To do. If the current position is within a predetermined distance from the target location, the content associated with the target image and the target location is distributed to the smartphone 210. The end user's smartphone 210 receives and reproduces the content associated with the target image and the target location from the content providing server 230 and attracts and guides the end user according to the purpose of marketing.

  FIG. 3B is a diagram showing registration information in the information processing system 200 according to the present embodiment. In addition, the content of the 1st starting information 360, the 2nd starting information 370, and the content 380 is not limited to FIG. 3B.

  In the present embodiment, color image information is registered as the first activation information 360. Position information is registered as the second activation information 370. Further, as the content 380, text, image, sound, moving image, 3D display, navigation, Web link (URL), and the like are registered.

(Content registration sequence)
FIG. 4A is a sequence diagram illustrating a procedure of content registration processing in the information processing system 200 according to the present embodiment.

  In step S411, communication is established between the client PC 220 and the content providing server 230. Next, in step S413, the client PC 220 and the client are authenticated by the content providing server 230 or the authentication server (not shown).

  In step S415, the client PC 220 uploads the content to the content providing server 230, and the content providing server 230 registers the content.

  In step S419, the client PC 220 sets the first activation information for the content providing server 230. If necessary, a target location is set as the second activation information. The content providing server 230 registers the first activation information, the second activation information, and the content in association with each other so as to be searchable by the first activation information and the second activation information. In the present embodiment, the first activation information is a target image and is stored not by image data itself but by a color feature amount.

  Note that the order of steps S415 and S419 may be reversed.

(Content provision sequence)
FIG. 4B is a sequence diagram illustrating a procedure of content providing processing in the information processing system 200 according to the present embodiment. FIG. 4B shows a sequence in which the content providing server 230 extracts image feature amounts and performs image matching (collation). However, if the function is high, the smartphone 210 may collate image feature amounts. Good.

  In step S <b> 421, communication is established between the end user's smartphone 210 and the content providing server 230. Next, in step S423, authentication processing of the smartphone 210 and the end user is performed by the content providing server 230 or the authentication server (not shown). If authenticated, the smartphone 210 downloads the application from the content providing server 230 in step S425.

  When the end user intends to receive the service provision of this embodiment, the downloaded application is activated in step 431. Then, the image is captured by the camera of the smartphone 210 while the application is activated, and the captured image is transmitted to the content providing server 230 in step S433. In step S435, the content providing server 230 extracts the feature amount of the captured image received from the smartphone 210, and collates it with the feature amount of the target image stored in advance in the content providing server 230.

  In step S437, the smartphone 210 calculates the current position based on GPS or the like and transmits it to the content providing server 230. Then, if the feature amount of the captured image and the feature amount of the target image match within a predetermined range, the content providing server 230 acquires a target location corresponding to the target image, and in step S439, the current location is determined from the target location. Determine if it is within distance. If the current position is within a predetermined distance from the target location, the content providing server 230 searches for content associated with the target image and the target location in step S441. In step S445, the content providing server 230 transmits the searched and acquired content to the smartphone 210, and the smartphone 210 reproduces the received content and notifies the end user. In FIG. 4B, since the content is not in the content providing server 230, an example in which the content is provided from the external provider 350 in step S443 is illustrated. Note that the current position information may be used during the reproduction of the content.

《Functional configuration of content providing server》
FIG. 5A is a block diagram illustrating a functional configuration of the content providing server 230 as the information processing apparatus according to the present embodiment.

  The content providing server 230 includes a communication control unit 501, a registration information receiving unit 502, a provided content database 503, a user registration unit 504, and an application transmission unit 505. The content providing server 230 further includes an image feature matching unit 506, a current position determination unit 510, a content acquisition unit 507, a content transmission unit 508, and a content reception unit 509. The provided content database 503 is not included in the content providing server 230, and a dedicated server may be provided outside.

  The communication control unit 501 controls communication between the content providing server 230 and the end user's smartphone 210, the business owner's client PC 220, or the external provider 350. The registration information receiving unit 502 includes an authentication unit that authenticates the client, receives the target image and content transmitted from the client PC 220, and instructs registration associated with the provided content database 503. The provided content database 503 registers the target image and content transmitted from the client PC 220 in association with each other. The provided content database 503 stores associations between target images, target locations, and contents as a table. When the captured image that matches the target image is received from the smartphone 210, the content associated with the target image and the target location is searched and passed to the content acquisition unit 507. The user registration unit 504 has an authentication unit that authenticates end users, and registers users who receive the service of the information processing system. And the application transmission part 505 transmits an application to the smart phone 210 of the authenticated end user.

  The image feature collating unit 506 extracts the feature amount of the captured image received from the smartphone 210 and collates it with the feature amount of the target image registered in the provided content database 503. If the feature amounts match within a predetermined range, the result is passed to the current position determination unit 510. The current position determination unit 510 receives the matching result from the image feature matching unit 506, acquires the target location corresponding to the target image, receives the current location information from the smartphone 210, and the current location is within a predetermined distance from the target location. Is determined. If the current position is within a predetermined distance from the target location, the current position determination unit 510 instructs the content acquisition unit 507 to acquire the content associated with the target image and the target location from the provided content database 503. The current position determination unit 510 instructs the content transmission unit 508 to transmit the content acquired by the content acquisition unit 507 from the provided content database 503. The content acquisition unit 507 acquires content associated with the target image and the target location from the provided content database 503 based on the content acquisition instruction from the current position determination unit 510. When the content is provided from the external provider 350, the content acquisition unit 507 acquires the content received from the external provider 350 by the content reception unit 509. The content transmission unit 508 transmits the content associated with the target image and the current position acquired by the content acquisition unit 507 to the smartphone 210. The content reception unit 509 receives content from the external provider 350 and passes it to the content acquisition unit 507.

  5A shows a configuration in which the image feature matching unit 506 and the current position determination unit 510 sequentially perform processing. However, the matching result of the image feature matching unit 506 and the determination result of the current position determination unit 510 are illustrated. It can be realized by providing a function configuration unit having a function of inputting and determining content acquisition by logical product, adding the function to the content acquisition unit 507, or other configurations. It is not limited to.

(Image feature matching unit)
FIG. 5B is a block diagram illustrating a functional configuration of the image feature matching unit 506 according to the present embodiment.

  The image feature matching unit 506 includes an image receiving unit 561, a feature extraction unit 562, a registered feature amount acquisition unit 563, a feature amount comparison unit 564, a target location acquisition instruction unit 565, and an optional matching reliability determination unit 566. And having.

  The image receiving unit 561 receives an image captured by the smartphone 210 via the communication control unit 501. The feature extraction unit 562 extracts the feature amount of the image captured by the smartphone 210. The registered feature amount acquisition unit 563 sequentially acquires feature amounts of target images registered in the provided content database 503. The feature amount matching unit 564 checks the feature amount of the image captured by the smartphone 210 and the feature amount of the target image, and notifies the target location acquisition instruction unit 565 if they match within a predetermined range. The target location acquisition instruction unit 565 instructs the current position determination unit 510 to acquire the target location based on the feature amount match notification from the feature amount matching unit 564. The optional matching reliability determination unit 566 determines the matching reliability in the feature value matching unit 564 and feeds back to the company A and the company B that produce the target image in order to improve the matching reliability of the target image.

(Current position determination unit)
FIG. 5C is a block diagram illustrating a functional configuration of the current position determination unit 510 according to the present embodiment. FIG. 5C shows a case where the current position determination unit 510 of the content providing server 230 performs a process of starting the content providing process with reference to the current position received from the smartphone 210.

  The current position receiving unit 511 receives the current position calculated by the smartphone 210 via the communication control unit 501. The target location acquisition unit 512 acquires the target location from the provided content database 503 in accordance with the target location income instruction from the target location acquisition instruction unit 565 of the image feature matching unit 506. The determination unit 513 determines whether the current position received by the current position reception unit 511 is within the predetermined distance D from the target location acquired by the target location acquisition unit 512, and the current position is within the predetermined distance D from the target location. If so, the content provision instructing unit 514 is notified of the provision of the content. The content provision instruction unit 514 instructs the provision content database 503, the content acquisition unit 507, and the content transmission unit 508 to provide the smartphone 210 with content registered in the provision content database 503 in association with the target image and the target location. To do.

(Provided content database)
FIG. 6 is a diagram showing a configuration of the provided content database 503 according to the present embodiment. The provided content database 503 includes an association table 610 that associates a target image as first activation information, a target location as second activation information, and provided content, and a content storage unit 620 that stores each content. The configuration of the provided content database 503 is not limited to FIG.

  The association table 610 includes first activation information (in this example, a target image that is collated with an image captured from the current position) 611 and second activation information (in this example, a target location that calculates a distance from the current position) 612. The provided content ID 614 is stored in association with. The association table 610 further includes an end-user guidance scenario (marketing flow line) 615 such as how the activation information and the provided content are produced or selected, and how the current position information is reflected. The client ID 616 of the client that registered the information is stored. FIG. 6 also stores a second activation condition 613 that is a threshold value for determining the distance between the current position and the target location. If there is no change in the second activation condition 613, it may not be stored. On the other hand, when there is a change in the similar threshold value in image collation depending on the target image, it may be stored as the first activation condition.

  The content storage unit 620 stores the content 622 and the external content ID 623 in association with the content ID 621. When the external content ID 623 is stored, it is provided from the external provider 350. The content storage unit 620 further stores a use language 624 of the content. By storing the language used 624, it is possible to determine the language used by the smartphone 210 or the like carried by the guiding end user and provide content in the corresponding language.

《Functional configuration of smartphone》
FIG. 7 is a block diagram illustrating a functional configuration of the smartphone 210 as the mobile terminal device according to the present embodiment.

  The smartphone 210 includes a communication control unit 701, a user registration unit 702, an application acquisition unit 703, an application execution unit 704, an imaging unit (camera) 705, a current position calculation unit (GPS) 706, and an output unit 707. And an input unit 708.

  The communication control unit 701 controls communication with the content providing server 230 via the network. The user registration unit 702 stores user information and requests the content providing server 230 to register a user. When the user is authenticated and registered, the application acquisition unit 703 acquires the application downloaded from the content providing server 230. The application execution unit 704 operates the smartphone 210 to execute the downloaded application of the present embodiment and receive and reproduce the content associated with the target image and the current position.

  An imaging unit (camera) 705 acquires an image or video by imaging processing. If this application is being executed, the captured image or video is passed to the application in order to collate with the target image. A current position calculation unit (GPS) 706 calculates the current position of the smartphone 210. If this application is being executed, the current position information is passed to the application during content selection and during navigation. The output unit 707 includes a display unit and an audio output unit. The display unit is used as an operation screen such as a menu display for starting an application, a monitor during imaging, or a content reproduction screen. The audio output unit is a speaker and outputs user instruction audio, content reproduction audio, or navigation audio. The input unit includes an operation unit and a voice input unit. The operation unit includes a touch panel and inputs user operation instructions. The voice input unit is at least one microphone and inputs a user operation voice.

(Application execution part)
The application execution unit 704 includes an image information transmission unit 741, an image feature extraction unit 742, a position information transmission unit 743, a content reception unit 745, a content reproduction unit 746, and a content end notification unit 747.

  The image information transmission unit 741 transmits the image or video captured by the imaging unit 705 or the feature amount of the image or video extracted by the image feature extraction unit 742 to the content providing server 230. The optional image feature extraction unit 742 extracts the feature amount of the image or video captured by the imaging unit 705, in this example, the color feature amount. The position information transmission unit 743 transmits the current position calculated by the current position calculation unit 706 to the content providing server 230.

  The content receiving unit 745 receives content associated with the target image from the content providing server 230. The content playback unit 746 plays back the content received by the content reception unit 745 and outputs the content to the output unit 707. The content end notification unit 747 notifies the content providing server 230 that the content reproduction by the content reproduction unit 746 has been completed.

<< Hardware configuration of content providing server >>
FIG. 8 is a block diagram showing a hardware configuration of the content providing server 230 according to the present embodiment.

  In FIG. 8, a CPU (Central Processing Unit) 810 is a processor for arithmetic control, and implements the functional components shown in FIG. 5A by executing a program. A ROM (Read Only Memory) 820 stores fixed data and programs such as initial data and programs. The communication control unit 501 controls communication between the content providing server 230 and other devices.

  A RAM (Random Access Memory) 840 is a random access memory used by the CPU 810 as a work area for temporary storage. The RAM 840 has an area for storing data necessary for realizing the present embodiment. The registered image / registered image feature amount 841 stores the first activation image (target image and its feature amount) instructed to be registered from the client PC 220. The registered content 842 stores content instructed to be registered from the client PC 220. The received image / received image feature amount 843 stores a captured image received from the smartphone 210 and collated with the target image and its feature amount. The image matching result (image ID) 844 stores the result of matching the registered image feature amount 841 and the received image feature amount 843. If the collation results match within a predetermined range, the image collation result (image ID) 844 also stores an image ID for searching for the content to be provided. The target location 845 stores the target location acquired from the provided content database 503 corresponding to the target image. The current position 846 stores the current position received from the smartphone 210. The current position determination result 847 stores a determination result as to whether or not the current position 846 is within a predetermined distance from the target location. The provided content 848 stores the content to be provided when the received image matches the target image within a predetermined range and the current position 846 is within a predetermined distance from the target location. The transmission / reception data 849 stores data to be transmitted / received via the communication control unit 501.

  The storage 850 stores a database, various parameters, or the following data or programs necessary for realizing the present embodiment. The provided content database 503 is a database that registers the startup image, the current position, and the content in association with each other. The storage 850 stores the following programs. The content providing server control program 851 is a program that controls the entire content providing server 230. The content registration module 852 is a module for associating and registering the first activation information and the second activation information requested by the client PC 220 in the provided content database 503. The activation information determination module 853 is a module that determines whether the captured image or video transmitted from the smartphone 210 and the current position serve as first and second activation information serving as a trigger for providing content. The content providing module 854 provides the content retrieved from the provided content database 503 if the captured image or video transmitted from the smartphone 210 and the current position are the first and second activation information for providing the content. It is.

  Note that the RAM 840 and the storage 850 in FIG. 8 do not show programs and data related to general-purpose functions and other realizable functions of each server.

<< Processing procedure of content providing server >>
FIG. 9A is a flowchart showing a processing procedure of the content providing server 230 according to the present embodiment. This flowchart is executed by the CPU 810 of FIG. 8 using the RAM 840, and implements the functional component of FIG. 5A.

  In step S911, the content providing server 230 determines whether or not the activation information from the client PC 220 is a content registration request corresponding to the activation information. If it is the registration information of the activation information and the content corresponding to the activation information, the content providing server 230 authenticates the client in step S913, and executes the provided content registration process in step S915.

  If it is not a registration request for the activation information and the content corresponding to the activation information, the content providing server 230 determines in step S921 whether or not it is a user registration request from the smartphone 210. If the request is a user registration request, the content providing server 230 authenticates the user in step S923. And the content provision server 230 provides an application with respect to the smart phone 210 which a user carries in step S925.

  If it is neither a content registration request nor a user registration request, in step S931, the content providing server 230 determines whether or not the activation information is received according to the application executed by the smartphone 210. If the activation information is received according to the application, the content providing server 230 authenticates the user in step S933. In step S935, the content providing server 230 executes a content providing process for providing the content registered in association with the activation information.

(Content registration process)
FIG. 9B is a flowchart showing a procedure of content registration processing (S915) according to the present embodiment.

  In step S941, the content providing server 230 assigns a client ID to the authenticated client. In step S943, the content providing server 230 acquires content from the client PC 220 together with the client ID.

  In step S947, the content providing server 230 acquires the target image as the first activation information from the client PC 220. In step S949, the content providing server 230 extracts the feature amount of the acquired target image. In step S951, the content providing server 230 acquires the target location as the second activation information. The target location may be received from the client PC 220 or may be found in the content providing server 230. In step S953, the content providing server 230 registers the content in association with the feature amount of the target image and the target location, and notifies the client PC 220 of the completion of registration.

(Content provision processing)
FIG. 9C is a flowchart showing a procedure of content providing processing (S935) according to the present embodiment.

  In step S961, the content providing server 230 acquires an image or video captured by the smartphone 210 listed by the user as activation information. In step S963, the content providing server 230 extracts the feature amount when the captured image itself is acquired, and compares the feature amount with the feature amount of the target image registered in advance when the feature amount is acquired. Then, it is determined whether or not they match within a predetermined range. When the feature value matching process can be executed by the smartphone 210, a matching result is acquired. If there is no matching target image feature amount, the acquired captured image is not activation information, and the process returns without doing anything. In addition, you may notify that it does not correspond with the smart phone 210. FIG.

  When the matching target image feature amount is found, the content providing server 230 determines the language used from the language information from the portable terminal device such as the smartphone 210 in the optional step S965. Then, control is performed so that content corresponding to the language used determined at the time of subsequent acquired content is acquired or the content is translated into the language used. In step S967, the content providing server 230 acquires a target location corresponding to the target image.

  The content providing server 230 receives the current position information from the smartphone 210 in step S969. Next, the content providing server 230 acquires the content associated with the target image of the first activation information and the target location of the second activation information from the provided content database 503 in step S971. Then, the content providing server 230 provides the acquired content to the smartphone 210 in step S973.

《Smartphone hardware configuration》
FIG. 10 is a block diagram illustrating a hardware configuration of the smartphone 210 according to the present embodiment.

  In FIG. 10, a CPU 1010 is a processor for arithmetic control, and implements a functional configuration unit of the smartphone 210 in FIG. 7 by executing a program. The ROM 1020 stores fixed data and programs such as initial data and programs. The communication control unit 701 controls communication with the content providing server 230 in this embodiment.

  The RAM 1040 is a random access memory that the CPU 1010 uses as a work area for temporary storage. The RAM 1040 has an area for storing data necessary for realizing the present embodiment. The content providing server address 1041 stores the IP address of the content providing server 230. The registration information 1042 stores terminal information and user information registered in the content providing server 230. The captured image (feature amount) 1043 stores an image or video captured by the imaging unit (camera) 705 during application activation and the feature amount thereof. The image collation flag 1044 is optional information, and stores the match between the captured image and the target image when the smartphone 210 can execute image matching. The current position information 1045 stores the current position calculated by the current position calculation unit 706. The received content 1046 stores the content that is retrieved and received when the captured image matches the target image and the current position is within a predetermined distance from the target location. The input / output data 1047 stores data input / output by the input / output device via the input / output interface 1060. The transmission / reception data 1048 stores data to be transmitted / received to / from the content providing server 230 via the communication control unit 701.

  The storage 1050 stores a database, various parameters, or the following data or programs necessary for realizing the present embodiment. The image matching data 1051 is data that acquires and holds the target image or its feature value from the content providing server 230 when the smartphone 210 can perform image matching. The storage 1050 stores the following programs. The smartphone control program 1052 is a program that controls the entire smartphone. The application program 1053 is an application for acquiring downloaded content. The captured image transmission module 1054 is a module that is included in the application program 1053 and transmits an image or video captured by the imaging unit 705, or a feature amount and a collation result to the content providing server 230. The optional feature amount extraction module 1055 is included in the application program 1053, and is a module for extracting the feature amount of the captured image by the smartphone 210. The optional image matching module 1056 is a module that is included in the application program 1053 and determines the first activation information by matching the captured image with the target image when the smartphone 210 can perform image matching. The current position transmission module 1057 is a module that is included in the application program 1053 and transmits the current position information calculated by the current position calculation unit 706 to the content providing server 230. The content reception / playback module 1058 is included in the application program 1053 and receives content acquired in association with the first and second activation information from the content providing server 230 based on the captured image or video and the current position. It is a module to play.

  The input / output interface 1060 performs an interface for controlling data input / output with the input / output device. In the present embodiment, the input / output interface 1060 is connected to an output unit 707 including a display unit and an audio output unit, and an input unit 708 including an operation unit and an audio input unit. An imaging unit (camera) 705 and a current position calculation unit (GPS) 706 are further connected to the input / output interface 1060.

  Note that the RAM 1040 and the storage 1050 in FIG. 10 do not show programs and data related to general-purpose functions and other realizable functions that the smartphone 210 has.

《Smartphone processing procedure》
FIG. 11A is a flowchart illustrating a processing procedure of the smartphone 210 according to the present embodiment. This flowchart is executed by the CPU 1010 of FIG. 10 using the RAM 1040, and implements the functional configuration unit of FIG.

  In step S <b> 1111, the smartphone 210 determines whether to request user registration for the service provided by the content providing server 230. When requesting user registration, the smartphone 210 transmits user information to the content providing server 230 in step S1113. In step S1115, the smartphone 210 completes user authentication and registration, and downloads the application of the service transmitted from the content providing server 230. In step S1117, the smartphone 210 displays an icon for starting the application.

  If it is not a user registration request, the smartphone 210 determines in step S1121 whether it is an application activation instruction. If there is an application activation instruction, the smartphone 210 performs an application execution process in step S1123.

  If the request is not a user registration request nor an application activation instruction, the smartphone 210 determines whether another application is selected in step S1131. In the case of selecting another application, the smartphone 210 executes another application in step S1133.

(Application execution process)
FIG. 11B is a flowchart showing a procedure of application execution processing (S1123) according to the present embodiment.

  At an arbitrary timing after the application is activated, the smartphone 210 acquires an imaged image by capturing an image or video with an imaging unit (camera) in step S1141. The smart phone 210 performs collation with the target image in optional step S1143. If the smartphone 210 has an image matching function, the smartphone 210 transmits the matching result to the content providing server 230 in step S1145. On the other hand, when the smartphone 210 does not have an image matching function, the smartphone 210 transmits the acquired captured image or its feature amount to the content providing server 230 in step S1145. Subsequently, the smartphone 210 acquires the current position in step S1147, and transmits the current position information to the content providing server 230 in step S1149. The smartphone 210 receives the target location from the content providing server 230, determines whether the distance from the current position is within a predetermined distance by the smartphone 210, and notifies the content providing server 230 of the determination result. There may be.

  In step S1151, the smartphone 210 waits for reception of the captured image and the content associated with the current position from the content providing server 230. Meanwhile, the smartphone 210 waits for a notification that the activation condition is not satisfied from the content providing server 230 in step S1153. If there is a notification that the activation condition is not satisfied from the content providing server 230, it is determined that the image does not satisfy the activation condition, and the process returns to step S1141 to wait for the next imaging. If the content providing server 230 is configured not to notify that the activation condition is not satisfied, a step for determining a timeout may be added.

  If the content is received from the content providing server 230, the smartphone 210 plays back the received content in step S1155. Next, in step S1157, the smartphone 210 determines whether an application end instruction has been issued. If the end of the application is not instructed, the smartphone 210 returns to step S1141 and waits for the next captured image. If an application termination instruction is given, the application is terminated.

  According to the present embodiment, in addition to the image information of the building or landscape that can be imaged from the position of the user as the first activation information, the content is searched based on the current position as the second activation information, and the mobile terminal device Since it provides, the user who carries a portable terminal device can be guided efficiently.

[Third Embodiment]
Next, an information processing system including an information processing apparatus according to the third embodiment of the present invention will be described. In the information processing system according to the present embodiment, compared to the second embodiment, the first activation information is related to the current position, and the second activation information is related to the image captured from the medium, and the content is distributed. Is different in that it became two. Since other configurations and operations are the same as those of the second embodiment, the same configurations and operations are denoted by the same reference numerals, and detailed description thereof is omitted.

《Information processing system》
Hereinafter, with reference to FIG. 12A thru | or FIG. 13, the structure and operation | movement of the information processing system containing the information processing apparatus of this embodiment are demonstrated.

(Operation concept)
FIG. 12A is a conceptual diagram showing an operation of the information processing system 1200 including the information processing apparatus according to the present embodiment. In FIG. 12A, the same components as those in FIG. 2A are denoted by the same reference numerals, and description thereof is omitted.

  The first activation information of the present embodiment relates to the current position calculated by the smartphone 210. While the application is activated, a target location that is within a predetermined distance from the current location of the smartphone 210 is found, and a target image representing the target location is acquired. If there are a plurality of target locations that are within a predetermined distance from the current position, a plurality of target images are acquired.

  The second activation information of this embodiment relates to an image captured from a paper medium. The image on the medium is represented by paper, but may be another printable medium, or may be a coating on a metal medium such as a sign. Examples of paper media images include, but are not limited to, architectural photographs (company, shop, etc.), product photographs, printed images including entertainment information, etc. on advertisement pamphlets or magazines. Even if it is not an image indicating a direct guidance target, registering it as a related image widens the opportunities to attract users.

  While the application is activated, an image on a paper medium serving as the second activation information is captured by an imaging unit (camera) included in the smartphone 210 as a mobile terminal device carried by the end user. The image captured from the paper medium is collated with the target image acquired based on the current position. If the captured image and the target image match within a predetermined range, the content stored in association with the first activation information and the second activation information in response to the captured image from the smartphone 210 and the current position is the content providing server 1230. Provided to the smartphone 210. Others are the same as FIG. 2A.

  FIG. 12B is a diagram illustrating an operation of the smartphone 210 as the mobile terminal device in the information processing system 1200 according to the present embodiment. In FIG. 12B, the same components as those in FIG. 2B are denoted by the same reference numerals, and description thereof is omitted. That is, it is assumed that the application “BR bonus” 211 of the present embodiment is installed and activated on the smartphone 210.

  During the activation of the application of the present embodiment, the smartphone 210 calculates the current position using GPS and transmits it to the content providing server 1230. The content providing server 1230 finds a target location within a predetermined distance (200 m in FIG. 12B) from the current position, and acquires a corresponding target image. FIG. 12B shows an example in which two target locations (Sky Tree and Kaminarimon) are found within a predetermined distance. Next, the user takes an image 1215 of a tourist brochure or magazine 1214 with the camera of the smartphone 210, for example. The smartphone 210 or the content providing server 1230 determines the Kaminarimon in the image 1215 from the feature amount, and starts application processing. As application processing, in association with the target image and the target location, the content providing server 1230 provides, as the first content 1216, a video including sightseeing guidance on the route around Kaminarimon and the route toward Kaminarimon to the smartphone 210. Subsequently, the second content indicating the navigation 1217 to Kaminarimon, which is the guidance destination, is provided to the smartphone 210. In that case, navigation content and other guidance contents provide an area including Kaminarimon “●” 1219 and current position “◯” 1218, which are the guidance destinations, in dimensions that can be displayed on the smartphone 210, and from the current position to Kaminarimon. Continue navigation. Therefore, guidance by accurate navigation based on the current position and the target position that do not require the user's position determination is possible.

(Constitution)
The configuration of the information processing system 1200 according to the present embodiment is such that the operation procedure in the smartphone 210 carried by the end user is acquisition of a target image corresponding to a target position within a predetermined distance from the current position, and a captured image and target from the medium. Although the image is changed to collation and content reproduction, since the others are the same as in FIG. 3A, illustration and description are omitted.

《Functional configuration of content providing server》
FIG. 13A is a block diagram illustrating a functional configuration of a content providing server 1230 as an information processing apparatus according to the present embodiment. In FIG. 13A, the same functional components as those in FIG. 5A are denoted by the same reference numerals, and description thereof is omitted.

  The content providing server 1230 includes a provided content database 1303, a current position determination unit 1310, and an image feature matching unit 1306 that are unique to the present embodiment. The provided content database 1303 stores the association between the target location, the target image, and the content as a table. The current position determination unit 1310 receives the current position information from the smartphone 210, finds a target location within a predetermined distance from the current position from the provided content database 1303, and obtains a target image corresponding to the target location. 1306 is instructed. The image feature collating unit 1306 extracts the feature amount of the captured image received from the smartphone 210 and collates it with the feature amount of the target image at the target location acquired from the provided content database 1303. Then, if the feature amounts match within a predetermined range, the image feature matching unit 1306 instructs the content acquisition unit 507 to acquire the content associated with the target location and the target image from the provided content database 1303. In addition, the image feature matching unit 1306 instructs the content transmission unit 508 to transmit the content acquired by the content acquisition unit 507 from the provided content database 1303.

  13A shows a configuration in which the current position determination unit 1310 and the image feature matching unit 1306 sequentially perform processing, but the determination result of the current position determination unit 1310 and the matching result of the image feature matching unit 1306 are shown. It is possible to provide a functional configuration unit having a function of inputting and determining content acquisition by logical product, adding the function to the content acquisition unit 507, or other configurations, and the configuration of FIG. 13A It is not limited to.

(Current position determination unit)
FIG. 13B is a block diagram illustrating a functional configuration of the current position determination unit 1310 according to the present embodiment. In FIG. 13B, the same functional components as those in FIG. 5C are denoted by the same reference numerals, and description thereof is omitted.

  The target location acquisition unit 1312 acquires the target location from the provided content database 1303. The determination unit 1313 determines whether the current position received by the current position reception unit 511 is within a predetermined distance D from the target location acquired by the target location acquisition unit 1312, and if the current position is within the predetermined distance D from the target location. For example, the target image acquisition instruction unit 1314 is instructed to acquire the target image. The target image acquisition instructing unit 1314 instructs the image feature matching unit 1306 to obtain the target image corresponding to the target location from the provided content database 1303.

(Image feature matching unit)
FIG. 13C is a block diagram illustrating a functional configuration of the image feature matching unit 1306 according to the present embodiment. In FIG. 13C, the same functional components as those in FIG. 5B are denoted by the same reference numerals, and description thereof is omitted.

  The target image feature amount acquisition unit 1363 sequentially acquires feature amounts of target images corresponding to the target locations registered in the provided content database 1303 in accordance with the target image acquisition instruction from the current position determination unit 1310. The feature amount matching unit 564 checks the feature amount of the image captured by the smartphone 210 and the feature amount of the target image corresponding to the target location, and notifies the content provision instructing unit 1365 if they match within a predetermined range. . Based on the feature amount matching notification from the feature amount matching unit 564, the content provision instructing unit 1365 provides the smartphone 210 with the content registered in the provision content database 1303 in association with the target location and the target image. The provided content database 1303, the content acquisition unit 507, and the content transmission unit 508 are instructed.

(Provided content database)
FIG. 14 is a diagram showing a configuration of the provided content database 1303 according to the present embodiment. In FIG. 14, the same components as those in FIG. 6 are denoted by the same reference numerals, and description thereof is omitted. The configuration of the provided content database 1303 is not limited to FIG.

  The provided content database 1303 includes an association table 1410 that associates a target location as first activation information with a captured image as second activation information and provided content, and a content storage unit 620 that stores each content.

  The association table 1410 is associated with the first activation information (in this example, the target location) 1411 and the second activation information (in this example, the target image corresponding to the captured image on the medium) 1412 in association with the first provided content. The ID 1413 and the second provided content ID 1414 are stored. The related table 1410 further includes a guidance scenario (marketing flow line) 1415 of the end user who created or selected the first and second activation information and the two provided contents, and the client ID 616 of the client that registered the related information. Remember. Note that FIG. 14 does not show the first activation condition, which is a threshold for determining the distance between the current position and the target location, and the second activation condition, which is a similar threshold for image matching, but the conditions change. You may memorize it.

(Content provision processing)
FIG. 15 is a flowchart illustrating a procedure of content providing processing (S935) according to the present embodiment. In FIG. 15, steps similar to those in FIG. 9C are denoted by the same step numbers, and description thereof is omitted.

  The content providing server 1230 acquires the current position information from the smartphone 210 in step S1561. In step S 1563, the content providing server 1230 finds the target location whose distance from the target location is within the predetermined threshold D. If not found, the content providing server 1230 ends the process and returns. If found, the content providing server 1230 acquires a captured image captured from the smartphone 210 from the medium in step S1567. In step S1569, the content providing server 1230 collates the feature amounts of the target image and the captured image that are found in step S1563 and that correspond to the target location whose distance is within the predetermined threshold value D. If there is a target image whose feature amount matches within the predetermined range, the content providing server 1230 acquires the content associated with the target location and the target image in step S1571. If there is no target image whose feature quantity matches within the predetermined range, the process is terminated and the process returns.

  According to this embodiment, in addition to the target location related to the current position as the first activation information, a plurality of contents are sequentially searched based on the image information captured from the medium as the second activation information. Since it provides to a portable terminal device, the user who carries a portable terminal device can be guided efficiently.

[Fourth Embodiment]
Next, an information processing system including an information processing apparatus according to the fourth embodiment of the present invention will be described. Compared with the second embodiment and the third embodiment described above, the information processing system according to the present embodiment has content that is distributed in consideration of the current date and time in addition to the captured image captured by the mobile terminal device and the current position. It is different in that it is selected and three contents are distributed. Other configurations and operations are the same as those in the second embodiment and the third embodiment. Therefore, the same configurations and operations are denoted by the same reference numerals, and detailed description thereof is omitted.

《Information processing system》
Hereinafter, the configuration and operation of the information processing system of this embodiment will be described with reference to FIGS. 16A and 16B.

(Operation concept)
FIG. 16A is a conceptual diagram illustrating an operation of an information processing system 1600 including the information processing apparatus according to the present embodiment. In FIG. 16A, the same components as those in FIG. 2A or FIG.

  The first activation information in the present embodiment is an image captured from a paper medium. While the application is activated, an image on a paper medium serving as the first activation information is captured by an imaging unit (camera) included in the smartphone 210 as a mobile terminal device carried by the end user. The image captured from the paper medium is collated with the target image stored in advance in association with the content. If the captured image matches the target image within a predetermined range, the second activation information is determined based on the transmission of the current position information from the smartphone 210 and the transmission of the current date and time information. If the three activation conditions are satisfied, the stored three contents are sequentially provided from the content providing server 1630 to the smartphone 210. Others are the same as FIG. 2A.

  FIG. 16B is a diagram illustrating an operation of the smartphone 210 as the mobile terminal device in the information processing system 1600 according to the present embodiment. In FIG. 16B, the same components as those in FIG. 2B or FIG. That is, it is assumed that the application “BR bonus” 211 of the present embodiment is installed and activated on the smartphone 210.

  For example, a tourist brochure or magazine image 1612 is captured by the camera of the smartphone 210 while the application of the present embodiment is activated. In the present embodiment, the image 1612 is an exhibition guide for the exhibition. The smartphone 210 or the content providing server 1630 determines the exhibition, the venue, and the date / time from the feature amount of the image 1612, and sets the first activation information that activates the processing of the application. At that time, the smartphone 210 calculates the current position using the GPS, and the time measuring unit calculates the current date and time, which are used as the second activation information. In association with the first activation information and the plurality of second activation information, the content providing server 1630 has an event “Manufacturing ...” or a place to be held from “10:00 on June 24, X” that is the target date and time. A display image 1615 such as “Big Sight” is provided and reproduced. Next, when the playback of the 1st content is completed, or when the user gives a playback skip instruction, a live video 1616 of the event venue entrance, in the venue, and the guidance booth is provided and played as the 2nd content. Then, when the playback of the 2nd content is completed, or when the user gives a playback skip instruction, the navigation 1617 to the booth in the venue, which is the guide location related to the target date and time, is executed as the 3rd content. Therefore, guidance by accurate navigation based on the current position and the target position that do not require the user's position determination is possible.

《Functional configuration of content providing server》
FIG. 17A is a block diagram illustrating a functional configuration of a content providing server 1630 as an information processing apparatus according to the present embodiment. In FIG. 17A, the same functional components as those in FIG. 5A are denoted by the same reference numerals, and description thereof is omitted.

  The provided content database 1703 registers the 1st content, the 2nd content, and the 3rd content in association with the target image, the current position, and the current date and time transmitted from the client PC 220. When the target image, the current position, and the current date / time are received from the smartphone 210, the 1st content, the 2nd content, and the 3rd content associated with the target image, the current position, and the current date / time are searched and passed to the content acquisition unit 507. In addition, the current position determination unit 1710 determines the target date and time with respect to the current date and time determination unit 1711 if the distance between the target location acquired according to the target location acquisition instruction from the image feature matching unit 506 and the current position is within a predetermined range. The acquisition of The current date and time determination unit 1711 also provides the provided content database 1703, when the difference between the target date and time acquired according to the target date and time acquisition instruction from the current position determination unit 510 and the current date and time received from the smartphone 210 is within a predetermined time. The content acquisition unit 507 and the content transmission unit 508 are instructed to transmit the 1st content, the 2nd content, and the 3rd content acquired from the provided content database 1703.

  17A shows a configuration in which the image feature matching unit 506, the current position determination unit 1710, and the current date and time determination unit 1711 sequentially perform processing. However, the matching result of the image feature matching unit 506 and the current position determination unit 1710 are illustrated. And a determination result of the current date and time determination unit 1711 are input, and a function configuration unit having a function of determining acquisition of content by logical product is provided, or the function is added to the content acquisition unit 507, etc. This configuration can also be realized, and is not limited to the configuration of FIG. 17A.

(Current position determination unit)
FIG. 17B is a block diagram illustrating a functional configuration of the current position determination unit 1710 according to the present embodiment. In FIG. 17B, the same functional components as those in FIG. 5C are denoted by the same reference numerals, and description thereof is omitted. The target date / time acquisition instruction unit 1714 transmits the content registered in the provided content database 1703 in association with the target image, the target location, and the target date / time to the smartphone 210, and the content transmission. Section 508 is instructed.

(Current date / time determination section)
FIG. 17C is a block diagram illustrating a functional configuration of the current date and time determination unit 1711 according to the present embodiment. FIG. 17C is a configuration in the case where the current date and time determination unit 1711 of the content providing server 1630 starts the content providing process with reference to the current date and time received from the smartphone 210.

  The current date and time reception unit 1761 receives the current date and time calculated by the smartphone 210 via the communication control unit 501. The target date and time acquisition unit 1762 acquires the target date and time from the provided content database 1703 in accordance with the target date and time acquisition instruction from the current position determination unit 1710. The determination unit 1763 determines whether the current date and time received by the current date and time reception unit 1761 is within the predetermined time T until the target date and time acquired by the target date and time acquisition unit 1762, and the current date and time reaches the target date and time. If it is within, the content provision instructing unit 1764 is notified of the provision of the content. The content provision instructing unit 1764 is configured to provide the first content, the second content, and the third content registered in the provision content database 1703 in association with the target image, the target location, and the target date and time to the smartphone 210. The content acquisition unit 507 and the content transmission unit 508 are instructed.

(Provided content database)
FIG. 18 is a diagram showing the configuration of the provided content database 1703 according to this embodiment. In FIG. 18, the same components as those in FIG. 6 or FIG. The configuration of the provided content database 1703 is not limited to FIG.

  The provided content database 1703 includes a relation table 1810 that associates the target image as the first activation information, the target location and target date and time as the second activation information, and the content to be provided, and a content storage unit 620 that stores each content. Have

  The association table 1810 is associated with the first activation information (in this example, the target image corresponding to the captured image on the medium) 611 and the second activation information (target location, target date and time) 1812, and the first provided content ID 1413. And the second provided content ID 1414 and the third provided content ID 1814 are stored. The association table 1810 further guides the end user, such as how the first and second activation information and the three provided contents are produced or selected, and how the current position information and current date information are reflected. The scenario (marketing flow line) 1815 is stored.

  According to the present embodiment, in addition to the captured image taken as the first activation information and the current position as the second activation information, a plurality of contents are sequentially provided to the mobile terminal device based on the current date and time. Therefore, it is possible to efficiently guide the user who carries the mobile terminal device.

[Fifth Embodiment]
Next, an information processing system including an information processing apparatus according to the fifth embodiment of the present invention will be described. In the information processing system according to the present embodiment, the application processing start in the mobile terminal device is performed with various information including the captured image, the current position, and the current date and time as compared with the second to fourth embodiments. The content selection information is different in that it is performed with various information. Since other configurations and operations are the same as those in the second to fourth embodiments, the same configurations and operations are denoted by the same reference numerals, and detailed description thereof is omitted.

  In the present embodiment, the trigger of the application process start takes into account the characteristics of the captured image, various information including the current position and current date, or a combination thereof. And according to the feature, the selection of the content to be provided to the portable terminal device carried by the user and the order of the provision are in line with the flow line for guiding the user to the destination related to each information and the combination thereof. Pre-selected and registered according to the scenario. The combination of the activation information, the selection information, and the content becomes a powerful marketing weapon when created and registered by a business owner who is also a client of the information processing system of the present embodiment.

(Provided content database)
FIG. 19 is a diagram showing a configuration of the provided content database 1903 according to the present embodiment. FIG. 19 shows only the association table that associates the first activation information, the second activation information, and the provided content to be provided, and the content storage unit that stores each content is omitted.

  The related table of the provided content database 1903 includes the provided content 1913, the guidance scenario (marketing flow line) 1914, the client ID 1915, and the corresponding language 1916 in association with the first activation information 1911 and the second activation information 1912. Remember. The corresponding language 1916 indicates the language used by the smartphone 210 that the provided content 1913 can support.

(Registration information)
FIG. 20 is a diagram showing registration information in the information processing system according to the present embodiment. The contents of the first activation information, the second activation information, and the content are not limited to FIG.

  In the present embodiment, as the first activation information 2060, color image information, position information, date and time information, audio information, direction information, acceleration information, speed information, illuminance information, and environment information that can be acquired by a mobile terminal device such as a smartphone. , Biometric information, or a combination thereof is registered. The environmental information includes information such as temperature and humidity, and the biological information includes body temperature and blood pressure. A thermometer, a hygrometer, etc. are incorporated in acquisition of environmental information. In addition, a wearable portable terminal device is suitable for obtaining biometric information, in which a thermometer, a hygrometer, or the like is incorporated.

  As the second activation information 2070, color image information, position information, date and time information, audio information, direction information, acceleration information, speed information, illuminance information, environmental information, biological information, which can be acquired by a mobile terminal device such as a smartphone. Any one or a combination thereof is registered. The environmental information includes information such as temperature and humidity, and the biological information includes body temperature and blood pressure. A thermometer, a hygrometer, etc. are incorporated in acquisition of environmental information. In addition, a wearable portable terminal device is suitable for obtaining biometric information, in which a thermometer, a hygrometer, or the like is incorporated.

  The first content 2080 or the second content 2090 is selected from text, image, sound, video, 3D display, navigation, web link (URL), etc., and is selected and ordered based on the guidance scenario.

  According to the present embodiment, since a combination of various information detected by the mobile terminal device is used as a trigger, at least two contents adapted to the information can be provided sequentially, so according to a scenario along the flow line in marketing required by the business owner The user can be actively guided to a predetermined place.

[Sixth Embodiment]
Next, an information processing system according to the sixth embodiment of the present invention will be described. The information processing system according to the present embodiment is different from the second to fifth embodiments in that it evaluates and learns a guidance scenario including a start condition and provided content in an application. Since other configurations and operations are the same as those in the second to fifth embodiments, the same configurations and operations are denoted by the same reference numerals, and detailed description thereof is omitted.

(Provided content database)
FIG. 21 is a diagram showing the configuration of the provided content database 2103 according to this embodiment. In FIG. 21, the same components as those in FIG. 19 are denoted by the same reference numerals, and description thereof is omitted.

  The provided content database 2103 is configured to propose a selection or combination that further enhances the guidance effect by accumulating guidance results when the first activation information, the second activation information, and the provision content are selected and combined according to the guidance scenario. Have. For this purpose, guidance scenario evaluation 2117 and countermeasure 2118 are stored in association with each guidance scenario 1914. The guidance scenario evaluation 2117 includes a guidance result, condition evaluation, content evaluation, and the like.

  According to the present embodiment, by accumulating execution results such as start conditions and provided contents in an application based on a guidance scenario and improving the evaluation, the user is brought to a predetermined location according to the scenario along the flow line in marketing required by the business owner. Can be guided more efficiently.

[Seventh Embodiment]
Next, an information processing system according to a seventh embodiment of the present invention will be described. The information processing system according to the present embodiment is different from the second to sixth embodiments in that the present invention is applied to a game-like event rather than a user's guidance to a target location. Since other configurations and operations are the same as those in the second to sixth embodiments, the same configurations and operations are denoted by the same reference numerals, and detailed description thereof is omitted.

<< Operation concept of information processing system >>
FIG. 22A is a diagram illustrating an operation of the smartphone 210 as the mobile terminal device in the information processing system according to the present embodiment. In FIG. 22A, components similar to those in FIG. 2B are denoted by the same reference numerals, and description thereof is omitted. That is, it is assumed that the application “BR bonus” 211 of the present embodiment is installed and activated on the smartphone 210.

  While the application according to the present embodiment is activated, the user captures the “treasure hunting” brochure or the target photograph 2213 of the magazine page 2212 with the camera and transmits it to the content providing server. The content providing server collates the target image (bag in this example) corresponding to the client ID with the captured image, and acquires the target prize display location 2214 (target location). Then, the smartphone 210 calculates the current position using GPS and transmits it to the content providing server. When the current position is within a predetermined distance 2215 (100 m in FIG. 22A) from the display location of the target prize, the content providing server starts processing the application. The content providing server provides the bag store introduction video 2216 associated with the target image and the target location, and the smartphone 210 plays the introduction video 2216. Next, the user takes an image of the bag 2217 that seems to be the target prize with the camera of the smartphone 210. The content providing server collates the captured image of the bag 2217 with the target image with higher accuracy, and if it matches, the content providing server determines that the bag is the target prize and notifies the smartphone 210 that the bag that is the target prize is obtained. Send to user. According to this example, by providing a simple application, it is possible to efficiently solicit users for the promotion of a bag, the promotion of a store, or the activation of a shopping street or a region.

  FIG. 22B is a diagram illustrating another operation of the smartphone 210 as the mobile terminal device in the information processing system according to the present embodiment. In FIG. 22A, the same reference numerals are given to the same components as in FIG. 2B, and the description will be omitted. That is, it is assumed that the application “BR bonus” 211 of the present embodiment is installed and activated on the smartphone 210.

  During the activation of the application of the present embodiment, the smartphone 210 calculates the current position using GPS and transmits it to the content providing server. The content providing server finds a place within a predetermined distance from the current position among a plurality of places where each stamp image of the stamp rally, which is the target place of this example, and acquires a corresponding target image. Next, the user captures an image 2224 that seems to be the target image with the camera of the smartphone 210. Note that a hint image capable of conceiving the image 2224 may be provided to the smartphone 210. If it is determined from the feature amount of the image 2224 that the content providing server matches the stamp image at the target location, the content providing server starts processing of the application. In association with the target location and the target image, the content providing server provides the smartphone 210 with a promotion video including tourist information on the target location as the first content 2225 as the processing of the application. Subsequently, the second content indicating the stamp rally progress table 2226 is provided to the smartphone 210. According to this example, by providing a simple application, it is possible to efficiently invite users for introduction of each target place or introduction and activation of a region.

  According to the present embodiment, it is possible to efficiently solicit users by simply executing a game-like application on a smartphone.

[Other Embodiments]
Although the present invention has been described with reference to the exemplary embodiments, the present invention is not limited to the above exemplary embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. In addition, a system or an apparatus in which different features included in each embodiment are combined in any way is also included in the scope of the present invention.

  In addition, the present invention may be applied to a system composed of a plurality of devices, or may be applied to a single device. Furthermore, the present invention can also be applied to a case where an information processing program that implements the functions of the embodiments is supplied directly or remotely to a system or apparatus. Therefore, in order to realize the functions of the present invention on a computer, a program installed in the computer, a medium storing the program, and a WWW (World Wide Web) server that downloads the program are also included in the scope of the present invention. . In particular, at least a non-transitory computer readable medium storing a program for causing a computer to execute the processing steps included in the above-described embodiments is included in the scope of the present invention.

Claims (13)

  1. First activation information related to the current position calculated by the current position calculation means of the mobile terminal device for starting the content providing process, at least one second activation information for starting the content providing process, content, Content holding means for holding
    The mobile terminal device or et previous SL receives the first activation information, the first information determining means determines that the current position is within a predetermined distance from at least one target location,
    Second information setting means for setting the second activation information when the first information determination means determines that the distance is within the predetermined distance;
    Second information determination means for determining that the second activation information has been received from the portable terminal device;
    If it is determined that the second information determining unit receives the previous SL second activation information, and content providing means for providing the content held in the content holding unit to the portable terminal device,
    An information processing apparatus comprising:
  2. The second activation information includes a target image related to the target location imaged by the mobile terminal device,
      When the second information determination unit determines that the target image has been received, the content providing unit notifies the user who carries the portable terminal device that the privilege information corresponding to the target image is provided. The information processing apparatus according to claim 1, which provides content.
  3. The information processing apparatus according to claim 2, wherein the privilege information is acquisition information of a product corresponding to the target image.
  4. The information processing apparatus according to claim 2, wherein the privilege information is acquisition information of a stamp corresponding to the target image.
  5. The second activation information includes information on a target date and time related to the current date and time acquired by the mobile terminal device,
    The content providing unit, when said current date second information determination unit receives is determined from the target date to be within the predetermined time, the guidance content to entertainment associated with the current date and time in the target location providing information processing apparatus according to claim 1 or 2.
  6. The second activation information includes predetermined azimuth information acquired by the mobile terminal device,
      The content providing unit provides the content held in the content holding unit to the portable terminal device when the second information determination unit determines that the predetermined orientation information is received. Alternatively, the information processing apparatus according to 2 or 5.
  7. The content providing means provides first content for guiding a user carrying the mobile terminal device to the target location and then playing the second content for sequentially guiding the user to the target location. to, the information processing apparatus according to claim 1.
  8. The second activation information further includes language information indicating a language that the portable terminal device is used,
    The information processing apparatus according to claim 2 , wherein the content providing unit provides content in a language corresponding to the language information to the mobile terminal device.
  9. 9. The content registration unit according to claim 1, further comprising: a content registration unit configured to associate and register the first activation information, the at least one second activation information, and the content in the content holding unit. The information processing apparatus described.
  10. From the portable terminal device, for starting the content providing process, currently receiving the first start information about the current position calculated by the position calculating means, a predetermined distance from the current position of at least one target location of the mobile terminal device A first information determination step for determining that the
    A second information setting step for setting second activation information for activating the content providing process when it is determined that the distance is within the predetermined distance in the first information determining step;
    A second information determination step for determining that the second activation information has been received from the portable terminal device;
    If it is determined that it has received the previous SL second activation information Te said second information determining step odor, said been content held in the content holding unit in association with the first start information and the second activation information A content providing step to be provided to the mobile terminal device;
    An information processing program that causes a computer to execute.
  11. An application execution means for downloading and executing an application for executing content providing processing;
    During execution of the application, said to start the content providing process, now obtains the first boot information about the current position calculated by the position calculation means of the mobile terminal device, an information processing apparatus that provides the content First information transmitting means for transmitting to
    When it is determined that the current position is within a predetermined distance from at least one target location, second activation information set by the information processing apparatus for activating the content providing process is acquired, Second information transmitting means for transmitting to the information processing device;
    A content reproducing means for reproducing the contents provided from the information processing apparatus in response to the transmission of the previous SL second activation information,
    A mobile terminal device comprising:
  12. An application execution step for downloading and executing an application for executing the content providing process;
    During execution of the application, said to start the content providing process, now obtains the first boot information about the current position calculated by the position calculation means of the mobile terminal device, an information processing apparatus that provides the content A first information transmission step of transmitting to
    When it is determined that the current position is within a predetermined distance from at least one target location, second activation information set by the information processing apparatus for activating the content providing process is acquired, A second information transmission step of transmitting to the information processing device;
    A content reproduction step of reproducing the content provided from the information processing apparatus in response to the transmission of the previous SL second activation information,
    A program for controlling a portable terminal device that causes a computer to execute.
  13. First activation information related to the current position calculated by the current position calculation means of the mobile terminal device for starting the content providing process, at least one second activation information for starting the content providing process, content, Content holding means for holding
    In the first activation information acquired by the mobile terminal device, first information determination means for determining that the current position is within a predetermined distance from at least one target location;
    Second information setting means for setting the second activation information when the first information determination means determines that the distance is within the predetermined distance;
    Second information determination means for determining that the mobile terminal device has acquired the second activation information;
    Content providing means for providing the content held in the content holding means from the information processing device to the portable terminal device when it is determined that the portable terminal device has acquired the second activation information ;
    An information processing system comprising:
JP2015125180A 2015-06-22 2015-06-22 Information processing system, information processing device, information processing program, portable terminal device and control program thereof Active JP6059298B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2015125180A JP6059298B1 (en) 2015-06-22 2015-06-22 Information processing system, information processing device, information processing program, portable terminal device and control program thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2015125180A JP6059298B1 (en) 2015-06-22 2015-06-22 Information processing system, information processing device, information processing program, portable terminal device and control program thereof

Publications (2)

Publication Number Publication Date
JP6059298B1 true JP6059298B1 (en) 2017-01-11
JP2017010284A JP2017010284A (en) 2017-01-12

Family

ID=57756291

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2015125180A Active JP6059298B1 (en) 2015-06-22 2015-06-22 Information processing system, information processing device, information processing program, portable terminal device and control program thereof

Country Status (1)

Country Link
JP (1) JP6059298B1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6181893B1 (en) * 2017-03-31 2017-08-16 株式会社Cygames Browsing confirmation system, portable terminal device, server, program, and method

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3548459B2 (en) * 1998-11-20 2004-07-28 富士通株式会社 Guidance information presentation device, guidance information presentation processing method, a recording medium which records the guidance information presentation program, guiding script generating device, guidance information providing apparatus, guidance information providing method and guidance information providing program recording medium
JP2009181186A (en) * 2008-01-29 2009-08-13 Mitsubishi Space Software Kk Information providing server, information display terminal, information providing system, information providing program, information display program, information providing method and information display method
WO2010116483A1 (en) * 2009-04-06 2010-10-14 株式会社ナビタイムジャパン Navigation system, route search server, route search agent server, and navigation method
WO2013121470A1 (en) * 2012-02-15 2013-08-22 パナソニック株式会社 Content presentation device, terminal, system, program, and method
JP5194198B1 (en) * 2012-02-15 2013-05-08 パナソニック株式会社 Content presentation device, terminal, system, program, and method

Also Published As

Publication number Publication date
JP2017010284A (en) 2017-01-12

Similar Documents

Publication Publication Date Title
JP5879637B2 (en) Intuitive computing method and system
US9424255B2 (en) Server-assisted object recognition and tracking for mobile devices
EP2616901B1 (en) Moving information between computing devices
US20140168056A1 (en) Enabling augmented reality using eye gaze tracking
US20110279445A1 (en) Method and apparatus for presenting location-based content
Schmalstieg et al. Augmented Reality 2.0
KR20130027081A (en) Intuitive computing methods and systems
US9699375B2 (en) Method and apparatus for determining camera location information and/or camera pose information according to a global coordinate system
KR101796008B1 (en) Sensor-based mobile search, related methods and systems
US20110310227A1 (en) Mobile device based content mapping for augmented reality environment
US20060190812A1 (en) Imaging systems including hyperlink associations
CN102770862B (en) Mixture of the position sensor data and visual query to return a local visual query included
US20110096992A1 (en) Method, apparatus and computer program product for utilizing real-world affordances of objects in audio-visual media data to determine interactions with the annotations to the objects
CN103632626B (en) Intelligent guide method for implementing a mobile Internet-based, and mobile client device
US8819030B1 (en) Automated tag suggestions
WO2013023705A1 (en) Methods and systems for enabling creation of augmented reality content
WO2012021267A2 (en) Location and contextual-based mobile application promotion and delivery
CN103502982A (en) Method and apparatus for displaying interactive preview information in a location-based user interface
KR20100124947A (en) Ar contents providing system and method providing a portable terminal real-time regional information by using augmented reality technology
JP2009540414A (en) Media identification
CN107092646A (en) Systems and methods for image recognition using mobile devices
CN102467574B (en) Mobile terminal and metadata setting method
KR20110126180A (en) Human assisted techniques for providing local maps and location-specific annotated data
EP2831807A1 (en) User-guided object identification
CN102893129B (en) The method of determining the terminal position determination system, the mobile terminal and the terminal position

Legal Events

Date Code Title Description
TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20161117

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20161208

R150 Certificate of patent or registration of utility model

Ref document number: 6059298

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350

S531 Written request for registration of change of domicile

Free format text: JAPANESE INTERMEDIATE CODE: R313531

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350

S111 Request for change of ownership or part of ownership

Free format text: JAPANESE INTERMEDIATE CODE: R313113

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350