JP6628516B2 - Information providing system and computer program - Google Patents

Description

  The present invention relates to an information providing system that mainly provides information via business cards.

  As a technology for providing information via a business card, a person who receives the business card prints the identification code of the registered user representing the personal information as a QR code (registered trademark) on the business card so that the person who has received the business card can use the terminal device. There has been known a technology that enables a business card to acquire detailed information of a person representing personal information using a business card (for example, Patent Document 1).

  Here, in this technology, the terminal device captures a business card, accesses a management server that manages a QR code printed on the business card, and manages detailed information of each person, and indicates the decoded identification code. It has a function of acquiring and displaying detailed information about a person.

In addition, as a technology for performing AR display that displays an image obtained by synthesizing AR (Augmented Reality: Augmented Reality) content with an image of the actual situation, information on acquiring the AR content is printed on a card as a QR code There is known a technique in which a card receiver can use an AR display in which an AR content is combined with an image obtained by photographing the card using a terminal device by using a terminal device (for example, Patent Document 2).

  Here, in this technology, the terminal device captures a card, decodes a QR code printed on the card, accesses a content database storing image content as AR content, and decodes the information obtained by decoding the QR code. It has a function of acquiring an image content by using the acquired image content, and displaying an image obtained by synthesizing the acquired image content on an image obtained by photographing the card in an arrangement determined with respect to an outer frame figure printed as a marker on the card. .

JP 2011-86198 A JP 2011-130386 A

  By applying the AR display technology as described above and printing information for acquiring the AR content on the business card as a QR code, the recipient of the business card can print the AR content on the image of the business card. When the AR display that displays a video obtained by synthesizing is used, the following problem occurs.

  In other words, according to this technique, since the outer frame figure printed on the business card as a marker is common to each business card, after acquiring the AR content using the information of the QR code of one business card, when capturing another business card, In other words, the acquired AR content is synthesized and displayed on the video of the other business card.

  For this reason, when using different AR contents for each person on a business card, such as a photograph image of a person whose business card represents personal information, a photograph image of another person is synthesized and displayed on the business card. In addition, there is a problem in that AR content that does not correspond to a business card is synthesized and displayed on a business card video.

  Therefore, the present invention provides an AR display that combines and displays AR content corresponding to the person of the business card on the image of the business card, and combines the AR content with only the image of the business card of the person who correctly corresponds to the AR display. And display it.

  In order to achieve the object, the present invention provides a terminal as an information providing system for performing an AR display for combining and displaying AR content corresponding to an object whose information is described in a printed matter with an image of the printed matter. Providing information including an apparatus and an AR content server that stores AR content provided for each target object in association with the target object in association with an identification character string that is a character string representing the identification of the target object Provide system. Here, the terminal device includes a camera, an AR content acquisition unit, and an AR display unit, and the AR content acquisition unit corresponds to an object whose information is described in a printed matter taken by the camera. The information including the AR content and the identification character string associated with the AR content is obtained from the AR content server as AR content information, and the AR display unit converts the image of the print taken by the camera into the print. Recognize the described identification character string, and when the recognized identification character string matches the identification character string included in the AR content information acquired by the AR content acquisition means, the AR acquired by the AR content acquisition means. The AR content included in the content information is combined with the image of the printed matter taken by the camera and displayed.

  In order to achieve the above object, the present invention provides an information providing system for performing AR display in which AR content corresponding to an object whose information is described in a printed matter is combined with an image of the printed matter and displayed. , A terminal device, and an AR content server that stores AR content provided for each target object in association with the target object in association with an identification character string that is a character string representing the identification of the target object. Provide an information providing system. Here, the terminal device has a camera, an AR content acquisition unit, an AR display unit, and a storage device, and the AR content acquisition unit is an object whose information is described in a printed matter taken by the camera. The information including the AR content corresponding to the AR content and the identification character string associated with the AR content is obtained from the AR content server as AR content information, and stored in the storage device. The camera recognizes an identification character string described in the printed matter from the image of the printed matter, and the recognized identification character string is an identification character string included in any of the AR content information stored in the storage device. If they match, the AR content included in the AR content information including the identification character string that matches the recognized identification character string is combined with the image of the printed matter taken by the camera. To display.

  Here, the above information providing system, the printed matter, a two-dimensional code that is a two-dimensional code of a character string representing an identification code of an object whose information is described in the printed matter, the printed thing, The AR content acquiring means decodes the two-dimensional code printed on the printed matter captured by the camera, and associates the AR content with the AR content corresponding to the target indicated by the identification code represented by the decoded character string and the AR content. AR content information including an identification character string may be obtained from the AR content server as the AR content information.

In the above information providing system, the printed matter may be a business card, the object may be a person, and the identification character string may be a name of a person whose identification character string indicates identification.
In order to achieve the above object, the present invention provides an information providing system for performing AR display in which AR content corresponding to an object whose information is described in a printed matter is combined with an image of the printed matter and displayed. The present invention also provides an information providing system including a terminal device and an AR content server that stores AR content provided for each target object in association with the target object in association with the identification code of the target object. Here, the terminal device has a camera, an AR content acquisition unit, and an AR display unit, and the printed matter includes a character string representing an identification code of an object whose information is described in the printed matter. The two-dimensional code converted into a two-dimensional code is printed, and the AR content acquisition unit decodes the two-dimensional code printed on the printed matter taken by the camera, and associates the two-dimensional code with the identification code represented by the decoded character string. The information including the stored AR content and the identification code associated with the AR content is obtained from the AR content server as AR content information, and the AR display unit obtains the information from a printed image taken by the camera. The two-dimensional code printed on the printed matter is decoded, the identification code represented by the decoded character string is identified, and the identified identification code is used as the AR code. When the content matches the identification code included in the AR content information obtained by the content obtaining means, the AR content included in the AR content information obtained by the AR content obtaining means is synthesized and displayed on an image of a print taken by the camera. I do.

  In order to achieve the above object, the present invention provides an information providing system for performing AR display in which AR content corresponding to an object whose information is described in a printed matter is combined with an image of the printed matter and displayed. The present invention also provides an information providing system including a terminal device and an AR content server that stores AR content provided for each target object in association with the target object in association with the identification code of the target object. Here, the terminal device has a camera, an AR content acquisition unit, an AR display unit, and a storage device, and the printed matter represents an identification code of an object whose information is described in the printed matter. A two-dimensional code obtained by converting a character string into a two-dimensional code is printed, and the AR content acquisition unit decodes the two-dimensional code printed on a printed matter taken by the camera, and an identification code represented by the decoded character string. The information including the AR content stored in association with the AR content and the identification code associated with the AR content is acquired as the AR content information from the AR content server, stored in the storage device, and displayed in the AR display. The means decodes a two-dimensional code printed on the printed matter from the image of the printed matter taken by the camera, and recognizes an identification code represented by the decoded character string. Separately, when the identified identification code matches the identification code included in any of the AR content information stored in the storage device, the AR content information including the identification code matching the recognized identification code is used. The included AR content is combined with an image of a print taken by the camera and displayed.

Here, in these information providing systems, the printed matter may be a business card, and the object may be a person.
Here, in the above information providing system, a still image or a moving image can be used as the AR content.
The information providing system as described above recognizes the identification character string / identification code of the object represented by the identification character string / two-dimensional code of the photographed print, and includes the recognized identification character string / identification code and the acquired AR content information. Only when the identification character string / identification code of the target object matches, the image of the AR content included in the acquired AR content information is synthesized and displayed on the photographed image of the printed matter.

  Therefore, according to the present invention, it is possible to display the AR content by synthesizing it only with the photographed image of the printed matter of the corresponding object correctly. Therefore, when the object is a person and the printed matter is a business card, the AR content corresponding to the person whose personal information is described in the business card is synthesized only with the captured image of the business card of the corresponding person. Can be displayed. When the printed matter is a business card, the name printed on the business card can be used as the identification character string.

  As described above, according to the present invention, an AR display for synthesizing and displaying AR content corresponding to the person of the business card on the image of the business card is performed, and in the AR display, the business card of the person who correctly corresponds to the AR content is displayed. Can be synthesized and displayed only on the captured image.

It is a block diagram showing composition of an information service system concerning an embodiment of the present invention. FIG. 1 is a block diagram illustrating a configuration of a smart device according to an embodiment of the present invention. FIG. 3 is a diagram showing the contents of an AR content database and an AR content data set according to the embodiment of the present invention. It is a figure showing a business card concerning an embodiment of the present invention. 6 is a flowchart illustrating an AR content acquisition process according to the embodiment of the present invention. It is a flowchart which shows AR display processing which concerns on embodiment of this invention. It is a figure showing the example of AR display concerning the embodiment of the present invention. 9 is a flowchart illustrating another example of the AR display process according to the embodiment of the present invention. 9 is a flowchart illustrating another example of the AR display process according to the embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described.
FIG. 1 shows a configuration of an information providing system according to the present embodiment.
1, the information providing system includes an AR service system 1, an APP download service system 2, and a smart device 3.
The AR service system 1 and the APP download service system 2 are connected to a WAN 4 such as the Internet, and the smart device 3 is connected to the WAN 4 via a mobile communication network or a wireless LAN to connect the AR service system 1 and the APP download service system 2. Can be accessed.

Here, the smart device 3 is, for example, a smartphone or a tablet device.
Further, the AR service system 1 responds to a request from the Web server 11, the AR content database 12 storing the AR content, and the smart device 3 to store the AR content stored in the AR content database 12 in the smart device 3. And an AR content server 13 for transferring the content to the AR content server 13.

  Next, the APP download service system 2 converts the application data stored in the APP database 21 in response to a request from the smart device 3 with the APP database 21 storing the application data of various applications for the smart device. An APP download server 22 for transferring the data to the device 3 is provided. The application data is a package of an application program executed by the smart device 3 and data used by the application program.

Here, FIG. 2 shows a configuration of the smart device 3.
As illustrated, the smart device 3 includes an operation unit 301, a display 302, a voice input / output unit 303 including a microphone and a speaker, a wireless interface 304 connected to a wireless LAN via wireless communication, and a mobile device via mobile communication. The mobile communication device 305 includes a mobile communication device 305 connected to a communication network, a storage device 306, a camera 307, other peripheral devices 308, an operating system 309 of the smart device 3, and a plurality of applications running on the operating system 309.

  Here, the smart device 3 includes, as applications, a mobile phone 305, a telephone 311 that is an application that provides a mobile phone function using the voice input / output unit 303, and the operation unit 301, and an application for browsing a web page. , An APP downloader 313 that is an application that downloads an application and installs it on the smart device 3, an AR client 314 that is an application that performs AR display, and the like.

  Here, the application data of the AR client 314 is stored in the APP database 21 of the APP download service system 2. The AR client 314 of each smart device 3 is downloaded by the APP downloader 313 from the APP download service system 2 and installed in the smart device 3.

  Note that the smart device 3 is configured using a computer having a microprocessor, a memory, and other peripheral devices as hardware, and the operating system 309 and each application are executed by a microprocessor having a predetermined configuration. This is realized by executing a program.

Next, the contents of the AR content database 12 of the AR service system 1 are shown in FIG.
As shown in FIG. 3a1, the AR content database 12 includes a personal management database 121 and a media database 122.
The personal management database 121 stores a personal management table provided for each registered user who is a user registered in the AR service system 1. The media database 122 stores the AR service as AR content. A media file which is a file of an image such as an image or a moving image provided by the system 1 is stored.

  Next, as shown in FIG. 3a2, the personal management table stores personal IDs, personal data, business card data, front side content file pointers, back side content file pointers, etc., which are identifiers of the corresponding registered users. .

  Then, in the personal data, for each of the Japanese and English notations, personal information such as the name, name, company, department, post, address, telephone, and mail of the corresponding registered user is registered.

  In the business card data, the layout type of the business card of the corresponding registered user and the two-dimensional code printed on the business card of the corresponding user are registered for each of the front and back sides of the business card. . Here, as the two-dimensional code, for example, a QR code (registered trademark) can be used.

  The front side content file pointer includes a still image or a moving image stored in the media database 122 and used for AR display of the front surface of the corresponding registered user's business card (combined with the front surface image). Pointer information indicating a media file of an image such as an image is registered. Here, as the media file indicated by the front side content file pointer, for example, an image file of a photograph of the corresponding registered user is used.

  The backside content file pointer includes an image such as a still image or a moving image stored in the media database 122 and used for AR display of the back side of the corresponding registered user's business card (combined with the front side image). Pointer information indicating the media file is registered. Here, as the media file indicated by the backside content file pointer, for example, a moving image file of a moving image for PR of the company of the corresponding registered user is used.

Next, FIG. 4 shows an example of a business card of a registered user.
FIG. 4A shows the front surface of the business card, and FIG. 4B shows the back surface of the business card.
Here, the business card of each registered user of the AR service system 1 is registered in the AR service system 1 according to the layout type registered in the personal management table of the registered user. Each item of personal data registered in the table is laid out. The business card of each registered user of the AR service system 1 is printed according to the layout type and personal data registered in the personal management table of the registered user registered in the AR service system 1 and distributed to each registered user. May be used.

  In the illustrated example, the front side of the business card is a business card written in Japanese, and the back side of the business card is a business card written in English. A two-dimensional code 401, which is a two-dimensional code, is printed on the front surface of the business card.

  Here, as shown in FIG. 4C, the two-dimensional code 401 printed on the business card is a character string indicating that the business card is a business card of a registered user of the information providing system and a business card corresponding to the AR service system 1 ( card_ARservice) and a character string (pid = 541ahdi744) indicating the personal ID (541ahdi744) of the registered user of the business card.

Hereinafter, an operation of performing an AR display of a registered user's business card using the AR client 314 in the smart device 3 will be described.
In this case, the receiving user who is the user of the smart device 3 that has received the business card of the registered user of the AR service system 1 first activates the AR client 314, and shoots the two-dimensional code printed on the business card with the camera 307. .

On the other hand, the activated AR client 314 executes the AR content acquisition processing shown in FIG.
As illustrated, the AR client 314 starts a process of acquiring an image captured by the camera 307 and displaying it on the display 302 in the AR content acquisition process (step 502). Thereafter, the operation of recognizing the two-dimensional code in the image captured by the camera 307 (step 504) is repeated until the recognition is successful (step 506).

  If the recognition of the two-dimensional code is successful, the character string represented by the two-dimensional code is decoded, and the personal ID in the decoded character string is extracted (step 508). Here, in step 508, it is determined whether the character string obtained by decoding the two-dimensional code includes a character string (card_ARservice) indicating that the business card described above is a business card corresponding to the AR service system 1. In this case, the personal ID (541ahdi744) is identified and extracted from the character string (pid = 541ahdi744) indicating the personal ID included in the character string obtained by decoding the two-dimensional code, and the extraction of the personal ID is determined to be successful. . On the other hand, if the character string obtained by decoding the two-dimensional code does not include a character string (card_ARservice) indicating that the business card described above is a business card corresponding to the AR service system 1, the two-dimensional code indicates information provision. Since it is not the two-dimensional code printed on the business card of the registered user of the system, it is determined that the extraction of the personal ID has failed.

If the extraction of the personal ID fails (step 510), the process returns to step 504.
On the other hand, if the extraction of the personal ID has succeeded (step 510), the AR client 314 transmits a transfer request for the AR content of the extracted personal ID to the AR content server 13 of the AR service system 1 (step 512).

  On the other hand, the AR content server 13 that has received the AR content transfer request reads the personal management table in which the personal ID for which the AR content has been requested is registered from the personal management database 121, and reads the front side content file of the read personal management table. The media file indicated by the pointer is read from the media database 122 as a front side content file, and the media file indicated by the back side content file pointer of the read personal management table is read from the media database 122 as a back side content file. Also, personal management data including the personal ID and the personal data of the read personal management table is created.

  Then, as shown in FIG. 3B, an AR content data set including the created personal management data, the front side content file, and the back side content file is created and transmitted to the AR client 314.

  When the AR client 314 receives the AR content data set from the AR content server 13 (step 514), the AR client 314 stores the received AR content data set in the storage device 306 (step 516).

Then, the AR display process is started (step 518), and the AR content acquisition process ends.
The AR content acquisition processing performed by the AR client 314 has been described above.
Next, the AR display process started in step 518 of the AR content acquisition process will be described.
The AR display process is a process performed by the AR client 314. As shown in FIG. 6, in the AR display process, the AR client 314 first starts a process of acquiring a captured image of the camera 307 and displaying the captured image on the display 302. (Step 602). Thereafter, an OCR (optical character recognition) operation for recognizing a character string in an image captured by the camera 307 (step 604), an end of AR display is instructed by the user (step 606), or an OCR operation is performed. Until the recognition of the character string by is successful (step 608).

Then, when the user instructs the end of the AR display (step 606), the AR display process ends.
On the other hand, if the recognition of the character string is successful (step 608), the following processing is performed.
That is, if the recognized character string contains a character string that matches the front name of the personal management data in the stored AR content data set (step 610), Using an image pattern recognizing a character string that matches the name of the user as a marker, calculating the image portion of the business card in the photographed image from the size, orientation, and position of the image pattern, and calculating the relative position of the calculated business card image. The size, orientation, and arrangement of the images of the front side content file are set so that the position, relative orientation, and relative size become the predetermined relative position, relative orientation, and relative size. After the adjustment, the image is synthesized and displayed on the image of the business card (step 612).

Then, the processing returns to the processing from step 604.
Here, in step 612, for example, while the photographed image 701 of the front surface of the business card photographed as shown in FIG. If the character string that matches the name is successfully recognized, the character string is displayed according to the size, orientation, and position of the image pattern 70 that recognizes the character string that matches the front face name, as shown in FIG. 7A2. The image 702 of the front side content file is placed on the image 701 of the front side of the business card, and the two-dimensional code in the image 702 is set in a direction that matches the direction of the business card and that does not protrude from the image of the business card. Are displayed so as to be displayed at a position overlapping with. More specifically, for example, the up, down, left, and right directions of the image portion 701 of the business card in the captured image are obtained as directions corresponding to the up, down, left, and right of the character of the image pattern 70 in which the character is recognized. The size of the image portion is determined according to the relationship between the business card size and the print size of the name on the business card from the size of the portion corresponding to one character or a plurality of characters in the image pattern 70 in which the character of the name is recognized. The position of the image portion of the business card in the image is obtained from the position of the image pattern 70 that recognizes the first character of the name according to the layout of the name on the business card.

  On the other hand, if the recognized character string includes a character string that matches the back name of the personal management data in the saved AR content data set (step 614), the character that matches the back name is used. Using the image pattern that recognized the row as a marker, calculating the image portion of the business card in the captured image from the size, orientation, and position of the image pattern, and calculating the relative position and relative orientation to the calculated business card image By adjusting the size, orientation, and arrangement of the image of the backside content file so that the relative size becomes a predetermined relative position, relative orientation, and relative size, Are combined and displayed on the business card image (step 616).

Then, the processing returns to the processing from step 604.
Here, in step 616, for example, while the captured image 711 of the back side of the business card captured as shown in FIG. 7b1 is displayed on the display 302, a character string matching the back side name is obtained from the captured image of the camera 307. If the recognition is successful, as shown in FIG. 7B2, according to the size, orientation, and position of the image pattern 70 in which the character string matching the name on the back side is recognized, the image 711 on the back side of the displayed business card is displayed. Then, the image 712 of the backside content file is synthesized and displayed so that the image 712 is displayed on the entire back side of the business card in the captured image in a direction that matches the direction of the business card in the captured image.

  In addition, a character string that matches the front name of the personal management data in the saved AR content data set and a character string that matches the back name of the personal management data are also included in the recognized character strings. If not (step 614), if the above composite display is being performed at that point, the composite display is canceled and only the image captured by the camera 307 is displayed ( Step 618), and return to the processing from step 604.

  By the way, in the operation of the AR display processing as described above, in step 606, when the success of the character string recognition is not detected, that is, when the recognition failure occurs, If the combined display is being performed, the combined display is canceled so that only the image captured by the camera 307 is displayed (step 620).

The AR display processing performed by the AR client 314 has been described above.
The embodiments of the present invention have been described above.
As described above, in the present embodiment, the name of the photographed business card is recognized by the OCR operation, and only when the recognized name matches the name registered in the acquired AR content data set, the acquired AR content is acquired. The image of the AR content (front side content file or back side content file) of the data set is synthesized and displayed with the captured image of the business card.

Therefore, according to the present embodiment, the AR content can be correctly combined with the captured image of the business card of the corresponding person and displayed.
In the above embodiment, the AR display process shown in FIG. 8 may be performed instead of the AR display process shown in FIG.
As illustrated, in the AR display process illustrated in FIG. 8, the AR client 314 starts a process of acquiring a captured image of the camera 307 and displaying the captured image on the display 302 (step 802). Thereafter, an operation of recognizing a two-dimensional code in an image captured by the camera 307 is performed (step 804), the end of AR display is instructed by the user (step 806), or the recognition of the two-dimensional code is successful. (Step 808).

Then, when the user instructs the end of the AR display (step 806), the AR display process ends.
On the other hand, if the recognition of the two-dimensional code is successful (step 808), the following processing is performed.
That is, if the personal ID identified from the character string obtained by decoding the recognized two-dimensional code matches the personal ID of the personal management data in the stored AR content data set (step 810), the captured image is displayed. Using the two-dimensional code as a marker, from the size, orientation and position of the two-dimensional code, calculate the image portion of the business card in the captured image, relative to the calculated business card image, relative position, relative orientation, Adjust the size, orientation and arrangement of the image of the front side content file so that the relative size becomes a predetermined relative position, relative direction and relative size, and The image is synthesized and displayed on the image of the business card (step 812). However, in step 810, if the character string obtained by decoding the recognized two-dimensional code does not include a character string (card_ARservice) indicating that the business card is a business card corresponding to the AR service system 1, the personal ID is identified. Failure.

Then, the processing returns to the processing from step 804.
On the other hand, when the personal ID identified from the character string obtained by decoding the recognized two-dimensional code does not match the personal ID of the personal management data in the stored AR content data set, or when the identification of the personal ID fails. (Step 814), if the composite display as described above is being performed at that time, the composite display is canceled and only the image captured by the camera 307 is displayed (step 814). , And returns to the processing from step 804.

  In the operation of the AR display processing as described above, even if the recognition of the two-dimensional code is not detected in step 806, if the above-described composite display is performed at that time, the composite The display is canceled so that only the image captured by the camera 307 is displayed (step 814).

  According to such AR display processing, the two-dimensional code printed on the photographed business card is recognized, and the AR content data set corresponding to the person ID represented by the two-dimensional code is acquired, and the AR content data set acquired afterward is acquired. Only when the two-dimensional code printed on the business card matches the personal ID registered in the acquired AR content dataset, the AR content (front side content file or back side content file) of the acquired AR content dataset Is superimposed on the photographed image of the business card.

Therefore, even with the AR display processing shown in FIG. 8, the AR content can be correctly synthesized and displayed only with the captured image of the business card of the corresponding person.
In the AR display processing shown in FIG. 8, the AR display is performed in which the image of the front side content file is combined with the photographed image of the business card only on the front surface on which the two-dimensional code of the business card is printed. In this case, the AR display for synthesizing the captured image of the business card with the image of the backside content file may be performed on the back side of the business card by the same processing as the AR display processing shown in FIG.

  Here, in the above-described embodiment, at each time point, the AR content data set acquired by the AR client 314 in the above-described AR content acquisition processing is stored in the storage device 306, and the AR display processing is performed by the user's AR display start instruction operation The AR client 314 may be configured so as to be executed at any time according to the above, and then the AR display processing shown in FIG. 9 may be performed as the AR display processing.

  As shown in the figure, in the AR display processing shown in FIG. 9, the AR client 314 starts processing of acquiring an image captured by the camera 307 and displaying it on the display 302 (step 902). Thereafter, an OCR (optical character recognition) operation for recognizing a character string in the image captured by the camera 307 is performed (step 904), and the end of the AR display is instructed by the user (step 906). Is performed until the recognition of the data is successful (step 908).

Then, when the user instructs the end of the AR display (step 906), the AR display process ends.
On the other hand, if the recognition of the character string is successful (step 908), the following processing is performed.
In other words, if the recognized character string includes a character string that matches the front face name of the personal management data in any of the AR content data sets stored in the storage device 306 ( Step 910) The relative position, relative orientation, and relative size with respect to the business card image are determined in advance using the image pattern in which the character string matching the front face name is recognized as a marker. AR content data set containing management data in which the name of the front surface that matches the character string in the recognized character string is registered so that the relative position, relative orientation, and relative size The size, orientation, and arrangement position of the image of the inside front side content file are adjusted, and the image is synthesized and displayed on the business card image (step 912).

Then, the process returns to step 904.
On the other hand, if the character string that matches the name on the back of the personal management data in any of the AR content data sets stored in the storage device 306 is included in the recognized character string (step 914). Using an image pattern that recognizes a character string that matches the name on the back side as a marker, the relative position, relative orientation, and relative size to the business card image are predetermined relative positions, The image of the backside content file image in the AR content data set that contains the management data in which the name of the back side that matches the character string in the recognized character string is registered so that it has a relative orientation and relative size The size, direction, and arrangement position are adjusted, and the image is synthesized and displayed on the business card image (step 916).

  In addition, a character string that matches the front name of the personal management data in any of the AR content data sets stored in the storage device 306 is also stored in the storage device 306. If the character string that matches the name on the back of the personal management data in the content data set is not included in the recognized character string (step 914), the composite display as described above is performed at that time. If yes, the composite display is canceled, only the image captured by the camera 307 is displayed (step 918), and the process returns to step 904.

  By the way, in the operation of the AR display processing as described above, in step 908, if the success of the character string recognition is not detected, that is, if the recognition failure occurs, If the composite display is being performed, the composite display is canceled so that only the image captured by the camera 307 is displayed (step 920).

  By doing so, when a business card is photographed, the AR content (front side content file or back-side content file) of the stored AR content data set corresponding to the registered user of the photographed business card is selected. Since the image of the side content file) is superimposed on the captured image of the business card, the receiving user can use the AR display of the business card simply by photographing the registered user's business card at any time. Become like

  Note that the AR client 314 replaces the AR display process shown in FIG. 9 with the steps 810 and 812 of the AR display process shown in FIG. 8 in one of the AR content data sets stored in the storage device 306. It is determined whether the personal ID is indicated by a character string obtained by decoding the recognized two-dimensional code. If the personal ID is not indicated, the process proceeds to step 814. If the personal ID is indicated, the personal ID indicated by the recognized character string is indicated. Of the front side content file in the AR content data set that contains the registered management data, adjusts the size, orientation, and arrangement position of the image, and replaces the image with the process of combining and displaying the image on the business card image. It may be performed.

Even in this case, the receiving user can use the AR display of the registered card simply by photographing the registered user's business card at any time.
In the above embodiments, the AR content may be audio or a combination of audio and image. In this case, in the AR display process, a process of reproducing and outputting the audio included in the AR content data set corresponding to the registered user of the photographed business card is performed.

  In addition, the above embodiments are also applicable to printed materials other than business cards, such as catalogs, brochures, company information, posters, magazines and books, hanging advertisements for trains, signs, printed cars and airplane bodies and airframes, and the like. Can be applied.

  That is, in this case, assuming that the user is replaced with the object whose information represents the printed matter, the name is replaced with a character string representing the identification of the object, and the two-dimensional code is represented as the identification of the object, The embodiment of the invention may be applied. As a character string representing the identification of the object, for example, if the printed matter is a product catalog or pamphlet, a character string of the product name or model number of the product is used. Is used.

  DESCRIPTION OF SYMBOLS 1 ... AR service system, 2 ... APP download service system, 3 ... Smart device, 4 ... WAN, 11 ... Web server, 12 ... AR content database, 13 ... AR content server, 21 ... APP database, 22 ... APP download server, 70 image pattern, 121 personal management database, 122 media database, 301 operating unit, 302 display, 303 audio input / output unit, 304 wireless interface, 305 mobile communication device, 306 storage device, 307 Camera, 308 peripheral device, 309 operating system, 311 telephone, 312 web browser, 313 APP downloader, 314 AR client.

Claims (7)

An information providing system for performing AR display in which AR content corresponding to an object whose information is described in a printed matter is combined with an image of the printed matter and displayed,
A terminal device;
An AR content server that stores AR content provided for each target object in association with the target object in association with an identification character string that is a character string representing the identification of the target object;
The terminal device has a camera, an AR content acquisition unit, and an AR display unit,
The AR content obtaining means includes, as AR content information, information including an AR content corresponding to a target whose information is described in a printed matter taken by the camera and an identification character string associated with the AR content. Get from AR content server,
The AR display unit recognizes an identification character string described in the print from an image of the print taken by the camera, and the recognized identification character string is included in the AR content information acquired by the AR content acquisition unit. An information providing system, wherein when the identification character string matches, an AR content included in the AR content information acquired by the AR content acquisition unit is combined with an image of a printed matter taken by the camera and displayed. An information providing system for performing AR display in which AR content corresponding to an object whose information is described in a printed matter is combined with an image of the printed matter and displayed,
A terminal device;
An AR content server that stores AR content provided for each target object in association with the target object in association with an identification character string that is a character string representing the identification of the target object;
The terminal device has a camera, an AR content acquisition unit, an AR display unit, and a storage device,
The AR content obtaining means includes, as AR content information, information including an AR content corresponding to a target whose information is described in a printed matter taken by the camera and an identification character string associated with the AR content. Obtained from the AR content server, stored in the storage device,
The AR display means recognizes an identification character string described in the printed matter from an image of the printed matter taken by the camera, and the recognized identification character string is any of AR content information stored in the storage device. In the case of matching with the identification character string included in the above, the AR content included in the AR content information including the identification character string matching the recognized identification character string is combined with the image of the print taken by the camera and displayed. An information providing system characterized by the following. The information providing system according to claim 1 or 2,
On the printed matter, a two-dimensional code obtained by converting a character string representing an identification code of an object whose information is described in the printed matter into a two-dimensional code is printed,
The AR content acquisition unit decodes a two-dimensional code printed on a printed matter taken by the camera, and associates the AR content with the AR content corresponding to the target indicated by the identification code represented by the decoded character string and the AR content. An information providing system, comprising: acquiring, from the AR content server, AR content information that includes an identification character string included therein. The information providing system according to claim 1, 2, or 3,
The information providing system, wherein the printed matter is a business card, the object is a person, and the identification character string is a name of a person whose identification character string indicates identification. The information providing system according to claim 1, 2, 3, or 4,
  The information providing system, wherein the AR content is an image that is a still image or a moving image. A computer program read and executed by a computer having a camera,
Said computer,
Accessing an AR content server that stores AR content provided for each target object in association with an identification character string, which is a character string representing the identification of the target object, and prints taken by the camera. AR content corresponding to the object whose information is described in
AR content acquisition means for acquiring information including an identification character string associated with the content as AR content information from the AR content server,
The camera recognizes an identification character string described in the printed matter from the image of the printed matter taken by the camera, and the recognized identification character string matches the identification character string included in the AR content information acquired by the AR content acquisition unit. In this case, the computer program causes the AR content included in the AR content information acquired by the AR content acquisition unit to function as an AR display unit that combines the AR content with an image of a printed matter taken by the camera and displays the image. A computer program according to claim 6, wherein
A computer program, wherein the printed matter is a business card, the object is a person, and the identification character string is a name of a person whose identification character string indicates identification.