WO2018135272A1

WO2018135272A1 - Information processing device, display method, program, and computer-readable recording medium

Info

Publication number: WO2018135272A1
Application number: PCT/JP2017/046923
Authority: WO
Inventors: 真人柴田; 昌敏竹谷
Original assignee: 合同会社ＩＰＢｒｉｄｇｅ１号
Priority date: 2017-01-18
Filing date: 2017-12-27
Publication date: 2018-07-26

Abstract

An information processing device provided with: a recognition unit for recognizing a code included in an image; a position acquisition unit for acquiring a user-selected position in the image; and a control unit for performing control so as to execute the content of the code when the code is located at the in-image position acquired by the position acquisition unit.

Description

Information processing apparatus, display method, program, and computer-readable storage medium

The present invention relates to an information processing apparatus, a display method, a program, and a computer-readable storage medium.

In recent years, two-dimensional codes that can hold more information than barcodes are known. Two-dimensional codes are often formed in black and white, but in order to hold more information, two-dimensional codes in which data areas are formed in colors such as red, green, and blue are also used (for example, Patent Document 1). However, when a two-dimensional code is embedded in an image, the two-dimensional code becomes conspicuous and the design of the image is impaired.

JP 2004-234318 A

On the other hand, outdoor advertising such as digital signage is increasing. Many of such digital signage are large and are installed at remote locations (for example, the rooftop of a building or a wall surface) where it is difficult for the user to make direct contact. In addition, when digital signage is installed in a car or in a store, it is often installed for the purpose of providing information to a large number of users.

* Displaying a 2D code or the like in the content displayed on such digital signage will cause the design to break. In addition, the content displayed on the display is often intended to be provided for a large number of people. In such a case, even if an individual user is interested in the content, it is difficult to induce search behavior by the user's own terminal device.

Therefore, an object of the present invention is to make it possible to operate content provided to a large number of people in a terminal device owned by each user.

An image acquisition unit that acquires an image, a recognition unit that recognizes a code included in the image, a position acquisition unit that acquires a position in the image selected by a user, and the image acquired by the position acquisition unit An information processing apparatus comprising: a control unit configured to perform control so that the content held by the code is executed when the code is located at a middle position.

Further features and aspects of the present invention will become apparent from the detailed description of embodiments set forth below, taken in conjunction with the accompanying drawings.

According to the present invention, content provided to a large number of people can be operated on a terminal device owned by each user.

1 is a diagram illustrating an overall configuration of an information processing system according to a first embodiment. The block diagram which shows the content server 10, the portable terminal device 20, and the server 30 which concern on 1st Embodiment. The figure which shows an example of the code image 60 with which the invisible code which concerns on 1st Embodiment was embedded. 5 is a flowchart showing a process for generating a code image 60 according to the first embodiment. 5 is a flowchart showing a process for generating a code image 60 according to the first embodiment. The image figure for demonstrating operation | movement of the system which concerns on 1st Embodiment. The image figure for demonstrating operation | movement of the system which concerns on 1st Embodiment. The image figure for demonstrating operation | movement of the system which concerns on 1st Embodiment. The image figure for demonstrating operation | movement of the system which concerns on 1st Embodiment. The image figure for demonstrating operation | movement of the system which concerns on 1st Embodiment. The flowchart for demonstrating the process regarding the live view of the portable terminal device which concerns on 1st Embodiment. The image figure for demonstrating the multilingual guide map which concerns on 2nd Embodiment. The image figure for demonstrating the multilingual guide map which concerns on 2nd Embodiment. The image figure for demonstrating the multilingual guide map which concerns on 2nd Embodiment. The flowchart for demonstrating the process regarding the switching of the language display method by the form information terminal side which concerns on 2nd Embodiment.

Hereinafter, an information processing apparatus, a display method, a reading method, and a computer-readable non-transitory storage medium according to embodiments will be described with reference to the drawings.

(First embodiment)
<Overall configuration of information processing system>
FIG. 1 is a diagram illustrating an overall configuration of an information processing system according to the first embodiment. The information processing system includes a content server 10, a mobile terminal device 20, and a server 30. The content server 10, the mobile terminal device 20, and the server 30 are connected via a network N so as to communicate with each other.

The network N includes, for example, a part or all of a WAN (Wide Area Network), a LAN (Local Area Network), the Internet, a provider device, a wireless base station, a dedicated line, and the like. The mobile terminal device 20 is, for example, a tablet computer or a smartphone. The mobile terminal device 20 communicates with the content server 10 and the server 30 via the network N by performing wireless communication W.

The content server 10 is connected to a display D that displays content and the like. Here, the display D is a so-called digital signage. Digital signage is an electronic signboard for displaying advertisements and the like on the relatively large display to the public. The digital signage itself may be provided with a storage device or the like that temporarily holds content. Therefore, the content server 10 and the display D may be integrated as digital signage.

For convenience, a computer that requests information is called a client, and a computer that sends information in response to a request for information is called a server. In the present embodiment, the content server 10 and the server 30 function as a server, and the mobile terminal device 20 mainly functions as a client.

FIG. 2 is a block diagram showing the content server 10, the mobile terminal device 20, and the server 30 according to the first embodiment. The content server 10 includes a CPU (Central Processing Unit) 11, a memory 12, a bus 13, and a storage 14.

The CPU 11 as an information processing unit and the memory 12 as an information holding unit are connected to each other via a bus 13. The CPU 11 accesses information stored in the memory 12 via the bus 13 according to a program. The memory 12 is a RAM (Random Access Memory), but is not limited thereto. For example, the memory 12 may be another type of memory such as a flash memory. The memory 12 temporarily stores information processed by the CPU 11.

CPU 11 reads and writes various information to and from storage 14. The storage 14 is, for example, an HDD (Hard Disk Drive), but is not limited to this. For example, the storage 14 may be an external device accessible by the content server 10 such as a flash memory, NAS (Network Attached Storage), or an external storage server.

The content server 10 may store information in the storage 14 instead of the memory 12. Further, alternatives such as a memory and a storage may be used according to a change in the architecture of the content server 10.

The mobile terminal device 20 includes a CPU 21 as a control unit, a memory 22, a bus 23, a camera 24, and a display 25 (see FIG. 6 and the like). The display 25 has a function of detecting and acquiring selection by a user such as a touch panel. Further, the CPU 21 serving as a position acquisition unit acquires information on the position (coordinates) selected by the user from a touch panel or the like. The CPU 21 accesses information stored in the memory 22 via the bus 23 according to a program. The memory 22 is a RAM, but is not limited to this. For example, the memory 22 may be another type of memory such as a flash memory. The memory 22 temporarily stores information processed by the CPU 11.

The image read by the camera 24 can be processed by the CPU 21 via the bus 23. Further, the camera 24 can display the image on the display 25 in order to present the captured information to the user even when there is no recording instruction (shutter) (so-called live view function). Although details will be described later, the CPU 21 performs image processing on the image acquired by the camera 24 and displays the result of the image processing on the display 25. In the case of expressing as an image, it may be interpreted as including a moving image as well as a still image. An apparatus in which a camera and a display are configured as one device may be interpreted as an information processing apparatus, or a system configured by linking a device having a camera function and a plurality of devices having a display function as information. It may be interpreted as a processing device.

The server 30 includes a CPU 31, a memory 32, a bus 33, and a storage 34. The CPU 31 accesses information stored in the memory 32 via the bus 33 according to the program. The memory 32 is a RAM, but is not limited to this. For example, the memory 32 may be another type of memory such as a flash memory. The memory 32 temporarily stores information processed by the CPU 31.

CPU 31 reads and writes various information to and from storage 34. The storage 34 is, for example, an HDD, but is not limited to this. For example, the storage 34 may be an external device accessible by the server 30 such as a NAS or an external storage server.

Note that, similarly to the mobile terminal device 20, a camera that captures an image may be connected to the content server 10 and the server 30. In addition, the server 30 may include a display, like the content server 10 and the mobile terminal device 20.

<Invisible code>
In general, blinking of an image of less than 0.03 [seconds] is difficult to recognize with the naked eye. In other words, blinking of an image having a frequency exceeding 33 [Hz] is difficult to recognize with the naked eye. When an image blinks at a frequency exceeding 33 [Hz], it is difficult to recognize with the naked eye even when an image with a large difference in brightness is easily blinking. For this reason, if the difference between brightness and darkness is small, even if the blinking frequency is slightly lower than 33 [Hz], it is possible to create a state in which blinking of an image is difficult to be recognized with the naked eye. Such a state that is difficult to be recognized with the naked eye is referred to as “invisible” in the present embodiment.

FIG. 3 is a diagram illustrating an example of the code image 60 in which the invisible code according to the first embodiment is embedded. Hereinafter, a method for generating the code image 60 will be described with reference to FIG. The code image 60 is an image in which the first frame image 60a and the second frame image 60b are alternately switched. The code image 60 is generated by embedding the code information 50 in the background image 40 in an invisible state. Hereinafter, the code information 50 embedded in the background image 40 in the invisible state is referred to as “invisible code”.

The background image 40 is an arbitrary image designated by a creator (content provider) that generates content, but is not limited thereto. For example, the background image 40 may be an image displayed as an operation screen of application software. The code information 50 is described by taking a QR code (registered trademark) as an example, but is not limited thereto. For example, the code information 50 may be another two-dimensional code such as DataMatrix (registered trademark) or VeriCode (registered trademark), or may be a one-dimensional code such as a barcode. The code information 50 is information in which data designated by the user (for example, URL of a WEB page, account information of messenger software, account information of a social networking service) is coded.

The code information 50 includes a position detection marker 51, an attitude correction marker 52, and a data dot 53. The position detection marker 51 is a marker used to detect the position of the code information 50 and is arranged at three corners of the code information 50. The posture correction marker 52 is a marker used to correct the posture of the code information 50 read by the camera 24 as a reading unit. The data dot 53 is a dot for expressing information related to data held by the code.

Since the camera 24 as an image acquisition unit is provided in the mobile terminal device 20, there is a possibility that camera shake may occur in an image taken by the camera 24. For this reason, the data dot 53 includes an error correction code such as a Reed-Solomon code. In order to ensure robustness, an error correction code having an error correction rate of 15% or more is employed. It is preferable to employ an error correction code having an error correction rate of 30% or more.

The CPU 21 of the mobile terminal device 20 acquires the background image 40 and the code information 50. Based on the acquired background image 40 and code information 50, the CPU 21 generates a first frame image 60a and a second frame image 60b. For example, the CPU 21 generates the first frame image 60 a by reducing the brightness of the pixels of the background image 40 at the position overlapping the code information 50. Further, the CPU 21 generates the second frame image 60b by increasing the brightness of the pixel of the background image 40 at the position overlapping the code information 50. The CPU 21 generates, as the code image 60, an image in which the first frame image 60a and the second frame image 60b are alternately switched at intervals of less than 0.03 [seconds].

As described above, blinking of an image of less than 0.03 [seconds] is difficult to recognize with the naked eye. For this reason, when the first frame image 60a and the second frame image 60b are alternately switched at intervals of less than 0.03 [seconds], the average brightness of the first frame image 60a and the second frame image 60b is obtained. The lightness of the code image 60 is recognized by the naked eye. For this reason, even if the code information 50 is embedded in the background image 40, it is reduced that the designability of the background image 40 is impaired.

If the code information 50 is simply superimposed on the background image 40, the code information 50 becomes conspicuous and the design of the image is impaired. Therefore, the CPU 21 alternately switches the first frame image 60a and the second frame image 60b at intervals of less than 0.03 [seconds]. As a result, the code information 50 becomes invisible to the naked eye. In other words, it is difficult to distinguish between the background image 40 and the code image 60 with the naked eye even though the first frame image 60a and the second frame image 60b are alternately displayed. In digital signage that is often installed outdoors, the light ID method such as LinkRay (registered trademark) that causes the LED light source to blink at high speed is easily wiped out by sunlight. For this reason, it is preferable to employ an invisible code in digital signage. In FIG. 3, for the sake of explanation, the description is made using a black and white image. However, it is assumed that a color image is used as the background image, and a high recognition rate can be ensured even outdoors by changing the value in the hue direction instead of the brightness.

<Code image display processing>
FIG. 4 is a flowchart showing the display processing of the code image 60 according to the first embodiment. The display processing of the code image 60 to be displayed on the display D will be described in detail with reference to FIG. As for an image (moving image) in which an invisible code is embedded, an image (moving image) generated by a system other than the content server 10 may be stored in the server 10 and distributed to the display D.

The CPU 11 of the mobile terminal device 10 acquires data to be stored as an invisible code to be displayed on the digital signage (S101). Data to be stored as an invisible code displayed on the digital signage may be data designated by the user. For example, the data designated by the user includes a URL of a WEB server that can purchase a product displayed on the digital signage, messenger software account information, SNS account information, and the like. The CPU 11 generates code information 50 based on the acquired data (S102).

Next, the CPU 11 as the acquisition unit acquires the background image 40. For example, the CPU 11 acquires the background image 40 from the memory 12 or another server 30 (S103).

If the background image 40 contains many high-frequency components, it is difficult to extract invisible codes. Therefore, the CPU 11 calculates the frequency characteristics of the background image 40 by performing Fourier transform on the background image 40 (S104).

Thereafter, the CPU 11 removes a spatial frequency component equal to or greater than a predetermined value of the calculated background image 40 (S105). Note that the CPU 11 may hold both the background image 40 from which the high frequency component has been removed and the background image 40 from which the high frequency component has been removed, in the memory 22.

For example, the CPU 11 performs a filtering process on the background image 40 to remove components having a spatial frequency included in the background image 40 of a predetermined value or more. Specifically, the CPU 11 performs a filtering process on the coordinates (i, j) of the background image 40 based on the following equation (1). Here, Xi, j indicates a pixel value before filtering, and Yi, j indicates a pixel value after filtering.

If the background image 40 includes a component having a high spatial frequency, the background image 40 is susceptible to camera shake when the background image 40 is captured. For this reason, if the background image 40 is blurred, it becomes difficult to extract an invisible code. For this reason, the CPU 11 performs a filtering process on the background image 40 to remove components having a spatial frequency equal to or higher than a predetermined value. Thereby, the influence of camera shake can be reduced.

Thereafter, the CPU 11 performs a code image generation process to be described later using the code information 50 and the background image 40 (S106). In the code image generation process, the CPU 11 as the generation unit generates the code image 60 by embedding the code information 50 in the background image 40 in an invisible state. Although details will be described later, the CPU 11 embeds the code information 50 in the background image 40 in an invisible state by changing the pixel value of the background image 40 at a position overlapping the code information 50. The moving image in which the invisible code generated as described above is embedded is transmitted from the content server 10 to the display D so as to be reproducible. Thereby, the moving image with the invisible code embedded can be presented to the public by digital signage installed outdoors.

FIG. 5 is a flowchart showing a code image generation process according to the first embodiment. The flowchart shown in FIG. 5 shows the process of S106 of FIG. 4 in detail.

First, the CPU 11 analyzes the background image 40 and the code information 50 (S201). For example, the CPU 11 acquires the color data of the background image 40 at the position where the code information 50 is arranged, and acquires the RGB value of the acquired color data. The CPU 11 determines whether the code information 50 is suitable for embedding in the background image 40 based on the acquired RGB values. For example, when the majority of the color data of the background image 40 at the position where the code information 50 is arranged is not suitable for embedding the code information 50, the CPU 11 determines that the code information 50 is not suitable for embedding in the background image 40. judge. Here, in consideration of the application for display on the digital signage, it is not always required to increase the position accuracy for embedding the code. Therefore, the position where the code is embedded may be shifted in consideration of the background image 40.

The color data that is not suitable for embedding the code information 50 means a pixel value that cannot afford to change the color. Specifically, when the pixel value is expressed in 8-bit RGB, (255, 255, 255), (0, 0, 0), and the like correspond to color data that is not suitable for embedding the code information 50. When the saturation direction is set as the variation direction of the pixel value, the pixel value whose saturation is the limit value also corresponds to color data that is not suitable for embedding the code information 50.

When the pixel value is near the limit value of the expression range, it is difficult to ensure a sufficient fluctuation range for changing the pixel value. For this reason, when the pixel value is expressed in 8-bit RGB (256 gradations), the code information 50 is suitably used for the background image 40 if the sum of differences between the RGB values and the limit value (0 or 255) can be secured. Can be embedded.

When the CPU 11 determines that the code information 50 is not suitable for embedding in the background image 40, the CPU 11 regenerates the code information 50 in consideration of the background image 40 (S202). For example, the CPU 11 uses the margin bits of the error correction code of the code information 50 to change the arrangement of the data dots 53 while maintaining the identity of the information held by the data dots 53. As a result, the CPU 11 regenerates the code information 50. The CPU 11 repeats the generation of the code image 60 until the code information 50 can be suitably embedded in the background image 40.

Next, the CPU 11 determines the fluctuation direction and fluctuation width of the pixel value of the code information 50 (S203). In the present embodiment, the CPU 11 changes the pixel value to two complementary colors on the hue with the pixel value of the code information 50 as the center.

CPU21 acquires the image value of the background image 40 of the position which overlaps with the code information 50 (S204). The CPU 11 performs image processing on the acquired image value under the conditions determined in S203 (S205).

For example, the CPU 11 converts the pixel values (R, G, B) acquired in S204 into pixel values (H, S, V) in the HSV model. Here, H (Hue) represents a hue, S (Saturation Chroma) represents saturation, and V (Value Lightness Brightness) represents lightness. The CPU 11 converts the pixel values (R, G, B) into pixel values (H, S, V) based on the following formulas (2) to (4). Here, MAX indicates the maximum value of RGB, and MIN indicates the minimum value of RGB.

The CPU 11 performs color data A obtained by subtracting 5 from the pixel value in the saturation direction with respect to the converted pixel value (H, S, V), and color data B obtained by adding 5 to the pixel value in the saturation direction. Is generated. That is, the color data A is (H, S-5, V), and the color data B is (H, S + 5, V). In the present embodiment, the fluctuation range of the pixel value is set to 5, but the present invention is not limited to this.

The CPU 11 generates a first frame image and a second frame image by changing the pixel value of the background image 40 at a position overlapping the code information 50. The CPU 11 generates a code image 60 (moving image) in which an invisible code is embedded by alternately switching the generated first frame image and second frame image at intervals of less than 0.03 [seconds] ( S206). In other words, the CPU 11 generates the code image 60 (moving image) in which the invisible code is embedded by alternately switching the generated first frame image and second frame image at a frequency higher than 30 [Hz]. .

As described above, the CPU 11 generates the code image 60 in which the code information 50 is embedded in the invisible state in the background image 40 subjected to the filter processing. Specifically, the CPU 11 generates the code image 60 by changing the pixel value of the background image 40 at the position overlapping with the code information 50 in a predetermined direction around the pixel value. As a result, it is possible to provide a link or the like while maintaining the design for the public outdoors while suppressing the mobile terminal on the photographing side from being difficult to read the invisible code due to camera shake.

In the present embodiment, RGB pixel values and HSV model pixel values are used. However, the present invention is not limited to this. For example, a pixel value in the HSL color space or a pixel value of L * a * b * (JIS Z 8730) may be used.

<Code image reading process>
FIG. 6 is a diagram for explaining a link when a mobile terminal device according to the first embodiment simultaneously displays a moving image in which an invisible code displayed on digital signage is embedded in live view (obtained display as needed). FIG. Conventionally, a code reader is designed to change a screen according to data held by a code when the code is recognized. For example, when a QR code reader recognizes a scanned QR code, the QR code reader performs screen transition in accordance with presentation of retained contents (for example, URL information) and stored data. As for the invisible code, an image that is alternately switched at a predetermined time interval is read by the camera 24, and a code that is invisiblely embedded between a plurality of frames is extracted by difference calculation. About the difference, even if it is the change in the brightness direction, the change in the saturation direction, and the change in the hue direction, it can be extracted by the difference calculation in the same manner. This extraction process is performed by the CPU 21 that processes the image data.

On the other hand, even if the invisible code is embedded in the digital signage, the mobile terminal device 20 including the camera 24 and the display 25 according to the present embodiment is immediately stored in the code when the code is recognized. The screen does not transition to etc. 6 to 9 are image diagrams for explaining an operation when the mobile terminal device 20 captures an image including the display D by the live view function. In each figure, a region C hatched in gray in the display D indicates a position where an invisible code is embedded. Similarly, a region C hatched in gray in the display 25 indicates a position corresponding to a region hatched in gray in the display D.

FIG. 6 is an image diagram for explaining a state in which an image in which an invisible code is embedded is presented on the entire display D of the digital signage. In FIG. 6, the mobile terminal device 20 shoots a part of a building including digital signage in the live view mode. The lower part of FIG. 6 shows an image displayed on the display 25 of the mobile terminal device 20 in the live view state in the situation as shown in the upper part.

The mobile terminal device 20 is in a state of taking an image including the display D in which the invisible code is embedded, but does not perform processing such as screen transition by interpreting the invisible code. However, the mobile terminal device 20 is instructed by the invisible code only when the user performs a selection operation such as a touch on the portion corresponding to the area where the invisible code of the display 25 displayed in the live view is embedded. Perform the action. As an example, URL information for guiding to a product purchase page may be embedded as an invisible code in an image in which a product advertisement is displayed so that it can be seen by human eyes on digital signage. In this case, if the user is guided to the product purchase page just by holding the smartphone in the live view state over the cityscape so as to cut out the scenery, a reaction unintended by the user is caused. Therefore, when the user intentionally selects a place where the invisible code is embedded, the screen transition is performed, thereby avoiding a situation of transition to a purchase page for a product that is not of interest to the user. Here, a smartphone has been described as an example, but screen transition processing may be performed when selection is performed by a selection unit such as a ring shape, line-of-sight recognition, glove shape, or brain interface as an alternative to a selection operation such as touch. Further, when the camera 24 that acquires an image and the display 25 are separated, the devices may cooperate to perform the same operation.

7A and 7B are image diagrams for explaining states in which the posture and zoom state taken by the camera 24 are different. In the state shown in FIG. 7A, an image including the entire digital signage display D is taken. On the other hand, in the state shown in FIG. 7B, only a part of the display D is photographed with the display D and the camera 24 not facing each other. In this case, by adopting random dots or the like as a code to be embedded as an invisible code, the amount of retained data is limited, but even if only a part of the invisible code is displayed on the display 25, the invisible code can be selected. Can be presented. The invisible code may be embedded in a part of the display D, or the invisible code may be embedded at a position that overlaps with an article (product) to be guided to the purchase page. Specifically, in order to make it easy to recognize a product with an invisible code embedded, a mark such as “.” May be added to indicate that it can be selected, or a leader line is displayed to display an advertisement page. Or it may be shown that it is possible to link to the purchase page.

6, 7 </ b> A, and 7 </ b> B, the operation when the invisible code is embedded in the entire display D has been described. On the other hand, FIG. 8 is an image diagram for explaining a form in which the invisible code embedded in the digital signage moves. In FIG. 8, it is assumed that the area C (the area where the invisible code is embedded) hatched in gray in the display D moves as indicated by a solid arrow. For example, the hatched area C is moved in accordance with the movement of the image related to the product that can be recognized by the person as indicated by the solid arrow. Thereby, it is possible to prevent transition to the purchase page unless the user intentionally selects the product itself.

FIG. 9 is an image diagram for explaining a state in which there are a plurality of digital signage, each of which has an area in which an invisible code is embedded. Each digital signage includes a first area C1 and a second area C2 hatched in gray. These multiple areas are moved as indicated by solid arrows in accordance with the contents displayed on the digital signage. In this case, if the user selects C1, processing corresponding to data held by the invisible code embedded in the C1 region is performed. Similarly, if the user selects C2, processing corresponding to the data held by the invisible code embedded in the area C2 is performed. Although FIG. 9 illustrates the case where selectable areas are arranged on each display, a plurality of selectable areas may be arranged on a single display.

Subsequently, the operation on the mobile terminal device 20 side will be described with reference to FIG. FIG. 10 is a flowchart for explaining the operation of the mobile terminal device 20. The portable terminal device 20 has a live view function, and acquires external image information from the camera 24 (S301). The CPU 21 acquires the image data acquired from the camera 24 and realizes the live view function by displaying the image data on the display 25. At this time, the CPU 21 as a recognition unit for recognizing the code analyzes the image data acquired from the camera 24, and analyzes (recognizes) whether the invisible code is embedded in the image data (S302).

If the invisible code is present in the image data as a result of the analysis, the CPU 21 identifies the position where the invisible code is embedded, and embeds data held by the invisible code as a link at the identified position (S303). . Thereafter, the CPU 21 acquires touch information indicating that the user has touched the display 25 displaying the live view from the display 25 including a touch panel or the like (S304). Specifically, when the user touches the position where the invisible code specified in S303 is embedded, the CPU 21 performs control based on the link embedded in the touched position. As described above, even if the invisible code is embedded in the image displayed on the display D, the CPU 21 does not change the screen when there is no user selection operation.

Therefore, the CPU 21 acquires touch information (user selection information) indicating that the area C in which the invisible code is embedded is touched, and executes the process of S306 when the user selects the area C (touches the embedded link). (S305: Yes). On the other hand, when the area C in which the invisible code is embedded is not selected (touched) by the user, the CPU 21 repeatedly executes the processing from S301 to S304 (S305: No).

When the user touches the embedded link, the CPU 21 transmits a request to the Web server or the like via the network in order to display the embedded link (S306). In the present embodiment, the CPU 21 performs the decoding process on the code in S303, but this operation may be performed immediately before S306. Thereby, additional information such as information about the advertisement and URL can be given to the moving image displayed on the facility such as digital signage. In other words, an interactive user experience can be provided by presenting content displayed on a public display without a touch panel function so as to be selectable on a smartphone or the like.

(Second Embodiment)
In the first embodiment, application to a terminal that displays an advertisement or the like has been described with emphasis. In this embodiment, an application mode for a transfer guide, a traffic sign, a guide map, and the like in a train that requires multilingual support will be described. In addition, about the location which overlaps with the part demonstrated in 1st Embodiment, description shall be abbreviate | omitted by attaching | subjecting the same code | symbol.
This embodiment improves the deterioration of convenience due to multilingual descriptions on the screen presenting highly public information or frequent switching of the display language.

FIG. 11 is a diagram for explaining a route map of the subway displayed by the digital signage arranged in the vehicle. As shown in FIG. 11A, information on routes is presented in many languages to cope with the recent increase in foreigners visiting Japan. However, the display area of the digital signage display is limited, and there is a limit to supporting multiple languages. For this reason, it is adopted to switch between a route map written in Japanese as shown in FIG. 11B and a route map written in English as shown in FIG. 11C at regular intervals. However, in the method of switching the display language every predetermined period, there is a problem in convenience because it is necessary to wait until the user displays in a readable language.

Therefore, as shown in FIG. 11B, an invisible code serving as a pointer regarding the operation status and route map is embedded in a part of the digital signage. The area C where the invisible code is embedded may be a part of the display or the entire surface.

The user's own mobile terminal individually holds language settings related to the user's regular language. Using the user's language setting, the CPU 21 controls to display an overlay on the image acquired by the camera 24. Thereby, the user can confirm the route map described in the language which a user can read with the own portable terminal device 20. FIG.

The operation of the mobile terminal device 20 of this embodiment will be described with reference to FIG. Since S401 to S404 are processes similar to S301 to S304, respectively, description thereof will be omitted. The CPU 21 analyzes data held by the invisible code. Then, the CPU 21 downloads the route map information described in the user-readable language via the network without waiting for the user's instruction (S405). Thereafter, the CPU 21 switches the display language in accordance with an instruction such as a user touch. If the language information that can be identified by the user cannot be acquired, the CPU 21 controls the display language to be switched every time the route map is touched on the live view screen (S407).

As described above, by using the system of the present embodiment, it is possible to provide means for individual users to acquire and convert information provided to a large number of people through digital signage based on their own setting information. it can. In other words, the content displayed on the public display can be customized and provided for the user using a simple method. Moreover, since an invisible code is used, it is not visually recognized by a user who is not interested in the information held in the code, so that the conventional landscape can be maintained.

In addition, the content server 10, the portable terminal device 20, and the server 30 in the said embodiment have a computer system inside. Each process of the content server 10, the mobile terminal device 20, and the server 30 described above is stored in a computer-readable recording medium in the form of a program. Then, the above-described various processes are performed by the computer reading and executing the stored program. Here, the computer-readable recording medium means a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, or the like. Alternatively, the computer program may be distributed to the computer via a communication line, and the computer that has received the distribution may execute the program.

Also, a part of processing executed on a specific computer may be shared by other computers. For this reason, the entire information processing system may be regarded as an information processing apparatus. Even if an element that performs characteristic information processing is intentionally installed outside the country, if the region that refers to the processing result is in Japan, the information processing is considered to have been executed in the country.

The preferred embodiments of the present invention have been described above, but the present invention is not limited to these embodiments. Additions, omissions, substitutions, and other modifications can be made without departing from the spirit of the present invention. The present invention is not limited by the above description, but only by the scope of the appended claims.

10 content server 11 CPU
12 memory 13 bus 14 storage 20 portable terminal device 21 CPU
22 Memory 23 Bus 24 Camera 25 Display 30 Server 31 CPU
32 memory 33 bus 34 storage C area D display

Claims

An image acquisition unit for acquiring images;
A recognition unit for recognizing a code included in the image;
A position acquisition unit for acquiring a position in the image selected by the user;
A control unit that controls to execute the content held by the code when the code is located at a position in the image acquired by the position acquisition unit;
An information processing apparatus.
A display unit for displaying the image acquired by the image acquisition unit;
The information processing apparatus according to claim 1, wherein the display unit displays an image acquired by the image acquisition unit as needed.
The information processing apparatus according to claim 2, wherein the display unit is a touch panel, and the position acquisition unit acquires a position selected by the user from the touch panel.
4. The information processing apparatus according to claim 1, wherein when the recognition unit recognizes a code, the control unit decodes content held by the code before the user selects the code.
The information processing apparatus according to any one of claims 1 to 4, wherein the code is an invisible code.
An image acquisition unit for acquiring images;
A recognition unit for recognizing an invisible code included in the image;
A display unit for displaying the image acquired by the image acquisition unit,
An information processing apparatus that changes a part of an image including the invisible code based on contents held by the invisible code.
A display method executed by the information processing apparatus according to claim 2.
A program for causing an information processing apparatus to execute the display method according to claim 7.
A computer-readable storage medium in which the program according to claim 8 is stored.