JP6740598B2 - Program, user terminal, recording device, and information processing system - Google Patents

Description

The present invention relates to a program, a user terminal, a recording device, and an information processing system.

Techniques for providing content to users are known. For example, in Patent Document 1, when a mobile phone with a built-in camera shoots a document provided with a media marker, a digital medium in which a link is associated with the media marker shown in the taken picture (content in the present invention) A technique for displaying is disclosed.

JP, 2011-135556, A

The technology of Patent Document 1 is utilized for DM (Direct Mail), which is a printed matter in which an image or the like is formed and distributed to a user, and an image representing a media marker formed in the DM is used for a web page or the like. It is possible to guide the user to the content. In that case, if the user's behavior such as whether the user has watched each DM or even the contents has been understood, it can be utilized in the marketing activity.
Therefore, it is an object of the present invention to collect information indicating a user's behavior using printed matter in a recording device.

Program according to claim 1 of the present invention, first performing the computer controlling the user terminal including a photographing unit, the reading process reads the user identification information identifying the user is a print distribution destination from the captured image of the printed matter a processing unit, is included in the captured image of the printed material have use specific information used to identify the image associated with the content an image formed on the printed material together with the user identification information a second processing unit that performs a specific process for specifying an image, the reading process and of the specific treatment, as a notification unit for notifying the recording device in association with actual results of the performed process to the user identification information It is characterized by making it function.

A program according to claim 2 of the present invention, in the configuration according to claim 1, causes the computer to function also as a third processing unit that performs an acquisition process for acquiring content associated with the image, The notifying unit notifies the recording device of a result of a process actually performed among the reading process, the specifying process, and the obtaining process in association with the user identification information.
According to a third aspect of the present invention, in the configuration according to the second aspect, the image is also associated with destination information indicating a destination requesting the content associated with the image. The third processing unit is characterized by performing, as the acquisition process, a process of requesting the content associated with the image to the destination represented by the destination information associated with the image identified by the identification process.

In the program according to claim 4 of the present invention, in the configuration according to any one of claims 1 to 3 , the specifying information is information representing a feature amount of the image, and the second processing unit is , said collating the characteristic amount supplied from the feature amount of the image and information providing apparatus included in the captured image, and identifies the image contained in the captured image.

In the program according to claim 5 of the present invention, in the configuration according to any one of claims 1 to 4 , the content associated with the image identified by the identifying process is previously acquired by the user terminal group. The computer is made to function as a first output unit that outputs a number.

A program according to claim 6 of the present invention is, in the configuration according to any one of claims 1 to 5 , associated with an image formed on a printed matter together with user identification information read by the reading process, and It is characterized in that the computer is made to function as a second output unit that outputs the number of contents that have not been acquired yet.

The program according to claim 7 of the present invention, in the configuration according to any one of claims 1 to 6 , causes the computer to function as a positioning unit that measures the position of the user terminal, and the notification unit performs measurement. It is characterized in that the recorded information is notified to the recording device in association with the user identification information, the position information representing the generated position.

The program according to claim 8 of the present invention, in the configuration according to claim 2 or 3 , causes the computer to generate an image formed on the printed matter together with the user identification information, the image being associated with content. The image processing apparatus is caused to function as a requesting unit that requests the information providing device to use the specifying information used for specifying, and the image is associated with a plurality of contents, and the third processing unit is specified by the specifying process. It is characterized in that a process for acquiring any one of the contents selected from the plurality of contents associated with the image is performed as the acquisition process.

A program according to claim 9 of the present invention, in the configuration according to claim 8 , causes the computer to function as a positioning unit that measures the position of the user terminal, and the third processing unit causes the third processing unit to From the above, the processing for acquiring the content selected according to the measured position is performed as the acquisition processing.

User terminal according to claim 10 of the present invention includes a photographing unit, a first processing unit for performing reading processing to read user identification information for identifying a user from the captured image of the printed material is a printed material was distributed to the user identification the specific process for specifying an image contained in the captured image of the printed material have use specific information used to identify the image associated with the content an image formed on the printed material with information a second processing unit that performs the reading process and of the specific processing, characterized by comprising a notification unit that notifies actually performed and the result of the processing in the recording apparatus in association with the user identification information ..

The user terminal according to claim 11 of the present invention is, in the configuration according to claim 10, a third processing unit that performs an acquisition process for acquiring content associated with the image, the reading process, and the identification. Of the processing and the acquisition processing, a notification unit that notifies the recording device of the result of the processing actually performed is associated with the user identification information.
Recording apparatus according to claim 12 of the present invention, in response to a request from the user terminal to the user identification information identifying the user is a print distribution destination was photographed image or Ra読 viewed take reading processing of the printed matter A transmission unit for transmitting, to the user terminal, identification information used to identify an image formed on the printed matter and associated with the content together with the user identification information, and the reading process and transmission from the user terminal A specific process of specifying the image included in the captured image of the printed matter performed by the user terminal using the specified information, and the content associated with the image specified by the specific process are acquired. Of the acquisition process for performing the above, a recording unit that records the result of the process actually performed in association with the user identification information is provided.

An information processing system according to claim 13 of the present invention comprises the user terminal according to claim 10 or 11 , and the recording device according to claim 12 .

According to a fourteenth aspect of the present invention, in the configuration according to the thirteenth aspect , the recording device obtains an attribute of a user to whom the printed matter on which the user identification information is formed is distributed. The recording unit of the recording apparatus records the user identification information and the attribute of the user acquired for the user identification information in association with each other.

According to a fifteenth aspect of the present invention, in the configuration according to the thirteenth or fourteenth aspect , the recording unit of the recording device is identified by the identification process in association with a result of the identification process. The feature is that the attribute of the image is recorded.

According to the inventions according to claims 1, 2, 3, 10 , 11, 12 , and 13 , it is possible to collect the information indicating the behavior of the user using the printed matter in the recording device.
According to the invention of claim 4 , it is possible to eliminate the need to communicate with an external device every time an image formed on a printed matter is captured.
According to the invention of claim 5 , it is possible to inform the user how many users are browsing the content.
According to the invention of claim 6 , it is possible to inform the user of the remaining number of contents acquired by photographing an image formed on a printed matter.
According to the invention of claim 7 , it is possible to record the position where the action represented by the recorded information is taken.
According to the invention of claim 8 , it is possible to provide different contents even if the same image is captured.
According to the invention of claim 9 , if the position where the user acts is different, the provided content can be different even if the same image is taken.
According to the invention of claim 14 , the attribute of the user who has taken the action represented by the action history information can be recorded.
According to the fifteenth aspect , it is possible to record the attribute of the image photographed by the user from among the images formed on the printed matter.

The figure showing the whole structure of the content provision system which concerns on an Example. Diagram showing an example of printed matter distributed to users Diagram showing the hardware configuration of the content distribution device Diagram showing the hardware configuration of the image forming apparatus Diagram showing the hardware configuration of the server device Diagram showing the hardware configuration of the smartphone Diagram showing the functional configuration realized by each device Figure showing an example of the screen displayed during shooting Figure showing an example of the screen displayed during shooting Figure showing an example of displayed name information Diagram showing an example of output content Diagram showing an example of the action history database Diagram showing an example of the operation procedure of each device in the history collection process The figure showing an example of the functional structure which the content provision system of a modification implement|achieves. The figure showing an example of the output content acquisition number The figure showing an example of the functional structure which the content provision system of a modification implement|achieves. A diagram showing an example of the number of unacquired contents that have been output Diagram showing the hardware configuration of the smartphone of the modification The figure showing an example of the functional structure which the content provision system of a modification implement|achieves. The figure showing an example of the action history database of a modification. The figure showing an example of the functional composition which the smartphone of a modification realizes. Diagram showing an example of a selection table The figure showing an example of the functional structure which the content provision system of a modification implement|achieves. The figure showing an example of the action history database of a modification. The figure showing an example of the action history database of a modification. The figure showing an example of the functional structure which the content provision system of a modification implement|achieves.

[1] Example FIG. 1 shows the overall configuration of a content providing system 1 according to an example. The content providing system 1 provides contents such as web pages and videos to users to whom printed matter such as direct mail is delivered, and collects information for using the behavior of the user who received the printed matter for marketing. System. The content providing system 1 includes a communication line 2, a content distribution device 10, an image forming device 20, a server device 30, and a smartphone 40.

The communication line 2 includes, for example, a mobile communication network and the Internet, and mediates exchange of data between devices. Each device included in the content providing system 1 is connected to the communication line 2 in a wired or wireless manner. The content distribution device 10 stores contents such as web pages and moving images, and distributes those contents in response to a request from an external device.

The image forming apparatus 20 is an apparatus that forms an image on a medium by, for example, an electrophotographic method, and creates a printed matter such as the above-described direct mail. The image forming apparatus 20 forms the identification information associated with the user who is the distribution destination of the printed matter and the image associated with the content distributed by the content distribution apparatus 10 on the printed matter.

FIG. 2 shows an example of a printed matter distributed to the user. In FIG. 2, the direct mail 3 in which the DM code “xxxxx-xxxxx-xx01”, the image A of the product α, the image B of the product β, and the image C of the product γ are printed is shown as an example of the printed matter. .. The DM code is information for identifying each distributed direct mail, and is an example of the “identification information” of the present invention. In this example, image A corresponds to web page a (introduction page for product α), image B corresponds to web page b (introduction page for product β), and image C corresponds to web page c (introduction page for product γ). It is assumed that these web pages are the contents distributed by the content distribution device 10.

The server device 30 performs a process for linking the printed matter distributed to the user with the content. The server device 30 also records action history information, which is information indicating a history of actions performed by the user who receives the printed matter. The server device 30 is an example of the “recording device” in the present invention. This action history information is information for using the above-described user action for marketing. The server device 30 records the result of the process performed by the smartphone 40 by the user's operation as action history information.

The smartphone 40 is a device used by a user and is an example of the “user terminal” in the present invention. The smartphone 40 includes a touch screen and a digital camera. When the smartphone 40 captures an image of a printed material by a user operation, the smartphone 40 reads a DM code from a captured image obtained as a result of the capturing, and transmits the read DM code to the server device 30.

The server device 30 is associated with the DM code, is associated with the image, the feature amount of the image formed on the printed material on which the DM code is formed, the image ID (Identification) for identifying the image, and the image. The destination information indicating the destination requesting the content and the name information of the content are stored. The destination information is, for example, a URL (Uniform Resource Locator), an IP (Internet Protocol) address, or the like, and is information indicating a destination of request data (HTTP (Hypertext Transfer Protocol) request or the like) indicating that the content is requested. The destination information is also information associated with the image formed on the printed matter.

Hereinafter, these pieces of information associated with the DM code are referred to as “image information”. As the image feature amount, a local feature amount such as SIFT (Scale-Invariant Feature Transform), PCA (Principal Component Analysis)-SIFT, SURF (Supeeded-Up. Robust Features) is used. The image feature amount is used, for example, to identify the images A, B, and C from the captured image of the direct mail 3 captured by the smartphone 40 using the image matching technique.

That is, the image information is an image formed in the direct mail 3 together with the DM code, and is used to identify an image associated with the content (for example, the images A, B, and C shown in FIG. 2). .. The image information is an example of the “specification information” of the present invention. The server device 30 reads out the image information (feature amount, image ID, destination information, and name information) stored in association with the transmitted DM code, and transmits the image information to the smartphone 40. For example, when the DM code shown in FIG. 2 is transmitted, the server device 30 displays the feature amounts of the images A, B, and C, the image IDs, and the destination information of the web pages a, b, and c. The information is transmitted to the smartphone 40. The server device 30 is a device that provides image information to the smartphone 40, and is also an example of the “information providing device” of the present invention.

After that, when the user photographs the image A formed on the printed matter using the smartphone 40, the smartphone 40 determines that the image A was photographed by comparing the characteristic amount of the photographed image with the transmitted characteristic amount. To do. Then, the smartphone 40 displays the name information associated with the image A, and when the user performs an instruction operation to instruct acquisition of the content of the displayed name, the smartphone 40 displays the destination information associated with the image A. Use to request content. In response to this request, the smartphone 40 acquires the web page a, which is the content distributed from the content distribution device 10, and displays it on the touch screen. In the content providing system 1, the content is provided to the user in this way.

FIG. 3 shows the hardware configuration of the content distribution device 10. The content distribution device 10 is a computer including a CPU (Central Processing Unit) 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, a storage 14, and a NIC (Network Interface Card) 15. .. The CPU 11 controls the operation of each unit by using the RAM 12 as a work area and executing a program stored in the ROM 13 or the storage 14. The storage 14 is a storage medium such as an HDD (Hard Disk Drive) and an SSD (Solid State Drive), and stores data and programs used by the CPU 11 for control. The NIC 15 has a communication circuit and communicates with an external device via the communication line 2.

FIG. 4 shows the hardware configuration of the image forming apparatus 20. The image forming apparatus 20 is a computer including a CPU 21, a RAM 22, a ROM 23, a storage 24, a NIC 25, a printer 26, and an operation panel 27. The CPU 21 to the NIC 25 are common to the hardware of the same name shown in FIG. The printer 26 includes a photoconductor, a developing device, a transfer device, a fixing device, and the like, and forms an image on a medium such as paper by an electrophotographic method.

FIG. 5 shows the hardware configuration of the server device 30. The server device 30 is a computer including a CPU 31, a RAM 32, a ROM 33, a storage 34, and a NIC 35. These are common to the hardware of the same name shown in FIG.

FIG. 6 shows the hardware configuration of the smartphone 40. The smartphone 40 is a computer including a CPU 41, a RAM 42, a ROM 43, a storage 44, a NIC 45, a touch screen 46, and a digital camera 47. The CPU 41, RAM 42, and ROM 43 are common to the hardware of the same name shown in FIG. The storage 44 is a storage medium such as a flash memory, and stores data and programs used by the CPU 41 for control. The storage 44 stores a content acquisition program that realizes a function for acquiring content based on a printed matter distributed to a user.

The NIC 45 has a communication circuit for mobile communication and communicates with an external device via the communication line 2 that performs wireless communication. The touch screen 46 includes a display and a touch panel provided on the surface of the display, displays an image, and receives an operation from the user. The digital camera 47 includes a lens, an image sensor, and the like, and takes a photograph of a subject. The digital camera 47 also has a function of displaying an image of a subject on the touch screen 46 in order to determine the angle of a photograph or the like.

The functions described below are realized by the CPU of each device included in the content providing system 1 executing a program (the smartphone 40 described above is the content acquisition program) to control each unit.
FIG. 7 shows a functional configuration realized by each device. The content distribution device 10 includes a content reading unit 101 and a content transmitting unit 102. The server device 30 includes an image information reading unit 301, an image information storage unit 302, an image information transmission unit 303, a processing result recording unit 304, and a processing result output unit 305. The smartphone 40 includes a photographing unit 401, a reading processing unit 402, an image information requesting unit 403, an image specifying processing unit 404, a content acquisition processing unit 405, a content output unit 406, and a processing result notification unit 407. ..

The image capturing unit 401 of the smartphone 40 captures an image of a subject and supplies the captured image of the subject to a function corresponding to the screen displayed at the time of capturing.
FIG. 8 shows an example of a screen displayed at the time of shooting. In FIG. 8, the DM code recognition screen is displayed on the touch screen 46. In this screen, the character string "Recognize the DM code. Please align the frame with the DM code.", the recognition area A1, the print image B1 of the direct mail 3 which is the photographed print, and the print image B1. The DM code image C1 included in is displayed. When the DM code recognition screen is displayed, the imaging unit 401 supplies the image displayed in the recognition area A1 to the reading processing unit 402.

The reading processing unit 402 of the smartphone 40 performs a reading process of reading the identification information formed on the printed matter (printed matter distributed to the user) received by the user from the captured image of the medium. The reading processing unit 402 is an example of the “first processing unit” in the present invention. The identification information mentioned here is, specifically, information for identifying a user who is a distribution destination of the printed matter or the printed matter. In this embodiment, the reading processing unit 402 reads a DM code that identifies each direct mail as identification information.

The reading processing unit 402 extracts, for example, the characters included in the image displayed in the recognition area A1 by the technique of OCR (Optical Character Recognition). When the three sets of character strings connected by “-(hyphen)” are extracted and each character string is four characters, the reading processing unit 402 recognizes the three sets of character strings as a DM code. To do. The reading processing unit 402 supplies the read DM code, which is the identification information, to the image information requesting unit 403.

When the DM code is read by the reading processing unit 402, the image information requesting unit 403 of the smartphone 40 is used to identify the image formed on the printed matter together with the DM code and associated with the content. The image information is requested from the server device 30. The image information requesting unit 403 is an example of the “requesting unit” in the present invention. The image information requesting unit 403 makes this request by transmitting, for example, request data indicating that the read DM code and the image information corresponding thereto are requested to the server device 30. The transmitted request data is supplied to the image information reading unit 301 of the server device 30.

The image information reading unit 301 of the server device 30 reads the image information requested by the smartphone 40 from the image information storage unit 302. The image information storage unit 302 stores the image information in association with the DM code. The image information reading unit 301 reads the image information stored in the image information storage unit 302 in association with the DM code represented by the supplied request data, as the image information requested by the smartphone 40, and the image information transmitting unit 303. Supply to.

The image information transmitting unit 303 of the server device 30 transmits the image information corresponding to the DM code to the requesting smartphone 40 in response to the request from the smartphone 40 that has performed the reading process of reading the DM code. The image information transmission unit 303 is an example of the “transmission unit” in the present invention. The image information transmitted by the image information transmission unit 303 is supplied to the image identification processing unit 404 of the smartphone 40.

When the image information requested by the image information requesting unit 403 is provided, the image specifying processing unit 404 of the smartphone 40 performs the specifying process of specifying the image included in the captured image of the printed matter using the image information. .. Specifically, the image identification processing unit 404 identifies the image included in the captured image by comparing the feature amount of the image included in the captured image with the feature amount provided by the server device 30. The image identification processing unit 404 is an example of the “second processing unit” in the present invention. The image identification processing unit 404 performs identification processing when the captured image of the printed matter is displayed on the touch screen 46.

FIG. 9 shows an example of a screen displayed at the time of shooting. In FIG. 9, the image specifying screen is displayed on the touch screen 46. On this screen, the character string "Please hold the camera over the image of interest.", the recognition area A2, and the printed matter image B2 of the direct mail 3 are displayed. When the image specifying screen is displayed, the photographing unit 401 supplies the image displayed in the recognition area A2 (the image including the image C and its periphery in the example of FIG. 9) to the image specifying processing unit 404.

The image identification processing unit 404 compares the feature amount represented by the supplied image information with the feature amount represented by the captured image, and for example, the image information may include a feature amount whose difference is less than a threshold value. For example, the image representing the feature amount is specified as the image included in the captured image. In the example of FIG. 9, the image identification processing unit 404 identifies the image C shown in FIG. 2 as an image included in the captured image. In this embodiment, the image identification processing unit 404 displays the name information of the content associated with the identified image in the image information.

FIG. 10 shows an example of the displayed name information. In the example of FIG. 10, the name information D1 of “introduction page of product γ” is displayed on the touch screen 46 in an overlapping manner on the printed material image B2 illustrated in FIG. 9. When the user performs an operation of tapping the name information D1, for example, the image identification processing unit 404 accepts this operation as the above-described instruction operation (an operation of instructing acquisition of the content with the displayed name), and for example, the identified image is displayed. The destination information of the content associated in the image information is supplied to the content acquisition processing unit 405.

When the image identification processing unit 404 identifies an image, the content acquisition processing unit 405 of the smartphone 40 performs an acquisition process for obtaining content associated with the image. The content acquisition processing unit 405 is an example of the “third processing unit” in the present invention. In the present embodiment, the content acquisition processing unit 405 performs, as an acquisition process, a process of requesting the content associated with the image to the destination represented by the destination information associated with the image identified by the identification process.

Specifically, the content acquisition processing unit 405 first requests the content from the destination information supplied from the image identification processing unit 404. If the destination information is a URL, the content acquisition processing unit 405 performs a process of transmitting an HTTP request requesting the content to the URL as the acquisition process. The transmitted HTTP request is supplied to the content reading unit 101 of the content distribution device 10.

When the content reading unit 101 of the content distribution apparatus 10 is requested by the external apparatus for the content stored in the storage unit of the apparatus (for example, the storage 14 shown in FIG. 3), the content reading unit 101 reads the requested content from the storage unit. The content reading unit 101 supplies the read content to the content transmitting unit 102. The content transmitting unit 102 transmits the content read by the content reading unit 101 to the external device (smartphone 40 in this embodiment) that is the source of the content.

The content acquisition processing unit 405 acquires the content thus transmitted as content corresponding to the image identified by the identification processing. The content acquisition processing unit 405 supplies the acquired content to the content output unit 406. The content output unit 406 outputs the content acquired by the content acquisition processing unit 405. The content output unit 406 outputs to, for example, a display unit (for example, the touch screen 46 shown in FIG. 6).

FIG. 11 shows an example of the output content. In the example of FIG. 10, the web page c shown in FIG. 2 is displayed on the touch screen 46 as the content E1. Note that, for example, when the content includes audio information, the content output unit 406 may output the content to sound emitting means such as a speaker. Further, the output destination of the content is not limited to the own device but may be an external device.

The processing result notification unit 407 of the smartphone 40 notifies the server device of the result of the processing actually performed among the above-described reading processing, identification processing, and acquisition processing in association with the identification information (DM code in this embodiment). To do. The processing result notification unit 407 is an example of the “notification unit” in the present invention. The processing result notification unit 407 is supplied with processing result information indicating the result of the processing performed by each processing unit (the reading processing unit 402, the image identification processing unit 404, the content acquisition processing unit 405).

For example, when the reading processing unit 402 performs the reading process to read the DM code, the reading processing unit 402 supplies the read DM code to the processing result notifying unit 407 as processing result information. When the image identifying processing unit 404 identifies the image by performing the identifying process, the image identifying processing unit 404 supplies the image ID of the identified image as the processing result information to the processing result notifying unit 407. When the content acquisition processing unit 405 performs the acquisition process to acquire the content, the content acquisition processing unit 405 supplies the name information of the acquired content as the processing result information to the processing result notification unit 407.

When the processing result information is supplied from each processing unit, the processing result notification unit 407 transmits to the server device 30 the notification data in which the processing result information and the DM code read by the reading process are associated with each other. By doing so, the above notification is given. When the DM code is supplied from the reading processing unit 402, the processing result notifying unit 407 transmits the notification data representing the DM code to the server device 30. By transmitting this notification data, the result that the DM code is read by the reading process is notified to the server device 30 in association with the read DM code. The transmitted notification data is supplied to the processing result recording unit 304 of the server device 30.

When the processing result recording unit 304 of the server device 30 notifies the smartphone 40 of the processing result in association with the identification information (DM code in this embodiment), the processing result recording unit 304 records the notified processing result in association with the identification information. To do. The processing result recording unit 304 is an example of the “recording unit” in the present invention. When the notification data is supplied, the processing result recording unit 304 associates the processing result represented by the notification data with the DM code and stores them in the action history database stored in the own device. The action history database is a database for storing the above-described action history information (information indicating a history of actions performed by the user who receives the printed matter).

FIG. 12 shows an example of the action history database. In the example of FIG. 12, the result of the reading process “xxxx-xxxx-xx01”, that is, the result of the specific process “image C” and the result of the acquisition process “product γ introduction page” in association with the DM code. Are stored, and the result of the specific process of “image A” and the result of the acquisition process of “introduction page of product α” are stored. Image A and image C are image IDs of the respective images.

An image ID “image A” is stored in association with the result (DM code) of the reading process “xxxx-xxxx-xx02” as the result of the specific process, but the result of the acquisition process is not stored. .. Further, neither the result of the specific process nor the result of the corresponding acquisition process is stored in the result (DM code) of the reading process “xxxx-xxxx-xx03”. Note that these DM codes are assumed to be formed on the printed matter in which the images A, B, and C are formed like the direct mail 3 shown in FIG. 2 and are distributed to different users.

The processing result recording unit 304 stores each piece of information as shown in FIG. 12, thereby recording the processing result in association with the DM code. The processing result thus recorded is the information indicating the history of the action taken by the user who received the printed matter, that is, the action history information described above. For example, the result of the reading process represents an action history in which the user takes an action of photographing the DM code formed on the printed matter with the smartphone 40. In addition, the result of the specific process represents an action history that the user has taken an action of photographing one of the images formed on the printed matter with the smartphone 40, and the result of the acquisition process indicates an operation for instructing acquisition of the content. Indicates the history of actions performed.

For example, with respect to the user who received the printed material on which the DM code “xxxx-xxxx-xx01” is formed, it is said that the user has performed the operation of instructing acquisition of the corresponding web page by photographing the DM code, the image A and the image C. The action history is displayed. Regarding the user who received the printed material on which the DM code "xxxx-xxxx-xx02" was formed, the user captured the DM code and the image A, but did not perform an operation to instruct the acquisition of the web page corresponding to the image A. The action history is displayed. In addition, regarding the user who receives the printed material on which the DM code “xxxxx-xxxxx-xx03” is formed, the action history indicating that the DM code was photographed but the image photographing operation was not performed is displayed. .. In this way, the processing result recording unit 304 records the action history information of various users to whom the printed matter is distributed.

The processing result output unit 305 of the server device 30 outputs the processing result recorded in the processing result recording unit 304. The output destination of the processing result is, for example, an information processing device of a business operator who wants to utilize the collected action history information for marketing. This business operator uses the output action history information, for example, among the users who have received the distributed direct mail, the user who has read the DM code and the user who has taken the image and displayed the name information of the content. The information processing device is caused to perform a process of calculating the ratio of users who have performed an operation for instructing acquisition of content. As a result, it is possible to analyze which image among the images formed on the printed matter the user is interested in, and how many users are interested in the printed matter itself. It will be used for the next product development and advertising investment.

Each device included in the content providing system 1 performs history collection processing for collecting user's action history information based on the above configuration.
FIG. 13 shows an example of the operation procedure of each device in the history collection process. This operation procedure is started when the user photographs the DM code of the printed matter using the smartphone 40. First, the smartphone 40 (imaging unit 401) captures an image of a printed matter (step S11). Next, the smartphone 40 (reading processing unit 402) performs a reading process of reading the DM code from the captured image of the printed matter (step S12).

Next, the smartphone 40 (processing result notification unit 407) notifies the server device 30 of the result of the reading process in association with the DM code read in step S12 (step S1). The server device 30 (processing result recording unit 304) records the notified processing result in association with the DM code (step S2). Subsequently, the smartphone 40 (image information requesting unit 403) requests the server device 30 for image information corresponding to the read DM code (step S13). The server device 30 (image information reading unit 301) reads the image information corresponding to the requested DM code (step S14). The server device 30 (image information transmitting unit 303) transmits the read image information to the requesting smartphone 40 (step S15). The operations of steps S1 and S2 may be performed after the operations of steps S13 to S15, or may be performed in parallel with these operations.

Next, it is assumed that the user uses the smartphone 40 to photograph any image formed on the printed matter. Then, first, the smartphone 40 (imaging unit 401) captures an image of the printed matter (step S21). Next, the smartphone 40 (image identification processing unit 404) performs identification processing for identifying an image included in the captured image of the printed matter (step S22). Subsequently, the smartphone 40 (processing result notification unit 407) notifies the server device 30 of the result of the specific processing in association with the DM code read in step S12 (step S3). The server device 30 (processing result recording unit 304) records the notified specific result in association with the DM code (step S4).

Next, the smartphone 40 (content acquisition processing unit 405) requests the content distribution device 10 for content corresponding to the specified image (step S23). The content distribution device 10 (content reading unit 101) reads out the requested content (step S24). The content distribution device 10 (content transmission unit 102) transmits the read content to the requesting smartphone 40 (step S25). The operations of steps S3 and S4 may be performed after the operations of steps S23 to S25, or may be performed in parallel with these operations.

Subsequently, the smartphone 40 (content acquisition processing unit 405) acquires the transmitted content (step S26). Next, the smartphone 40 (processing result notification unit 407) notifies the server device 30 of the result of the acquisition processing in association with the DM code read in step S12 (step S5). The server device 30 (processing result recording unit 304) records the notified acquisition result in association with the DM code (step S6). Then, the smartphone 40 (content output unit 406) outputs the acquired content to, for example, the display unit of the own device (step S27). The operations of steps S5 and S6 may be performed after the operation of step S27, or may be performed in parallel with this operation. Then, the server device 30 (processing result output unit 305) outputs the processing results recorded in steps S2, S4, and S6 as action history information based on, for example, an instruction from the external device (step S7).

In the content providing system 1, as described above, the action history information indicating the action of the user who used the printed matter is collected in the server device 30. Further, in the present embodiment, image information is provided to the smartphone 40, and the smartphone 40 performs the identification process of identifying the image using the image information. As a result, it is not necessary to communicate with the server device 30 that stores image information each time an image formed on a printed material is captured.

[2] Modified Example The above-described embodiment is merely an example of the implementation of the present invention, and may be modified as follows. Further, the embodiments and the respective modified examples may be implemented in combination as necessary.

[2-1] Output of Content Acquisition Number The number of times the content associated with the image captured on the image specifying screen shown in FIG. 9 has been acquired in the past by the user terminal group (hereinafter referred to as “content acquisition number”) is defined by the user. May be presented.
FIG. 14 shows an example of a functional configuration realized by the content providing system 1a of this modification. The smartphone 40a included in the content providing system 1a includes a content acquisition number acquisition unit 408 and a content acquisition number output unit 409 in addition to the units illustrated in FIG.

The image information reading unit 301 of the server device 30 of the present modification supplies the read image information to the processing result output unit 305 together with the destination information (IP address of the smartphone 40, etc.) of the request source of the image information. Of the images recorded as the result of the specific processing, the processing result output unit 305 determines the number of contents associated with the image in which the supplied image information includes the feature amount as the result of the acquisition processing. It is output to the destination information of the request source as the content acquisition number corresponding to.

For example, when the image information reading unit 301 reads the image information associated with the DM code “xxxxx-xxxxx-xx01” illustrated in FIG. 8, the processing result output unit 305 causes the processing result output unit 305 to read each of the images A, B, and C. The number of contents recorded in association with the image ID (for example, ◯ for image A, □ for image B, and Δ for image C) is output to the smartphone 40 as the number of acquired contents. For example, when the processing result shown in FIG. 12 is recorded, the number of contents recorded in association with the image IDs of the images A and C is one, and the content is recorded in association with the image ID of the image B. The number of recorded contents is 0 (two images A are recorded, but one is recorded in association with it).

The content acquisition number acquisition unit 408 acquires the content acquisition number thus output from the server device 30 and supplies it to the content acquisition number output unit 409. The image identification processing unit 404 supplies the image ID for identifying the image identified to the content acquisition number acquisition unit 408, and the content acquisition number acquisition unit 408 requests the server device 30 for the content acquisition number corresponding to the image ID. Therefore, the content acquisition number may be acquired.

The image identification processing unit 404 of the present modified example supplies the image ID for identifying the identified image to the content acquisition number output unit 409. The content acquisition number output unit 409 outputs the content acquisition number thus supplied, that is, the content acquisition number corresponding to the image identified by the image identification processing unit 404, for example, to the display unit of its own device. The content acquisition number output unit 409 is an example of the “first output unit” in the present invention.

FIG. 15 shows an example of the output content acquisition count. In the example of FIG. 15, with the name information D1 of the web page c displayed on the touch screen 46 in the image specifying screen shown in FIG. 10, the number of acquired contents of the web page c is displayed. The character string "Matter" is displayed as the content acquisition count F1. The content acquisition number output unit 409 may output the content acquisition number to a sound emitting unit such as a speaker or an external device. In either case, the output of the content acquisition number informs the user how many users are viewing the content.

[2-2] Output of the number of unacquired contents When a plurality of images are formed on the printed matter, the number of contents that have not been acquired yet among the contents associated with each image (hereinafter referred to as “unacquired contents”). “Number”) may be presented to the user.
FIG. 16 shows an example of a functional configuration realized by the content providing system 1b of this modification. The smartphone 40b included in the content providing system 1b includes an unacquired content number acquisition unit 410 and an unacquired content number output unit 411 in addition to the units illustrated in FIG.

The image information reading unit 301 supplies the read image information and the DM code used to request the image information to the processing result output unit 305 together with the destination information of the sender of the DM code. The processing result output unit 305 determines the number of pieces of name information not recorded in association with the supplied DM code, out of the name information of the content represented by the supplied image information, as a printed matter on which the DM code is formed. It is output to the destination information of the transmission source as the number of unacquired contents in.

For example, when the processing result shown in FIG. 12 is recorded, the number of pieces of name information that is not recorded in association with the DM code “xxxx-xxxxx-xx01” is 1 (only the introduction page of the product β). In addition, the number of pieces of name information that are not recorded in association with the DM codes of "xxxxx-xxxxx-xx02" and "xxxx-xxxxx-xx03" is 3 (each product α, β, γ introduction page). is there.

The unacquired content number acquisition unit 410 acquires the unacquired content number output from the server device 30 in this manner, and supplies the unacquired content number output unit 411. The unacquired content number acquisition unit 410 may acquire the unacquired content number by requesting the server device 30 for the unacquired content number corresponding to the DM code read by the reading processing unit 402.

The number of unacquired contents output unit 411 is the number of unacquired contents thus supplied, that is, the number of contents that are associated with the image formed on the printed material together with the DM code read by the reading processing unit 402 and have not been acquired yet. Is output to the display means of the own device, for example. The unacquired content number output unit 411 is an example of the “second output unit” in the present invention.

FIG. 17 shows an example of the number of unacquired contents that have been output. In the example of Fig. 17, in the image specifying screen (the image specifying screen displayed after the DM code is recognized) shown in Fig. 9, "There are two unacquired contents. Please hold the camera over the image you are interested in. .” is displayed as the number of unacquired contents G1. The unacquired content number output unit 411 may output the unacquired content number to a sound emitting means such as a speaker or an external device. In any case, by outputting the number of unacquired contents, the user is notified of the remaining number of contents acquired by photographing the image formed on the printed matter.

[2-3] Recording of Position Information In addition to the action history information described above, for example, position information indicating the position where the user has acted may be recorded.
FIG. 18 shows the hardware configuration of the smartphone 40c of this modification. The smartphone 40c includes a GPS (Global Positioning System) 48 in addition to the hardware shown in FIG. The GPS 48 measures the current position based on signals from GPS satellites. The GPS 48 supplies the CPU 41 with position information indicating the current position in latitude and longitude.

FIG. 19 shows an example of a functional configuration realized by the content providing system 1c of this modification. The smartphone 40c included in the content providing system 1c includes a positioning unit 412 in addition to the units illustrated in FIG. The positioning unit 412 measures the position of its own terminal (smartphone 40c). In the positioning unit 412, the processing result information (for example, the DM code, the image ID, and the name information of the content) described above when each processing is performed from the reading processing unit 402, the image identification processing unit 404, and the content acquisition processing unit 405. ) Is supplied. The positioning unit 412 measures the position of its own terminal when the processing result information is supplied. When measuring the position of the own terminal, the positioning unit 412 supplies the processing result notifying unit 407 with the position information as the measurement result together with the supplied processing result information.

The processing result notifying unit 407 notifies the server device 30 of the position information indicating the position of the terminal itself measured by the positioning unit 412 in association with the DM code. For example, when the DM code is supplied as the processing result information together with the position information, the processing result notifying unit 407 sends the notification data by associating the information to the server device 30 to make this notification. Further, when the image ID and the content name information are supplied as the processing result information together with the position information, the processing result notification unit 407 notifies the processing result information, the DM code, and the position information in association with each other. This notification is performed by transmitting the data to the server device 30.

FIG. 20 shows an example of the action history database of this modification. In the example of FIG. 20, the position information is stored in association with the results of the reading process, the specific process, and the acquisition process (however, the part where the action history information is not stored (the part represented by “−”). ), the corresponding position information is not stored)). As described above, according to this modification, the position where the action represented by the action history information is taken is recorded.

[2-4] Selection of Contents Plural contents may be associated with the image formed on the printed matter.
FIG. 21 shows an example of a functional configuration realized by the smartphone 40d of this modification. The smartphone 40d includes a positioning unit 412 and a content selection unit 413 in addition to the units illustrated in FIG. The positioning unit 412 measures the position of its own terminal (smartphone 40d), similar to that shown in FIG. The positioning unit 412 measures the position when requested by the content selection unit 413, for example, and supplies the measurement result to the content selection unit 413.

The content selection unit 413 selects any one of the plurality of contents associated with the image identified by the identification process, according to the position of the terminal itself measured by the positioning unit 412. The image ID and image information of the image identified by the image identification processing unit 404 are supplied to the content selection unit 413. When these pieces of information are supplied, the content selection unit 413 requests the positioning unit 412 to supply the measurement result of the position of its own device, and is associated with the supplied image ID in the supplied image information. The content corresponding to the position indicated by the measurement result is selected from among the content of the name.

The content selection unit 413, for example, uses the selection table in which the position of the user terminal and a plurality of contents associated with the image formed on the printed matter are associated with each other to perform the above selection.
FIG. 22 shows an example of the selection table. In the example of FIG. 22, user home appliances such as “home”, “commuting road”, and “workplace” are called “home appliance introduction page”, “portable product introduction page”, and “office supplies introduction page”, respectively. It is associated with the name of the content. It is assumed that all of these contents are associated with the image A formed in the direct mail 3 shown in FIG. 2, for example.

In the smartphone 40d, the location candidates of the user such as the home address of the user who uses the terminal, the work address, the route used for the commuting route, and the station are registered. Using the registration information, the content selection unit 413 determines that the location where the distance from the position represented by the measurement result supplied from the positioning unit 412 is equal to or less than the threshold is the location of the user terminal, and the location of the user terminal is determined. The content having the name associated with the position in the selection table is selected. Upon selecting the content in this way, the content selection unit 413 supplies the name and image information of the selected content to the content acquisition processing unit 405.

The content acquisition processing unit 405 performs a process of acquiring the content of the name supplied from the content selection unit 413 as the acquisition process. That is, the content acquisition processing unit 405 performs, as an acquisition process, a process for acquiring any one of the contents selected from the plurality of contents associated with the image specified by the specifying process. Then, this selection is performed according to the position of the own terminal measured by the positioning unit 412 as described above.

As a result, for example, if the user shoots the image A while at home, an introduction page for home electric appliances is provided, and if the user shoots the image A while commuting to work or home, an introduction page for the portable product is provided. If you take Image A while you are at work, you will be provided with an introduction page for office supplies. As described above, in this modification, even if the same image is captured, the content to be provided is different. Specifically, if the position where the user acts is different, the content provided is different even if the same image is taken.

The selection of the content according to the position of the own device may be utilized as follows. For example, depending on the position measured when an image of a printed matter distributed by a business operator of a chain store was taken, content that introduces the store closest to the position is provided, or the store closest to the position is provided. If the distance is less than or equal to the threshold value, a special coupon is provided as content.

Also, the selection of the content may be performed by other methods. For example, the content selection unit 413 may select content according to the current time. In this case, the content selection unit 413 selects a web page of a television program at night, for example, and selects a news web page during commuting hours. As described above, the content selection unit 413 selects content according to various situations such as the position of the user terminal and the current time. Other conditions to be used for selection include weather, temperature, moving speed, and altitude. When these situations are used, the smartphone may be equipped with a sensor that measures each situation.

In addition, other situations such as the user's schedule and purchase history may be used. In these cases, the information (schedule, purchase history, etc.) stored by other functions such as the schedule management application and the household account application installed on the smartphone may be acquired. Whichever situation is used, it is desirable that the content selection unit 413 selects the content that the user is likely to be interested in in that situation.

Also, the content selection unit 413 may select content according to the OS of the smartphone, the type of browser that displays the content, the version of the browser, and the like. For example, if the content corresponding to the type of OS is associated with the image, the content selection unit 413 selects the content corresponding to the OS of the smartphone. Thereby, the content that can be output by the smartphone used by the user is selected.

Further, the content selection unit 413 may not select the content depending on the situation. For example, the content selection unit 413 does not select the content during the day, so that the content is not acquired even when an image is captured during work. Further, the content selection unit 413 selects content such as a coupon in a specific time zone, does not select the content in other time zones, and provides the coupon only in the specific time zone. Good. As a result, the content is provided only in a specific time zone.

Further, the content selection unit 413 may select the content according to the order in which the images are specified. For example, in the direct mail 3 shown in FIG. 2, content selection is performed depending on the case where images are specified in the order of image A, the image B, and image C, and the case where images are specified in the order of image B, image A, and image C. The unit 413 changes the content selected from the plurality of contents corresponding to the last image C. The content selection unit 413 may select the content only when the images are specified in a specific order, and may not select the content when the images are specified in another order. This provides the enjoyment of changing the order in which the images are taken and collecting more types of content.

Further, when the smartphone includes sound collecting means such as a microphone, the content selection unit 413 may select content according to the volume of the collected sound. The content selection unit 413 selects the content represented by only the image if the volume is equal to or higher than the threshold, and selects the content represented by the image and the sound if the volume is lower than the threshold. As a result, the content with sound is provided when the user can hear the sound.

[2-5] Recording of User Attribute In addition to the action history information described above, for example, user attribute may be recorded.
FIG. 23 shows an example of a functional configuration realized by the content providing system 1e of this modification. The server device 30e included in the content providing system 1e includes a user attribute storage unit 306 and a user attribute acquisition unit 307 in addition to the units illustrated in FIG. The user attribute storage unit 306 stores the user attribute of the user who uses the content providing system 1e in association with, for example, the terminal ID of the user terminal used by the user.

The user attributes are, for example, the user's sex, age, residence area, occupation, hobbies and the like. For example, the user attribute is used such that the tendency of the action history is checked for each user attribute and it is useful for marketing activities. In the content providing system 1e, the user is supposed to register his or her own user attribute in advance. Other than that, for example, the smartphone 40 may store the user attribute information and periodically transmit the user attribute information to the server device 30e, or the user attribute may be acquired from another system.

In this modification, the processing result notification unit 407 transmits the notification data to which the terminal ID assigned to the own terminal is added. This notification data is supplied to the processing result recording unit 304 and the user attribute acquisition unit 307 of the server device 30e. The user attribute acquisition unit 307 uses the supplied notification data to acquire the attribute of the user of the smartphone 40 that is the transmission source of the notification data, that is, the user to whom the printed material in which the DM code is formed is distributed. The user attribute acquisition unit 307 is an example of the “first acquisition unit” in the present invention.

The user attribute acquisition unit 307 reads out the user attribute stored in the user attribute storage unit 306 in association with the device ID represented by the notification data, and acquires the user attribute as the user attribute of the user to whom the above-mentioned printed matter is distributed. The user attribute acquisition unit 307 supplies the acquired user attribute to the processing result recording unit 304. The processing result recording unit 304 stores the DM code represented by the notification data and the user attribute acquired for the DM data (the user attribute acquired by the user attribute acquisition unit 307 using the notification data notified of the DM code). Correlate and record.

FIG. 24 shows an example of the action history database of this modification. In the example of FIG. 24, the user attribute is stored in association with each of the results of the reading process, the specific process, and the acquisition process (however, the part in which the action history information is not stored (the part represented by “−”). ), the corresponding user attributes are not stored). As described above, according to this modification, the attribute of the user who took the action represented by the action history information is recorded, and the recording result is used for analyzing the field of interest for each attribute of the user.

[2-6] Recording of Image Attribute In addition to the action history information described above, the attribute of the image identified by the identifying process may be recorded, for example. The image attributes include, for example, the classification of the subject represented in the image (people, animals, vehicles, buildings, landscapes, etc.), the position where the image is formed in the printed matter, the size of the image, the color of the image (full color Or black and white) is used. In this modified example, it is assumed that the business operator involved in the printed matter registers the image attribute in the server device in advance.

The processing result recording unit 304 records the attribute of the image specified by the specific processing in association with the result of the specific processing.
FIG. 25 shows an example of the action history database of this modification. In the example of FIG. 25, the image attribute is stored in association with the result of each specific process. As described above, according to this modification, the attributes of the image captured by the user among the images formed on the printed matter are recorded.

[2-7] Device that realizes each unit The device that realizes each unit illustrated in FIG. 7 and the like is not limited to the above.
FIG. 26 shows an example of a functional configuration realized by the content providing system 1g of this modification. The content providing system 1g includes a first server device 30g-1, a second server device 30g-2, and a smartphone 40.

The first server device 30g-1 realizes the image information reading unit 301, the image information storage unit 302, and the image information transmitting unit 303 shown in FIG. 7. The second server device 30g-2 implements the content reading unit 101 and the content transmitting unit 102 implemented by the content delivery apparatus 10, in addition to the process result recording unit 304 and the process result output unit 305 illustrated in FIG.

In the embodiment, the server device 30 plays the roles of both the information providing device that provides the image information and the recording device that records the action history information. However, as in the example of FIG. You may play the role of a providing apparatus and a recording apparatus. Further, the server device may play the role of a device that distributes the content. The functional configuration shown in FIG. 26 is also an example, and various devices other than this may realize each unit shown in FIG. 7 and the like.

[2-8] User Terminal Although the smartphone is used as the user terminal in the embodiments, the present invention is not limited to this, and a tablet terminal, a notebook computer, a mobile phone, or the like may be used as the user terminal. Any terminal may be used as the user terminal as long as it is a terminal that realizes each function, such as a smartphone shown in FIG. 7 and the like.

[2-9] Printed matter In the embodiment, the direct mail is distributed to the user as a printed matter, but it is not limited to this. For example, it may be a leaflet, a publication, a pamphlet, a poster, or the like. Further, these are print media such as sheet-shaped paper and vinyl, but other than that, a three-dimensional object such as a tissue, a fan, a pen, or a container may be distributed to the user as a print.

[2-10] Route of Printed Material The direct mail described in the embodiment is mainly distributed to users by mail, but the present invention is not limited to this, and a business operator such as a courier may distribute the printed material. However, the user may take the printed matter distributed to a store such as a convenience store in the neighborhood to receive it. Further, the printed matter described so far is a distributed matter to be distributed to the user, but the present invention is not limited to this, and the printed matter placed at a restaurant or the like is taken by the user in the form of being taken by the user. May be. As described above, any route may be used as long as the printed matter can be delivered to the user.

[2-11] Identification Information Although the DM code for identifying each direct mail is used as the identification information in the embodiment, the identification information is not limited to this. For example, a DM code that identifies a group of direct mails that have common products, regions, distribution times, or the like may be used, or information such as a user ID that identifies an individual user may be used as identification information. In short, the identification information may be information corresponding to the granularity that the business operator that analyzes the behavior history of the user based on the recorded behavior history information wants to analyze. For example, the DM code of the embodiment is used to analyze the behavior of each printed matter, and the identification information common to the area is used to analyze the behavior tendency of each area. Further, the identification information is not limited to the one represented by a character string as shown in FIG. For example, it may be represented by a barcode or a QR (Quick Response) code (registered trademark).

[2-12] Image Identification Method The image identification processing unit 404 may identify an image by a method different from that in the embodiment. For example, the image identification processing unit 404 may identify an image by using a feature amount different from that in the embodiment or by using a technique such as binarization or contour extraction. In short, any specifying method may be used as long as the image included in the captured image is specified.

[2-13] Content Acquisition Method The content acquisition processing unit 405 may acquire content by a method different from that in the embodiment. For example, when the image information transmitted from the server device does not include the destination information of the content, the destination information of the content associated with the image identified by the identifying process is transmitted to the server device (or another external device). ), and the content may be acquired using the destination information notified by the response.

[2-14] Positioning Method The positioning method by the positioning unit 412 shown in FIG. 19 and the like is not limited to GPS. For example, various known positioning methods such as a positioning method using an assisted GPS, a DGPS (Differential GPS), and a BLE (Bluetooth (registered trademark) Low Energy) beacon may be used.

[2-15] Category of the Invention The present invention includes the above-mentioned user terminal (for example, smartphone), recording device (for example, server device), information providing device (for example, server device), content distribution device, and the like, as well as those devices. It can be understood as an information processing system. Further, the present invention can be understood as an information processing method for realizing the processing executed by those devices, and as a program for causing a computer that controls those devices to function. This program may be provided in the form of a recording medium such as an optical disc having the program stored therein, or may be downloaded to a computer via a communication line such as the Internet and installed and made available. May be provided.

DESCRIPTION OF SYMBOLS 1... Content providing system, 2... Communication line, 10... Content distribution apparatus, 20... Image forming apparatus, 30... Server apparatus, 40... Smartphone, 101... Content reading section, 102... Content transmitting section, 301... Image information reading section , 302... Image information storage unit, 303... Image information transmission unit, 304... Processing result recording unit, 305... Processing result output unit, 306... User attribute storage unit, 307... User attribute acquisition unit, 401... Imaging unit, 402... Read processing unit, 403... Image information requesting unit, 404... Image specifying processing unit, 405... Content acquisition processing unit, 406... Content output unit, 407... Processing result notifying unit, 408... Content acquisition number acquisition unit, 409... Content acquisition Number output unit, 410... Unacquired content number acquisition unit, 411... Unacquired content number output unit, 412... Positioning unit, 413... Content selection unit

Claims (15)

A computer that controls a user terminal equipped with an imaging unit,
A first processing unit for performing reading processing to read the user identification information identifying a user who is a printed material was distributed to the captured image of the printed material,
Specifying an image contained in the captured image of the printed material have use specific information used to identify the image with the user identification information associated with the content an image formed on the printed matter A second processing unit that performs specific processing,
The reading process and of the specific processing, a program to function actually performed and the result of the processing as a notification unit for notifying the recording device in association with the user identification information. The computer,
It also functions as a third processing unit that performs an acquisition process for acquiring the content associated with the image,
The notification unit notifies the recording device of the result of the process actually performed among the reading process, the specific process, and the acquisition process in association with the user identification information.
The program according to claim 1. The image is also associated with destination information indicating a destination requesting the content associated with the image,
The third processing unit, according to claim 2 for performing the process of requesting the content associated with the image to a destination indicated by the destination information corresponding to the specific image by said specific processing as the acquisition process Program of. The identification information is information representing the feature amount of the image,
The second processing unit, said collating the characteristic amount supplied from the feature amount of the image and information providing apparatus included in the captured image, claim 1 specifying the image contained in the captured image The program according to any one of 3 above. According to any one of the four claims 1 causes the computer to function as first output unit for outputting the number of content associated with the specified image by the specifying process is obtained in the past by the user terminal groups Program of. Claims 1 causes the computer to function as second output unit for outputting the number of the reading along with the user identification information read by the processing associated with the image being formed on a printed material and content that is not yet acquired the 5 The program according to any one of items. Causing the computer to function as a positioning unit that measures the position of the user terminal,
The program according to any one of claims 1 to 6 , wherein the notification unit notifies the recording device of position information indicating the measured position in association with the user identification information. The computer,
Causing the information providing device to function as a requesting unit for specifying information used for specifying an image formed on the printed matter together with the user identification information, the image being associated with the content,
A plurality of contents are associated with the image,
The third processing unit, according to claim 2 performs a process for obtaining any of the content selected from among said plurality of contents corresponding to the specific image by said specific processing as the acquisition process or The program according to 3 . Causing the computer to function as a positioning unit that measures the position of the user terminal,
The program according to claim 8 , wherein the third processing unit performs, as the acquisition processing, processing for acquiring content selected according to the measured position from the plurality of contents. The shooting department,
A first processing unit for performing reading processing to read the user identification information identifying a user who is a printed material was distributed to the captured image of the printed material,
Specifying an image contained in the captured image of the printed material have use specific information used to identify the image with the user identification information associated with the content an image formed on the printed matter A second processing unit that performs specific processing,
Among the reading process and the specific processing, the user terminal and a notifying unit that notifies actually performed and the result of the processing in the recording apparatus in association with the user identification information. A third processing unit that performs acquisition processing for acquiring the content associated with the image;
Of the reading process, the specifying process, and the acquiring process, a notification unit that notifies the recording device of the result of the process actually performed in association with the user identification information.
The user terminal according to claim 10, further comprising: In response to a request from the user terminal to the user identification information identifying the user is a print distribution destination was photographed image or Ra読 viewed take reading processing of the printed material, and is formed on the printed material together with the user identification information A transmission unit that transmits the identification information used to identify the image associated with the content to the user terminal,
The reading process from the user terminal, the specifying process of specifying the image included in the captured image of the printed matter performed by the user terminal using the specifying information transmitted, and the specifying process Of the acquisition processing for acquiring the content associated with the image, a recording unit that records the result of the processing actually performed in association with the user identification information. A user terminal according to claim 10 or 11 ,
An information processing system comprising: the recording device according to claim 12 . The recording device includes a first acquisition unit that acquires an attribute of a user to whom the printed matter on which the user identification information is formed is distributed.
The information processing system according to claim 13 , wherein the recording unit of the recording device records the user identification information and the attribute of the user acquired for the user identification information in association with each other. The information processing system according to claim 13 or 14 , wherein the recording unit of the recording device records the attribute of the image identified by the identification process in association with the result of the identification process.

Images