JP6610192B2 - Information processing apparatus, information processing system, and program - Google Patents

Description

  The present invention relates to an information processing apparatus, an information processing system, and a program.

  Techniques for providing content to users are known. For example, in Patent Document 1, when a mobile phone with a built-in camera captures a document provided with a media marker, digital media in which a link is associated with the media marker that appears in the captured photograph (content in the present invention) A technique for displaying is disclosed.

JP 2011-135556 A

  Using the technology of Patent Document 1, for example, a pamphlet is read by a scanner device, and a pamphlet is received by associating a specific area of the read pamphlet image as a media marker with a URL to a related web page. Users can be directed to a web page. In the technique of Patent Document 1, in order to perform this association, it is necessary for the user to display a pamphlet image on a screen and to perform an operation of pointing a specific position of the pamphlet image on the screen.

This operation is easier to perform as the screen for displaying the brochure image is larger.For example, when performing this operation by displaying the brochure image on the operation panel of the scanner device that has read the brochure, the operation panel is compared to a display such as a personal computer. Because it is often small, work is difficult.
Therefore, an object of the present invention is to designate a specific area of an image even when an operation for pointing to a specific position of the image is not performed on a screen on which an image read from a medium is displayed.

An information processing apparatus according to claim 1 of the present invention includes an acquisition unit that acquires a second image read from a first medium on which a first image is formed, and the first image is acquired from the acquired second image. As a third image representing a specific area , the first image having a size, shape, number, or arrangement corresponding to any of the size of the first image, the size of the first medium, or the size of the second image. An extraction unit that extracts three images; and a destination of content associated with the third image to a user terminal that has captured the region of the first image of the second medium on which the first image is formed The notification device that notifies information includes a transmission unit that transmits the destination information and a third image that is associated with the destination information and extracted by the extraction unit.

An information processing apparatus according to a second aspect of the present invention includes an acquisition unit that acquires a second image read from a first medium on which a first image is formed , and the first image from the acquired second image. An extraction unit that extracts a plurality of third images each representing a specific area of the first image, and a user terminal that has captured the area of the first image of the second medium on which the first image is formed. A transmission unit that transmits the destination information and the third image that is associated with the destination information and extracted by the extraction unit to a notification device that notifies the destination information of the content associated with the three images; A transmission unit configured to transmit the extracted third images and the common destination information associated with the plurality of third images .

An information processing apparatus according to a third aspect of the present invention includes an acquisition unit that acquires a second image read from a first medium on which a first image is formed , and the first image from the acquired second image. An extraction unit for extracting a third image representing a specific area of the first image, and the third image for the user terminal that has captured the area of the first image of the second medium on which the first image is formed. A transmission unit that transmits the destination information and a third image that is associated with the destination information and extracted by the extraction unit to a notification device that notifies the destination information of the associated content; A boundary recognizing unit that recognizes the boundary from the second image when the second image on which the boundary image representing the image is acquired is obtained, and the extracting unit displays the image delimited by the recognized boundary and wherein the extracting as the third image That.

An information processing apparatus according to a fourth aspect of the present invention includes an acquisition unit that acquires a second image read from a first medium on which a first image is formed , and the first image from the acquired second image. An extraction unit for extracting a third image representing a specific area of the first image, and the third image for the user terminal that has captured the area of the first image of the second medium on which the first image is formed. The notification device that notifies the destination information of the associated content includes a transmission unit that transmits the destination information and the third image that is associated with the destination information and extracted by the extraction unit, The unit includes a boundary recognition unit that acquires a fourth image read from a third medium on which a boundary image representing the boundary of the region is drawn, and recognizes the boundary from the acquired fourth image, The extractor is delimited by the recognized boundary And extracts an image as the third image.

An information processing apparatus according to claim 5 of the present invention includes an acquisition unit that acquires a second image read from a first medium on which a first image is formed , and the first image from the acquired second image. A first receiving unit that receives an operation for selecting one of the displayed candidates and an extracting unit that extracts a third image representing a specific area of the image; a display unit that displays a plurality of destination information candidates; And a notification device that notifies destination information of content associated with the third image to a user terminal that has captured the region of the first image of the second medium on which the first image is formed A transmission unit that transmits the destination information selected by the operation received by the first reception unit from the destination information, and the third image that is associated with the destination information and extracted by the extraction unit; be characterized in that it comprises a .

An information processing apparatus according to a sixth aspect of the present invention includes an acquisition unit that acquires a second image read from a first medium on which a first image is formed , and the first image from the acquired second image. An extraction unit that extracts a third image representing a specific area of the image, and when the second image in which the destination information of the content associated with the third image is drawn is acquired from the second image A destination information recognition unit that recognizes information, and a notification device that notifies the destination information to a user terminal that has captured the region of the first image of the second medium on which the first image is formed. And a transmission unit that transmits the destination information and the third image associated with the destination information and extracted by the extraction unit .

An information processing device according to claim 7 of the present invention includes an acquisition unit that acquires a second image read from a first medium on which a first image is formed , and the first image from the acquired second image. An extraction unit for extracting a third image representing a specific area of the first image, and the third image for the user terminal that has captured the area of the first image of the second medium on which the first image is formed. The notification device that notifies the destination information of the associated content includes a transmission unit that transmits the destination information and the third image that is associated with the destination information and extracted by the extraction unit, The unit includes a destination information recognition unit that acquires a fifth image read from the fourth medium on which the destination information is drawn, and recognizes the destination information from the acquired fifth image. , sending recognized the destination information And butterflies.

An information processing apparatus according to claim 8 of the present invention includes an acquisition unit that acquires a second image read from a first medium on which a first image is formed , and the first image from the acquired second image. An extraction unit that extracts a third image representing a specific area of the image, a storage unit that stores the acquired second image in a predetermined storage location, and a second medium on which the first image is formed The notification device that notifies the destination information of the content associated with the third image to the user terminal that has captured the region of the first image is associated with the destination information and the destination information and is extracted. A transmission unit that transmits the third image extracted by the transmission unit, the transmission unit transmitting information indicating the destination of the request for the second image stored in the storage location as the destination information. And

According to a ninth aspect of the present invention, there is provided an information processing apparatus in which a second image read from a first medium on which a first image is formed is stored in a storage location, and a storage designated by a user among the plurality of storage locations. An acquisition unit that acquires the second image stored in a place, an extraction unit that extracts a third image representing a specific area of the first image from the acquired second image, and the first image The notification device for notifying the destination information of the content associated with the third image to the user terminal that has captured the region of the first image of the formed second medium. A transmission unit that transmits the third image associated with the destination information and extracted by the extraction unit; the second image stored in the storage location where the acquired second image is stored; Information indicating the destination of the request is sent to the destination Characterized in that it comprises a transmitter which transmits a broadcast.

An information processing system according to a tenth aspect of the present invention is based on the information processing apparatus according to any one of the first to ninth aspects, the destination information and the third image transmitted from the information processing apparatus. And a notification device that notifies the destination information to the user terminal that has captured the second medium and acquired the third image .

According to an eleventh aspect of the present invention, there is provided a program that causes a computer to acquire a second image read from a first medium on which a first image is formed, and to acquire the first from the acquired second image. A third image representing a specific area of the image has a size, shape, number or arrangement according to any of the dimensions of the first image, the first medium, or the second image. Content associated with the third image for an extraction unit that extracts the third image and a user terminal that has captured the region of the first image of the second medium on which the first image is formed The notification device for notifying the destination information functions as a transmission unit that transmits the destination information and the third image associated with the destination information and extracted by the extraction unit .
According to a twelfth aspect of the present invention, there is provided a program that causes a computer to acquire a second image that is read from a first medium on which a first image is formed, and the first image from the acquired second image. An extraction unit that extracts a plurality of third images each representing a specific region of the image, and a user terminal that has captured the region of the first image of the second medium on which the first image is formed A transmission unit that transmits the destination information and the third image that is associated with the destination information and extracted by the extraction unit to a notification device that notifies the destination information of the content associated with the third image. Then, the plurality of extracted third images and the common destination information associated with the plurality of third images are caused to function as a transmission unit.
According to a thirteenth aspect of the present invention, there is provided a program that causes a computer to acquire a second image read from a first medium on which a first image is formed, and the first from the acquired second image. An extraction unit that extracts a third image representing a specific area of the image; and the third image for a user terminal that has captured the area of the first image of the second medium on which the first image is formed. A transmission unit that transmits the destination information and a third image that is associated with the destination information and extracted by the extraction unit to a notification device that notifies the destination information of the content associated with A program for functioning as a boundary recognition unit for recognizing a boundary from the second image when the second image on which a boundary image representing a boundary is drawn is acquired, wherein the extraction unit is recognized Delimited by the boundary And extracts an image as the third image.

According to a fourteenth aspect of the present invention, there is provided a program for causing a computer to acquire a second image read from a first medium on which a first image has been formed, and to acquire the first from the acquired second image. An extraction unit that extracts a third image representing a specific area of the image; and the third image for a user terminal that has captured the area of the first image of the second medium on which the first image is formed. For notifying the destination information of the content associated with the destination information and functioning as a transmission unit that transmits the destination information and the third image associated with the destination information and extracted by the extraction unit In the program, the acquisition unit acquires a fourth image read from a third medium on which a boundary image representing a boundary of the region is drawn, and the computer is acquired from the acquired fourth image. Recognize boundaries To function as a boundary recognition unit that, the extraction unit may extract an image separated by recognized the boundary as the third image.
According to a fifteenth aspect of the present invention, there is provided a program that causes a computer to acquire a second image read from a first medium on which a first image is formed, and to acquire the first from the acquired second image. An extraction unit that extracts a third image representing a specific area of the image; a display unit that displays a plurality of candidates for destination information; and a first unit that accepts an operation for a user to select one of the displayed candidates. A notification for notifying the destination information of the content associated with the third image to the receiving terminal and the user terminal that has captured the region of the first image of the second medium on which the first image is formed A transmission unit that transmits, to the device, destination information selected by the operation received by the first reception unit from the destination information, and a third image that is associated with the destination information and extracted by the extraction unit As machine Characterized in that to.
According to a sixteenth aspect of the present invention, there is provided a program that causes a computer to acquire a second image read from a first medium on which a first image is formed, and to acquire the first from the acquired second image. An extraction unit that extracts a third image representing a specific region of the image; and when the second image in which destination information of content associated with the third image is drawn is acquired from the second image A destination information recognition unit that recognizes the destination information, and a notification device that notifies the destination information to a user terminal that has captured the region of the first image of the second medium on which the first image is formed. It is made to function as a transmission part which transmits the said destination information recognized and the 3rd image matched with the said destination information and extracted by the said extraction part.

According to a seventeenth aspect of the present invention, there is provided a program for causing a computer to acquire a second image read from a first medium on which a first image is formed, and to acquire the first from the acquired second image. An extraction unit that extracts a third image representing a specific area of the image; and the third image for a user terminal that has captured the area of the first image of the second medium on which the first image is formed. For notifying the destination information of the content associated with the destination information and functioning as a transmission unit that transmits the destination information and the third image associated with the destination information and extracted by the extraction unit In the program, the acquisition unit acquires a fifth image read from a fourth medium on which the destination information is drawn, and recognizes the destination information from the acquired fifth image. Destination information To function as a recognition unit, and the transmission unit, and transmits recognized the destination information.
According to an eighteenth aspect of the present invention, there is provided a program that causes a computer to acquire a second image that is read from a first medium on which a first image is formed, and the first image from the acquired second image. An extraction unit for extracting a third image representing a specific area of the image; a storage unit for storing the acquired second image in a predetermined storage location; and a second medium on which the first image is formed. Of these, the notification device that notifies the destination information of the content associated with the third image to the user terminal that has captured the region of the first image is associated with the destination information and the destination information, and A transmission unit that transmits the third image extracted by the extraction unit, and that functions as a transmission unit that transmits information indicating a destination of a request for the second image stored in the storage location as the destination information. Features.
According to a nineteenth aspect of the present invention, a computer stores a second image read from a first medium on which a first image is formed in a storage location, and a user designates the plurality of storage locations. An acquisition unit that acquires the second image stored in a storage location; an extraction unit that extracts a third image representing a specific region of the first image from the acquired second image; and the first image To the notification device that notifies the destination information of the content associated with the third image to the user terminal that has captured the region of the first image of the second medium on which the image is formed. A transmission unit that transmits a third image associated with the destination information and extracted by the extraction unit, the second image stored in the storage location where the acquired second image was stored Table of destinations for requests It characterized in that to function as a transmission unit for transmitting the information as the destination information.

According to the inventions according to claims 1 to 19 , a specific area of an image can be designated even if an operation for pointing to a specific position of the image is not performed on a screen on which an image read from a medium is displayed. Can do.
According to the invention which concerns on Claim 1, 11 , the 3rd image matched with each dimension of this invention can be extracted.
According to the second and twelfth aspects of the present invention, it is possible to make it easier to provide specific content to the user than when only one third image is extracted.
According to the inventions according to claims 3, 4 , 13, and 14 , the user can determine the boundary of the third image by handwriting.
According to the fifth and fifteenth aspects of the present invention, the destination information can be transmitted simply by the user performing an operation of selecting from a plurality of candidates .
According to the invention of 請 Motomeko 6,7,16,17, the user can specify the destination information by hand.
According to the eighth and 18th aspects of the invention, the destination information can be specified by storing the second image in the storage location.
According to the ninth and nineteenth aspects of the present invention, the user can specify the destination information by specifying the storage location of the second image.

The figure showing the whole structure of the content provision system which concerns on 1st Example. The figure showing the hardware constitutions of a content delivery apparatus The figure showing the hardware constitutions of an image reading apparatus Diagram showing hardware configuration of server device A diagram showing the hardware configuration of a smartphone Diagram showing functional configuration realized by each device The figure showing an example of the area image extracted The figure showing an example of the displayed candidate of several destination information A figure showing an example of the displayed input field The figure showing an example of the generated correspondence file A figure showing an example of stored destination information A figure showing an example of a photographed image A diagram showing an example of displayed content The figure showing an example of the operation | movement procedure of each apparatus in a matching process The figure showing an example of the operation | movement procedure of each apparatus in a content provision process The figure showing the functional structure which each apparatus of 2nd Example implement | achieves. The figure showing an example of the screen for accepting continuous reading operation The figure showing an example of the two media from which an image is read continuously The figure showing the functional structure which each apparatus of 3rd Example implement | achieves. The figure showing an example of the screen for accepting storage location designation operation The figure showing the example of the area image extracted by a modification The figure showing the example of the medium from which a region image is extracted in a modification The figure showing the example of the medium from which a region image is extracted in a modification The figure showing the functional composition which each device of a modification realizes The figure showing an example of the screen on which the storage location of the read image is displayed The figure showing an example of the functional composition which the image reading device of a modification realizes The figure showing an example of the functional structure which the content provision system of a modification implement | achieves

[1] First Example FIG. 1 shows an overall configuration of a content providing system 1 according to a first example. The content providing system 1 is a system that provides content such as a web page and a video to a user who uses the system. The content providing system 1 includes a communication line 2, a content distribution device 10, an image reading device 20, a server device 30, and a smartphone 40.

  The communication line 2 includes, for example, a mobile communication network and the Internet, and is a system that mediates data exchange between devices. Each device provided in the content providing system 1 is connected to the communication line 2 by wire or wirelessly. The content distribution device 10 stores content such as web pages and moving images, and distributes such content in response to a request from an external device. In the present embodiment, the content distribution apparatus 10 distributes the content A1 “Web page of XX prefecture sightseeing spot” shown in FIG.

  The image reading apparatus 20 includes an image scanner, and performs a process of reading an image formed on a medium such as paper. The image reading apparatus 20 is an example of the “information processing apparatus” in the present invention. The image reading device 20 is used, for example, by a person who provides the content A1 to the user or a person who receives a work request from the person. In this embodiment, these workers set the medium P1 on which the pamphlet image B1 representing the “XX prefecture pamphlet” shown in FIG. 1 is formed in the image reading device 20 and perform the image reading work. To do. The image reading device 20 transmits an image reading result (details will be described later) to the server device 30.

  The server device 30 is a device that notifies destination information of content provided to the user in the present system to a user terminal used by the user. The server device 30 is an example of the “notification device” in the present invention. The content destination information is information indicating a destination from which content is requested when the content is acquired, and is, for example, a URL (Uniform Resource Locator) of a web page. In the present embodiment, the server device 30 notifies the user terminal of the URL of the content A1 as destination information. The server device 30 stores this destination information in association with the image reading result transmitted from the image reading device 20.

  The smartphone 40 is a device used by a user and is an example of the “user terminal” in the present invention. The smartphone 40 includes a touch screen and a digital camera. In the present embodiment, it is assumed that the above-described worker distributes the medium P2 on which the pamphlet image B1 is formed to the user as with the medium P1. When the user operates the smartphone 40 to display the image of the medium P2 on the touch screen, the smartphone 40 captures the displayed medium P2 and requests the server device 30 for destination information using the captured result.

  The server device 30 compares the photographing result by the smartphone 40 and the image reading result by the image reading device 20, and if it is determined that the photographed image and the read image are common, this request is made. In response, the smartphone 40 is notified of the destination information of the content A1 stored in association with the image reading result. The smartphone 40 requests content from the destination indicated by the notified destination information, acquires the content A1, and displays it on the touch screen. In the content providing system 1, content is provided to the user in this way.

  FIG. 2 shows a hardware configuration of the content distribution apparatus 10. The content distribution apparatus 10 is a computer that includes a central processing unit (CPU) 11, a random access memory (RAM) 12, a read only memory (ROM) 13, a storage 14, and a network interface card (NIC) 15. . The CPU 11 controls the operation of each unit by executing a program stored in the ROM 13 or the storage 14 using the RAM 12 as a work area. The storage 14 is a storage medium such as a hard disk drive (HDD) or a solid state drive (SSD), and stores data and programs used by the CPU 11 for control. The NIC 15 includes a communication circuit and communicates with an external device via the communication line 2.

  FIG. 3 shows a hardware configuration of the image reading apparatus 20. The image reading device 20 is a computer including a CPU 21, a RAM 22, a ROM 23, a storage 24, a NIC 25, an image scanner 26, and an operation panel 27. The CPU 21 to the NIC 25 are common to the hardware of the same name shown in FIG. The image scanner 26 includes an image sensor, and optically reads an image represented on the surface of the medium from a sheet-like medium. This image includes an image formed by an image forming apparatus, an image drawn by a human hand, and the like. The image scanner 26 supplies image data representing the read image to the CPU 21. The operation panel 27 includes a touch screen, displays information indicating the state of the device itself, the state of processing, and the like, and displays an image such as an operator to accept a user operation.

  FIG. 4 shows the hardware configuration of the server device 30. The server device 30 is a computer that includes a CPU 31, a RAM 32, a ROM 33, a storage 34, and a NIC 35. These are common to the hardware of the same name shown in FIG.

  FIG. 5 shows a hardware configuration of the smartphone 40. The smartphone 40 is a computer that includes a CPU 41, a RAM 42, a ROM 43, a storage 44, a NIC 45, a touch screen 46, and a digital camera 47. The CPU 41, the RAM 42, and the ROM 43 are common to the hardware of the same name shown in FIG. The storage 44 is a storage medium such as a flash memory, and stores data and programs used by the CPU 41 for control. The storage 44 stores a content acquisition program that realizes the function of the device itself to acquire content.

  The NIC 45 has a communication circuit for mobile communication, and communicates with an external device via the communication line 2 that performs wireless communication. The touch screen 46 includes a display and a touch panel provided on the surface of the display, displays an image, and accepts an operation from a user. The digital camera 47 includes a lens, an image sensor, and the like, and takes a photograph. The digital camera 47 supplies photograph data representing the photographed photograph to the CPU 41.

The functions described below are realized by the CPU of each device provided in the content providing system 1 executing a program to control each unit.
FIG. 6 shows a functional configuration realized by each device. The content distribution device 10 includes a content reading unit 101 and a content transmission unit 102. The image reading apparatus 20 includes an image reading unit 201, a read image acquisition unit 202, a region image extraction unit 203, a destination information candidate registration unit 204, a destination information candidate display unit 205, a selection operation reception unit 206, an input A column display unit 207, an input operation reception unit 208, a destination information association unit 209, and a destination information / region image transmission unit 210 are provided. The server device 30 includes a first feature value calculation unit 301, a destination information / region image storage unit 302, a second feature value calculation unit 303, a region image specification unit 304, and a destination information notification unit 305. The smartphone 40 includes an imaging unit 401, a destination information request unit 402, a content request unit 403, and a content display unit 404.

  The image reading unit 201 of the image reading apparatus 20 reads an image from a medium on which an image is formed. The image reading unit 201 reads an image from a distribution medium distributed to the user, such as a brochure, a flyer, a poster, or a catalog. In these distribution media, images representing common pictures and sentences are formed. The image is formed by printing or copying, for example. In this embodiment, it is assumed that an image is formed on a medium by printing, and an image formed on the medium is referred to as a “print image” (this is merely an example, and image formation is not limited to printing). An image read by the image reading unit 201 from a medium on which a print image is formed is referred to as a “read image”. The print image is an example of the “first image” in the present invention, and the read image is an example of the “second image” in the present invention.

  The print image may be formed on the entire medium or may be formed on a part of the medium. In the latter case, a plurality of print images may be formed on the medium. For example, a brochure image B1 is formed as a print image on the medium P1 shown in FIG. When the image reading unit 201 reads an image from a medium on which a print image is formed, the image reading unit 201 supplies image data representing the read image obtained as a result of the reading to the read image acquisition unit 202.

  The read image acquisition unit 202 of the image reading apparatus 20 acquires a read image read from a medium on which a print image is formed. The read image acquisition unit 202 is an example of the “acquisition unit” in the present invention, and the medium on which the print image is formed is an example of the “first medium” in the present invention. The read image acquisition unit 202 acquires an image represented by the image data supplied from the image reading unit 201 as a read image. The read image acquisition unit 202 supplies the acquired read image to the region image extraction unit 203.

  The area image extraction unit 203 of the image reading apparatus 20 extracts an area image representing a specific area of the print image from the read image acquired by the read image acquisition unit 202. The region image extraction unit 203 is an example of the “extraction unit” in the present invention, and the region image is an example of the “third image” in the present invention. For example, when the read image read from the medium P1 illustrated in FIG. 1 is acquired, the region image extraction unit 203 displays an image representing a specific region of the brochure image B1 that is a print image formed on the medium P1. Extract as an image.

  FIG. 7 shows an example of the extracted region image. 7A shows the medium P1 on which the brochure image B1 shown in FIG. 1 is formed, and FIG. 7B shows the read image D1 read from the medium P1. In this embodiment, it is assumed that the brochure image B1 is formed on the entire medium P1, and the read image D1 is an image obtained by reading the entire medium P1. Therefore, the read image D1 represents the entire pamphlet image B1.

  In this embodiment, the area image extraction unit 203 applies the maximum square area C1 that is located on the uppermost side of the maximum square area (the largest one of the square areas) included in the read image D1. The represented image is extracted as a region image, and a region image E1 shown in FIG. 7C is generated. This area image E1 is an image representing the square area C2 of the brochure image B1 shown in FIG. When the region image is extracted in this way, the region image extraction unit 203 supplies the extracted region image to the destination information association unit 209.

  The destination information candidate registration unit 204 of the image reading apparatus 20 registers destination information candidates. For example, the above-described operator operates an external device such as a personal computer to create registration data in which content names and destination information are associated with each other, and transmits the registration data to the image reading device 20. This registration data is supplied to the destination information candidate registration unit 204, and the destination information candidate registration unit 204 registers the content name and destination information represented by the registration data by storing the supplied registration data in its own device.

  The destination information candidate display unit 205 of the image reading apparatus 20 displays the destination information candidates registered by the destination information candidate registration unit 204. The destination information candidate display unit 205 reads and displays the destination information candidates registered by the destination information candidate registration unit 204 when the user performs an operation for associating the destination information with the region image.

  FIG. 8 shows an example of the displayed destination information candidates. In FIG. 8, a management screen for destination information displayed on the operation panel 27 is shown. The destination information candidate display unit 205 displays a plurality of destination information candidates F1 for accessing each of a plurality of contents with names such as “C01”, and a selection button G1 corresponding to each URL. doing. Further, the destination information candidate display unit 205 displays a character string “Select a content name / URL and press a setting button” and a setting button G2. The destination information candidate display unit 205 supplies the selection button G1 and setting button G2 coordinates, the content name displayed in association with each selection button G1, and a plurality of destination information candidates F1 to the selection operation reception unit 206. .

  The selection operation accepting unit 206 of the image reading apparatus 20 is a function that accepts an operation for the user to select one of the displayed plurality of destination information candidates, and is an example of the “first accepting unit” in the present invention. When any of the coordinates of the supplied selection button G1 is operated by the user and the setting button G2 is subsequently operated, the selection operation receiving unit 206 corresponds to the operated selection button G1. Is accepted as an operation to select destination information (URL in this example). The selection operation reception unit 206 supplies the content name and URL for which the selection operation has been received to the destination information association unit 209.

The input field display unit 207 of the image reading apparatus 20 displays an input field for the user to input a character string.
FIG. 9 shows an example of the displayed input field. In FIG. 9, as in FIG. 8, a management screen for destination information displayed on the operation panel 27 is shown. The input field display unit 207 displays an input field H1 for inputting a content name and URL, a keyboard image G3, and a setting button G4. The input field display unit 207 notifies the input operation reception unit 208 of the contents and coordinates of each key of the keyboard image G3 and the coordinates of the setting button G4.

  The input operation accepting unit 208 of the image reading apparatus 20 is a function that accepts an operation in which a user inputs a character string representing destination information, and is an example of the “second accepting unit” in the present invention. The input operation accepting unit 208 determines the character string input by the user's operation based on the supplied coordinates of each key, and when the setting button G4 is subsequently operated, the operation is changed to the content name and the destination. This is accepted as an operation for inputting a character string representing information (URL in this example). The input operation accepting unit 208 supplies the content name and URL for which the character string input operation has been accepted to the destination information associating unit 209. In addition to the example of FIG. 9, for example, input by a software keyboard capable of flicking may be accepted, or if the image reading apparatus includes a physical keyboard, input by that may be accepted.

  The destination information association unit 209 of the image reading apparatus 20 associates the region image extracted by the region image extraction unit 203 with the destination information specified by the user. For example, the worker described above sequentially performs a task of reading a medium and a task of selecting or inputting destination information. When the region image is supplied from the region image extraction unit 203, the destination information association unit 209 receives the content name and URL supplied from the selection operation reception unit 206 or the input operation reception unit 208 and the region image. Associate. Specifically, the destination information association unit 209 generates an association file in which, for example, the file name of the region image, the content name, and the URL are associated with each other.

Specifically, the destination information association unit 209 generates an association file in which, for example, the file name of the region image, the content name, and the URL are associated with each other.
FIG. 10 shows an example of the generated association file. In the example of FIG. 10, the file name of the area image “R01.jpg”, the content name “C01”, and the URL “http...” Are associated with each other. This URL is the URL of the content A1 shown in FIG. The destination information association unit 209 supplies the generated association file and the supplied region image, content name, and URL to the destination information / region image transmission unit 210.

  The destination information / region image transmission unit 210 of the image reading apparatus 20 notifies the destination information and region image associated with each other by the destination information association unit 209 to the user terminal (smart phone 40 in this embodiment). To the notification device (the server device 30 in this embodiment). The destination information / region image transmission unit 210 is an example of the “transmission unit” in the present invention. The destination information / region image transmission unit 210 stores a destination (IP (Internet Protocol) address or the like) of the server device 30 in advance, and is associated with the association file supplied from the destination information association unit 209. Each information (region image, content name, URL) is transmitted to the server device 30.

  When the URL supplied from the selection operation reception unit 206 is associated with the region image, the destination information / region image transmission unit 210 is selected from the URL, that is, the operation received by the selection operation reception unit 206. When the destination information is transmitted and the content name and URL supplied from the input operation accepting unit 208 are associated with the region image, the URL, that is, input from the operation accepted by the input operation accepting unit 208 Sends the destination information indicated by the character string.

  Each piece of information transmitted by the destination information / region image transmission unit 210 is supplied to the first feature amount calculation unit 301 of the server device 30. The first feature amount calculation unit 301 calculates a feature amount from the region image transmitted from the image reading device 20. The first feature quantity calculation unit 301 calculates local feature quantities such as SIFT (Scale-Invariant Feature Transform), PCA (Principal Component Analysis) -SIFT, SURF (Supeed-Up. Robust Features), and the like. The first feature amount calculation unit 301 supplies the calculated feature amount to the destination information / region image storage unit 302 together with each piece of supplied information (region image, content name, URL).

  The destination information / region image storage unit 302 of the server device 30 stores the destination information and the region image transmitted from the image reading device 20 in association with each other. The destination information / region image storage unit 302, when the feature amount calculated by the first feature amount calculation unit 301 satisfies the predetermined condition, the region image for which the feature amount is calculated and the destination information associated with the region image. And the content name are stored. As the determined condition, for example, a condition that is satisfied when the number of feature points whose feature values are calculated is equal to or greater than a threshold value is used. The smaller the number of feature points, the lower the probability that a region image will be recognized from the feature amount. Therefore, the destination information / region image storage unit 302 has an insufficient recognition probability for a region image whose number of feature points is less than the threshold value. The area image, the destination information, and the content name are not stored.

  FIG. 11 shows an example of stored destination information and the like. The destination information / region image storage unit 302 stores the feature amount calculated for the region image in association with the file name, content name, and URL of the region image associated in the example of FIG. Yes. In the example of FIG. 11, feature amounts, content names, and URLs are stored in association with three region images “R01.jpg”, “R02.jpg”, and “R03.jpg”, respectively. These pieces of information are referred to from an area image specifying unit 304 described later.

The photographing unit 401 of the smartphone 40 has a function of generating photographed image data representing a photographed image. For example, the photographing unit 401 photographs a medium on which a print image is formed and generates photographed image data representing the image of the medium.
FIG. 12 shows an example of a captured image of the medium. FIG. 12 illustrates an example in which the medium P2 (see FIG. 12A) on which the brochure image B1 illustrated in FIG. 1 is formed is captured. This medium P2 is distributed to a user who uses the smartphone 40, for example, by an operator who distributes the medium P1 shown in FIG.

  When the user points the lens of the digital camera 47 of the smartphone 40 toward the medium P2, the medium image K1 including the medium P2 is displayed in the image display area J1 on the shooting screen of the touch screen 46 (see FIG. 12B). . On this shooting screen, a shooting button G5 is displayed. When the user performs an operation of pressing the shooting button G5, the shooting unit 401 takes a picture of the displayed medium image K1 and shows the shot image. Generate image data. The imaging unit 401 supplies the generated captured image data to the destination information request unit 402.

  The destination information request unit 402 of the smartphone 40 transmits the destination information associated with the area image if the area image of the print image formed on the medium is included in the image of the medium captured by the device itself. The server device 30 is requested to do so. In other words, when a region image is acquired by photographing by the photographing unit 401, the destination information request unit 402 requests transmission of destination information associated with the region image. When the photographed image data is supplied from the photographing unit 401, the destination information requesting unit 402 transmits request data representing the photographed image data and the request described above to the server device 30. This request data is supplied to the second feature amount calculation unit 303 of the server device 30.

  The 2nd feature-value calculation part 303 of the server apparatus 30 calculates a feature-value from the image image | photographed by the user terminal (this example smart phone 40). When the above-described request data is supplied, the second feature amount calculation unit 303 calculates a feature amount in an image represented by the request data, that is, an image photographed by the user terminal. At this time, the second feature amount calculation unit 303 calculates a feature amount (for example, SIFT or the like) common to that calculated by the first feature amount calculation unit 301, and the feature amount of each feature point is calculated as a region image. This is supplied to the specifying unit 304.

  For example, when the smartphone 40 performs an operation of pressing the shooting button G5 in the state illustrated in FIG. 12B, the feature amount of each feature point of the medium image K1 representing the entire pamphlet image B1 is obtained by the second feature amount calculation unit 303. Calculated. Since the medium image K1 includes the entire region corresponding to the region image E1 shown in FIG. 7C, among the feature points whose feature values are calculated by the second feature value calculation unit 303 at this time. Each of the feature points calculated by the first feature amount calculation unit 301 from the region image E1 is included in the feature points. Hereinafter, the feature amount calculated by the first feature amount calculation unit 301 is referred to as a first feature amount, and the feature amount calculated by the second feature amount calculation unit 303 is referred to as a second feature amount.

  The area image specifying unit 304 of the server device 30 specifies an area image included in the medium image captured by the user terminal. When the second feature amount is supplied from the second feature amount calculation unit 303, the region image specifying unit 304 receives the second feature amount and the first feature amount stored in the destination information / region image storage unit 302. Are compared, and the region image included in the image for which the second feature amount is calculated is specified. For example, if the second feature quantity includes a portion whose difference from a certain first feature quantity is less than the threshold value, the area image specifying unit 304 specifies the area image from which the first feature quantity is calculated. .

  For example, when the second feature amount of the medium image K1 is calculated, the region image specifying unit 304 uses the first feature amount calculated for the region image E1 included in the brochure image B1 shown in the medium image K1 as the second feature amount. Since it is included in the feature amount, the region image E1 is specified as the region image included in the medium image captured by the user terminal. The area image specifying unit 304 supplies the specified area image to the destination information notification unit 305.

  The destination information notification unit 305 of the server device 30 notifies the requesting user terminal of the destination information requested from the user terminal. When the region image is supplied from the region image specifying unit 304, the destination information notifying unit 305 uses the URL stored in the destination information / region image storage unit 302 in association with the region image as a user terminal (in this example). It is specified as the destination information requested from the smartphone 40). When the area image E1 is specified, the destination information notification unit 305 specifies the URL associated with “R01.jpg” representing the area image E1 in FIG. 11 as the destination information.

  The destination information notification unit 305 notifies the user terminal of the requested destination information by reading the specified URL from the destination information / region image storage unit 302 and transmitting the URL to the transmission source of the request data described above. As described above, the server device 30 specifies the print image of the medium (for example, the medium P2) on which the print image (for example, the brochure image B1) is formed based on the destination information and the region image transmitted from the server device 30. The destination information of the content (for example, content A1 shown in FIG. 1) associated with the region image that is an image of the specific region is notified to the user terminal that captured the region. This medium P2 is an example of the “second medium” in the present invention.

  The transmitted destination information is supplied to the content request unit 403 of the smartphone 40. The content request unit 403 requests content from the destination indicated by the destination information notified from the server device 30. For example, the content request unit 403 transmits an HTTP (Hypertext Transfer Protocol) request with the notified URL as a destination, thereby requesting the content whose location is specified by the URL.

  In the present embodiment, the transmitted HTTP request is received by the content distribution apparatus 10 storing content such as a web page and supplied to the content reading unit 101. The content reading unit 101 reads the content requested from the user terminal. Specifically, the content reading unit 101 reads the content represented by the supplied HTTP request and supplies the content to the content transmission unit 102. The content transmission unit 102 transmits the content read by the content reading unit 101 to the user terminal that is the request source of the content.

  In the present embodiment, first, an HTML (HyperText Markup Language) document is transmitted from the content distribution apparatus 10 to the smartphone 40, and the HTML document is supplied to the content display unit 404 and the content request unit 403. The content request unit 403 requests an image file or a moving image file described in the HTML document, and the content distribution apparatus 10 transmits the requested file to the smartphone 40 in response thereto. The content display unit 404 displays the content transmitted from the content distribution device 10 in response to the request from the device itself.

  FIG. 13 shows an example of the displayed content. In the example of FIG. 13, the content display unit 404 displays the content A <b> 1 shown in FIG. 1 on the touch screen 46 of the smartphone 40. Each device provided in the content providing system 1 performs an association process for associating the region image with the destination information and a content providing process for providing the content to the user based on the above configuration.

  FIG. 14 shows an example of the operation procedure of each device in the association process. This operation procedure is started when an operator who performs work related to the association processing performs an image reading operation on the image reading apparatus 20. First, the image reading device 20 (image reading unit 201) reads an image from a medium on which a print image is formed (step S11). Next, the image reading device 20 (read image acquisition unit 202) acquires a read image read from the medium (step S12). Subsequently, the image reading device 20 (region image extraction unit 203) extracts a region image from the acquired read image (step S13).

  In the example of FIG. 14, next, the image reading apparatus 20 (destination information candidate display unit 205) displays candidates for destination information as shown in FIG. 8, for example (step S14). Subsequently, the image reading apparatus 20 (selection operation reception unit 206) receives a user operation for selecting one of the displayed candidates (step S15). Then, the image reading apparatus 20 (destination information association unit 209) associates the extracted region image with the selected destination information (step S16). In step S14, the image reading device 20 (input field display unit 207) displays an input field as shown in FIG. 9, for example. In step S15, the image reading device 20 (input operation receiving unit 208) displays destination information. A user's operation for inputting a character string representing the character string may be accepted.

  The image reading device 20 (destination information / region image transmission unit 210) transmits the destination information and the region image associated in step S16 to the server device 30 (step S17). The server device 30 (first feature amount calculation unit 301) calculates a first feature amount from the transmitted region image (step S21). The server device 30 (destination information / region image storage unit 302) calculates the destination information and the region image transmitted in step S17 and the region information when the calculated first feature amount satisfies the condition. The first feature value is stored in association with each other (step S22).

  In the above association processing, in order to extract the region image, for example, a read image is displayed on the screen, and the operator designates a range of the region image by performing an operation indicating a specific position on the screen. Conceivable. However, if this operation is to be performed by the image reading device 20, the read image is displayed on the operation panel 27. However, the operation panel often has a smaller screen than a display such as a personal computer, for example. Is difficult to do. In this embodiment, the region image extraction unit 203 extracts a region image, so that an operation for pointing a specific position of the image on the screen on which the image read from the medium is displayed is not performed. A specific area is specified.

  In this embodiment, the destination information is transmitted simply by the user performing an operation of selecting from a plurality of destination information candidates (for example, those shown in FIG. 8), and the user can input the destination information (for example, FIG. 9). The destination information is transmitted simply by performing an operation of inputting a character string into the item (1).

  FIG. 15 shows an example of the operation procedure of each device in the content providing process. This operation procedure is started when a user using the smartphone 40 performs an operation of photographing a medium. First, the smartphone 40 (photographing unit 401) photographs a medium shown on the screen (step S31). Next, the smartphone 40 (destination information request unit 402) transmits request data for requesting destination information to the server device 30 based on the image of the captured medium (step S32).

  The server device 30 (second feature amount calculator 303) calculates the second feature amount of the image represented by the transmitted request data, that is, the image of the medium photographed in step S31 (step S33). Next, the server device 30 (region image specifying unit 304) obtains the region image included in the image represented by the calculated second feature amount and the request data transmitted from the first feature amount stored in the own device. Specify (step S34). Subsequently, the server device 30 (destination information notification unit 305) reads out the destination information stored in association with the specified region image (step S35), and transmits the read destination information to the smartphone 40 (step S36). ).

  The smartphone 40 (content request unit 403) requests content from the destination indicated by the destination information notified from the server device 30 (in this example, the URL of the content distribution device) (step S41). The content distribution apparatus 10 (content reading unit 101) first reads the content requested in step S41 (step S42). Next, the content distribution apparatus 10 (content transmission unit 102) transmits the read content to the smartphone 40 (step S43). The smartphone 40 (content display unit 404) displays the content transmitted in step S43 (step S44).

[2] Second Embodiment Hereinafter, a second embodiment of the present invention will be described focusing on differences from the first embodiment. In the second embodiment, destination information is specified by a method different from that in the first embodiment.
FIG. 16 illustrates a functional configuration realized by each device included in the content providing system 1a according to the second embodiment. The content distribution device 10, the server device 30, and the smartphone 40 include each unit illustrated in FIG. The image reading apparatus 20a includes a continuous reading operation reception unit 211, a boundary recognition unit 212, and a destination information recognition unit 213 instead of the units from the destination information candidate registration unit 204 to the input operation reception unit 208 illustrated in FIG. Prepare.

The continuous reading operation reception unit 211 receives a continuous reading operation by a user. The continuous reading operation is an operation for instructing a process of acquiring a read image continuously from two media.
FIG. 17 shows an example of a screen for receiving a continuous reading operation. In FIG. 17, a sentence “Do you want to read images in succession?” And a button image G11 including a character string “Yes” are displayed. When the user performs an operation of pressing the button image G11, the continuous reading operation reception unit 211 receives this operation as a continuous reading operation and notifies the reading image acquisition unit 202 to that effect.

  When the read image acquisition unit 202 of the present embodiment receives the notification from the continuous reading operation reception unit 211, it acquires the read image read from the first medium supplied from the image reading unit 201 after that. When the read image read from the second medium is acquired and supplied to the region image extraction unit 203, it is supplied to the boundary recognition unit 212 and the destination information recognition unit 213. In this embodiment, it is assumed that the user reads an image from the next two media after performing a continuous reading operation.

  FIG. 18 shows an example of two media on which images are continuously read. FIG. 18A shows the first medium P11. A print image B2 is formed on the medium P11. A photograph of the XX park is represented in the area C21 of the print image B2, a photograph of the XX Memorial is represented in the area C22, and a photograph of the XX plateau is represented in the area C23. In FIG. 18B, the second medium P12 is shown. The medium P12 includes line segments L11, L12, L13, L14, L15, and L16 representing the boundaries of the areas C21, C22, and C23 of the print image B2 (referred to as “boundary line segment L10” if these are not distinguished), and each area. Destination information U21, U22, U23 associated with the area image shown in FIG. The boundary line segment L10 is a line segment drawn by the user, and the destination information U21, U22, U23 is a character string drawn by the user.

  In this way, the boundary image representing the boundary of a specific area of the print image and the destination information are drawn on the medium P12. The medium P12 on which the boundary image is drawn is an example of the “third medium” of the present invention, and the medium P12 is an example of the “fourth medium” of the present invention because the destination information is drawn. It is also. The read image acquisition unit 202 acquires a read image including a boundary image from the medium P12 as an example of the third medium, and the acquired read image is an example of the “fourth image” in the present invention. . The read image acquisition unit 202 acquires a read image including destination information from the medium P12 as an example of the fourth medium, and the acquired read image is an example of the “fifth image” of the present invention. It is.

  The boundary recognition unit 212 recognizes the boundary of a specific area from the read image acquired by the read image acquisition unit 202. Specifically, when the read image read from the second medium is supplied from the read image acquisition unit 202, the boundary recognition unit 212 extracts, for example, a line segment having a length longer than the threshold from the read image. . For this line segment extraction, a known method such as a method using Hough transform or a method using thinning may be used. The boundary recognition unit 212 extracts line segments representing the respective boundary line segments L10 from the read image read from the medium P12 illustrated in FIG. 18B, and recognizes these extracted line segments as boundaries of a specific region. To do. The boundary recognition unit 212 supplies boundary information representing the recognized boundary (for example, a mathematical expression representing each extracted line segment in the XY coordinate system set for the read image) to the region image extraction unit 203.

  The region image extraction unit 203 extracts an image divided by the boundary recognized by the boundary recognition unit 212 as a region image. When the read image read from the first medium is supplied from the read image acquisition unit 202 and the boundary information is supplied from the boundary recognition unit 212, the region image extraction unit 203 displays the boundary image from the read image using the boundary information. An image divided by the line segments to be extracted is extracted as a region image. For example, in the example of FIG. 18, the region image extraction unit 203 extracts, as a region image, an image represented in a region C21 divided by line segments L11, L12, L13, and L14 from the read image of the medium P11. Further, the region image extraction unit 203 extracts, as a region image, an image represented in the region C22 divided by the line segments L12, L13, L15, and L16 from the read image of the medium P11, and the line segments L13, L14, and L15. , The image represented in the region C23 delimited by L16 is extracted as a region image.

  The destination information recognition unit 213 recognizes destination information from the read image acquired by the read image acquisition unit 202. Specifically, when the read image read from the second medium is supplied from the read image acquisition unit 202, the destination information recognition unit 213 uses a known OCR (Optical character recognition) technique from the read image. The character string is extracted by using and the extracted character string is recognized as the destination information. In the example of FIG. 18, the destination information recognition unit 213 recognizes the destination information U21, U22, and U23 shown in FIG. The destination information recognition unit 213 supplies the recognized destination information and arrangement information indicating the arrangement of the destination information to the destination information association unit 209. The arrangement information is represented by, for example, the coordinates of the head of each destination information in the XY coordinate system set for the read image described above.

  The destination information association unit 209 associates the destination information recognized by the destination information recognition unit 213 with the region image extracted by the region image extraction unit 203. The destination information associating unit 209 identifies the region image of the region where the destination information is arranged based on the arrangement information supplied from the destination information recognition unit 213, and associates the destination information with the region image. In the example of FIG. 18, the destination information association unit 209 associates the region image of the region C21 with the destination information U21, associates the region image of the region C22 with the destination information U22, and associates the region image of the region C23 with the region image. Corresponds to destination information U23.

  The destination information / region image transmission unit 210 transmits the destination information recognized by the destination information recognition unit 213 to the server device 30 as described above. As described above, according to this embodiment, as in the medium P12 illustrated in FIG. 18B, the boundary of the area image is determined by the user's handwriting, and the destination information is specified by the user's handwriting.

[3] Third Embodiment Hereinafter, a third embodiment of the present invention will be described focusing on differences from the first and second embodiments. In the third embodiment, destination information is specified by a method different from that in the first and second embodiments.
FIG. 19 shows a functional configuration realized by each device included in the content providing system 1b of the third embodiment. The content providing system 1b includes a content storage device 10b, an image reading device 20b, a server device 30, and a smartphone 40. The content storage device 10b includes a content reading unit 101, a content transmission unit 102, and a content storage unit 103 illustrated in FIG. The image reading apparatus 20b includes a storage operation reception unit 214 and a read image storage unit 215 instead of the units from the destination information candidate registration unit 204 to the input operation reception unit 208 illustrated in FIG.

  The content storage unit 103 of the content storage device 10b stores the read image read from the medium by the image reading device 20b. The content reading unit 101 reads content from the storage location where the content storage unit 103 stores the read image. In this embodiment, destination information such as a URL is determined in association with this storage location. For example, when the smartphone 40 requests content with a URL associated with a certain storage location as destination information, the content reading unit 101 reads the read image from the storage location associated with the URL, thereby reading the content. The image is transmitted as content to the smartphone 40 by the content transmission unit 102.

The storage operation reception unit 214 receives an operation for designating the storage location of the read image acquired by the read image acquisition unit 202.
FIG. 20 shows an example of a screen for accepting a storage location designation operation. In FIG. 20, the text “Please specify the location to store the scanned image.”, The names of the storage locations “Folder 01”, “Folder 02”, “Folder 03”,... A button image G12 including the character string is displayed. These storage locations represent the storage locations of the content storage device 10b. When the user selects any storage location (in this example, “Folder 02” is selected) and presses the button image G12, the storage operation reception unit 214 performs this operation as a storage location designation operation. Received and notified of the designated storage location to the read image storage unit 215.

  The read image acquisition unit 202 of this embodiment supplies the read image supplied from the image reading unit 201 to the region image extraction unit 203 and the read image storage unit 215. The read image storage unit 215 stores the read image acquired by the read image acquisition unit 202 in a predetermined storage location. The read image storage unit 215 is an example of the “storage unit” in the present invention. For example, the read image storage unit 215 stores the read image in a storage location notified from the storage operation reception unit 214, that is, a storage location designated by an operation received by the storage operation reception unit 214.

  The read image storage unit 215 transmits storage request data representing the read image supplied from the read image acquisition unit 202 and storage in the designated storage location to the content storage device 10b. Upon receiving this request data, the content storage unit 103 of the content storage device 10b stores the read image represented by the received request data in the requested storage location. The read image storage unit 215 stores the read image in the storage location thus determined. The read image storage unit 215 supplies the destination information association unit 209 with information (for example, URL) indicating the destination of a request for acquiring the read image from the storage location where the read image is stored.

  The destination information associating unit 209 associates the information supplied from the read image storage unit 215 with the region image as destination information, so that the destination information / region image transmission unit 210 stores the read image in the read image storage unit 215. Information indicating a destination of a request for acquiring the read image from the storage location is transmitted to the server device 30 as destination information. Thereby, the read image stored in the content storage device 10b is provided to the user as the content. According to this modification, as described above, the destination information is specified by storing the read image in the storage location.

[4] Modifications Each of the above-described embodiments is merely an example of the present invention and may be modified as follows. Moreover, you may implement combining each Example and each modification as needed.

[4-1] Region Image Extraction Method The region image extraction unit 203 extracts an image represented in the maximum square region as a region image in the embodiment, but extracts an image represented in a different region as the region image. May be.
FIG. 21 shows an example of a region image extracted in this modification. In FIG. 21A, the region image extraction unit 203 extracts an image represented in a rectangular region C11 as a region image. In FIG. 21B, the region image extraction unit 203 extracts the image represented in the elliptical region C12 as a region image.

  In FIG. 21C, the region image extraction unit 203 extracts images respectively represented by a plurality of square regions C13, C14, C15, and C16 as region images. In this case, the area image extraction unit 203 extracts, for example, four square areas that are determined in size and touch the corners of the read image (may be separated by a determined distance). When the region image extraction unit 203 extracts a plurality of region images in this manner, the destination information association unit 209 associates different destination information with each of the plurality of extracted region images, for example. Thereby, a plurality of contents are associated with one read image.

  Further, the destination information association unit 209 may associate common destination information with the plurality of extracted region images. In that case, the destination information / region image transmission unit 210 transmits the plurality of region images and the common destination information associated with each other. As a result, the content is requested for the common destination information regardless of which of the plurality of region images is captured by the user terminal, and compared with the case where only one region image is extracted, the specific content ( Content requested using common destination information) is easily provided to the user.

  In the embodiment, the area image extraction unit 203 extracts the area image represented by the maximum square area of the read image, that is, the area image having a size corresponding to the size of the read image, but the present invention is not limited to this. For example, the region image extraction unit 203 may extract a region image having a size corresponding to the size of the medium on which the print image is formed. Since the size of the medium is detected when the image reading unit 201 reads an image from the medium, for example, the region image extraction unit 203 determines the size of the region image using the detected size.

  FIG. 22 shows an example of a medium from which an area image is extracted in this modification. 22A, 22B, and 22C show media P21, P22, and P23, respectively, and the relationship between the dimensions of these media is P21 <P22 <P23. The region image extraction unit 203 extracts the image represented in the region C21 as a region image from the read image read from the medium P21. Further, the region image extraction unit 203 extracts the image represented in the region C22 from the read image read from the medium P22 as a region image, and the image represented in the region C23 from the read image read from the medium P23. Are extracted as region images.

  The relationship between the sizes of the regions shown in FIG. 22 is C21 <C22 <C23. The area image extraction unit 203 determines the size of the area image using, for example, a table in which the size of the medium is associated with the size of the area image. Thus, by extracting the area image having a size corresponding to the size of the medium, for example, when the same print image is formed on the medium in a size according to the size of the medium, the same print image is obtained. An image representing a region can be extracted as a region image.

  Note that the area image extraction unit 203 is not limited to the size of the read image or the medium, but, for example, according to the size of the print image (such as when the print image is formed with a predetermined size even if the size of the medium changes). A region image having a size may be extracted. In this case, for example, even if the dimensions of the print image are different, the ratio of the area image to the print image can be made constant. Further, the region image extraction unit 203 is not limited to the size according to each dimension described above, and may extract a region image having a shape according to each dimension. For example, the region image extraction unit 203 extracts a vertically long rectangular region image as the size of the read image increases.

Further, the area image extraction unit 203 may extract the number or arrangement of area images corresponding to the above-described dimensions.
FIG. 23 shows an example of a medium from which an area image is extracted in this modification. 23A, 23B, and 23C show read images D31, D32, and D33, respectively, and the relationship between the dimensions of these read images is D31 <D32 <D33. The area image extraction unit 203 extracts the images represented in the three area groups C31 as area images from the read image D31.

  Further, the region image extraction unit 203 extracts the images represented by the five region groups C32 from the read image D32 as region images, and the images represented by the nine region groups C33 from the read image D33 as region images. Extract. In the example of FIG. 23, the number of area images is increased as the size of the read image increases, and the area images are included in the captured image captured by the smartphone by arranging the area images so as to be separated from each other. A region image is extracted so as to be easily captured. In the present modification, as described above, the region image extraction unit 203 uses the dimensions of the medium, the print image, or the read image to extract the region image, so that the region image corresponding to these dimensions is extracted.

[4-2] Specification of storage location In the third embodiment, the read image stored in the storage location of the content storage device by the image reading device is provided as the content, but the read image stored in the storage location by another device is provided. It may be provided as content.
FIG. 24 illustrates a functional configuration realized by each device included in the content providing system 1c according to the present modification. The content providing system 1c includes a content storage device 10c, an image reading device 20c, a server device 30, and a smartphone 40. The content storage device 10c includes each unit common to the content storage device 10b illustrated in FIG. 19 and a storage location / destination information notification unit 104. The image reading apparatus 20c includes a storage location display unit 216 and a storage location designation operation reception unit 217 instead of the units from the destination information candidate registration unit 204 to the input operation reception unit 208 illustrated in FIG.

  The content storage unit 103 of the content storage device 10c stores a read image read from a medium on which a print image is formed (an example of the “first medium” in the present invention). The reading of the read image may be performed by the image reading device 20c or may be performed by a device that reads other images. The storage location / destination information notification unit 104 notifies the storage location where the read image is stored by the content storage unit 103 and the destination information associated with the storage location in response to an inquiry from an external device.

  The storage location display unit 216 of the image reading device 20c displays the storage location of the read image stored in the content storage device 10c. When an operation for displaying the storage location is performed by the user, the storage location display unit 216 inquires the content storage device 10c about the storage location and displays the storage location notified in response thereto. The storage location designation operation accepting unit 217 accepts a designation operation for designating any of the storage locations displayed by the storage location display unit 216.

  FIG. 25 shows an example of a screen on which the storage location of the read image is displayed. In FIG. 25, the storage location display unit 216 displays the text “Please specify the location where the scanned image to be provided as content is stored.”, “Folder 01”, “Folder 02”, “Folder 03”, ... And the button image G13 including the character string “OK” are displayed. When the user selects one of the storage locations and presses the button image G13, the storage location designation operation receiving unit 217 accepts this operation as a storage location designation operation, and acquires the specified storage location as a read image. Notification to the unit 202. At that time, the storage location designation operation accepting unit 217 notifies the destination information association unit 209 of the destination information notified together with the storage location.

  The read image acquisition unit 202 acquires a read image stored in a storage location designated by the user among a plurality of storage locations. When the storage location is notified from the storage location designation operation receiving unit 217, the read image acquisition unit 202 requests the content storage device 10c for a read image stored in the storage location. Upon receiving this request, the content storage unit 103 of the content storage device 10c supplies the requested read image to the request source (the image reading device 20c in this example). The read image acquisition unit 202 acquires the read image supplied in this way.

  When the region image extraction unit 203 extracts the region image from the read image thus acquired, the destination information association unit 209 receives the extracted region image and the destination information notified from the storage location designation operation reception unit 217, that is, the acquisition. The destination information of the storage location of the read image that has been read is associated. As a result, the destination information / region image transmission unit 210 transmits, as destination information, information indicating the destination of the read image request stored in the storage location where the read image acquired by the read image acquisition unit 202 was stored. . As described above, in this modification, the destination information is specified by the user specifying the storage location of the read image.

[4-3] Selection of Destination Information Candidates In the example of FIG. 8, both the content name and the URL are displayed on the screen for selecting a destination information candidate, but only one of them may be displayed. However, even when only the content name is displayed, it is assumed that the destination information is already associated with the content or is associated later. In this way, if the content name is displayed as information indicating the candidate destination information, the destination information is also selected by selecting the content name.

In the first embodiment, the destination information candidate display unit 205 displays all registered destination information candidates. However, the present invention is not limited to this.
FIG. 26 illustrates an example of a functional configuration realized by the image reading apparatus 20d according to the present modification. The image reading apparatus 20d includes a destination information candidate selection unit 218 in addition to the units illustrated in FIG. The destination information candidate selection unit 218 is supplied with the destination information registered by the destination information candidate registration unit 204. The destination information candidate selection unit 218 selects a candidate to be presented to the user from among the supplied destination information candidates.

  The destination information candidate selection unit 218 is supplied with the read image acquired from the read image acquisition unit 202 and the region image extracted from the region image extraction unit 203. For example, the destination information candidate selection unit 218 recognizes characters from the supplied image using the OCR technique, and selects destination information indicating a destination to content related to the word represented by the recognized characters. For example, if destination information candidates for content related to each region from Hokkaido to Okinawa are registered, and the word “Tohoku” is recognized from the image, the destination information candidate selection unit 218 sets the destination information for the content related to the Tohoku region. Select a candidate.

  In this manner, the destination information candidate selection unit 218 selects a destination information candidate corresponding to the read image or the extracted region image, and supplies the selected destination information candidate to the destination information candidate display unit 205. . The destination information candidate display unit 205 displays the supplied destination information candidates (that is, the selected destination information candidates). This makes it easier for the user to select destination information candidates as much as the candidates related to the read image or the region image are displayed instead of displaying all the registered destination information candidates.

  The destination information candidate selection unit 218 is not limited to the above example, and may select a destination information candidate according to the current time, for example. For example, the destination information of content related to winter is selected in winter, and the destination information of content related to summer is selected in summer. In this case as well, the user can easily select destination information candidates as described above.

[4-4] Method for Recognizing Region Boundary and Destination Information The method for recognizing region boundary and destination information in the second embodiment is not limited to the above. For example, the destination information U21, U22, U23 shown in FIG. 18B is written on the medium P11 shown in FIG. 18A, and the destination information recognition unit 213 shown in FIG. 16 recognizes the destination information from the medium P11. The destination information recognition unit 213 of this modification recognizes the destination information from the read image when the read image on which the destination information is drawn is acquired. That is, this read image is an image read from a medium on which a print image on which destination information is drawn is formed, and is an example of the “second image” of the present invention described above. This destination information may be drawn by a user by hand, or may be drawn by an image forming apparatus, for example.

  If the line segment representing the boundaries of the areas C21, C22, and C23 can be recognized on the medium P11, the boundary recognition unit 212 recognizes the boundary of each area from the read image of the medium P11. The boundary recognition unit 212 of this modification recognizes the boundary of a specific area from the read image when the read image on which the boundary image is drawn is acquired. That is, this read image is an image read from a medium on which a print image on which a boundary image is drawn is formed, and is an example of the “second image” of the present invention described above. The boundary image may be drawn by a user by hand, or may be drawn by an image forming apparatus, for example.

  In this case, the area image extraction unit 203 extracts the area image after removing the destination information written on the medium P11 from the read image of the medium P11. For example, if the destination information is written with a pen of a specific color, the region image extraction unit 203 removes the destination information by interpolating pixels of that color with surrounding pixels. This eliminates the need to prepare a second medium as in the example of FIG.

  Further, the destination information written by the user may be a word or a sentence related to the content instead of the URL. In that case, the destination information recognition unit 213 searches the recognized character string with a search service provided on the Internet or the like, and recognizes the URL of the web page obtained as the search result as the destination information. The destination information recognizing unit 213 may recognize, for example, the first displayed search result as the destination information, or display the upper one of the search results and select the one selected by the user from there. It may be recognized as destination information. As a result, the destination information is recognized without the user knowing the URL of the content.

[4-5] Extraction Method of Region Image The method by which the region image extraction unit 203 extracts a region image is not limited to the above. For example, the area image extraction unit 203 may recognize a specific object from the read image and extract an area image including the recognized object. The specific object is, for example, a person's face, a person's whole body, a vehicle, a building, an animal, or the like. The region image extraction unit 203 learns the tendency of the feature amount from a sample image including a specific object such as a person or a vehicle (what is called a cascade type classifier or Haar classifier), for example. A rectangular image circumscribing the contour of the recognized object is extracted as a region image.

  When a user shoots a print image formed on a medium, if the print image includes a person, a vehicle, or the like, the user often shoots such an object to fit on a screen. According to this modification, since the region image including such an object is extracted, the region image is more likely to be included in the image captured by the user as compared with the case where the object is not recognized.

[4-6] Apparatus for Implementing Each Unit The apparatus for implementing each unit illustrated in FIG. 6 and the like is not limited to that described above.
FIG. 27 shows an example of a functional configuration realized by the content providing system according to this modification. In FIG. 27, a content providing system 1e including a content distribution device 10, an image reader 20e, an image reading device 20f, a server device 30e, and a smartphone 40e is shown.

  The image reader 20e includes the image scanner 26 shown in FIG. 3, and realizes the image reading unit 201 shown in FIG. The image reader 20e supplies the read image (the read image in this example) to the image reading device 20f. The image reading device 20f has nine functions from the read image acquisition unit 202 to the destination information / region image transmission unit 210 shown in FIG. Thus, the image reading device may be externally attached to the image scanner.

  The server device 30e includes each unit excluding the second feature amount calculation unit 303 among the units included in the server device 30 illustrated in FIG. On the other hand, the smartphone 40e includes a second feature amount calculation unit 303 in addition to the units included in the smartphone 40 illustrated in FIG. As described above, the second feature amount may be calculated on the user terminal side. In addition to the example shown in FIG. 27, various devices may implement the units shown in FIG.

[4-7] Category of the Invention The present invention relates to an information processing apparatus such as the above-described image reading apparatus, a notification apparatus such as a server apparatus, a user terminal such as a smartphone (not limited to a smartphone, a tablet terminal or a notebook personal computer). In addition to a content distribution device and a content storage device, the information processing system can be regarded as an information processing system such as a content providing system including these devices. In addition, the present invention can be understood as an information processing method for realizing the processing performed by these devices, or as a program for causing a computer that controls these devices to function. This program may be provided in the form of a recording medium such as an optical disk in which it is stored, or may be provided in the form of being downloaded to a computer via a network such as the Internet, installed and made available for use. May be.

DESCRIPTION OF SYMBOLS 1 ... Content provision system, 2 ... Communication line, 10 ... Content delivery apparatus, 20 ... Image reading apparatus, 30 ... Server apparatus, 40 ... Smartphone, 101 ... Content reading part, 102 ... Content transmission part, 103 ... Content storage part, DESCRIPTION OF SYMBOLS 104 ... Storage location / destination information notification part, 201 ... Image reading part, 202 ... Read image acquisition part, 203 ... Area image extraction part, 204 ... Destination information candidate registration part, 205 ... Destination information candidate display part, 206 ... Selection operation Reception unit, 207 ... Input field display unit, 208 ... Input operation reception unit, 209 ... Destination information association unit, 210 ... Destination information / region image transmission unit, 211 ... Continuous reading operation reception unit, 212 ... Boundary recognition unit, 213 ... Destination information recognition unit, 214 ... Storage operation reception unit, 215 ... Read image storage unit, 216 ... Storage location display unit, 217 ... Storage location designation operation reception 218 ... Destination information candidate selection unit 301 ... First feature quantity calculation unit 302 ... Destination information / region image storage unit 303 ... Second feature amount calculation unit 304 ... Region image specification unit 305 ... Destination information notification , 401 ... Shooting unit, 402 ... Destination information request unit, 403 ... Content request unit, 404 ... Content display unit

Claims (19)

An acquisition unit for acquiring a second image read from the first medium on which the first image is formed;
As a third image representing a specific region of the first image from the acquired second image , the size of the first image, the size of the first medium, or the size of the second image An extraction unit for extracting the third image having a size, shape, number or arrangement according to the
A notification device that notifies destination information of content associated with the third image to a user terminal that has captured the region of the first image of the second medium on which the first image is formed. An information processing apparatus comprising: a transmission unit configured to transmit destination information and a third image associated with the destination information and extracted by the extraction unit. An acquisition unit for acquiring a second image read from the first medium on which the first image is formed;
An extraction unit that extracts a plurality of third images respectively representing specific regions of the first image from the acquired second image;
A notification device that notifies destination information of content associated with the third image to a user terminal that has captured the region of the first image of the second medium on which the first image is formed. A transmission unit that transmits destination information and a third image that is associated with the destination information and extracted by the extraction unit , and that corresponds to the extracted third images and the plurality of third images. An information processing apparatus comprising: a transmission unit that transmits the common destination information attached thereto . An acquisition unit for acquiring a second image read from the first medium on which the first image is formed;
An extraction unit that extracts a third image representing a specific region of the first image from the acquired second image;
A notification device that notifies destination information of content associated with the third image to a user terminal that has captured the region of the first image of the second medium on which the first image is formed. A transmission unit that transmits destination information and a third image that is associated with the destination information and extracted by the extraction unit ;
A boundary recognition unit for recognizing the boundary from the second image when the second image in which a boundary image representing the boundary of the region is drawn is acquired;
The extraction unit is an information processing apparatus that extracts an image divided by the recognized boundary as the third image . An acquisition unit for acquiring a second image read from the first medium on which the first image is formed;
An extraction unit that extracts a third image representing a specific region of the first image from the acquired second image;
A notification device that notifies destination information of content associated with the third image to a user terminal that has captured the region of the first image of the second medium on which the first image is formed. A transmission unit that transmits destination information and a third image that is associated with the destination information and extracted by the extraction unit ;
The acquisition unit acquires a fourth image read from a third medium on which a boundary image representing a boundary of the region is drawn;
A boundary recognition unit for recognizing the boundary from the acquired fourth image;
The extraction unit is an information processing apparatus that extracts an image divided by the recognized boundary as the third image . An acquisition unit for acquiring a second image read from the first medium on which the first image is formed;
An extraction unit that extracts a third image representing a specific region of the first image from the acquired second image;
A display unit for displaying a plurality of destination information candidates;
A first accepting unit that accepts an operation for the user to select one of the displayed candidates;
A notification device that notifies destination information of content associated with the third image to a user terminal that has captured the region of the first image of the second medium on which the first image is formed. Information including: a destination unit selected from an operation received by the first receiving unit among the destination information, and a third unit associated with the destination information and extracted by the extracting unit. Processing equipment. An acquisition unit for acquiring a second image read from the first medium on which the first image is formed;
An extraction unit that extracts a third image representing a specific region of the first image from the acquired second image;
A destination information recognition unit that recognizes the destination information from the second image when the second image in which the destination information of the content associated with the third image is drawn is acquired;
A notification device for notifying a pre Kiate destination information to the user terminal which photographs the region of the first image of the second medium in which the first image has been formed, and recognized the destination information, the An information processing apparatus comprising: a transmission unit that transmits a third image associated with destination information and extracted by the extraction unit. An acquisition unit for acquiring a second image read from the first medium on which the first image is formed;
An extraction unit that extracts a third image representing a specific region of the first image from the acquired second image;
A notification device that notifies destination information of content associated with the third image to a user terminal that has captured the region of the first image of the second medium on which the first image is formed. A transmission unit that transmits destination information and a third image that is associated with the destination information and extracted by the extraction unit ;
The acquisition unit acquires a fifth image read from a fourth medium on which the destination information is drawn,
A destination information recognition unit for recognizing the destination information from the acquired fifth image;
The transmission unit is an information processing apparatus that transmits the recognized destination information . An acquisition unit for acquiring a second image read from the first medium on which the first image is formed;
An extraction unit that extracts a third image representing a specific region of the first image from the acquired second image;
A storage unit for storing the acquired second image in a predetermined storage location;
A notification device that notifies destination information of content associated with the third image to a user terminal that has captured the region of the first image of the second medium on which the first image is formed. A transmission unit that transmits destination information and a third image that is associated with the destination information and extracted by the extraction unit , and that represents information indicating a destination of a request for the second image stored in the storage location. An information processing apparatus comprising: a transmission unit that transmits the destination information . The second image read from the first medium on which the first image is formed is stored in the storage location, and the second image stored in the storage location designated by the user among the plurality of storage locations is acquired. An acquisition unit;
An extraction unit that extracts a third image representing a specific region of the first image from the acquired second image;
A notification device that notifies destination information of content associated with the third image to a user terminal that has captured the region of the first image of the second medium on which the first image is formed. A transmission unit that transmits destination information and a third image associated with the destination information and extracted by the extraction unit , and is stored in the storage location where the acquired second image is stored An information processing apparatus comprising: a transmission unit that transmits information indicating a destination of the request for the second image as the destination information . The information processing apparatus according to any one of claims 1 to 9 ,
Based on the destination information and the third image transmitted from the information processing device, a notification device that notifies the destination information to the user terminal that has captured the second image by capturing the second medium An information processing system comprising: Computer
An acquisition unit for acquiring a second image read from the first medium on which the first image is formed;
As a third image representing a specific region of the first image from the acquired second image , the size of the first image, the size of the first medium, or the size of the second image An extraction unit for extracting the third image having a size, shape, number or arrangement according to the
A notification device that notifies destination information of content associated with the third image to a user terminal that has captured the region of the first image of the second medium on which the first image is formed. The program for functioning as a transmission part which transmits destination information and the 3rd image matched with the said destination information and extracted by the said extraction part. Computer
An acquisition unit for acquiring a second image read from the first medium on which the first image is formed;
An extraction unit that extracts a plurality of third images respectively representing specific regions of the first image from the acquired second image;
A notification device that notifies destination information of content associated with the third image to a user terminal that has captured the region of the first image of the second medium on which the first image is formed. A transmission unit that transmits destination information and a third image that is associated with the destination information and extracted by the extraction unit , and that corresponds to the extracted third images and the plurality of third images. The program for functioning as a transmission part which transmits the attached said common destination information . Computer
An acquisition unit for acquiring a second image read from the first medium on which the first image is formed;
An extraction unit that extracts a third image representing a specific region of the first image from the acquired second image;
A notification device that notifies destination information of content associated with the third image to a user terminal that has captured the region of the first image of the second medium on which the first image is formed. A transmission unit that transmits destination information and a third image that is associated with the destination information and extracted by the extraction unit ;
A program for functioning as a boundary recognition unit for recognizing a boundary from the second image when the second image in which a boundary image representing a boundary of the region is drawn is acquired ,
The extraction unit is a program that extracts an image delimited by the recognized boundary as the third image . Computer
An acquisition unit for acquiring a second image read from the first medium on which the first image is formed;
An extraction unit that extracts a third image representing a specific region of the first image from the acquired second image;
A notification device that notifies destination information of content associated with the third image to a user terminal that has captured the region of the first image of the second medium on which the first image is formed. A program for functioning as a transmission unit that transmits destination information and a third image associated with the destination information and extracted by the extraction unit ,
The acquisition unit acquires a fourth image read from a third medium on which a boundary image representing a boundary of the region is drawn;
Causing the computer to function as a boundary recognition unit that recognizes the boundary from the acquired fourth image;
The extraction unit is a program that extracts an image delimited by the recognized boundary as the third image . Computer
An acquisition unit for acquiring a second image read from the first medium on which the first image is formed;
An extraction unit that extracts a third image representing a specific region of the first image from the acquired second image;
A display unit for displaying a plurality of destination information candidates;
A first accepting unit that accepts an operation for the user to select one of the displayed candidates;
A notification device that notifies destination information of content associated with the third image to a user terminal that has captured the region of the first image of the second medium on which the first image is formed. In order to function as a transmission unit that transmits destination information selected from the operation received by the first reception unit among the destination information and a third image that is associated with the destination information and extracted by the extraction unit. Program. Computer
An acquisition unit for acquiring a second image read from the first medium on which the first image is formed;
An extraction unit that extracts a third image representing a specific region of the first image from the acquired second image;
A destination information recognition unit that recognizes the destination information from the second image when the second image in which the destination information of the content associated with the third image is drawn is acquired;
A notification device for notifying a pre Kiate destination information to the user terminal which photographs the region of the first image of the second medium in which the first image has been formed, and recognized the destination information, the A program for causing a transmission unit to transmit a third image associated with destination information and extracted by the extraction unit. Computer
An acquisition unit for acquiring a second image read from the first medium on which the first image is formed;
An extraction unit that extracts a third image representing a specific region of the first image from the acquired second image;
A notification device that notifies destination information of content associated with the third image to a user terminal that has captured the region of the first image of the second medium on which the first image is formed. A program for functioning as a transmission unit that transmits destination information and a third image associated with the destination information and extracted by the extraction unit ,
The acquisition unit acquires a fifth image read from a fourth medium on which the destination information is drawn,
Causing the computer to function as a destination information recognition unit that recognizes the destination information from the acquired fifth image;
The transmission unit is a program for transmitting the recognized destination information . Computer
An acquisition unit for acquiring a second image read from the first medium on which the first image is formed;
An extraction unit that extracts a third image representing a specific region of the first image from the acquired second image;
A storage unit for storing the acquired second image in a predetermined storage location;
A notification device that notifies destination information of content associated with the third image to a user terminal that has captured the region of the first image of the second medium on which the first image is formed. A transmission unit that transmits destination information and a third image that is associated with the destination information and extracted by the extraction unit , and that represents information indicating a destination of a request for the second image stored in the storage location. The program for functioning as a transmission part which transmits as said destination information . Computer
The second image read from the first medium on which the first image is formed is stored in the storage location, and the second image stored in the storage location designated by the user among the plurality of storage locations is acquired. An acquisition unit;
An extraction unit that extracts a third image representing a specific region of the first image from the acquired second image;
A notification device that notifies destination information of content associated with the third image to a user terminal that has captured the region of the first image of the second medium on which the first image is formed. A transmission unit that transmits destination information and a third image associated with the destination information and extracted by the extraction unit , and is stored in the storage location where the acquired second image is stored A program for functioning as a transmission unit that transmits information indicating a destination of a request for a second image as the destination information .

Images