JP2014049017A - Terminal device, server device, communication system, information processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide electronic data highly convenient for a user by using a positional relationship between a plurality of barcodes.SOLUTION: A terminal device 100 photographs a subject including a plurality of barcodes. The terminal device 100 recognizes a URL recorded as a barcode for each barcode on the basis of the photographed image. The terminal device 100 specifies positions of the respective plurality of barcodes in the subject image obtained by the photographing. The terminal device 100 transmits the recognized plurality of URLs respectively to a server device 200 in association with coordinates representing the positions of the barcodes recording the URLs.

Description

  The present invention relates to a terminal device, a server device, a communication system, an information processing method, and a program. In particular, the present invention relates to a terminal device that can read a barcode, a server device that communicates with the terminal device, a communication system including the terminal device and the server device, an information processing method in the terminal device, an information processing method in the server device, and a terminal The present invention relates to a program for controlling a device and a program for controlling a server device.

  Conventionally, a barcode printed on a paper medium or the like is known. Regarding barcode reading, Patent Document 1 discloses a barcode reading apparatus that aims to read a plurality of barcodes printed on a single packaging box in association with each other. Also, Patent Document 2 discloses a barcode reader for the purpose of reading a plurality of barcodes printed on a medium.

  Conventionally, it has been considered to manually designate a physical document (that is, print document) portion and search a network for digital content corresponding to the designated portion. Patent Document 3 discloses the following configuration as a system for performing such processing. That is, the system of Patent Document 3 includes: (i) identifying a physical document; (ii) identifying an electronic copy of the identified physical document; and (iii) one or more pages of the physical document Identifying a portion of the page marked on a digitizing tablet that is continuously placed and aligned above (or below) the identified page; and (iv) the electronic of the physical document Searching for an electronic copy of the identified page from the copy; (v) automatically marking the user selected portion of the physical document on the received electronic page; and (vi) searching. Copying the marked portion of the marked electronic page to the operating system clipboard, and (vii) the clipboard contents by the user And a step of paste constant destination file.

  A QR code (registered trademark) is known as a two-dimensional barcode. Non-Patent Document 1 discloses that data to be encoded can be divided and expressed in a QR code. It is disclosed that the same data can be extracted from one QR code and a plurality of QR codes. Further, it is disclosed that information stored separately in a plurality of QR codes can be linked as one data. Up to 16 divisions are possible, and if this function is used, printing on a narrow and long area is possible.

JP 2004-240874 A JP-A-9-319821 Special table 2007-527039 gazette

“QR code dot com-QR code features”, [online], [search August 15, 2012], Internet <URL: http://www.qrcode.com/qrfeature.html>

  However, the techniques of Patent Documents 1 and 2 cannot identify the positional relationship between a plurality of printed barcodes. Further, the system of Patent Document 3 is not configured using a barcode. Non-Patent Document 1 has a configuration in which information indicated by one barcode is divided into a plurality of pieces of information using a plurality of barcodes, and only one piece of information can be obtained even if a plurality of barcodes are read.

  The present application has been made in view of the above-described problems, and a terminal device and a server device that can provide electronic data that is highly convenient for the user by using the positional relationship between a plurality of barcodes. A communication system, an information processing method, and a program.

  According to an aspect of the present invention, the terminal device includes an imaging unit that images a subject including a plurality of barcodes, a recognition unit that recognizes the first information recorded as a barcode for each barcode based on the imaging. , Specifying means for specifying the position of each of the plurality of barcodes in the subject image obtained by imaging, and each of the plurality of recognized first information, and the position of the barcode on which the first information is recorded Transmission means for transmitting to the server device in association with the expressed second information.

  Preferably, the terminal device transmits a plurality of pieces of first information and second information to the server device, and the plurality of pieces of third information specified by each of the plurality of pieces of first information in the server device. Is further provided with receiving means for receiving the electronic data generated by placing the second information from the server device and display means for displaying the received electronic data.

  According to another aspect of the present invention, the terminal device includes an imaging unit that images a subject including a plurality of barcodes, and a recognition unit that recognizes the first information recorded as a barcode for each barcode based on the imaging. And a determination means for determining the positional relationship between the plurality of barcodes based on the imaging, the plurality of recognized first information, and the second information representing the positional relationship are associated with each other and transmitted to the server device. Transmitting means.

  Preferably, the determination unit determines the position of each barcode image of the plurality of barcodes in the subject image obtained by imaging using a two-dimensional coordinate system. The determining means determines the positional relationship between the plurality of barcodes based on the coordinate values of the barcode images on one coordinate axis constituting the coordinate system.

  Preferably, the determination unit determines the positional relationship between the plurality of barcodes based on the coordinate values of the barcode images on both coordinate axes constituting the coordinate system.

  Preferably, the terminal device transmits a plurality of pieces of first information and second information to the server device, and the plurality of pieces of third information specified by each of the plurality of pieces of first information in the server device. Is further provided with receiving means for receiving the electronic data generated by placing the second information from the server device and display means for displaying the received electronic data.

  According to still another aspect of the present invention, the terminal device includes an imaging unit that images a plurality of barcodes, a recognition unit that recognizes the first information recorded as a barcode for each barcode based on the imaging, Based on the imaging, a determination unit that determines a positional relationship between a plurality of barcodes, a transmission unit that transmits a plurality of recognized first information items to the server device, and a server device, for each first information, Receiving means for receiving second information specified based on the first information, and display means for displaying a plurality of second information based on the determined positional relationship.

  Preferably, the determination unit determines the position of each barcode image of the plurality of barcodes in the subject image obtained by imaging using a two-dimensional coordinate system. The determining means determines the positional relationship between the plurality of barcodes based on the coordinate values of the barcode images on one coordinate axis constituting the coordinate system.

  Preferably, the determination unit determines the positional relationship between the plurality of barcodes based on the coordinate values of the barcode images on both coordinate axes constituting the coordinate system.

  When the further another situation of this invention is followed, a server apparatus communicates with the terminal device which images the to-be-photographed object containing a some barcode. The server device uses each of the plurality of first information obtained by recognizing the first information recorded as a barcode for each barcode as the first information in the subject image obtained by imaging. A receiving means for receiving from the terminal device in association with the second information representing the position of the recorded barcode; a judging means for judging the positional relationship between the plurality of barcodes based on the plurality of second information; And generating means for generating electronic data by arranging a plurality of third information specified by each of the plurality of first information using the determined positional relationship, and generating the generated electronic data with a terminal Transmitting means for transmitting to the apparatus.

  Preferably, the determination unit determines the position of each barcode image of the plurality of barcodes in the subject image using a two-dimensional coordinate system. The determining means determines the positional relationship between the plurality of barcodes based on the coordinate values of the barcode images on one coordinate axis constituting the coordinate system.

  Preferably, the determination unit determines the positional relationship between the plurality of barcodes based on the coordinate values of the barcode images on both coordinate axes constituting the coordinate system.

  When the further another situation of this invention is followed, a server apparatus communicates with the terminal device which images the to-be-photographed object containing a some barcode. The server device includes a plurality of first information obtained by recognizing the first information recorded as a barcode for each barcode, and second information representing a positional relationship between the plurality of barcodes. Receiving means from the terminal device, and generating means for generating electronic data by arranging a plurality of third information specified by each of the plurality of first information using the second information, Transmitting means for transmitting the generated electronic data to the terminal device.

  When the further another situation of this invention is followed, a communication system is provided with a terminal device and a server apparatus. The terminal device includes an imaging unit that images a subject including a plurality of barcodes, a recognition unit that recognizes first information recorded as a barcode for each barcode based on the imaging, and a subject image obtained by imaging And specifying means for specifying the position of each of the plurality of barcodes, and associating each of the plurality of recognized first information with the second information representing the position of the barcode in which the first information is recorded And transmitting means for transmitting to the server device. The server device associates each of the plurality of pieces of first information with the second information representing the position of the barcode on which the first information is recorded, and receives a plurality of second information from the terminal device. Based on the information, the determination means for determining the positional relationship between the plurality of barcodes and the plurality of third information specified by each of the plurality of first information are arranged using the determined positional relationship. By this, the production | generation means which produces | generates electronic data and the transmission means which transmits the produced | generated electronic data to a terminal device are included.

  When the further another situation of this invention is followed, a communication system is provided with a terminal device and a server apparatus. The terminal device includes an imaging unit that images a subject including a plurality of barcodes, a recognition unit that recognizes the first information recorded as a barcode for each barcode based on the imaging, a plurality of barcodes based on the imaging. A determination unit that determines the positional relationship between the codes; and a transmission unit that associates the plurality of recognized first information and the second information representing the positional relationship and transmits the second information to the server device. The server device receives the plurality of first information and the second information from the terminal device, and receives the plurality of third information specified by each of the plurality of first information as the second information. By arranging using the information, a generating unit that generates electronic data and a transmitting unit that transmits the generated electronic data to the terminal device are included.

  When the further another situation of this invention is followed, a communication system is provided with a terminal device and a server apparatus. The terminal device includes an imaging unit that images a plurality of barcodes, a recognition unit that recognizes the first information recorded as a barcode for each barcode based on the imaging, and a plurality of barcodes based on the imaging. Based on the first information, the determination means for determining the positional relationship, the transmission means for transmitting the plurality of recognized first information to the server device, and the server device for each first information Receiving means for receiving second information; and display means for displaying a plurality of second information based on the determined positional relationship. The server device includes receiving means for receiving a plurality of first information from the terminal device and transmitting means for transmitting the plurality of second information to the terminal device.

  When the further another situation of this invention is followed, the information processing method is performed in a terminal device. The information processing information includes a step of imaging a subject including a plurality of barcodes, a step of recognizing first information recorded as a barcode for each barcode based on imaging, and a subject image obtained by imaging Identifying the position of each of the plurality of barcodes, and associating each of the plurality of recognized first information with the second information representing the position of the barcode in which the first information is recorded, And transmitting to the server device.

  When the further another situation of this invention is followed, the information processing method is performed in a terminal device. The information processing information includes a step of imaging a subject including a plurality of barcodes, a step of recognizing the first information recorded as a barcode for each barcode based on the imaging, and a plurality of barcodes based on the imaging. A step of determining a positional relationship between each other, and a step of associating the plurality of recognized first information and the second information representing the positional relationship and transmitting the second information to the server device.

  When the further another situation of this invention is followed, the information processing method is performed in a terminal device. The information processing method includes a step of imaging a plurality of barcodes, a step of recognizing the first information recorded as a barcode for each barcode based on the imaging, and a position between the plurality of barcodes based on the imaging. A step of determining a relationship; a step of transmitting a plurality of pieces of recognized first information to the server device; and a second specified from the server device for each first information based on the first information Receiving information, and displaying a plurality of pieces of second information based on the determined positional relationship.

  If the further another situation of this invention is followed, the information processing method will be performed in the server apparatus which communicates with the terminal device which images the to-be-photographed object containing a some barcode. In the information processing method, each of a plurality of pieces of first information obtained by recognizing the first information recorded as a barcode for each barcode, the first information in the subject image obtained by imaging. A step of receiving from the terminal device in association with the second information representing the position of the barcode recorded, and a step of determining the positional relationship between the plurality of barcodes based on the plurality of second information; A step of generating electronic data by arranging a plurality of third information specified by each of the plurality of first information using the determined positional relationship, and generating the generated electronic data in a terminal device Transmitting.

  If the further another situation of this invention is followed, the information processing method will be performed in the server apparatus which communicates with the terminal device which images the to-be-photographed object containing a some barcode. The information processing method includes a plurality of first information obtained by recognizing the first information recorded as a barcode for each barcode, and second information representing a positional relationship between the plurality of barcodes. And a step of generating electronic data by arranging a plurality of third information specified by each of the plurality of first information using the second information, Transmitting the generated electronic data to the terminal device.

  If the further another situation of this invention is followed, a program will control a terminal device. The program includes a step of imaging a subject including a plurality of barcodes, a step of recognizing first information recorded as a barcode for each barcode based on the imaging, and a plurality of subject images obtained by imaging. A server device for identifying each position of the barcode and associating each of the plurality of recognized first information with the second information representing the position of the barcode in which the first information is recorded; To the processor of the terminal device.

  If the further another situation of this invention is followed, a program will control a terminal device. The program includes a step of imaging a subject including a plurality of barcodes, a step of recognizing the first information recorded as a barcode for each barcode based on the imaging, and a plurality of barcodes based on the imaging. The processor of the terminal device executes the step of determining the positional relationship, the step of associating the plurality of recognized first information and the second information representing the positional relationship, and transmitting to the server device.

  If the further another situation of this invention is followed, a program will control a terminal device. The program includes a step of imaging a plurality of barcodes, a step of recognizing first information recorded as a barcode for each barcode based on the imaging, and a positional relationship between the plurality of barcodes based on the imaging. A step of determining, a step of transmitting the plurality of recognized first information to the server device, and a second information specified based on the first information for each first information from the server device. The step of receiving and the step of displaying the plurality of pieces of second information based on the determined positional relationship are executed by the processor of the terminal device.

  When the further another situation of this invention is followed, a program controls the server apparatus which communicates with the terminal device which images the to-be-photographed object containing a some barcode. The program records each of the plurality of first information obtained by recognizing the first information recorded as a barcode for each barcode, in the subject image obtained by imaging. A step of receiving from the terminal device in association with the second information representing the position of the barcode, a step of determining a positional relationship between the plurality of barcodes based on the plurality of second information, A step of generating electronic data by arranging a plurality of third information specified by each of the first information using the determined positional relationship, and transmitting the generated electronic data to the terminal device Step is executed by the processor of the server device.

  When the further another situation of this invention is followed, a program controls the server apparatus which communicates with the terminal device which images the to-be-photographed object containing a some barcode. The program includes a plurality of first information obtained by recognizing the first information recorded as a barcode for each barcode, and a second information representing the positional relationship between the plurality of barcodes. Receiving from the terminal device, generating a plurality of pieces of third information specified by each of the plurality of pieces of first information using the second information, and generating electronic data Transmitting the electronic data to the terminal device to the processor of the server device.

  ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to provide electronic data with a high convenience for a user using the positional relationship between several barcodes.

It is the figure which showed the outline | summary of the communication system. It is a figure for demonstrating the 1st example of the process performed in a communication system. It is a figure for demonstrating the 2nd example of the process performed in a communication system. It is a figure for demonstrating the 3rd example of the process performed in a communication system. It is a figure showing the hardware constitutions of the terminal device. It is the figure which showed the hardware constitutions of the server apparatus. It is a functional block diagram for demonstrating the functional structure of a terminal device. It is a functional block diagram for demonstrating the functional structure of a server apparatus. It is a figure for demonstrating the case (1st example of FIG. 2 etc.) of determining the arrangement order of the content in the case of arranging the content which comprises electronic data in a line. It is a figure for demonstrating the case (the 2nd example of FIG. 3, the 3rd example of FIG. 4, etc.) of determining the arrangement | sequence order of the content in the case where the content which comprises electronic data is arranged two-dimensionally. It is a flowchart for demonstrating the flow of the process in a terminal device. It is a flowchart for demonstrating the flow of the process in a server apparatus. It is the flowchart which showed the flow of the process for producing | generating new_data [j] from data [i]. It is the flowchart which showed the flow of the process for producing | generating list [s] [t] from new_data [j]. It is a flowchart for demonstrating the flow of the process (layout process) which arrange | positions a some content and produces one electronic data. It is a functional block diagram for demonstrating the functional structure of a terminal device. It is a functional block diagram for demonstrating the functional structure of a server apparatus. It is a flowchart for demonstrating the flow of the process in a terminal device. It is a flowchart for demonstrating the flow of the process in a server apparatus. It is a functional block diagram for demonstrating the functional structure of a terminal device. It is a functional block diagram for demonstrating the functional structure of a server apparatus. It is a flowchart for demonstrating the flow of the process in a terminal device. It is a flowchart for demonstrating the flow of the process in a server apparatus. It is a figure for demonstrating the user interface displayed with a terminal device.

  The communication system according to each embodiment of the present invention will be described below with reference to the drawings. In the following description, the same reference numerals are assigned to the same members. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

  In the following description, a smart phone will be described as an example of a portable electronic device (hereinafter referred to as “terminal device”). However, the terminal device is not limited to a smartphone and may be any portable electronic device having a communication function. For example, the terminal device may be a mobile phone (such as a feature phone), a tablet terminal, a laptop computer, a digital camera, or a PDA (Personal Digital Assistant).

  In the following, “barcode” includes various barcodes such as a one-dimensional barcode, a two-dimensional barcode, and a three-dimensional barcode. An example of the two-dimensional barcode is a QR code (registered trademark). Note that the three-dimensional barcode is a temporal change of the two-dimensional barcode. In the following, a QR code will be described as an example of a barcode. For convenience of explanation, “QR code” is simply referred to as “bar code”.

  In the first and second embodiments, a configuration in which a server device to be described later performs a composition process of a plurality of contents will be described. In the third embodiment, a configuration in which a terminal device to be described later performs a composition process of a plurality of contents will be described. The difference between the first embodiment and the second embodiment will be described later.

[Embodiment 1]
<A. System overview>
FIG. 1 is a diagram showing an outline of a communication system. Referring to FIG. 1, in communication system 1, terminal device 100 and server device 200 are connected to each other via a network 300 so that they can communicate with each other.

  The server device 200 stores a plurality of contents in advance. In FIG. 1, for convenience of explanation, only one server device is shown, but the present invention is not limited to this. The communication system 1 may include a plurality of server devices, and the terminal device 100 may communicate with the plurality of server devices. Further, a configuration in which one server device (for example, a Web server) communicates with another server device (for example, a content server) based on a command from the terminal device 100 may be employed.

  The terminal device 100 has a barcode reader function for reading a barcode. The terminal device 100 captures a barcode with the camera of the terminal device 100 and recognizes information recorded as a barcode based on the captured image data. In the following description, it is assumed that the URL is recorded as a barcode.

<B. Outline of processing>
In the following, an outline of processing performed in the communication system 1 (the terminal device 100 and the server device 200) will be described by giving three examples. That is, three examples in which the communication system 1 can be used will be described.

(B1. First example)
FIG. 2 is a diagram for explaining a first example of processing performed in the communication system 1. With reference to FIG. 2, as shown in the state (A), the user of the terminal device 100 arranges a plurality of materials 710, 720, and 730 distributed at a school or the like in an order that the user can easily grasp. Bar codes 711, 721, and 731 are printed on the materials 710, 720, and 730, respectively.

  Note that the material 710 whose title is material B describes a plurality of FIGS. 1 and 2 for the items explained in the class. A material 720 whose title is material A describes an overview of the above items. The details of the above items are described in the material 730 whose title is the material C.

  Next, as shown in the state (B), the user uses the terminal device 100 to image a subject including a plurality of barcodes 711, 721, 731. Thereby, the terminal device 100 recognizes the URL recorded as the barcode for each barcode. That is, the terminal device 100 recognizes a plurality of barcodes at the same time.

  Furthermore, the terminal device 100 specifies the position of each of the plurality of barcodes 711, 721, 731 in the subject image obtained by imaging. Specifically, the terminal device 100 calculates coordinate values (x, y) in the two-dimensional coordinate system (XY coordinate system) for each of the barcodes 711, 721, and 731.

  The XY coordinate system can be set as follows, for example. That is, the origin O of the XY coordinate system is one of the four corners of the display. Here, the distance between one corner of the four corners in the barcode without the cutout symbol and the origin O of the XY coordinate system is longer than the other three corners (the corner with the cutout symbol). One of the four corners is selected as the origin O. Further, the corner without the cut-out symbol is defined as the “first corner”, and the three corners are sequentially rotated from the first corner in the order of “second corner”, “third corner”, “fourth corner”. ”, One side of the display (display surface) in the direction from the third corner (diagonal corner of the first corner) to the fourth corner is the X axis, and the third corner is directed to the second corner. One side of the directional display is the Y axis.

  The method of setting the coordinate system is not particularly limited. Further, as the position of the barcode, the position of a predetermined portion of the barcode (for example, the second corner or the center) may be used.

  When the terminal device 100 finishes recognizing each URL of each barcode 711, 721, 731 and specifying each coordinate value, each of the recognized URLs represents the position of the barcode where the URL is recorded. The data is transmitted to the server device 200 in association with the coordinate value. Specifically, the terminal device 100 transmits the URL recorded as the barcode 711 to the server device 200 in association with the coordinate value representing the position of the barcode 711 where the URL is recorded. The terminal device 100 transmits the URL recorded as the barcode 721 to the server device 200 in association with the coordinate value representing the position of the barcode 721 where the URL is recorded. The terminal device 100 transmits the URL recorded as the barcode 731 to the server device 200 in association with the coordinate value representing the position of the barcode 731 where the URL is recorded.

  Here, it is assumed that the positions of the barcodes 711, 721, 731 are (x71, y71), (x72, y72), (x73, y73), respectively. Further, x71 <x72 <x73 and y71 <y72 <y73.

  Based on each URL, the server device 200 reads the contents of the materials A, B, and C (for example, PDF (Portable Document Format) file, HTML (HyperText Markup Language) file) from the database. The server device 200 determines the positional relationship between the plurality of barcodes 711, 721, 731 based on the three coordinate values. As an example, the server apparatus 200 determines that the barcodes 711, 721, and 731 are arranged in ascending order of Y coordinate values.

  As shown in the state (C), the server device 200 generates one piece of electronic data 790 by arranging three contents specified by each of the three URLs using the determined positional relationship. The server device 200 transmits the generated electronic data 790 to the terminal device 100. Examples of the electronic data format include a PDF format and an HTML (HyperText Markup Language) format. The format of electronic data is not particularly limited.

  The terminal device 100 receives the electronic data 790 from the server device 200. The terminal device 100 displays the received electronic data 790 on the display.

  With such a configuration, the user of the terminal device 100 can obtain one piece of electronic data including the contents of the paper medium materials that are desired to be managed together in the desired order in the desired order. That is, the communication system 1 can provide electronic data that is highly convenient for the user by using the positional relationship between a plurality of barcodes.

(B2. Second example)
FIG. 3 is a diagram for explaining a second example of processing performed in the communication system 1. In the first example, the positional relationship between the plurality of barcodes is determined using only the Y coordinate, whereas in the second example, the positions of the plurality of barcodes are used using the X coordinate and the Y coordinate. Determine the relationship. It should be noted that whether only one coordinate or both coordinates are used can be changed by user settings.

  Referring to FIG. 3, as shown in state (A), the user of terminal device 100 arranges a plurality of maps 810, 820, 830, and 840 of places visited by travel in the positional relationship of east, west, south, and north. Barcodes 811, 821, 831, and 841 are printed on the maps 810, 820, 830, and 840, respectively.

  Next, as shown in state (B), the user uses the terminal device 100 to image a subject including a plurality of barcodes 811, 821, 831, 841. Thereby, the terminal device 100 recognizes the URL recorded as the barcode for each barcode. That is, the terminal device 100 recognizes a plurality of barcodes at the same time.

  Furthermore, the terminal device 100 specifies the position of each of the plurality of barcodes 811, 821, 831, 841 in the subject image obtained by imaging. Specifically, the terminal device 100 calculates coordinate values (x, y) in the XY coordinate system for each of the barcodes 811, 821, 831, and 841.

  When the terminal device 100 finishes recognizing each URL of each barcode 811, 821, 831, 841 and specifying each coordinate value, each terminal device 100 recognizes each of the plurality of recognized URLs as the position of the barcode where the URL is recorded. The information is transmitted to the server device 200 in association with the expressed coordinate value.

  Here, it is assumed that the positions of the barcodes 811, 821, 831, and 841 are (x81, y81), (x82, y82), (x83, y83), and (x84, y84), respectively. Further, x81 <x82 <x84 <x83 and y81 <y83 <y82 <y84.

  The server device 200 reads each content (for example, PDF (Portable Document Format) file) of each map from the database based on each URL. The server device 200 determines the positional relationship between the plurality of barcodes 811, 821, 831, 841 based on the four coordinate values. Specifically, the server device 200 determines that the barcodes 811, 821, 831, and 841 are arranged two-dimensionally based on the X coordinate value and the Y coordinate value.

  As shown in the state (C), the server apparatus 200 generates electronic map data 890 by arranging four maps (contents) specified by each of the four URLs using the determined positional relationship. To do. As an example, the server apparatus 200 has a map recorded as a barcode 831, with a map specified by a URL recorded as a barcode 811 in the upper left and a map specified by a URL recorded as a barcode 821 in the lower left. The electronic data 890 configured to display the map specified by the above in the upper right and the map specified by the URL recorded as the barcode 841 in the lower right is generated. The server device 200 transmits the generated electronic data 890 to the terminal device 100.

  The terminal device 100 receives the electronic data 890 from the server device 200. The terminal device 100 displays the received electronic data 890 on the display.

  With such a configuration, the user of the terminal device 100 has one piece of electronic data including the contents of the paper medium materials that the user wants to manage in a desired order (arrangement) in the desired order (arrangement). Can be obtained. That is, the communication system 1 can provide electronic data that is highly convenient for the user by using the positional relationship between a plurality of barcodes.

(B3. Third example)
FIG. 4 is a diagram for explaining a third example of processing performed in the communication system 1. In the second example, the terminal device 100 is in the vertical position, whereas in the third example, the terminal device 100 is in the horizontal position. In the third example, as in the second example, the positional relationship between the plurality of barcodes is determined using the X coordinate and the Y coordinate.

  Referring to FIG. 4, as shown in state (A), the user of terminal device 100 receives two business cards 910 and 920 and a pamphlet 930 such as a product, and business cards 910 and 920 are pamphlets 930. Arrange them so that they are on top. Bar codes 911 and 921 are printed on the business cards 910 and 920, respectively. In addition, a barcode 931 is printed on the brochure 930.

  Next, as shown in state (B), the user uses the terminal device 100 to image a subject including a plurality of barcodes 911, 921, and 931. Thereby, the terminal device 100 recognizes the URL recorded as the barcode for each barcode. That is, the terminal device 100 recognizes a plurality of barcodes at the same time.

  Furthermore, the terminal device 100 specifies the position of each of the plurality of barcodes 911, 921, 931 in the subject image obtained by imaging. Specifically, the terminal device 100 calculates coordinate values (x, y) in the XY coordinate system for each of the barcodes 911, 921, 931, and 941. In the third example, the longitudinal direction of the display is the X-axis direction, and the lateral direction is the Y-axis direction.

  When the terminal device 100 finishes recognizing each URL of each barcode 911, 921, 931, 941 and specifying each coordinate value, the terminal device 100 sets each of the recognized URLs to the position of the barcode that records the URL. The information is transmitted to the server device 200 in association with the expressed coordinate value.

  Here, it is assumed that the positions of the barcodes 911, 921, 931 are (x91, y91), (x92, y92), (x93, y93), respectively. Further, x91 <x92 <x93 and y2 <y91 <y93 are set.

  The server device 200 reads each content of the business card and the pamphlet from the database based on each URL. The server apparatus 200 determines the positional relationship between the plurality of barcodes 911, 921, and 931 based on the three coordinate values. Specifically, the server device 200 determines that the barcodes 911, 921, and 931 are arranged two-dimensionally based on the value of the X coordinate and the value of the Y coordinate.

  As shown in the state (C), the server apparatus 200 arranges the four contents (business card and pamphlet) specified by each of the three URLs using the determined positional relationship, thereby obtaining one piece of electronic data 890. Is generated. As an example, the server device 200 includes a business card specified by a URL recorded as a barcode 911 in the upper left, a map specified by a URL recorded as a barcode 921 in the upper right, and a URL recorded as a barcode 931. The electronic data 990 configured to display the pamphlet specified in (1) on the upper right is generated. The server device 200 transmits the generated electronic data 990 to the terminal device 100.

  The terminal device 100 receives the electronic data 990 from the server device 200. The terminal device 100 displays the received electronic data 990 on the display.

  With such a configuration, the user of the terminal device 100 has one piece of electronic data including the contents of the paper medium materials that the user wants to manage in a desired order (arrangement) in the desired order (arrangement). Can be obtained. That is, the communication system 1 can provide electronic data that is highly convenient for the user by using the positional relationship between a plurality of barcodes.

  Hereinafter, a specific configuration of the terminal device 100 and a specific configuration of the server device 200 for realizing the above processing will be described.

<C. Hardware configuration>
FIG. 5 is a diagram illustrating a hardware configuration of the terminal device 100. Referring to FIG. 5, terminal apparatus 100 includes antenna 151, RF (Radio Frequency) circuit 152, baseband circuit 153, GPS receiver 154, application processor 155 that executes a program, and ROM (Read Only). Memory) 156, RAM (Random Access Memory) 157, NAND-type flash memory 158, touch screen 159, camera 167, microphone 161, speaker 162, and operation key 163 for receiving an instruction input by the user. , A communication IF (Interface) 164, an IC (Integrated Circuit) card reader / writer 165, and a power supply unit 166.

  The GPS receiver 154 includes an antenna 1541, an RF circuit 1542, and a baseband circuit 1543. The GPS receiver 154 receives a GPS satellite signal using the antenna 1541. Further, the GPS receiver 154 outputs processing results (that is, current location information) in the RF circuit 1542 and the baseband circuit 1543 to the application processor 155. The current location information is information indicating latitude, longitude, and altitude.

  The antenna 151, the RF circuit 152, and the baseband circuit 153 are used for wireless communication with another mobile terminal, a fixed telephone, and a PC (Personal Computer) via a base station. Specifically, the antenna 151, the RF circuit 152, and the baseband circuit 153 are used for the terminal device 100 to perform communication using a mobile phone network.

  The flash memory 158 is a nonvolatile semiconductor memory. The flash memory 158 volatilely stores various programs for controlling the terminal device 100 and various data such as data generated by the terminal device 100 and data acquired from an external device of the terminal device 100.

  The components 153 to 159 and 161 to 167 are connected to each other by a data bus. A memory card 1651 is attached to the IC card reader / writer 165.

  The processing in the terminal device 100 is realized by software executed by each hardware and the application processor 155. Such software may be stored in advance in the flash memory 158. The software may be stored in a memory card 1451 or other storage medium and distributed as a program product. Alternatively, the software may be provided as a program product that can be downloaded by an information provider connected to the so-called Internet. Such software is read from the storage medium by an IC card reader / writer 165 or other reader, or downloaded via the antenna 151, the RF circuit 152, the baseband circuit 153, or the communication IF 164, Once stored in the flash memory 158. The software is read from the flash memory 158 by the application processor 155 and further stored in the flash memory 158 in the form of an executable program. The application processor 155 executes the program.

  Each component which comprises the terminal device 100 shown by the figure is general. Therefore, it can be said that the essential part of the present invention is software stored in the flash memory 158, the memory card 1451, or other storage medium, or software that can be downloaded via a network. In addition, since the operation | movement of each hardware of the terminal device 100 is known, detailed description is not repeated.

  The recording medium is not limited to DVD-ROM, CD-ROM, FD, and hard disk, but is fixed such as semiconductor memory such as magnetic tape, cassette tape, optical disk, optical card, mask ROM, EPROM, EEPROM, and flash ROM. A medium carrying a program may be used. The recording medium is a non-temporary medium that can be read by the computer. The program here includes not only a program directly executable by the CPU but also a program in a source program format, a compressed program, an encrypted program, and the like.

  FIG. 6 is a diagram illustrating a hardware configuration of the server device 200. Referring to FIG. 6, server apparatus 200 includes, as main components, CPU 251 that executes a program, ROM 252 that stores data in a nonvolatile manner, data generated by execution of the program by CPU 251, or an input device (see FIG. 6). (Not shown) RAM 253 for storing data volatilely, HDD 254 for storing data in a nonvolatile manner, LED 255, switch 256, communication IF (Interface) 257, power supply circuit 258, monitor 259 and operation keys 260. Each component is connected to each other by a data bus.

  The processing in the server device 200 is realized by each hardware and software executed by the CPU 251. Such software may be stored in the HDD 254 in advance. The software may be stored in other storage media and distributed as a program product. Alternatively, the software may be provided as a program product that can be downloaded by an information provider connected to the so-called Internet. Such software is read from the storage medium by a reading device or downloaded via, for example, a communication IF (not shown) and then temporarily stored in the HDD 254. The software is read from the HDD 254 by the CPU 251 and stored in the RAM 253 in the form of an executable program. The CPU 251 executes the program.

  Each component which comprises the server apparatus 200 shown by the figure is a general thing. Therefore, it can be said that the essential part of the present invention is RAM 253, HDD 254, software stored in a storage medium, or software that can be downloaded via a network. Since the operation of each hardware of server device 200 is well known, detailed description will not be repeated.

  The recording medium is not limited to a DVD-RAM, and a fixed program such as a semiconductor memory such as a DVD-ROM, a CD-ROM, an FD, a hard disk, a magnetic tape, a cassette tape, an optical disk, an EEPROM, and a flash ROM is supported. It may be a medium to be used. The recording medium is a non-temporary medium that can be read by the computer. The program here includes not only a program directly executable by the CPU but also a program in a source program format, a compressed program, an encrypted program, and the like.

<D. Functional configuration>
FIG. 7 is a functional block diagram for explaining a functional configuration of the terminal device 100. Referring to FIG. 7, terminal device 100 includes an imaging unit 111, a recognition unit 112, a specifying unit 113, a transmission unit 114, a reception unit 115, and a display unit 116.

  The imaging unit 111 images a subject including a plurality of barcodes. The imaging unit 111 sends the subject image obtained by imaging to the recognition unit 112 and the specifying unit 113.

  The recognition unit 112 recognizes the URL recorded as a barcode for each barcode based on the imaging. The recognition unit 112 sends a plurality of URLs as recognition results to the transmission unit 114.

  The specifying unit 113 specifies the position of each of the plurality of barcodes in the subject image obtained by the imaging. The identification unit 113 sends a plurality of coordinate values, which are identification results, to the transmission unit 114.

  The transmission unit 114 transmits each of the plurality of recognized URLs to the server apparatus 200 in association with the coordinate value representing the position of the barcode where the URL is recorded.

  The receiving unit 115 receives electronic data from the server device 200. Specifically, the receiving unit 115 transmits a plurality of contents specified by each of the plurality of URLs in the server device 200 based on the transmission of the plurality of URLs and coordinate values to the server device 200 using the plurality of coordinate values. The electronic data generated by the placement is received from the server device 200.

The display unit 116 displays the received electronic data.
FIG. 8 is a functional block diagram for explaining a functional configuration of the server device 200. Referring to FIG. 8, server apparatus 200 includes a reception unit 211, a determination unit 212, a content DB 213, a generation unit 214, and a transmission unit 215.

  The receiving unit 211 receives the URL and the coordinate value transmitted from the transmitting unit 114 of the terminal device 100. Specifically, the receiving unit 211 recognizes each of a plurality of URLs obtained by recognizing the URL recorded as a barcode for each barcode, and stores the URL recorded in the subject image obtained by imaging. It is received from the terminal device 100 in association with the coordinate value representing the position.

  The determination unit 212 determines the positional relationship between the plurality of barcodes based on the plurality of coordinate values received by the reception unit 211. The determination unit 212 sends the determination result to the generation unit 214.

The content DB 213 stores a plurality of contents.
The generation unit 214 generates electronic data by arranging a plurality of contents specified by each of a plurality of URLs using the determined positional relationship. In other words, the generation unit 214 reads content from the content DB 213 using the URL, and generates one electronic data from the plurality of read content.

  The transmission unit 215 transmits the electronic data generated by the generation unit 214 to the terminal device 100.

  The processing of the determination unit 212 will be further described as follows. The determination unit 212 determines the position of each barcode image of the plurality of barcodes in the subject image using a two-dimensional coordinate system. For example, when only one of the X coordinates and the Y coordinates is used (in the case of the first example), the determining unit 212 is based on the coordinate value of each barcode image on one coordinate axis constituting the coordinate system. The positional relationship between a plurality of barcodes is determined. In addition, when using the X coordinate and the Y coordinate (second example, third example), the determination unit 212 uses a plurality of bars based on the coordinate values of each barcode image on both coordinate axes constituting the coordinate system. Determine the positional relationship between codes.

  Note that specifying the positional relationship between a plurality of barcodes also specifies the positional relationship between a plurality of corresponding contents.

<E. Data>
Next, data (coordinate values) representing the position of the barcode and data (information representing the positional relationship) representing the content arrangement will be described. Note that data representing the arrangement of content ("new_data [j]" described later) is, for example, rearranged (sorted) data representing a barcode position (coordinate value). “J” is a value from 0 to N (number of barcodes).

  FIG. 9 is a diagram for explaining a case where the arrangement order of the contents in the case where the contents constituting the electronic data are arranged in a line (the first example of FIG. 2 and the like). FIG. 9A shows data representing the positions of four barcodes. The data is composed of four coordinate values (x, y). FIG. 9B is a diagram showing data for determining the arrangement of the rearranged contents.

  When the server apparatus 200 receives four coordinate values from the terminal apparatus 100, the server apparatus 200 compares the x-coordinate values of data [0] and data [3] having a small Y-coordinate value. The server apparatus 200 sets data [3] to new_data [0] because the x-coordinate value of data [3] is smaller than the x-coordinate value of data [0]. Further, the server apparatus 200 sets data [0] to new_data [1].

  Next, the server apparatus 200 compares the Y coordinate values of the remaining data, data [1] and data [2]. Since the Y-coordinate values of both are close, the server device 200 compares the x-coordinate values in data [1] and data [2]. The server apparatus 200 sets data [2] to new_data [3] because the x-coordinate value of data [2] is smaller than the x-coordinate value of data [1]. Further, the server apparatus 200 sets data [1] to new_data [3].

  A specific process for obtaining the data of FIG. 9B from the data of FIG. 9A will be described later (FIG. 13).

  The server apparatus 200 has the barcode printed at the position indicated by data [3] in the order shown in FIG. 9B, that is, the content where the barcode is printed at the position indicated by data [0]. An electronic device configured to display each content in the order of content, content having a barcode printed at a position indicated by data [2], and content having a barcode printed at a position indicated by data [1]. Generate data. That is, electronic data including the corresponding content is generated in ascending order of the value of “j” in “new_data [j]”.

  As described above, the order in which the value of j is small, that is, the arrangement order of data [0], data [3], data [2], and data [1] is such that data [i] (0 ≦ i ≦ 3). It represents the positional relationship between the corresponding four contents. Further, this arrangement order corresponds to information representing the positional relationship between a plurality of barcodes.

  Note that, as shown in FIG. 2, the order of contents may be determined by only the Y coordinate value without considering the X coordinate value.

  FIG. 10 is a diagram for explaining a case where the arrangement order of contents in the case where contents constituting electronic data are arranged two-dimensionally (second example in FIG. 3, third example in FIG. 4). It is.

  FIG. 10A shows data representing the positions of four barcodes. The data is composed of four coordinate values (x, y). FIG. 10B shows data for determining the arrangement of the rearranged contents. The data shown in FIG. 10 (A) is the same as the data shown in FIG. 9 (A), and the data shown in FIG. 10 (B) is the same as the data shown in FIG. 9 (B). That is, for the rearrangement from FIG. 10A to FIG. 10B, the method described based on FIG. 9 is used.

  FIG. 10C shows data that is two-dimensionally rearranged based on the value of the x coordinate, with a group of approximate ones having the same value of the Y coordinate but similar. Here, list [s] [t] is used for determining the arrangement using a matrix. In the case of FIG. 10, since four barcodes are read, it is necessary to arrange the contents in 2 rows and 2 columns. For this reason, the values of s and t take 0 or 1.

  Note that list [0] [0] is in the first row and the first column, and means the upper left position. list [0] [1] is in the first row and the second column, and means the upper right position. list [1] [0] is in the second row and the first column, and means the lower left position. list [1] [1] is in the second row and the second column, and means the lower right position.

  In the order shown in FIG. 9B, the server device 200 prints the content where the barcode is printed at the position indicated by new_data [0] on the upper left and the barcode at the position indicated by new_data [1]. The displayed content is displayed in the upper right, the content having the barcode printed at the position indicated by new_data [2] is displayed in the lower left, and the content having the barcode printed in the position indicated by new_data [3] is displayed in the lower right. The electronic data configured as described above is generated.

  As described above, four contents corresponding to new_data [j] (0 ≦ i ≦ 3) (that is, four contents corresponding to new_data [i]) are determined by the values of s and t in list [s] [t]. ) Is represented. Further, the arrangement order specified by s and t (the arrangement order in the two-dimensional arrangement) corresponds to information representing the positional relationship between a plurality of barcodes. In the present embodiment, the “positional relationship” means not a coordinate value itself but a relative rough positional relationship such that a barcode is above or below another barcode. To do.

  Specific processing for obtaining the data shown in FIG. 10C from the data shown in FIG. 10B will be described later (FIG. 14). The content layout process in the electronic data will also be described later (FIG. 15).

<F. Control structure>
(F1. Terminal device processing flow)
FIG. 11 is a flowchart for explaining the flow of processing in the terminal device 100. Referring to FIG. 11, in step S102, terminal apparatus 100 captures an image of a subject including a plurality of barcodes. In step S104, the terminal device 100 recognizes the URL recorded as a barcode for each barcode. In step S106, the terminal device 100 specifies the position of each of the plurality of barcodes in the subject image.

  In step S <b> 108, the terminal device 100 transmits each of the recognized URLs to the server device 200 in association with a coordinate value that represents the position of the barcode that records the URL. In step S <b> 110, the terminal device 100 receives, from the server device 200, electronic data generated by arranging a plurality of contents specified by each of the plurality of URLs using the coordinate values in the server device 200. In step S112, the terminal device 100 displays the received electronic data.

(F2. Processing flow of server device)
FIG. 12 is a flowchart for explaining the flow of processing in the server apparatus 200. Referring to FIG. 12, in step S202, server device 200 associates each of a plurality of URLs with a coordinate value that represents the position of a barcode that records the URL in a subject image obtained by imaging. Received from the terminal device 100. In step S204, the server device 200 determines the positional relationship between the plurality of barcodes based on the plurality of coordinate values. In step S206, electronic data is generated by arranging a plurality of contents specified by each of a plurality of URLs using the positional relationship. In step S208, the server apparatus 200 transmits the generated electronic data to the terminal apparatus 100.

  FIG. 13 is a flowchart showing a flow of processing for generating new_data [j] from data [i]. In other words, FIG. 13 is also a diagram for explaining a part of the process of step S204 in FIG.

  Referring to FIG. 13, in step S302, server apparatus 200 sets the value of variable j to 0. In step S304, the server apparatus 200 has a plurality of data [i] having a minimum Y coordinate value (y), or one or more data [i] having a difference from the minimum y within a predetermined value. Determine if it exists.

  When a positive determination is made in step S304 (in the case of YES), the server apparatus 200 sets these data [i] as search targets in step S305. In step S306, data [i] having the minimum X coordinate value (x) is found from the data [i] to be searched. In step S308, the server apparatus 200 sets data [i] with the smallest x as new_data [j]. In step S310, the server apparatus 200 excludes data [i] having the smallest x from the search target. In step S312, the server apparatus 200 increments the value of j. That is, the server apparatus 200 increases the value of j by 1.

  In step S314, the server apparatus 200 determines whether or not data [i] whose y is the minimum and data [i] whose difference from the minimum y is within a predetermined value are excluded from the search target. If a positive determination is made in step S314 (in the case of YES), server apparatus 200 advances the process to step S316. When a negative determination is made in step S314 (in the case of NO), server apparatus 200 advances the process to step S306.

  When a negative determination is made in step S304 (in the case of NO), the server apparatus 200 sets data [i] with the smallest y as new_data [j] in step S318. In step S320, the server apparatus 200 excludes data [i] having the smallest y from being searched. In step S322, the server apparatus 200 increments the value of j.

  In step S316, the server apparatus 200 determines whether the value of j is greater than the maximum value of i. When a positive determination is made in step S316 (in the case of YES), server apparatus 200 ends the series of processes. If a negative determination is made in step S316 (in the case of NO), server apparatus 200 advances the process to step S304.

  FIG. 14 is a flowchart showing a flow of processing for generating list [s] [t] from new_data [j]. In other words, FIG. 14 is also a diagram for explaining a part of the processing in step S204 in FIG. Note that new_data [j] is data after the rearrangement is performed in the order of the value of the X coordinate and the value of the Y coordinate in a manner in which the Y coordinate is prioritized over the X coordinate as described above.

  Referring to FIG. 14, in step S402, server apparatus 200 sets the value of variable j to 1, and sets the values of variables s and t to 0. In step S404, the server apparatus 200 sets list [s] [t] to new_data [j-1]. In step S406, the server apparatus 200 determines whether or not the value of y in new_data [j] and the value of y in new_data [j−1] are approximated (whether or not the difference is within a predetermined range). Judging.

  When a positive determination is made in step S406 (in the case of YES), server apparatus 200 increments the value of s in step S408. If a negative determination is made in step S406 (in the case of NO), the server device 200 increments t and sets the value of s to “0” in step S416.

  In step S410, the server apparatus 200 sets list [s] [t] to new_data [j]. In step S412, the server apparatus 200 increments the value of j.

  In step S414, the server apparatus 200 determines whether all new_data have been processed. If a positive determination is made in step S414 (in the case of YES), the server device 200 ends the series of processes. If a negative determination is made in step S414 (in the case of NO), server apparatus 200 advances the process to step S406.

  FIG. 15 is a flowchart for explaining the flow of processing (layout processing) for arranging a plurality of contents and creating one electronic data. Referring to FIG. 15, in step S502, server apparatus 200 sets the values of variables s and t to “0”. In step S504, the server apparatus 200 recognizes new_data [j] set as list [s] [t], and specifies a URL associated with the new_data [j] from the plurality of URLs.

  In step S506, the server apparatus 200 reads the content file [s] [t] from the content DB 213 and opens the file. The file [s] [t] is a content file corresponding to a paper medium in which a barcode is recorded at the coordinate value of data [i] specified by list [s] [t].

  In step S508, the server apparatus 200 arranges (lays out) the contents of the opened file at the upper left of the new file that is the source of the electronic data to be transmitted to the terminal apparatus 100. In step S510, the server apparatus 200 determines whether or not list [s + 1] [t] exists.

  When a positive determination is made in step S510 (in the case of YES), server apparatus 200 increments the value of s in step S512. In step S514, the server apparatus 200 recognizes new_data [j] set as list [s] [t], and identifies the URL associated with the new_data [j] from the plurality of URLs.

  In step S516, the server apparatus 200 reads the content file [s] [t] from the content DB 213 and opens the file. In step S518, the server apparatus 200 arranges the contents of the opened file to the right of the position of list [s-1] [t] in the new file.

  If a negative determination is made in step S510 (in the case of NO), the server apparatus 200 determines whether or not list [s] [t + 1] exists in step S520.

  If a positive determination is made in step S520 (in the case of YES), server apparatus 200 increments the value of t in step S522. In step S524, the server apparatus 200 recognizes new_data [j] set as list [s] [t], and specifies the URL associated with the new_data [j] from the plurality of URLs.

  In step S526, the server apparatus 200 reads the content file [s] [t] from the content DB 213 and opens the file. In step S528, the server apparatus 200 arranges the contents of the opened file immediately below the position of list [s] [t-1] in the new file.

[Embodiment 2]
In the first embodiment, the terminal device 100 transmits the coordinate values of each of the plurality of barcodes, and the server device 200 has been described with a configuration in which the positional relationship between the plurality of barcodes is determined. In the present embodiment, a configuration will be described in which the terminal device determines the positional relationship between a plurality of barcodes and transmits the determined positional relationship to the server device. Note that generation of electronic data including a plurality of contents is performed by the server device as in the first embodiment.

  Hereinafter, the terminal device and the server device in the present embodiment are referred to as “terminal device 100A” and “server device 200A” in order to distinguish them from the terminal device 100 and the server device 200 in the first embodiment.

<G. Functional configuration>
FIG. 16 is a functional block diagram for explaining a functional configuration of the terminal device 100A. Referring to FIG. 16, terminal device 100A includes an imaging unit 111, a recognition unit 112, a transmission unit 114A, a reception unit 115, a display unit 116, and a determination unit 117. The terminal device 100A is different from the terminal device 100 of the first embodiment in that the determination unit 117 and the transmission unit 114A are provided without the specifying unit 113 (FIG. 7) and the transmission unit 114.

  The imaging unit 111 images a subject including a plurality of barcodes. The imaging unit 111 sends a subject image obtained by imaging to the recognition unit 112 and the determination unit 117.

  The recognition unit 112 recognizes the URL recorded as a barcode for each barcode based on the imaging. The recognition unit 112 sends a plurality of URLs as recognition results to the transmission unit 114A.

The determination unit 117 determines the positional relationship between a plurality of barcodes based on the above imaging.
Specifically, the determination unit 117 first specifies the position of each of the plurality of barcodes in the subject image obtained by the imaging. The determination unit 117 determines the positional relationship between the plurality of barcodes based on the plurality of specified coordinate values. That is, the determination unit 117 executes the processing by the specifying unit 113 of the terminal device 100 and the processing by the determination unit 212 of the server device 200 described in the first embodiment. The determination unit 117 sends the determined positional relationship to the transmission unit 114A.

  The transmission unit 114A associates the plurality of recognized URLs with the information representing the positional relationship, and transmits them to the server device 200A.

  The receiving unit 115 receives electronic data from the server device 200A. Specifically, the receiving unit 115 transmits a plurality of contents specified by each of the plurality of URLs in the server device 200A based on the fact that the plurality of URLs and information representing the positional relationship are transmitted to the server device 200A. The electronic data generated by arranging using the information representing the above is received from the server device 200A.

The display unit 116 displays the received electronic data.
The processing of the determination unit 117 will be further described as follows. The determination unit 117 determines the position of each barcode image of the plurality of barcodes in the subject image using a two-dimensional coordinate system. For example, when only one of the X coordinates and the Y coordinates is used, the determination unit 117 determines the positions of a plurality of barcodes based on the coordinate values of the barcode images on one coordinate axis constituting the coordinate system. Determine the relationship. In addition, when using the X coordinate and the Y coordinate, the determination unit 117 determines the positional relationship between a plurality of barcodes based on the coordinate values of the barcode images on both coordinate axes constituting the coordinate system.

  FIG. 17 is a functional block diagram for explaining a functional configuration of the server device 200A. Referring to FIG. 17, server apparatus 200 </ b> A includes reception unit 211 </ b> A, content DB 213, generation unit 214, and transmission unit 215. Server device 200A differs from server device 200 in that it does not include determination unit 212. Further, the server device 200A includes a receiving unit 211A instead of the receiving unit 211.

  The receiving unit 211A receives the URL transmitted from the transmitting unit 114A of the terminal device 100A and information representing the positional relationship. Specifically, the receiving unit 211A displays a plurality of URLs obtained by recognizing a URL recorded as a barcode for each barcode, and information indicating the positional relationship between the plurality of barcodes, as a terminal device 100A. Receive from.

  The generation unit 214 generates electronic data by arranging a plurality of contents specified by each of a plurality of URLs using the received positional relationship. In other words, the generation unit 214 reads content from the content DB 213 using the URL, and generates one electronic data from the plurality of read content.

  The transmission unit 215 transmits the electronic data generated by the generation unit 214 to the terminal device 100.

<H. Control structure>
FIG. 18 is a flowchart for explaining the flow of processing in the terminal device 100A. Referring to FIG. 18, in step S602, terminal device 100A images a subject including a plurality of barcodes. In step S604, the terminal device 100A recognizes the URL recorded as a barcode for each barcode. In step S606, the terminal device 100A determines the positional relationship between the plurality of barcodes.

  In step S608, the terminal device 100A transmits the recognized URLs and information representing the positional relationship to the server device 200A. In step S610, the terminal device 100A transmits electronic data generated by arranging a plurality of contents specified by each of a plurality of URLs using information representing a positional relationship in the server device 200A from the server device 200A. Receive. In step S612, the terminal device 100A displays the received electronic data.

  FIG. 19 is a flowchart for explaining the flow of processing in server apparatus 200A. Referring to FIG. 19, in step S <b> 702, server apparatus 200 </ b> A receives a plurality of URLs and information representing a positional relationship from terminal apparatus 100 </ b> A. In step S704, electronic data is generated by arranging a plurality of contents specified by each of a plurality of URLs using information representing the positional relationship. In step S606, the server device 200A transmits the generated electronic data to the terminal device 100A.

  In the second embodiment, similarly to the first embodiment, electronic data that is highly convenient for the user can be provided to the user by using the positional relationship between the plurality of barcodes.

[Embodiment 3]
In the first embodiment, the terminal device 100 transmits the coordinate value to the server device 200 together with the URL. In the second embodiment, the terminal device 100A transmits information indicating the positional relationship together with the URL to the server device 200A. In the present embodiment, a configuration will be described in which the terminal device does not transmit information on the position of the barcode such as information representing the coordinate value and the positional relationship to the server device. That is, a configuration in which the terminal device generates electronic data instead of the server device will be described.

  Hereinafter, in order to distinguish the terminal device and the server device in the present embodiment from the terminal devices 100 and 100A and the server devices 200 and 200A in the first and second embodiments, “terminal device 100B”, “server device” 200B ".

<I. Functional configuration>
FIG. 20 is a functional block diagram for explaining a functional configuration of the terminal device 100B. Referring to FIG. 20, terminal device 100B includes an imaging unit 111, a recognition unit 112, a transmission unit 114B, a reception unit 115B, a display unit 116B, and a determination unit 117B.

  The imaging unit 111 images a subject including a plurality of barcodes. The imaging unit 111 sends the subject image obtained by imaging to the recognition unit 112 and the determination unit 117B.

  The recognition unit 112 recognizes the URL recorded as a barcode for each barcode based on the imaging. The recognition unit 112 sends a plurality of URLs as recognition results to the transmission unit 114B.

The determination unit 117B determines the positional relationship between the plurality of barcodes based on the imaging.
Specifically, the determination unit 117B first identifies the position of each of the plurality of barcodes in the subject image obtained by the imaging. The determination unit 117B determines the positional relationship between the plurality of barcodes based on the identified plurality of coordinate values. Determination unit 117B sends the determined positional relationship to display unit 116B.

The transmission unit 114B transmits the plurality of recognized URLs to the server device 200B.
The receiving unit 115B receives a plurality of contents from the server device 200B. Specifically, the receiving unit 115B receives a plurality of contents specified by each of the plurality of URLs in the server device 200B from the server device 200B based on the transmission of the plurality of URLs to the server device 200B. Receiving unit 115B sends the received plurality of contents to display unit 116B.

  Display unit 116B displays a plurality of contents based on the determined positional relationship. Specifically, the display unit 116B first generates electronic data by arranging a plurality of contents specified by each of a plurality of URLs using the positional relationship. Next, the display unit 116B displays the generated electronic data.

  The processing of the determination unit 117B will be further described as follows. The determination unit 117B determines the position of each barcode image of the plurality of barcodes in the subject image using a two-dimensional coordinate system. For example, when only one of the X coordinates and the Y coordinates is used, the determination unit 117B determines the positions of the plurality of barcodes based on the coordinate values of the barcode images on one coordinate axis constituting the coordinate system. Determine the relationship. In addition, when the X coordinate and the Y coordinate are used, the determination unit 117B determines the positional relationship between the plurality of barcodes based on the coordinate values of the barcode images on both coordinate axes constituting the coordinate system.

  FIG. 21 is a functional block diagram for explaining a functional configuration of the server device 200B. Referring to FIG. 21, server apparatus 200B includes a reception unit 211B, a content DB 213, and a transmission unit 215. Server device 200B is different from server device 200A (FIG. 17) of the second embodiment in that at least generation unit 214 is not provided.

  The reception unit 211B receives the URL transmitted from the transmission unit 114B of the terminal device 100B. Specifically, the reception unit 211B receives, from the terminal device 100B, a plurality of URLs obtained by recognizing the URL recorded as a barcode for each barcode.

  The transmission unit 215B reads content specified by a plurality of URLs from the content DB 213. The transmission unit 215B transmits the plurality of read contents to the terminal device 100B.

<J. Control structure>
FIG. 22 is a flowchart for explaining the flow of processing in the terminal device 100B. With reference to FIG. 22, in step S <b> 802, terminal apparatus 100 </ b> B images a subject including a plurality of barcodes. In step S804, the terminal device 100B recognizes the URL recorded as a barcode for each barcode. In step S806, the terminal device 100B determines the positional relationship between the plurality of barcodes.

  In step S808, the terminal device 100B transmits the plurality of recognized URLs to the server device 200B. In step S810, terminal device 100B receives a plurality of contents specified by each of a plurality of URLs in server device 200B from server device 200B. In step S812, the terminal device 100B displays the received plurality of contents based on the determined positional relationship.

  FIG. 23 is a flowchart for explaining the flow of processing in the server device 200B. Referring to FIG. 23, in step S902, server apparatus 200B receives a plurality of URLs from terminal apparatus 100B. In step S904, the server apparatus 200B transmits the plurality of contents specified by each of the plurality of URLs to the terminal apparatus 100B.

  As described above, also in the third embodiment, similarly to the first and second embodiments, electronic data that is highly convenient for the user can be provided to the user by using the positional relationship between the plurality of barcodes. .

<K. User interface>
FIG. 24 is a diagram for explaining a user interface displayed on the terminal devices 100, 100A, and 100B. Specifically, FIG. 24 is a diagram illustrating a user interface that is displayed when the user confirms the configuration of electronic data in the terminal devices 100, 100A, and 100B of the embodiments. FIG. 24A shows a first example of the user interface, and FIG. 24B shows a second example of the user interface.

  Referring to FIG. 24, terminal apparatuses 100, 100A, and 100B have, for example, material 710 whose title is material B, material 720 whose title is material A, and material C whose title is material C, as shown in FIG. When a plurality of barcodes 711, 721, and 731 are imaged in a state where the materials 730 are arranged in this order from above, the user interface 610 shown in FIG. 24A or the user interface 620 shown in FIG. Is generated.

  The user interface 610 is associated with an object 690 that represents the generation date of electronic data generated based on recognition of a plurality of barcodes, and a plurality of objects that represent contents (included in the electronic data) that constitute the electronic data. 710d, 720d, and 730d are included. In this case, the terminal devices 100, 100A, and 100B clearly show the connection between the objects 710d, 720d, and 730d and the object 690 by drawing them with lines. Further, the terminal devices 100, 100A, and 100B display arrows for indicating the order between the objects so that the user can easily grasp the arrangement of the materials 710, 720, and 730.

  Similar to the user interface 610, the user interface 620 includes a plurality of objects 710d, 720d, and 730d in association with the object 690. The user interface 620 has the same configuration as the user interface 610 except that the objects 720d and 730d and the object 690 are not connected by a line.

  The user interface displayed by the terminal devices 100, 100A, 100B is not limited to the above. The user interface displayed by the terminal devices 100, 100A, and 100B may be configured such that the user can distinguish the contents that constitute the electronic data and the arrangement order (arrangement) of the contents in the electronic data for each electronic data. That's fine.

  When an object is selected, the content file corresponding to the selected object is opened and the content is displayed (or only the data of the content portion of the generated electronic data is displayed. In addition, a communication system may be configured. In order to open the content file and display the content, the terminal device needs to acquire the content itself from the server device.

<L. Modification>
(1) In each embodiment, the “positional relationship” is not the coordinate value itself, but a relative rough positional relationship such that a certain barcode is above or below another barcode. Meant. However, the present invention is not limited to this, and the positional relationship may be an absolute or relative positional relationship specified according to coordinate values. That is, the terminal device and the server device may be configured so that the user generates electronic data according to the arrangement of the plurality of paper media that are the imaging targets.

  (2) In the above description, a paper medium barcode has been described as an example. However, the present invention is not limited to this. The medium on which the barcode is recorded may be a medium other than paper. Alternatively, the barcode may be displayed on a display surface of a portable portable electronic device. For example, even if the plurality of electronic devices each display one barcode and move the position of the electronic device, the terminal devices 100, 100A, and 100B recognize the plurality of barcodes collectively. Good.

  (3) In the first embodiment, the server apparatus 200 has been described by taking as an example a configuration in which electronic data is transmitted to the terminal apparatus 100 that has transmitted the URL and the coordinate value to the server apparatus 200 (FIG. 7, 8 etc.). However, the present invention is not limited to this. The communication system 1 may be configured such that the server device 200 transmits electronic data to one or more predetermined terminal devices other than the terminal device 100 that has transmitted the URL and the coordinate value. Alternatively, the communication system 1 is configured such that the server device 200 transmits electronic data to the terminal device 100 that has transmitted the URL and the coordinate value and to one or more predetermined terminal devices other than the terminal device 100. May be. It suffices to decide between which terminal device the electronic data is transmitted between the terminal device 100 and the server device 200.

  Further, in the second embodiment, the server apparatus 200A has been described by taking as an example a configuration in which the electronic data is transmitted to the terminal apparatus 100A that has transmitted the URL and the positional relationship to the server apparatus 200A (FIG. 16). , 17 etc.). However, the present invention is not limited to this. The communication system 1 may be configured such that the server device 200A transmits electronic data to one or more predetermined terminal devices other than the terminal device 100A that transmitted the URL and the positional relationship. Alternatively, the communication system 1 is configured such that the server device 200A transmits electronic data to one or more predetermined terminal devices other than the terminal device 100A and the terminal device 100A that transmitted the URL and the positional relationship. May be. The terminal device 100A and the server device 200A may determine which terminal device the electronic data is transmitted to.

  Furthermore, in the third embodiment, the server device 200B has been described with an example of a configuration in which content is transmitted to the terminal device 100B that has transmitted a URL to the server device 200B (FIGS. 20, 21, and the like). However, the present invention is not limited to this. The communication system 1 may be configured such that the server device 200B transmits content to one or more predetermined terminal devices other than the terminal device 100B that transmitted the URL. Alternatively, the communication system 1 may be configured such that the server device 200B transmits the content to the terminal device 100B that transmitted the URL and one or more predetermined terminal devices other than the terminal device 100B. The terminal device 100B and the server device 200B may determine which terminal device the content is transmitted to.

  The embodiment disclosed this time is an exemplification, and the present invention is not limited to the above contents. The scope of the present invention is defined by the terms of the claims, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  1 communication system, 100, 100A, 100B terminal device, 111 imaging unit, 112 recognition unit, 113 identification unit, 114, 114A, 114B, 215, 215B transmission unit, 115, 115B, 211, 211A, 211B reception unit, 116, 116B display unit, 117, 117B, 212 determination unit, 155 application processor, 158 flash memory, 159 touch screen, 167 camera, 200, 200A, 200B server device, 213 content DB, 214 generation unit, 251 CPU, 253 RAM, 300 Network, 610, 620 user interface, 690, 710d, 720d, 730d object, 710, 720, 730 document, 711, 721, 731, 811, 821, 831, 41,911,921,931,941 bar code, 790,890,990 electronic data, 810, 820, 830, 840 maps, 910, 920 business cards, 930 brochures.

Claims (26)

Imaging means for imaging a subject including a plurality of barcodes;
Recognition means for recognizing the first information recorded as the barcode for each barcode based on the imaging;
Specifying means for specifying the position of each of the plurality of barcodes in the subject image obtained by the imaging;
A terminal device comprising: transmission means for associating each of the plurality of recognized first information with the second information representing the position of the barcode on which the first information is recorded, and transmitting the information to the server device. . Based on the transmission of the plurality of first information and the second information to the server device, a plurality of third information specified by each of the plurality of first information in the server device Receiving means for receiving electronic data generated by arranging using the second information from the server device;
The terminal device according to claim 1, further comprising display means for displaying the received electronic data. Imaging means for imaging a subject including a plurality of barcodes;
Recognition means for recognizing the first information recorded as the barcode for each barcode based on the imaging;
Determination means for determining a positional relationship between the plurality of barcodes based on the imaging;
A terminal device comprising: transmission means for associating the plurality of recognized first information with the second information representing the positional relationship and transmitting the information to a server device. The determination means includes
Determining the position of each barcode image of the plurality of barcodes in the subject image obtained by imaging using a two-dimensional coordinate system;
The terminal device according to claim 3, wherein a positional relationship between the plurality of barcodes is determined based on a coordinate value of each barcode image on one coordinate axis constituting the coordinate system.   The terminal device according to claim 4, wherein the determination unit determines a positional relationship between the plurality of barcodes based on coordinate values of the barcode images on both coordinate axes constituting the coordinate system. Based on the transmission of the plurality of first information and the second information to the server device, a plurality of third information specified by each of the plurality of first information in the server device Receiving means for receiving electronic data generated by arranging using the second information from the server device;
The terminal device according to claim 3, further comprising display means for displaying the received electronic data. Imaging means for imaging a plurality of barcodes;
Recognition means for recognizing the first information recorded as the barcode for each barcode based on the imaging;
Determination means for determining a positional relationship between the plurality of barcodes based on the imaging;
Transmitting means for transmitting the plurality of recognized first information to a server device;
Receiving means for receiving second information specified based on the first information for each first information from the server device;
A terminal device comprising: display means for displaying the plurality of second information based on the determined positional relationship. The determination means includes
Determining the position of each barcode image of the plurality of barcodes in the subject image obtained by imaging using a two-dimensional coordinate system;
The terminal device according to claim 7, wherein a positional relationship between the plurality of barcodes is determined based on a coordinate value of each barcode image on one coordinate axis constituting the coordinate system.   The terminal device according to claim 8, wherein the determination unit determines a positional relationship between the plurality of barcodes based on coordinate values of the barcode images on both coordinate axes constituting the coordinate system. A server device that communicates with a terminal device that images a subject including a plurality of barcodes,
Each of the plurality of first information obtained by recognizing the first information recorded as the barcode for each barcode is used as the first information in the subject image obtained by the imaging. A receiving means for receiving from the terminal device in association with the second information representing the position of the recorded barcode;
Determination means for determining a positional relationship between the plurality of barcodes based on the plurality of second information;
Generating means for generating electronic data by arranging a plurality of third information specified by each of the plurality of first information using the determined positional relationship;
A server apparatus comprising: transmission means for transmitting the generated electronic data to the terminal apparatus. The determination means includes
Determining the position of each barcode image of the plurality of barcodes in the subject image using a two-dimensional coordinate system;
The server device according to claim 10, wherein a positional relationship between the plurality of barcodes is determined based on a coordinate value of each barcode image on one coordinate axis constituting the coordinate system.   The server device according to claim 11, wherein the determination unit determines a positional relationship between the plurality of barcodes based on coordinate values of the barcode images on both coordinate axes constituting the coordinate system. A server device that communicates with a terminal device that images a subject including a plurality of barcodes,
A plurality of the first information obtained by recognizing the first information recorded as the barcode for each barcode, and a second information representing a positional relationship between the plurality of barcodes; Receiving means for receiving from the terminal device;
Generating means for generating electronic data by arranging a plurality of third information specified by each of the plurality of first information using the second information;
A server apparatus comprising: transmission means for transmitting the generated electronic data to the terminal apparatus. A communication system including a terminal device and a server device,
The terminal device
Imaging means for imaging a subject including a plurality of barcodes;
Recognition means for recognizing the first information recorded as the barcode for each barcode based on the imaging;
Specifying means for specifying the position of each of the plurality of barcodes in the subject image obtained by the imaging;
Transmitting means for transmitting each of the recognized plurality of first information to the server device in association with second information representing the position of the barcode on which the first information is recorded,
The server device
Receiving means for receiving each of the plurality of pieces of first information from the terminal device in association with second information representing the position of the barcode on which the first information is recorded;
Determination means for determining a positional relationship between the plurality of barcodes based on the plurality of second information;
Generating means for generating electronic data by arranging a plurality of third information specified by each of the plurality of first information using the determined positional relationship;
A communication system comprising: transmission means for transmitting the generated electronic data to the terminal device. A communication system including a terminal device and a server device,
The terminal device
Imaging means for imaging a subject including a plurality of barcodes;
Recognition means for recognizing the first information recorded as the barcode for each barcode based on the imaging;
Determination means for determining a positional relationship between the plurality of barcodes based on the imaging;
Transmission means for associating the plurality of recognized first information with the second information representing the positional relationship and transmitting the second information to the server device;
The server device
Receiving means for receiving the plurality of first information and the second information from the terminal device;
Generating means for generating electronic data by arranging a plurality of third information specified by each of the plurality of first information using the second information;
A communication system comprising: transmission means for transmitting the generated electronic data to the terminal device. A communication system including a terminal device and a server device,
The terminal device
Imaging means for imaging a plurality of barcodes;
Recognition means for recognizing the first information recorded as the barcode for each barcode based on the imaging;
Determination means for determining a positional relationship between the plurality of barcodes based on the imaging;
Transmitting means for transmitting the recognized plurality of first information to the server device;
Receiving means for receiving second information specified based on the first information for each first information from the server device;
Display means for displaying the plurality of second information based on the determined positional relationship;
The server device
Receiving means for receiving the plurality of first information from the terminal device;
A communication system comprising: a transmission unit configured to transmit the plurality of second information to the terminal device. Information processing information in the terminal device,
Imaging a subject including a plurality of barcodes;
Recognizing the first information recorded as the barcode for each barcode based on the imaging;
Identifying the position of each of the plurality of barcodes in the subject image obtained by the imaging;
An information processing method comprising: associating each of the recognized plurality of pieces of first information with second information representing a position of a barcode in which the first information is recorded, and transmitting the information to a server device. . Information processing information in the terminal device,
Imaging a subject including a plurality of barcodes;
Recognizing the first information recorded as the barcode for each barcode based on the imaging;
Determining a positional relationship between the plurality of barcodes based on the imaging;
An information processing method comprising: associating the plurality of recognized first information with second information representing the positional relationship and transmitting the second information to a server device. An information processing method in a terminal device,
Imaging a plurality of barcodes;
Recognizing the first information recorded as the barcode for each barcode based on the imaging;
Determining a positional relationship between the plurality of barcodes based on the imaging;
Transmitting the recognized plurality of first information to a server device;
Receiving, from the server device, second information specified based on the first information for each of the first information;
Displaying the plurality of pieces of second information based on the determined positional relationship. An information processing method in a server device that communicates with a terminal device that images a subject including a plurality of barcodes,
Each of the plurality of first information obtained by recognizing the first information recorded as the barcode for each barcode is used as the first information in the subject image obtained by the imaging. Receiving from the terminal device in association with second information representing the position of the recorded barcode;
Determining a positional relationship between the plurality of barcodes based on the plurality of second information;
Generating electronic data by arranging a plurality of third information specified by each of the plurality of first information using the determined positional relationship;
Transmitting the generated electronic data to the terminal device. An information processing method in a server device that communicates with a terminal device that images a subject including a plurality of barcodes,
A plurality of the first information obtained by recognizing the first information recorded as the barcode for each barcode, and a second information representing a positional relationship between the plurality of barcodes; Receiving from the terminal device;
Generating electronic data by arranging a plurality of third information specified by each of the plurality of first information using the second information;
Transmitting the generated electronic data to the terminal device. A program for controlling a terminal device,
Imaging a subject including a plurality of barcodes;
Recognizing the first information recorded as the barcode for each barcode based on the imaging;
Identifying the position of each of the plurality of barcodes in the subject image obtained by the imaging;
Associating each of the plurality of recognized first information with second information representing the position of the barcode on which the first information is recorded, and transmitting to the server device, A program to be executed by a processor. A program for controlling a terminal device,
Imaging a subject including a plurality of barcodes;
Recognizing the first information recorded as the barcode for each barcode based on the imaging;
Determining a positional relationship between the plurality of barcodes based on the imaging;
The program which makes the processor of the said terminal device perform the step which links | relates the said some recognized 1st information and the 2nd information showing the said positional relationship, and transmits to a server apparatus. A program for controlling a terminal device,
Imaging a plurality of barcodes;
Recognizing the first information recorded as the barcode for each barcode based on the imaging;
Determining a positional relationship between the plurality of barcodes based on the imaging;
Transmitting the recognized plurality of first information to a server device;
Receiving, from the server device, second information specified based on the first information for each of the first information;
A program causing the processor of the terminal device to execute the step of displaying the plurality of second information based on the determined positional relationship. A program for controlling a server device that communicates with a terminal device that images a subject including a plurality of barcodes,
Each of the plurality of first information obtained by recognizing the first information recorded as the barcode for each barcode is used as the first information in the subject image obtained by the imaging. Receiving from the terminal device in association with second information representing the position of the recorded barcode;
Determining a positional relationship between the plurality of barcodes based on the plurality of second information;
Generating electronic data by arranging a plurality of third information specified by each of the plurality of first information using the determined positional relationship;
A program for causing a processor of the server device to execute the step of transmitting the generated electronic data to the terminal device. A program for controlling a server device that communicates with a terminal device that images a subject including a plurality of barcodes,
A plurality of the first information obtained by recognizing the first information recorded as the barcode for each barcode, and a second information representing a positional relationship between the plurality of barcodes; Receiving from the terminal device;
Generating electronic data by arranging a plurality of third information specified by each of the plurality of first information using the second information;
A program for causing a processor of the server device to execute the step of transmitting the generated electronic data to the terminal device.

Images