JP6213513B2 - Information identification program and information identification method - Google Patents

Description

  The present invention relates to an information specifying program and an information specifying method.

  2. Description of the Related Art Conventionally, a store-specific receipt data collection device that collects form data such as receipts printed in a format different for each store is known. The receipt data collection device disclosed in Patent Document 1 stores store information including print layout information of receipts and item names printed on receipts in association with identification codes for the stores. The receipt data collection device acquires the store information and the image information of the receipt by reading a receipt sandwiched between receipt sandwiching sheets on which a code image indicating an identification code for each store is displayed. An optical character recognition (OCR) process and a process corresponding to the store information are performed on the acquired image information of the receipt.

JP 2012-32979 A

  In order to use the receipt data collection device described in Patent Document 1, it is necessary to prepare a receipt holding sheet for each store in order to specify store information. For example, an individual user may use the receipt data collection device instead of the tenant management department, but the store used by the individual user is not always predetermined as the management target store of the tenant management department. . For this reason, it is complicated for an individual user to prepare a receipt sandwiching sheet for each store. For this reason, it is desirable that the store information can be specified directly from the image information of the form without using the identification code for each store displayed on the receipt holding sheet. In specifying store information, it may be possible to specify store information by performing OCR processing on image information of a form and extracting a store name or the like based on the processing result. However, the store name printed on the form is unique to each store, and a special font may be used for printing the store name, or the store name may be printed with decorative characters such as logo marks. is there. Specifying such a store name by OCR processing specifies a character string (for example, “total amount”, “consumption tax amount”, etc.) that is commonly used as an item name printed on a receipt or a numeric string representing the amount. There is a problem that it is very difficult.

  An object of the present invention is to provide an information specifying program and an information specifying method capable of automatically specifying a form issuer from image data corresponding to the print contents of a form.

The information specifying program exemplified in this specification is an information specifying program executed by a computer constituting the information specifying device, and is sent from the image reading device connected to the information specifying device to the computer for each issuer. An image data acquisition step of acquiring image data obtained by reading a form issued in a different format, a text data acquisition step of acquiring text data from the image data, and analyzing an array of characters in the text data, An extraction step of extracting a part of text data as a search key, a transmission step of transmitting the search key to a database connected to the information specifying device via a network, and the publisher corresponding to the search key Identification information including identifiable information is received from the database. A plurality of form templates in which the issuer is associated with the receiving step, the specifying step of specifying the issuer based on the identification information, and the print layout information of the form and item information indicating items to be printed on the form A template acquisition step for acquiring a form template corresponding to the issuer specified in the specifying step from the storage means for storing, and the text data acquisition step with reference to the form template acquired in the template acquisition step By analyzing the text data acquired in step, an item value acquisition step of acquiring an item value that is a value corresponding to the item information, and the item value acquired in the item value acquisition step Corresponds to the publisher specified in Characterized in that to execute the item value storage step of storing.

According to the information specifying program exemplified in this specification, when the computer acquires text data from the image data corresponding to the print contents of the form, the computer searches for identification information including information that can identify the issuer of the form. Extract the search key. The computer acquires identification information from the database based on the search key, and identifies the issuer from the identification information. The computer can automatically collect item values corresponding to the item information printed on the form, using the form template associated with the specified issuer. Therefore, the information specifying program according to the first aspect of the present invention can automatically specify the form issuer from the image data corresponding to the print contents of the form.

1 is an overall configuration diagram of an information identification system 100. FIG. This is a specific example of the receipt 110. It is a figure which shows the data structure of template DB50. It is a figure which shows the data structure of receipt analysis result DB60. It is a flowchart which shows an information specific process. It is a flowchart which shows an information specific process. It is a flowchart which shows a receipt analysis process. This is a specific example of the XML document 70. This is a specific example of the HTML document 80. 5 is a specific example of a reading completion screen 130.

  Embodiments of the present invention will be described with reference to the drawings. First, the system configuration of the information specifying system 100 will be described with reference to FIG. The information specifying system 100 includes an information specifying device 1, an image reading device (hereinafter referred to as “scanner”) 2, a management server 3, and a service server 4. The information specifying device 1 can transmit / receive data and commands to / from the management server 3 and the service server 4 connected to the LAN 9 or the service server 4 connected to the Internet 8 via the Internet 8.

  The configuration of the information specifying device 1 will be described. The information specifying device 1 is a device that can specify a form issuer from image data corresponding to the print contents of a form issued in a different format for each issuer. The form mentioned here may be a slip indicating the contents of consumption activity, such as a receipt, a receipt, or a billing statement for mobile phone / gas / water / electricity. Unless otherwise specified, the following description will be made assuming that the form is a receipt. The information identification device 1 may be a device dedicated to information identification, or may be a general-purpose device such as a so-called personal computer. In this embodiment, a general-purpose information specifying device 1 is illustrated. As shown in FIG. 1, the information specifying device 1 includes a CPU 10 that is a controller that controls the information specifying device 1. The CPU 10 is electrically connected to a ROM 11, a RAM 12, a hard disk drive (hereinafter referred to as “HDD”) 13, and an input / output interface (hereinafter referred to as “input / output I / F”) 14 via a data bus. . The ROM 11 stores a program such as a BIOS for operating the information specifying device 1, an initial value, and the like. The RAM 12 temporarily stores various data such as calculation results obtained by calculation processing by the CPU 10.

  The HDD 13 is a non-volatile storage device, and stores a program that causes the CPU 10 to execute an information specifying process (see FIGS. 5 and 6) described later. The HDD 13 stores a template DB 50 (see FIG. 3) and a receipt analysis result DB (see FIG. 4), which will be described later, as a database (hereinafter referred to as “DB”) for managing various types of information. The HDD 13 is an example of a non-temporary storage medium. The non-temporary storage medium may be any storage medium that can store information regardless of the period in which the information is stored. The non-transitory storage medium may not include a temporary storage medium (for example, a signal to be transmitted). The storage device in the present embodiment is the HDD 13, but the storage device is composed of another non-transitory storage medium that can hold data regardless of the length of time for storing information, for example, flash memory, RAM, and the like. May be.

O I / F1 4 performs mediation of transfer of data. The input-output I / F1 4 is an external communication interface (hereinafter, "external communication I / F" hereinafter.) 15, a display unit 16 for displaying an image, an operation unit 17 such as a keyboard and a mouse, a general-purpose image A scanner 2 as a reading device is electrically connected. The external communication I / F 15 connects the information specifying device 1 to the Internet 8.

The information specifying process (see FIGS. 5 and 6) is a part of the process of the household account book management software for managing the household account book. O I / F1 4 is for connecting the external device such as information identifying device 1 and the scanner 2, a predetermined standard (e.g., Universal Serial Bus (USB)) is an interface device adapted to. CPU10 via the input-output I / F1 4, for transmitting and receiving data to and from external devices such as the scanner 2. The scanner 2 performs a reading operation for capturing an image of the receipt 110 or the like according to an instruction from the information specifying device 1, forms image data, and transmits the image data to the information specifying device 1. The scanner 2 is a sheet feed scanner or a flat bed scanner. The scanner 2 may be a read-only scanner for the receipt 110 or a general-purpose scanner.

  The management server 3 may be a general-purpose server. The management server 3 includes a CPU 30 that controls the management server 3. The CPU 30 is electrically connected to the ROM 31, RAM 32, HDD 33, and input / output I / F 34 via a data bus. The ROM 11 stores a program such as a BIOS for operating the management server 3 and an initial value. The RAM 32 temporarily stores various data such as calculation results obtained by calculation processing by the CPU 30. The HDD 33 stores various settings. An external communication I / F 35 is connected to the input / output I / F 34. The external communication I / F 35 connects the management server 3 to the LAN 9.

  The service server 4 is a general-purpose server that provides various web services. The service server 4 includes external service servers such as a location information service server 41 and a search service server 42. The location information service server 41 is an external service server that provides a location information service that can provide location information corresponding to a character string. As an example of the location information service, for example, there is a Google Places API provided by Google (registered trademark). The Google Places API is an external application programming interface (API) that can transmit information about facilities, geographical locations, and famous spots corresponding to an arbitrary character string indicating a telephone number or the like to Internet users. The search service server 42 is an external service server that provides a search service for searching for information corresponding to an arbitrary character string from a web page accessible on the Internet 8. An example of a search service is a search engine provided by Google (registered trademark), for example. As a search result by the search engine, a web page determined by the search engine to match the search character string used for the search is presented, or a search corresponding to the number of web pages determined by the search engine to match the search character string The number of hits is presented.

  The receipt 110 will be described with reference to FIG. The receipt 110 is an example of a receipt received from a store clerk when the user makes a purchase at a store having the store name “XXX STORE”. The receipt 110 includes various item information indicating the contents of consumption activities. The logo column 111 shows a logo mark of the store. A logo column 111 of the receipt 110 indicates that shopping is performed at a store having a store name “XXX STORE”. The address column 112 indicates the store address, and the phone number column 113 indicates the store phone number. The date column 114 shows a purchase date. The item column 115 shows the purchased product name, and the unit price column 116 shows the purchase price for each product. The total payment column 117 indicates the total payment amount, and the tax amount column 118 indicates a tax amount such as a consumption tax. The payment method column 119 shows a payment method. The advertisement columns 121 and 122 indicate advertisements, notices, etc. for consumers. In the receipt 110, the advertisement column 122 includes URL information 123 indicating the URL of the store.

  A template DB 50 stored in the HDD 13 will be described with reference to FIG. In the template DB 50, a plurality of receipt templates (hereinafter simply referred to as “templates”) in which item information indicating items to be printed on the receipt 110 are associated with the name of the store from which the receipt 110 is issued are registered. In the example shown in FIG. 3, the template DB 50 includes “total payment”, “tax amount”, “date”, and “payment method” that are item information necessary for household account management among the item information included in the receipt 110. It is registered. The template DB 50 may include item information other than the example shown in FIG. The T-ID is an ID given by the CPU 10 every time a template is registered in the template DB 50.

  For each issuer of the receipt 110, there are various forms for printing the item information “total payment” on the receipt 110. In the template DB 50, as shown in FIG. 2, in the receipt 110 of the store name “XXX STORE”, “total payment” is printed with the item name “total”. Therefore, as illustrated in FIG. 3, the item name “total” is associated with the item information “total payment” in the template with the T-ID “T0001”. Similarly, for the templates with T-IDs “T0002” to “T0005”, the item names “Total”, “AMOUNT”, “total”, and “total” are associated with the item information “total payment”, respectively. . Similarly, for item information “tax amount”, “date”, and “payment method”, item names to be printed corresponding to the item information are registered in the receipts 110 issued by the respective stores.

  Note that the template DB 50 is devised so that the date from the receipt 110 can be accurately read by expressing the print format of the date that is the corresponding item name with a regular expression for the item information “date”. . Specifically, in the T-ID “T0001”, the item information “date” is displayed on the receipt 110 as “(4-digit number) / (1-digit or 2-digit number) / (1-digit or 2-digit number)”. ”(For example,“ A.D./Month/Day ”) is associated. For the T-ID “T0002”, the item information “Date” is displayed on the receipt 110 as “(4-digit number) (1-digit or 2-digit number) (1-digit or 2-digit number) (2-digit number): (2 digit number): (2 digit number) ”format (for example,“ A.D. month day hour: minute: second ”format) is associated. Further, the item information “payment method” is associated with which of a plurality of payment methods such as cash, BB card, CC card, and DD card can be printed on the receipt 110. Here, “BB card”, “CC card”, and “DD card” indicate the type of credit card.

  The print layout of the receipt 110 may differ depending on the manufacturer of the cash register, which is generally called “register”. In addition, the type of item information included in the receipt 110 and the printing position of each item information may be different for each issuer of the receipt 110. Although not shown, the template DB 50 sets print layout information such as a print start position of the receipt 110, a reading range for reading an image from the receipt 110, and a printing position of each item information in the reading range for each template.

The receipt analysis result DB 60 stored in the HDD 13 will be described with reference to FIG. In the receipt analysis result DB 60, each value acquired as an item value corresponding to the item information of the receipt 110 in the receipt analysis process (see FIG. 7) performed in the information specifying process (see FIGS. 5 and 6) described later. Is stored for each receipt 110. The receipt analysis result DB 60 stores R-ID, T-ID, total payment amount, tax amount, date, and payment method in association with each other. The R-ID is an ID assigned by the CPU 10 for each receipt 110 when information is stored in the receipt analysis result DB 60. The T-ID indicates which template is the template corresponding to the store name indicating the issuer of the receipt 110. The store name of the issuer of the receipt 110 may be stored instead of the T-ID. The total payment amount is a value of the total payment amount printed on the receipt 110. The tax amount is the value of the tax amount printed on the receipt 110. The date is the date printed on the receipt 110. The payment method is a payment method printed on the receipt 110. In addition, the receipt analysis result DB 60 may store image data obtained by reading the receipt 110 with the scanner 2 for each receipt 110 in association with the R-ID or the like.

  With reference to FIG. 5 to FIG. 10, information specifying processing in the present embodiment will be described. The information specifying process is a process of specifying the store name that is the issuer of the receipt 110 from the information described in the receipt 110 based on the image data of the receipt image obtained by capturing the receipt 110. Further, in the information specifying process, the item value corresponding to the item information is acquired from the receipt 110 with reference to the template corresponding to the specified store name, and each acquired item value is stored in the receipt analysis result DB 60 for each receipt 110. It is processing to do. Hereinafter, a case where the receipt 110 (see FIG. 2) is read by the scanner 2 and each item value is stored in the receipt analysis result DB 60 for each receipt 110 will be described as a specific example. The specific example shows a process of storing information associated with the R-ID “R0001” illustrated in FIG. 4 from a state in which no information is stored in the receipt analysis result DB 60.

  As shown in FIG. 5, when the information specifying process is started, the CPU 10 executes image reading (S1). Specifically, the CPU 10 transmits an instruction to perform an image reading operation to the scanner 2. Upon receiving the instruction, the scanner 2 captures the receipt 110, generates image data of the receipt image, and transmits it to the CPU 10. The CPU 10 receives the image data transmitted from the scanner 2 and stores it in the RAM 12.

  The CPU 10 acquires text data from the received image data (S2). The text data is data indicating characters included in the image data of the receipt 110 and information corresponding to the item information of the receipt 110. That is, the CPU 10 extracts a character from the image data stored in the RAM 12 by extracting a feature corresponding to the outline of the character from the image and executing a known OCR process for comparing with a standard pattern of the character. The CPU 10 analyzes the extracted character and specifies the character printed on the receipt 110. The CPU 10 generates text data indicating the specified character and stores it in the RAM 12. At this time, the CPU 10 acquires a text start coordinate and a text end coordinate corresponding to the specified text data. In the OCR process, a known library including a feature extraction step such as a known edge detection technique and a matching step for comparing the extracted feature with a character template can be used.

  As shown in FIG. 2, in the logo column 111, a special font in which each character has a border is used for the portion of “XXX” among characters indicating the store name “XXX STORE”. For the “STORE” portion, a handwritten-style special font is used. Further, an arcuate pattern is arranged so as to overlap a part of the characters “STORE”. When a special font is used for a character or an OCR process is performed for such a logo mark in which a character and a design are combined, the character “XXX STORE” indicating the store name is difficult to be identified from the logo mark. Sometimes. In the following description, it is assumed that the character “XXX STORE” indicating the store name is not specified by the OCR process from the logo mark in the logo column 111 and that the character indicating “XXX STORE” is not included in the text data.

  CPU10 cuts out and acquires the character string of the predetermined range among the character strings contained in the acquired text data (S3). The CPU 10 stores the acquired character string in the RAM 12. In the present embodiment, the predetermined range is a predetermined range in the text start coordinate direction from the text start coordinate of the text data, and a predetermined range in the text start coordinate direction from the text end coordinate. That is, the CPU 10 determines a character string based on information printed on the side near the upper end of the receipt 110 and a character string based on information printed on the side near the lower end of the receipt 110 among the character strings included in the text data. cut. In the present embodiment, data including a character string cut out by the CPU 10 is defined as first text data.

  The process of S3 is performed as a pre-process for identifying the store name that is the issuer of the receipt 110 from the text information obtained from the image data of the receipt 110. As shown in FIG. 2, the information printed in the item column 115, unit price column 116, total payment column 117, tax amount column 118, and payment method column 119 of the receipt 110 is often information that is not related to the store name. The store name is difficult to identify from the information in these fields. Although the print layout of the receipt 110 is various, the item column 115, unit price column 116, total payment column 117, tax amount column 118, and payment method column 119 are arranged at the center of the receipt 110 in the vertical direction (for example, the longitudinal direction). Relatively often printed. For this reason, it is preferable to perform the process for specifying a subsequent store name based on the item information printed in the area other than the central portion of the receipt 110. Therefore, the CPU 10 acquires, in the process of S3, the first text data excluding the character string printed in the central portion of the receipt 110 from the text data acquired in the process of S2. Note that the method of acquiring the first text data from the text data is not limited to this. For example, when acquiring the text data, the CPU 10 may acquire the number of character strings in the text data together. In this case, the CPU 10 includes data corresponding to the character string included in the predetermined number of lines from the start position to the end position of the text data and the character string included in the predetermined number of lines from the end position to the start position of the text data. Can be acquired as the first text data.

The CPU 10 refers to the first text data stored in the RAM 12 and searches for a character string indicating a telephone number from the first text data (S4). As shown in FIG. 2, the telephone number of the issuer is printed in the telephone number column 113 of the receipt 110 in the format “052-333-XXX”. For each receipt 110, the printing format of the telephone number in the telephone number column 113 varies depending on the presence / absence of “-(hyphen)”, the presence / absence of “() (parentheses)”, the type of parenthesis used, and the like. In the present embodiment, the CPU 10 searches for the character string indicating the telephone number from the first text data, using the unique character arrangement in the character string including the number for expressing the telephone number. For example, the CPU 10
(\ D {2,4})-(\ d {2,4})-(\ d {4})
(\ D {2,4}) (\ d {2,4}) (\ d {4})
(\ D {2,4})-(\ d {2,4})-(\ d {4})
[(\ D {2,4})] (\ d {2,4})-(\ d {4})
A character string indicating a telephone number is searched from a character string included in the first text data by pattern matching using a regular expression such as. When the character string indicating the telephone number can be retrieved, the CPU 10 extracts the character string indicating the telephone number from the first text data and stores it in the RAM 12. Hereinafter, a character string indicating a telephone number is referred to as a “telephone number string”. That is, the CPU 10 extracts a telephone number string from the first text data as a search key for acquiring location information that can identify the store name using the location information service.

  Depending on the receipt 110, the telephone number column 113 is not provided, so the CPU 10 cannot always extract the telephone number string from the first text data. The CPU 10 determines whether or not the telephone number string has been extracted (S5). When the telephone number string can be extracted (S5: YES), the CPU 10 transmits the extracted telephone number string to the location information service server 41 (S6). For example, when the location information service server 41 is the service server 4 that provides the above Google Places API, the CPU 10 transmits an HTTP request based on the telephone number string to a predetermined URL indicating the location information service server 41.

  The location information service server 41 outputs, for example, an XML format data file (hereinafter referred to as “XML file”) as an HTTP response to an HTTP request relating to a character string such as a telephone number string. This XML file includes position information corresponding to the HTTP request. When the XML file including the position information is transmitted as an HTTP response from the position information service server 41, the CPU 10 receives the transmitted XML file.

  When there is an error in the telephone number indicated by the telephone number string, or when the information corresponding to the telephone number indicated by the telephone number string is not stored in the location information service server 41, an effective request corresponding to the HTTP request based on the telephone number string is obtained. An XML file including location information may not be transmitted from the location information service server 41 to the CPU 10. For some reason, a plurality of pieces of location information corresponding to the telephone number string may be detected by the API. The CPU 10 determines whether or not an XML file including valid position information corresponding to the HTTP request has been received (S7). When the CPU 10 refers to the XML file stored in the RAM 12 and receives an XML file including valid position information (S7: YES), the CPU 10 determines whether or not the received XML file includes a plurality of pieces of position information. Is determined (S8).

  When a plurality of pieces of position information are included in the XML file (S8: YES), the CPU 10 acquires an XML file including a plurality of pieces of position information (S9). The CPU 10 stores the acquired XML file in the RAM 12. That is, the CPU 10 acquires an XML file including a plurality of pieces of position information as identification information for identifying the store name. Next, the CPU 10 acquires current position information indicating the position information of the current position (S10). The current position information of the information specifying device 1 is obtained by a well-known method such as using Geolocation API defined by W3C (registered trademark), which is a web standardization organization, by wireless LAN, WiFi, mobile phone base station, GPS, IP, etc. It can be obtained via an address. The CPU 10 analyzes the XML file, and among the plurality of pieces of position information, stores the name of the store that is the issuer of the receipt 110 corresponding to the position information that indicates the position closest to the current position information specified in the process of S10. Specify (S11). Thereafter, the CPU 10 shifts the process to S20.

  On the other hand, when the position information included in the XML file is one (S8: NO), the CPU 10 acquires an XML file including one position information (S12). The CPU 10 stores the acquired XML file in the RAM 12. That is, the CPU 10 acquires an XML file including one piece of position information as identification information for identifying the store name. The CPU 10 analyzes the XML file and identifies the store name that is the issuer of the receipt 110 corresponding to the position information included in the XML file (S13). Thereafter, the CPU 10 shifts the process to S20.

  The store name identification using the telephone number string in the processing from S6 to S13 will be specifically described. FIG. 8 is an example of an XML document 70 indicating the contents of an XML file that is an HTTP response transmitted from the location information service server 41 in response to an HTTP request based on the telephone number string transmitted from the CPU 10 to the location information service server 41. is there. The XML document 70 indicates the contents of an XML file including valid position information corresponding to the telephone number string transmitted by the CPU 10 to the position information service server 41 in the process of S6. The XML document 70 shows the search result searched by the API corresponding to the telephone number string.

  When the XML file includes valid position information, the XML document 70 includes a name element 71 surrounded by “<name>” and “</ name>”, “<geometry>” and “</ geometry>”. Each includes a geometry element 72 surrounded by. The name element 71 and the geometry element 72 are elements indicating a part of the search result of the position information corresponding to the telephone number string. The name element 71 mainly includes names of stores, services, etc. corresponding to the search result of the position information. The geometry element 72 includes a value indicating the longitude and latitude of the position information corresponding to the telephone number string. As shown in FIG. 8, the name element 71 includes “XXX STORE” which is a store name searched by the API corresponding to the telephone number string. Further, the geometry element 72 includes a value indicating the longitude and latitude of the position information searched by the API corresponding to the telephone number string. The CPU 10 acquires an XML file including position information (S12). The CPU 10 stores the acquired XML file in the RAM 12. The CPU 10 refers to the acquired XML file, pays attention to the description of the name element 71 and the geometry element 72, and specifies the store name that is the issuer of the receipt 110 corresponding to the position information (S13).

  Further, the XML document 70 may include a plurality of name elements 71 and geometry elements 72. In such a case, the CPU 10 determines that a plurality of pieces of position information are included in the XML file (S8: YES). The CPU 10 acquires an XML file including a plurality of pieces of position information (S9). The CPU 10 stores the acquired XML file in the RAM 12. The CPU 10 refers to the acquired XML file and extracts a geometry element 72 indicating the history closest to the current location from among the values indicating the history indicated by the plurality of geometry elements 72. The CPU 10 specifies the store name from the name element 71 corresponding to the extracted geometry element 72 (S11).

  In this embodiment, the CPU 10 pays attention to the name element 71 and the geometry element 72 and identifies the store name from which the receipt 110 is issued. Various corresponding element information is included. The CPU 10 may specify the store name based on other element information included in the XML document 70.

  The format of the XML document 70 may be different for each location information service. In addition, the location information service server 41 may transmit the location information search result to the CPU 10 in a format other than the XML format. The CPU 10 may analyze the search result contents in accordance with the position information search result transmitted from the position information service server 41 and specify the store name of the receipt 110. There are various types of location information service server 41. The CPU 10 uniformly analyzes the location information search result for specifying the store name by previously unifying the location information service server 41 that transmits the telephone number string in the processing of S6 into a specific one. be able to.

  Returning to the description of FIG. On the other hand, when the telephone number string cannot be extracted from the first text data (S5: NO), or the CPU 10 cannot receive an XML file including valid location information corresponding to the telephone number string (S7: NO). ), The process proceeds to S14. In the following description, it is assumed that the CPU 10 could not extract a telephone number string from the first text data or could not receive an XML file containing valid location information corresponding to the telephone number string. CPU10 searches the character string which shows URL from 1st text data with reference to the 1st text data memorize | stored in RAM12 (S14). As shown in FIG. 2, the advertisement column 122 of the receipt 110 includes URL information 123. In general, a special character string such as “http: //”, “https: //”, “www.”, Or the like is used for the notation of the URL. The CPU 10 retrieves the URL information 123 from the first text data using a specific character string used for such URL notation. If the URL information 123 can be retrieved, the CPU 10 extracts a character string indicating the URL indicated by the URL information 123 from the first text data and stores it in the RAM 12. That is, CPU10 extracts the character string which shows URL from 1st text data as a search key for acquiring the HTTP response which can identify a shop name using a web service.

  Depending on the receipt 110, the URL information 123 is not printed, so the CPU 10 cannot always extract the URL information 123 from the first text data. The CPU 10 determines whether or not the URL information 123 has been extracted (S15). When the URL information 123 can be extracted (S15: YES), the CPU 10 transmits an HTTP request to the service server 4 corresponding to the URL in order to access the URL indicated by the extracted URL information 123 (S16).

  When the service server 4 corresponding to the URL indicated by the URL information 123 is operating effectively, the service server 4 transmits an HTTP response to the CPU 10. When the HTTP response is transmitted from the service server 4 corresponding to the URL, the CPU 10 receives the transmitted HTTP response.

  The CPU 10 determines whether an HTTP response is received from the service server 4 corresponding to the URL (S17). When receiving the HTTP response (S17: YES), the CPU 10 acquires header information of the HTTP response (S18). The CPU 10 stores the acquired header information of the HTTP response in the RAM 12. That is, the CPU 10 acquires header information of the HTTP response as identification information for identifying the store name. The CPU 10 refers to the header information of the acquired HTTP response and specifies the store name that is the issuer of the receipt 110 (S19). Thereafter, the CPU 10 shifts the process to S20.

  The store name identification based on the header information of the HTTP response in the process of S19 will be specifically described. FIG. 9 shows an HTML format data file (hereinafter referred to as “HTML file”) corresponding to an HTTP response transmitted from the service server 4 in response to an HTTP request transmitted from the CPU 10 to the service server 4 corresponding to the URL. It is an example of the HTML document 80 which shows the content. The HTML document 80 indicates the contents of the HTML file corresponding to the HTTP response transmitted from the service server 4 that operates effectively in response to the HTTP request transmitted from the CPU 10 to the service server 4 corresponding to the URL in the process of S15. And In FIG. 9, an HTML document 80 is a response body (“<body>” and “</ </ part>”, which is a part that mainly stores information necessary for displaying a web page in an HTML file constituting an HTTP response. The description of the part surrounded by “body>” is omitted.

  As shown in FIG. 9, the header information of the HTTP response is a portion surrounded by “<head>” and “</ head>” in the HTML file. The header information indicates the contents, attributes, etc. of the HTTP response. The header information includes a title element 81 in principle. The title element 81 of the header information is an element that represents the title of the web page, and corresponds to the wording that is shown in the search result in the search engine or displayed in the title bar of the browser. For this reason, it is considered that a person who opens a web page for introducing a store devises the title element 81 to include a store name in the title element 81. Therefore, in the present embodiment, the CPU 10 pays attention to the title element 81 of the header information and identifies “XXX STORE” described in the title element 81 as the name of the store from which the receipt 110 is issued.

  Since there are various web pages, it may be possible to specify the name of the store from which the receipt 110 is issued by searching for data such as images included in the response body. Therefore, the CPU 10 may identify the store name that is the issuer of the receipt 110 by referring to the response body in the HTML file of the HTTP response and analyzing the content.

  Returning to the description of FIG. On the other hand, when the URL information 123 cannot be extracted from the first text data (S15: NO), or the HTTP response cannot be received (S17: NO), the CPU 10 proceeds to S21 (see FIG. 6). To do. In the following description, it is assumed that the CPU 10 could not extract the URL information 123 from the first text data or could not receive an HTTP response from the service server 4 corresponding to the URL corresponding to the extracted URL information 123.

  As shown in FIG. 6, the CPU 10 extracts a plurality of character strings indicating words from the upper text data in the first text data (S21). As described above, in the present embodiment, the first text data is a character string based on information printed on the side closer to the upper end of the receipt 110 among the character strings included in the text data, and closer to the lower end of the receipt 110. Contains character strings based on information printed on the side. The upper text data refers to data composed of a character string based on information printed on the side closer to the upper end of the receipt 110 in the first text data.

  As shown in FIG. 2, on the side close to the upper end of the receipt 110, direct contents about the store, such as the advertisement column 121 and the address column 112, may be printed. On the other hand, on the side close to the lower end of the receipt 110, indirect contents about the store such as the advertisement column 122 may be printed. For this reason, in this embodiment, paying attention to the upper text data, a plurality of character strings indicating words are cut out from the character strings included in the upper text data, and a plurality of character strings indicating the extracted words are used to store the store name. Trying to identify. As for the method of cutting out the character string indicating the word from the upper text data, the character string is divided into meaningful words using a known morphological analysis tool, and the character string is kanji or hiragana using a regular expression. Various methods can be used, such as separating katakana and alphanumeric boundaries. That is, the CPU 10 extracts a plurality of words from the upper text data as a search key for acquiring a search result that can identify the store name using the search service.

  Note that the print format of the receipt 110 is various, and a valid word for specifying the receipt 110 issuance store name may be easily obtained from text data other than the upper text data. For this reason, the text data that is the target of extracting words is not limited to the upper text data. The developer of the information specifying device 1 may determine the range of text data to be extracted from words based on a prior experiment or the like.

  In the case of the receipt 110 shown in FIG. 2, the CPU 10 can extract, from the upper text data, for example, “Western confectionery” “XXX STORE” from the advertisement column 122, the store address printed in the address column 112, and the like as words. The CPU 10 transmits each of these words extracted in the process of S21 to the search service server 42 (S22). CPU10 receives the search result by the search service corresponding to each word, and acquires for every word (S23). The CPU 10 stores the search result acquired for each word in the RAM 12. That is, the CPU 10 acquires a search result by the search service corresponding to each word as identification information for identifying the store name. The CPU 10 determines whether or not all the words extracted in the process of S21 have been transmitted to the search service server 42 (S24). CPU10 repeats the process of S22 and S23, when not having extracted all the extracted words to the search service server 42 (S24: NO).

  When all the extracted words are transmitted to the search service server 42 (S24: YES), the CPU 10 refers to the search result for each word, and selects the word corresponding to the search result having the maximum number of search hits as the receipt 110. It is specified as the store name that is the issuer of (S25). In the present embodiment, when the word “XXX STORE” is extracted from the upper text data, and the number of search hits corresponding to the word “XXX STORE” is higher than the other extracted words, the CPU 10 determines that “XXX “STORE” is specified as the store name. Thereafter, the CPU 10 shifts the processing to S19 (see FIG. 5).

  Returning to the description of FIG. The CPU 10 executes a receipt analysis process (S19). The receipt analysis process is a process of analyzing text data based on the template corresponding to the store name specified in any one of S11, S12, S18, and S25, and registering the analysis result in the receipt analysis result DB 60. is there.

  The receipt analysis process (S20, see FIG. 5) will be described with reference to FIG. As shown in FIG. 7, when the receipt analysis process is started, the CPU 10 refers to the template DB 50 stored in the HDD 13 and performs any of the processes S11, S12, S18, and S25 (see FIG. 5). A template corresponding to the specified store name is acquired (S41). The CPU 10 stores the acquired template in the RAM 12. In the present embodiment, it is assumed that the store name “XXX STORE” is specified by any one of the processes S11, S12, S18, and S25 (see FIG. 5). In this case, the CPU 10 acquires a template with the T-ID “T0001” associated with the store name “XXX STORE” from the template DB 50.

  CPU10 acquires each item value which is values, such as a character corresponding to the item information of a template, and a number among the character strings contained in text data based on the acquired template of T-ID "T0001" ( S42). The CPU 10 associates each item value acquired in S42 and stores it in the receipt analysis result DB 60 of the HDD 13 (S43). In this case, the CPU 10 creates a data area corresponding to the R-ID “R0001” in the receipt analysis result DB 60.

  The CPU 10 displays a reading completion screen 130 (see FIG. 10) for displaying the item value corresponding to the R-ID “R0001” stored in the receipt analysis result DB 60 of the HDD 13 on the display unit 16 (S44). As shown in FIG. 10, the reading completion screen 130 includes a result display field 131, an OK button 133, and a correction button 134. The CPU 10 displays each item value acquired from the receipt 110 and stored in the receipt analysis result DB 60 in the result display column 131. The CPU 10 accepts an editing operation such as correction of an item value for the data area corresponding to the R-ID “R0001” in the receipt analysis result DB 60 by operating the correction button 134 via the operation unit 17. The CPU 10 ends the storage of the item value in the receipt analysis result DB 60 by operating the OK button 133 via the operation unit 17. The CPU 10 returns the process to the information specifying process (see FIG. 5). Thereafter, the CPU 10 ends the information specifying process.

  As described above, the CPU 10 of the information specifying device 1 transmits an instruction to perform an image reading operation to the scanner 2 and receives image data of a receipt image obtained by capturing the receipt 110 from the scanner 2 (S1). When the CPU 10 obtains text data from the image data by OCR processing (S2), the CPU 10 obtains a telephone number string from the first text data as a search key for obtaining position information that can identify the store name from which the receipt 110 is issued. Extract (S4). Moreover, CPU10 extracts the character string which shows URL from 1st text data as a search key for acquiring the HTTP response which can identify a shop name using a web service (S14). Moreover, CPU10 extracts a some word from upper text data as a search key for acquiring the search result which can identify a shop name using a search service (S21). CPU10 acquires the identification information which can identify the store name which issued the receipt 110 based on each search key (S9, S12, S18, S23), and specifies a store name from identification information (S11, S13, S19, S25). Therefore, the information specifying device 1 can automatically specify the store name of the issuer of the receipt 110 from the image data corresponding to the print contents of the form.

  The receipt 110 may be printed with the telephone number of the issuer of the receipt 110. The telephone number may be printed as a character string including a numeric string having a specific arrangement. CPU10 can extract the character string which shows a telephone number as a search key by searching the character string containing the numerical sequence which has a specific arrangement | sequence from the text data corresponding to the image data of the receipt 110 (S4). Therefore, the information specifying device 1 can specify the issuer of the receipt 110 by using the location information service using a character string indicating a telephone number as a search key.

  The receipt 110 may be printed with a URL indicating the issuer of the receipt 110. The URL may be printed as a character string including a character string having a specific arrangement such as “http: //” or “www”. The CPU 10 can extract a character string indicating the URL as a search key by searching a character string having a specific arrangement indicating the URL from text data corresponding to the image data of the receipt 110 (S14). The CPU 10 transmits an HTTP request to the extracted URL (S16), and receives an HTTP response (S17). Therefore, the information specifying device 1 can specify the issuer of the receipt 110 by analyzing the header information of the HTTP response.

  Various characters (for example, store names) indicating the issuer of the receipt 110 may be printed on the receipt 110. In particular, the character string related to the issuer store name of the receipt 110 is often printed on the side closer to the upper end of the receipt 110. The CPU 10 extracts a plurality of character strings indicating words as search keys from the upper text data composed of character strings based on information printed on the side closer to the upper end of the receipt 110 in the first text data (S21). The CPU 10 transmits each of the extracted words to the search service server 42 (S22), receives the search results from the search service corresponding to each word, and acquires the search results for each word (S23). Therefore, the information specifying device 1 can specify the word corresponding to the search result having the maximum number of search hits as the name of the store from which the receipt 110 is issued with reference to the search result for each word.

  The information specifying device 1 can use various information that can be acquired via the Internet 8 or the LAN 9 as identification information for identifying the store name of the issuer of the receipt 110.

  The CPU 10 of the information specifying device 1 receives from the location information service server 41 an XML file containing location information corresponding to the telephone number string extracted as a search key as identification information for specifying the issuer of the receipt 110. It can be acquired (S9, S12).

  The arrangement of telephone number strings printed on the receipt 110 may be fixed. In addition, the URL may be printed on the receipt 110 less than the telephone number string. The CPU 10 extracts the telephone number string as a search key in preference to the character string indicating the URL (S5: NO, or S7: NO, S13). Therefore, the information specifying device 1 can improve the accuracy of specifying the issuer of the receipt 110.

  The CPU 10 of the information specifying device 1 receives from the search service server 42 search results corresponding to each of a plurality of character strings indicating words extracted as search keys as identification information for specifying the issuer of the receipt 110. (S23).

  The CPU 10 receives the HTTP response from the service server 4 (S17) by transmitting an HTTP request to the service server 4 corresponding to the character string indicating the URL extracted as the search key (S16). Therefore, the information specifying device 1 can acquire the header information of the received HTTP response as identification information for specifying the issuer of the receipt 110.

  The plurality of character strings indicating words extracted from the first text data as the search key may include character strings that do not contribute to specifying the issuer of the receipt 110. On the other hand, the character string indicating the URL printed on the receipt 110 may indicate information about the issuer of the receipt 110. The CPU 10 extracts a character string indicating a URL as a search key in preference to a plurality of character strings indicating a word (S15: NO, or S17: NO, S21). Therefore, the information specifying device 1 can improve the accuracy of specifying the issuer of the receipt 110.

  The service server 4 may open a web page that provides information about the issuer of the receipt 110. When the character string indicating the word is a character string corresponding to the store name indicating the issuer of the receipt 110, the number of search hits indicated by the search result information is higher than when the character string indicating the word does not indicate the issuer. It is considered to be. CPU10 specifies an issuer from search result information with the largest number of search hits among several search result information (S25). Therefore, the information specifying device 1 can improve the accuracy of specifying the issuer of the receipt 110.

  The XML file transmitted from the position information service server 41 includes position information corresponding to the telephone number string transmitted by the CPU 10. Therefore, the information specifying device 1 can improve the accuracy of specifying the issuer of the receipt 110 by using the location information service.

  There is a possibility that the user who wants to specify the issuer of the receipt 110 has received the receipt 110 from the issuer having the store name corresponding to the location information indicating the positional relationship closest to the current location. Therefore, the CPU 10 acquires the current position information of the information specifying device 1 (S10), and specifies the issuer from the position information indicating the closest positional relationship to the acquired current position information (S11). Therefore, the information specifying device 1 can improve the accuracy of specifying the issuer of the receipt 110.

  The header information of the HTTP response includes information such as a title element 81 that specifies the title of the web page, for example. The information specifying apparatus 1 can improve the accuracy of specifying the issuer by specifying the issuer of the receipt 110 from the header information of the HTTP response.

CPU10 acquires the template corresponding to the store name specified in each step of S11, S13, S19, S25 from template DB50 (S41). CPU10 acquires the item value corresponding to the item information of a template based on the acquired template (S42). The CPU 10 stores the acquired item value in the receipt analysis result DB (S43).
Therefore, the information specifying device 1 can automatically collect item values corresponding to the item information printed on the receipt 110 in association with the specified store name.

  The CPU 10 of the information specifying device 1 can specify the issuer of the receipt 110 from the image data corresponding to the print contents of the receipt 110 received from the scanner 2 according to the information specifying method of the above embodiment.

  In the above embodiment, the information specifying device 1 corresponds to the “information specifying device” of the present invention. The scanner 2 corresponds to the “image reading device” of the present invention. The HDD 13 that stores the template DB 50 and the receipt analysis result DB 60 corresponds to the “storage unit” of the present invention. The Internet 8 and the LAN 9 correspond to the “network” of the present invention. The service server 4 corresponds to the “database” of the present invention, the location information service server 41 corresponds to the “location information service server” of the present invention, and the search service server 42 corresponds to the “search service server” of the present invention. Equivalent to.

  The process of S1 corresponds to the “image data acquisition step” of the present invention. The process of S2 corresponds to the “text data acquisition step” of the present invention. The processes of S4, S14, and S21 correspond to the “extraction step” of the present invention, the process of S4 corresponds to the “telephone number search step” of the present invention, and the process of S14 corresponds to the “URL search step of the present invention. The process of S21 corresponds to the “word extraction step” of the present invention. The processes of S6 to S13, S16 to S19, S22, and S23 correspond to the “identification information acquisition step” of the present invention. The processes of S6, S16, and S22 correspond to the “transmission step” of the present invention, the process of S6 corresponds to the “telephone number transmission step” of the present invention, and the process of S16 corresponds to the “HTTP request transmission of the present invention. Step S22 corresponds to the “word transmission step” of the present invention. The processing of S7, S8, S17, and S23 corresponds to the “reception step” of the present invention, the processing of S7 and S8 corresponds to the “location information reception step” of the present invention, and the processing of S17 is the processing of the present invention. This corresponds to an “HTTP response receiving step”, and the process of S23 corresponds to a “search result receiving step” of the present invention. The processing of S11, S13, S19, and S25 corresponds to the “specific step” of the present invention. The process of S10 corresponds to the “present location acquisition step” of the present invention. The process of S41 corresponds to the “template acquisition step” of the present invention. The process of S42 corresponds to the “item value acquisition step” of the present invention. The process of S43 corresponds to the “item value storage step” of the present invention.

  In addition, this invention is not limited to said embodiment, A various change is possible. The information specifying device 1 of the present invention is not limited to a general-purpose device such as the personal computer of the above embodiment. For example, the information specifying device 1 may be a mobile terminal. The portable terminal may acquire the image data by capturing the receipt 110 with an imaging device such as a camera built in the portable terminal (S1), and may execute the processes after S2. Further, the method of acquiring image data corresponding to the print contents of the form is not limited to reading the form by the scanner 2, and the form image data is obtained by using an imaging device such as a camera capable of capturing the print contents of the form. You may get it.

  Further, the management server 3 may function as the information specifying device 1. In this case, the CPU 30 of the management server 3 receives image data obtained by imaging the receipt 110 transmitted via the Internet 8 and the LAN 9 from the information specifying device 1 that is a personal computer or other devices in the above embodiment. You may acquire (S1). The CPU 30 may acquire text data from the image data by executing a known character recognition process on the image data (S2), and execute the processes after S3.

  In the above embodiment, the CPU 10 extracts position information that can identify the store name, an HTTP response, and a search key for acquiring a search engine search result from the first text data (S4, S13, S21). . The extracted search key is transmitted to the external service server 4 such as the location information service server 41 and the search service server 42, but the configuration of the present invention is not limited to this. For example, the information specifying device 1 may acquire identification information that can identify the store name corresponding to the search key from a database stored in the HDD 13 in advance. In this case, as the database stored in the HDD 13, a telephone number list of the issuing store of the receipt 110, a URL list of the issuing store of the receipt 110, and a specific character string that is always printed on the receipt 110 by the issuing store of the receipt 110 are issued. A list associated with each store is applicable. These databases may store information equivalent to the information provided from the location information service server 41, the search service server 42, etc., or store information customized to specify the issuer of the receipt 110. It may be a thing. Such a database may be stored in the HDD 33 of the management server 3 or the like. The CPU 10 may acquire identification information via the Internet 8 or the LAN 9 by transmitting a search key to the CPU 30 of the management server 3.

  Either one or both of the template DB 50 and the receipt analysis result DB 60 may be stored in the HDD 33 of the management server 3.

  In the above embodiment, in the process of S21, the CPU 10 extracts a character string indicating a word from the upper text data in the acquired first text data. However, the present invention is not limited to this, and the word is extracted from the entire first text data. May be extracted. In each process of S4, S14, and S21, the CPU 10 extracts the search key from the first text data. However, the present invention is not limited to this, and the search key is extracted from the entire text data acquired in the process of S2. It may be extracted.

  In the above embodiment, the CPU 10 identifies the word corresponding to the search result having the maximum number of search hits as the name of the store that issued the receipt 110 in the process of S25. In this process, the CPU 10 may specify the store name based on, for example, the header information of the HTML file of the web page indicating the highest search result among the search results.

  In the above-described embodiment, store name specification using a telephone number string as a search key is performed in preference to store name specification using a character string indicating a URL as a search key. Further, store name identification using a character string indicating a URL as a search key is prioritized over store name identification using a character string indicating a URL as a search key. These priorities can be appropriately changed according to the tendency of the print contents of the receipt 110 and the like. Further, the information specifying device 1 does not have to execute all of these specifying methods and may appropriately select the specifying method.

  The CPU 10 functions as a processor that executes an information specifying process by expanding a program stored in the HDD 13 on the RAM 12. A general-purpose processor may be used as the CPU 10. A microcomputer, ASIC, FPGA, or the like may be used as a processor instead of the CPU 10. When the management server 3 functions as the information specifying device 1, the CPU 30 functions as a processor that executes information specifying processing by developing a program stored in the HDD 33 on the RAM 32. Similarly to the above, a microcomputer, ASIC, FPGA, or the like may be used as a processor instead of the CPU 30.

  The information specifying process may be distributed by a plurality of electronic devices (that is, a plurality of CPUs). For example, a part of the information specifying process may be executed by another server such as the management server 3 connected to the Internet 8. The program may be downloaded from another server such as the management server 3 connected to the Internet 8 and stored in the HDD 13 of the information specifying device 1.

1 Information Identification Device 2 Scanner 3 Management Server 4 Service Server 8 Internet 9 LAN
10 CPU
13 HDD
41 Location Information Service Server 42 Search Service Server 50 Template DB
60 Receipt analysis result DB

Claims (17)

An information identification program executed by a computer constituting the information identification device,
In the computer,
An image data acquisition step of acquiring image data obtained by reading a form issued in a different format for each issuer from an image reading device connected to the information specifying device;
A text data acquisition step for acquiring text data from the image data ;
And execute
The form includes at least one of a special font and a graphic, and the first area in which the issuer is indicated in a manner in which it is difficult to acquire the text data indicating the issuer in the text data acquisition step, A second area in which at least one of a unit price, a total payment amount, a tax amount, and a payment method is shown, and a third area different from the first area and the second area,
An extraction step of extracting a character string indicating a telephone number by searching a character string including a numeric string having a specific sequence from the text data extracted from the third region ;
A server providing a location information service capable of providing a data file including location information indicating a geographical location corresponding to a telephone number, the character string indicating the phone number extracted in the extraction step; A transmission step of transmitting to an external location information service server that is not a dedicated server for the information identification program ;
A receiving step of receiving, from the location information service server, a data file including location information corresponding to the character string indicating the telephone number transmitted in the transmission step ;
A specifying step of specifying the issuer based on position information included in the data file received in the receiving step ;
A form template corresponding to the issuer specified in the specifying step, from storage means for storing a plurality of form templates in which the print layout information of the form and item information indicating items printed on the form are associated with the issuer A template acquisition step for acquiring
Item value for acquiring an item value corresponding to the item information by analyzing the text data acquired in the text data acquisition step with reference to the form template acquired in the template acquisition step An acquisition step;
An information specifying program that further executes an item value storing step of storing the item value acquired in the item value acquiring step in association with the issuer specified in the specifying step. An information identification program executed by a computer constituting the information identification device,
In the computer,
An image data acquisition step of acquiring image data obtained by reading a form issued in a different format for each issuer from an image reading device connected to the information specifying device;
A text data acquisition step for acquiring text data from the image data;
And execute
The form includes at least one of a special font and a graphic, and the first area in which the issuer is indicated in a manner in which it is difficult to acquire the text data indicating the issuer in the text data acquisition step, A second area in which at least one of a unit price, a total payment amount, a tax amount, and a payment method is shown, and a third area different from the first area and the second area,
An extraction step of extracting a character string indicating a URL from the text data by searching the text data for a character string of a specific sequence indicating a URL from the text data extracted from the third region;
An external web server that corresponds to the URL indicated in the character string extracted in the extraction step and is connected to the Internet and provides a web service and is not a dedicated server for the image reading apparatus and the information specifying program A transmission step of transmitting an HTTP request to
A receiving step of receiving an HTTP response corresponding to the HTTP request transmitted in the transmitting step from the web server;
A specifying step of specifying the issuer based on an HTML element included in the HTTP response received in the receiving step;
A form template corresponding to the issuer specified in the specifying step, from storage means for storing a plurality of form templates in which the print layout information of the form and item information indicating items printed on the form are associated with the issuer A template acquisition step for acquiring
Item value for acquiring an item value corresponding to the item information by analyzing the text data acquired in the text data acquisition step with reference to the form template acquired in the template acquisition step An acquisition step;
An item value storing step for storing the item value acquired in the item value acquiring step in association with the issuer specified in the specifying step;
It characterized by further executing the information identification program. An information identification program executed by a computer constituting the information identification device,
In the computer,
An image data acquisition step of acquiring image data obtained by reading a form issued in a different format for each issuer from an image reading device connected to the information specifying device;
A text data acquisition step for acquiring text data from the image data;
And execute
The form includes at least one of a special font and a graphic, and the first area in which the issuer is indicated in a manner in which it is difficult to acquire the text data indicating the issuer in the text data acquisition step, A second area in which at least one of a unit price, a total payment amount, a tax amount, and a payment method is shown, and a third area different from the first area and the second area,
An extraction step of extracting a plurality of character strings indicating words from the text data extracted from the third region;
A server connected to the Internet that provides a search service for searching for information corresponding to an arbitrary character string from a web page accessible via the Internet for a plurality of character strings indicating the words extracted in the extraction step Transmitting to an external search service server that is not a dedicated server for the image reading device and the information specifying program;
A receiving step of receiving, from the search service server, search result information capable of specifying a search hit number indicating the number of web pages searched for a plurality of character strings indicating the words transmitted in the transmitting step;
Of the search result information received in the receiving step, a specifying step of specifying the word corresponding to the search result information having the maximum number of search hits as the issuer,
A form template corresponding to the issuer specified in the specifying step, from storage means for storing a plurality of form templates in which the print layout information of the form and item information indicating items printed on the form are associated with the issuer A template acquisition step for acquiring
Item value for acquiring an item value corresponding to the item information by analyzing the text data acquired in the text data acquisition step with reference to the form template acquired in the template acquisition step An acquisition step;
An item value storing step for storing the item value acquired in the item value acquiring step in association with the issuer specified in the specifying step;
It characterized by further executing the information identification program. An information identification program executed by a computer constituting the information identification device,
In the computer,
An image data acquisition step of acquiring image data obtained by reading a form issued in a different format for each issuer from an image reading device connected to the information specifying device;
A text data acquisition step for acquiring text data from the image data;
And execute
The form includes at least one of a special font and a graphic, and the first area in which the issuer is indicated in a manner in which it is difficult to acquire the text data indicating the issuer in the text data acquisition step, A second area in which at least one of a unit price, a total payment amount, a tax amount, and a payment method is shown, and a third area different from the first area and the second area,
An extraction step of extracting a plurality of character strings indicating words from the text data extracted from the third region;
A server connected to the Internet that provides a search service for searching for information corresponding to an arbitrary character string from a web page accessible via the Internet for a plurality of character strings indicating the words extracted in the extraction step Transmitting to an external search service server that is not a dedicated server for the image reading device and the information specifying program;
A reception step of receiving search result information indicating search results for a plurality of character strings indicating the words transmitted in the transmission step from the search service server;
Of the search result information received in the reception step, a specifying step of specifying a word corresponding to the search result information indicating the highest search result as the issuer,
A form template corresponding to the issuer specified in the specifying step, from storage means for storing a plurality of form templates in which the print layout information of the form and item information indicating items printed on the form are associated with the issuer A template acquisition step for acquiring
Item value for acquiring an item value corresponding to the item information by analyzing the text data acquired in the text data acquisition step with reference to the form template acquired in the template acquisition step An acquisition step;
An item value storing step for storing the item value acquired in the item value acquiring step in association with the issuer specified in the specifying step;
It characterized by further executing the information identification program. An information identification program executed by a computer constituting the information identification device,
In the computer,
An image data acquisition step of acquiring image data obtained by reading a form issued in a different format for each issuer from an image reading device connected to the information specifying device;
A text data acquisition step for acquiring text data from the image data;
And execute
The form includes at least one of a special font and a graphic, and the first area in which the issuer is indicated in a manner in which it is difficult to acquire the text data indicating the issuer in the text data acquisition step, A second area in which at least one of a unit price, a total payment amount, a tax amount, and a payment method is shown, and a third area different from the first area and the second area,
A first extraction step of extracting a character string indicating a telephone number by searching for a character string including a numeric string having a specific sequence from the text data extracted from the third region, extracted from the third region A second extraction step of extracting a character string indicating a URL from the text data by searching the text data for a character string having a specific sequence indicating the URL from the text data thus extracted. An extraction step including a third extraction step of extracting a plurality of character strings indicating words from the text data;
A server providing a location information service capable of providing a data file including location information indicating a geographical location corresponding to a telephone number, the character string indicating the phone number extracted in the first extraction step; Corresponding to the URL shown in the character string extracted in the first transmission step and the second extraction step to be transmitted to an external location information service server that is not a dedicated server for the device and the information specifying program, connected to the Internet A second providing step of sending an HTTP request to an external web server that is a server that provides a web service and is not a dedicated server for the image reading device and the information specifying program, and the words extracted in the third extracting step Access multiple strings shown via the Internet External search service server that is connected to the Internet and provides a search service for searching for information corresponding to an arbitrary character string from an active web page, and is not a dedicated server for the image reading device and the information specifying program A third transmission step for transmitting to the transmission step;
A first reception step of receiving a data file including location information corresponding to a character string indicating a telephone number transmitted in the first transmission step from the location information service server; an HTTP request transmitted in the second transmission step; It is possible to specify the number of search hits indicating the number of web pages searched for a plurality of character strings indicating words transmitted in the second reception step and the third transmission step of receiving a corresponding HTTP response from the web server. Receiving a search result information from the search service server; receiving a search result information indicating a search result for a plurality of character strings indicating a word transmitted in the third transmission step from the search service server; A receiving step including four receiving steps;
A first identifying step for identifying the issuer based on location information included in the data file received in the first receiving step; and an HTML element included in an HTTP response received in the second receiving step. Second identification step for identifying the issuer, and third identification for identifying the word corresponding to the search result information having the maximum number of search hits as the issuer among the search result information received in the third reception step A step of identifying, among the search result information received in the fourth reception step, a fourth specification step of specifying a word corresponding to the search result information indicating the highest-order search result as the issuer;
A form template corresponding to the issuer specified in the specifying step, from storage means for storing a plurality of form templates in which the print layout information of the form and item information indicating items printed on the form are associated with the issuer A template acquisition step for acquiring
Item value for acquiring an item value corresponding to the item information by analyzing the text data acquired in the text data acquisition step with reference to the form template acquired in the template acquisition step An acquisition step;
An item value storing step for storing the item value acquired in the item value acquiring step in association with the issuer specified in the specifying step;
Is executed further,
The first extraction step is executed after the image data acquisition step is executed, and if the execution of the first extraction step is successful, the first transmission step is executed, and after the execution of the first transmission step, the first transmission step is executed. When the first reception step is executed and the execution of the first reception step is successful, the first specific step is executed,
When the execution of the first extraction step fails or when the execution of the first reception step fails, the second extraction step is executed, and when the execution of the second extraction step is successful, the second transmission Step, execute the second reception step after the second transmission step, and if the second reception step is successful, execute the second specifying step,
If execution of the second extraction step fails or execution of the second reception step fails, execute the third extraction step, and execute the third transmission step after execution of the third extraction step is allowed, the third or to execute the third receiving step and the third specifying step after the execution of the transmission step, or information you characterized in that to execute the fourth receiving step and said fourth specific step Information identification program. An information specifying program executed by a computer constituting the information specifying device for specifying the issuer from image data corresponding to the print contents of the form issued in a format different for each issuer,
In the computer,
Executing a text data acquisition step of acquiring text data from the image data;
The form includes at least one of a special font and a graphic, and the first area in which the issuer is indicated in a manner in which it is difficult to acquire the text data indicating the issuer in the text data acquisition step, A second area in which at least one of a unit price, a total payment amount, a tax amount, and a payment method is shown, and a third area different from the first area and the second area,
An extraction step of extracting a character string indicating a telephone number by searching a character string including a numeric string having a specific sequence from the text data extracted from the third region;
A server providing a location information service capable of providing a data file including location information indicating a geographical location corresponding to a telephone number, the character string indicating the phone number extracted in the extraction step; An identification information acquisition step of acquiring a data file including location information corresponding to a character string indicating a telephone number by transmitting to an external location information service server that is not a dedicated server for the information identification program;
A specifying step of specifying the issuer corresponding to the character string indicating the telephone number extracted in the extraction step based on the data file acquired in the identification information acquisition step;
It characterized by further executing the information identification program. An information specifying program executed by a computer constituting the information specifying device for specifying the issuer from image data corresponding to the print contents of the form issued in a format different for each issuer,
In the computer,
Executing a text data acquisition step of acquiring text data from the image data;
The form includes at least one of a special font and a graphic, and the first area in which the issuer is indicated in a manner in which it is difficult to acquire the text data indicating the issuer in the text data acquisition step, A second area in which at least one of a unit price, a total payment amount, a tax amount, and a payment method is shown, and a third area different from the first area and the second area,
An extraction step of extracting a character string indicating a URL from the text data by searching the text data for a character string of a specific sequence indicating a URL from the text data extracted from the third region;
An external web server that corresponds to the URL indicated in the character string extracted in the extraction step and that is connected to the Internet and provides a web service, and is not a dedicated server for the information identification device and the information identification program An identification information acquisition step of acquiring an HTTP response corresponding to the HTTP request by transmitting an HTTP request to
A specifying step of specifying the issuer based on an HTML element included in the HTTP response acquired by the identification information acquiring step;
Further execute a you wherein Rukoto information identification program. An information specifying program executed by a computer constituting the information specifying device for specifying the issuer from image data corresponding to the print contents of the form issued in a format different for each issuer,
In the computer,
Executing a text data acquisition step of acquiring text data from the image data;
The form includes at least one of a special font and a graphic, and the first area in which the issuer is indicated in a manner in which it is difficult to acquire the text data indicating the issuer in the text data acquisition step, A second area in which at least one of a unit price, a total payment amount, a tax amount, and a payment method is shown, and a third area different from the first area and the second area,
An extraction step of extracting a plurality of character strings indicating words from the text data extracted from the third region;
A server connected to the Internet that provides a search service for searching for information corresponding to an arbitrary character string from a web page accessible via the Internet for a plurality of character strings indicating the words extracted in the extraction step The number of search hits indicating the number of web pages searched for a plurality of character strings indicating words is transmitted to an external search service server that is not a dedicated server for the information specifying device and the information specifying program. An identification information acquisition step of acquiring identifiable search result information;
Of the search result information acquired by the identification information acquisition step, a specifying step of specifying a word corresponding to the search result information having the maximum number of search hits as the publisher
It characterized by further executing the information identification program. An information specifying program executed by a computer constituting the information specifying device for specifying the issuer from image data corresponding to the print contents of the form issued in a format different for each issuer,
In the computer,
Executing a text data acquisition step of acquiring text data from the image data;
The form includes at least one of a special font and a graphic, and the first area in which the issuer is indicated in a manner in which it is difficult to acquire the text data indicating the issuer in the text data acquisition step, A second area in which at least one of a unit price, a total payment amount, a tax amount, and a payment method is shown, and a third area different from the first area and the second area,
An extraction step of extracting a plurality of character strings indicating words from the text data extracted from the third region;
A server connected to the Internet that provides a search service for searching for information corresponding to an arbitrary character string from a web page accessible via the Internet for a plurality of character strings indicating the words extracted in the extraction step An identification information acquisition step of acquiring search result information indicating search results for a plurality of character strings indicating words by transmitting to an external search service server that is not a dedicated server for the information specifying device and the information specifying program; ,
Of the search result information acquired by the identification information acquisition step, a specifying step of specifying a word corresponding to the search result information indicating the highest search result as the issuer,
It characterized by further executing the information identification program. An information specifying program executed by a computer constituting the information specifying device for specifying the issuer from image data corresponding to the print contents of the form issued in a format different for each issuer,
In the computer,
Executing a text data acquisition step of acquiring text data from the image data;
The form includes at least one of a special font and a graphic, and the first area in which the issuer is indicated in a manner in which it is difficult to acquire the text data indicating the issuer in the text data acquisition step, A second area in which at least one of a unit price, a total payment amount, a tax amount, and a payment method is shown, and a third area different from the first area and the second area,
A first extraction step of extracting a character string indicating a telephone number by searching for a character string including a numeric string having a specific sequence from the text data extracted from the third region, extracted from the third region A second extraction step of extracting a character string indicating a URL from the text data by searching the text data for a character string having a specific sequence indicating the URL from the text data thus extracted. An extraction step including a third extraction step of extracting a plurality of character strings indicating words from the text data;
A server providing a location information service capable of providing a data file including location information indicating a geographical location corresponding to a telephone number, the character string indicating the phone number extracted in the first extraction step; A first identification information acquisition step of acquiring a data file including position information corresponding to a character string indicating a telephone number by transmitting to an external position information service server that is not a dedicated server for the apparatus and the information specifying program; An external web corresponding to the URL indicated in the character string extracted in the second extraction step, connected to the Internet and providing a web service, and not a dedicated server for the information identification device and the information identification program By sending an HTTP request to the server, A second identification information acquisition step of acquiring a corresponding HTTP response, and a plurality of character strings indicating the words extracted in the third extraction step corresponding to an arbitrary character string from a web page accessible via the Internet A plurality of character strings indicating words by being transmitted to an external search service server which is a server connected to the Internet which provides a search service for searching for information and which is not a dedicated server for the information specifying device and the information specifying program An identification information acquisition step including a third identification information acquisition step of acquiring search result information capable of specifying the number of search hits indicating the number of web pages searched for,
Based on the data file acquired in the first identification information acquisition step, a first specification step for specifying the issuer corresponding to the character string indicating the telephone number extracted in the extraction step, the second identification information acquisition Of the search result information acquired by the second identification step and the third identification information acquisition step that specify the issuer based on the HTML element included in the HTTP response acquired by the step, the search hit number is the maximum. Corresponds to the search result information indicating the highest search result among the search result information acquired by the third specifying step for specifying a word corresponding to certain search result information as the issuer and the third identification information acquiring step. A specifying step including a fourth specifying step of specifying a word as the publisher;
Is executed further,
The first extraction step is executed after the text data acquisition step is executed, and when the first extraction step is executed successfully, the first identification information acquisition step is executed, and the first identification information acquisition step is executed. If the execution is successful, the first specific step is executed,
When the execution of the first extraction step fails, or when the execution of the first identification information acquisition step fails, the second extraction step is executed, and when the execution of the second extraction step is successful, the first Second identification information acquisition step is executed, and when the second identification information acquisition step is successfully executed, the second identification step is executed,
When the execution of the second extraction step fails or when the execution of the second identification information acquisition step fails, the third extraction step is executed, and the third identification information is executed after the execution of the third extraction step. whether to execute the acquisition step and the third specifying step, or, the third identification information acquiring step and information specific program that is characterized in that to execute the fourth specific step. 6. The information specifying program according to claim 1, wherein the image data acquisition step acquires the image data read by the image reading device excluding a camera. The image data, information specifying program according to claim 6 10, characterized in der Rukoto those read by the image reading apparatus except camera. A current location acquisition step for acquiring current location information indicating location information of the current location;
In the identifying step, when the data file includes a plurality of pieces of position information, the issuer is identified from position information indicating a position relationship closest to the current position information among the plurality of pieces of position information. The information specifying program according to claim 1 or 6 . The information specifying program according to claim 2 or 7, wherein the specifying step specifies the issuer from a title element among HTML elements included in header information of the HTTP response . The information specifying program according to claim 2 or 7 , wherein the specifying step specifies the issuer from information included in a response body of the HTTP response. A form template corresponding to the issuer specified in the specifying step, from storage means for storing a plurality of form templates in which the print layout information of the form and item information indicating items printed on the form are associated with the issuer A template acquisition step for acquiring
Item value for acquiring an item value corresponding to the item information by analyzing the text data acquired in the text data acquisition step with reference to the form template acquired in the template acquisition step An acquisition step;
The item value obtained in the item value acquiring step, one of claims 6 10, characterized in that a item value storage step of storing in association with said issuer identified in the identifying step Information identification program described in An information specifying method executed by a computer constituting the information specifying device for specifying the issuer from image data corresponding to the print contents of a form printed in a format different for each issuer,
The form includes at least one of a special font and a graphic, and includes a first area in which the publisher is indicated in a manner in which it is difficult to obtain text data indicating the publisher by character recognition processing, an item, a unit price, A second area in which at least one of a total payment amount, a tax amount, and a payment method is indicated; and a third area different from the first area and the second area;
An image data acquisition step of acquiring image data obtained by reading the document from the image reading apparatus,
A text data acquisition step of acquiring text data from the image data by character recognition processing;
An extraction step of extracting a character string indicating a telephone number by searching a character string including a numeric string having a specific sequence from the text data extracted from the third region ;
A server that provides a location information service that can provide a data file that includes location information that indicates a geographic location corresponding to a telephone number, using a character string that indicates the phone number extracted in the extraction step, for the image reading device An identification information acquisition step of acquiring a data file including location information corresponding to a character string indicating a telephone number by transmitting to an external location information service server that is not a dedicated server ;
A specification step of specifying the issuer corresponding to the character string indicating the telephone number extracted in the extraction step based on the data file acquired in the identification information acquisition step. Method.

Images