JP2017146745A - Information processing apparatus, control method, information processing system, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing apparatus that efficiently creates quality teacher data enabling construction of an accurate classification rule.SOLUTION: An MFP 101 receives image data for each of classes to be classified as positive example data, acquires image data to which document file information matching at least one piece of information included in document file information provided to the received image data as negative example data, and creates a classification rule used for classifying the image data by type by using the positive example data and negative example data.SELECTED DRAWING: Figure 4

Description

  The present invention relates to an information processing device, a control method, an information processing system, and a program.

  Image classification using machine learning has been proposed as one technique for improving the efficiency of a workflow for handling documents. Image classification using machine learning generally has two processes, learning and classification (operation), and a classification rule is constructed by learning by providing an image data group (teacher data, learning set). The input image is classified based on the classification rule.

  In order to classify data into a plurality of types, it is often only necessary to classify which type of data is given as a learning set. There is a need to classify as “not applicable”. For example, when a large number of documents are read by the scanner of the MFP, only specific types of documents are stored in a predetermined folder, and other types of documents are classified as “not applicable”. The case where it stores in the folder of the place can be considered.

  In machine learning, a classification rule is constructed based on data given as a learning set. Therefore, it is preferable that the data in the learning set has a feature amount close to that of data input during operation. In addition, in order to realize the “not applicable” classification, it is better to prepare the data of the “other” type (negative example data) in addition to the data of the type to be originally classified (positive example data). As example data, it is desirable to prepare a document that is likely to be input at the time of classification.

  However, when a user handles many types of documents, it is a heavy burden on the user to prepare a large amount of document data other than the type of data (positive example data) that the user wants to classify as negative example data. End up. In machine learning, if the same type of data is mixed in the positive example data and the negative example data, the classification rule cannot be constructed correctly. Therefore, it is necessary not only to prepare the learning set for the first time, but also to check whether the positive example data of the newly added type is mixed in the negative example data every time the type of positive example data is added. .

  Patent Document 1 extracts a feature word from a positive example document (positive example data), and contains as little of the positive example feature word as possible from a negative example candidate document extracted from a file server. A document classification system that selects a document including many feature words other than as a negative example is disclosed.

JP 2014-96086 A

  However, it is difficult to efficiently construct a highly accurate classifier only by removing document data that is unlikely to be the same type as the positive example data as in Patent Document 1. In general, the learning time increases according to the data amount of the learning set. For this reason, for example, in the case of a system that constructs classification rules on the spot from data prepared by the user, it is necessary to narrow down the learning set. However, if the data used for learning is reduced uniformly, such as selecting a certain number of files at random from the file server, the data that is likely to be input at the time of classification also decreases, and the classification accuracy decreases. End up.

  An object of the present invention is to provide an information processing apparatus that efficiently generates high-quality teacher data that enables construction of highly accurate classification rules.

  An information processing apparatus according to an embodiment of the present invention includes a receiving unit that receives image data for each type to be classified as positive example data, and at least one of information included in file information attached to the received image data matches Acquisition means for acquiring image data to which file information to be assigned is provided as negative example data, and generation means for generating a classification rule used for classifying image data by type using the positive example data and the negative example data And comprising.

  According to the information processing apparatus of the present invention, it is possible to generate high-quality teacher data that enables construction of a highly accurate classification rule.

It is a figure which shows the information processing system structure in 1st Embodiment. 2 is a diagram illustrating a configuration example of an MFP. FIG. It is a figure which shows an example of the hardware constitutions of a server. 10 is a flowchart for explaining processing in which an MFP learns a classification rule. It is a flowchart which shows the process which builds negative example data. It is a figure which shows an example of document file information. It is a figure which shows the screen where a user edits the narrowing-down conditions of a negative example data candidate group. It is a figure which shows the example of the narrowing-down result by document file information. It is a figure which shows an example of the machine learning using a learning set. It is a figure explaining the calculation method of a feature-value. It is a figure explaining the method of cutting out a patch image from image data. It is a figure explaining the process which produces | generates a learning set and learns a classification rule.

Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.
(First embodiment)
FIG. 1 is a diagram showing a configuration of an information processing system in the present embodiment.
The information processing system in the first embodiment includes an MFP 101 and a server 102 that are information processing apparatuses.

  An MFP 101 is connected to the LAN 103. The LAN 103 is connected to a server 102 that provides a service via the Internet 104. The MFP 101 and the server 102 are connected to each other via the LAN 103 and transmit / receive image data and various types of information. Note that the MFP 101 and the server 102 may be connected to each other and can transmit and receive image data and various types of information, may be directly connected by wire, or may be connected by wireless communication.

  The server 102 functions as a file server that stores the image data input from the MFP 101 and transmits image data that satisfies the conditions specified by the MFP 101 to the MFP 101. In this embodiment, the MFP 101 executes generation of a learning set used when learning a classification rule and construction of a classification rule using the learning set, but even if the server 102 executes similar processing. Good.

FIG. 2 is a diagram illustrating a configuration example of the MFP 101.
FIG. 2A is a diagram illustrating an example of a hardware configuration of the MFP 101. As shown in FIG. 2A, the MFP 101 includes a controller 20, an image reading unit 201, an image output unit 205, and an operation unit 207. The controller 20 includes an apparatus control unit 200, an image processing unit 202, a storage unit 203, a CPU 204, and a network I / F unit 206.

  The device control unit 200 exchanges data with the outside of the MFP 101 and the outside via the network I / F unit 206 and accepts an operation from the operation unit 207. The image reading unit 201 reads an image of a document and outputs image data to the controller 20. The image processing unit 202 converts print information including image data input from the image reading unit 201 or the outside into intermediate information (hereinafter referred to as “object”), and stores the intermediate information in an object buffer of the storage unit 203.

  Objects have text, graphic, and image attributes. Further, bitmap data is generated based on the object stored in the object buffer and stored in the buffer of the storage unit 203. At that time, pseudo halftone processing such as color conversion processing, density adjustment processing, toner total amount control processing, video count processing, printer gamma correction processing, and dithering is performed. The storage unit 203 includes a ROM (Read Only Memory), a RAM (Random Access Memory), an HDD (Hard Disk Drive), and the like.

  The ROM stores various control programs and image processing programs executed by the CPU 204. The RAM is used as a reference area or work area in which the CPU 204 stores data and various types of information. The RAM and HDD are used for the above-described object buffer and the like. The controller 20 accumulates image data on the RAM and HDD, sorts pages, accumulates the originals over a plurality of sorted pages, and prints out a plurality of copies.

  Note that the HDD constituting the storage unit 203 functions as a file server and stores image data input via the image reading unit 201 and the network I / F unit 206. The image output unit 205 forms and outputs a color image on a recording medium such as recording paper. A network I / F unit 206 connects the MFP 101 to the LAN 103 and transmits / receives various information to / from the Internet 104 and other devices. The operation unit 207 includes a touch panel and operation buttons, receives an operation from the user, and transmits information on the operation to the apparatus control unit 200.

  FIG. 2B is a diagram illustrating an example of the appearance of the MFP 101. The image reading unit 201 has a plurality of light receiving pixels. If the sensitivity of each light receiving pixel is different, even if the density of each pixel on the document is the same, it is recognized that each pixel has a different density. Therefore, the image reading unit 201 first exposes and scans a white plate (uniformly white plate), converts the amount of reflected light obtained by the exposure scan into an electrical signal, and outputs the electrical signal to the controller.

  The image processing unit 202 recognizes the difference in sensitivity of each light receiving pixel based on the electrical signal obtained from each light receiving pixel, and scans the image on the document using the difference. A known shading correction processing unit that corrects the value of the received electric signal is provided. Further, when the shading correction unit receives gain adjustment information from the CPU 204 in the controller, the shading correction unit performs gain adjustment according to the information.

  The gain adjustment is used to adjust how an electric signal value obtained by exposing and scanning a document is assigned to a luminance signal value of 0 to 255. By this gain adjustment, the value of the electrical signal obtained by exposing and scanning the document can be converted into a high luminance signal value or converted into a low luminance signal value. That is, the dynamic range of the read signal can be adjusted by gain adjustment.

Next, a configuration for scanning the image on the document will be described.
The image reading unit 201 converts information of an image into an electrical signal by inputting reflected light obtained by exposing and scanning an image on a document to a light receiving pixel. Further, the electrical signal is converted into a luminance signal composed of each color of red R, green G, and blue B, and the luminance signal is output as an image to the controller 20.

  The document is set on the tray 212 of the document feeder 211. When the user gives an instruction to start reading from the operation unit 207, a document reading instruction is given from the controller 20 to the image reading unit 201. Upon receiving this instruction, the image reading unit 201 feeds originals one by one from the tray 212 of the original feeder 211, and performs an original reading operation. The document reading method is not limited to the automatic feeding method by the document feeder 211, and is a method of scanning the document by placing the document on a glass surface (not shown) and moving the exposure unit. Also good.

  The image output unit 205 is an image forming device that forms an image received from the controller 20 on a sheet. In this embodiment, the image forming method is an electrophotographic method using a photosensitive drum or a photosensitive belt, but is not limited thereto. For example, the present invention can be applied even to an ink jet system that prints on paper by ejecting ink from a minute nozzle array. Further, the image output unit 205 is provided with a plurality of paper cassettes 213 to 215 that can select different paper sizes or different paper orientations. The printed paper is discharged to the paper discharge tray 216.

FIG. 3 is a diagram illustrating an example of a hardware configuration of the server.
The server 102 includes a CPU 301, a RAM 302, a ROM 303, a network I / F unit 304, an HDD 305, and a data bus 306. The CPU 301 reads out the control program stored in the ROM 303, loads it into the RAM 302, and executes various control processes. The RAM 302 is used as a temporary storage area such as a program executed by the CPU 301 or a work memory.

  A network I / F unit 304 connects the server 102 to the Internet 104 and transmits / receives various information to / from other devices. The HDD 305 stores image data, feature amount data, various programs, and the like. Image data received from the MFP 101 via the network I / F unit 304 is transmitted / received to / from the CPU 301, RAM 302, and ROM 303 via the data bus 306.

  When the CPU 301 executes an image processing program stored in the ROM 303 or the HDD 305, image processing for the image data is realized. Also, it is assumed that the HDD 305 can input data from an external device other than the MFP 101 via the network I / F unit 304 and already stores a large amount of files including document image data.

<Detailed Description of First Embodiment>
FIG. 4 is a flowchart for explaining processing for generating a learning set and learning classification rules.
The process shown in FIG. 4 is executed by the MFP 101 and the server 102. Processing executed in the MFP 101 is realized by the CPU 204 loading and executing a processing program stored in the storage unit 203. The processing executed in the server 102 is realized by the CPU 301 loading a processing program stored in the HDD 305 into the RAM 302 and executing it.

  In this embodiment, the user scans a document (original) using the MFP 101, and stores image data of the document in the server 102 for each type. Assumes a system that learns. In this way, when applying classification rules using machine learning to workflows that handle documents, storage destinations and distribution destinations of documents input from input devices such as MFPs with scanners are automatically determined, and file names are automatically assigned. Is possible. Further, by learning from a document prepared for each user, it is possible to construct an individually customized classification rule.

  Note that the timing for scanning a document and storing it in the server and learning the classification rules is not limited to the above workflow, and the scanning of the document and storage in the server and the learning of the classification rules are performed separately. Also good. In the second embodiment, a case will be described in which data already stored in the server 102 is acquired during learning of classification rules. The learning of the classification rule is not necessarily performed by the MFP 101 that has read the data. For example, the processing related to the classification of the present embodiment may be performed by the server that stores the image data.

  In step S <b> 401, the MFP 101 receives a storage setting for image data from the user via the operation unit 207. Note that the image data storage setting includes a folder path indicating a storage destination of image data read by the MFP 101, a file name at the time of storage, a file format, and the like.

  In step S <b> 402, when the MFP 101 receives a user instruction from the operation unit 207, the MFP 101 feeds documents one by one from the tray 212 of the document feeder 211, and reads the document by the image reading unit 201. In the present embodiment, the documents set on the tray 212 are the same type of document. Documents of the same type are documents that are classified into the same class in the classification rule.

  In step S403, the MFP 101 stores the image data group read by the image reading unit 201 in step S402 in the storage unit 203 as a positive example data candidate group of the learning set. When storing an image data group, document file information is assigned to each image data. The document file information is used for narrowing down a negative example data candidate group to be described later. Document file information includes general meta information given when creating an electronic file with an application, such as title, creator name, file format, creation tool, creation device, conversion tool, keyword, generation date, and update date Is used.

  A keyword is a group of character strings representing the characteristics of a document file. In this embodiment, character recognition is performed when a document is read, and the result is used. For example, characteristic characters such as the upper center of the first page that becomes the title, a character string in a characteristic position of the document such as a header, footer, or an item in the table, or a character string with a different font compared to other characters Use columns as keywords.

  In addition, if the keyword group for the document file information and the corresponding item are stored as a dictionary and character recognition is performed and character strings that match the keyword in the dictionary are included, the item corresponding to the keyword is displayed. You may give as a keyword of document file information. As a keyword group for document file information, commonly used document titles such as “decision document”, “application form”, “order form”, and company name list are used.

  The document file information is not limited to the meta information as described above, and may be document structure information such as character string position information and font size obtained in the process of character recognition. In addition, scan settings such as reading resolution, color, and assignment assigned when reading with the MFP 101 may be used. In the present embodiment, the image data group is stored together with the document file information. However, the present invention is not limited to this. For example, the feature amount data calculated from the read image data group may be stored. Good.

  In step S <b> 404, the MFP 101 transmits the image data group read by the image reading unit 201 through the network I / F unit 206 to the server 102. The server 102 receives an image data group from the MFP 101 via the LAN 103 and the Internet 104. The CPU 301 of the server 102 records the received image data in the HDD 305 based on the image data storage setting set in step S 401.

  In step S <b> 405, the MFP 101 receives an instruction from the user via the operation unit 207 as to whether or not to continue reading the document. If the reading of the document is continued, the process returns to step S401. If the reading of the document is not continued, the process proceeds to step S406. Note that the determination of whether or not to continue reading the document is not limited to the above method. For example, the number of document readings in step S401 may be counted, and document reading may be continued until the number of document readings set in advance by the user via the operation unit 207 is reached.

  In step S <b> 406, the MFP 101 stores the document image data stored in the storage unit 203 and the document image data acquired from the server 102 via the Internet 104 and the LAN 103 in the storage unit 203 as a negative example data candidate group. Store. In step S407, the MFP 101 extracts the negative example data candidate group file acquired in step S406. Details of the extraction process of the negative example data candidate group will be described later with reference to FIG.

  In step S408, the MFP 101 uses the positive example data candidate group stored in step S403 and the negative example data candidate group stored in step S407 as learning sets for learning classification rules using machine learning. In the present embodiment, learning of the classification rule will be described later with reference to FIGS.

<Detailed Description of Extraction Process of Negative Example Data Candidate Group (Step S407)>
Document data that is unlikely to be input during operation becomes useless data that does not contribute to classification accuracy during operation. For example, if 50 pieces of document data randomly selected from the server contain 5 unused data and 10 pieces of data to be used in another job, this data is effective for constructing a classification rule. Will be 35. As described above, simply acquiring data at random can reduce the number of documents that are likely to be actually input during classification.

  Even when data redundancy is not taken into account, it is a factor of reducing documents that are likely to be input at the time of classification. For example, when 50 pieces of document data used as negative example data are 10 documents for each of 5 types and 1 document for each of 50 types, the former may be input at the time of classification. Reduce the number of expensive documents. In the present embodiment, it is possible to acquire negative example data that enables highly accurate classification by extracting the negative example data candidate group.

FIG. 5 is a flowchart showing a process of constructing negative example data from the negative example data candidate group.
Specifically, the process illustrated in FIG. 5 is a process of extracting negative example data that is a part of the learning set used for constructing the classification rule from the negative example data candidate group acquired in step S406. The processing shown in FIG. 5 is realized by the CPU 204 of the MFP 101 loading and executing a processing program stored in the storage unit 203.

  In step S501, the MFP 101 obtains a narrowing condition for the negative example data candidate group based on the document file information given in step S403 and the user instruction from the positive example data candidate group recorded in the storage unit 203 in step S403. To do. In step S502, the MFP 101 narrows down (extracts) the negative example data candidate group acquired in step S406 based on the narrow-down condition acquired in step S501. Details of step S501 and step S502 will be described later with reference to FIGS.

  In step S503, the MFP 101 reduces redundant data from the negative example data candidate group extracted in step S502. In order to specify redundant data, for example, clustering using a keyword that is one of document file information as a feature amount is used. This is because documents having the same keyword are likely to be the same type of document, and are therefore determined to be the same type of document.

  The identification of redundant data is not limited to the above method. For example, as a feature amount other than a keyword, clustering with feature amounts such as a title character string, a font size of the title character string, a position of the title character string based on document structure information may be used, and other methods may be used. It may be used. If there are a large number of documents determined to be of the same type, redundant data can be reduced by extracting some of them and deleting the remaining documents. At this time, for example, the upper limit of the data capacity or the number of documents may be determined in advance, and when the data capacity or the number exceeds the upper limit, the document may be deleted so that they are less than the upper limit value.

  In step S504, the MFP 101 deletes a document that may be of a type included in the positive example data candidate group from the negative example data candidate group in which redundant data is reduced in step S503. Whether or not the type is included in the positive example data candidate group is determined based on the probability (matching rate) that the document of the positive example data matches the keyword. Note that the determination of whether or not the type is included in the positive example data candidate group is not limited to the above method.

  The determination here does not require accuracy when determining whether to classify as “others” using the classification rule. It is only necessary to delete a document that is suspected of being a positive type using the matching rate of document structure information and the matching rate of image feature amounts. Further, if the classification rule has already been constructed and the type of positive example data is to be added, it may be determined whether or not the type of positive example data is applied by applying the classification rule that has been constructed. .

  In this embodiment, the negative example data is created by the MFP 101 executing the process of narrowing down the image data group acquired from the server 102. However, the present invention is not limited to this. For example, the server 102 may narrow down data according to the narrowing conditions described with reference to FIG.

<Detailed Description of Narrowing Condition Acquisition and Narrowing Processing (Steps S501 and S502)>
The process of obtaining and narrowing down the narrowing conditions is a process executed by the CPU 204 of the MFP 101. Acquisition of the narrowing-down conditions will be described with reference to FIGS.
FIG. 6 is a diagram illustrating an example of document file information. The document file information is shown for three types of documents given as the positive example data candidate group. FIG. 7 is a diagram showing an example of a screen for the user to confirm and edit the narrowing-down conditions based on the document file information of the positive example data candidate group.

  The screen in FIG. 7 has radio buttons 701 and 702. With the radio buttons 701 and 702, it is possible to switch whether or not to set a narrowing-down condition. A button 703 is a button for instructing (requesting) acquisition of a negative example data candidate group, and switches processing to be acquired according to the states of the radio buttons 701 and 702.

  Specifically, when the radio button 701 is selected, the documents in the storage unit 203 and the HDD 305 are narrowed down and acquired based on the contents set in the conditional expression field 705 and the conditional field 706. When the radio button 702 is selected, the documents in the storage unit 203 and the HDD 305 are acquired as they are without being narrowed down. A button 704 is a button for instructing automatic acquisition of a narrowing condition.

  When an automatic acquisition of the narrowing condition is instructed by the button 704, the MFP 101 acquires the narrowing condition from the document file information of the positive example data candidate group and displays it in the conditional expression field 705 and the condition field 706. Specifically, in the conditional expression field 705 and the condition field 706, at least one item common to the document file information of the positive example data candidate group based on the document file information of the positive example data candidate group shown in FIG. Conditional expressions that can be acquired for documents including In the condition field 706, each item of document file information of the positive example data candidate group is displayed as a condition.

  In the conditional expression field 705, a conditional expression in which the conditions in the condition field 706 are combined with “+” representing the union is input. That is, a document including at least one common item with each item of document file information of the positive example data candidate group is extracted. The automatic acquisition of conditional expressions is not limited to the method of extracting the union of each item of document file information as described above. For example, the co-occurrence of document file information may be calculated between the positive example data candidate groups, and the condition may be displayed so as to narrow down documents including combinations of document file information items with high co-occurrence. That is, the documents may be narrowed down using a combination of document file information items that are frequently assigned in the positive example data candidate group.

  The conditional expression field 705 and the conditional field 706 not only display conditions but also accept editing by the user. The user can edit the condition automatically acquired using the button 704, and can also arbitrarily set a condition for narrowing down a user-desired document. It is also possible to clear the conditions displayed in the conditional expression field 705 and the condition field 706 by the button 709.

  In the example shown in FIG. 7, conditions up to condition number 7 are listed in the condition field 706, but these numbers are variable, and the upper limit is not limited to 10 currently displayed. When the button 710 instructs to add a condition, the number of conditions (number of lines) can be increased. In the conditional expression field 705, it is also possible to input a conditional expression like a multinomial operation using the condition number displayed in the condition field 706 and parentheses or operators. For example, a union set is represented by a “+” operator, and a product set is represented by a “*” operator.

  Further, the narrow-down conditions expressed in the conditional expression field 705 and the conditional field 706 can be saved or read in a file. A button 707 is a button for saving the narrow-down conditions to a file. When the button 707 is pressed, the narrow-down conditions displayed in the conditional expression field 705 and the condition field 706 are saved in a text file format.

  A button 708 is a button for reading a narrowing condition from a file. When the button 708 is pressed, the narrowing condition read from the file is displayed in the conditional expression field 705 and the condition field 706. Note that the format of the file for storing the filtering conditions is not limited to the text file format, and is not particularly limited as long as the conditions can be expressed. For example, it may be saved in XML format.

FIG. 8 is a diagram illustrating an example of a result of narrowing down the document files in the storage unit 203 and the HDD 305 according to the above-described narrowing conditions.
Data 801 is one of the data adopted as negative example data when narrowing down according to the conditions shown in FIG.

  The items 802 (name of creator), item 803 (format), and item 804 (creation device) of the document file information are adopted because they match the conditions 711, 712, and 713, respectively. On the other hand, one of data that is not adopted as negative example data is data 805. Since the item of the document file information does not match any of the conditions shown in the condition field 706, it is not adopted as negative example data and is deleted from the negative example data candidate group.

<Detailed Description of Construction of Classification Rule Using Machine Learning (Step S408)>
Next, a machine learning technique used for constructing a classification rule in this embodiment will be described. In the present embodiment, a known technique called Real AdaBoost is used as a machine learning technique. Real AdaBoost is a technique capable of selecting a feature amount suitable for classification of a given learning set from a large amount of feature amounts and combining the feature amounts to form a classifier (classification rule).

  If a large amount of feature amount is used at the time of image classification, there is a possibility that the performance is lowered due to the calculation load of the feature amount. As in Real AdaBoost, it is a great advantage that a classifier can be configured by selecting feature quantities suitable for classification and using only some of the feature quantities. However, Real AdaBoost is a two-class classifier and classifies data with two types of labels. That is, as it is, it cannot be used for classification of three or more types of image data.

  Therefore, in the present embodiment, a known method called OVA (One-Versus-All) that extends the two-class classifier to a multi-class classifier is used. OVA creates classifiers for classifying one class (target class) and other classes by the number of classes, and uses the output of each classifier as the reliability of the target class. That is, in one classifier, data belonging to a class classified by the classifier is used as positive example data, and data belonging to other classes is used as negative example data to learn a classification rule.

  Each classifier performs learning so as to increase the output reliability when data of one class targeted by the classifier is input. At the time of classification, data to be classified is input to all classifiers, and the class having the highest reliability is set as the classification destination. Further, when the reliability output from all the classifiers is low, or when the reliability output from a plurality of classifiers is high, the determination such as “not applicable” or “unknown” is performed.

FIG. 9 is a diagram illustrating an example of machine learning using a learning set.
In this example, three classes (types) of positive example data candidate groups (Document A, Document B, Document C) and “other” documents of negative example data candidate groups (Document A, Document B) are used as learning sets. , A feature amount corresponding to each of the documents (not the document C) is prepared. In order to classify the three types of documents A, B, and C, OVA prepares three types of classifiers. The three types of classifiers are document A classifier for classifying documents into document A and other classes, document B classifier for classifying documents into document B and other classes, document C and This is a document C classifier for classifying documents into classes other than.

  Here, a method for constructing the document A classifier will be described. First, the CPU 204 of the MFP 101 acquires positive example data and negative example data necessary for learning the classification rule from the learning set. In the document A classifier, positive example data is data of document A, and negative example data is data of other classes. Therefore, the CPU 204 acquires image data to which the label of the document A is assigned from the positive example data candidate group, and sets it as positive example data.

  In addition, the CPU 204 acquires image data with a label other than the document A (document B, document C) as negative example data from the positive example data candidate group. Further, the CPU 204 acquires image data as negative example data from the negative example data candidate group. At this time, if the negative example data candidate group contains data of document A, which is positive example data, the classification rule cannot be learned correctly. For this reason, it is assumed that a document A that has a high possibility of being document A has been removed by the processing in step S504.

  The CPU 204 constructs a document A classifier using Real AdaBoost based on the acquired feature values of the positive example data and the negative example data. In the document A classifier, a large output value (reliability) is output when the feature amount of the document A is input, and a small output value (reliability) when a feature amount of a document of another class is input. Degree) is output. The same applies to the document B classifier and the document C classifier.

  Note that the machine learning technique that can be used in the present embodiment is not limited to the above technique. A known method such as Support Vector Machine or Random Forest may be used. In addition, if the feature selection framework is not included in the machine learning method and you want to improve the classification speed during classification, you can use known features such as feature selection using principal component analysis or discriminant analysis. Quantity selection may be performed. When the device learning method is a two-class classifier, a known method such as All-Versus-All (AVA) or Error- Correcting Output-Coding (ECOC) other than OVA may be used.

<Details of features used to construct classification rules>
A feature amount used for constructing a classification rule in the present embodiment will be described with reference to FIGS. 10 and 11.

FIG. 10 is a diagram illustrating a feature amount calculation method.
In the present embodiment, the feature amount is a nine-dimensional feature amount calculated based on the gradient information for the patch image 1002 cut out from the input image 1001. The CPU 204 of the MFP 101 pays attention to each pixel in the patch image 1002 and calculates the gradient strength and gradient direction from the gradation value of the pixel adjacent to the pixel of interest.

  Then, the CPU 204 performs edge determination based on the gradient strength, thereby determining that a pixel having a gradient strength equal to or greater than a certain value is an edge pixel and a pixel having a gradient strength smaller than the certain value is a non-edge pixel. As a result of the edge determination, the pixel 1003 is an example of a pixel determined as a non-edge pixel, and the pixel 1004 is an example of a pixel determined as an edge pixel. An arrow in the pixel 1004 that is an edge pixel represents a gradient direction.

  Since the gradient direction expresses the direction of the line of characters and ruled lines, the angle rotated by 180 degrees is regarded as the same direction and is normalized to 0 to 180 degrees. The CPU 204 quantizes the gradient direction from the edge pixel group into 8 directions every 22.5 degrees, calculates the gradient intensity integrated value / number of patch pixels for each direction, and creates an 8-bin histogram. Further, the CPU 204 calculates the number of non-edge pixels / the number of patch pixels from the non-edge pixel group, and calculates a nine-dimensional feature amount from one patch image together with the histogram created from the edge pixel group.

  By using edge pixels and non-edge pixels, it is possible to express not only ruled line and character information but also a margin part which is a major feature of a document image. Although the description so far has been a description of feature amounts in one patch image 1002, in practice, a large number of feature amounts are used by cutting out and using a plurality of patch images from one input image.

FIG. 11 is a diagram illustrating a method for cutting out a patch image from read image data.
The CPU 204 cuts margins from the input image 1101 and deletes the image edge 1102 where noise is likely to appear. The CPU 204 creates a multi-scale (multiple resolution) image by reducing the image 1103 after the margin cut. The reason why the multi-scale image is prepared is that the structure of the edge changes for each resolution, so that the image reading unit 201 can cope with even a slightly different reading resolution or document resolution.

  An image 1104 is an image obtained by reducing the image 1103 after the margin cut to 1/4. A patch image is cut out from the image 1103 after the margin cut and the reduced image 1104 while changing the patch size and cutout position. Specifically, first, from 16 1/16 size patch images obtained by equally dividing the reduced image 1104 into 16 pieces and 64 1/64 size patch images obtained by equally dividing 64 pieces. A total of 80 patch images are created.

  Further, 80 patch images are similarly divided from the image 1103 after the margin cut, and a total of 160 patch images are obtained from one input image 1101. Since a 9-dimensional feature value is calculated from each patch image, it is possible to calculate a 9 × 160 = 1440-dimensional feature value from one input image 1101.

  Note that the parameters relating to the image resolution, patch size, and patch cutout position are not limited to the above numbers. In addition, a color histogram, color dispersion, or the like may be used as the feature amount in order to use the color information of the document as the feature amount to be calculated. Further, the feature amount used for the construction of the classification rule is not limited to the feature amount related to the image data as described above. For example, document file information such as meta information and document structure information used for narrowing down the negative example data candidate group may be used.

  In the present embodiment, a case has been described in which a document is read as image data by the MFP 101 and the image data is classified. However, the present invention is not limited to this. For example, the narrowing of negative example data using the positive example data of the present invention can also be applied to text format data.

  As described above, according to the present embodiment, it is possible to efficiently generate high-quality negative example data that enables construction of a highly accurate classification rule.

(Second Embodiment)
In the first embodiment, it is assumed that a document set on the tray 212 and read at a time by the image reading unit 201 is used as positive example data. On the other hand, in the present embodiment, it is assumed that a document already stored on the server 102 is used as positive example data in addition to a document set on the tray 212 and read at once by the image reading unit 201. Only differences from the first embodiment will be described below.

<Detailed Description of Second Embodiment>
FIG. 12 is a flowchart for explaining processing for generating a learning set and learning classification rules.
The process shown in FIG. 12 is executed by the MFP 101 and the server 102. Processing executed in the MFP 101 is realized by the CPU 204 loading and executing a processing program stored in the storage unit 203. The processing executed in the server 102 is realized by the CPU 301 loading a processing program stored in the HDD 305 into the RAM 302 and executing it.

  In this embodiment, the user scans a document (original) using the MFP 101, and stores the image data of the document in the server 102 for each type. A system for learning rules is assumed. Furthermore, in the present embodiment, it is assumed that a document used for learning classification rules is acquired from the server 102.

  In step S <b> 1201, the MFP 101 accepts whether a document to be used as positive example data is read from the document feeder 211 or selected from the server 102. When reading from the document feeder 211, the process proceeds to step S1202, and when selecting from the server 102, the process proceeds to step S1205. Steps S1202 and S1203 are the same as steps S401 and S402 in FIG. Step S1204 is the same as step S404 in FIG.

  In step S <b> 1205, the MFP 101 receives an instruction from the user as to which document in the HDD 305 of the server 102 is to be used via the operation unit 207. The CPU 301 of the server 102 transmits the image data group in the HDD 305 to the MFP 101 via the Internet 104 and the LAN 103 based on a user instruction. In step S1206, the MFP 101 stores the image data group read by the image reading unit 201 in step S1203 or the image data group received from the server 102 in step S1205 in the storage unit 203 as a positive example data candidate of a learning set. Store as a group.

  When storing, document file information used for narrowing down the negative example data candidate group is assigned to each image data. The document file information added to the image data group read by the image reading unit 201 is the same as the document file information added in step S403 in FIG. On the other hand, when document file information is already assigned to the image data group received from the server 102, the document file information is used. If the document file information is insufficient, the document file information similar to the document file information provided in step S403 in FIG. 4 is assigned to the missing item.

  In step S <b> 1207, the MFP 101 receives an instruction from the user via the operation unit 207 as to whether or not to continue registration of positive example data. If registration of positive example data is to be continued, the process returns to step S1201. If registration of the positive example data is not continued, the process proceeds to step S1208. Note that the determination as to whether or not to continue to register positive example data is not limited to the above method. For example, the registration number of the positive example data in step S1206 may be counted, and the registration of the positive example data may be continued until the registration number of the positive example data set in advance by the user via the operation unit 207 is reached. Steps S1208 to S1210 are the same as steps S406 to S408 in FIG.

  As described above, according to this embodiment, when creating negative example data, in addition to document file information obtained from image data (positive example data) input from the MFP, image data obtained from the server is added. Document file information that has already been assigned can be used. This makes it possible to collect data that is highly likely to be input at the time of classification as negative example data while suppressing the volume of data from a large amount of documents, and can efficiently generate highly accurate classification rules. It becomes.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

101 MFP
102 servers

Claims (10)

Receiving means for receiving image data for each type to be classified as positive example data;
Obtaining means for obtaining, as negative example data, image data provided with file information that matches at least one of the information included in the file information provided in the received image data;
An information processing apparatus comprising: generating means for generating a classification rule used for classifying image data for each type using the positive example data and the negative example data. The information processing apparatus according to claim 1, wherein the file information includes at least meta information of image data. The information processing according to claim 2, wherein the meta information includes at least one of a title of image data, a creator name, a file format, a creation device, a generation date and time, or a keyword included in the image data. apparatus. The acquisition unit acquires, as negative example data, image data to which file information including information having high co-occurrence in the positive example data is included among the information included in the file information assigned to the positive example data. The information processing apparatus according to any one of claims 1 to 3. The acquisition means includes image data to which file information having a high coincidence rate with information included in file information added to the positive example data among image data acquired as the negative example data is added to the negative example data. The information processing apparatus according to claim 1, wherein the information processing apparatus is not used as an information processing apparatus. The acquisition means classifies the image data acquired as the negative example data for each type based on the keyword of the image data, and when the number of image data classified in each type is greater than the upper limit value, The information processing apparatus according to any one of claims 1 to 5, wherein the image data is deleted so that the number of the classified image data is equal to or less than the upper limit value. The acquisition means includes
A screen that displays information included in the file information attached to the received image data for each item;
The information processing according to any one of claims 1 to 6, wherein image data satisfying a value specified for each item specified on the screen satisfying a specified condition is acquired as the negative example data. apparatus. A system comprising an information processing device and a server,
The information processing apparatus includes:
Receiving means for receiving image data for each type to be classified as positive example data;
Acquisition means for acquiring, as negative example data, image data to which file information that matches at least one of the information included in the file information provided to the received image data is assigned;
Using the positive example data and the negative example data, and generating means for generating a classification rule used for classifying the image data by type,
The server
An information processing system comprising: transmission means for transmitting image data to the information processing apparatus in response to a request from the information processing apparatus. A reception process for receiving image data for each type to be classified as positive example data;
An acquisition step of acquiring, as negative example data, image data provided with file information that matches at least one of the information included in the file information provided in the received image data;
And a generating step of generating a classification rule used for classifying image data for each type using the positive example data and the negative example data.   The program for functioning a computer as each means with which the information processing apparatus of any one of Claims 1 thru | or 7 is provided.

Images