JP2013246677A - Learning device for dictionary for pattern recognition, pattern recognition device, coding device, division device and learning method for dictionary for pattern recognition - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a learning device for a dictionary for pattern recognition, a pattern recognition device, a coding device, a division device and a learning method for a dictionary for pattern recognition, for easily achieving the learning of a pattern recognition dictionary without imposing any manual labor.SOLUTION: The learning device for a dictionary for pattern recognition includes: a generation part; a feature extraction part; a setting part; and an update part. The generation part generates a pattern area candidate in a recognition object image in which a pattern input by an input person seems to be described. The feature extraction part calculates a feature value in a pattern area candidate generated by the generation part. The setting part sets probability that the pattern input by the input person is described in the pattern area candidate generated by the generation part on the basis of the visual line position of the input person who has input the pattern. The update part updates the dictionary for pattern recognition on the basis of the probability set by the setting part and the feature value calculated by the feature extraction part.

Description

  Embodiments described herein relate generally to a pattern recognition dictionary learning device, a pattern recognition device, a coding device, a sorting device, and a pattern recognition dictionary learning method.

  For example, the pattern recognition apparatus can improve the recognition accuracy by updating the dictionary using learning data in which a pattern image and information indicating a correct pattern are associated with each other. As a learning method of a dictionary for recognizing a character as an example of a pattern, learning data is prepared by associating a character image with information indicating a character (character that is a correct answer) indicated by the character image. There is a method for updating dictionary data based on data. As a method of associating a character image with a correct character, there is a method in which a person designates character information for each character image. However, there is a problem that a method in which a person directly associates each character image with a correct character takes a lot of time and effort.

Japanese Patent Laid-Open No. 7-191796 Japanese Patent No. 3051628

  In order to solve the above problems, a pattern recognition dictionary learning device, a pattern recognition device, a coding device, a sorting device, and a pattern recognition dictionary learning method capable of efficiently learning a pattern recognition dictionary are provided. With the goal.

  According to the embodiment, the pattern recognition dictionary learning apparatus includes a generation unit, a feature extraction unit, a setting unit, and an update unit. The generation unit generates a pattern region candidate in an image to be recognized that seems to describe a pattern input by the input user. The feature extraction unit calculates a feature amount in the pattern area candidate generated by the generation unit. The setting unit sets a probability that the pattern input by the input user is described in the pattern area candidate generated by the generation unit based on the line-of-sight position of the input user who inputs the pattern. The update unit updates the pattern recognition dictionary based on the probability set by the setting unit and the feature amount calculated by the feature extraction unit.

FIG. 1 is a block diagram schematically illustrating a configuration example of a paper sheet processing apparatus as a sorting apparatus according to the present embodiment. FIG. 2 is a flowchart for explaining an operation example of the VCD as the video coding apparatus according to the present embodiment. FIG. 3 is a diagram showing an example of a recognition target image supplied to the VCD as the video coding apparatus according to the present embodiment. FIG. 4 is a diagram illustrating the line-of-sight position and the input timing on the display screen of the image to be recognized according to the present embodiment. FIG. 5 is a diagram illustrating a configuration example of a learning data file created by the VCD according to the present embodiment. FIG. 6 is a block diagram illustrating a configuration example of the learning unit according to the present embodiment. FIG. 7 is a diagram illustrating an example of a hypothesis in a recognition target image according to the present embodiment. FIG. 8 is a diagram showing prior probabilities for each hypothesis according to the present embodiment. FIG. 9 is a diagram showing posterior probabilities for the respective hypotheses according to the present embodiment. FIG. 10 is a flowchart for explaining an operation example of learning processing by the learning unit according to the present embodiment.

Hereinafter, the present embodiment will be described with reference to the drawings.
The learning device according to the present embodiment updates the pattern recognition dictionary. Here, the pattern to be recognized is information such as characters, symbols, or codes, and the pattern recognition dictionary is a storage unit that stores dictionary data for recognizing those patterns. In this embodiment, a sorting apparatus that performs character recognition processing using a character recognition dictionary as a pattern recognition dictionary will be described.

  The sorting device according to the present embodiment is a system including functions such as a character recognition dictionary learning device, a pattern recognition device, and a coding device. The sorting apparatus according to the present embodiment sorts a sorting object based on sorting information written in letters on a sorting object such as paper sheets or articles. The sorting apparatus has a character recognition function for recognizing characters as sorting information written on sorting objects such as paper sheets or articles. The sorting device sorts sorting objects such as paper sheets or articles based on sorting information obtained as a recognition result by character recognition processing or sorting information input by video coding processing described later.

  The sorting apparatus according to the present embodiment is a video coding system (hereinafter, abbreviated as VCS) in which a person inputs character information as sorting information that could not be recognized by a character recognition process using a character recognition dictionary in the sorter body. Have The VCS has a plurality of coding devices. Each coding device displays an image including classification information that could not be recognized by the character recognition processing of the classification machine body on the display screen, and a person who viewed the display screen displays the classification information as character information included in the character image. input. In addition, the coding apparatus according to the present embodiment not only returns the classification information as character information input by the person to the sorting machine body, but also when the image including the classification information, the character information input by the person, and the character information are input. It has a function of supplying data including position information viewed by a person to a character recognition dictionary learning device such as a learning unit provided in the sorter body as data for updating the character recognition dictionary (learning data). .

  The sorting apparatus according to the present embodiment includes a learning unit that updates (learns) the character recognition dictionary based on learning data supplied from the VCS. The learning unit refers to the character information input by the person included in the learning data and the position information viewed by the person when the character information is input, and the dictionary data of each character stored in the character recognition dictionary Update. The learning unit only needs to update a dictionary used for character recognition processing in the sorting machine body, and may be a learning device provided separately from the sorting device.

  The sorting device according to the present embodiment sorts articles such as a paper sheet processing device (for example, a mail sorting device) that sorts paper sheets such as postal items, and packages and cargo (for example, parcels, parcel delivery). An article sorting apparatus that sorts articles according to sorting information described in tags attached to articles or the like is assumed. In the following description, as an example of a sorting device, a description will be given assuming a paper sheet processing device that sorts paper sheets based on address information as sorting information written in characters.

FIG. 1 is a block diagram schematically showing a configuration example of a paper sheet processing apparatus as a sorting apparatus having functions such as a pattern recognition dictionary learning apparatus, a pattern recognition apparatus, and a coding apparatus according to the present embodiment.
For example, the paper sheet processing apparatus 1 performs character recognition on classification information expressed by characters such as an address written on a paper sheet such as a postal matter or a form, and the paper sheet is based on the result of the character recognition. Is processed separately. In the configuration example shown in FIG. 1, the sheet processing apparatus 1 includes a sorter body 3 and a video coding system (hereinafter abbreviated as VCS) 4. The sorter main body 3 and the VCS 4 are connected so that they can communicate with each other.

First, the sorting machine body 3 will be described.
The sorter main body 3 of the paper sheet processing apparatus shown in FIG. 1 sorts paper sheets based on address information as sorting information. The sorter main body 3 includes a character recognition unit and an address recognition unit. The character recognition unit of the sorter main body 3 refers to the dictionary for character recognition and recognizes each character that seems to be address information in the paper sheet image read by the scanner. The address determination unit of the sorting machine main body 3 determines address information described on the paper sheet with reference to the result of character recognition by the character recognition unit and the address information stored in the address database. The sorter body 3 of the paper sheet processing apparatus sends an image of the paper sheet whose address information could not be recognized (specified) to the VCS 4. The VCS 4 receives the address information in the image of the paper sheet by the operator and returns the input result to the sorter main body 4. The sorter main body 4 also has a function of sorting paper sheets whose address information could not be recognized based on the address information input by the VCS 4.

  The sorter body 3 includes an operation panel 10, a supply unit 11, a main transport path 12, a barcode reader (hereinafter referred to as BCR) 13, a scanner 14, a barcode writer (hereinafter referred to as BCW) 15, a sorting unit 16, a control unit 17, A character recognition unit 18, a dictionary 18a, an address identification unit 19, an address database (hereinafter referred to as an address DB) 19a, a learning unit 20, and the like are provided.

  The control unit 17 controls the operation of each unit of the paper sheet processing apparatus 1 in an integrated manner. The control unit 17 includes a CPU, a buffer memory, a program memory, a nonvolatile memory, and the like. The CPU performs various arithmetic processes. The buffer memory temporarily stores the results of calculations performed by the CPU. The program memory and the nonvolatile memory store various programs executed by the CPU, control data, and the like. The control unit 17 can perform various processes by executing a program stored in the program memory by the CPU.

The operation panel 10 allows an operator (operator) to specify a processing mode, to specify processing start, and to display an operation state of the sheet processing apparatus 1.
The supply unit 11 stocks paper sheets to be taken into the paper sheet processing apparatus 1. The supply unit 11 collectively receives the stacked paper sheets. The supply unit 11 supplies paper sheets one by one to the main transport path 12. For example, the supply unit 11 includes a separation roller. When a paper sheet is input to the supply unit 11, the supply roller 11 contacts the lower end of the input paper sheet in the stacking direction. The separation roller rotates to supply paper sheets set in the supply unit 11 one by one from the lower end in the stacking direction to the main transport path 12 at a constant pitch.

  The main transport path 12 is a transport unit that transports paper sheets to each unit in the paper sheet processing apparatus 1. The main conveyance path 12 includes a conveyance belt and a drive pulley. The main conveyance path 12 drives a drive pulley by a drive motor. The conveyor belt is operated by a driving pulley. On the main transport path 12, a bar code reader 13, a scanner 14, a bar code writer 15, and a sorting unit 16 are provided.

  The BCR 13 reads a barcode such as an ID barcode or a destination barcode printed on a sheet conveyed on the main conveyance path 12. The BCR 13 includes a reading unit that reads an image of a barcode and a recognition unit that recognizes the barcode in the read image. When the reading unit reads the barcode, the reading unit supplies an image of the barcode to the recognition unit. The recognition unit processes the supplied barcode image and recognizes data included in the barcode. The recognized data is supplied to the control unit 17.

  The scanner 14 acquires an image from paper sheets conveyed by the main conveyance path 12. The scanner 14 includes, for example, illumination and an optical sensor. The illumination irradiates light on the paper sheet 1 conveyed by the main conveyance path 12. The optical sensor includes a light receiving element such as a charge coupled device (CCD) and an optical system (lens). The optical sensor receives the reflected light reflected by the paper sheets by the optical system, forms an image on the CCD, and acquires an electrical signal (image). The scanner 14 obtains an image of the entire paper sheet by continuously obtaining images from the paper sheet 1 conveyed by the main conveyance path 12. The scanner 14 supplies the acquired image to the character recognition unit 18. The scanner 14 may be constituted by a video camera or the like.

  The character recognizing unit 18 functions as a pattern recognizing unit that recognizes a character as a pattern included in an image of a paper sheet read by the scanner 14. The character recognizing unit 18 extracts a character image region (character candidate region) that seems to be each character constituting the address information from the paper sheet image read by the scanner 14. The character recognition unit 18 recognizes each character in the extracted character candidate region image with reference to dictionary data stored in the character recognition dictionary 18a. For example, the character recognition unit 18 calculates the similarity between the feature parameters as characters (patterns) in the character candidate area and the feature parameters (character dictionary patterns) of each character stored as dictionary data in the dictionary, The character recognition result is determined based on the calculated similarity.

  The dictionary 18a stores dictionary data such as feature parameters obtained by converting the characteristics of each character into data. The dictionary 18 a is used for character recognition processing (OCR processing) by the character recognition unit 18. For example, the dictionary 18a stores the characteristic parameters of each character as dictionary data indicating the characteristics of each character. Here, the character feature parameter is a numerical value of the feature in the character image. For example, the characteristic parameter of the character as dictionary data in the dictionary 18a may be one obtained by extracting the luminance gradient information after blurring the character image as a 128-dimensional vector. The character feature parameters stored as dictionary data in the dictionary 18a may be a Gaussian distribution having an average m and a covariance matrix Σ.

  The character recognizing unit 18 extracts, as a character candidate region, a region that seems to be a region where one character is described from a paper sheet image to be recognized. The extraction method of the character candidate region is not limited to a specific method. When a character candidate area where one character is likely to be described is extracted, the character recognition unit 18 extracts feature parameters of a pattern that seems to be a character in the character candidate area. When the feature parameters are extracted, the character recognition unit 18 compares the extracted feature parameters with the feature parameters of each character stored in the dictionary 18a, and the characters described in the character candidate area are the characters compared. Probability (or similarity) is calculated.

  For example, the character recognizing unit 18 compares the extracted feature parameter with the feature parameter “a” stored in the dictionary 18 a, and the probability that the character described in the character candidate area is “a” (or Similarity) is calculated. Further, the character recognition unit 18 calculates the similarity (probability) in order for each character such as “I”, “U”,.

  The character recognition unit 18 specifies a character included in the character candidate area from the calculated similarity (probability) for each character. For example, the character recognition unit 18 sets a character whose calculated probability exceeds a predetermined threshold as a recognition result. When there are a plurality of characters exceeding the predetermined threshold, the character recognition unit 18 outputs these characters to the address determination unit 19 as a character recognition result in descending order of probability. Further, when there are a plurality of characters exceeding a predetermined threshold, the character recognition unit 18 outputs a predetermined number of characters among the characters in descending order of probability to the address determination unit 19 as recognition results. good. In addition, when a character is uniquely specified, if there are a plurality of characters exceeding a predetermined threshold, the character recognizing unit 18 selects one character having the highest probability among the characters as a character recognition result as an address determining unit. 19 may be output.

  Further, in the paper sheet processing apparatus, the character recognition unit 18 is a process that presupposes address determination, and therefore may output a character recognition result without comparing with a threshold value. For example, the character recognition unit 18 may use a predetermined number of characters as character recognition results in descending order of probability calculated for all characters. In this case, the character recognition unit 18 outputs a predetermined number of characters to the address determination unit 19 as a character recognition result in descending order of probability. Further, the character recognition unit 18 may uniquely specify a character in the character candidate area. When the character is uniquely specified, the character recognition unit 18 may output the character with the highest probability as the character recognition result for the character candidate area. The character recognizing unit 18 similarly performs recognition processing for all the character images described in the extracted image, and recognizes all the characters described in the extracted image. The character recognition processing in the character recognition unit 18 may be any method as long as it is character recognition using dictionary data registered in the dictionary 18a, and is not limited to the above processing example.

  The address determination unit 19 recognizes address information including a combination of characters based on the recognition result of each character by the character recognition unit 18. The address determination unit 19 identifies (recognizes) the address information described on the paper sheet by comparing the character recognition result obtained by the character recognition unit 18 with the address information stored in the address DB 19a. For example, when a plurality of character candidates are obtained for each character candidate area as a result of character recognition by the character recognition unit 18, the address determination unit 19 obtains the combination of the obtained character candidates and the address information stored in the dictionary 18a. Compare and identify address information. When the address information can be specified (recognized), the address determination unit 19 supplies the specified address information to the control unit 17. When the address information cannot be specified (recognized), the address determination unit 19 transmits the image of paper sheets and video coding information (coding data) including the character recognition result to the image storage / distribution device 21.

  The address DB 19a stores address information to be recognized. In the present embodiment, it is assumed that the address information registered in the address DB 19a is information composed of multiple layers of data. In the address DB 19a, for example, all address information existing in an area to be processed of paper sheets (for example, mail) is stored in a tree structure (hierarchical structure). The address DB 19a may be updatable. The address DB 19a may be any storage device that can be accessed by the address determination unit 19. For example, the address DB 19a may be provided outside the sorter main body 3.

  The BCW 15 prints an ID barcode or a destination barcode on a paper sheet as necessary. For example, the BCW 15 prints a destination barcode obtained by converting the address information as a recognition result into a barcode on a paper sheet whose address information can be recognized by the address determination unit 19. Further, the BCW 15 prints an ID barcode obtained by converting the identification information (ID code) given from the control unit 17 into a barcode for paper sheets whose address information cannot be recognized by the address determination unit 19. That is, the BCW 15 prints the recognition result as a destination barcode on a paper sheet for which address information can be recognized, and prints an ID barcode on a paper sheet for which address information cannot be recognized. That is, the destination barcode is a barcode indicating the address information itself, and the ID barcode is a barcode indicating identification information for identifying the paper sheet. The identification information of the paper sheet indicated by the ID barcode is information for associating the address information input by keystroke with the VCS 4 and the paper sheet. In other words, the paper sheet on which the ID barcode is printed is a paper sheet to be processed by the VCS 4.

  On the downstream side of the BCW 15 in the sheet transport direction, a sorting unit 16 is provided for sorting the sheets according to the address information. The partitioning section 16 includes a plurality of partition pockets (not shown) partitioned into a plurality of stages and a plurality of rows. Each pocket is set corresponding to each sorting destination, and sheets are sequentially accumulated in the pocket corresponding to the address information based on the address information or the machine code.

  Further, the sorting unit 16 is provided with a VCS exclusion pocket (not shown) in which paper sheets whose sorting destination cannot be recognized are accumulated. The paper sheets accumulated in the VCS exclusion pocket are re-supplied to the supply unit 11 after the address information is input by the VCS 4. The paper sheets re-supplied to the supply unit 11 are re-sorted based on the ID code printed on the paper sheets and the address information input by the VCS 4. The control unit 17 sorts the paper sheets into the respective pockets of the sorting unit 16 based on the address information as the sorting information.

  The learning unit 20 manages a learning function for the dictionary 18a for character recognition or the address DB 19a for address determination. In the VCS 4, the learning unit 20 updates the dictionary data stored in the dictionary 18a or the address data stored in the address DB 19a based on information including character information and address information input by the operator. The learning unit 20 will be described in detail later. In the present embodiment, a processing example in which the learning unit 20 updates the character recognition dictionary 18a will be mainly described. However, a learning process can be performed by a similar processing method for a dictionary for recognizing a character string by changing the recognition target from a character unit to a character string unit such as a word.

Next, the VCS 4 will be described.
The VCS 4 includes an image storage / distribution device 21, a plurality of video coding disks (hereinafter abbreviated as VCD) 22, and the like. The image storage / distribution device 21 is realized by a computer having a control unit, a storage unit, various interfaces, and the like. The VCD 22 is realized by a computer having a display unit, an input unit, a control unit, a storage unit, various interfaces, and the like, for example.

  The image storage / distribution device 21 is connected to the sorter main body 3 and each VCD 22. The image storage / distribution device 21 stores video coding information (coding data) including image of paper sheets whose address information could not be recognized by the character recognition unit 18 and the address determination unit 19 in the sorter body 3. 3 is received. The image storage device 21 monitors the operating status of each VCD 22 and distributes the coding data including the sheet image received from the sorter main body 3 to each VCD 22 according to the operating status of each VCD 22.

  Each VCD 22 displays a paper sheet image included in the coding data distributed from the image storage device 21 on the display unit 27 and prompts the operator to input correct address information (character information). The VCD 22 returns input information including character information as address information input by the operator to the image storage / distribution device 21 in a state where the image of the paper sheet is displayed on the display unit 27. The image storage / distribution device 21 performs a process of returning the input information acquired from each VCD 22 to the sorter body 3.

In the configuration example illustrated in FIG. 1, the VCD 22 includes a CPU 23, a nonvolatile memory 24, a RAM 25, a ROM 26, a display unit 27, an input unit 28, and a line-of-sight detection unit 29.
The CPU 23 functions as a control unit that controls the entire VCD 22. The CPU 23 performs various processes based on the control program and control data stored in the ROM 26 or the nonvolatile memory 24. For example, the CPU 23 performs basic operation control of the VCD 22 by executing an operating system program. Note that some of the various functions may be realized by a hardware circuit.

  The nonvolatile memory 24 is configured by a nonvolatile memory capable of writing and rewriting data, such as an EEPROM, a flash ROM, an HDD (hard disk drive), or an SSD (Solid State Disk). The non-volatile memory 24 stores a control program, control data, and various data in accordance with the operation application of the VCD 22. For example, the nonvolatile memory 24 stores coding data including an image for video coding (an image including a character image) supplied from the image storage / distribution device 21. Further, the nonvolatile memory 24 may store input information input by an operator, line-of-sight position information described later, and the like.

  The RAM 25 is a volatile memory. The RAM 25 temporarily stores data being processed by the CPU 23. For example, the RAM 25 stores image data for display, and stores input information and line-of-sight position information input by the operator. The ROM 26 is a non-rewritable nonvolatile memory in which a control program and control data are stored in advance.

  The display unit 27 is configured by a liquid crystal display or the like. For example, the display unit 27 displays an image for video coding (for example, an image of a paper sheet) supplied from the image storage / distribution device 21. The display unit 27 displays not only paper sheet images (images including characters to be recognized) as video coding images, but also information on the range recognized on the sorter body 3 side. Also good. The input unit 28 is a device for an operator to input character information as address information included in an image displayed on the display unit 27. For example, the input unit 28 includes a keyboard and a pointing device.

  The line-of-sight detection unit 29 detects an operator's line-of-sight position. The line-of-sight detection unit 29 detects a position (line-of-sight position) at which the operator is gazing on the display screen of the display unit 27 in a state where an image of a paper sheet that is a target of video coding is displayed. The line-of-sight detection unit 29 only needs to be able to detect the operator's line-of-sight position on the display screen of the display unit 27.

  For example, the line-of-sight detection unit 29 includes two cameras and a processing unit. The two cameras are set apart from each other by a certain distance, and each of them captures an image including the operator's pupil. Each camera transmits an image including the photographed image of the operator's pupil to the processing unit. The processing unit that has received an image from each camera estimates the shape of the eyeball of the operator based on the difference between the two images taken by the two cameras, and estimates the tangent plane with the eyeball at the iris center. The processing unit calculates a normal from the estimated tangent plane, and detects the line of sight of the operator. Further, the processing unit estimates the position of the operator's eyeball. The processing unit estimates at which position on the display screen of the display unit 27 the operator's line-of-sight position is based on the operator's eyeball position and line-of-sight direction.

  The line-of-sight detection unit 29 is not limited to the configuration described above. For example, the line-of-sight detection unit 29 may be configured by a plurality of cameras, and another processing unit such as the CPU 23 may perform arithmetic processing for detecting the line-of-sight position. In this case, the camera as the line-of-sight detection unit 29 outputs the captured image to a processing unit such as the CPU 23 having a processing function for detecting the line-of-sight position from the image. Further, the line-of-sight detection unit 29 is not limited to the configuration having a camera, and may be one that detects the line-of-sight position on the display screen by a device worn by the operator.

  The line-of-sight detection unit 29 detects the line-of-sight position of the operator at predetermined intervals. The line-of-sight detection unit 29 transmits information indicating the line-of-sight position of the operator detected at predetermined intervals to the CPU 23 as line-of-sight position information associated with time information. For example, the line-of-sight detection unit 29 detects an operator's line-of-sight position in a cycle of about 10 to 30 times per second, and displays line-of-sight position information indicating the detected line-of-sight position of the operator and the time when the line-of-sight position is detected. You may make it transmit to CPU23.

Next, an operation example of the VCD 22 will be described.
FIG. 2 is a flowchart showing an operation example of the VCD 22 according to the present embodiment.
The image storage / distribution device 21 sequentially receives data for coding including an image of a paper sheet whose address could not be read by the address determination unit 19 of the sorter body 3 from the sorter body 3. For example, the coding data is data including an image of a paper sheet and an ID of the paper sheet. The coding data may further include processing results (for example, information indicating recognized characters) by the character recognition unit 18 and the address determination unit 19. The image storage / distribution device 21 stores the coding data supplied from the address determination unit 19. The image storage / distribution device 21 distributes the stored coding data to each VCD 22 in accordance with the operation status of each VCD 22.

  The VCD 22 acquires coding data including a paper sheet image distributed from the image storage / distribution device 21 (step 11). That is, the VCD 22 uses the image storage / distribution device 21 to store coding data including an image of a paper sheet (an image including characters to be recognized) for which the address determination unit 19 could not identify an address. Get from 3.

FIG. 3 is a diagram illustrating an example of a paper sheet image included in the coding data supplied to the VCD 22. In the following description, it is assumed that the sheet image included in the coding data received by the VCD 22 is an image as shown in FIG. 3 as an example.
When the coding data including the paper sheet image is acquired, the CPU 23 displays the paper sheet image included in the received coding data on the display unit 27 (step 12). The paper image is an image including a character that is a target of character (pattern) recognition. In a state where the image of the paper sheet is displayed on the display unit 27, the CPU 23 receives an input to the input unit 28 by the operator.

  Further, in the state where the image of the paper sheet is displayed on the display unit 27, the line-of-sight detection unit 29 detects the line-of-sight position of the operator on the display screen of the display unit 27, and detects the detected line-of-sight position and the line-of-sight position. Information including the time is output to the CPU 23 as line-of-sight position information (steps S13 and S14). That is, the line-of-sight detection unit 29 acquires information for specifying the operator's line-of-sight position (for example, an image including the operator's pupil imaged by the camera) (step 13), and the acquired information (image captured by the camera) From this, the line-of-sight position of the operator is detected (step S14).

  For example, the line-of-sight detection unit 29 captures an image including the eyes of the operator who is viewing the display screen of the display unit 27 with a plurality of (for example, two) cameras. When the operator's pupil is photographed by a plurality of cameras, the line-of-sight detection unit 29 estimates the shape of the eyeball of the operator based on the difference between the plurality of images photographed by the plurality of cameras by the processing unit, and contacts the eyeball at the iris center. Estimate the plane. When the tangent plane is estimated, the processing unit of the line-of-sight detection unit 29 calculates the normal from the tangent plane and calculates the line-of-sight direction of the operator. The processing unit of the line-of-sight detection unit 29 detects where the operator is looking on the display screen of the display unit 27 based on the calculated line-of-sight direction, that is, the operator's line-of-sight position.

  Steps 13 and 14 are not limited to the above-described method, as long as they are processes that can detect the line-of-sight position in the recognition target image of the operator who is performing the character input operation. For example, as a process for detecting the line-of-sight position, a method of detecting the line-of-sight position of the operator on the display screen based on information from a device worn by the operator may be applied. A method of detecting the line-of-sight position on the display screen from the line-of-sight direction specified based on the matching may be applied.

  When the line-of-sight detection unit 29 detects the line-of-sight position, the CPU 23 associates the information indicating the line-of-sight position detected by the line-of-sight detection unit 29 with the time when the line-of-sight position is detected, and stores it in the nonvolatile memory 24 (step 15). . Information indicating the line-of-sight position on the display screen of the display unit 27 and information indicating the time when the line-of-sight position was displayed are held in the nonvolatile memory 24 as line-of-sight position information when the image of the paper sheet is coded. As a result, time-series eye-gaze position information is stored in the nonvolatile memory 24. The CPU 23 may store the line-of-sight position information in the RAM 25.

  In parallel with the process of storing the line-of-sight position information in the non-volatile memory 24, the CPU 23 performs the process of storing information indicating the character input by the operator using the input unit 28 such as a keyboard as input character information in the non-volatile memory 24. (Steps S16 and S17). That is, the CPU 23 detects information input through the input unit 28 such as a keyboard or a pointing device (step 16). When a character (address) is input by the input unit 28 (step 16, YES), the CPU 23 uses information associating the input character (or word) and the input time as character input information to the nonvolatile memory 24. (Step 17).

  The CPU 23 stores the line-of-sight position information and the character input information in the nonvolatile memory 24 as learning data of the character recognition dictionary 18a obtained from one sheet of paper image. That is, the nonvolatile memory 24 stores the line-of-sight position information as time-series position information, and stores the input character information in association with the input time. In this way, according to the learning data including the time-series gaze position information stored in the nonvolatile memory 24 and the input character information associated with the time, the position (gaze position) that the operator was viewing when each character was entered. ), Or the position (line-of-sight position) that the operator was looking at before entering each character can be easily specified.

  Further, the CPU 23 determines whether or not the character input process is completed in accordance with the input from the input unit 28 (step S18). For example, the CPU 23 determines that the character (address) input process as the coding process for the paper sheet image displayed on the display unit 27 has been completed when the input unit 28 inputs a character input completion instruction. You can do it.

  If the character input process (coding process) for the image displayed on the display unit 27 is not completed (step S18, NO), the CPU 23 returns to step 13 and repeats the process described above. For example, the CPU 23 repeats the line-of-sight position detection process in steps S13 to S14 so that the line-of-sight position of the operator can be detected at a predetermined timing (for example, about 10 to 30 times per second) until the character input process is completed. Run. Further, the CPU 23 performs a character input information storage process as a process of step S17 in accordance with the input of characters until the character input process is completed.

  When the character input process (coding process) for the paper sheet image displayed on the display unit 27 is completed (step S18, YES), the CPU 23 uses the input information including the input characters as the image of the paper sheet. Address information. The CPU 23 uses the image storage / distribution device 21 to classify information indicating the coding result associated with the address information including the input characters and the information for identifying the paper sheet (ID of the paper sheet). It transmits to the main body 3 (step S19).

  When the character input process is completed (YES in step S18), the CPU 23 uses the line-of-sight position information and the input character information stored in the nonvolatile memory 24 as learning data in the character recognition dictionary 18a. Is supplied to the learning unit 20 of the sorting machine main body 3 (step S20). The CPU 23 may transfer the learning data for one case to the learning unit 20 every time the coding process (character input processing) of the address information for one case is completed, or the non-volatile memory at a predetermined timing. The learning data stored in the learning data may be transferred to the learning unit 20, or a predetermined number of pieces of learning data may be collectively transferred to the learning unit 20.

Next, the relationship between the line-of-sight position and the input character will be described.
FIG. 4 is a diagram showing an example of the line-of-sight position of an operator who inputs character (address) information by looking at an image of a paper sheet as shown in FIG. .
A line l in FIG. 4 shows the movement of the operator's line-of-sight position in time series. In the example shown in FIG. 4, the line-of-sight position of the operator is concentrated on “stock Holm” in the image of the paper sheet, as indicated by the line l. A point a shown in FIG. 4 indicates the line-of-sight position at the time when the operator inputs “H” as an input character by the input unit 28. As described above, the VCD 22 uses the time-series gaze position information and the input character information in association with the time information, and the gaze position at the time when each character (for example, “k”, “o”, etc.) is input. It also has information indicating. Input characters are also stored as address information. For this reason, the VCD 22 can also obtain address information described in the image of the paper sheet.

  When it is determined that the character input process (coding process) has been completed, the CPU 23 reads the address information as the character information input by key entry stored in the nonvolatile memory 24 or the RAM 25. The CPU 23 transmits the read address information to the sorter main body 3 through the image storage / distribution device 21 in association with the identification information (ID code) of the paper sheet. As a result, the sorting machine main body 3 can acquire the address information that is key-input by the VCS corresponding to the ID barcode assigned to the paper sheet as the address information of each paper sheet.

  Further, the CPU 23 associates the image of the paper sheet stored in the nonvolatile memory 24 with the image storage / distribution device 21, the line-of-sight position information indicating the line-of-sight position at each time, and the input character and the input time. Input character information is transmitted to the sorter body 3 as learning data. Since the learning data is information for updating the character recognition dictionary 18a, the learning data may be supplied before the learning unit 20 executes the learning process. That is, the learning data does not have to be transmitted to the sorting machine main body 3 every time the character input process is completed, and may be transmitted to the sorting machine main body 3 at an arbitrary timing.

Next, the learning data created by the VCS 4 will be described.
FIG. 5 is a configuration example of the learning data created by the VCD 22.
The learning data includes paper sheet image data (XXX.JPG in the example shown in FIG. 5), line-of-sight position information indicating the line-of-sight position at each time, and input character information in which the input character is associated with the input time. Have In the example shown in FIG. 5, it is shown that the line-of-sight position of the operator at time a is the position of coordinates (x100, y100) on the display screen. Further, the example shown in FIG. 5 indicates that the character information “H” is input at time b (in other words, the time when the character “H” is input is “b”).

  The example shown in FIG. 5 indicates that no character is input at time a and “H” is input as a character at time b. Further, the line-of-sight position at time b when the character “H” is input can be specified as coordinates (x200, y200) on the display screen based on the line-of-sight position information. That is, the learning data as shown in FIG. 5 includes time series line-of-sight positions and information in which input characters are associated with input times. For this reason, according to the learning data as shown in FIG. 5, it is possible to specify which position the operator was looking at when inputting which character.

Next, the learning unit 20 will be described.
Here, a process (learning function) in which the learning unit 20 updates the character recognition dictionary 18a using the learning data supplied from the VCS (VCD) will be described. However, the learning process to be described later can be applied to dictionary learning for pattern recognition. For example, the learning process described later can be applied not only to a single character but also to a word, and therefore may be applied to updating address information in the address DB 19a. Further, the learning process described later may be applied not only to characters or words but also to a dictionary learning process used for other pattern recognition such as a biometric authentication process or a specific object detection process.

  That is, the learning unit 20 updates the dictionary 18a using the learning data supplied from the VCS 4. The learning unit 20 identifies a partial image (character region candidate) that is estimated to have a character written in a paper sheet image. The learning unit 20 calculates the probability (input probability) that the input character input by the operator is written in the partial image for each identified partial image. That is, the learning unit 20 calculates which portion of the image to be authenticated is likely to have an input character written or is low. The learning unit 20 updates the dictionary 18a based on the feature amount extracted from the partial image and the probability that the input character exists in the partial image.

FIG. 6 is a diagram illustrating a configuration example of the learning unit 20 as a learning device.
In the configuration example illustrated in FIG. 6, the learning unit 20 includes a data storage unit 31, a hypothesis generation unit 32, a hypothesis prior probability setting unit 33, a hypothesis posterior probability setting unit 34, a feature extraction unit 36 and an update unit 37. The learning unit 20 implements various processing functions including the processing of each unit described above by the processor executing the control program. For example, the learning unit 20 can be realized by a computer having a processor, a memory, and an interface.

  The data storage unit 31 stores learning data supplied from the VCS 4. The data storage unit 31 has a configuration for storing learning data supplied from the VCS 4 in a nonvolatile memory. The data storage unit 31 stores, for example, information including image data of an image to be recognized, line-of-sight position information, and input character information as learning data. The learning data stored in the data storage unit 31 can be read as appropriate. The learning data stored in the data storage unit 31 is supplied to the hypothesis generation unit 32, the hypothesis prior probability setting unit 33, and the like.

  The hypothesis generation unit 32 generates, as a hypothesis, an area in which the character (correct answer character) input by the VCS 4 is described in an image (paper image) that is a character recognition target. The hypothesis generation unit 32 generates a partial image (character area candidate) that is likely to be a description area of the character from an image that is a recognition target for one input character (correct character). The generated character area candidate as a partial image is also referred to as a hypothesis. For example, the hypothesis generation unit 32 extracts a circumscribed rectangular region for a connected pixel component that seems to be a character as a partial image (character region candidate) as a hypothesis in the recognition target image. A method of extracting a partial image (character area candidate) as a hypothesis is not limited to a specific method.

  Further, the hypothesis generation unit 32 may narrow down character region candidates as hypotheses with reference to the line-of-sight position information. That is, the hypothesis generation unit 32 refers to the line-of-sight position information included in the learning data, and narrows down hypotheses (character area candidates) that are likely to have a character to be learned (input correct character) written therein. You may do it. In this case, the hypothesis generation unit 32 selects, as a hypothesis, a character area candidate that exists around the line-of-sight position at the time when the character is input (or the line-of-sight position at the time before the character is input). For example, when generating a hypothesis in which “H” is written, the hypothesis generation unit 32 uses, as hypotheses, character region candidates around the line-of-sight position at the time when the character “H” is input among the character region candidates. It may be selected (generated). Further, the hypothesis generation unit 32 may generate a hypothesis for each character based on character characteristics or the like, or may generate a hypothesis independently of the character. The method of generating a hypothesis is not limited to a specific method.

The hypothesis prior probability setting unit 33 calculates the probability that the character is written in the character region candidate generated as a hypothesis by the hypothesis generation unit 32. This probability is also referred to as prior probability. A hypothesis in which the character is written is also called a correct hypothesis. The hypothesis prior probability setting unit 33 calculates the probability without using the feature amount in each hypothesis (character region candidate).
The hypothesis prior probability setting unit 33 calculates, for example, the probability (prior probability) that each hypothesis is a correct hypothesis based on the line-of-sight position of the operator at the time when the operator inputs a certain character. The operator inputs characters while looking at the recognition target image (paper sheet image) displayed on the display unit 27. For this reason, there is a high probability that the hypothesis around the position where the line of sight was at the time when the operator entered the character (the line of sight position when inputting the character) is the correct hypothesis. That is, a hypothesis near the position where the line of sight was at the character input time is likely to be a correct hypothesis, and a hypothesis far from the line of sight position has a low probability of being a correct hypothesis.

  In addition, the operator may input a character after a little time has elapsed since viewing the image displayed on the display unit 27. In other words, there may be a time lag from when the operator sees the displayed character until when the character is actually input. Assuming such a case, the hypothesis around the position where the line of sight was slightly before the character input time (the line of sight immediately before the character input) may have a high probability of being the correct hypothesis. Note that the prior probability determination method is not limited to a specific method. For example, the prior probability determination method for the hypothesis may be changed for each operator. By adopting a prior probability determination method that reflects an operator's habit (the tendency of the line-of-sight position when each operator inputs a character) and the like, a highly accurate prior probability can be calculated.

  The hypothesis posterior probability setting unit 34 uses, for each hypothesis for a certain character generated by the hypothesis generation unit 32, the prior character and the feature parameter (feature amount) in the character region candidate as each hypothesis, so that the character is assigned to each hypothesis. Calculate the probability described (probability that each hypothesis is a correct hypothesis). This probability is also called posterior probability.

  The hypothesis posterior probability setting unit 34 acquires the character feature amount (feature parameter) in the character region candidate as each hypothesis from the feature extraction unit 36 described later. The hypothesis posterior probability setting unit 34 compares, for example, the character parameters of each hypothesis with the character parameters of the input characters stored in the dictionary 18a, thereby determining the characters and input characters (correct characters) in each hypothesis. (The probability that the hypothesis is correct in the current dictionary 18a) is calculated. The hypothesis posterior probability setting unit 34 calculates the similarity (probability that the hypothesis is correct in the current dictionary 18a) calculated by the feature parameter of each hypothesis and the input character feature parameter of the dictionary 18a and the hypothesis pre-setting unit 33. Use the prior probabilities to calculate the probability that each hypothesis is the correct hypothesis (posterior probability).

  For example, the hypothesis posterior probability setting unit 34 calculates the probability (posterior probability) that each hypothesis is a correct hypothesis from the feature amount of the input character and the prior probability based on the operator's line-of-sight position. The method for calculating the posterior probability is not limited to a specific method. The learning unit 20 can estimate the correct hypothesis more accurately than the prior probability calculated from the line-of-sight position by using the posterior probability calculated from the prior probability and the similarity.

  The feature extraction unit 36 extracts feature parameters from a partial image (character candidate region) as a hypothesis. The feature parameter extracted by the feature extraction unit 36 is data to be compared with the feature parameter used in the dictionary 18a. For example, the feature extraction unit 36 may extract features as feature parameters using brightness gradient information after blurring a partial image as a hypothesis as a 128-dimensional vector. The feature extraction unit 36 may use the density value of the pixel as a feature parameter, or may use higher-order information as a feature. The feature parameter extraction method is not limited to a specific method.

  The updating unit 37 updates (learns) the dictionary data stored in the dictionary 18a based on the posterior probability and the hypothetical feature parameter (character feature parameter of the character region candidate). The updating unit 37 updates the dictionary 18a so that the feature parameter of the hypothesis having a small posterior probability is not largely reflected and the feature parameter of the hypothesis having a large posterior probability is largely reflected. Further, the update unit 37 may reflect information obtained from new learning data more largely than information obtained from old learning data. The updating method for the dictionary 18a is not limited to a specific method. A specific example of the update method will be described later.

Next, an example of an algorithm used for the learning process of the dictionary 18a in the learning unit 20 will be described.
First, the learning unit 20 reads learning data from the data storage unit 31. Here, as a specific example, it is assumed that the recognition target image included in the learning data is an image of a paper sheet shown in FIG. In the image of the paper sheet shown in FIG. 3, “Taro Toshishi TOSHIBE vaguen 151 stockHolm SWEDEN” is described. The learning data including the image shown in FIG. 3 includes “s” “t” “o” “c” “k” “H” “o” “l” “m” as input characters. Further, the learning data including the image illustrated in FIG. 3 includes information indicating the line-of-sight position as illustrated in FIG. 4 as the line-of-sight position information. The line-of-sight position shown in FIG. 4 is concentrated at the position described as “stockHolm”. Point a in FIG. 4 is the operator's line-of-sight position at the time when the character “H” is input.

  The hypothesis generation unit 32 acquires learning data from the data storage unit 31. When the learning data is acquired, the hypothesis generation unit 32 extracts, from the image data, a region (partial image) as a hypothesis that the character is described with respect to one input character (correct character). Create one or more. For example, it is assumed that the hypothesis generation unit 32 generates a hypothesis estimated that “H” is written. The hypothesis generation unit 32 selects, as a hypothesis, a character candidate region around the line-of-sight position (“×” shown in FIG. 4) when the character “H” is input.

FIG. 7 shows an example of a hypothesis generated by the hypothesis generation unit 32.
Hypotheses 41 to 51 are examples of hypotheses as character candidate regions estimated to include the letter “H”. Each hypothesis 41 to 51 is an example of a hypothesis (character candidate region) selected with reference to the line-of-sight position among the character candidate regions generated based on the circumscribed rectangle for the connected pixel component. The hypothesis generation unit 32 may generate one character region candidate (hypothesis) by combining adjacent rectangles among circumscribed rectangles of the connected pixel components. For example, hypothesis 45 and hypothesis 48 include a plurality of hypotheses. Hypothesis 45 and hypothesis 48 are hypotheses considering the possibility that a plurality of circumscribed rectangles are regions including one character. Hypotheses composed of a plurality of circumscribed rectangles can be combined according to the distance between adjacent rectangles, the relative size of each adjacent rectangle, and the like. The generated hypothesis (character candidate area) is not necessarily limited to a rectangle.
The hypothesis generation unit 32 transmits the generated hypothesis to the hypothesis prior probability setting unit 33 and the feature extraction unit 36.

  The hypothesis prior probability setting unit 33 receives a hypothesis for each input character from the hypothesis generation unit 32. The hypothesis prior probability setting unit 33 calculates the prior probability for each hypothesis based on the line-of-sight position information. The hypothesis prior probability setting unit 33 calculates the probability that each hypothesis is likely to be a correct hypothesis based on the line-of-sight position when the operator inputs a character (immediately before).

Here, if the input character is c and the hypothesis is hi (i = 1 to N, N is the number of hypotheses for c), the prior probability is expressed as P (hi | c). In this case, the prior probability satisfies the following formula.

  Here, it is assumed that there is a hypothesis that “does not exist” in consideration of the case where there is no correct hypothesis in the hypothesis. Therefore, Formula 1 is always satisfied.

  FIG. 8 is a diagram showing prior probabilities for the respective hypotheses 41 to 51 when the input character is “H”. In the example shown in FIG. 8, hypotheses 41 to 51 are in order of 0.1%, 0.2%, 0.5%, 1%, 3%, 5%, 15%, 30%, 40%, 0.1 % And 0.1% are set. Therefore, hypotheses (such as hypothesis 48 and hypothesis 49 in the example shown in FIG. 8) around the line-of-sight position (“×” shown in FIG. 8) at the time when the character “H” is input have a high prior probability. Is set. Further, a hypothesis at a position just before the line-of-sight position (such as hypothesis 47 and hypothesis 46 in the example shown in FIG. 8) is also set with a relatively large probability compared to other hypotheses.

Conversely, hypotheses subsequent to the line-of-sight position (hypothesis 50 and hypothesis 51 in the example shown in FIG. 8) have a low probability despite being hypotheses close to the line-of-sight position. This is because the probability that the operator is looking at a character after the line-of-sight position at the time of character input is low (that is, the operator is considered to input a character that has already been viewed while looking at the character in the image). Because. Further, the probability that none of the hypotheses 41 to 51 is a correct hypothesis (that is, the probability of a hypothesis that the correct hypothesis does not exist) is set to 5%. Therefore, the hypothetical example shown in FIG.
The hypothesis prior probability setting unit 33 transmits the prior probabilities calculated for each hypothesis to the hypothesis posterior probability generation unit 34.

The feature extraction unit 36 extracts a feature parameter (feature amount) X for each hypothesis received from the hypothesis generation unit 32. Here, X represents all N features extracted from each hypothesis for the input character c. In other words, X = (x1, x2,... XN), where xi is a character feature extracted from the hypothesis hi. As described above, the feature xi is data to be compared with the feature parameter as dictionary data stored in the dictionary 18a.
The feature extraction unit 36 transmits the feature X as a character extracted from the character region candidate as each hypothesis to the hypothesis posterior probability setting unit 34 and the update unit 37.

The hypothesis posterior probability setting unit 34 that has received the prior probability and feature X of each hypothesis calculates the posterior probability from the prior probability and the feature X. The posterior probability is represented by P (hi | X, c). The posterior probability satisfies the following formula.

The posterior probability can be transformed as follows using Bayes' theorem.

Here, it is approximated that the features x1, x2,... XN for each hypothesis are independent of each other. The probability that the hypothesis hi is a correct hypothesis is assumed to be affected by the i-th feature xi. Therefore, if k ≠ i, it can be approximated as follows.

Therefore, the first factor of the numerator on the right side of Equation 3 can be approximated as follows.

By substituting Equation 5 into the first factor in the numerator of Equation 3 and the first factor in Σ of the denominator and dividing the denominator numerator, the following equation is derived.

Where in Equation 6

Or

Can be calculated using dictionary data stored in the dictionary 18a. Therefore, the hypothesis posterior probability setting unit 34 can calculate the posterior probability using Equation 6. However, the method of calculating the posterior probability is not limited to a specific method such as the method described above.

  FIG. 9 is a diagram illustrating a calculation result of the posterior probability for the input character “H”. In the example shown in FIG. 9, the hypothesis 47, which is the correct answer hypothesis, has a probability of 90%. 8 and FIG. 9, the posterior probability of the hypothesis 47 shown in FIG. 9 is higher than the prior probability shown in FIG. The hypothesis posterior probability setting unit 34 compares the feature parameter extracted from the hypothesis with the dictionary data stored in the dictionary 18a and refers to the line-of-sight position when the character is input, so that the correct hypothesis Set a posterior probability higher than the probability.

In the example shown in FIG. 9, each hypothesis other than the hypothesis 47, which is the correct hypothesis, is set with a posterior probability lower than the prior probability shown in FIG. That is, the hypothesis posterior probability setting unit 34 gives high confidence to the correct hypothesis (higher posterior probability) for each hypothesis for which the prior probability is set, and lowers the confidence for hypotheses other than the correct hypothesis. (Lower posterior probability).
The hypothesis posterior probability setting unit 34 transmits the posterior probability calculated for each hypothesis for each input character to the update unit 37 as a learning unit.

The update unit 37 acquires the posterior probability calculated based on the learning data from the hypothesis posterior probability setting unit 34, and acquires the feature parameters of each hypothesis from the feature extraction unit 36. When the posterior probability and the feature parameter X are received, the updating unit 37 updates the dictionary data stored in the dictionary 18a using the acquired posterior probability and the feature parameter X for the input character. Here, as an example, it is assumed that the dictionary 18a stores a Gaussian distribution of feature parameters for each character as dictionary data. It is assumed that the Gaussian distribution parameters as character feature parameters are the mean m and the covariance matrix Σ. In this case, the update unit 37 calculates the average m and the covariance matrix Σ by the following formula.

  Here, the feature parameter Xc is a set of features extracted from all hypotheses corresponding to the input character c. The feature parameter xi is a feature extracted from the hypothesis hi. Wi is a posterior probability that the hypothesis hi is a correct hypothesis. That is, Equation 9 calculates a weighted average with wi as the “weight” of the feature. Also, Equation 10 calculates a weighted covariance matrix with wi as the feature weight. When the calculation of the mean m and the covariance matrix Σ is finished, the updating unit 37 updates the average m and the covariance matrix Σ, which are stored in the dictionary 18a, with the calculated parameters of the Gaussian distribution as the characteristic parameters for the input character c. To do.

  The updating unit 37 calculates the average m and the covariance matrix Σ by using Equation 9 and Equation 10 for each input character that has acquired the posterior probability and the feature parameter for each hypothesis. The update unit 37 updates the parameters of the Gaussian distribution of the feature parameters for each input character c to the calculated mean m and covariance matrix Σ.

  Note that the updating unit 37 may update the dictionary 18a every time the posterior probability and feature parameter of each hypothesis for one input character are acquired, or all the input characters included in one learning data The dictionary 18 may be updated every time the posterior probabilities and feature parameters of each hypothesis are acquired. Further, the updating unit 37 obtains the posterior probabilities and feature parameters of the respective hypotheses for all input characters included in all the learning data stored in the data storage unit 31 periodically or at an arbitrary timing. May be updated. In this case, the learning unit 20 repeats the above processing until it acquires the posterior probabilities and feature parameters of each hypothesis for all input characters included in all the learning data stored in the data storage unit 31, and then updates The dictionary 18a may be updated by the unit 37. The updating unit 37 updates the dictionary 18a using learning data created based on information (input character information and line-of-sight position information) input in the past by the VCS 4.

For example, the update unit 37 updates the dictionary data stored in the dictionary 18a by the following update formula.

Here, m _old and Σ _old are parameters of a Gaussian distribution as feature quantities for the input character c stored as dictionary data in the dictionary 18a. m _{new new} and sigma _{new new} are parameters of the Gaussian distribution of the updated. Also, w _old is the “weight” of the parameter before update. The “weight” may be an arbitrary value. For example, when the “weight” is a large value, the current parameter is emphasized, and the change of the dictionary 18a becomes moderate. On the other hand, when the “weight” is a small value, the current parameter is not emphasized, and the change of the dictionary 18a becomes abrupt.

Further, when the character feature parameter as the dictionary data stored in the dictionary 18a is updated, the accuracy of the posterior probability calculated by the hypothesis posterior probability setting unit 34 increases. Therefore, the learning unit 20 can further improve the accuracy of the dictionary 18a by performing learning processing using the same learning data again using the updated dictionary 18a. Note that the learning process is not limited to a specific method such as the processing method described above.
In addition, the hypothesis generation unit 32, the hypothesis prior probability setting unit 33, the hypothesis posterior probability setting unit 34, the feature extraction unit 36, and the update unit 37 are realized by a processor executing a program. All may be realized by hardware.

Next, the flow of learning processing by the learning unit 20 will be described.
FIG. 10 is a flowchart for explaining the flow of the learning process of the dictionary 18a by the learning unit 20.
First, the learning unit 20 reads learning data from the data storage unit 31 (step 21). The read learning data is transmitted to the hypothesis generation unit 32.
When the learning data is received, the hypothesis generation unit 32 generates a hypothesis for a certain input character c from the line-of-sight information (step 22). For example, the hypothesis generation unit 32 extracts a region that seems to be a character region based on a circumscribed rectangle for a connected pixel component in an image to be recognized. Further, the hypothesis generation unit 32 may select a character area as a hypothesis based on the line-of-sight position when the character is input from an area that seems to be a character area. The hypothesis generation unit 32 transmits the generated hypothesis to the hypothesis prior probability setting unit 33 and the feature extraction unit 36.

  When the hypothesis generated by the hypothesis generator 32 is received, the hypothesis prior probability setting unit 33 calculates a probability that each hypothesis is a correct hypothesis, that is, a prior probability based on the line-of-sight position information (step 23). For example, the hypothesis prior probability setting unit 33 sets the prior probability of each hypothesis such that the probability increases as the line of sight is closer to the time when the character is input (or immediately before the character is input). The hypothesis prior probability setting unit 33 transmits the prior probabilities set for each hypothesis to the hypothesis posterior probability setting unit 34.

  The feature extraction unit 36 that has received the hypothesis generated by the hypothesis generation unit 32 extracts the feature parameters of each hypothesis (step 24). The feature parameter here is to be compared with the feature parameter of each character stored as dictionary data in the dictionary 18a. When the feature parameters of each hypothesis are extracted, the feature extraction unit 37 transmits the feature parameters of each hypothesis to the posterior probability setting unit 34 and the update unit 37. Steps 23 and 24 may be performed in the reverse order or in parallel.

When the prior probability from the prior probability setting unit 33 and the feature parameter from the feature extraction unit 36 are acquired for each hypothesis for the character, the posterior probability setting unit 34 calculates the posterior probability of each hypothesis for the character from the acquired prior probability and feature parameter. Calculate (step 25). The hypothesis posterior probability setting unit 34 transmits the calculated posterior probability of each hypothesis to the update unit 37.
When the posterior probabilities and feature parameters of each hypothesis for the character are acquired, the updating unit 37 updates the dictionary data of the character stored in the dictionary 18a (step S26). For example, the updating unit 37 updates the dictionary data (feature parameter) of the character stored in the dictionary 18a using the feature parameter with the posterior probability as a weight.

  Note that the update unit 37 may update the dictionary 18a every time the posterior probability and feature parameter of each hypothesis for one character are acquired, or the posterior probability of each hypothesis for each character in units of learning data. The dictionary 18a may be updated after acquiring the feature parameters, or the posterior probabilities of each hypothesis and the feature parameters for each character included in all the learning data stored in the data storage unit 31 are acquired. After that, the dictionary 18a may be updated.

  Based on the above flow, the learning unit specifies the character description region and the characteristic parameter with high probability based on the character input by the operator and the line-of-sight position of the operator during the character input operation. The learning unit stores the input character in the dictionary based on the probability that the input character is a description region of the character and the feature parameter obtained from the description region, and updates the dictionary data of the character. Thereby, according to the learning unit according to the present embodiment, the correct character and the correct character in the recognition target image are described without the person directly specifying the correct character and the probability that the correct character is described. The dictionary can be efficiently learned from the probability of being.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  DESCRIPTION OF SYMBOLS 1 ... Paper sheet processing apparatus, 3 ... Sorting machine main body, 4 ... VCS, 10 ... Operation panel, 11 ... Supply part, 12 ... Main conveyance path, 13 ... BCR, 14 ... Scanner, 15 ... BCW, 16 ... Sorting machine , 17 ... control unit, 18 ... character recognition unit, 18a ... dictionary, 19 ... address determination unit, 19a ... address DB, 20 ... learning unit, 21 ... image storage / distribution device, 22 ... VCD, 23 ... CPU, 24 ... non-volatile 25 ... RAM, 26 ... ROM, 27 ... display unit, 28 ... input unit, 29 ... gaze detection unit, 31 ... data storage unit, 32 ... hypothesis generation unit, 33 ... hypothesis prior probability setting unit, 34 ... hypothesis A posteriori probability setting unit, 36... Feature extraction unit, 37.

Claims (12)

A generation unit that generates pattern region candidates in an image to be recognized that a pattern input by an input person seems to be described;
A feature extraction unit that calculates a feature amount in the pattern area candidate generated by the generation unit;
Based on the line-of-sight position of the input person who input the pattern, a setting unit that sets a probability that the pattern input by the input person is described in the pattern area candidate generated by the generation unit;
An update unit that updates the pattern recognition dictionary based on the probability set by the setting unit and the feature amount calculated by the feature extraction unit;
A device for learning a pattern recognition dictionary. The setting unit
A prior probability setting unit for setting a prior probability that a pattern input by the input person is described in a pattern region candidate generated by the generation unit based on the line-of-sight position of the input person;
A posterior probability setting unit that sets a posterior probability for the pattern region candidate based on the prior probability and the feature amount in the pattern region candidate extracted by the feature extraction unit;
The update unit updates a pattern recognition dictionary based on the posterior probability and the feature amount for the pattern region candidate.
The learning apparatus for the pattern recognition dictionary according to claim 1. The generation unit selects a pattern region candidate from a plurality of pattern region candidates based on the line-of-sight position of the input person who has input the pattern.
The learning apparatus of the pattern recognition dictionary of any one of Claim 1 or 2. The prior probability setting unit sets a prior probability for each character region candidate based on the line of sight of the input person at the time when the input person inputs the pattern,
The learning apparatus for a dictionary for pattern recognition according to any one of claims 1 to 3. The prior probability setting unit sets a prior probability for a pattern region candidate based on the line-of-sight position of the input person at a time before the input person inputs the pattern,
5. The pattern recognition dictionary learning device according to claim 1. The posterior probability setting unit applies the pattern region candidate to the pattern region candidate based on the prior probability, the feature amount in the pattern region candidate, and the similarity between the pattern feature amount stored in the pattern recognition dictionary. Set the posterior probability,
The pattern recognition dictionary learning device according to claim 1. The pattern is a character.
The pattern recognition dictionary learning device according to any one of claims 1 to 6. A display unit for displaying an image including a pattern to be recognized;
An input unit for an input person to input a pattern included in the image displayed on the display unit;
A line-of-sight detection unit that detects the line-of-sight position of the input person on the display screen of the display unit;
A coding device. And a data generation unit that generates learning data for a pattern recognition dictionary including information indicating a pattern input by the input unit and information indicating a line-of-sight position of the input person who has input the pattern.
The coding apparatus according to claim 8. A storage unit storing dictionary data for pattern recognition;
An image acquisition unit for acquiring an image including a pattern to be recognized;
A pattern recognition unit that recognizes a pattern included in the image acquired by the image acquisition unit using dictionary data stored in the storage unit;
A display unit for displaying an image in which a pattern could not be recognized by the pattern recognition unit;
An input unit for an input person to input a pattern included in the image displayed on the display unit;
A line-of-sight detection unit that detects the line-of-sight position of the input person on the display screen of the display unit;
A generation unit that generates pattern region candidates in the image in which the pattern input by the input unit seems to be described;
A feature extraction unit that calculates a feature amount in the pattern area candidate generated by the generation unit;
A setting unit that sets the probability that the pattern input by the input unit is described in the pattern area candidate generated by the generation unit based on the line-of-sight position detected by the line-of-sight detection unit;
An update unit that updates the dictionary data stored in the storage unit based on the probability set by the setting unit and the feature amount calculated by the feature extraction unit;
A pattern recognition apparatus. A sorting device for sorting sorting objects based on sorting information,
An image reading unit for reading an image of a description area of the classification information in the classification object;
A storage unit storing dictionary data for recognizing a pattern that can be classification information of a classification target;
A pattern recognition unit that recognizes a pattern included in an image read by the image reading unit using dictionary data stored in the storage unit;
A classification unit that classifies a classification target object based on classification information including a pattern obtained as a recognition result by the pattern recognition unit;
A display unit for displaying a read image of a classification object whose pattern constituting the classification information could not be recognized by the pattern recognition unit;
An input unit for an input person to input a pattern constituting the segment information included in the read image of the segment object displayed on the display unit;
A line-of-sight detection unit that detects the line-of-sight position of the input person on the display screen of the display unit;
A generation unit for generating pattern region candidates in the image of the segmented object that seems to describe the pattern input by the input unit;
A feature extraction unit that calculates a feature amount in the pattern area candidate generated by the generation unit;
A setting unit that sets the probability that the pattern input by the input unit is described in the pattern area candidate generated by the generation unit based on the line-of-sight position detected by the line-of-sight detection unit;
An update unit that updates the dictionary data stored in the storage unit based on the probability set by the setting unit and the feature amount calculated by the feature extraction unit;
Sorting device having. Generate pattern area candidates that seem to describe the pattern entered by the input person in the image to be recognized,
Calculating a feature amount in the generated pattern region candidate;
A probability that the pattern input by the input person is described in each generated pattern area candidate is set using the line-of-sight position of the input person,
Updating a pattern recognition dictionary based on the set probability and the calculated feature value;
Learning method for a pattern recognition dictionary.

Images