JP2023012225A - Support system, support method, and program - Google Patents

Abstract

To support a work for detecting an error in writing a hand-written character.SOLUTION: A support system according to the present invention has an acquisition unit for acquiring a learning data set including, as a set, a learning image and a first label associated with the learning image, an estimation unit for subjecting the learning image of the learning data set acquired by the acquisition unit to image recognition to thereby estimate a second label estimated to be associated with the learning image, a comparison unit for comparing the first label with the second label, a display unit for displaying the learning image having the first label and the second label different from each other based on the comparison result by the comparison unit, and an input unit input with input information indicating whether or not the image indicated as the learning image displayed by the display unit is different from the first label associated with the learning image.SELECTED DRAWING: Figure 1

Description

The present invention relates to a support system, support method and program.

In recent years, techniques for machine learning and character recognition using deep learning for handwritten characters written on forms and the like have been actively developed. Deep learning machine learning requires a large amount of handwritten characters and labels that represent the character types of the characters written there. At this time, if there is a writing error in the handwritten characters used for learning, learning the handwritten characters may cause a decrease in the accuracy of character recognition. Therefore, it is necessary to remove from the data for learning the characters that are incorrectly written and the characters that cannot be clearly read as correct characters by human eyes. For this reason, handwritten characters are inspected in advance for writing errors before learning, but handwritten characters have various strokes, so it is necessary to rely on human eyes to detect errors. However, since the data to be used for deep learning is enormous, the number of characters to be inspected is extremely large, and there is a possibility that oversights may occur. There is

Patent Literature 1 describes a technique for improving the efficiency of work related to correcting labels assigned to learning data. In the technique described in Patent Document 1, high-dimensional data obtained from an image is converted to low-dimensional data and displayed as a plot diagram, and a reference point selected on the plot diagram has the same label as the reference point. Corrects the label given to the image using the distance to the point. However, when the learning data is characters, it is not effective to use a plot diagram as a display method because the number of classes (types of characters) is as large as thousands.

There is also a technology for detecting errors in handwritten characters that aims at automatic scoring of Chinese characters. In Patent Document 2, by inputting an input handwritten character image and a correct character label that should be written there into a machine learning model, a correct character image having the characteristics of the input handwritten character image is generated. It describes a technique for detecting misspellings of kanji characters by taking a difference between a correct character image and an input handwritten character image. However, the method of taking the difference between the input character and the correct character is effective when judging whether the character shape is correct, such as scoring kanji, but it is effective for handwritten characters in a data set used for character recognition such as forms. It is not necessary to accurately determine character shapes such as , tome, han, and harai.

JP 2020-135631 A Japanese Patent Application Laid-Open No. 2021-26729

The technique described above has the problem that it is not possible to efficiently detect writing errors contained in a huge amount of handwritten characters.

SUMMARY OF THE INVENTION In view of the above problems, an object of the present invention is to provide a support system, a support method, and a program that can support the task of detecting handwriting errors.

A support system according to an aspect of the present invention includes an acquisition unit that acquires a learning data set in which a learning image and a first label associated with the learning image are a set; an estimation unit for estimating a second label estimated to be associated with the learning image by image recognition of the learning image of the training data set; the first label; the second label; a display unit for displaying the learning image in which the first label and the second label are different based on the comparison result by the comparison unit; and the learning image displayed by the display unit. an input unit for inputting input information indicating whether or not the image shown in the learning image is different from the first label associated with the learning image.

A support method according to an aspect of the present invention includes an acquisition process of acquiring a learning data set in which a learning image and a first label associated with the learning image are a set; comparing the first label and the second label with an estimation process of estimating a second label that is estimated to be associated with the training image by image recognition of the training image of the dataset; a comparing step, a displaying step of displaying the learning image having the different first label and the second label based on the comparison result, and the image shown in the displayed learning image being the and an input process of inputting input information indicating whether or not the label is different from the first label associated with the learning image.

A program according to an aspect of the present invention is configured to provide a computer with a step of acquiring a learning data set in which a learning image and a first label associated with the learning image are a set; estimating a second label estimated to be associated with the learning image by image recognition of the learning image of the learning data set; and comparing the first label and the second label. a step of displaying the learning image having the first label and the second label different from each other based on the comparison result; a step of inputting input information indicating whether the label is different from the first label associated with the image.

Advantageous Effects of Invention According to the present invention, it is possible to support the task of detecting handwriting errors in handwritten characters.

1 is a block diagram showing a schematic configuration of an examination support device according to an embodiment of the present invention; FIG. FIG. 4 is a diagram for explaining a procedure for the inspection support apparatus according to the embodiment of the present invention to cut out handwritten character images in units of characters from an image of a manuscript sheet; 4 is a schematic diagram showing the data structure of an acquisition label table stored by the inspection support device according to the embodiment of the present invention; FIG. 4 is a schematic diagram showing the data structure of an estimated label table stored by the examination support device according to the embodiment of the present invention; FIG. FIG. 4 is an image diagram showing an example of a visual inspection screen displayed by the inspection support device according to the embodiment of the present invention; FIG. 5 is an image diagram showing another example of a visual inspection screen displayed by the inspection support device according to the embodiment of the present invention; FIG. 5 is an image diagram showing another example of a visual inspection screen displayed by the inspection support device according to the embodiment of the present invention; It is a flowchart which shows the procedure of the test|inspection assistance process which the test|inspection assistance apparatus by embodiment of this invention performs.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a schematic configuration of an examination support apparatus 1 according to this embodiment.
The examination support apparatus 1 (support system) includes a data input unit 101, an acquisition unit 102, an acquired label storage unit 103, an estimation unit 104, a discriminator 105, an estimated label storage unit 106, a comparison unit 107, a selection data storage unit 108, a display A section 109, a correct/incorrect information input section 110, and an error data storage section 111 are provided.

The data input unit 101 inputs an image captured by a camera, scanner, or the like of a manuscript sheet in which a writing position is specified for each type of character (hereinafter referred to as “character type”). A plurality of handwritten characters of different character types are written in advance on each designated position on the manuscript paper. It is desirable that the characters are imaged under such conditions that they can be recognized by the classifier 105 included in the acquisition unit 102 and the estimation unit 104 in the latter stage. Since the image input by the data input unit 101 is data used for deep learning, the data input unit 101 normally inputs images of a plurality of manuscript sheets for the same character type.

The acquiring unit 102 acquires a learning data set in which a learning image and a first label associated with the learning image are set. The learning image is a handwritten character image showing handwritten characters. Also, the first label is information indicating the type of characters shown in the handwritten character image. Specifically, the acquisition unit 102 performs image processing on the image of the document sheet input by the data input unit 101, and cuts out characters in the image on a character-by-character basis. A character image (handwritten character image) cut out in units of one character is a learning image used for deep learning. After that, the acquisition unit 102 identifies the correct character type corresponding to each handwritten character image according to the position in the image of the manuscript paper, and associates it with the data of the extracted handwritten character image (hereinafter referred to as “character image data”). The combination with the character code (first label) representing the correct character type is written in the acquired label storage unit 103 . Means for extracting a handwritten character image in units of characters from the image of the manuscript paper will be described later.

The acquired label storage unit 103 stores an acquired label table. This acquisition label table stores character image data in units of characters and character codes (first labels) representing character types corresponding thereto in association with each other.

The estimating unit 104 estimates a second label that is estimated to be associated with the learning image by performing image recognition on the learning image of the learning data set acquired by the acquiring unit 102 . For example, the estimator 104 includes the classifier 105 . The discriminator 105 is a previously learned CNN (Convolutional Neural Network) that estimates the probability of the character type of the characters written in the image when a handwritten character image is input. The estimating unit 104 inputs the handwritten character image cut out by the acquiring unit 102 to the classifier 105, and determines the character type output as the character type with the highest probability from the estimation result for the input handwritten character image as the handwritten character. The character type is assumed to be associated with the image. Then, the estimation unit 104 writes a combination of the address of the handwritten character image cut out by the acquisition unit 102 and the character code (second label) representing the estimated character type into the estimated label storage unit 106 .

The estimated label storage unit 106 stores an estimated label table. The estimated label table contains a character image data address indicating the head address of an area where character image data for each character is stored, and a corresponding character code (second label) indicating the character type estimated by the estimation unit 104. They are stored in association with each other. The character image data address indicates the address of the area of the obtained label storage unit 103 in which each character image data is stored, and serves as an index when reading the character image data from the estimated label storage unit 106 .

A comparison unit 107 compares the first label and the second label. Specifically, the comparison unit 107 reads character codes corresponding to the same handwritten character image from the obtained label storage unit 103 and the estimated label storage unit 106 and compares them. Then, the comparison unit 107 selects handwritten character images with different character codes as objects for visual inspection, combines the addresses with the character codes (first label) read out from the acquired label storage unit 103, and selects data storage unit 108. write to

The selection data storage unit 108 stores a selection data table. The selection data table includes a character image data address indicating the head address of an area storing character image data for each character to be visually inspected and selected by the comparison unit 107, and the corresponding character image data address obtained by the obtaining unit 102. A character code (first label) representing the character type is associated with each other and stored. The character image data address indicates the address of the area of the acquisition label storage unit 103 in which each character image data is stored, and serves as an index when reading the character image data from the selection data storage unit 108 .

The display unit 109 displays learning images with different first labels and second labels based on the comparison result by the comparison unit 107 . At this time, the display unit 109 collectively displays learning images with different first labels and second labels in tiles for each type of first label. Specifically, the display unit 109 arranges all the handwritten character images having the same character code among the handwritten character images stored in the selection data storage unit 108 into tiles and displays them on a display or the like. At this time, the display unit 109 displays the reference image indicated by the first label together with the learning image. For example, the display unit 109 may display a correct character image (reference image) generated from two or more different digital fonts for comparison, together with the handwritten character image. In addition, the display unit 109 may display characters represented by the correct character code in the title bar.

The correct/wrong information input unit 110 inputs whether or not the handwritten character image shown in the learning image displayed by the display unit 109 is different from the first label associated with the handwritten character image. Information is entered. Specifically, the correct/wrong information input unit 110 indicates whether or not the handwritten character image to be visually inspected displayed by the display unit 109 is a erroneously written character (differs from the first label). Accepts input of input information. Then, the correct/incorrect information input unit 110 writes the combination of the address of the erroneously written character and the corresponding character code (first label) into the error data storage unit 111 . For example, the inspector visually inspects the group of handwritten character images displayed by the display unit 109 by comparing them with correct character images, and determines that the handwritten characters are incorrectly written or the characters are extremely deformed and unreadable. A character image is selected as a scribbled character. As a general method, a method of selecting a handwritten character image of the erroneously written character by clicking with a mouse is conceivable. In the case of this method, the correct/wrong information input unit 110 determines that the handwritten character image selected by mouse click is an incorrectly written character, combines the character image data address with the corresponding character code, and stores the error data storage unit 111. Write.

The error data storage unit 111 stores an error data table. The error data table corresponds to the character image data address indicating the head address of the area storing the character image data determined as the erroneous character by the correct/wrong information input unit 110 and the corresponding character code (first label). memorize it. The character image data address indicates the address of the area of the acquisition label storage unit 103 in which each character image data is stored, and serves as an index when reading the character image data from the error data storage unit 111 .

Next, with reference to FIG. 2, the means by which the acquisition unit 102 cuts out handwritten character images in units of characters from the image of the manuscript paper will be described. FIG. 2 is a diagram for explaining a procedure for the examination support apparatus 1 according to the present embodiment to cut out a handwritten character image in units of characters from an image of a manuscript sheet.
The manuscript paper in the present embodiment is a graph paper having a frame (square) that can divide each character. Since the squares of the manuscript paper are squares arranged at equal intervals, if the coordinates of the four corners of the outer frame rectangle are known, the relative positions of all the squares and the characters contained in the squares can be calculated, and the handwritten character image can be obtained. can be cut out. However, since the position of the outer frame of the scanned manuscript paper is not constant due to inclination or misalignment during scanning, the acquisition unit 102 detects the coordinates of the four corners of the rectangular outer frame according to the following procedure, Crop the image.

Acquisition unit 102 binarizes an image of a document sheet (document sheet character image 201 ) input from data input unit 101 and performs black-and-white reversal to obtain a binarized/black-and-white reversal result image 202 . After that, the acquiring unit 102 performs contour extraction from the binarized/black-and-white inverted result image 202, and detects a contour with the widest area, such as reference numeral 2031 of the outer frame detection result image 203, as the outer frame.

A non-straight frame line enlarged view 204 is an enlarged view of a region 2032 in the upper right corner of the outer frame detection result image 203 . As shown in the non-straight border enlargement 204, image contours are not usually straight due to fine line irregularities. Therefore, the acquisition unit 102 performs linear approximation on the extracted outline of the outer frame, and straightens the outer frame as shown in a linear approximation enlarged frame line diagram 205 . A linear approximation frame line enlarged view 205 is an enlarged view of a region 2032 of the outer frame detection result image 203 after linear approximation is performed on the outer frame contour. As a result, the outer frame outline has four end points, and the coordinates of the four corners of the outer frame rectangle are determined.

Since the detected outer frame quadrilateral is mostly a distorted quadrilateral that is neither a rectangle nor a parallelogram, the acquisition unit 102 performs image conversion so that the rectangles face each other by homography. After that, the acquiring unit 102 acquires a handwritten character image for each character by cutting out squares according to the dimensions of the original paper from the image-converted one-character image frame line detection result image 206 .

Next, an acquired label table stored in the acquired label storage unit 103 will be described with reference to FIG. FIG. 3 is a schematic diagram showing the data structure of the acquisition label table stored in the inspection support apparatus 1 according to this embodiment. The obtained label table stores the character image data of each handwritten character image extracted by the obtaining unit 102 in association with the character code (first label) representing the correct character type of the handwritten character image.

Next, the estimated label table stored in the estimated label storage unit 106 will be described with reference to FIG. FIG. 4 is a schematic diagram showing the data structure of the estimated label table stored by the examination support apparatus 1 according to this embodiment. The estimated label table represents a character image data address indicating the head address where the character image data of each handwritten character image extracted by the acquisition unit 102 is stored, and the character type of the handwritten character image estimated by the estimation unit 104. Character codes (second labels) are associated with each other and stored. The character image data address serves as an index for reading character image data from the acquisition label storage unit 103 .
By storing the address of the region of the acquisition label storage unit 103 in which each character image data is stored in this manner, the storage capacity can be reduced compared to the case where the character image data itself is stored.
Note that the selection data table stored in the selection data storage unit 108 and the error data table stored in the error data storage unit 111 are also similar to the estimated label table shown in FIG. (first label) and stored.

Next, visual inspection screens displayed by the display unit 109 will be described with reference to FIGS. 5 to 7. FIG. FIG. 5 is an image diagram showing an example of a visual inspection screen displayed by the inspection support apparatus 1 according to this embodiment.
The visual inspection screen is a screen displayed on the display DP or the like by the display unit 109 so that the inspector can visually inspect for spelling errors. In the example shown in this figure, the display unit 109 displays a group of handwritten character images (handwritten character images 301-1 to 301-301) having the same character code among the handwritten character images stored in the selection data storage unit 108. -9), a textbook-style correct character image 302 and a gothic-style correct character image 303 are displayed on the visual inspection screen 300 . In this example, the display unit 109 displays a handwritten character image group (handwritten character images 301-1 to 301-9) corresponding to the character code of the character "Kashiwa" and two types of correct character images 302 and 303. it's shown. Here, the display unit 109 reverses the correct character images 302 and 303 in black and white ( Display text with white and background with black. Also, the display unit 109 displays the correct character images 302 and 303 in the center of the visual inspection screen 300 . In addition, the display unit 109 displays the character “Kashiwa” 304 represented by the correct character code in the title bar of the visual inspection screen 300 .

Here, the correct character type is "Kashiwa", while the handwritten character image group (handwritten character images 301-1 to 301-9) is the fourth handwritten character image 301-6 from the left in the middle row and the lower row. In the second handwritten character image 301-8 from the left, "beat" is written, and it can be seen that the handwriting is erroneous. In addition, the third handwritten character image 301-3 from the upper left of the group of handwritten character images (handwritten character images 301-1 to 301-9) is written with the “tree” part of “oak” broken down. Therefore, it is difficult to determine whether it is ``Kashiwa'' or ``Matsu'', so it is selected as a erroneous character.

For example, the inspector inputs the erroneously written character by clicking the mouse on the image of the handwritten character determined to be the erroneously written character on the visual inspection screen. The correct/wrong information input unit 110 determines that a handwritten character image for which a mouse click is received on the visual inspection screen displayed by the display unit 109 is a erroneous character, and converts the character image data address to the corresponding character code (first label) in the error data storage unit 111.

In this manner, the display unit 109 displays only the handwritten character images whose character image data addresses are stored in the selection data storage unit 108 as targets for visual inspection. It eliminates the need for visual inspection of character images. In other words, the inspector only needs to visually inspect the handwritten character images that may contain handwriting errors. Therefore, the burden on the inspector is reduced, and the inspection can be performed efficiently.
As shown in this display example, each character type is grouped into tiles for visual inspection, which saves the trouble of comparing handwritten character images to be inspected one by one with correct character images. inspection can be efficiently performed.
In addition, by displaying a plurality of types (two types in this example) of correct character images, the inspector can confirm the differences due to fonts and typefaces in the same character type. For example, in the case of "Kashiwa", the first stroke of the "white" part is located in the upper left part in the textbook typeface, but in the central part in the Gothic typeface. Therefore, the inspector can compare the two types of correct character images and know that the first stroke of "white" can be either in the upper left or in the center.
In addition, by displaying the correct character image with black and white reversed, the inspector can easily distinguish between the correct character image and the handwritten character image to be inspected.
In addition, by displaying the correct character image in the center and displaying each handwritten character image in a nearly square tile shape, the distance between each handwritten character image to be inspected and the correct character image is approximately the same. When the inspector compares any handwritten character image, the moving distance of the line of sight becomes substantially the same, which facilitates the comparison.

FIG. 6 is an image diagram showing another example of the visual inspection screen displayed by the inspection support apparatus 1 according to this embodiment.
The group of handwritten character images (handwritten character images 301-1 to 301-9) and the correct character images 302 and 303 displayed on the visual inspection screen 300A shown in this figure are the same as those displayed on the visual inspection screen 300 shown in FIG. is identical to that shown in In the visual inspection screen 300A shown in this figure, the correct character images 302 and 303 are displayed after the group of handwritten character images (handwritten character images 301-1 to 301-9) (lower right of the screen). is different from the visual inspection screen 300 . Since other displays are the same as those of the visual inspection screen 300, description thereof will be omitted. The correct character images 302 and 303 may be displayed at any positions on the visual inspection screen, as long as they are visible to the inspector.

FIG. 7 is an image diagram showing another example of the visual inspection screen displayed by the inspection support apparatus 1 according to this embodiment.
The group of handwritten character images (handwritten character images 301-1 to 301-9) and the correct character images 302 and 303 displayed on the visual inspection screen 300B shown in FIG. is identical to that shown in While the visual inspection screen 300 is a horizontally long screen, the visual inspection screen 300B shown in this figure is a vertically long screen. In addition, while the two types of correct character images 302 and 303 displayed on the visual inspection screen 300 are arranged in a horizontal row, the visual inspection screen 300B shown in this figure displays two types of correct character images. 302 and 303 are arranged in a vertical line. The visual inspection screen 300 is not limited to the example shown in this figure, and may be a horizontally long screen or a vertically long screen.

In the display example described above, two types of correct character images are displayed. good too.

Next, the examination support processing by the examination support apparatus 1 will be described with reference to FIG. 8 . FIG. 8 is a flow chart showing the procedures of examination support processing executed by the examination support apparatus 1 according to this embodiment.

(Step S1) The data input unit 101 receives an input of a character image to be inspected for writing errors. A character image is, for example, an image in which a plurality of character types of handwritten characters are written on a manuscript sheet or the like.

(Step S2) The acquiring unit 102 performs image processing on the character image input in step S1, cuts out the handwritten character image for each character, and acquires the first label representing the corresponding character type.

(Step S3) The estimation unit 104 inputs each handwritten character image cut out in step S2 to the classifier 105, and estimates each character type (second label).

(Step S4) The comparison unit 107 selects one handwritten character image cut out in step S2, and compares the corresponding first label obtained in step S2 with the second label estimated in step S3. do.

(Step S5) The comparison unit 107 determines whether the comparison result in step S4 is different between the first label and the second label. If the labels are different (step S5: YES), the process proceeds to step S6. If the labels are the same (step S5: NO), the process proceeds to step S7.

(Step S6) The comparison unit 107 writes and saves the character image data address of the handwritten character image with a different label in the selection data storage unit 108 in association with the corresponding first label acquired in step S2.

(Step S7) The comparison unit 107 determines whether or not the labels of all handwritten character images cut out in step S2 have been compared. If the labels of all handwritten character images have been compared (step S7: YES), the process proceeds to step S8. If the labels of all handwritten character images have not been compared (step S7: NO), the process returns to step S4.

(Step S8) The display unit 109 reads the handwritten character images of the same character type (first label) from the selection data storage unit 108, arranges all the read handwritten character images together with the correct character image in a tile form, and displays them. to display.

(Step S9) The correct/wrong information input unit 110 acquires the selection input of the erroneously written character. For example, the inspector visually inspects the group of handwritten character images displayed in step S8, and if there is an incorrectly written handwritten character image, selects the handwritten character image by a mouse click or the like. The correct/wrong information input unit 110 determines that the handwritten character image selected by mouse click is an incorrectly written character, and writes the character image data address to the error data storage unit 111 in association with the first label. When all the displayed handwritten character images have been visually inspected, the process proceeds to step S10. For example, when the displayed visual inspection screen is closed by the inspector's operation, the display unit 109 determines that the visual inspection has been completed for all displayed handwritten character images.

(Step S10) The display unit 109 determines whether or not all handwritten character images stored in the selection data storage unit 108 have been visually inspected. If there is a handwritten character image that has not been visually inspected (step S10: NO), the process proceeds to step S8 for the character type that has not been visually inspected. If all the handwritten character images stored in the selection data storage unit 108 have been visually inspected (step S10: YES), the process ends.

In this way, by storing the character image data addresses of the handwritten character images with writing errors in the error data storage unit 111, the error data among the handwritten character images stored in the acquired label storage unit 103 can be Images whose character image data addresses are stored in the storage unit 111 can be excluded from deep learning learning images. In other words, only handwritten character images without spelling mistakes can be used as training images for deep learning.

As described above, according to the present embodiment, the inspection support apparatus 1 includes the acquisition unit 102 that acquires a learning data set in which a learning image and a first label associated with the learning image are set; An estimating unit 104 that estimates a second label that is estimated to be associated with the learning image by image recognition of the learning image of the learning data set acquired by the acquiring unit 102, a first label, a first A comparison unit 107 that compares the two labels, a display unit 109 that displays learning images with different first labels and second labels based on the comparison result of the comparison unit 107, and a learning image displayed by the display unit 109. and a correct/incorrect information input unit 110 for inputting input information indicating whether or not the handwritten character image shown in the handwritten image is different from the first label associated with the handwritten character image.

With the above-described configuration, when visually inspecting the learning image for errors in the learning data set, the second label obtained by image recognition is compared with the first label assigned in advance to obtain a label. By selecting only the training images with different values, it is possible to save the trouble of inspecting correct training images that occupy most of the training data set. Therefore, the inspector does not need to visually inspect all the learning images, and only needs to visually inspect the learning images for which there is a possibility that the characters shown in the handwritten character image are different from the first label. Get better. That is, since the number of learning images to be visually inspected can be reduced, the burden on the inspector is reduced, and the inspection can be performed efficiently. Therefore, according to the present embodiment, it is possible to support and efficiently detect errors from a huge amount of learning data sets.

In addition, the display unit 109 collectively displays learning images with different first labels and second labels in tiles for each type of first label. In this way, the training images to be inspected, which have different first labels and second labels, are collectively displayed in tiles for each of the same labels. Since it can be inspected, efficient inspection becomes possible.

Also, the display unit 109 displays the reference image indicated by the first label together with the learning image. As a result, the inspector can inspect the learning image by comparing it with the reference image on the display screen, so that the work can be made more efficient.

The learning image is a handwritten character image showing handwritten characters, and the first label is information indicating the type of the character shown in the handwritten character image. Therefore, according to the present embodiment, it is possible to support the task of detecting writing errors in a large number of handwritten characters. That is, it is possible to perform an efficient visual inspection with few oversights on the handwritten character image.

In addition, the acquiring unit 102 acquires a handwritten character image for each character by binarizing and black-and-white inverting an image of a manuscript paper on which a plurality of handwritten characters of different character types are written, and obtains a handwritten character image for each character from a position on the manuscript paper. Identify the character type of each handwritten character image. As a result, if shadows are reflected in the image of the scanned manuscript paper, they can be removed and the straight lines of the manuscript paper can be extracted with high accuracy. can be obtained accurately.

All or part of the examination support apparatus 1 in the above-described embodiment may be realized by a computer. In that case, a program for realizing this function may be recorded in a computer-readable recording medium, and the program recorded in this recording medium may be read into a computer system and executed. It should be noted that the "computer system" referred to here includes hardware such as an OS and peripheral devices. The term "computer-readable recording medium" refers to portable media such as flexible discs, magneto-optical discs, ROMs and CD-ROMs, and storage devices such as hard discs incorporated in computer systems. Furthermore, "computer-readable recording medium" means a medium that dynamically retains a program for a short period of time, like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. It may also include something that holds the program for a certain period of time, such as a volatile memory inside a computer system that serves as a server or client in that case. Further, the program may be for realizing a part of the functions described above, or may be capable of realizing the functions described above in combination with a program already recorded in the computer system. It may be implemented using a programmable logic device such as FPGA.

Although the embodiment of the present invention has been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and design and the like are included within the scope of the gist of the present invention.

For example, in the above-described embodiment, the examination support apparatus 1 acquires a handwritten character image from a document sheet on which handwritten characters are written. Handwritten character images may be obtained by other methods such as handwriting input of each character with a touch pen.

Further, in the above-described embodiment, an example of a handwritten character image as a learning image has been described, but the learning image is not limited to this. Any image can be targeted, such as weather images (labeled as sunny, cloudy, rainy, etc.), or images of other objects. For example, in the case where an animal image obtained by imaging an animal is labeled with the type of the animal, the examination support apparatus 1 detects that an image of "bear" is included in the training images whose first label is "cat". At certain times, it may be displayed as an image to be inspected.

1... Inspection support device (support system)
101 ... data input unit (input unit)
DESCRIPTION OF SYMBOLS 102... Acquisition part 103... Acquisition label storage part 104... Estimation part 105... Discriminator 106... Estimated label storage part 107... Comparing part 108... Selection data storage part 109... Display part 110... True/False information input part 111... Error data storage part 201... Document paper character image 202... Binary/black and white inverted result image 203... Outer frame detection result image 204... Non-linear frame line enlarged view 205... Linear approximation frame line enlarged view 206... Single character image frame line detection result image 300, 300A, 300B... Visual inspection screen 301-1 to 301-9... Handwritten character image 302... Correct character image 303... Correct character image DP... Display

Claims (8)

an acquisition unit that acquires a learning data set in which a learning image and a first label associated with the learning image are a set;
an estimating unit that estimates a second label that is estimated to be associated with the learning image by image recognition of the learning image of the learning data set acquired by the acquiring unit;
a comparison unit that compares the first label and the second label;
a display unit that displays the learning image with the first label and the second label that are different from each other based on the comparison result by the comparison unit;
an input unit for inputting input information indicating whether or not the image shown in the learning image displayed by the display unit is different from the first label associated with the learning image;
Support system with The display unit collectively displays the learning images with different first labels and second labels for each type of the first label.
The assistance system of claim 1. The display unit displays the learning images with different first labels and second labels in tiles for each type of the first label.
3. A support system according to claim 2. The display unit displays a reference image indicated by the first label together with the learning image.
4. A support system according to any one of claims 1 to 3. The learning image is a handwritten character image showing handwritten characters,
The first label is information indicating the type of characters shown in the handwritten character image.
5. A support system according to any one of claims 1 to 4. The acquisition unit acquires a handwritten character image for each character by binarizing and black-and-white inverting an image of a manuscript paper on which a plurality of handwritten characters of different character types are written, and obtains a handwritten character image for each character from a position on the manuscript paper. Identifying the type of characters in a handwritten character image,
6. A support system according to claim 5. an acquisition process of acquiring a learning data set in which a learning image and a first label associated with the learning image are a set;
an estimation process of estimating a second label estimated to be associated with the learning image by image recognition of the learning image of the acquired learning data set;
a comparing step of comparing the first label and the second label;
a display step of displaying the learning image having the different first label and the second label based on the comparison result in the comparison step;
an input step of inputting input information indicating whether or not the image shown in the displayed learning image is different from the first label associated with the learning image;
Assistance methods including; to the computer,
obtaining a learning data set in which a learning image and a first label associated with the learning image are a set;
estimating a second label estimated to be associated with the learning image by image recognition of the learning image of the acquired learning data set;
comparing the first label and the second label;
a step of displaying the learning image with the first label and the second label different based on the comparison result in the comparing step;
a step of inputting input information indicating whether or not the image shown in the displayed learning image is different from the first label associated with the learning image;
program to run the

Images