JPH05303619A - Electronic scrap book - Google Patents

Abstract

PURPOSE:To enable edition and reuse of data by electrically fetching character strings, figures and photographs on paper and recording one part or all of it. CONSTITUTION:The image data electrically fetched by an input means 20 is divided into areas for character strings, figures and photographs by means of an area division means 21. The area of the character strings is converted into a code by a character recognition means 22, and a correction means 23 corrects the erroneously recognized code. An edition means 24 reuses and edits one part of or all data, and a preservation means 25 records a code of one part of the character string and records image data on the other part.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention captures a document image including characters, figures, and photographs as digitized image data, performs character recognition processing on character areas, and binarizes area on figures and photographs. The present invention relates to an electronic scrapbook in which a database is created and displayed by editing after doing.

[0002]

2. Description of the Related Art Conventionally, there is a scrapbook in which necessary portions of one or a plurality of printed documents are cut off with scissors and the like and attached with glue or the like. In addition, an optical filing system has been developed that records handwritten or printed documents as electronic image data.

[0003]

By the way, in the conventional scrapbook, it takes a lot of time to cut and paste, and it is also possible to change a part of the text of the scraped data or cut and paste the same data as another scrap data again. could not.

Further, in the optical filing system, the text of the image data to be edited cannot be changed.

Therefore, according to the present invention, by electronically processing the image data, the labor of scrap is reduced and the same data can be used a plurality of times.
Further, the data in the character area enables character data to be changed by encoding the character by character recognition processing.

[0006]

Image input means for inputting image data including handwritten or printed characters / drawings / photographs, and image data input by the image input means for character area, figure area and photograph area Area separating means for separating the area into two areas, and character recognizing means for recognizing the image data separated into character areas by the area separating means as a specific character,
It is provided with a correction means for correcting the character erroneously recognized by the recognition means, and an editing means for editing part or all of the character data corrected by the correction means and the image data of figures and photographs. Electronic scrapbook to do.

[0007]

【Example】

(Embodiment 1) An embodiment of the present invention will be described below with reference to the accompanying drawings. FIG. 1 is a block configuration diagram of an electronic scrapbook of the present invention. CPU 10 for controlling each process
And an image input module 20 which is an image input means for converting a document image including characters, figures, and photographs into image data for each pixel and inputting the same, and an image input such as an image scanner operating under the control of the image input module 20. The device 50, the area separation module 21 which is an area separation means for separating an image into a character area, a photograph area and a drawing area, and a character recognition module 22 which is a character recognition means for recognizing image data as a specific character A correction module 23 that is a correction unit that corrects a recognized character, an editing module 24 that edits a character string, a storage module 25 that stores data, and a RAM 30 that is temporarily used when each processing module operates. A communication device 40 for exchanging data with an external device, and information and processing required for each processing module to operate. An external storage device 51 such as a hard disk, a magneto-optical disk, a floppy disk, a magnetic tape, an IC memory card for recording the completed data, and a tablet device 52 (described later) for displaying image data or character data after recognition and also for inputting instructions. Is equipped with. Figure 2
FIG. 3 is a schematic diagram of a tablet device 52. Similar to the CRT device, the image data is displayed on the display / instruction input screen 5201 by the electric signal from the tablet input terminal 5202 (FIG. 2A). In addition, a dedicated electronic pen 52
When the display / instruction input screen is traced at 04, the locus of the electronic pen is converted into an electric signal and externally output from the output terminal 5203 (FIG. 2B).

Next, the processing contents of this embodiment will be described with reference to FIG. The processing of the electronic scrapbook according to the present embodiment is performed by an image data input processing 2000 in which image data to be processed by the image input module is captured.
An area separation processing 2100 for separating the captured image data by the area separation module into a character area, a photograph area, and a drawing area, and a character recognition processing 2200 for image data by a character recognition module that performs character segmentation and character recognition of the character area. A correction processing 2300 of data erroneously recognized by the character recognition by the correction module, and an editing processing 2430 by the editing module for changing the arrangement of the corrected character data and changing the character and the arrangement of the photograph area and the drawing area.
00, and a saving process 250 by a saving module that saves image data and edited data after area separation
It consists of zero.

First, the image data input process 2000 will be described. The image data input process is performed by reading by the operator using the image input device 50. Specifically, an operator activates the image input module 20 under the control of the CPU 10 and reads a printed matter such as a handwritten document or a magazine as binary image data by using the image input device 50.

Next, the area separation processing 2100 will be described with reference to FIG.
Will be described with reference to FIG. FIG. 4 is a schematic diagram showing how the regions are separated. The area separation module is shown in FIG.
Image data is divided into a character area, a drawing area, and a photograph area as shown in FIG. When the area separation is performed, one image data is separated into a plurality of image data such as characters, drawings, and photographs, and an area separation flag indicating which area is added to each image data. In addition,
After the region separation is completed, if each region of the character string / drawing / photo is surrounded by lines of different colors and types, it can be seen at a glance whether the region separation is completed correctly.

The area separation processing is performed by the CPU 10 shifting the processing from the input module to the area separation module 21 after inputting the image data. Area separation is C
Under the control of the PU, the operator writes the area division boundary on the display / instruction input screen 5201 of the tablet device 52 with the electronic pen 5204, thereby transmitting the area division information to the area separation module. FIG. 5 is an operation example of the area separation processing. Image data is displayed on the display / instruction input screen 5201 (FIG. 5A), a character area is surrounded by the electronic pen 5204 (FIG. 5B), and the surrounded area is a character string using the electronic pen 5204. It is menu 210
Select from 1 (FIG. 5 (c)). If you have enclosed a picture or photo, select the picture or photo from the menu.

Next, the character recognition process 2200 will be described. In the character recognition process, the character recognition module 22 is activated under the control of the CPU 10, the character string is cut out for each character, and then the feature amount is extracted for each character to extract the external storage device 5.
This is performed by comparing the feature amount recorded in 1 and the character having the most similar feature amount. As a method of character recognition, a pattern matching method can be used in which an input pattern is compared with a reference pattern recorded in a storage device by observing the entire pattern, and a recognized character is determined based on the similarity between the input pattern and the reference pattern. Alternatively, it can be realized by using a structural analysis method or the like in which the local observation of the pattern and the determination based on the observation are accumulated and the whole is determined from the part. Any pattern observing means may be used.

In the character recognition, the character having the most similar feature amount to the dictionary recorded in the storage device is the first recognition result, the second character having the most similar feature amount is the second recognition result, and so on. 3rd place ... The recognition result of the nth place. The recognized character is encoded into, for example, a JIS code and recorded in the external storage device 51 as a recognition result. When the character recognition is completed, the first-ranked recognition result is displayed on the display / instruction input screen 5201.

Next, the correction process 2300 will be described with reference to FIG. The correction process is performed by activating the correction module 23 under the control of the CPU 10 and inputting a correction instruction after confirming the recognition result of the character recognition process 2300 by the operator. In the correction process, the recognition result after the character recognition process is displayed on the display / instruction input screen 5201 (2301), and the operator confirms whether all the displayed character strings are correctly recognized. Then, the character which is erroneously recognized is found, and the display / instruction input screen 5201 is displayed by the electronic pen 5204.
When pointing to the above erroneously recognized character (2302), the recognition result of the second, third, ..., Nth position during character recognition is displayed (2303). The operator confirms whether the correct character is present in the recognition result displayed (230).
4). If the correct character exists, the character is selected with the electronic pen (2305) and the character is corrected to the target character (2312). When the recognition result up to the nth position does not include the correct character, when "handwriting input" is selected (2306), a handwriting character input screen is displayed (230).
In step 7), a character is written with the electronic pen and the character is recognized (2308). The operator confirms whether or not the input is correct (2309), and when the input is correct, "decide" is selected (2310) and the characters are replaced, and the processing ends (2312). When the recognition result is not the target character, "Clear" is selected and the handwriting input is performed again (2311).

FIG. 7 shows a display example during the actual correction process.
FIG. 7 shows the display / instruction input screen 5201 when the correction process is being performed. FIG. 7A shows the processing result of the character recognition processing. In FIG. 7A, “shi” is “ma”.
7B, the electronic pen 5204 incorrectly recognizes “M” as shown in FIG. 7B.
The menu opens as shown in (c). This menu has 2
The recognition results from the first to the nth position and "handwriting input" are displayed. When correct characters are included in the menu as shown in FIG. 7C, the character or the number at which the character is displayed is designated by the electronic pen, and then FIG.
The characters are erroneously recognized as shown, and the correction ends. If the menu shown in FIG. 7C does not include the correct character, the handwriting recognition menu opens as shown in FIG. 7E when "handwriting input" is instructed with the electronic pen. Since there is a frame for handwritten characters in this menu,
Use the electronic pen to write the correct characters in this frame, and "recognize"
The character in the frame written by instructing is recognized. When the characters can be correctly recognized, if the "decision" is instructed with the electronic pen, the correction process is finished as shown in FIG. 7D. If the characters are mistakenly recognized, select "Clear" and perform handwriting again.

Kazuhiko Yamamoto: "Handwriting Educational Kanji Recognition by Relaxation Matching Method", which uses relaxation method matching that seeks a rough solution and then corrects it to gradually obtain a true solution for handwritten character recognition. Argument theory (D), vol. J65-
D, pp. 351-358 (1982), or DP matching originally developed for speech recognition (Hiroaki Sakoe, Narumi Chiba: "Continuous word recognition based on time normalization using dynamic programming", ASJ Magazine, vol.
27, pp. 483-490 (1971)) applied to character recognition or neural network applied Katsuji Imai, Kazutoshi Gohara, Yoshiki Uchikawa: "Handwritten characters for noisy original images using 3-layer BP model" Cognition ", Theological Theory (D-II), vol. J75-D-I
I, pp. 314-323 (1992) and the like, but any method may be used.

In order to correctly recognize the corrected character input by handwriting, the type of character to be recognized may be reduced by limiting the input to alphabet or katakana. At this time, a desired kanji can be obtained by using a method such as hiragana-kanji conversion currently used in a Japanese word processor. Further, in the character recognition process, when n = 1 using only the first-ranked recognition result, the correction method is always handwritten character recognition.

Although the handwritten character recognition is used for the erroneous recognition correction process, the keyboard may be displayed on the tablet and the correction character may be selected with the electronic pen. At this time, since it is not possible to display all kanji on the tablet, input is limited to alphabets and katakana, and kana is obtained using kana-kanji conversion.

Next, the editing process 2400 will be described with reference to FIG. In the editing process, the arrangement of the character string / drawing / photo and the character string separated in the area separation process are changed. First, the change of arrangement will be described. Figure 8 (a)
Is a display / instruction input screen 5201 of the tablet device 52.
It is an example of the recognized characters and pictures displayed in. The line surrounding each area is a boundary line created by the area separation processing. By pointing on the line surrounding the area to be changed with the electronic pen 5204, the area becomes movable. Therefore, the line surrounding the area is moved with the electronic pen and the area is moved to the target location (FIG. 8).
(B)). If the two areas overlap as a result of the movement,
The last moved area is displayed with priority. Further, the same thing is performed for another area (FIG. 8C). Next, changing the character string will be described. When you want to change the character string, specify the character part with the electronic pen and erase it,
By writing a character on the tablet with an electronic pen and recognizing the character in the same manner as the correction means, the character can be erased or newly added. In this way, since character recognition is performed and characters are encoded, it is possible to change character data and create a new sentence. When the editing process is completed, the edited data is recorded in the external storage device 51.

Next, the saving process 2500 will be described. In the saving process, the image data after the area separation and the data edited in the editing process are saved in the external storage device 51. In the storage processing after the area separation, when the operator instructs the storage processing, the image data of the character area, the photograph area, and the drawing area is stored. In the storage after the editing is completed, the data in the character area stores the data encoded by the character recognition processing and the correction processing, and the photograph area and the drawing area store the image data. The image data may be MR encoding, MMR encoding, which is a facsimile encoding method, or
Data compressed using entropy coding may be stored. Further, since the data after the region separation is less necessary than the data after the editing is completed, when saving the data after the region separation, the data is saved by the operator's instruction.

Next, an external view of the electronic scrapbook of the present invention will be described with reference to FIG. The scanner 61 corresponds to the image input device 50. The printer output terminal 62, the image input terminal 63, and the communication terminal 64 operate under the control of the communication device 40. The printer output terminal 61 is connected to a printing device such as a printer to print the edited result on paper. Image input terminal 62
Can be connected to an image scanner or the like to read large image data or the like that cannot be read by the image scanner 61 equipped with an electronic scrapbook, and can also be connected to an external storage device storing the image data to read the image data. Also do. The communication terminal 64 is connected to a computer or a word processor to transfer data.

In the present embodiment, the editing process has been described as editing within one image data, but if a plurality of image data are read and displayed on the tablet device and the displayed image data are moved, a plurality of image data can be displayed. You can freely edit the information of the image data of. At this time, if the processed data is read from the external storage device 50, the past data can be edited. It is also possible to read the processed data from another device using the communication terminal 64 and edit the data. Further, the data recorded in the electronic scrapbook of the present invention can be output to a computer or a word processor using the communication terminal 64 and used as a part of the data created by the operator.

In the present embodiment, each process was performed using a dedicated module, but in addition to using the ROM to perform each process, a RAM for the process is prepared and the operation content is transferred from the external storage device to the RAM. It may be read and processed.

(Embodiment 2) A case in which area separation processing is automatically performed in Embodiment 1 will be described with reference to FIG. In the area separation processing, after inputting the image data, C
The PU 10 switches the input module to the area separation module 21.
It is performed by transferring the process to. As means for region separation, for example, Teruo Akiyama, “Region Extraction from Document Image Using Marginal Distribution, Linear Density, and Circumscribing Rectangle Feature”, NTT Research Practical Report, Vol. 37, No. 8, pp. 409-4
16 (1988) and the like can be used. Figure 10
FIG. 6 is a flow chart of this automatic area separation method. Generate a one-character circumscribed rectangle of the input image data (210
3), a) If the circumscribing rectangle of the same size has a high appearance frequency and the circumscribing rectangle is small (2104), it is determined to be a character string (2105).

If it is not judged as a character string, the line density in one circumscribed rectangle is measured (2106), and b) if the aspect ratio of the circumscribed rectangle is abnormal and the line density is small (2107). , A ruled line (2108) c) If the circumscribed rectangle is large and the line density is large, (210
9, 2110), which is a chart (2111) d) When the circumscribed rectangle is large and the linear density is small, (210
9, 2110) and the heading (2112) are repeated to separate the image data areas.

The area separation processing may be prepared in two modes of automatic area separation processing and manual area separation processing so that the operator can select the mode.

(Example 3) In Example 1 or 2,
A case where a CRT, a keyboard and a mouse are used instead of the tablet device will be described with reference to FIG.

In FIG. 10, the tablet device 52 of the block diagram of FIG. 1 is replaced with a CRT device 53 and an instruction input device 54.
It has been replaced with (keyboard, mouse). The tablet device 52 is displayed by the CRT device 53. The operation performed by the electronic pen in the first embodiment is performed by using the mouse or the keyboard. Menu 2201, 23
The selection of 012302 is valid when the mouse is clicked.

The correction process is not limited to handwriting character recognition.
It also enables typing input from the keyboard. At this time, if you want to obtain kanji, you can use kana-kanji conversion. In the editing process as well, the character to be changed can be typed from the keyboard as well as the handwritten character recognition.

[0030]

As described above, according to the present invention, since the image data of the character area is handled as the character code, the data of the character area can be changed and a part of the character area can be pasted to another sentence. is there. Also, by extracting a keyword from the encoded character data, a database of edited image data can be automatically created, and the image data can be searched using this keyword. Since the data in the character area is encoded, the data can be sent directly to the translator. At this time, the sentence translated into another language can be taken into the electronic scrapbook of the present invention again, and the data of the drawing and the photograph can be combined to create data in which only the sentence is changed to another language.

Since only the code is recorded in the character area,
The amount of data can be reduced compared to when all data is recorded as image data. Therefore, the external storage device can be downsized and data can be transferred at high speed.

[Brief description of drawings]

FIG. 1 is a block diagram of the present embodiment.

FIG. 2 is a schematic diagram of a tablet.

FIG. 3 is a flowchart of processing of this embodiment.

FIG. 4 is an explanatory diagram of region separation processing.

FIG. 5 is an explanatory diagram of manual area separation processing.

FIG. 6 is a flowchart of correction processing.

FIG. 7 is an explanatory diagram of correction processing.

FIG. 8 is an explanatory diagram of editing processing.

FIG. 9 is an external view of an electronic scrapbook.

FIG. 10 is a flowchart of automatic area separation processing.

FIG. 11 is a block diagram of an electronic scrapbook.

[Explanation of symbols]

 10 CPU 20 Image Input Module 21 Area Separation Module 22 Character Recognition Module 23 Correction Module 24 Editing Module 25 Storage Module 30 RAM Device 40 Communication Device 50 Image Input Device 51 External Storage Device 52 Tablet Device 53 CRT Device 54 Instruction Input Device (Keyboard, Mouse 60) Electronic scrapbook 61 Image scanner 62 Printer output terminal 63 Image input terminal 64 Communication terminal 65 Printer 66 Image scanner 67 Computer word processor, etc. 2000 Image input processing 2100 Area separation processing 2101 Area separation menu 2200 Character recognition processing 2300 Correction processing 2313 Correction processing menu 2314 Handwriting input menu 2400 Editing processing 2500 Saving processing 5201 Display / instruction The Power screen 5202 tablet input terminal 5203 tablet output terminal 5204 electronic pen

Claims (4)

[Claims] 1. An image input unit for converting a document image including characters, drawings, and photographs into image data for each pixel and inputting the image data, and image data input by the image input unit for a character region, a drawing region, and a photograph region. Area separating means for separating areas, character recognizing means for recognizing a character of image data separated into character areas by the area separating means as a specific character, and correcting a character erroneously recognized by the character recognizing means. Correcting means, editing means for editing character data corrected by the correcting means and image data of figures and photographs according to an operator's instruction using a display device and an instruction input device; and data separated into areas by the area separation And an electronic scrapbook comprising a storage unit for storing the data edited by the editing unit. 2. The electronic scrapbook according to claim 1, wherein the area separation means uses a display device and an instruction input device to perform area separation according to an operator's instruction. 3. The electronic scrapbook according to claim 1, wherein the area separation means automatically performs area separation by extracting a feature from the image data. 4. The electronic scrapbook according to claim 1, wherein the correction means includes handwriting correction by an operator for the correction.