JP2021166365A - Image processing apparatus and image forming apparatus - Google Patents

Abstract

To create another document that incorporates character strings entered in item entry columns included in an image of a document.SOLUTION: An image forming apparatus 10 comprises: a display 21; an image reading unit 11; an operation unit 22; and a control unit 31. The control unit 31 analyzes a document image obtained through reading of a document performed by the image reading unit 11, acquires the layout of the document image, displays the document image on the display 21, when an item column and an entry column in the document image are designated by a user with a touch operation through a touch panel 23, extracts a character string in the entry column associated with the item, and inserts the extracted character string into an item included in a predetermined document.SELECTED DRAWING: Figure 2

Description

The present invention relates to an image processing apparatus and an image forming apparatus, and more particularly to a technique for extracting a necessary character string from an image on a manuscript.

In some image processing devices, the image of the original is read by the image reading unit, and the image of the original is formed on the recording paper by the image forming unit. Further, the image processing apparatus described in Patent Document 1 accepts the setting of the mark entry area for the image of the original, performs the mark recognition process on the mark entry area, and indicates the result of the mark recognition process. (For example, a rectangular and transparent area having the same size as the mark entry area) is generated, and the generated object is superimposed on the mark entry area and displayed. As a result, the work of confirming the mark entered by the user is reduced, and the work of correction by the operator becomes easy.

Japanese Unexamined Patent Publication No. 2013-0122223

Here, as the content described in the manuscript, the layout is such that the item column in which the item name (for example, name and address) is entered and the entry column in which the specific content corresponding to the item name is entered are arranged side by side. There is something. Many of these manuscripts take a long time because they rely on human labor to recognize and extract the specific contents entered in the entry fields. Therefore, automation of such work is desired.

In Patent Document 1, the mark recognition process is performed on the mark entry area, and the object showing the result of the mark recognition process is superimposed on the mark entry area and displayed, thereby reducing the mark confirmation work. However, it is not possible to create other documents that incorporate the character string entered in the above entry field.

The present invention has been made in view of the above circumstances, and an object of the present invention is to make it possible to create another document incorporating a character string described in an entry field of an item included in an image of a manuscript.

The image processing device according to one aspect of the present invention analyzes the display unit, the operation unit operated by the user, and the manuscript image, and each item included in the manuscript image and each entry associated with each item. The field is detected, the manuscript image is displayed on the display unit, and the character string of the entry field associated with the item included in the manuscript image specified by the operation of the operation unit is extracted and determined in advance. It is provided with a control unit for inserting the extracted character string into the items included in the obtained document.

Further, the image forming apparatus according to one aspect of the present invention includes the image processing apparatus and an image forming unit for forming an image on a recording medium, and the control unit obtains the combined original image. An image is formed by an image forming unit.

According to the present invention, it is possible to create another document incorporating the character string described in the entry field of the item included in the image of the manuscript.

It is sectional drawing which shows the image forming apparatus which is one Embodiment of the image processing apparatus which concerns on this invention. It is a functional block diagram which shows the main internal structure of an image forming apparatus. It is a figure which illustrates the layout of the image described in a manuscript. It is a flowchart which shows the process at the time of executing the character string extraction mode. It is a figure which shows the selection screen for instructing either new or existing layout of a manuscript image. It is a figure which shows the original image displayed on the display part. It is a figure which shows the state which highlighted the item and the entry column in the manuscript image displayed on the display part. It is a figure which shows another selection screen for selecting and instructing either save character string, save layout, and save character string and layout. (A) is a diagram illustrating a CSV file, and (B) is a diagram illustrating an updated CSV file. It is a figure which conceptually shows the arrangement of each item and an entry column constructed based on the information which a layout data shows. It is a figure which shows the manuscript image which has a different layout displayed on the display part.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

FIG. 1 is a cross-sectional view showing an image forming apparatus to which the image processing apparatus according to the embodiment of the present invention is applied. As shown in FIG. 1, the image forming apparatus 10 of the present embodiment is an MFP (multifunction device) having a plurality of functions such as a copy function, a printer function, and a facsimile function. The image forming apparatus 10 includes an image reading unit 11 and an image forming unit 12.

The image reading unit 11 has an image sensor that optically reads the image of the document MS, and the analog output of the image sensor is converted into a digital signal to generate image data indicating the image of the document MS.

The image forming unit 12 forms the image indicated by the image data on the recording paper, and is used for the image forming unit 3M for magenta, the image forming unit 3C for cyan, the image forming unit 3Y for yellow, and the black. The image forming unit 3Bk of the above is provided. In each of the image forming units 3M, 3C, 3Y, and 3Bk, the surface of the photoconductor drum 4 is uniformly charged, the surface of the photoconductor drum 4 is exposed, and an electrostatic latent image is formed on the surface of the photoconductor drum 4. Is formed, the electrostatic latent image on the surface of the photoconductor drum 4 is developed into a toner image, and the toner image on the surface of the photoconductor drum 4 is primarily transferred to the intermediate transfer belt 5. As a result, a color toner image is formed on the intermediate transfer belt 5. The toner image of this color is secondarily transferred to the recording paper P conveyed from the paper feed unit 14 through the transfer path 8 in the nip area N between the intermediate transfer belt 5 and the secondary transfer roller 6.

After that, the recording paper P is heated and pressurized by the fixing device 15, the toner image on the recording paper P is fixed by thermocompression bonding, and the recording paper P is further discharged to the discharge tray 17 through the discharge roller 16.

Next, FIG. 2 is a block diagram showing a main internal configuration of the image forming apparatus 10 of the present embodiment. As shown in FIG. 2, the image forming apparatus 10 of the present embodiment includes an image reading unit 11, an image forming unit 12, a display unit 21, an operation unit 22, a touch panel 23, and a network communication unit (NW communication unit). It includes 25, an image memory 27, a storage unit 28, and a control unit 29. These components are capable of transmitting and receiving data or signals to and from each other via a bus.

The display unit 21 is composed of a liquid crystal display (LCD), an organic EL (OLED: Organic Light-Emitting Diode) display, and the like.

The operation unit 22 includes physical keys such as a numeric keypad, an enter key, and a start key, and receives various instructions based on operations by the user of each key.

A touch panel 23 is arranged on the screen of the display unit 21. The touch panel 23 is a touch panel of a so-called resistive film method or capacitance method, and detects contact (touch) of a user's finger or the like with the touch panel 23 together with the contact position, and a detection signal indicating the coordinates of the contact position. Is output to the control unit 31 or the like described later of the control unit 29. The touch panel 23 serves as an operation unit in which user operations on the screen of the display unit 21 are input together with the operation unit 22.

The image memory 27 stores image data indicating an image of the original MS read by the image reading unit 11.

The storage unit 28 is a large-capacity storage device such as an SSD (Solid State Drive) or an HDD (Hard Disk Drive), and stores various application programs and various data.

The control unit 29 is composed of a processor, a RAM (Random Access Memory), a ROM (Read Only Memory), and the like. The processor is, for example, a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), an MPU (Micro Processing Unit), or the like. The control unit 29 functions as the control unit 31 when the control program stored in the ROM or the storage unit 28 is executed by the processor.

The control unit 29 comprehensively controls the image forming apparatus 10. The control unit 29 is connected to an image reading unit 11, an image forming unit 12, a display unit 21, an operation unit 22, a touch panel 23, a network communication unit 25, an image memory 27, a storage unit 28, and the like, and is a component thereof. It controls the operation of the above and sends and receives signals or data to and from each of the components.

The control unit 31 serves as a processing unit that executes various processes necessary for image formation by the image forming apparatus 10. Further, the control unit 31 receives an operation instruction input by the user based on the detection signal output from the touch panel 23 or the operation of the physical key of the operation unit 22. Further, the control unit 31 has a function of controlling the display operation of the display unit 21 and a function of controlling the communication operation of the network communication unit 25. Further, the control unit 31 processes the image data stored in the image memory 27.

In the image forming apparatus 10 having such a configuration, for example, when the user sets the document MS in the image reading unit 11 and operates the operation unit 22 to input a copy job execution instruction, the control unit 31 operates. According to the execution instruction received by the unit 22, the image reading unit 11 is made to read the image of the original MS. Further, the control unit 31 temporarily stores the image data indicating this image in the image memory 27, causes the image data to be input to the image forming unit 12, and the image forming unit 12 records the image indicated by the image data. To print.

Further, the image forming apparatus 10 includes a character string extraction mode. The control unit 31 sets the image forming apparatus 10 to the character string extraction mode according to the mode setting instruction from the user input to the operation unit 22. When the image forming apparatus 10 is set to the character string extraction mode, the control unit 31 sets the character image, the ruled line image, and the like constituting the original image based on the original image obtained by scanning the original by the image reading unit 11. Based on, each item and its entry field are detected, the character string included in the entry field of the item is extracted, and the extracted character string is inserted into the item included in other documents.

A case where the original image D1 formed on the original MS as shown in FIG. 3 is read by the image reading unit 11 when the character string extraction mode is executed will be described. In the manuscript image D1 shown in FIG. 3, each item IC in which an item name (for example, a name or an address) is entered and each entry field EC in which specific contents corresponding to the item are entered are arranged side by side. It has a layout (document structure). The control unit 31 analyzes the configuration of the manuscript image D1 using a known OCR technique or the like, and determines each item IC and each entry field EC. For example, the control unit 31 uses a character string as a predetermined item name from the manuscript image D1 and a character string obtained by characterizing the manuscript image D1 by OCR technology, for example, "application date", " The item IC determines the "name", "gender", "age", "address", "telephone number", etc., and the area in which the character string as the item name is surrounded by a ruled line. Further, the control unit 31 determines, for example, a rectangular area adjacent to the item IC and surrounded by a ruled line as an entry field EC associated with the item IC. The control unit 31 displays the original image D1 on the screen of the display unit 21.

Subsequently, the control unit 31 associates with each item IC in the document image D1 on the display screen designated by the instruction input via the touch panel 23 by the user touching the display screen of the display unit 21. Each character string described in the position in each entry field EC is extracted. Even when another document having a different layout is read by the image reading unit 11, the control unit 31 similarly fills the entry field EC designated by the user based on the document image obtained by this reading. Extract the described character string.

Next, the processing at the time of executing such a character string extraction mode will be described in detail with reference to the flowchart shown in FIG.

When the user inputs a character string extraction mode setting instruction via the touch panel 23 by operating the GUI (for example, the initial setting screen) displayed on the screen of the display unit 21, the control unit 31 responds to the setting instruction. Then, the character string extraction mode is set (S101). Then, the control unit 31 causes the display unit 21 to display the selection screen G1 as shown in FIG. 5 (S102). On the selection screen G1 shown in FIG. 5, a new key K11 for inputting information indicating that the layout of the manuscript is new, and that the layout of the manuscript is not new but existing. An existing key K12 for inputting the information to be shown is arranged.

When the user touches either the new key K11 or the existing key K12 on the selection screen G1, the control unit 31 controls the document based on the information input corresponding to the touch-operated key. Accepts the selection of whether the layout is new or existing (S103).

Subsequently, the user sets the document MS in the image reading unit 11 and operates the start key of the operation unit 22. The control unit 31 causes the image reading unit 11 to read the document MS according to the character string extraction instruction input by the operation of the start key, and displays the document image obtained by the reading on the screen of the display unit 21 (S104). ). For example, as shown in FIG. 6, the original image D1 is displayed on the screen of the display unit 21. The control unit 31 stores the original image in the image memory 27 (S105).

On the screen of the display unit 21 shown in FIG. 6, the control unit 31 displays the OK key K21 and the end key K22 in addition to the original image D1. If the entire document image D1 cannot be displayed on the screen of the display unit 21, the control unit 31 responds to a swipe operation on the screen of the display unit 21 input via the touch panel 23, and the display unit 21 displays the original image D1. The original image D1 is scrolled and displayed on the screen.

For example, when the control unit 31 accepts the designation of the original layout as a new one in S103 (S106 “new”), the control unit 31 uses a known OCR (Optical Character Recognition) technique or the like to obtain the original image in the image memory 27. The layout is analyzed to detect each item IC and its size and position in the manuscript image, and each entry field EC corresponding to each item IC and its size and position (S107).

Here, it is assumed that the user touch-operates any item IC among the item ICs on the screen of the display unit 21 and then touches the OK key K21. Based on the instructions input to the touch panel 23 based on each of the above touch operations, the control unit 31 fills in the touch-operated item IC and the entry field EC adjacent to the item IC in association with the item IC. As the column EC, it is detected based on the detected size and position, and the detected item IC is highlighted (for example, highlighted in yellow) on the screen of the display unit 21, and the detected entry column EC is highlighted. Display (for example, highlight in blue) (S108). When the control unit 31 detects a plurality of item ICs and the entry field EC associated therewith based on the touch operation, the control unit 31 highlights all the detected item ICs and the entry field EC.

Further, the control unit 31 determines the size and position of each item IC detected as described above, the item name of the selected item IC, and the size of each entry field EC associated with each item IC. Layout data indicating the position is generated (S109).

When the user touches the end key K22 displayed on the display screen shown in FIG. 7, the control unit 31 receives a display instruction on the selection screen for selecting a method for saving data through the touch panel 23. The selection screen G2 as shown in FIG. 8 is displayed on the display unit 21 (S110). On the selection screen G2 shown in FIG. 8, the control unit 31 gives a character string saving instruction to extract and save the character string described in each item IC detected in S108 and each associated entry field EC. Character string save key K31 for input, layout save key K32 for inputting layout data save instruction for saving layout data generated in S109, for inputting both the character string save instruction and the layout data save instruction. Display the character string layout save key K33.

For example, when the user touches the character string saving key K31 on the selection screen G2 shown in FIG. 8, the control unit 31 receives the character string saving instruction through the touch panel 23 (S111 “character string”), and in S109, it is used as layout data. Based on the acquired size and position of each item IC and the size and position of each entry field EC associated therewith, each item IC detected in S108 and each entry field EC associated therewith are determined. Each character string entered in each entry field EC is extracted from the manuscript image D1, and a document containing these character strings, for example, a CSV file (a file in a word processor format or a file in a spreadsheet format) , Etc.) is created, and the CSV file is stored in the storage unit 28 in association with the layout data generated in S109 (S112).

FIG. 9A is a diagram illustrating a CSV file. In the CSV file shown in FIG. 9A, the first line is a character string (item name) indicating each item IC detected in S108, for example, here, "application date", "name", "name", " Gender, age, address, and telephone number are separated by "," and arranged in order. In addition, in the second line, the character string described in each entry field EC associated with each item IC, here, "January 1, 1st year of Reiwa", "○○○ Taro", "Man", "30", "Osaka prefecture △△△", "*****" are separated by "," and arranged in the same arrangement as the corresponding items.

Further, when the user touches the layout save key K32 on the selection screen G2 shown in FIG. 8 and the control unit 31 receives the layout data save instruction for saving the layout data generated in S109 through the touch panel 23 (S111 " Layout "), the control unit 31 stores the layout data generated in S109 in the storage unit 28 (S113).

As described above, the layout data includes the size and position of each item IC selected in S108 in the original image D1, the item name of each selected item IC, and the size and position of each entry field EC selected in S108. However, for example, the arrangement of each item and the entry column configured based on the information indicated by the layout data for the manuscript image D1 is as shown in FIG.

Further, when the user touches the character string layout saving key K33 on the selection screen G2 shown in FIG. 8, the control unit 31 receives both the character string saving instruction and the layout data saving instruction through the touch panel 23 (S111). "Both"). Similar to S112, the control unit 31 creates a CSV file as described above from each character string entered in each item IC and each associated entry field EC, and stores this CSV file in the storage unit 28. The layout data is stored in the storage unit 28 in the same manner as in S113 (S114).

On the other hand, in S106, when the existing key K12 on the selection screen G1 shown in FIG. 5 is touch-operated in S102 (S106 “existing”), the control unit 31 is read by the image reading unit 11 in S104 and stored in the image memory 27. The stored manuscript image D1 is analyzed by a known OCR technique or the like, and the manuscript is existing layout data stored in the storage unit 28 and is based on the layout data specified by the operation of the operation unit 22 by the user. From the image D1, each item IC and each associated entry field EC are detected (S115). Then, the control unit 31 extracts each character string described in each entry field EC, and writes each character string in each corresponding item included in the CSV file associated with the layout data. Then, the CSV file is updated (S116).

As a result, when a plurality of layout data are stored in the storage unit 28, the size and position of each item indicated by any of the layout data A and the size and position of each item that match the size and position of each entry field. And the position and the character string of each entry field associated with each item in the manuscript image of the manuscript MS having the size and position of each entry field are quickly added to the CSV file associated with the layout data A. Can be updated.

For example, as shown in FIG. 9B, in the third line, the control unit 31 has "February 2, 1st year of Reiwa" in each entry field EC in the manuscript image of another manuscript having the same layout as the manuscript image D1. , "○○○ Hanako", "Woman", "27", "Kyoto Prefecture XXX", "*****" are added, and the corresponding items are sequentially arranged and updated. ..

As described above, in the present embodiment, the original image of the original MS is read by the image reading unit 11 in the state where the character string extraction mode is set, and each item IC and each entry field EC of the original image are determined. , The manuscript image is displayed on the screen of the display unit 21, and each character string described in the entry field EC associated with each item IC in the manuscript image specified by the user is extracted in advance. Each of the above-extracted corresponding character strings is inserted into each item included in the CSV file as a defined document.

Further, as another embodiment, it becomes “existing” in S106, and the control unit 31 is associated with each item IC from the manuscript image of the manuscript MS using the existing layout data A stored in the storage unit 28. When detecting each entry field EC, instead of this, the control unit 31 analyzes the manuscript image of the manuscript MS and detects each item, each associated entry field, and a character string of each item. Regardless of the size and position of each item and each entry field, if the character string indicating the detected item matches the character string indicating the item included in the layout data A, the CSV associated with the layout data A As a character string corresponding to the item stored in the file, the character string described in the entry field EC associated with the item IC detected from the manuscript image is inserted into the CSV file and updated. You may do it. In this case, the control unit 31 creates a new line in the CSV file as shown in FIG. 9B, and inserts and updates the character string.

As yet another embodiment, it is also possible to write the character string written for each item IC in the CSV file on a manuscript image having a layout different from the layout indicated by the layout data associated with the CSV file. Is. For example, it is assumed that the original image D2 of a document having a different layout as shown in FIG. 11 is read by the image reading unit 11 and stored in the image memory 27. This manuscript image D2 includes each item IC in which "application date", "name", "gender", "age", and "telephone number" are entered as item names, and each item corresponding to each item. It has a layout in which each entry field EC in which specific contents are entered is arranged side by side.

The control unit 31 analyzes the layout of the manuscript image D2, detects each item IC and each entry field EC, recognizes the character string of the item IC for each item IC, and determines a character string that matches this character string. The item to be possessed is searched from the CSV file stored in the storage unit 28 specified by the operation of the operation unit 22 by the user, the character string for the searched item is read, and the character indicating the read character string is read. A column image is generated, and the character string image is combined in the entry field EC associated with the item IC whose character strings match in the original image D2. The control unit 31 stores the original image D2 in the storage unit 28, or causes the image forming unit 12 to form an image on the recording paper P. As a result, for example, the character string in the entry field EC of the manuscript image D1 can be reprinted on the manuscript image D2 via the CSV file.

In the above embodiment, the original image to be analyzed by the control unit 31 is acquired by reading the original MS by the image reading unit 11, but the document is obtained from the USB memory via the USB interface (not shown). It may be acquired by reading, receiving from storage on the network through the network communication unit 25, or the like.

Further, the configuration and processing of the above-described embodiment described with reference to FIGS. 1 to 11 are merely examples of the present invention, and the present invention is not intended to be limited to the configuration and processing.

10 Image forming device 11 Image reading unit 12 Image forming unit 21 Display unit 22 Operation unit 23 Touch panel 25 Network communication unit 27 Image memory 28 Storage unit 29 Control unit 31 Control unit

Claims (5)

Display and
The operation unit operated by the user and
The manuscript image is analyzed, each item included in the manuscript image and each entry field associated with each item are detected, the manuscript image is displayed on the display unit, and the manuscript image is designated by the operation of the operation unit. An image process including a control unit that extracts a character string in an entry field associated with the item included in the manuscript image and inserts the extracted character string into an item included in a predetermined document. Device. It also has a storage unit to store data.
The control unit detects each item included in the manuscript image and its size and position, and an entry field associated with each item and its size and position by the analysis, and each detected item and its size. The image processing apparatus according to claim 1, wherein layout data including size and position, an entry field associated with each item, and data indicating the size and position is generated and stored in the storage unit. The control unit detects an item indicated by layout data stored in the storage unit and an entry field associated with the item from the manuscript image, and the characters in the entry field associated with the detected item. A column is extracted, and the extracted character string is an item included in the document that is associated with the layout data and stored in the storage unit, and is the same as the character string of the detected item. The image processing apparatus according to claim 2, wherein the image processing device is inserted into the document as a character string in an entry field associated with an item having a character string. The control unit analyzes the manuscript image, detects the item and the entry field associated with the item, recognizes the character string indicating the item, and selects an item having a character string matching the character string. , Search from the document stored in the storage unit specified by the operation of the operation unit by the user, read the character string of the entry field associated with the searched item from the document, and read this. Claim 2 or claim, which generates a character string image indicating the character string, and synthesizes the character string image in an entry field associated with an item in which the character string matches in the manuscript image. The image processing apparatus according to 3. The image processing apparatus according to claim 4 and
The recording medium is provided with an image forming unit for forming an image.
An image forming apparatus in which the control unit forms an image of the synthesized original image by the image forming unit.

Images