JP6844574B2 - Image processing device and image processing method - Google Patents

Description

The present invention relates to an image processing apparatus and an image processing method.

Patent Document 1 discloses an image processing apparatus that utilizes an image recognition technique. Specifically, the image processing apparatus disclosed in Patent Document 1 includes a reading unit, a RAM, and an image processing unit. The reading unit reads the image data of the original. The image processing unit includes a character recognition unit, a font recognition unit, and a file creation unit. The character recognition unit acquires the character code and the position information from each character image. The font recognition unit acquires data indicating a font type and font size data from individual character images. The file creation unit uses the font data stored in the RAM to read the character image of the image data read by the reading unit based on the character code, the character position information, the data indicating the font type, and the font size data. Replace with the expanded text image.

Japanese Unexamined Patent Publication No. 2005-275863

However, the image of the manuscript cannot be processed by the technique disclosed in Patent Document 1. Therefore, the user converts the image data read by the reading unit into text data, and then uses an electronic device such as a personal computer to process the text data. As a result, the convenience of the user may be reduced.

In view of the above problems, an object of the present invention is to provide an image processing device and an image processing method that improve user convenience.

The image processing apparatus according to the present invention includes a data input unit and a control unit. The data input unit acquires data of a symbol image indicating a symbol. The symbol includes a first character string and a specific symbol that specifies a processing target character that is a target of a predetermined processing in the first character string. The control unit controls the operation of the data input unit. The control unit determines whether or not the symbol image includes a specific symbol image indicating the specific symbol. When the control unit determines that the specific symbol image is included in the symbol image, the control unit identifies the processing target character. The control unit generates first processing data in which a predetermined process is executed on the character to be processed. The first processed data includes image data indicating an image of the second character string excluding the processing target character in the first character string. The control unit converts the image of the second character string into text data based on the image data indicating the image of the second character string, and generates the second processed data.

According to the present invention, the convenience of the user is improved.

It is a figure which shows the structure of the image processing apparatus which concerns on Embodiment 1 of this invention. It is a figure which shows an example of the symbol image which concerns on Embodiment 1 of this invention. It is an enlarged figure which shows the specific symbol image which concerns on Embodiment 1 of this invention, and the image which shows the character to be processed. It is a figure which shows an example of the 1st processed image which concerns on Embodiment 1 of this invention. It is a figure which shows an example of the 2nd processed image which concerns on Embodiment 1 of this invention. It is a flowchart which shows the processing process which concerns on Embodiment 1 of this invention. It is another example which shows the structure of the image processing apparatus which concerns on Embodiment 1 of this invention. It is a figure which shows another example of the 1st processed image which concerns on Embodiment 1 of this invention. It is a figure which shows another example of the 2nd processed image which concerns on Embodiment 1 of this invention. It is a figure which shows an example of the symbol image which concerns on Embodiment 2 of this invention. It is a figure which shows an example of the 1st processed image which concerns on Embodiment 2 of this invention. It is a flowchart which shows the processing process which concerns on Embodiment 2 of this invention. It is a figure which shows another example of the specific symbol which concerns on Embodiment 2 of this invention. It is a figure which shows another example of the specific symbol which concerns on Embodiment 2 of this invention.

Hereinafter, embodiments of the image processing apparatus and the image processing method according to the present invention will be described with reference to the drawings. In the drawings, the same or corresponding parts are designated by the same reference numerals and the description is not repeated.

[Embodiment 1]
First, the configuration of the image processing apparatus 1 according to the first embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing a configuration of an image processing device 1 according to the first embodiment.

As shown in FIG. 1, the image processing device 1 includes an operation unit 2, a scanner 3, a storage device 9, and a control unit 10. The scanner 3 is an example of a data input unit.

The operation unit 2 receives an instruction from the user to the image processing device 1. When the operation unit 2 receives an instruction from the user, the operation unit 2 transmits a signal indicating the instruction from the user to the control unit 10. In the present embodiment, the operation unit 2 is an operation panel. The operation panel has, for example, a touch display and a plurality of operation keys. The touch display has a display and a touch sensor. The plurality of operation keys have, for example, a start key.

The scanner 3 reads the image of the original and inputs the data indicating the image of the original to the control unit 10. Hereinafter, the process in which the scanner 3 reads the image of the original document and inputs the data indicating the image of the original document to the control unit 10 will be referred to as “data input process”.

The scanner 3 has a contact glass and a reading mechanism. The reading mechanism includes, for example, a CCD (Charge-Coupled Device) image sensor. The scanning mechanism reads an image of a document placed on a contact glass and generates data indicating the image of the document. The data indicating the image of the original is input to the control unit 10. In the present embodiment, the image of the manuscript includes an image showing a symbol. Hereinafter, the image showing the symbol will be referred to as a “symbol image”.

The storage device 9 stores various types of data. The storage device 9 is composed of a storage device and a semiconductor memory. The storage device is composed of, for example, an HDD (Hard Disk Drive) and / or an SSD (Solid State Drive). The semiconductor memory constitutes, for example, a RAM (Random Access Memory) and a ROM (Read Only Memory).

The control unit 10 is composed of a processor such as a CPU (Central Processing Unit). Further, the control unit 10 includes an integrated circuit for image formation processing. The integrated circuit for image formation processing is configured by, for example, an ASIC (Application Specific Integrated Circuit). The control unit 10 controls the operation of each unit of the image processing device 1 by executing the control program stored in the storage device 9.

The control unit 10 executes the machining process. In the processing, the control unit 10 acquires data indicating a symbol image. Hereinafter, the data indicating the symbol image will be referred to as "symbol image data". The symbol image data is input from the scanner 3. The control unit 10 generates the first processed data and the second processed data based on the symbol image data. Specifically, the control unit 10 determines whether or not the specific symbol image is included in the symbol image. The control unit 10 determines whether or not the specific symbol image GK1 is included in the symbol image G1 by using, for example, an image recognition technique.

When the control unit 10 determines that the symbol image includes the specific symbol image, the control unit 10 identifies the processing target character and generates the first processed data in which the predetermined processing is executed on the processing target character. Further, the control unit 10 generates the second processing data based on the first processing data by utilizing the technique of OCR (optical character recognition). Specifically, the control unit 10 generates second processed data obtained by converting an image of a character string excluding the processing target character into text data. The control unit 10 stores the generated second processing data in the storage device 9.

Next, the symbol image G1 according to the first embodiment will be described with reference to FIGS. 2 and 3. FIG. 2 is a diagram showing an example of the symbol image G1 according to the first embodiment. FIG. 3 is an enlarged view of the specific symbol image GK1 and the image showing the processing target character according to the first embodiment. Note that FIG. 2 shows an image (symbol image) of a symbol written horizontally by hand. Hereinafter, the writing direction SD in the symbol image is defined as the X direction, and the direction orthogonal to the X direction is defined as the Y direction.

As shown in FIG. 2, the symbol image G1 includes a first character string image GT1 and a specific symbol image GK1.

The first character string image GT1 is an image showing the first character string. The first character string image GT1 is composed of a plurality of characters. In the example shown in FIG. 2, the first character string image GT1 (first character string) is composed of the character string "I am interested in purchasing your property at 122 123 Main Stream."

The specific symbol image GK1 is an image showing a specific symbol. In the present embodiment, the specific symbol is a single line extending in the X direction, and overlaps with at least one character among a plurality of characters constituting the first character string image GT1. That is, the specific symbol is a strikethrough. In the example shown in FIG. 2, the specific symbol overlaps with the character string "122".

When the symbol image G1 includes the specific symbol image GK1, the control unit 10 specifies the character to be processed. The processing target character indicates a character string that is the execution target of a predetermined process. The control unit 10 specifies the processing target character according to a predetermined rule. Predetermined rules are associated with specific symbols. In the present embodiment, the strikethrough is associated with a rule that a character that overlaps with a specific symbol is specified as a processing target character. Therefore, the control unit 10 specifies a character that overlaps with the specific symbol as a processing target character.

Specifically, the control unit 10 specifies the character to be processed by acquiring the information indicating the position of the specific symbol image GK1 in the symbol image G1. For example, as shown in FIG. 3, the control unit 10 acquires position information indicating the position P1 of the start point of the specific symbol image GK1 in the symbol image G1 and position information indicating the position P2 of the end point. The control unit 10 extracts a character image in which at least a part is included in the range of the two acquired position information. The control unit 10 identifies the character to be processed by acquiring text data from the extracted character image by using the function of OCR. In the example shown in FIG. 2, the control unit 10 specifies the character string "122" as a character to be processed.

Subsequently, the first processing data according to the first embodiment will be described with reference to FIG. FIG. 4 is a diagram showing an example of the first processed image G2 according to the first embodiment. The first processed image G2 is an image based on the first processed data.

As shown in FIG. 4, the first processed image G2 includes the second character string image GT2. The second character string image GT2 is an image showing the second character string. The second character string image GT2 (second character string) is composed of the character string "I am interested in purchasing your product at 123 Main Stream.". In other words, the second character string is composed of a character string excluding the processing target character "122" from the first character string described with reference to FIG.

Next, the second processing data according to the first embodiment will be described with reference to FIG. FIG. 5 is a diagram showing an example of the second processed image G3 according to the first embodiment. The second processed image G3 shows the second processed data (text data). In the present embodiment, the second processing data is composed only of text data.

As shown in FIG. 5, the second processed image G3 includes the third character string image GT3. The third character string image GT3 is an image showing the third character string.

In the example shown in FIG. 3, the third character string image GT3 (third character string) is composed of the character string "I am interested in purchasing your property at 123 Main Stream." Specifically, the third character string image GT3 is composed of a character string excluding the processing target character string "122" from a plurality of characters constituting the first character string image GT1 shown in FIG. In other words, in the second processing data, the data indicating the processing target character string is deleted.

Subsequently, the processing process according to the first embodiment will be described with reference to FIG. FIG. 6 is a flowchart showing the processing process according to the first embodiment. The processing process is started when the control unit 10 receives a signal from the operation unit 2 instructing the start of the data input process.

As shown in FIG. 6, the control unit 10 causes the scanner 3 to execute the data input process (step S101). Specifically, the control unit 10 acquires the symbol image data by, for example, having the scanner 3 read the symbol image. Next, the control unit 10 determines whether or not the specific symbol image GK1 is included in the symbol image G1 based on the symbol image data (step S102).

When the control unit 10 determines that the specific symbol image GK1 is not included in the symbol image G1 (step S102: No), the control unit 10 ends the processing process.

On the other hand, when the control unit 10 determines that the specific symbol image GK1 is included in the symbol image G1 (step S102: Yes), the control unit 10 specifies the character to be processed (step S103).

When the control unit 10 specifies the character to be processed, the control unit 10 generates the first processing data (step S104). Specifically, the control unit 10 executes a predetermined process on the character to be processed. The predetermined process is stored in the storage device 9 in association with the specific symbol. In the present embodiment, the strikethrough is associated with a deletion process for deleting the character to be processed and the specific symbol. Therefore, the control unit 10 generates the first processed data in which the image showing the character to be processed and the image showing the specific symbol are deleted from the symbol image G1 based on the symbol image data. Next, the control unit 10 generates the second processed data by converting the second character string image GT2 included in the first processed data into text data by using the function of the OCR (step S105). Next, the control unit 10 stores the second machining data in the storage device 9 (step S106), and ends the machining process.

The first embodiment has been described above. For example, when a character image written with a ball pen contains an unnecessary character image, it is necessary to convert the character image into text data using OCR technology and then delete the text data indicating the unnecessary character. May become. However, according to the image processing device 1 of the present embodiment, the second processed data (text data) in which unnecessary characters are deleted is generated only by adding a strikethrough to the character image with a ballpoint pen or the like. Therefore, the user can process the symbol image data in the image processing device 1 without performing an operation for processing the symbol image data. Therefore, the convenience of the user is improved.

In the present embodiment, the configuration in which the data input unit is the scanner 3 has been described as an example, but as shown in FIG. 7, the data input device may be, for example, a communication device 4. FIG. 7 is another example showing the configuration of the image processing device 1 according to the first embodiment. The communication device 4 communicates with an external device 5 on which a communication device using the same communication method is mounted. The communication device 4 inputs the image data received from the external device 5 to the control unit 10. The communication device 4 is, for example, a USB (Universal Serial Bus) adapter. Alternatively, the communication device 4 is a LAN (Local Area Network) adapter. The external device 5 includes, for example, an imaging device such as a camera.

Further, in the present embodiment, the case where the strikethrough is a single line extending in the X direction has been described as an example, but the strikethrough may be, for example, a double line extending in the X direction.

Further, for example, when the symbol image G1 includes a ruled line image GL showing a ruled line as shown in FIG. 8, the control unit 10 generates first processed data in which the ruled line image GL is deleted as shown in FIG. To do. FIG. 8 is a diagram showing another example of the symbol image G1 according to the first embodiment. FIG. 9 is a diagram showing another example of the first processed image G2 according to the first embodiment. Specifically, the first processed image G2 shown in FIG. 9 shows the first processed data generated based on the symbol image G1 shown in FIG.

The control unit 10 determines whether or not the symbol image G1 includes the ruled line image GL before generating the first processed data. The control unit 10 determines whether or not the symbol image G1 includes the ruled line image GL by, for example, determining whether or not the line image extending in the X direction is included in the symbol image G1. Specifically, the control unit 10 determines whether or not there is a line image extending from the start point to the end point of the X coordinate of the symbol image G1 and the number of line images extending from the start point to the end point of the X coordinate includes the threshold value or more. By determining, it is determined whether or not the symbol image G1 includes the ruled line image GL. Alternatively, the control unit 10 determines whether or not the symbol image G1 includes the ruled line image GL by determining whether or not the line images extending in the X direction are lined up at equal intervals along the Y direction. By generating the first processed data in which the ruled line image GL is deleted, erroneous conversion in generating the second processed data is suppressed. Specifically, it is possible to prevent the control unit 10 from erroneously recognizing that the ruled line image GL is a strikethrough. The ruled line image GL is not limited to the line image extending in the X direction. The ruled line image GL may be a line image extending in the Y direction.

[Embodiment 2]
Next, the image processing apparatus 1 according to the second embodiment will be described with reference to FIGS. 10 to 12. The second embodiment is different from the first embodiment in that the specific symbol is composed of a specific character string and a range symbol indicating a range of the specific character string. Hereinafter, the matters different from the first embodiment will be described with respect to the second embodiment, and the description of the parts overlapping with the first embodiment will be omitted.

First, the symbol image G4 according to the second embodiment will be described with reference to FIG. FIG. 10 is a diagram showing an example of the symbol image G4 according to the second embodiment. In the example shown in FIG. 10, the range symbol is a rectangle surrounding a specific character string. In other words, the range symbol is a symbol indicating the position at the beginning of the specific character string and the position at the end of the specific character string.

As shown in FIG. 10, the symbol image G4 includes the fourth character string image GT4. The fourth character string image GT4 is an image showing the fourth character string. The fourth character string image GT4 (fourth character string) is composed of the character string "Hi sam, My name is John. I am interested in purchasing your product at 123 Main Street. To.

Further, the symbol image G4 includes two specific symbol images GK2. Hereinafter, the two specific symbol images GK2 shown in FIG. 10 will be referred to as "first specific symbol image GK21" and "second specific symbol image GK22". The first specific symbol image GK21 is a rectangle surrounding an image showing the first specific character string. The second specific symbol image GK22 is a rectangle surrounding an image showing the second specific character string. In the example shown in FIG. 10, the first specific character string is "logo.jpeg" and the second specific character string is "JOHN_signature.jpg".

Subsequently, the first processed image G5 according to the second embodiment will be described with reference to FIG. FIG. 11 is a diagram showing an example of the first processed image G5 according to the second embodiment.

As shown in FIG. 11, the first processed image G5 includes a fifth character string image GT5, a first converted image GH1, and a second converted image GH2. The fifth character string image GT5 is an image showing the fifth character string.

In the example shown in FIG. 11, the fifth character string image GT5 (fifth character string) is the character string “Hi sam, My name is John. It is composed of "you Regards,".

The first converted image GH1 is arranged at the position where the first specific symbol image GK21 described with reference to FIG. 10 was arranged. In other words, the first specific symbol image GK21 is replaced with the first converted image GH1. Further, the first converted image GH1 is adjusted to have substantially the same size as the size in the X direction and the size in the Y direction of the first specific symbol image GK21.

The second converted image GH2 is arranged at the position where the second specific symbol image GK22 described with reference to FIG. 10 was arranged. In other words, the second converted image GH2 is replaced with the second converted image GH2. Further, the second converted image GH2 is adjusted to have substantially the same size as the Y direction of the second specific symbol image GK22.

FIG. 12 is a flowchart showing the processing process according to the second embodiment. As shown in FIG. 12, when the control unit 10 determines that the symbol image G4 includes the specific symbol image GK2 (step S102: Yes), the control unit 10 determines the type of the specific symbol (step S201). The control unit 10 determines the type of the specific symbol by using, for example, an image recognition technique.

When the control unit 10 determines the type of the specific symbol, the control unit 10 specifies the character to be processed according to the rule associated with the determined type of the specific image (step S103). In the present embodiment, when the specific symbol is composed of the specific character string and the range symbol, a rule is associated with specifying the specific character string included in the range symbol as the processing target character. Therefore, the control unit 10 specifies the specific character string as the processing target character. In the example shown in FIG. 10, the control unit 10 specifies "logo.jpeg" and "JOHN_signature.jpeg" as processing target characters, respectively.

When the control unit 10 specifies the character to be processed, the control unit 10 generates the first processing data (step S104). Specifically, a predetermined process is executed for the character to be processed. In the present embodiment, when the specific symbol is composed of the specific character string and the range symbol, a conversion process for converting the specific symbol image GK2 into a conversion image is associated with the conversion process. Therefore, the control unit 10 generates the first processed data obtained by converting the specific symbol image GK2 into a converted image from the symbol image G4 based on the symbol image data. Specifically, when the control unit 10 specifies the processing target character, the control unit 10 acquires an extracted image obtained by extracting an image indicating the processing target character. The control unit 10 acquires text data indicating a character string to be processed by using the OCR technique based on the extracted image. The control unit 10 acquires the identification information indicated by the processing target character string (text data), and acquires the converted image from the storage device 9 based on the identification information. The converted image is associated with the identification information and stored in advance in the storage device 9.

When the control unit 10 acquires the converted image from the storage device 9, the control unit 10 generates the first processed data obtained by converting the specific symbol image GK2 into the converted image. Specifically, the control unit 10 arranges the converted image at the same position as the specific symbol image GK2 based on the position information indicating the position of the specific symbol image GK2. Further, the control unit 10 adjusts so that at least one of the sizes of the converted image in the X direction and the Y direction substantially matches the size of at least one of the specific symbol image GK in the X direction and the Y direction.

Next, the control unit 10 generates the second processed data by converting the second character string image GT2 included in the first processed data into text data (step S105). Next, the control unit 10 stores the second machining data in the storage device 9 (step S106), and ends the machining process.

The second embodiment has been described above. According to the present embodiment, the image processing device 1 can convert an image indicating a character to be processed into a converted image. Therefore, the user can acquire the second processed data replaced with the converted image indicated by the identification information only by adding the identification information indicating the converted image to the symbol image. Therefore, the convenience of the user is improved.

Although the case where the specific symbol is a rectangle has been described in the present embodiment, the specific symbol may be, for example, an ellipse.

Further, the specific character string may be enclosed in double mountain brackets “<< logo.jpg >>”, for example, as shown in FIG. As a result, the control unit 10 can acquire the specific character string more clearly. The specific character string may be enclosed in double parentheses "((logo.jpeg))" as shown in FIG. 13 and 14 are diagrams showing other examples of the specific symbols according to the second embodiment.

Further, in the present embodiment, the case where the size of the converted image is adjusted according to the size of the specific symbol image GK2 has been described, but the size of the converted image may be adjusted according to, for example, the font size of the character string. Good. The control unit 10 may adjust the size of the converted image so that the ratio between the size of the converted image in the Y direction and the size of the font in the Y direction becomes a predetermined ratio, for example. The specified ratio may be specified in advance, or may be specified by the user each time the scanner 3 executes the data input process.

The embodiments of the present invention have been described above with reference to the drawings (FIGS. 1 to 14). However, the present invention is not limited to the above-described embodiment, and can be implemented in various embodiments without departing from the gist thereof. Further, the configuration and shape shown in the above-described embodiment are merely examples and are not particularly limited, and various changes can be made without substantially deviating from the effects of the present invention.

For example, in the embodiment of the present invention, the case where the present invention is applied to an image processing device has been described as an example, but the present invention can also be applied to, for example, an image processing device included in an image forming device.

In addition, the items described in the first and second embodiments can be combined as appropriate.

The present invention is useful, for example, in the field of image processing equipment. Image processing devices are also useful in the field of image forming devices.

1 Image processing device 2 Operation unit 3 Scanner 10 Control unit

Claims (5)

A data input unit that acquires symbol image data indicating symbols, and
It is provided with a control unit that controls the operation of the data input unit.
The symbol includes a first character string and a specific symbol that specifies a processing target character that is a predetermined processing target in the first character string.
The control unit
It is determined whether or not the specific symbol image indicating the specific symbol is included in the symbol image, and it is determined.
When it is determined that the specific symbol image is included in the symbol image, the processing target character is specified, and the processing target character is specified.
Further, it is determined whether or not the specific symbol includes a specific character string and a range symbol indicating the range of the specific character string.
The specific symbol, when determined to include the specific character string and said range symbols, to generate a first processed data executing the predetermined processing to the processing target characters,
The first processed data includes image data indicating an image of the second character string of the first character string excluding the character to be processed.
Based on the image data showing the image of the second character string, the image of the second character string is converted into text data to generate the second processed data .
The predetermined process is an image processing apparatus including a conversion process for converting the specific symbol image into a converted image. The image processing apparatus according to claim 1 , wherein the range symbol includes a symbol indicating a position at the beginning of the specific character string and a position at the end of the specific character string. The image processing apparatus according to claim 1 or 2 , wherein the range symbol includes a rectangle surrounding the specific character string. The specific character string indicates information identifying the transformed image, the image processing apparatus according to any one of claims 1 to 3. A data acquisition step for acquiring image data of a symbol image indicating a symbol including a character string, and
A determination step for determining whether or not a specific symbol image indicating a specific symbol is included in the symbol image, and
When it is determined that the specific symbol image is included in the symbol image, the specific character to be processed is specified among the character strings based on the specific image data indicating the specific symbol image. Steps and
A step of further determining whether or not the specific symbol includes a specific character string and a range symbol indicating the range of the specific character string, and
The specific symbol, when determined to include the specific character string and said range symbols, a first generation step of generating a first processed data executing the predetermined processing to the processing target characters,
Including a second generation step of converting the character image into text data and generating the second processed data based on the image data of the character image indicating the character excluding the character to be processed included in the first processed data. See,
The predetermined process is an image processing method including a conversion process for converting the specific symbol image into a converted image.

Images