CN113196293A

CN113196293A - Apparatus and method for recognizing one or more characters

Info

Publication number: CN113196293A
Application number: CN201880098908.7A
Authority: CN
Inventors: 范宇颖
Original assignee: Fresenius Medical Care Deutschland GmbH
Current assignee: Fresenius Medical Care Deutschland GmbH
Priority date: 2018-10-16
Filing date: 2018-10-16
Publication date: 2021-07-30
Also published as: WO2020077530A1

Abstract

An apparatus for recognizing one or more characters is provided. The apparatus comprises an adjustment unit (12) for adjusting a color level of a first image of an object to generate an adjusted image, the first image of the object being generated from a color image of the object, the color image of the object comprising one or more characters to be recognized; a conversion unit (13) for converting the adjusted image into a binary image; and a character recognition unit (14) for recognizing the one or more characters based on the binary image. By this apparatus, one or more characters can be accurately recognized from an image having a plurality of color information, particularly, background color information.

Description

Apparatus and method for recognizing one or more characters

Technical Field

The present invention relates generally to optical character recognition and, in particular, to recognizing one or more characters from a color image.

Background

In some cases, the parameters are measured and displayed to the user in different foreground and background colors. This is particularly common in the medical field. For example, for a dialysis machine, the parameters may be shown in different colors (e.g., blue and/or green) in areas with a gray background color, and for a bedside monitor, the parameters may be shown in different colors (e.g., green and/or red) in areas with a black background color.

In some applications, especially in medical applications, a user (e.g. a physician or a caregiver) may need to record parameters. However, some medical devices (such as some relatively inexpensive medical devices) may only display measured parameters to a user, but there is no output interface that can output and record the measured parameters electronically. In this case, the user can only record the parameters by handwriting with a pen. Such parameter recording is error-prone and time-consuming when the user needs to monitor and record certain parameter(s) of the patient for a long time.

Optical Character Recognition (OCR) techniques may be used to recognize one or more characters from a display of a medical device, however, due to a variety of color information in the display of the medical device, and in particular background color information in the display of the medical device, the recognition of the characters may be inaccurate.

Disclosure of Invention

Accordingly, it would be desirable to provide a method and apparatus that can automatically identify one or more characters from an image having multiple color information, particularly background color information, to facilitate the recording of parameters shown in a display.

In one aspect, one embodiment of the invention provides an apparatus for recognizing one or more characters. The apparatus includes an adjustment unit to adjust a color level of a first image of an object to generate an adjusted image, the first image of the object generated from a color image of the object, the color image of the object including one or more characters to be recognized; a conversion unit for converting the adjusted image into a binary image; and a character recognition unit for recognizing the one or more characters based on the binary image.

Generally, in order to perform OCR, an image should be first converted into a binary image. The multiple color information of the image (including background color information and foreground color information) can introduce noise into the binary image after conversion and further adversely affect the OCR results. For example, some regions having a background color may be converted into black regions in a binary image like characters so as to interfere with recognition of characters, and thus characters cannot be accurately recognized from a color image having various color information.

According to the apparatus of the present invention, the color level of a color image is first adjusted before converting the color image into a binary image to perform OCR on characters in the image. In this way, one or more characters can be accurately recognized from an image having a plurality of color information, particularly, background color information.

According to one embodiment, the adjustment unit adjusts the brightness of the color image of the object to generate the first image of the object. Therefore, before adjusting the color level, the brightness of the color image is first adjusted, whereby a part of the background color can be removed.

According to another embodiment, the adjusting unit adjusts a contrast of the color image of the object to generate an intermediate color image of the object, and then adjusts the brightness of the intermediate color image of the object to generate the first image of the object. In this way, the contrast of the color image is first adjusted, and thus, overexposure and/or underexposure in the color image can be overcome.

According to a further embodiment, the adjusting unit sharpens the first image of the object and then adjusts a color level of the sharpened first image of the object to generate the adjusted image. In this way, sharpening the image before adjusting in view of the color level can remove noise in the image.

According to another embodiment, the device further comprises an identification unit for identifying a region of interest as the object from the initial color image, the region of interest showing one or more characters to be identified; and determining image data indicative of the region of interest from the initial color image to generate the color image of the object. According to this embodiment, the region of interest is determined from the initial color image, and all subsequent processing can be performed on the color image of only the region of interest.

According to a further embodiment, the device further comprises a comparison unit for comparing the initial color image with one or more predetermined templates indicative of an image of interest showing the object, wherein the identification unit identifies the region of interest as the object from the initial color image if the initial color image corresponds to the one or more predetermined templates.

According to a further embodiment, the device further comprises a comparison unit for comparing the color image of the object with one or more predetermined templates, the one or more predetermined templates being indicative of an image of interest of the object, wherein the first image of the object is generated from the color image of the object if the color image of the object corresponds to the one or more predetermined templates.

According to the above two embodiments, only color images or initial color images corresponding to a predetermined template may be processed according to embodiments of the present invention. Thus, the color image may be checked for validity first, and invalid images may be discarded. In this way, the apparatus of the present invention can be designed to process a specified kind of image.

In another aspect, an embodiment of the present invention provides a method for recognizing one or more characters. The method includes adjusting a color level of a first image of an object to generate an adjusted image, the first image of the object generated from a color image of the object and including one or more characters to be recognized; converting the adjusted image into a binary image; and identifying the one or more characters based on the binary image.

In a further aspect, embodiments of the invention provide an image processor configured to perform a method according to embodiments of the invention; an imaging system comprising a device for recognizing one or more characters or an image processor configured to perform a method according to an embodiment of the invention; and a machine-readable storage medium encoded with instructions for, when executed, performing a method according to an embodiment of the invention.

Various aspects and features of the disclosure are described in more detail below. Still other objects and advantages of the present invention will become more apparent and will be readily understood by reference to the description taken in conjunction with the accompanying drawings.

Drawings

The invention will be described and explained in more detail hereinafter with reference to embodiments and with reference to the accompanying drawings, in which:

FIG. 1 illustrates a system 10 for recognizing one or more characters from a color image according to one embodiment of the present invention;

FIG. 2 is a flow diagram 100 illustrating adjusting the contrast of a color image according to one embodiment of the present invention;

FIG. 3 is a flow diagram 200 illustrating adjusting the brightness of a color image according to one embodiment of the present invention;

FIG. 4 is a flow diagram 300 illustrating unsharp masking (USM) sharpening of a color image, according to one embodiment of the present invention;

FIG. 5 is a flow diagram 400 illustrating adjusting the color levels of a color image according to one embodiment of the present invention;

FIG. 6 is a flowchart 500 illustrating an optical character recognition procedure based on a binary image including a character to be recognized according to one embodiment of the present invention;

FIG. 7 illustrates a system 20 for recognizing one or more characters from a color image according to another embodiment of the present invention;

FIG. 8 is a flow diagram illustrating identification of a region of interest from an initial image according to one embodiment of the present invention; and

FIG. 9 is a flow diagram of a method for recognizing one or more characters from a color image according to one embodiment of the present invention.

The same reference numbers in the drawings indicate similar or corresponding features and/or functions.

The present invention will be described with respect to particular embodiments and with reference to certain drawings but the invention is not limited thereto but only by the claims. The drawings described are only schematic and are non-limiting. In the drawings, the size of some of the elements may be exaggerated and not drawn on scale for illustrative purposes.

Detailed Description

The invention will be described hereinafter in relation to medical applications, in particular dialysis machines, such as 4008A and advanced dialysis machines of fisheries. It should be noted, however, that the apparatus and method of the present invention may also be used in other fields and for other images, as long as these images have a variety of color information, in particular background color information, and there is a need to automatically record parameters, but the apparatus may not be able to provide such an interface to record parameters electronically.

In the 4008A dialysis machine of ferwensis, the dialysis parameters are displayed in a display with a variety of color information during dialysis, e.g. the background color may be grey, while one parameter is shown in blue and another parameter in green. Since such dialysis machines do not provide an interface to record parameters, in order to automatically record parameters, according to an embodiment of the present invention, the parameters should be identified from the displayed image in an automated manner.

Fig. 1 shows an apparatus 10 for recognizing one or more characters according to an embodiment of the invention. The device 10 comprises an adjustment unit 12, a conversion unit 13 and a character recognition unit 14. The adjusting unit 12 includes at least a color level adjusting unit 124. In one embodiment, as shown in fig. 1, the adjusting unit 12 further includes a contrast adjusting unit 121, a brightness adjusting unit 122, and a sharpening unit 123. It should be noted that the contrast adjustment unit 121, the brightness adjustment unit 122, and the sharpening unit 123 are optional in the device. It is contemplated that the adjustment unit 12 itself performs its functions without being further divided into multiple units.

The adjustment unit 12 may receive a color image of the object. For example, in one embodiment the adjustment unit 12 receives a color image of the display of the fisheries '4008A dialysis machine directly from an image acquisition unit (such as a camera), or in another embodiment, an initial color image is first acquired from the display of the fisheries' 4008A dialysis machine, the adjustment unit 12 receiving a color image obtained by processing the initial color image, e.g., the adjustment unit 12 receiving a color image of the region of interest identified from the initial color image. The pre-processing of the initial color image for the display of the dialysis machine will be described below with respect to fig. 7-8.

Upon receiving the color image of the subject, the contrast adjustment unit 121 determines the contrast of the color image and adjusts the contrast of the color image as necessary. Fig. 2 shows a flow diagram 100 illustrating adjusting the contrast of a color image according to one embodiment of the invention.

According to block 101-. By adjusting the contrast of the color image, overexposure and underexposure of the color image can be eliminated.

The intermediate image from the contrast adjustment unit 121 may be further input to a brightness adjustment unit 122 to adjust the brightness of the color image. Fig. 3 shows a flow diagram 200 illustrating adjusting the brightness of a color image according to one embodiment of the invention.

According to block 201 and 205, for example, the brightness adjustment unit 122 may determine the brightness of the color image and evaluate the brightness with respect to, for example, a threshold or a threshold range, set the adjustment value Δ _2 if the determined brightness is higher with respect to the threshold or the threshold range, set the adjustment value Δ _ 2' if the determined brightness is lower with respect to the threshold or the threshold range, and then adjust the brightness of the color image based on the adjustment value to obtain the brightness-adjusted first image.

By adjusting the brightness of the color image relative to a threshold or threshold range, background color information of the color image may be partially removed and character recognition may be improved. In general, if the brightness of a color image is within a threshold range, the brightness of the color image need not be adjusted, however, to efficiently remove background color information, it is preferable to still adjust the brightness of the color image to make the color image brighter. In one embodiment, it is also possible to adjust the threshold range according to different requirements to achieve a similar effect.

After adjusting the brightness of the color image, the color image is input to the sharpening unit 123, and the sharpening unit 123 performs image sharpening on the first image to remove interference and noise signals in the first image. The sharpening unit 123 may employ various sharpening methods, as one example, fig. 4 shows a flowchart 300 illustrating unsharp masking (USM) sharpening performed on a color image according to one embodiment of the present invention. In embodiments for removing interference and noise signals, the USM sharpening method is preferred.

Based on the USM sharpening method, the sharpening unit 123 may first obtain pixel data of the color image and apply gaussian blur to the image, and then apply the USM formula to the blurred image to calculate a new pixel value for each pixel of the color image based on the input parameters, which may be determined from user input, and which are well known in the art, and finally construct a new sharpened image based on the new pixel values, according to block 301-.

After adjusting the contrast and brightness of the color image and sharpening the color image, background color information of the color image may be partially removed and noise may be reduced. The sharpened first image from the sharpening unit 123 may then be input to the color level adjustment unit 124 for further processing.

The color level adjustment unit 124 may adjust the color level of the color image. Fig. 5 shows a flowchart 400 illustrating adjusting the color levels of a color image according to one embodiment of the invention. Specifically, according to block 401-.

In general, by evaluating the color level of a color image relative to a color level table and then adjusting the color level of the color image, the color information of the corresponding characters in the color image may be maintained or enhanced while the background color information may be removed, thereby reducing potential noise introduced by the background color information during subsequent optical character recognition procedures.

Although the contrast adjustment unit 121, the brightness adjustment unit 122, the sharpening unit 123, and the color level adjustment unit 124 are described above with reference to fig. 2 to 5, the processes mentioned above and shown in fig. 2 to 5 are not restrictive but illustrative, and other processes and algorithms for achieving similar effects are also contemplated.

Furthermore, although the various processes of the adjustment unit 12 are described above in view of first adjusting the contrast and brightness, then sharpening, and finally adjusting the color level, other orders of the above processes are also contemplated, or one or more of the above processes may be omitted and/or combined to meet different requirements. For example, one or more of the contrast adjusting unit 121, the brightness adjusting unit 122, and the sharpening unit 123 may be omitted; or even all of them may be omitted, in which case the received color image of the subject is determined as the first image to be adjusted by the color level adjustment unit 124.

The adjusted image may be input to the conversion unit 13 to be converted into a binary image. Since the background color information has been almost removed, noise and interference can be reduced and characters can be clearly recognized in the binary image.

The binary image obtained after the conversion is input to the character recognition unit 14 to be used in the optical character recognition program. Fig. 6 shows a flow chart 500 illustrating an optical character recognition procedure based on a binary image comprising characters to be recognized. According to the embodiment as shown in fig. 6, at block 501, if the binary image has a black foreground and a white background, it is first converted to an inverse binary image, which step may be omitted in some cases; at block 502, calculating a sum of projection values in a horizontal direction of the binary image for each row of pixels; at block 503, the projected value sum for each row of pixels is compared to a horizontal projected value threshold and then based on a determination that the projected value sum for a row of pixels is below the horizontal projected value threshold, the binary image is divided into one or more row images, each row image comprising a row of characters; at block 504, for each line image, erosion and dilation algorithms are performed to make adjacent characters a unit of text; at block 505, the sum of the projection values in the vertical direction for each row image is calculated for each column pixel; at block 506, the sum of the projection values for each column of pixels is compared to a vertical projection value threshold and then the line image is divided into each text unit based on a determination that the sum of the projection values for a column of pixels is below the vertical projection value threshold; finally, at block 507, the characters in each text unit may be recognized by any known optical character recognition technique. The above flow chart performed by the character recognition unit 14 is illustrative and not restrictive, and the steps may be adjusted according to the binary image to be processed and different requirements.

In light of the above, with reference to FIG. 1, an apparatus 10 for recognizing one or more characters is described in accordance with one embodiment of the present invention. Fig. 7 shows a device 20 for recognizing one or more characters according to another embodiment of the invention. According to this embodiment, the device 20 comprises a recognition unit 25, a comparison unit 26, an adjustment unit 22, a conversion unit 23 and a character recognition unit 24, wherein the adjustment unit 22, the conversion unit 23 and the character recognition unit 24 have similar functions as the adjustment unit 12, the conversion unit 13 and the character recognition unit 14 as shown in fig. 1, the main differences of the device 20 from the device 10 relate to the recognition unit 25 and the comparison unit 26. In one example, as shown in fig. 7, the recognition unit 25 and the comparison unit 26 may be referred to as a preprocessing unit 21.

According to an embodiment as shown in fig. 7, an initial color image received from an image acquisition unit (not shown) may be input to the preprocessing unit 21 to obtain a color image of the object to be processed in the adjustment unit 22.

The identification unit 25 of the pre-processing unit 21 may receive the initial color image to identify the region of interest from the initial color image. For medical devices that do not provide an interface to record parameters, such as dialysis machines, in order to identify and record the parameters shown in the display of the medical device in an automated manner, a physician or caregiver may capture an image of the display of the medical device by an image acquisition unit, such as a camera, which may be an initial color image to be processed.

Sometimes, the captured image may comprise some information not related to the display of the medical device (e.g. a person standing near the display of the medical device) or not related to the region of interest comprising the characters to be identified, from the initial image only the region of interest (such as the region related to the display or the region comprising the characters to be identified) is identified using the identification unit 25, and thus from the captured image corresponding image data is determined to obtain a color image of the object, which in turn is adjusted in the adjustment unit 22, other information may be removed from the initial image.

FIG. 8 is a flow diagram 600 illustrating the identification of a region of interest from an initial image in accordance with one embodiment of the present invention. The region of interest may be a region showing a display of the medical device. According to the embodiment as shown in fig. 8, at

blocks

601 and 602, the size of the initial color image is first determined and adjusted; at block 603, the adjusted initial color image is converted to a grayscale image; at block 604-; at

blocks

607 and 608, the four vertices of the found boundary are calculated and a color image of the region of interest is obtained by determining image data indicative of the region of interest from the initial color image based on the four vertices, which may be used as a color image of the object to be further processed in the adjustment unit 22. The processes as shown in fig. 8 are illustrative, they may be modified, and other processes for identifying regions of interest are contemplated.

In an embodiment, after identifying and generating the color image of the region of interest as the color image of the object, the color image may be further input to the comparison unit 26 for further processing. The comparison unit 26 may compare the color image of the object with one or more predetermined templates, which may be pre-stored and indicate an image of interest of the object, wherein the color image of the object is further processed in the adjustment unit 22 if the color image of the object corresponds to the one or more predetermined templates, otherwise the color image of the object may be considered invalid and the procedure is not performed on it. In this way, the validity of the color image to be processed can be checked.

Although as shown in fig. 7 and explained above, the preprocessing of the initial color image is described with reference to identifying image data of a region of interest (such as a display region) to obtain a color image of the object and then checking the validity of the obtained color image of the object by comparing the color image to a predetermined template, it should be understood that this is not limiting, and that other processing(s) may be included in the preprocessing procedure, e.g., image keystone correction; and one or more of the processes of the recognition unit 25 and the comparison unit 26 may be omitted. For example, the identification unit 25 may be omitted if the initial color image shows only the region of interest.

In another embodiment, the comparison unit 26 first receives and compares the initial color image with one or more predetermined templates indicating the image of interest showing the object, wherein, if the initial color image corresponds to the one or more predetermined templates, the identification unit 25 identifies the region of interest as the object from the initial color image to obtain a color image of the object to be further processed.

Although fig. 7 shows the above processing only for preprocessing the initial color image, it is contemplated that one or both of the processes in the adjustment unit 12 may be performed in the preprocessing unit 21.

From the foregoing, the apparatus for recognizing one or more characters of the present invention has been described in relation to various embodiments, various flowcharts, and various units, it being understood that all flowcharts are illustrative and that one or more processes according to individual flowcharts may be deleted, modified, combined, and/or separated to implement and/or omit their corresponding functions. It should also be understood that the order of the processes may be changed if necessary. For example, the contrast of the initial color image may be adjusted first, and then a color image of the object is generated from the adjusted initial color image for further processing. Moreover, individual elements of the devices of the invention may be deleted, modified, combined, and/or separated to implement and/or omit their corresponding functions.

Fig. 9 is a flow diagram of a method 700 for identifying one or more characters from an initial color image in accordance with one embodiment of the present invention. According to the method 700, at 701, an initial color image acquired by, for example, a camera is received; at 702, a region of interest is identified as an object from an initial color image, and image data indicative of the region of interest is determined from the initial color image to generate a color image of the object, the region of interest showing one or more characters to be identified. At 703, the color image of the object may be compared to one or more predetermined templates indicative of the image of interest and showing the object, and if the color image of the object corresponds to the one or more predetermined templates, then at 704, the color image of the object is further processed; otherwise, processing returns to 701 to receive a new initial color image.

At 704, a contrast of the color image is determined and adjusted based on, for example, a threshold range to obtain an intermediate image; and then at 705, the brightness of the color image is determined and adjusted based on, for example, another threshold range to generate a first image. At 706, the first image is sharpened. At 707, the color level of the color image is determined and adjusted to generate an adjusted image, and then at 708, the adjusted image is converted to a binary image. Based on the binary image, at 709, one or more characters may be identified.

The method 700 is illustrative and not limiting; various steps may be omitted, combined, and/or modified to omit/modify corresponding functions. For example, in one embodiment, any one or more of the steps of identifying a region of interest, validating the image, adjusting image contrast and brightness, and sharpening the image may be omitted. The order of the steps may also be changed. In one embodiment, an initial color image may first be compared to a predetermined template(s), and after determining that the initial color image corresponds to one of the templates, a region of interest is identified from the initial color image.

In one embodiment, the device of the present invention may be embodied as an application program that may be run in, for example, a mobile phone. Alternatively, the device of the invention may be implemented as an image processor configured to perform the method of the invention as already described above. The device of the invention may be comprised in an imaging system which may further comprise an image acquisition unit for acquiring an initial image.

It is contemplated that the individual elements of the apparatus of the invention may be implemented by any of software, hardware or firmware. Further, a machine-readable storage medium may store instructions for performing the methods of the present invention.

It is noted that the device and the method according to the invention should not be limited to only those mentioned above. It will be apparent to those skilled in the art that the various aspects of the invention claimed may be practiced in other examples that depart from these specific details.

Furthermore, even if specific measures are recited in mutually different dependent claims, this does not indicate that a combination of these measures cannot be used to advantage.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps other than those listed in a claim or the specification. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. In the unit claims enumerating several units, several of these units can be embodied by one and the same item of software and/or hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words are to be interpreted as names.

Claims

1. An apparatus for recognizing one or more characters, comprising

An adjustment unit to adjust a color level of a first image of an object to generate an adjusted image, the first image of the object generated from a color image of the object, the color image of the object including one or more characters to be recognized;

a conversion unit for converting the adjusted image into a binary image; and

a character recognition unit for recognizing the one or more characters based on the binary image.

2. The apparatus of claim 1, wherein

The adjustment unit adjusts the brightness of the color image of the object to generate the first image of the object.

3. The apparatus of claim 1, wherein

The adjustment unit adjusts a contrast of the color image of the object to generate an intermediate color image of the object, and then adjusts a brightness of the intermediate color image of the object to generate the first image of the object.

4. The apparatus of claim 1, wherein

The adjustment unit sharpens the first image of the object and then adjusts the color level of the sharpened first image of the object to generate the adjusted image.

5. The apparatus of any of claims 1-4, further comprising

An identification unit for identifying a region of interest as the object from an initial color image, the region of interest showing the one or more characters to be identified; and determining image data indicative of the region of interest from the initial color image to generate the color image of the object.

6. The apparatus of any of claims 1-5, further comprising

A comparison unit for comparing the color image of the object with one or more predetermined templates, the one or more predetermined templates being indicative of an image of interest of the object, wherein the first image of the object is generated from the color image of the object if the color image of the object corresponds to the one or more predetermined templates.

7. The apparatus of claim 5, further comprising

A comparison unit for comparing the initial color image with one or more predetermined templates indicative of an image of interest showing the object, wherein the identification unit identifies the region of interest as the object from the initial color image if the initial color image corresponds to the one or more predetermined templates.

8. A method for recognizing one or more characters, comprising

Adjusting a color level of a first image of an object to generate an adjusted image, the first image of the object being generated from a color image of the object and including one or more characters to be recognized;

converting the adjusted image into a binary image; and

identifying the one or more characters based on the binary image.

9. The method of claim 8, further comprising

Adjusting the brightness of the color image of the object to generate the first image of the object.

10. The method of claim 8, further comprising

Adjusting a contrast of the color image of the object to generate an intermediate color image of the object; and

adjusting the brightness of the intermediate color image of the object to generate the first image of the object.

11. The method of claim 8, further comprising

Sharpening the first image of the object; and

adjusting the color level of a sharpened second image of the object to generate the adjusted image.

12. The method of any of claims 8-11, further comprising

Identifying a region of interest as the object from an initial color image, the region of interest showing the one or more characters to be identified; and

determining image data indicative of the region of interest from the initial color image to generate the color image of the object.

13. The method of any of claims 8-12, further comprising

Comparing the color image of the object with one or more predetermined templates indicative of an image of interest of the object; and

generating the first image of the object from the color image of the object if the color image of the object corresponds to the one or more predetermined templates.

14. The method of claim 12, further comprising

Comparing the initial color image to one or more predetermined templates indicative of an image of interest showing the object; and

identifying the region of interest as the object from the initial color image if the initial color image corresponds to the one or more predetermined templates.

15. An image processor configured to perform the method of any one of claims 8-14.

16. An imaging system comprising an apparatus for recognizing one or more characters according to any one of claims 1-8 or an image processor according to claim 15.

17. The imaging system of claim 16, further comprising

An image acquisition unit for acquiring an initial image, an

Wherein the one or more characters are identified from the initial image.

18. A machine-readable storage medium encoded with instructions that, when executed, perform the method of any of claims 8-14.