JP2021149136A - Server, method, and program for extracting character string such as serial number - Google Patents

Abstract

To provide a server, a method, and a program for extracting an object character string such as serial numbers from a format in an indefinite shape.SOLUTION: A character string extraction server is for extracting a plurality of object character strings included in an unspecific area of an image and includes: rectangle detection means which detects a plurality of closed areas being regions surrounded by a margin from a binarization image of an image, and detects a rectangle related to the respective closed areas; skew angle detection means which detects a skew angle of character strings inside the respective detected rectangles; skew correction means detecting a rectangle of a rotation image in accordance with a skew angle with respect to each rectangle; and character recognition means which recognizes a character string in the internal region of the rectangle and/or the rectangle of the rotation image. The recognized character string includes a plurality of object character strings.SELECTED DRAWING: Figure 2

Description

The present invention relates to a character string extraction system. In particular, it relates to a server, a method and a program for extracting a target character string such as a serial number from an amorphous format.

Conventionally, in the campaign application management system that manages the application of the campaign that can be applied by either the application method of sending the code from the terminal or the application method of adding the code to a predetermined form and mailing it, the data extraction unit reads the image. It was proposed to read the code indicating the serial number from the application form image acquired by the department by image recognition, decode the image data of the read code, and extract the mailed serial number represented by the text data (special feature). Open 2016-091113 (Patent Document 1).

Even if numbers and characters other than the serial code are mixed in the image data, as a mobile terminal device that can appropriately extract the serial code, an image in which various symbols such as the serial code and characters and numbers around the serial code are reflected. It has been proposed to extract a portion of the data that matches the format information stored in the information storage unit as a serial code (Japanese Unexamined Patent Publication No. 2018-136830: Patent Document 2).

JP-A-2016-091113 JP-A-2018-136830 Patent No. 5940615 Japanese Patent No. 6353893

The form of the campaign for the purpose of sales promotion is diversifying, and multiple application tickets (paper pieces such as stickers printed with a unique serial number for each point) attached to the product (packaging) are collected and set to the specified points. There is also a campaign form in which a prize (lottery qualification) is given when the product is reached, and a campaign form in which the points of the application tickets collected in sequence are sequentially added and the points are consumed on the due date to draw a lottery. While such a campaign form can work well on consumers' purchasing motivation, it tends to complicate the aggregation of points. In order to reduce manual work such as visual check and manual input, it is conceivable to incorporate an OCR (Optical Character Recognition) engine into a web application to recognize the serial code, but it is generally difficult to recognize it successfully. There is a tendency to limit the format and it is not effective.

Therefore, it is desirable to have a campaign support system that is configured so that it does not take time for both the applicant and the applicant to accept the application, depending on the form of the various campaigns.

In view of the above, it is an object of the present invention to provide a system in which a purchaser of a product extracts a character string such as a serial number from a photographed image of an application ticket related to a campaign. In particular, the number of application tickets included in the image may be one or more (optional), and the purpose is to provide a server, method, and program capable of extracting the target character string regardless of the format. do.

One aspect of the present invention for solving the above problems is a character string extraction server for extracting a plurality of target character strings included in an unspecified area of an image, and a margin from a binarized image of the image. A rectangle detecting means for detecting a plurality of closed areas surrounded by a closed area and detecting a rectangle related to each closed area, and a skew angle detecting means for detecting a skew angle of a character string inside each detected rectangle. A recognized character string including a skew correction means for detecting the rectangle of the rotated image according to the skew angle of the rectangle and a character string recognizing means for recognizing the character string in the internal region of the rectangle and / or the rectangle of the rotated image. Is characterized by containing a plurality of target character strings.

According to the present invention, it is possible to detect an area of the target character string in an unspecified portion of the image by using the feature that the target character string such as a serial number has a margin around it. In addition, the inclination of the character string is detected and corrected based on the margins and the like. As a result, it is possible to perform successful character recognition even from an image in which a plurality of individually collected application tickets are arbitrarily arranged and photographed. This makes it possible to easily carry out a campaign using a plurality of application tickets without limiting the format or taking the trouble of totaling. Applicants can easily participate in the campaign by registering multiple serial numbers on the campaign site simply by shooting any number of application tickets with a smartphone, etc., without downloading the application mount from the campaign site. ..

Preferably, the character string recognition means determines whether or not the character string inside each of the rectangles includes the target character string based on the aspect ratio of the rectangle and / or the rectangle of the rotated image, and determines that the character string includes the target character string. Recognize the character string in the internal area of the rectangle. According to the present invention, the shape of a rectangular region including a target character string such as a serial number is extracted substantially uniformly by skew correction or the like. Therefore, only the area including the target character string can be extracted based on the aspect ratio of the rectangle. By doing so, the OCR processing range can be limited regardless of the format information, and the target character string can be efficiently extracted.

The target character string may be a character string having a margin around it, which is composed of a combination of characters including numbers, alphabets, kana characters, and symbols. The character string may be one line (one column) or two or more lines (two columns), and the skew angle is detected from the upper and lower margins and the line spacing of the character string according to the format.

In the binarized image, the frequency of pixels less than a predetermined brightness value included in the grayscale image of the image is calculated, and the difference in brightness value between the pixels having a predetermined brightness value or more included in the grayscale image and the neighboring pixels is calculated. The absolute value of is subtracted from the intermediate value to calculate the frequency to create a histogram in which the luminance values of all the pixels are equal to or less than the intermediate value, and the threshold value can be determined based on the histogram to be generated.

By doing so, a binarized image for analysis suitable for the analysis according to the present invention is generated even from an image with insufficient shooting conditions, and finally, successful character recognition is facilitated, and the effectiveness is improved. It can be a high system.

The rectangle and / or the rectangle of the rotated image may include a rectangle in which a plurality of rectangles adjacent to each other in the vertical and horizontal directions are combined.

It is considered that the serial number etc. is described with sufficient margin from the surroundings (it may be separated by a ruled line, but the ruled line may be removed to make a margin), and the detected rectangle is only the target character string. In many cases including. A rectangle containing an unnecessary character string often does not include the target character string. When an application sticker with a serial number spanning multiple lines is photographed in close proximity, a rectangle containing each line may be detected, and the rectangle may include a part of the serial number. If the serial number spans multiple digits, each rectangle containing each digit may be detected. Since the character strings that make up the serial number are considered to be close to each other, it is possible to synthesize a rectangle consisting of the entire serial number by synthesizing rectangles that are sufficiently close to each other.

Preferably, the rectangle of the rotated image is detected based on the coordinates obtained by rotating the coordinates related to the determination of the rectangle detected by the rectangle detecting means based on the line segment interpolation table. By doing so, the processing range of OCR can be detected with a very small amount of calculation.

Another aspect of the present invention is a character string extraction method for extracting a plurality of target character strings included in an unspecified area of an image, in a region surrounded by margins from a binarized image of the image. A rectangle detection step that detects multiple closed areas and detects a rectangle related to each closed area, a skew angle detection step that detects the skew angle of the character string inside each detected rectangle, and a skew angle for each rectangle. A skew correction step for detecting the rectangle of the rotated image and a character recognition step for recognizing the character string in the internal area of the rectangle and / or the rectangle of the rotated image are included, and the recognized character string includes a plurality of target character strings. It is characterized by including.

Yet another aspect of the present invention is a program that causes a computer to execute the above character extraction method.

According to the present invention, a highly versatile character string extraction server, method, and program capable of supporting various forms of campaigns and the like are provided. Even in a campaign that requires aggregation of a plurality of application tickets, a serial code or the like can be automatically extracted from the captured images of an arbitrary number of application tickets. As a result, the applicant can easily carry out the campaign, and the applicant can easily participate.

Further, according to the present invention, a series of processes are performed so that characters are successfully recognized according to the state of an image taken by a general user. As a result, the target character string such as the serial number can be extracted with high accuracy even from the image in which an arbitrary number of application tickets are arranged arbitrarily, and the campaign can be substantially supported.

FIG. 1 schematically shows a campaign support system according to the present invention. FIG. 2 is a functional block diagram of the character string extraction server according to the present invention. FIG. 3 is an example of an image taken by a user terminal. FIG. 4A is a binarized image of a comparative example. FIG. 4B is a binarized image of the embodiment according to the present invention. FIG. 5 is an example of a histogram used in the binarization method according to the present invention. FIG. 6A schematically shows a binarized image. FIG. 6B schematically shows the horizontal blocking of the binarized image. FIG. 6C schematically shows the detection of a white background rectangle. FIG. 6D schematically shows the detection of a closed area. FIG. 6E schematically shows the detection of a rectangle. FIG. 7 schematically shows the detection of other closed areas and rectangles. FIG. 8 is a diagram schematically explaining the inclination (skew) of the character string with respect to the OCR direction. FIG. 9 is a diagram schematically showing a plurality of scan pixel regions determined by the line segment interpolation algorithm. FIG. 10 is a diagram schematically illustrating detection of a skew angle by scanning a scan pixel region. FIG. 11A is a diagram schematically illustrating the generation of a rotated image according to the skew angle. FIG. 11B schematically shows the detection of a rectangle in a rotated image. 12 (a) and 12 (b) are diagrams for explaining the generation of the rotation image and the detection of the rectangle based on the line segment interpolation table.

Hereinafter, various features of the invention will be described with reference to the drawings, along with suitable examples not intended to limit the invention. The drawings are simplified and outlined for illustration purposes.

With reference to FIG. 1, the campaign support system 1 includes a character string extraction server 100, a campaign server 200, and one or more user terminals 300 according to the present invention. The user terminal 300 can take an image I including an application ticket for applying for the campaign. The campaign server 200 can communicate with the user terminal 300, and the character string extraction server 100 can communicate with the campaign server 200 via a communication network 2 such as the Internet.

The character string extraction server 100 has a function for mounting a central processing unit (CPU), RAM, ROM, hard disk, etc., executing a programming language under the control of an appropriate operating system (OS), and executing various processes. Provide means. The program stored in the server is constructed by HTML, Javascript (registered trademark), a native program (object code), or the like. The server 100 may be provided as an ASP, and the user can access and use the server 100 through a Web browser, dedicated client software, or the like. Alternatively, the present invention can be incorporated into the campaign support system 1 as a character string extraction library.

The campaign server 200 may be an external server composed of a known server device that manages various information related to the campaign. The campaign server 200 may issue a user ID or passcode in response to access to the campaign site from the user terminal 300, and may acquire an image I taken by the camera of the user terminal. Alternatively, the operator of the campaign that has received the application sticker attached to the postcard may acquire the image I by photographing the postcard. The campaign server 200 can transmit the image I to the character string extraction server 100 and receive the analysis result. In addition, it is possible to collate with the regular serial number list stored in the database (not shown) as appropriate, check for duplicate applications, and use the point list associated with the serial number for each user. It is possible to collect the points related to the application and draw a prize. The external server is not limited to the campaign server, and may be any computer device that communicates with the character string extraction server 100 according to the present invention.

The user terminal 300 may be a high-performance terminal such as a known smartphone or a tablet-type high-performance terminal equipped with a camera device or browser software.

FIG. 2 shows a functional block diagram of the character string extraction logic implemented in the character string extraction server 100. The character string extraction server 100 includes a binarization means 10, a closed area / rectangular detection means 20, a skew angle detection means 30, a skew correction means 40, a target character string extraction means 50, and a storage means 60, and is provided from the campaign server 200 (or). , The result of extracting the target character string such as the serial number printed on each application ticket from the image I of one or more (arbitrary number) application tickets received (directly from the user terminal 300). The character string set) can be output and returned to the campaign server 200. Each means 10-60 of the string extraction server 100 can be implemented in software, firmware, hardware, or any combination thereof.

In the campaign envisioned by the present invention, the number of application tickets and the like is generally arbitrary (one or more), and their arrangement in the image may be arbitrary. For example, the user can access the campaign site 200 with a user terminal 300 such as a smartphone, and transmit the image I of the application tickets collected according to the instructions on the screen to the campaign site. FIG. 3 shows an exemplary image taken by the camera of the user terminal 300. In this example, the user shoots 10 application tickets side by side (pasted) on a whitish table (table, back of advertisement, etc.). In the image, the shadow of the user terminal is reflected in the center of the left corner, and some application tickets are tilted with respect to the vertical and horizontal directions of the image. It is considered that such an example is not uncommon in an image taken by a user with a mobile terminal such as a smartphone. Even if this image is binarized as before and OCR processed, the successful extraction of the serial number cannot be expected. Such a system is inconvenient and undesirable, as it can lead to frequent recognition errors, eventually requiring visual checks and manual input, and repeatedly requiring the user to send images until OCR processing is possible. ..

The binarization means 10 executes preprocessing including binarization in order to accurately recognize the finally required characters even from the image (I) as shown in FIG. For example, the binarizing means 10 appropriately scales the image I, trims the image I so as to be divisible by a predetermined block size, and executes an appropriate filter to generate a binarized image for analysis.

In order to explain the conventional binarization, when the image of FIG. 3 is binarized by the Otsu formula, the application ticket included in the shadow portion is painted black as shown in FIG. 4A. At this time, even if the threshold value is corrected to the dark side, the characters in the bright part become too light instead of the shadow becoming light. There are various binarization methods other than the Otsu method. For example, in the dynamic threshold method in which an image is divided to obtain each threshold value, a deviation occurs at the boundary depending on the size of the division, and the figure ( Foreground) may not be detected accurately.

The inventor of the present invention has devised a binarization method capable of easily distinguishing a ground (background) and a figure (foreground) even in an image containing shadows. According to the binarization method, the serial number in the shadow is not painted black as shown in FIG. 4B.

The binarization according to the present invention will be described with reference to FIG. First, pixels (black) having a brightness value less than a predetermined value (for example, less than 16 or less than 32) are extracted from the pixels constituting the grayscale image (256 gradations) of the image I. By doing so, a black part (such as a serial number printed in black) can be extracted even in the shadow. The predetermined luminance value may be determined by measuring in advance the threshold value of the brightness of the pixel to be left as black using the sample image as shown in FIG. For pixels (p) having a predetermined brightness value or more (for example, 16 or more, 32 or more, etc.), for example, when "p = intermediate value-|-s + p |" is calculated using the upper left pixel (s), the pixels. The smaller the difference in brightness between them, the closer the calculation result approaches the intermediate value (for example, 128), and the larger the difference in brightness, the closer to 0 (less than 0 is corrected to 0). FIG. 5 is a histogram calculated as described above (the horizontal axis is the luminance value and the vertical axis is the frequency), and all the pixels are distributed below the median value (128). Based on the histogram, for example, by obtaining the threshold value by the Otsu equation and generating a binarized image, it is possible to extract both black pixels in a dark place and black pixels in a bright place.

Appropriate embossing filters for binarization are not limited to those using the upper left pixel (s). If the number of neighborhoods is large, the contour (edge) is emphasized, and if it is small, the contour (edge) is weakened. ). Alternatively, two neighborhoods, an upper right corner and a lower left corner, can be used so as to be suitable for detecting a closed area, which will be described later. In addition, binarized embossed types such as 4 pixels at 4 corners of a matrix having a size of 3 × 3 or all pixels in the vicinity of 8 can be appropriately used for generating a histogram.

The closed area / rectangle detecting means 20 blocks the binarized image for analysis into a fixed block size, detects the closed area, and detects the rectangle based on the closed area. The detection of closed areas has the following advantages. Known OCR engines generally set the range of what to read for each format in the image. If the entire image is set as the OCR processing range, unnecessary characters and patterns may be processed as characters and the number of decipherable characters may be exceeded, which is not preferable. If the number and position of application tickets included in the image are arbitrary, the OCR processing range cannot be set in advance. According to the present invention, the OCR processing range can be detected from an image by using the feature that the printed characters (serial number, etc.) have a margin around them. In the present specification, a closed area surrounded by a margin may be referred to as a “closed area”.

The detection of closed areas and rectangles will be described with reference to FIGS. 6A-6E. FIG. 6A shows a portion of the binarized image Ia for analysis including two application tickets. In the illustrated example, as a result of binarization, characters other than the serial number such as the 7-digit two-stage serial number in the application ticket, the frame above and below it, and the automatic voice reception number are black pixels, and other parts Is a white pixel.

FIG. 6B shows the image Ia in the horizontal (width W) direction with a block size (Δw) of several pixels (2 pixels, 3 pixels, 4 pixels, 5 pixels, 6 pixels, or any other integer pixel). Indicates blocked horizontal blocking. By detecting as a white background blank when all the pixels of the block size in the horizontal direction are white pixels, even an image containing (partially) the inclination of a character string or the like is stretched in the horizontal direction (margins and line spacing). Etc.) will be easier to detect. Within each block (B1 to B14), a portion where a white background blank is continuous is detected as a white background rectangle.

The large number of white rectangles in FIG. 6C indicate the detected white rectangle R, respectively. The gray colored portion indicates a portion that is not extracted as a white background blank because the block size includes one or more black pixels. Each Rectangle R has the upper left of the image as the origin, and the coordinates (raster data) of the upper left and the lower right can be stored in the storage means 60 such as a memory.

Closed areas are detected by examining the connection of rectangles between adjacent blocks. Rectangle concatenation can be considered concatenated when one block of rectangles touches adjacent blocks of rectangles. Even if the rectangles between the blocks are not in contact with each other, they may be considered to be connected if certain conditions are met. Predetermined conditions can be preset as connection errors.

An example of a series of rectangles representing a closed area is shown in FIG. 6D with gray colored areas. A closed area is a closed area surrounded by a white background, for example, one rectangle of a block is connected to multiple (for example, two) rectangles in a subsequent block, and they are connected to the subsequent blocks. When finally connected to one common rectangle without interruption, those series of rectangles have a closed area inside. As the closed area detection logic, for example, the logic described in Patent Document 4 can be used, but this does not state that the program logic according to the present invention is known.

FIG. 6E schematically shows a rectangle (broken line portion) detected based on each detected closed area. The y-coordinates of the upper and lower sides of the rectangle and the x-coordinates of the left and right sides are stored in the storage means 60. In the case of detection by horizontal blocking, for example, the x-coordinate of the left side is the x-coordinate of the lower right corner of the left rectangle, the x-coordinate of the right side is the x-coordinate of the upper left corner of the right-hand rectangle, and the y-coordinate of the upper side is a series of rectangles on the upper side. The minimum value of the y-coordinate of the lower right end and the y-coordinate of the lower side may be the maximum value of the y-coordinate of the upper left end of the lower series of rectangles. The internal region of the rectangle detected in this way can be processed by OCR.

The above is a description of the detection of the closed area and the rectangle based on the horizontal blocking of the image Ia. Preferably, the image Ia is further blocked with a predetermined block size Δh in the vertical (height H) direction, and the closed area and the rectangle are formed. Detected (not shown). In the case of detection by vertical blocking, for example, the x-coordinate of the left side is the minimum value of the x-coordinate of the lower right end of the series of rectangles on the left side, and the x-coordinate of the right side is the maximum value of the x-coordinate of the upper left end of the series of rectangles on the right side. The y-coordinate of the upper side may be the y-coordinate of the lower right end of the upper rectangle, and the y-coordinate of the lower side may be the y-coordinate of the upper left end of the lower rectangle. The rectangle detected by horizontal blocking and the rectangle detected by vertical blocking may be subjected to OCR processing, respectively. Alternatively, it is determined whether the rectangle detected by horizontal blocking and the rectangle detected by vertical blocking overlap, and if they overlap, the minimum and maximum values of the left and right x-coordinates of the overlapping rectangle and the top and bottom One rectangle may be combined with the minimum and maximum values of the y-coordinate of, and the combined rectangle may be subjected to OCR processing or the like.

According to the detection of the closed area and the rectangle as described above, the processing range of the OCR can be appropriately detected even if the background of the plurality of application tickets includes a pattern such as a grain of a table.

Closed areas and rectangles that do not include the target character string can also be detected, as shown by the gray colored portion and the broken line in FIG. 7. As will be described later, by detecting the rectangle of the rotated image, the rectangle related to the target character string tends to have an aspect ratio according to the target character string. By setting such an aspect ratio (which may be a range) in advance, it is possible to perform OCR processing only on the rectangle including the target character string.

For example, when the serial number consists of 7 digits and 2 lines, the rectangle defined by the margins around the 7 digits and 2 lines has a predetermined aspect ratio. For example, the ratio of the width parallel to the OCR direction (wide: W) to the height perpendicular to the OCR direction (high: H) is in a predetermined range of 1.3 to 1.8. When the application ticket including the serial number is tilted with respect to the width (height) direction in the image, the rectangle detected by the margin around the serial number is not necessarily within the predetermined range. As will be described later, in the present invention, since the inclination is detected and corrected by the margins around the serial number and the line spacing, the aspect ratio of the rectangle including the serial number falls within a predetermined range. It should be noted that the rectangle in which the inclination is not detected cannot be skew-corrected, but it is a part that is not suitable for detection for detecting the skew of the serial number according to the present invention, that is, a part that does not include the target character string, and is generally a serial number or the like. It is considered that it does not interfere with the extraction of the target character string.

As shown in FIG. 8, when the character strings included in the rectangular areas 31 and 32 are upright with respect to the OCR direction (left-to-right direction / top-down direction, etc.) indicated by the arrows. Some are tilted. If the character string is tilted (not upright) with respect to a predetermined direction, the character cannot be recognized correctly in general. When the image contains a plurality of arbitrarily arranged application tickets, it is possible that the inclinations (skews) of the plurality of character strings are different from each other. In the present specification, the slope in which the character string is rotated from the reference direction (OCR direction) with the normal vector of the image as the central axis may be referred to as skew.

The amount of skew of a character string can be detected based on the margins of the character string and the line spacing. A method of calculating a horizontal or vertical histogram of an image is known in order to detect the direction of the line spacing (margin) related to the character string included in the image. In order to generate a histogram in any direction that is not limited to the horizontal or vertical direction, it is conceivable to scan the image in a certain direction while rotating the image sequentially, but this method increases the amount of calculation. The load is likely to be applied to the CPU. A logic that can instantly detect skew angles of a plurality of regions included in an image without imposing a load on the CPU is preferable.

For this purpose, omnidirectional scan pixels with the center of the detected rectangle as the center pixel are determined in advance. Such a scan pixel can be determined by the line segment interpolation method described in Patent Document 3, but this does not state that the program logic according to the present invention is known.

Specifically, the detected rectangular center pixel is obtained, and in a rectangular pixel region having the center pixel o and having a pair of opposite-side pixels of L ₁ and another pair of opposite-side pixels of L _2. Therefore, a plurality of rectangular scan pixel areas rotated by a predetermined angle (θ) in a range of −90 degrees to 90 degrees around the central pixel o are obtained by line segment interpolation and stored in the storage means 60. .. By the line segment interpolation algorithm, a series of pixels including the central pixel o and tilted by an arbitrary angle θ with respect to a series of pixels consisting of a predetermined number of pixels located in the reference direction creates a sine cosine table (trigonometric tables). It is determined only by the addition and subtraction used, and the scan position in substantially all directions can be determined with a small amount of calculation.

FIG. 9 shows an exemplary scan pixel region S1 having the center pixel o of the detected rectangle (gray colored portion) and rotated by 0 degrees, -15 degrees, and -30 degrees from the reference direction (arrow), respectively. , S2 and S3 are shown by dotted lines.

A histogram is calculated for each of these rectangular scan areas. FIG. 10 schematically shows the detection of the skew angle based on the histogram created by the scan. The histogram (shown below) may be a _{bar graph showing L 2 on} the horizontal axis and the histogram value (number of black pixels) on the vertical axis for each scan pixel area, and is a series of pixels in which black pixels do not exist in the histogram. If is present, the angle can represent the angle between the lines. In the illustrated example, for the rectangle 31 (broken line), the angle α ₁ = 0 degree corresponding to the scan pixel area S1'is determined as the skew angle, and for the rectangle 32 (broken line), the angle α _{2 corresponding to the scan pixel area S2.} = -15 degrees is determined as the skew angle.

The rectangle 31 whose detected skew angle is within the permissible range of OCR processing (for example, 5 degrees or less, 3 degrees or less, etc.) may be OCR-processed as it is by the target character string extracting means 50 equipped with a known OCR engine. The rectangle 32 having a skew angle that cannot be expected to read characters successfully by OCR is skew-corrected by the skew correction means 40.

A rotated image of the rectangular area 32 is generated in order to skew correct the rectangular area 32. At this time, if the input point (x, y) of the rectangular region 32 is rotated by (−α ₂ ) degrees with the origin o as the center of rotation to obtain the output point (x', y') of the rotated image, the corresponding pixel is obtained. There is a risk that the image will be thin and the accuracy of OCR processing will be reduced. Therefore, it is preferable to obtain the coordinates (x, y) of the output point of the rotated image from the input point (x', y') of the rectangular region 32 so that the accuracy in the OCR processing does not decrease.

FIG. 11A schematically shows the generation of a rotated image. The skew correction means 40 has a central pixel o, and the number of pixels on the four sides is, for example, a rectangular pixel region having a diameter of 2r of the circumscribed circle of the detected rectangular 32, and is a reference around the central pixel o. _{The rectangular pixel region C'rotated by the skew angle (α 2} ) with respect to the direction (region of the rotated image C) can be obtained by the line segment interpolation method. The rectangle 33 of the rotated image can be detected by copying the corresponding pixel of the pixel region C'(FIG. 11 (b)).

In the rectangle 33 of the rotated image, the character string stands upright with respect to the OCR direction, and the serial number which is the target character string can be recognized.

The detection of the coordinates of the rectangle 33 of the rotated image will be described with reference to FIG. In FIG. 12, each of the many small rectangles represents a pixel. FIG. 12A shows a region C including the rectangle 32 (a portion indicated by a rectangular outer frame) and a region C'(a portion indicated by shades of gray) rotated by _{α 2 = -15 degrees with respect to the region C.} ) And each coordinate (x', y') relating to the determination of the coordinates of the rectangle 32. Each coordinate (x', y') relating to the determination of the coordinates of the rectangle 32 is the coordinate of each rectangle that defines the closed area. Region C'can be generated based on the carry of pixels in the line segment interpolation table. _{That is, the X component of the second pixel located in the angle α 2} direction from the center pixel o (first pixel) _{is extended in the angle α 2} direction to the initial value = 0.5, which is the X component of the center pixel o. The increment in the X direction of the hypotenuse of the second pixel, that is, _{the value obtained by adding Δx (cosα 2} ), and the Y component is set to the initial value = 0.5, which is the Y component of the central pixel o, in the angle α ₂ direction. The increment in the Y direction of the hypotenuse of the second pixel to be extended, that is, _{the value obtained by adding Δy (sinα 2} ), rounding down each value to an integer, and then the carry value of the immediately preceding pixel (centered on the immediately preceding pixel). In the case of a pixel, the carry value is 0), so that the carry in the X direction and the carry in the Y direction from the pixel immediately before the second pixel are determined.

FIG. 12B shows a skew-corrected image (rotated image) based on the line segment interpolation table, rotation-transformed coordinates (x, y), and the like. The rectangle 33 of the rotated image is detected by rotationally transforming the coordinates related to the original rectangle and similarly taking the minimum and maximum values of the X and Y coordinates, and the internal region of the rectangle 33 is OCR processed.

In the embodiment, the time required for each process is 250 ms for filtering and closed area detection of one image, 350 ms for detecting and skew angle analysis of about 30 rectangles (about 12 ms per rectangle), and skew correction (rotation) for one rectangle. It was 2.0 to 3.0 ms, and each serial number was extracted with high accuracy.

As described above, according to the present invention, it is easy to apply even in a campaign using a plurality of (types) of application tickets, and it is also easy to tabulate. The character strings required for the campaign can be extracted instantly with high accuracy, and the system can be highly effective.

It is known to those skilled in the art that many various modifications can be made without departing from the ideas and aspects of the present invention. Therefore, needless to say, the aspects of the present invention are merely examples and do not limit the scope of the present invention.

100 Character string extraction server 10 Binarization means 20 Closed area / rectangle detection means 30 Skew angle detection means 40 Skew correction means 50 Target character string extraction means 60 Storage means

Claims (8)

A character string extraction server for extracting multiple target character strings contained in an unspecified area of an image.
A rectangle detecting means that detects a plurality of closed areas, which are areas surrounded by margins, from the binarized image of the image and detects a rectangle related to each closed area.
A skew angle detecting means for detecting the skew angle of the character string inside each of the detected rectangles,
Skew correction means for detecting the rectangle of the rotated image according to the skew angle for each rectangle,
A character string extraction server comprising the character string recognition means for recognizing the character string in the rectangular internal region of the rectangle and / or the rectangle of the rotated image, and the recognized character string including the plurality of target character strings. .. When the character string recognition means determines whether or not the character string inside each of the rectangles includes the target character string based on the aspect ratio of the rectangle and / or the rectangle of the rotated image, and includes the target character string. A character string extraction server that recognizes the character string in the determined internal area of the rectangle. The character string extraction server according to claim 1, wherein the target character string is a character string having a margin around it, which is composed of a combination of characters including numbers, alphabets, pseudonym characters, and symbols. The binarized image calculates the frequency of pixels having a luminance value less than a predetermined luminance value included in the grayscale image of the image, and the pixels having a luminance value equal to or higher than the predetermined luminance value included in the grayscale image are with neighboring pixels. It is generated by subtracting the absolute value of the difference in brightness value from the intermediate value, calculating the frequency, creating a histogram in which the brightness values of all pixels are equal to or less than the intermediate value, and determining the threshold value based on the histogram. The character string extraction server according to claim 1. The character string extraction server according to claim 1, wherein the rectangle and / or the rectangle of the rotated image includes a rectangle in which a plurality of rectangles adjacent to each other in the vertical and horizontal directions are combined. The first or second aspect of claim 1 or 2, wherein the rectangle of the rotated image is detected based on the coordinates obtained by rotationally transforming the coordinates related to the determination of the rectangle detected by the rectangle detecting means based on the line segment interpolation table. String extraction server. It is a character string extraction method for extracting a plurality of target character strings included in an unspecified area of an image.
A rectangle detection step of detecting a plurality of closed areas, which are areas surrounded by margins, from the binarized image of the image and detecting a rectangle related to each closed area.
A skew angle detecting means for detecting the skew angle of each of the detected rectangles,
A skew correction step for detecting a rectangle of a rotated image according to the skew angle for each rectangle, and
A character string extraction method comprising a character recognition step of recognizing a character string in the rectangular area and / or the rectangular internal area of the rotated image, and the recognized character string including the plurality of target character strings. A program that causes a computer to execute the character extraction method according to claim 7.

Images