JP2018137636A - Image processing device and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique capable of executing correction processing with high accuracy so that a precise looked-down image can be obtained for a card region regarding image processing related to photographic image data on a card or the like for identity check.SOLUTION: An image processing device includes an image correction unit that applies image correction processing to an image of photographic image data on an input identity card and outputs post-correction image data in a looked-down state from a vertical direction of a card surface. The image correction unit detects (S22) a color boundary point based on difference of colors between a card region and a background region from the image during the image correction processing (S2), detects (S26) the card region on the basis of the color boundary point, makes a conversion (S27) so that a trapezoidal distortion of the card region becomes a right angle quadrangle, and thereby obtain a post-correction image in the looked-down state.SELECTED DRAWING: Figure 5

Description

  The present invention relates to an image processing technique, and more particularly, to a technique for correcting image data such as an identification card.

  In some cases, the image data of a personal identification card such as a driver's license or health insurance card is used for identity verification. For example, the user images the surface of the card with a camera (such as a camera mounted on a digital camera, a tablet terminal, or a smartphone). The captured image data is input to an apparatus having an OCR (Optical Character Recognition) function. The device is a dedicated OCR device or a PC equipped with OCR software. The apparatus performs image processing including correction processing such as enlargement / reduction on the input image data, and obtains an image on the card surface as corrected image data. Further, the device obtains character information described on the card surface by performing OCR processing on the image.

  In a conventional OCR apparatus, an image is acquired from a sheet by a scanner or the like, and the image in the sheet area is a right-angled square image in an upright state (the image is vertically and horizontally and the like is in the same state). As a premise, OCR processing is performed at a preset reading position. When the image of the paper area is distorted, the accuracy of OCR is lowered.

  JP-A-2011-9986 (Patent Document 1) can be cited as an example of prior art relating to image processing such as identification verification. Japanese Patent Application Laid-Open No. 2004-228561 describes that an identification image capturing system or the like is capable of accurately correcting distortion of an image obtained by capturing an identity card, and the following. In the system, when a face is detected in an image obtained by an imaging device, a photo area is detected, and the aspect ratio of the image after tilt correction is corrected using the size of the obtained photo.

JP 2011-9986 A

  It is desirable that information such as characters written on the card surface can be recognized by a person or a computer or can be easily recognized in the imaged image data of the personal identification card by the camera. For this purpose, it is basically desirable to use an image (may be referred to as a bird's-eye view image) that is taken from a direction perpendicular to the card surface. In the overhead view image, ideally, the card surface is shown as a right-angled square.

  However, at the time of image capturing by the user, a tilt angle deviation from the vertical direction may occur with respect to the image capturing direction with respect to the card surface. When there is a deviation, the card surface is captured as a trapezoidally distorted image in the captured image instead of a right-angled quadrangle. In the case of image data having such distortion, information such as characters is difficult to recognize. Therefore, in an image processing apparatus such as an OCR apparatus or a PC, it is desirable to perform image processing including predetermined correction processing on the captured image so that an image suitable for predetermined processing such as OCR is obtained. In the correction process, for example, a quadrangle corresponding to an image area on the card surface (may be described as a card area) is detected from the captured image. In the correction processing, when the quadrangle has a trapezoidal distortion, the trapezoidal distortion is converted into a right-angled quadrangle. An accurate overhead view image is obtained by such distortion correction processing.

  When performing a correction process on the captured image data, the conventional image processing apparatus has problems in terms of accuracy and the like as follows. When the accuracy of the correction process is low, it becomes difficult for a person to visually recognize information such as characters in the image, and the accuracy of OCR by the computer is also reduced.

  (1) It may be difficult to accurately estimate and detect the quadrangle of the card area from the captured image. For example, the square side of the card area may be unclear with respect to the background area of the image, depending on the imaging direction at the time of imaging, lighting conditions, and the like. When the brightness difference, that is, the color difference between the background area and the card area of the image is small, the side of the rectangle, that is, the boundary line between the background area and the card area does not appear clearly. In many cases, it is difficult to accurately detect four sides and four corner points corresponding to the quadrangle of the card area from the image. That is, the detection accuracy of the card area is low. If the card area cannot be detected accurately, the accuracy in correcting the trapezoidal distortion is also lowered.

  (2) For example, in the case of a driver's license or the like, the surface layout has a relatively large amount of feature information such as ruled lines, marks, and photographs. Therefore, correction processing can be performed using the feature information. On the other hand, in the case of a specific type of card such as a health insurance card, for example, characteristic information such as ruled lines, marks, photographs, etc. is relatively small as a prescribed layout on the surface. In the card, description items (name, etc.) are prescribed, but the individual difference for each card is relatively large with respect to the character arrangement position for each description item. Even when the pixels at the same position coordinates on the card surface of each user are viewed, they often have different colors (pixel values). In the case of such a card, it is difficult to use the feature information and to detect the card area during the correction process. That is, the accuracy of the correction process is lowered.

  In Patent Document 1, detection is performed using a facial photograph in the card as a feature. However, in the case of a card without a facial photograph, when it is difficult to detect a boundary line of a facial photograph, This is not effective when there are large individual differences. In Patent Document 1, contour extraction and Hough transform are used for card area detection. However, when the color difference between the background area and the card area is small, or when the brightness in the image is uneven. Is not valid.

  An object of the present invention is to provide a technique capable of performing a high-precision correction process so that an accurate overhead image with respect to a card area can be obtained with respect to an image processing technique related to captured image data such as a personal identification card.

  A typical embodiment of the present invention is an image processing apparatus having the following configuration.

  The image processing apparatus according to the embodiment performs image correction processing on the image of the captured image data of the input identification card and outputs corrected image data in a state where the card surface is viewed from the vertical direction. An image correction unit, wherein the image correction unit detects a color boundary point based on a color difference between a card area and a background area from the image in the image correction process, and based on the color boundary point Then, the card area is detected and converted so that the trapezoidal distortion of the card area becomes a right-angled square, thereby obtaining the corrected image data in the overhead view.

  According to a typical embodiment of the present invention, high-accuracy correction processing can be performed with respect to an image processing technique related to captured image data such as a card for identity verification so that an accurate overhead image for a card area can be obtained.

It is a figure which shows the structure of the system containing the image processing apparatus of embodiment of this invention. In embodiment, it is a figure which shows the structural example of the layout of the surface of the object card | curd. In an embodiment, it is a figure showing an outline of use in a system. In an embodiment, it is a figure showing an outline of processing of each part of an image processing function in an OCR device. In an embodiment, it is a figure showing a flow of image processing in an OCR device. In an embodiment, it is a figure showing an example of an image corresponding to input image data. In an embodiment, it is a figure showing an example of other pictures. In an embodiment, it is a figure showing an example of other pictures. In an embodiment, it is a figure showing the 1st example of the state of the picture concerning S21. In embodiment, it is a figure which shows the 2nd example of the state of the image concerning S21. In an embodiment, it is a figure showing a state of an image concerning processing of S22. In an embodiment, it is a figure showing a state of an image concerning processing of S23. In an embodiment, it is a figure showing a state of an image concerning processing of S24. In an embodiment, it is a figure showing the state of the image concerning processing of S25 and processing of S26. In an embodiment, it is a figure showing a state of an image concerning processing of S27. In embodiment, it is explanatory drawing about the detail of the process of S22. It is a figure which shows the graph which plotted the color difference of S22 in embodiment. In embodiment, it is explanatory drawing about the detail of the process of S23. In embodiment, it is explanatory drawing about the detail of the 3rd process example of S23. In embodiment, it is explanatory drawing about the detail of the process of S24. In embodiment, it is explanatory drawing about a noise reduction process.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that components having the same function are denoted by the same reference symbols throughout the drawings for describing the embodiment, and the repetitive description thereof will be omitted.

(Embodiment)
The image processing apparatus according to the embodiment of the present invention will be described with reference to FIGS.

[Image processing apparatus (1)]
FIG. 1 shows a system configuration including an image processing apparatus according to an embodiment. The image processing apparatus according to the embodiment is configured as an OCR apparatus 1. This system has an OCR device 1 and a camera 2. The user uses the OCR device 1 and the camera 2. The user has a personal identification card 3 such as a health insurance card. Note that the user who uses the two devices of the OCR device 1 and the camera 2 may be the same user or different users. For example, a resident may take an image of his / her card 3 with the camera 2, and a staff member of an administrative organization may input the captured image data into the OCR device 1.

  The OCR device 1 is a dedicated device having a scanner function and an OCR function, and further an image processing device having a unique image processing function. Known techniques can be applied to the scanner function and the OCR function, and a description thereof will be omitted.

  The user images the surface of the card 3 with the camera 2. The camera 2 may be a digital camera or a camera function provided in a tablet terminal, a smartphone, or the like. The camera 2 can acquire, store, and output captured image data of the card 3. The user inputs the image data 4 </ b> A of the card 3 obtained by the camera 2 to the OCR device 1. The OCR device 1 inputs the image data 4A by an arbitrary means. For example, the OCR device 1 may be communicatively connected to the camera 2 and input image data 4A from the camera 2 through communication. Alternatively, the OCR device 1 may be connected to a PC or the like via a communication network and input the image data 4A from the PC or the like in which the image data 4A of the camera 2 is stored. Alternatively, the OCR device 1 may be connected to a memory card storing the image data 4A of the camera 2 and read and input the image data 4A from the memory card.

  The OCR device 1 performs image processing including correction processing on the input image data 4A (image data 122 of the storage device 102). This correction processing includes specific distortion correction processing in addition to known processing such as enlargement / reduction, position shift, and rotation. This distortion correction process is a process of converting the trapezoidal distortion of the card area in the image so that it becomes a right-angled overhead view image. As a result, image data suitable for OCR processing and human visual recognition is obtained as the corrected image data. The OCR device 1 performs OCR processing on the corrected image data, and obtains character data 123 as a result.

  The OCR device 1 includes an arithmetic device 101, a storage device 102, a scanner device 103, a communication interface device 104, an input / output interface device 105, an input device 106, a display device 107, an external storage device 108, etc., which are connected via a bus. Connected.

  The arithmetic device 101 includes a CPU, RAM, ROM, and the like, and realizes an image processing function based on software program processing by the CPU or the like. Specifically, the CPU reads the image processing program 121 in the storage device 102 into the memory and executes processing according to the image processing program 121. Thereby, each unit such as the image reading unit 11 constituting the image processing function is realized. The image processing function of the arithmetic device 101 includes, as each unit, an image reading unit 11, an image data input unit 12, an image correction unit 13, an optical character recognition unit 14, a character data output unit 15, an image data output unit 16, a setting unit 17, and the like. including.

  The storage device 102 stores an image processing program 121, image data 122, character data 123, setting information 124, and the like. The image processing program 121 may be stored and installed in the OCR device 1 in advance, or may be downloaded from a server on a communication network and installed. The image data 122 is image data based on the input of the image data 4A obtained by capturing the surface of the card 3 by the camera 2, and includes each image data before and after correction. The character data 123 is data including character information extracted by OCR processing based on the image data 122. The setting information 124 is setting information related to the image processing function.

  The image reading unit 11 controls the scanner device 103 to read an image from a sheet. The scanner device 103 has a known mechanism. In the embodiment, the image data 4A of the camera 3 is used instead of the image from the paper.

  The image processing apparatus is not limited to the dedicated OCR apparatus 1 described above. As a modification, it may be configured by PC and OCR software. The OCR software corresponds to an image processing program having the aforementioned OCR function and image processing function. The PC reads and executes an image processing program stored in a storage or the like by a CPU or the like, and similarly realizes the image correction unit 13 or the like in FIG.

[Image processing apparatus (2)]
The image processing apparatus according to the embodiment pays attention to the fact that there is a color difference between the background area and the card area in the image data of the card 3. The image processing apparatus detects the position of the card area in the image by searching and estimating the boundary lines (four sides of the quadrangle) between the background area and the card area. The image processing apparatus searches for a color boundary point that is a boundary point of a color difference on a search line that connects an end point and a center point of an image frame. The image processing apparatus collects and groups a plurality of color boundary points obtained by the search. The image processing apparatus draws an approximate line associated with the side of the card area for each group. The image processing apparatus detects four corner points of the card area using four approximate lines. The image processing apparatus performs a known projective transformation based on the detected four corner points of the card area, thereby converting the trapezoidal distortion of the card area into a right-angled square. As a result, a high-accuracy bird's-eye view image, that is, an image captured from a direction perpendicular to the card surface is obtained.

  The image processing apparatus performs a known process such as enlargement / reduction, position shift, rotation, and clipping from the corrected image including the right-angled rectangle as necessary, and includes an upright card area. A typical bird's-eye view image. For example, the image processing apparatus performs enlargement / reduction so that the corrected image becomes an image having a prescribed aspect ratio of the card. In the final bird's-eye view image, the card area is obtained as a right-angled rectangle having a predetermined suitable size, ratio, position, orientation, and the like.

  The image processing apparatus according to the embodiment sets a radially expanding line connecting a center point of a rectangular frame in a card image and a plurality of end points on the frame line as a search line (radiation). The image processing apparatus searches for a color boundary point that is a boundary point between the color of the background area and the color of the card area on the search line. The image processing apparatus calculates a vector between color boundary points for a plurality of color boundary points obtained by the search. The image processing apparatus divides the plurality of color boundary points into four groups based on the similarity of angles representing the direction of the vector. A group is associated with a side of the card area. The image processing apparatus detects the four sides of the card area by drawing an approximate line using the color boundary points for each group. The image processing apparatus detects four corner points of the card area as four intersections on the four sides. Based on the four corner points, the image processing apparatus performs projective transformation so that the trapezoidal distortion becomes a right-angled square.

  The image processing apparatus according to the embodiment can improve the detection accuracy of the card area and obtain an accurate bird's-eye view image by image processing related to imaged image data by the camera 2 of the card 3 such as a health insurance card. In particular, even in the case of an image having a trapezoidal distortion, such as the direction and illumination state when the card 3 is photographed by the camera 2, the card area detection accuracy and distortion correction accuracy can be increased. In particular, even in the case of an image having a trapezoidal distortion with a type of card 3 having little feature information in the layout, it is possible to increase the card area detection accuracy and the distortion correction accuracy.

[Identification card]
FIG. 2 shows the case of a health insurance card as a configuration example of the layout of the surface of the target card 3. In FIG. 2, the layout is shown in a simplified manner in monochrome, but an actual card has a predetermined color design. This card 3 has many character items and few feature information such as ruled lines, marks and photographs. The card 3 has a rectangular shape on the surface, has a prescribed size, has a vertical length V0, a horizontal length H0, and a thickness, and has four corner points rounded by a round process.

[Image processing apparatus (3)]
FIG. 3 shows an outline of use in a system including the image processing apparatus according to the embodiment. FIG. 3A shows an example in which the user images the card 3 with the camera 2. For example, in the case of a driver's license as a card, there are many characteristic information such as ruled lines in the card surface. In the case of a health insurance card as shown in FIG. 2, for example, the card 3 has little feature information such as ruled lines on the surface of the card 3. In the embodiment, effective image correction is realized even in the case of a card 3 of a type having a small amount of feature information.

  FIG. 3B shows an example of image data input. The input image data 4A has a card area on the background area. In this example, the card area has a trapezoidal distortion. This is caused by a shift in the imaging direction during imaging. Further, in this example, the brightness difference, that is, the color difference between the background area and the card area is small, and the boundary lines of the four sides of the card area are unclear.

  FIG. 3C shows an example of corrected image data obtained by image correction. In this corrected image data, the card area is acquired as an accurate overhead image. An accurate bird's-eye view image is an image in which the card area is a right-angled square. In this bird's-eye view image, it is easy for a person to visually recognize the written character information in the card surface. Moreover, in this bird's-eye view image, the character information in the card surface is easily recognized by OCR processing. The corrected image data may be an image obtained by cutting out the background area and cutting out only the card area. The corrected image data and the character information by OCR can be used for a predetermined use. For example, it can be used for documents to be submitted in administrative agencies.

[Image processing apparatus (4)]
FIG. 4 shows an outline of processing of each part of the image processing function in the OCR apparatus 1 which is the image processing apparatus of the embodiment. The image data input unit 12 inputs image data 4A to be subjected to image processing based on a user operation. For example, the image data input unit 12 inputs the image data 4A of the camera 2 through the communication interface device 104 or the input / output interface device 105, and stores the image data 122 in the storage device 102 as the image data 122, particularly the pre-correction image data 4B. The image data input unit 12 reads out the pre-correction image data 4B from the image data 122 of the storage device 102 as necessary, and inputs it to the image correction unit 13.

  The image correction unit 13 performs image processing including predetermined correction processing on the pre-correction image data 4B that is input image data. The image correction unit 13 performs image processing based on the setting information 124 by the setting unit 17. The image correction unit 13 outputs the corrected image data 4 </ b> C and stores it in the storage device 102.

  The optical character recognition unit (OCR unit) 14 performs a known optical character recognition process (OCR process) on the corrected image data 4C, which is input image data, and obtains character data 123 as a result. The optical character recognition unit 14 outputs the character data 123 and stores it in the storage device 102.

  The character data output unit 15 outputs the character data 123 to the user or an external device based on a user operation. For example, the character data output unit 15 displays the contents of the character data 123 on the screen of the display device 107. For example, the character data output unit 15 transmits the character data 123 to the external device via the communication interface device 104.

  The image data output unit 16 outputs the corrected image data 4C to the user or an external device based on a user operation. For example, the image data output unit 16 displays the content of the corrected image data 4C on the screen of the display device 107. For example, the image data output unit 16 transmits the corrected image data 4C to the external device via the communication interface device 104.

  The setting unit 17 provides a setting screen to the user, and performs user settings related to the image processing function based on a user operation. The setting unit 17 stores the setting information 124 in the storage device 102.

[Image processing flow]
FIG. 5 shows a flow of image processing in the OCR device 1. This flow shows a flow in the case where character data is output by performing OCR processing after image correction processing. FIG. 5 includes steps S1 to S5. Step S2 includes steps S2A to S2C. Step S2A includes steps S21 to S27. Hereinafter, it demonstrates in order of a step.

  (S1) As shown in FIG. 4, the image data input unit 12 of the OCR device 1 inputs the image data 4A from the camera 2 as pre-correction image data 4B.

  (S2) The image correction unit 13 of the OCR device 1 performs an image correction process on the input uncorrected image data 4B to obtain corrected image data 4C. The image correction processing in S2 includes step S2A for performing specific distortion correction processing, step S2B for performing processing such as enlargement / reduction, and step S2C for performing processing such as position shift, rotation, and clipping. S2B and S2C are known processes.

  (S2A) The image correction unit 13 performs a specific distortion correction process in S2A. This distortion correction process is a process of detecting the card area and position and correcting the trapezoidal distortion of the card area so that it becomes a right-angled square.

  (S21) When performing the distortion correction processing in S2A, the image correction unit 13 first calculates a plurality of end points in the image of the pre-correction image data 4B in S21. The image correction unit 13 sets each line connecting the center point of the image and each of the plurality of end points as a search line (radiation). Details are shown in FIG.

  (S22) Based on the search line of S21, the image correction unit 13 searches for a color boundary point that is a boundary point of the color difference between the background area and the card area on each of the plurality of search lines. The image correction unit 13 repeats the processing in the same manner in order from a certain search line. Thereby, a plurality of color boundary points are obtained. Details are shown in FIG.

  (S23) The image correction unit 13 calculates a vector between the color boundary points based on the color boundary points in S22. Details are shown in FIG.

  (S24) The image correction unit 13 groups color boundary points based on the vector of S23. Groups are associated with square sides of the card area. Details are shown in FIG.

  (S25) The image correction unit 13 calculates an approximate straight line based on the group of S24. The approximate line is associated with a square side of the card area. Details are shown in FIG.

  (S26) The image correcting unit 13 calculates the four corner points of the quadrangle of the card area by calculating the intersection based on the approximate straight line of S25. Details are shown in FIG.

  (S27) Based on the four corner points of S26, the image correction unit 13 corrects the square trapezoidal distortion of the card area so as to become a right-angled square by a known projective transformation process. Details are shown in FIG.

  (S2B) Based on the image after the projective transformation obtained in S2A and the four corner points, the image correction unit 13 causes the square of the card area to be a square having a specified aspect ratio and a suitable size in S2B. In addition, image enlargement / reduction processing or the like is performed.

  (S2C) Based on the image obtained in S2B, the image correction unit 13 performs a position shift process in S2C so that the square position of the card area in the image is a suitable position. Further, the image correction unit 13 performs in-plane rotation processing so that the card area in the image has a preferable orientation. Further, when only the card area is cut out from the image, the image correction unit 13 performs a process of cutting out the card area. Note that the processing of S2B and S2C can be omitted as appropriate.

  (S3) Based on the corrected image data 4C obtained in S2, the OCR unit 14 first detects a reading position and region for OCR. The reading position and area are set by user settings, for example. For example, the origin of the input image is the upper left point of the rectangle. A point (x1, y1) at a predetermined position coordinate is set as the reading start position (the upper left point of the reading area). A point (x2, y2) at a predetermined position coordinate is set as the reading end position (the lower right point of the reading area).

  (S4) The OCR unit 14 performs OCR processing on the read area of the corrected image data 4C to extract characters, and obtains character data 123 as a result.

  (S5) The OCR unit 14 performs OCR post-processing based on the character data 123 obtained in S4. The character data output unit 15 displays the resulting character information on the screen of the display device 107. The user checks and checks the character information displayed on the screen, and corrects if there is a problem.

[Input image example (1)]
FIG. 6 shows an example of the image 5 corresponding to the input image data 4A (pre-correction image data 4B) related to S21. The image 5 has a rectangle, and the upper left point is the origin (0, 0) of the position coordinates. The position coordinates of the pixel in the image 5 are represented by (x, y). The X direction is the in-plane horizontal direction (lateral direction), and the Y direction is the in-plane vertical direction (vertical direction). It is assumed that the horizontal length X1 and the vertical length Y1 of the image. The position coordinates of the upper right corner point are (X1, 0), the lower left corner point is (0, Y1), and the lower right corner point is (X1, Y1). The center point of the image 5 is a point G0, and the position coordinates are (X1 / 2, Y1 / 2). For the sake of explanation, the center point of the card area 7 in the image 5 is defined as a point C0. In the image 5, there are a background area 6 and a card area 7. Although the image 5 is simplified and shown in monochrome for the sake of explanation, it is actually full color and has gradation and unevenness depending on the situation such as illumination at the time of imaging.

  In the example of FIG. 6, the image 5 shows a case where the card region 7 is distorted in a trapezoidal shape due to a certain tilt angle with respect to the direction perpendicular to the card surface as the imaging direction. In this example, the aspect ratio of the card area 7 is different from the prescribed aspect ratio, and the X direction is slightly enlarged. Further, in this example, a case where the position of the card area 7 in the image 5 is slightly shifted from the center point G0 of the image 5 is shown.

[Input image example (2)]
FIG. 7 shows an example of another image 5. In this example, a case where there are many portions in the image 5 where the boundary line between the background region 6 and the card region 7, that is, the four sides which are the edges of the surface of the card 3, is unclear. For example, the upper and lower sides of the card area 7 are hardly shown. In the case of such an image, it is difficult to accurately detect the card area 7 with the conventional technique.

[Input image example (3)]
FIG. 8 shows an example of another image 5. In this example, the distortion of the trapezoidal shape of the card area 7 is small in the image 5, but the card area 7 is arranged at a position near the upper left in a state of being rotated obliquely in the plane, and the size of the image 5 On the other hand, the size of the card area 7 is relatively small. In the case of such an image 5, it is necessary to perform processing such as enlargement, position shift, and rotation as correction processing.

[Image processing (1) -S21]
FIG. 9 shows a state in which an end point and a search line are set as the state of the image 5 related to the image end point calculation processing in S21. The image correction unit 13 sets predetermined end points (also referred to as image end points) on the four frame lines of the rectangular frame of the image 5. The end point is represented by pi and is represented by a small square. In this example, in the coordinate system, the middle point of the right side of the image 5 is set as the end point p1 (position coordinates are (X1, Y1 / 2)), and the four sides are advanced counterclockwise from that point. Set the end point of. The image correction unit 13 calculates the position coordinates of each end point. In this example, a case where a total of 32 end points p1 to p32 are set is shown, but this is not a limitation. For example, the first end point p1 may be set near the origin (0, 0).

  On the right side of the image 5, nine end points {p1 to p5, p29 to p32} including the corner points are equally spaced between the upper right corner point (end point p5) and the lower right corner point (end point p29). It is set to (1/8 of the vertical length Y1). On the upper side, between the upper right corner point (end point p5) and the upper left corner point (end point p13), the nine end points {p5 to p13} including the corner points are equally spaced (8 in the horizontal length X1). 1). Although the case where the equal interval on the right side and the equal interval on the upper side are different is shown, the present invention is not limited to this and can be set. Similarly, nine end points {p13 to p21} are set on the left side, and nine end points {p21 to p29} are set on the lower side.

  The image correction unit 13 uses the center point G0 of the image 5 when setting the search line. The image correction unit 13 sets a line connecting the center point G0 and each of the plurality of end points as a search line (radiation). Search lines extend radially from the center point G0. A search line (radiation) is represented by Li and is represented by a broken line. In this example, search lines L1 to L32 are set. One end of the search line Li is the center point G0, and the other end is the end point pi. In this example, end points are also set at corner points of the rectangular frame of the image 5.

  FIG. 10 shows another setting example of the end points and search lines of the image 5. A case where search lines are set around the center point G0 at intervals of a constant angle α (for example, about 11 degrees) is shown. Thereby, a plurality of end points (a total of 32) are set on the border of the rectangular frame of the image 5. In this example, end points are not set at corner points of the rectangular frame of the image 5.

[Image processing (2) -S22]
FIG. 11 shows the state of the image 5 related to the color boundary point search processing in S22. A state in which the color boundary point is searched in the direction from the end point to the center point G0 on each search line is shown. A color boundary point between the background area 6 and the card area 7 is represented by qi, and is represented by a small rhombus. A color boundary point qi on the search line Li between the center point G0 and the end point pi. The search is performed in order from the search line L1 of the end point p1 to the search line L32 of the end point p32 in the counterclockwise direction. In 32 search lines L1 to L32, color boundary points q1 to q32 are obtained. In this example, a case where 32 color boundary points are successfully detected is shown, but some color boundary points may not be detected well.

[Image processing (3) -S23]
FIG. 12 shows a vector or the like as the state of the image 5 related to the vector calculation process of S23. Adjacent color boundary points at a plurality of color boundary points {q1 to q32} are connected by a vector. Vectors between color boundary points are indicated by vectors v1 to v32. In this example, a case where 32 vectors are successfully detected is shown. For example, the vector v1 is a vector having the color boundary point q1 as the start point and the color boundary point q2 as the end point, and has information on the angle in the plane.

  In this example, 32 vectors having the same number as the color boundary points are set by setting vectors between adjacent color boundary points between the 32 color boundary points. This is not limited to this. In a modification described later, it is possible to set a smaller number of vectors than the number of color boundary points, for example, by setting vectors by skipping color boundary points one by one.

[Image processing (4) -S24]
FIG. 13 shows a group or the like as the state of the image 5 related to the grouping process in S24. Four groups are indicated by g1 to g4. In this example, there are six color boundary points q1 to q4, q31 and q32 in the vicinity of the right side of the card area 7, and five vectors v1 to v3, v31 and v32 connecting them. The five vectors v1 to v3, v31, and v32 have similar angles (in other words, the angle difference is less than a predetermined value). Therefore, six color boundary points corresponding to them are grouped as one group g1. Similarly, in the vicinity of the upper side of the card area 7, there are nine color boundary points q5 to q13, eight vectors v5 to v12 connecting them, and the nine color boundary points are one. Grouped as group g2. Near the left side of the card area 7, there are six color boundary points q14 to q19, five vectors v14 to v18 connecting them, and these six color boundary points are set as one group g3. Grouped. In the vicinity of the lower side of the card area 7, there are ten color boundary points q20 to q30, nine vectors v20 to v29 connecting them, and these ten color boundary points are grouped as one group g4. Grouped.

  It should be noted that the vectors v4, v13, v19, v30 in the vicinity of the corner points of the card area 7 are not similar in angle to other vectors (in other words, the angle difference is greater than or equal to a predetermined value). Therefore, the color boundary points related to these vectors are excluded from consideration when grouping.

[Image processing (5) -S25]
FIG. 14 shows approximate straight lines, intersections, and the like as the states of the image 5 related to the approximate straight line calculation processing in S25 and the intersection calculation processing in S26. The image correction unit 13 draws an approximate line using the color boundary point for each group. In FIG. 14, an approximate straight line M1 is set in the image 5 using the color boundary points of the group g1. Similarly, an approximate straight line M2 is set using the color boundary points of the group g2. An approximate straight line M3 is set using the color boundary points of the group g3. An approximate straight line M4 is set using the color boundary points of the group g4. The approximate straight line can be calculated using, for example, a known least square method (a method for minimizing the sum of squares of residuals).

[Image processing (6) -S26]
The image correction unit 13 calculates the intersection of the approximate lines. In the image 5 of FIG. 14, four intersection points Q1 to Q4 are calculated using four approximate straight lines M1 to M4. For example, the intersection Q1 between the approximate straight line M1 and the approximate straight line M2. The intersection is indicated by a double circle. These four intersections Q1 to Q4 are points estimated as four corner points of the card area 7.

[Image processing (7) -S27]
FIG. 15 shows the state of the image 5 related to the projective transformation in S27. The image correction unit 13 performs a known projective transformation using the four intersection points Q1 to Q4. The trapezoid indicated by the four points before conversion becomes a right-angled square indicated by the four points after conversion. FIG. 15A shows four intersections Q1 to Q4 in the image 5 before projective transformation. A point C0a indicates a center point with respect to the intersections Q1 to Q4 of the card area 7 before conversion.

  FIG. 15B shows four intersections Q1b to Q4b in the image 5 after projective transformation. The point C0b indicates the center point with respect to the intersections Q1b to Q4b of the card area 7b after conversion. The card area 7b indicates an area within a right-angled rectangle defined by the intersections Q1b to Q4b after conversion. The background area 6b indicates an area other than the card area 7b.

  Thereafter, the image correction unit 13 appropriately performs the above-described S2B enlargement / reduction and S2C processing. The image correction unit 13 outputs the image 5 in FIG. 15B or the image of the card area 7b cut out from the image 5 as the corrected image data 4C.

[S22-Details]
Details of the processing of each step will be described.

  FIG. 16 is an explanatory diagram illustrating details of the color boundary point search process in S22. A partial region at the upper right of the image 5 is shown in an enlarged manner. In S <b> 22, the image correction unit 13 specifies a portion where the degree of color change is roughly large in units of blocks in the image 5, and then searches for color boundary points in detail in units of pixels. The process of S22 includes the following steps S22a to S22f.

  (S22a) The image correction unit 13 creates a block that passes on the above-described search line (FIG. 11). In FIG. 16, one search line Li is shown as an example. An end point pi and a color boundary point qi corresponding to the search line Li are shown. The case of the end point pi near the upper right corner point on the upper side of the image 5 is shown. The block is, for example, a square pixel area having a predetermined size. Examples of blocks are indicated by blocks b1 to b6 and the like. For example, the block b1 has the end point pi as a diagonal point. The pixel position through which the search line Li passes in the vicinity of the other diagonal point of the block b1 is the diagonal point (point r1) of the next block b2. The pixel position through which the search line Li passes in the vicinity of the other diagonal point of the block b2 is the diagonal point (point r2) of the next block b3. Similarly, blocks b4 to b6 and the like are set in order. From the result, the block b3 including the color boundary point qi is shown as the nth (#n) block. The block b1 is n-2th (# n-2), the block b2 is n-1th (# n-1), the block b4 is n + 1th (# n + 1), and the block b5 is n + 2 with respect to the nth block. (# N + 2). The search range 160 indicates the portions of the blocks b2, b3, and b4, and includes the blocks b2 and b4 before and after the nth block b3.

  The block is not limited to the above-described configuration, and may be, for example, an oblique state (a shape having a side perpendicular to the direction of the search line), a rectangle, or another shape.

  (S22b) The image correction unit 13 calculates the block representative color (ci) using the pixel values in the block sequentially for each block on the search line Li. As the representative color, for example, an average value, a mode value, or the like can be used as a statistical value of all the pixel values in the block. In this example, the representative color c1 and the like of the block b1 are shown.

  (S22c) The image correcting unit 13 sequentially compares the representative colors between the blocks on the search line Li, and the representative color (c3) of the focused block (eg, #n, b3) is the previous block ( It is determined whether or not there is a significant change from the representative color (c2) of # n-1, b2). In the determination, for example, a predetermined threshold is used. The image correction unit 13 determines whether the difference value between the representative colors of the preceding and succeeding blocks is equal to or greater than the threshold value. When the representative color of the target block has changed significantly from the representative color of the previous block, that is, when the difference value is equal to or greater than the threshold value, the image correcting unit 13 is the previous block (# n−1, b2). To the next block (# n + 1, b4) is set as the next search range 160. The processing up to S22c is processing that roughly specifies a place where the color boundary point qi is likely on the search line Li, and is presumed to be probably in the nth block (#n). The range including the preceding and succeeding blocks with respect to the estimated nth block is set as a search range 160, and the search is performed in detail in the processing subsequent to S22d.

  (S22d) The image correction unit 13 performs the following processing to search for the color boundary point qi for the blocks (for example, b2, b3, b4) in the search range 160. First, the image correction unit 13 uses the background color CB (representative color of the background region 6) as the representative color (c2) of the (n-1) th (or n-2th) block (# n-1, b2). ) As a temporary setting.

  (S22e) The image correction unit 13 uses the pixel value in the n + 1th (or n + 2th) block (# n + 1, b4) to obtain the background color and the black color (defined color) of the character component. The excluded color is temporarily set as the card color CC (representative color of the card area 7). This card color CC corresponds to the basic color of the surface of the actual card 3. The specified colors excluded here are all colors related to character information, marks, and other items described as a specified layout on the surface of the card 3. For example, if there is a red mark, the red color is also removed.

  (S22f) The image correction unit 13 searches for all pixels on the search line Li passing through the block (b2, b3, b4) in the search range 160 from the search start point (point r1) to the search end point (point r4). Do. In the search range 160, the image correction unit 13 calculates a color difference SB between the color of the pixel of interest and the background color CB, and a color difference SC between the color of the pixel of interest and the card color CC, and stores these data values in the memory. Hold temporarily.

  FIG. 17 shows a graph plotting data values of the color difference SB and the color difference SC. The horizontal axis represents pixels on the radiation Li within the search range 160 (from point r1 to point r4), and the vertical axis represents the color difference value. A curve 171 indicates the color difference SB from the background color CB, and a curve 172 indicates the color difference SC from the card color CC. A point 173 indicates an intersection point between the color difference SB curve 171 and the color difference SC curve 172. A pixel corresponding to the point 173 is determined as the color boundary point qi. A point 173 is a point that is far from the background color CB and approaches the card color CC.

  As the processing of S22f, basically, when certainty is given priority, pixel values are examined in order from the search start point (point r1) to the search end point (point r4), but when priority is given to processing speed, When the point 173 is obtained, the pixel may be determined as the color boundary point qi, and the process may be terminated.

  The image correction unit 13 repeats the above processing in the same manner for each of a plurality of search lines. In the above processing example, the search is performed from the end point pi toward the center point G0. However, as a modified example, similarly, the search may be performed from the center point G0 toward the end point pi. In addition, regarding the search for a plurality of search lines, the search processing may be performed in parallel instead of sequentially depending on the configuration of the arithmetic device 101 (for example, when parallel operation is possible).

  As a supplementary explanation, the definition of the color difference (color differences SB and SC of S22) is as follows. Each pixel has gradation values (for example, 256 gradations) of R (red), G (green), and B (blue) colors. The aforementioned background color CB and card color CC also have gradation values of R, G, and B, respectively. During the above-described processing, the Euclidean distance is calculated as the color difference SB between the target pixel value and the pixel value of the background color CB. Similarly, the Euclidean distance is calculated as the color difference SC between the target pixel value and the pixel value of the card color CC. For example, the Euclidean distance that is the color difference SB can be expressed by the following equation. The pixel value of interest is represented by {Kr, Kg, Kb}. Kr and the like indicate gradation values. The pixel value of the background color CB is represented by {K1r, K1g, K1b}. As an equation, √ {(Kr−K1r) ^ 2 + (Kg−K1g) ^ 2 + (Kb−K1b) ^ 2}. Note that the brightness is defined using, for example, the maximum value among the R, G, and B gradation values of the pixel.

[S23-Details]
FIG. 18 is an explanatory diagram for details of the vector calculation processing of S23. In the following, some possible processing examples including modifications are shown.

  FIG. 18A shows a first processing example. The first processing example is the same basic processing as that in FIG. 12, but is a processing for setting a vector between adjacent color boundary points at the closest positions in the image 5. In this example, the upper half area of the image 5 in FIG. 12 is shown. Nine color boundary points q5 to q13 are arranged in the vicinity of the upper side of the card area 7. Eight vectors v5 to v12 are set between adjacent color boundary points q5 to q13. For example, in the case of examining counterclockwise as shown in FIG. 11, the closest color boundary point on the left side with respect to the color boundary point q5 is the color boundary point q6. Therefore, a vector v5 from the color boundary point q5 to the color boundary point q6 is set. Next, the vector v6 is similarly set from the color boundary point q6 to the closest color boundary point q7 on the left side.

  FIG. 18B shows a second processing example as a modification. The second processing example is a process of selecting a color boundary point and setting a vector between the selected color boundary points by skipping N (for example, one) each of a plurality of arranged color boundary points. is there. According to this second processing example, it is possible to reduce the processing time by reducing the color boundary points and vectors to be calculated. In this example, the next color boundary point q7 is selected from the color boundary point q5 by skipping the next color boundary point q6, and the vector va1 from the color boundary point q5 to the color boundary point q7 is set. . Next, the next color boundary point q9 is selected from the color boundary point q7 by skipping the next color boundary point q8, and the vector va2 from the color boundary point q7 to the color boundary point q9 is set. Similarly, vectors va3 and va4 are set. In this case, the obtained vector is about half.

  FIG. 18C shows a third processing example as a modification. In the third processing example, color boundary points are selected by a method of sequentially skipping one by one for a plurality of arranged color boundary points, and a vector is set between the selected adjacent color boundary points. Furthermore, it is set so that an intermediate point between the selected adjacent color boundary points is replaced with a new color boundary point. The purpose of this process is to average the angle calculations for the vectors to improve the vector grouping accuracy when the color boundary points are dense. The new color boundary point which is an intermediate point is indicated by a black dot. In this example, when attention is paid to the color boundary point q6 next to the color boundary point q5, the vector vb1 is set between the color boundary points q5 and q7 before and after the color boundary point q5, and between the color boundary points q5 and q7. The intermediate point is set as a new color boundary point q6b in place of the color boundary point q6 (note that it is shifted because it overlaps). Next, paying attention to the color boundary point q7 next to the color boundary point q6, the vector vb2 is set between the preceding and following color boundary points q6 and q8, and the intermediate point between the color boundary points q6 and q8 is The color boundary point q7b is set as a new color boundary point q7b. Similarly, vectors vb3 to vb7 and new color boundary points q8b to q12b are obtained. In this case, new color boundary points q6b to q12b are used in the subsequent processing.

  FIG. 19 shows details of the third processing example. FIG. 19A shows a lower half area of the image 5 corresponding to the setting example of FIG. 10 described above. An area 191 shows an example of an area where a plurality of color boundary points are relatively densely arranged. An area 192 shows an example of an area where a plurality of color boundary points are relatively sparsely arranged. For example, the area 191 is near the middle of the lower side of the card area 7 and includes five color boundary points. On the other hand, the area 192 has a horizontal width similar to that of the area 191 and is near the right end of the lower side of the card area 7 and includes two color boundary points. As shown in FIG. 11, since the search is performed radially from the center point G0, a region having a dense color boundary point and a region having a sparse color boundary point are generated in the image 5 as described above.

  FIG. 19B shows a matrix enlarged at the pixel level for an example of an area where the color boundary points are dense. FIG. 19B shows a case where a vector is calculated between adjacent color boundary points in the first processing example. Pixels corresponding to the searched color boundary points are indicated by pixels px1 to px6. Vectors between the pixels px1 to px6 at the color boundary points are indicated by vectors vc1 to vc5. In this example, there is a shift in the Y direction with respect to the pixel positions of a plurality of color boundary points. The pixel px4 is shifted upward in the Y direction compared to the other pixels. The vector vc1 between the pixels px1 and px2 has an angle representing the direction of 180 (the same applies to the vectors vc2 and vc5). Here, as an angle reference, it is assumed that the right of the in-plane X axis is 0 degree and the angle is set to 360 degrees counterclockwise. On the other hand, regarding the pixel px4 at the color boundary point having a deviation, the angle is 153 degrees in the vector vc3, and the angle is 207 degrees in the vector vc4.

  When a large number of image end points and search lines are set in S21, the number of color boundary points increases, and the distance between the color boundary points may become very small. As in the above example, in an area where the color boundary points are dense, the angle of the vector may change abruptly even with a difference of one pixel at the position of the color boundary points. In this case, if the determination condition when grouping the vectors in S24 is, for example, an angle difference of 10 degrees or less, the vector vc1 and the vector vc3 are classified into different groups with a difference of one pixel, for example. Become. This is undesirable because it reduces the accuracy of the classification.

  Therefore, in the third processing example, the accuracy of vector grouping is increased. FIG. 19C shows a case where a vector is calculated by applying the third processing example and a new color boundary point is set with respect to an example of an area having a dense color boundary point similar to FIG. Pixels corresponding to the new color boundary point are indicated by pixels px2b to px5b. Vectors set by one skip are indicated by vectors vd1 to vd4. For example, the vector vd1 from the pixel px1 to the pixel px3 has an angle of 180 degrees, and the pixel px2 is the pixel px2b of the new color boundary point. The vector vd2 from the pixel px2 to the pixel px4 has an angle of 170 degrees, and the pixel px3 is the pixel px3b of the new color boundary point. The vector vd3 from the pixel px3 to the pixel px5 has an angle of 180 degrees, and the pixel px4 is the pixel px4b of the new color boundary point. The vector vd4 from the pixel px4 to the pixel px6 has an angle of 190 degrees, and the pixel px5 is the pixel px5b of the new color boundary point.

  As described above, the third processing example is applied in consideration of a region where the color boundary points are dense, and the distance between the color boundary points is secured using the front and rear color boundary points. Thereby, the difference in the change in the angle of the vector is smaller than in the case of the first processing example of (B), and it is possible to suppress the angle from changing suddenly as in the vector vc3. At the plurality of new color boundary points obtained, the positional deviation is small.

  The calculation method of the third processing example may be performed for all the color boundary points in the image 5, for example, the corresponding color boundary when it is determined that the distance between the color boundary points is close using a predetermined threshold value. It may be done only for points. When the third processing example is applied, the accuracy can be further increased.

[S24-Details]
FIG. 20 is an explanatory diagram for details of the grouping process of S24. The image correction unit 13 classifies the color boundary points associated with the vectors into groups based on the similarity of the angles of the vectors using the plurality of vectors obtained as in the example of FIG. 13 described above. . The classified group is associated with the side of the card area 7. FIG. 20 shows a histogram of vector angles in an example of an image (including a large number of color boundary points). The horizontal axis indicates the vector angle (0 to 360 degrees). The vertical axis represents the frequency value. Dashed circles indicate groups 201 to 204 as groups relating to vector angles. The group 201 is a group including a vector whose angle is around 90 degrees, and corresponds to the group g1 on the right side of the card area 7 in FIG. The group 202 is a group including a vector whose angle is around 180 degrees, and corresponds to the group g2 on the upper side. The group 203 is a group including a vector whose angle is around 270 degrees, and corresponds to the group g3 on the left side. The group 204 is a group including a vector whose angle is around 360 degrees (0 degree), and corresponds to the lower group g4.

  The process of S24 includes the following processes, for example. (S24a) The image correction unit 13 creates a histogram as shown in FIG. 20 using the angle of the vector. (S24b) The image correction unit 13 searches for four peak portions related to the angle and the frequency value from the histogram. (S24c) The image correcting unit 13 classifies the four vectors associated with the four peak portions into four groups g1 to g4. (S24d) In each group, the image correction unit 13 classifies a plurality of vectors near the vector corresponding to the peak portion as belonging to the group. The image correction unit 13 classifies the color boundary points associated with the vectors in the group as belonging to the group.

  A plurality of vectors between a plurality of color boundary points on one side of the card area 7 are substantially similar in angle. Therefore, it can be divided into groups based on the similarity. Normally, when the processing is successful, four groups can be detected corresponding to the four sides of the card area 7 in the vertical and horizontal directions.

  Note that, as a modified example related to the processing of S24, grouping may be performed using the position coordinates of the pixels of the color boundary point instead of the vector angle. Alternatively, the grouping may be performed using both the vector angle and the pixel coordinate of the color boundary point.

[Noise reduction processing]
FIG. 21 is an explanatory diagram of a processing example for noise reduction. By performing this process, the image processing apparatus according to the embodiment can reduce noise in the image 5 and improve accuracy. FIG. 21A shows an example of uneven brightness and noise in a part of the image 5. In this example, a case where the brightness is uneven in the X direction in the image 5 is shown by a simple expression (the unevenness in the card area 7 is omitted). The region on the right side in the X direction is brighter (that is, the color is closer to white), and the region on the left side is darker (that is, the color is closer to black). In this example, noise is included in the image 5. Noise points are indicated by white or black dots.

  FIG. 21B shows a known blur filter as an example of a filter applied to the image 5 for noise reduction. The image correction unit 13 applies a blur filter for each pixel when processing the input image 5. The left side shows nine pixels including a certain pixel (pix5) and surrounding pixels. The right side shows the configuration of a Gaussian filter that is a blurring filter. A filter coefficient is shown for each pixel. For example, for the center pixel (pix5), it indicates that the value of 4/16 of the pixel value is used. For pixels on the top, bottom, left and right (pix2, pix4, pix6, pix8), it indicates that the value of 2/16 of the pixel value is used. For pixels (pix1, pix3, pix7, pix9) at diagonal corner points, it indicates that a value of 1/16 of the pixel value is used. A total value obtained by multiplying each pixel value by a coefficient is applied as a new pixel value of the center pixel (pix5).

  By applying the blur filter, noise in the image 5 is reduced. The image correction unit 13 uses the image 5 to perform processing such as the search for the color boundary point described above. Thereby, the precision of detection of the card area | region 7 etc. can be improved.

  In the image processing system according to the embodiment, unlike the prior art, the boundary line is detected using the color difference between the background area 6 and the card area 7 in the image 5. Therefore, the image processing method according to the embodiment is effective with respect to variations in the combination of the color of the background area 6 and the color of the card area 7 in the image 5. However, the image captured by the camera 2 may include brightness unevenness and noise as in the above example. When viewed at the pixel level, the color of the pixel is the color of the background region 6 or the color of the card region 7. It may be difficult to determine whether it is noise or not. In other words, it may be difficult to determine whether a color change between pixels corresponds to a change from the background area 6 to the card area 7 or a change due to other factors.

  Therefore, in the embodiment, first, as described above, by performing the process of examining the color change in units of blocks, the influence of brightness unevenness and noise is reduced. Furthermore, by applying the noise reduction processing using a filter as described above, the influence of brightness unevenness and noise is reduced.

[Effects]
As described above, according to the image processing apparatus of the embodiment, with respect to the image processing related to the captured image data of the card 3 of the identity verification card, high-precision correction processing is performed so that an accurate overhead image with respect to the card area 7 can be obtained. it can. In particular, even in the case of an image in which the card 2 is imaged by the camera 2 and the lighting state are not appropriate and the card area 7 has a trapezoidal distortion, the detection accuracy of the card area 7 can be increased, and distortion correction processing can be performed. The accuracy can be increased. In particular, even in the case of a type of card 3 with little feature information in the layout, the detection accuracy of the card area 7 and the accuracy of distortion correction processing can be increased. According to the image processing apparatus of the embodiment, even when there are a lot of variations in the layout and color of the surface of the card 3, the background color at the time of shooting, etc. The accuracy of correction can be made higher than that of the prior art.

[Comparative example]
As a comparative example for the image processing apparatus according to the embodiment, the conventional image processing apparatus performs the following processing when detecting four sides of the card area from the image. The image processing apparatus generates a binary image (an image obtained by converting pixel values into black and white binary) from a full-color image that is an input image. The image processing apparatus detects straight line portions of four sides of the card area from the binary image, and detects intersections of the straight lines as four corner points of the card area. The image processing apparatus corrects the shape of the card area to be a right-angled square by performing projective transformation using these four corner points. However, when the four sides of the card area are not clearly shown in the binary image, the correction cannot be performed or the accuracy is lowered. For example, when the difference in brightness between the card area and the background area is small or the brightness is uneven, a binary image in which straight lines of four sides are reflected cannot be generated.

  In the case of the technique such as Patent Document 1, the face photo area in the card area is used as a feature, and the shape, size, and ratio of the card area are determined using the shape, size, and ratio of the face photo area. It can be corrected. However, when the face photograph area cannot be detected well, the correction cannot be performed or the accuracy is lowered.

[Modification]
Examples of modifications of the image processing apparatus according to the embodiment include the following. The known processes of S2B and S2C in FIG. 5 may be performed before the process of S2A as necessary.

  In the embodiment, the search line is set from the center point G0 of the image 5. As a modification, the center point (point C0 in FIG. 6) of the trapezoidal area corresponding to the card area 7 in the image 5 may be approximate position coordinates, and is detected and set by an arbitrary method. The search line Li and the end point pi are similarly set from the center point C0 of the trapezoidal region.

  The number of search lines Li and end points pi may be set arbitrarily, and may be set as appropriate in consideration of the balance between accuracy and processing speed. Various settings may be a design item on mounting, or user settings may be possible. The setting unit 17 manages setting information 124 for various settings.

  In the embodiment, when searching for the color boundary point qi in S22, a fixed set value is uniformly used as a determination threshold for each of the plurality of search lines Li. Not only this but in a modification, you may apply a different threshold value for every search line Li. The setting information 124 includes the threshold value. Further, it is assumed that the color boundary point qi cannot be detected as a result of determination using a certain threshold on a certain search line Li. In that case, the threshold value may be changed to another value, and the search on the search line Li may be retried. For example, the reference threshold value is applied uniformly at the beginning. At the time of retry, the changed threshold value is determined and applied by increasing / decreasing by a predetermined value in each of positive and negative directions from the reference threshold value.

  If a sufficient number of color boundary points are detected even if some of the plurality of search lines cannot be detected in the search in S22, the processing from S23 is established. . That is, four approximate straight lines S25 and four corner points S26 are obtained with a certain degree of accuracy. In consideration of accuracy, a minimum number for this sufficient number of color boundary points may be set. For example, the minimum number is 2 for each side of the image 5. When two color boundary points can be detected on the side, an approximate straight line connecting the two points can be drawn.

  Further, in the search of S22, if the color boundary points can be detected at a predetermined rate (%) or more for each group associated with the side out of the total number of end points and search lines, it is determined to be established. Also good. If true, four approximate straight lines and corner points are calculated using these color boundary points. In addition, when a predetermined number or more of color boundary points cannot be detected for each group of sides and it is not established, the threshold value may be changed and a retry may be performed.

  Although the present invention has been specifically described above based on the embodiments, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the invention.

  DESCRIPTION OF SYMBOLS 1 ... OCR apparatus, 2 ... Camera, 3 ... Card, 5 ... Image, 6 ... Background area | region, 7 ... Card area | region, 13 ... Image correction part.

Claims (6)

An image correction unit is provided that performs image correction processing on the image of the captured image data of the input identity verification card and outputs corrected image data in a state where the card surface is viewed from the vertical direction;
The image correction unit detects a color boundary point based on a color difference between a card area and a background area from the image and detects the card area based on the color boundary point in the image correction process. , By performing the conversion so that the trapezoidal distortion of the card area becomes a right-angled square, to obtain the corrected image data in the bird's-eye view state,
Image processing device. The image processing apparatus according to claim 1.
The image correction unit
Setting a plurality of search lines connecting between a point in the image and a plurality of end points on a border of the image;
Searching the color boundary points on each of the plurality of search lines to detect a plurality of color boundary points;
Classifying the plurality of color boundary points into four groups based on the similarity of vectors connecting the color boundary points in the plurality of color boundary points;
For each of the four groups, four approximate lines are detected by drawing approximate lines using the color boundary points to which the groups belong;
Based on the four approximate lines, four intersections are detected,
By performing projective transformation based on the four intersections, the corrected image data in the overhead state is obtained.
Image processing device. The image processing apparatus according to claim 2.
The image correction unit
Setting a block of pixels on the search line;
Calculate a representative color for each block,
Determining a search range including a block having a large degree of change in the representative color for each block;
Set the background color of the background area,
Set the card color of the card area,
On the search line of the search range block, for each pixel, calculate a first color difference with the background color and a second color difference with the card color;
An intersection of the first color difference and the second color difference is determined as the color boundary point;
Image processing device. The image processing apparatus according to claim 2.
The image correction unit
In the plurality of color boundary points, calculating the vector connecting adjacent color boundary points or color boundary points skipped by N pieces,
Classify into the four groups based on a histogram of angles representing the orientation of the vectors;
Image processing device. The image processing apparatus according to claim 4.
The image correction unit
Replacing the pixel at the midpoint of the vector connecting the color boundary points skipped by N as new color boundary points;
Image processing device. An image processing program for causing an image processing apparatus to perform image processing,
A program that realizes an image correction unit that performs image correction processing on an image of imaged image data of an input identity verification card and outputs corrected image data in a state where the card surface is viewed from the vertical direction. Prepared,
The image correction unit detects a color boundary point based on a color difference between a card area and a background area from the image and detects the card area based on the color boundary point in the image correction process. , By performing the conversion so that the trapezoidal distortion of the card area becomes a right-angled square, to obtain the corrected image data in the bird's-eye view state,
Image processing program.

Images