JP5240093B2 - ID card shooting system, ID card shooting method and program - Google Patents

Description

  The present invention relates to an image recognition technique.

  With the development of the Internet, Internet shopping has become popular in recent years. There are three main reasons why Internet shopping is so popular. One is the advantage that you can easily get rare products from all over the world. Next, there are merits due to the convenience of not having to visit the store, delivering the product to the house, and shopping at any time. Furthermore, there is a merit due to the cost of being able to shop at the cheapest store among various stores.

  As described above, there are a wide variety of products sold on the Internet. These include luxury goods such as liquor and cigarettes that have age restrictions on purchases. In the original face-to-face sales, it was possible to confirm by the store clerk when purchasing the product. On the other hand, this has become impossible in Internet shopping. For this reason, in an Internet shop that handles these products, the user is requested to present an identification card, and the sale is performed after the age is confirmed. There are some shops that sell without checking their age, but regulations are expected to advance and the sales style like the former is expected to be eliminated.

  Here, in order to present an identification card, the user is currently taking the following steps. 1. Go to a place where you can make a copy, such as a convenience store. 2. Pay money to make a copy. 3. Fill out the application form. 4). Buy stamps in envelopes. 5. Post envelopes. 6. Place an order after confirming the age.

  Basically, the main reason for Internet shopping is its simplicity and cost, as mentioned earlier. However, there is a problem that the age confirmation step as described above greatly reduces the merit of convenience and cost. Furthermore, as described above, the step of confirming the status such as age has a problem that the merit such as the convenience and cost of the user is impaired in various scenes. For example, opening a bank account has such a problem.

  With respect to such a problem, Patent Document 1 prepares a dedicated terminal, reads an automobile license, recognizes the date of birth, performs age verification, and transfers the age verification result to limit age. An invention has been made to allow a user to purchase a product. If the present invention is used, 2. Copy acquisition as described above. 4). Buy stamps in envelopes. 5. Since the operation of posting the envelope is reduced, the user can complete the age check relatively easily. However, it is necessary to go to a place where a dedicated terminal is installed, and the problem of 1. going to a predetermined place has not been solved.

  On the other hand, instead of preparing a dedicated terminal for this invention, it is easy to analogize the technology of photographing a driver's license using a digital camera attached to a user's mobile phone and performing age authentication. it can. According to this technology, age authentication can be freely performed anywhere, such as at home, and the convenience of the user is greatly improved.

  On the other hand, unlike a dedicated terminal or the like, an image suitable for recognition is not always obtained because a camera attached to a mobile phone cannot fix shooting conditions. For example, when a driver's license is photographed from an oblique direction, distortion called tilt distortion occurs as shown in FIG. When such distortion occurs, characters written on the driver's license are also distorted, making it difficult to recognize the contents written on the driver's license.

  Several inventions have been made to correct this tilt distortion by using image processing. For example, Patent Document 2 discloses an invention in which tilt correction is performed by designating two line segments in an image. The present invention is based on means for calculating a linear transformation matrix such as what kind of rectangle the post-correction trapezoid is converted to after correction, and correcting the tilt by applying the matrix to the entire surface. It is. As shown in FIG. 24, it is necessary to designate a total of 8 points by 4 points indicated by black circles before and after correction.

  In the above invention, the user designates four points before correction. Furthermore, four points after correction are calculated by calculation. However, since the four points after correction are simply calculated, they cannot be corrected accurately. Actually, in order to obtain the length of the side of the rectangle in the corrected image, the inclination angle of the driving license with respect to the camera and the distance from the camera to the driving license are required.

A mechanism in which the length of a rectangular side changes depending on the shooting angle will be described with reference to FIG.
Since the camera expresses the photographed three-dimensional space with a plane, each point in the three-dimensional space is described at a point where a line connecting the camera and the point intersects the plane where the image can be formed. As a result, the rectangular side originally having the length L1 + L2 is expressed as l1 + l2. The relationship between l1 and L1 can be described by the following mathematical formula.

  Here, d is the distance between the camera and the image-forming surface, and θ is the angle of how much the rectangle is inclined with respect to the image-forming surface. As described above, the size of the original rectangle cannot be determined unless θ and d are clarified. That is, accurate tilt correction is impossible unless the angle θ and the distance d are acquired by some means. Some means is, for example, a distance measuring sensor.

  On the other hand, Patent Document 3 discloses an invention in which an imaging device is provided with a means for measuring a distance and a means for allowing a user to input an angle, whereby accurate tilt correction is performed. However, there is a problem that such a means is not provided.

  In Patent Document 4, the corrected rectangle is simply calculated, and then correct tilt correction is performed by allowing the user to specify the aspect ratio of the rectangle. However, the task of specifying the aspect ratio is difficult for the user. There was a problem that it was difficult and complicated.

  The present invention is for solving the above-mentioned problems, and the contents described in the identification card can be corrected accurately and automatically by correcting the distortion of the image obtained by photographing the identification card. This is intended to make it suitable for recognition.

  The invention described in claim 1 is an imaging device that photographs an identification card, a face detection unit that determines whether or not a face is included in an image, a photo detection unit that detects a photographic region including a face, A tilt correction unit that corrects a tilt of the entire image, and when a face is detected by the face detection unit in the image obtained by the imaging device, a photo area is detected by the photo detection unit, and the photo detection unit The identification photograph photographing system is characterized in that the aspect ratio of the image after tilt correction is corrected using the size of the obtained photograph.

  According to a second aspect of the present invention, in the identification card photographing system according to the first aspect, a face detection process is performed on the image after the tilt correction.

  According to a third aspect of the present invention, in the identification card photographing system according to the first aspect, a face detection process is performed on the image before the correction of the tilt.

  According to a fourth aspect of the present invention, in the identification card photographing system according to the second or third aspect, the face detecting means is a face only when the image captured by the imaging device is generally facing a predetermined direction. It is possible to detect.

  According to a fifth aspect of the present invention, in the identification card photographing system according to the fourth aspect of the present invention, the information photographing system further includes a display unit, and displays the detected face photo area candidates. A plurality of results are displayed and the user is allowed to select.

  According to a sixth aspect of the present invention, in the identification card photographing system according to the fifth aspect, it is determined whether or not to start reading the description items of the identification card according to the size and position of the photo area. And

  According to a seventh aspect of the present invention, in the identification card photographing system according to the first aspect, the imaging device automatically enters a macro mode during photographing.

  The invention according to claim 8 is the identification card photographing system according to claim 1, further comprising OCR means for reading matters described in the identification card, wherein the OCR means is different from the imaging device. It is implemented in hardware.

  According to a ninth aspect of the present invention, in the identification card photographing system according to the first aspect, the tilt correcting means performs the tilt correction in accordance with the photo area obtained by the photo detecting means.

  According to the tenth aspect of the present invention, in the identification card photographing system according to the first aspect, the outline of the identification card is displayed on the display means at the time of photographing, and the user is allowed to photograph according to the rough shape. It is characterized by prompting.

  The invention described in claim 11 is an imaging device that photographs an identification card, a face detection unit that determines whether or not a face is included in an image, a photo detection unit that detects a photographic region including a face, An identification card photographing method in an identification card photographing system comprising a tilt correcting means for correcting a tilt of an entire image, wherein a face is detected by a face detecting means in an image obtained by an imaging device. The identification image photographing method is characterized in that a photo area is detected by a detecting means, and the aspect ratio of the image after tilt correction is corrected using the size of the photograph obtained by the photo detecting means.

  According to a twelfth aspect of the present invention, in the identification card photographing method according to the eleventh aspect, a face detection process is performed on the image after tilt correction.

  A thirteenth aspect of the invention is characterized in that, in the identification card photographing method of the eleventh aspect, face detection processing is performed on an image before tilt correction.

  According to a fourteenth aspect of the present invention, in the identification card photographing method according to the twelfth or thirteenth aspect, the face detection means is a face only when the image is photographed by the imaging device and is generally facing a predetermined direction. It is possible to detect.

  According to a fifteenth aspect of the present invention, in the identification card photographing method according to the fourteenth aspect, the identification card photographing system further includes a display unit, and displays the detected face photo area candidates, and there are a plurality of candidates. Is characterized in that a plurality of results are displayed on the display means and the user is allowed to select.

  According to a sixteenth aspect of the present invention, in the identification card photographing method according to the fifteenth aspect, it is determined whether or not to start reading the description items of the identification card according to the size and position of the photo area. And

  According to a seventeenth aspect of the present invention, in the identification card photographing method according to the eleventh aspect, the imaging device automatically enters a macro mode during photographing.

  The invention according to claim 18 is the identification card photographing method according to claim 11, further comprising OCR means for reading the items described in the identification card, wherein the OCR means is different from the imaging device. It is implemented in hardware.

  According to a nineteenth aspect of the present invention, in the identification card photographing method according to the eleventh aspect, the tilt correcting means corrects the tilt according to the photo area obtained by the photo detecting means.

  According to a twenty-second aspect of the present invention, in the identification card photographing method according to the eleventh aspect, the outline of the identification card is displayed on the display means at the time of photographing, and the user is allowed to photograph according to the rough shape. It is characterized by prompting.

  In the invention according to claim 21, a face is detected by face detection means for acquiring an image obtained by an imaging device for photographing an identification card in a computer and determining whether or not the image includes a face. In this case, a process for detecting a photo area by a photo detection unit that detects a photo area including a face, and correcting the aspect ratio of the image after tilt correction using the size of the photo obtained by the photo detection unit. It is a program characterized by executing.

  According to a twenty-second aspect of the present invention, in the program according to the twenty-first aspect, face detection processing is performed on the image after tilt correction.

  According to a twenty-third aspect of the invention, in the program according to the twenty-first aspect, face detection processing is performed on an image before tilt correction.

  According to a twenty-fourth aspect of the present invention, in the program according to the twenty-second or twenty-third aspect, the face can be detected only when the face detection unit is facing a predetermined direction in the image photographed by the imaging device. It is characterized by that.

  According to a twenty-fifth aspect of the present invention, in the program according to the twenty-fourth aspect, the detected face photograph area candidates are further displayed, and when there are a plurality of candidates, a plurality of results are displayed on the display means, and It is made to select.

  According to a twenty-sixth aspect of the present invention, the program according to the twenty-fifth aspect is characterized in that it is determined whether or not to start reading the items described in the identification card according to the size and position of the photo area.

  According to a twenty-seventh aspect of the invention, in the program according to the twenty-first aspect, the tilt correction unit performs the tilt correction according to the photographic area obtained by the photo detection unit.

  The invention according to claim 28 is the program according to claim 21, wherein the outline of the identification card is displayed on the display means at the time of photographing, and the user is encouraged to photograph according to the outline. And

  By making it possible to correct the tilt distortion of the image obtained by photographing the identification card accurately and automatically, the contents described in the identification card can be suitably recognized.

It is a figure explaining operation | movement of the system seen from the user which concerns on 1st embodiment of this invention. It is a figure explaining the transmission form which concerns on 1st embodiment of this invention. It is a figure which shows the image area | region which concerns on 1st embodiment of this invention. It is a flowchart which shows operation | movement of this application which concerns on 1st embodiment of this invention. It is a figure of the area | region of the face detected by the face detection process which concerns on 1st embodiment of this invention. It is a figure by which the position and size of the photograph area | region which concern on 1st embodiment of this invention are specified. It is a filter group concerning a first embodiment of the present invention. It is a figure explaining the correct image which concerns on 1st embodiment of this invention. It is the figure which showed the flowchart of the production | generation method of the classification rule by Adaboost which concerns on 1st embodiment of this invention. It is a figure explaining tilt correction concerning a first embodiment of the present invention. It is a figure explaining applying a filter to the whole picture concerning a first embodiment of the present invention. It is the figure which actually drawn many curves concerning a first embodiment of the present invention. It is a figure of the calculation method of four points after amendment concerning a first embodiment of the present invention. It is the flowchart which showed specification of the photograph area | region which concerns on 1st embodiment of this invention. It is a figure explaining the detection of the left side which concerns on 1st embodiment of this invention. It is a figure explaining the search range which concerns on 1st embodiment of this invention. It is a figure explaining the function which the server concerning a first embodiment of the present invention has. It is the flowchart which showed operation | movement of the license reading part which concerns on 1st embodiment of this invention. It is the figure which showed the rectangular area which concerns on 1st embodiment of this invention. It is a flowchart which shows operation | movement of the driver's license application which concerns on 2nd embodiment of this invention. It is a flowchart of the photographic area | region specific process which concerns on 2nd embodiment of this invention. It is a figure explaining performing edge detection processing, Hough transformation, and intersection detection processing to the whole picture concerning a third embodiment of the present invention. It is a figure explaining the conventional tilt correction. It is a figure explaining the conventional tilt correction. It is a figure explaining the mechanism in which the length of the side of a rectangle changes with the conventional photography angles.

  Hereinafter, as a preferred embodiment of the present invention, an identification card photographing system that can be used for client-server type Internet shopping will be described as an example.

First Embodiment Operation of System Viewed from User First, the operation of the system viewed from the user will be described with reference to FIG. The premise is as follows. 1. The user's PC is connected to the Internet and can communicate with the server through the Internet. 2. The user has a mobile phone. 3. The mobile phone can communicate with the server wirelessly over the Internet. 4). The mobile phone is accompanied by a camera. 5. The mobile phone can extract information from the QR code. 6). The mobile phone can receive mail over the Internet wirelessly.

  First, a user connects a PC to the Internet and performs various searches on a web page (step 001). When a desired product is found, a button arranged on the web page is pressed by a mouse click. Then, a notification of purchase decision of the product is transmitted from the web page to the server (step 002). If the product notified to the server is a product that can only be purchased by adults (step 003), the server will notify the necessity of adult authentication through the PC and a QR code will be displayed on the web page (step 004) . When the user captures the displayed QR code with his / her mobile phone (Step 005), the mobile phone recognizes the QR code and detects the address embedded in the QR code. The detected address is displayed on the display of the user's mobile phone, and when the user clicks on the address, the mobile phone requests the server to receive an application.

  A driver's license photographing application for adult authentication is downloaded from the server that has received the reception request to the mobile phone (step 006). When the user starts the application, the user is prompted to take a driver's license. Accordingly, the user photographs his driver's license (step 007). Note that shooting using the application will be described later.

  The driver's license image taken by the user is transmitted to the server (step 008). Based on the driver's license image, the server determines whether the user is an adult. If the result of the adult authentication is not an adult, the server sends an alert describing that the product cannot be purchased to the user's mobile phone by e-mail (step 011). In this case, the user cannot continue shopping and the shopping ends.

  On the other hand, if it is determined that the user is an adult, the server transmits an html mail in which a form for inputting information such as an address and a name for purchasing the product and a send button are filled in the user's mobile phone (see FIG. 2).

  The user who has received the mail (step 010) fills in the necessary information on the form (step 012), and transmits information to the server by pressing the send button (step 013). Finally, the merchandise providing side charges the user based on the contents entered in the form, and the merchandise is shipped to the user, thereby completing the shopping.

  If it is determined in step 003 that age verification is not required, the server sends an email with a form attached to the user (step 010). Unlike the case where age verification is required, the html is viewed by the user. It is sent to the web page inside. Thereby, when age verification is not required, the user's convenience is improved because it is not necessary to use a mobile phone.

Operation of Driver's License Shooting Application Next, an operation when a user photographs a driver's license using the above-described driver's license shooting application will be described. FIG. 4 is a flowchart showing the operation of the application. When the user starts photographing, an image is acquired by a camera attached to the mobile phone (step 100). It is desirable that the camera automatically enters the macro mode when shooting is started. The macro mode is a mode that makes it easy to focus on a close object. In order to accurately read the contents of the license, it is necessary to photograph the license at a relatively close position while focusing. Although the digital camera of the mobile phone equipped with the macro mode is common, the user generally does not know the existence of the macro mode. Therefore, it is desirable that the application automatically sets the macro mode setting.

  The application performs face detection processing on the acquired image (step 101). Detailed means of the face detection process will be described later. If there is no face (step 102), the application returns to image acquisition. On the other hand, when the face exists (step 102), the application determines whether or not the face is included in a predetermined area. The predetermined area is an area in which the center position of the face is located on the right 1/3 in the entire image and the center position of the face is included in the center 1/3 in the vertical direction of the image (FIG. 3). Shaded area).

  Further, the application determines whether the size of the detected face area occupies 3% or more and 10% or less of the entire screen. If all of these are satisfied, the application proceeds to the photo area identification step (step 104). If one of them is not satisfied, the application returns to image acquisition (step 100).

  The reason for determining the position of the face where the position is detected is to prevent erroneous detection of the face. Another purpose is to determine that all necessary areas fit on the screen for adult authentication. These numerical values are determined empirically and have no theoretical support.

  If the application notifies the user of the reason why the predetermined area is not satisfied through the screen of the mobile phone, the user can adjust the shooting conditions accordingly, so that shooting can be performed more smoothly. become.

  Subsequently, the application performs tilt correction (step 104) on the photographed license image. As described above, the tilt correction is a process of returning a rectangle that has been distorted into a trapezoid as shown in FIG. 23 to the original rectangle. Details of the processing will be described later, but as described in the background of the invention, it is impossible to correct the rectangular image having an accurate aspect ratio with respect to the original rectangle, so the aspect ratio is inaccurate. To correct images.

  Following the tilt correction, the application identifies the photographed area of the driver's license that was shot (step 104). In general, the face area detected by the face detection process described later is not the entire picture but only the face area as shown by the thick line in FIG. Therefore, in this embodiment, the application specifies not only the face area but also the photo area. In addition, the photographic area | region in this invention shows the area | region where the image obtained by the photograph is described. Actually, it may be an area where a photograph is pasted or an area where a photograph is printed. In addition, although a specific means for photographic area will be described later, a plurality of photographic area candidates may be detected. In this case, the application displays all candidates using a display attached to the mobile phone, and allows the user to select. As a result, mistakes in specifying the face area can be greatly reduced.

  When the application identifies the photograph area in Step 105, the position and size of the photograph area are identified in the entire license image (see FIG. 6). In Step 106, the entire screen is displayed according to the ratio of w and h obtained. The application changes the aspect ratio of. The photo of the license in Japan is w: h = 24: 30. In accordance with this, the vertical direction of the image is enlarged or reduced.

  When the application transmits the image subjected to the aspect ratio correction to the server (step 0107), the processing of the application is completed.

Face detection means (Fig.4-step101)
For the face detection, the method introduced in Document 1 is used. This time, the method described in Non-Patent Document 1 is used, but of course, a method such as the eigenface method may be used, or a simple pattern match may be used. Any technique can be used as long as it can detect a face.
Non-patent document 1
Rapid Object Detection using a Boosted Cascade of Simple Features.IEEE CVPR, 2001.
The details are given in Non-Patent Document 2, but here, an outline thereof will be described. The essence of the technique introduced in Non-Patent Document 1 can be said to be a filter called Rectangle Filter and a learning means called Adaboost (Non-Patent Document 2).
Non-Patent Document 2
The Elements of Statistical Learning: Data Mining, Inference, and Prediction.Springer Series in Statistics.2001.
The Rectangle Filter is a group of filters as shown in FIG.

  For example, consider applying 1. Edge Features (a) in FIG. 7 to an image. The filter is scanned, and the difference between the average value of the pixels in the area included in the white rectangle and the average value of the pixels in the area included in the black rectangle is used as the output value of the filter center. Thereby, the degree of coincidence with a specific edge pattern can be calculated.

  The reason why a simple filter such as Rectangle Filter is used here is that an acceleration called Integral Image can be used. I won't go into detail about Integral Image, but originally, to calculate the average value of the pixels included in each rectangle (the number of pixels included in the region) times is required, whereas Integral Image By using the average value, the average value can be calculated only by performing the summation three times, so that a very high speed calculation is possible. These calculations are performed while changing all the filters listed in the figure to various sizes. As a result, a multidimensional vector is obtained at each point of the input image.

  Subsequently, rectangles of various sizes and positions in the input image are assumed, and whether or not a face exists is determined from the vector values included in each rectangle. Of course, the determination involves handling a very large dimensional vector, and it is very difficult to create a determination means by human hands. Therefore, a technique called Adaboost, which creates a discriminator that binaryly determines whether the answer is correct or incorrect, is used.

  When various vectors and correct answer values are given as learning data for learning to Adaboost, a discriminator for determining whether the input vector is correct or incorrect is obtained. In the learning data in the present embodiment, the output value of the rectangle filter is a vector, and an image that is a face is correct and an image other than the face is incorrect.

  Here, a supplementary description will be given of a face image to be introduced as a correct image. If face images in various directions such as (a) to (d) in FIG. 8 are given as correct images, the created face detection means has various patterns (a) to (d). Can be identified. Conversely, if only the pattern (a) is given as a correct image and (b) to (d) are given as incorrect images, only the face having the pattern as shown in (a) can be detected.

Here, in the present embodiment, only the latter pattern (a) is given as correct answer data. This is because only the face facing in the direction of (a) can be detected, so it is possible to easily determine which direction the driver's license is facing in the photographed image when photographing with the mobile phone. This not only reduces the load on the server, but also improves the accuracy when reading the description of the driver's license on the server side.
Next, the operation principle of Adaboost will be described.

Adaboost
Adaboost is a means for generating classification rules. A highly accurate determination result can be obtained by taking a weighted majority of the results of a plurality of poor classifiers. The term “poor classifier” (weak classifier) here is used to compare a high-precision classifier obtained by combining them with a strong classifier. It may be a vessel. In other words, the face detection means in this embodiment may be anything as long as it can obtain a binary classification means by learning. For example, SVM or neural network may be used.

  First, a method for generating classification rules using Adaboost will be described. FIG. 9 is a diagram showing a flowchart of a classification rule generation method by Adaboost.

First, in Step 01, the weight D ₁ (i) for the sample is initialized. The initialization method is arbitrary as long as the sum of all the weights is 1, but this time, each sample is given a weight of 1 / m with respect to the number of samples m. Next, in Step 02, the weak classifier h _t (x _i ) is learned using the weighted samples. In Non-Patent Document 2, there is no definition of a weak classifier, and it is only necessary that the classification rate exceeds 0.5. The weak classifier used this time is a weak classifier that extracts a specific one dimension from the feature quantity dimension and sets a threshold value. As described above, since all the feature values are normalized to 0 to 1, an optimum weak classifier can be set by sequentially scanning the threshold value in 0.01 steps or the like.

Subsequently, in Step 03, all samples are identified by the weak classifier obtained by learning, and the error rate ε _t is calculated. However, this error rate is not the ratio of the number of mistaken samples, but the percentage of the sum of the weights assigned to the wrong samples. As described above, since the weights are normalized to 1 in total, the error rate can be calculated by taking the sum of the weights attached to the wrong samples. Then using Step04 In the mistake rate epsilon _t, we calculate the weights αt of the weak classifiers based on the number 2. In Formula 2, an excellent weak classifier is given a high weight, and an inferior weak classifier is given a low weight.

Subsequently, in Step 05, the weights of the samples are updated based on Equations 3 and 4 using the weights α _i of the weak classifiers. Equation 3 is designed to increase the weight of the sample mistaken in the current learning and decrease the weight of the correct sample. Zt is a value for normalization so that the sum of the updated weights becomes 1.

  Finally, at Step 06, convergence is confirmed. For example, when the error rate becomes 0, it can be said that it has converged, and it can also be said that it has converged when the change in the error rate does not exceed a preset threshold (for example, 1e-6). If not converged, the process from Step 02 to Step 06 is repeated once again. If converged, the process is terminated.

If a plurality of weak classifiers h _i (x) and weak classifier weights α _i finally calculated are used and calculation is performed based on Equation 5, the classifiers can be used. In Equation 5, T represents the number of times of learning until convergence, that is, the number of weak classifiers used for identification.

Tilt correction (Figure 4-step104)
Next, the tilt correction shown in FIG. 4-step 104 will be described. The tilt correction is achieved by a flow as shown in FIG.

  First, the edge detection means 300 applies a filter as shown in FIG. 11 to the entire image, calculates an edge component from the image, and further performs binarization processing to obtain a binary image of the edge. Hereinafter, a point that is an edge will be referred to as an edge point.

  Subsequently, the straight line detection unit 301 detects a straight line included in the image. Here, it is assumed that Hough transform, which is a general straight line detection algorithm, is used. Although the Hough transform is used here, any algorithm that can detect a straight line may be used. In the Hough transform, an edge point coordinate x, y is substituted into the following equation to obtain a curve in the rθ plane.

  When the coordinates of all the edge points are substituted into Equation 6 and a curve is drawn on the rθ plane, a point where many of these curves intersect will appear. FIG. 12 actually depicts a large number of curves. Here, the white line represents each curve, and it can be said that a brighter point is a point where many curves intersect.

  A desired straight line can be obtained by substituting r and θ indicating the bright point into the following equation (7). On the other hand, actually, as is clear from FIG. 12, a bright area is obtained instead of a bright point. Therefore, the brightest area is obtained in the local area, and r and θ indicating straight lines are obtained. That is, after drawing on the rθ plane, the target pixel is sequentially scanned on the rθ plane, and a straight line (hereinafter referred to as r and θ) where the target pixel is the whitest point in the 5 × 5 pixels around the target pixel (See the equation) is a straight line candidate.

  As is clear from FIG. 12, a plurality of straight line candidates appear. Therefore, the intersection detection unit 302 first calculates intersections of all detected straight line candidates. Among the calculated intersections, the one at the top left, the top right, the bottom left, and the bottom right in the image are output as four points before correction. These four points are expected to get four corners of the driver's license.

  Subsequently, the corrected 4-point calculation means 303 calculates the corrected 4 points based on the coordinates of the 4 points before the correction. As shown in FIG. 13, the correction method of the four points after correction is each vertex of a rectangle having four sides of a vertical or horizontal straight line passing through the middle point of each point. The points indicated by white circles in the figure are the four points before correction, and the points indicated by black circles are the four points after correction.

Thereafter, the coordinates of the four points before correction are (x ₀ , y ₀ ) to (x ₃ , y ₃ ), and the coordinates of the four points after correction are (x ′ ₀ , y ′ ₀ ) to (x ′ ₃ , Continue talking as y ' ₃ ).

  Subsequently, the matrix calculation means 304 obtains it by solving the 8-ary simultaneous equations of Equation 8.

  The conversion matrix can be obtained by solving the quaternary simultaneous equations by the Gauss-Jordan method or the like.

  The tilt correction is completed by applying the conversion matrix obtained here to the entire image by the matrix calculation means 305.

Photo area determination means (Fig.4-step105)
Next, the photographic area specifying means described in step 105 of FIG. 4 will be described. The photo area is specified by detecting the left, right, top, and bottom sides according to the flow shown in FIG. First, the left side detection will be described as an example.

  First, since the left side is a vertical edge, a vertical edge is detected. Specifically, a filter that strongly responds to the vertical edge as shown in FIG.

  Subsequently, binarization processing is performed (401) to determine edge pixels and non-edge pixels. In the edge count 402, first, a search range as indicated by a dotted line in FIG. 16 is determined with reference to the detected face area. In the search range, the edges are counted in the vertical direction. Result: Count how many vertical edge pixels exist in each x. Thereafter, in the maximum count search 403, the number of vertical edge pixels of the x coordinate where the vertical edge pixel is most present is searched. Thereafter, the number of vertical edges of each x coordinate is compared with the maximum number of vertical edges, and 80% or more of the maximum number of edges is set as a photo edge candidate. Similarly, the right side is detected.

  Further, for the upper side and the lower side, if a filter as shown in FIG. 15B is used as an edge detection filter and the edge count is changed to the horizontal direction instead of the vertical direction, a photographic edge candidate can be similarly obtained. Note that the photograph edge does not pick up only the point with the maximum edge count, but is a candidate point if it exceeds 80% of the maximum, so a plurality of photograph region candidates appear as described above.

Operation of the system viewed from the server side Next, the operation of the system viewed from the server side will be described. FIG. 17 below lists the functions of the server in the system according to the embodiment. First, the server includes a communication unit 107 and can communicate with PCs of various users through the Internet. The server has a product DB for storing product information. As shown in Table 1, each DB stores a product ID, a product name, a price, an image URL, whether age authentication is necessary, and a product image.

  When the user accesses the page of the server through the Internet, the server transmits the information to the user's PC through the product information transmission unit 100. When a user who has browsed these pieces of information searches for a product and then presses a purchase decision button, the product ID of the product that has been purchased is transmitted to the server.

  The server searches the product DB for the transmitted product ID, and the age authentication necessity determination unit 101 determines whether or not the product requires adult authentication. When age authentication is not necessary, the server generates a form by the form generation / transmission unit 102 and transmits the form to the web page being browsed by the user via the communication unit 105 and the Internet.

  On the other hand, when age verification is required, the server creates a QR code using the QR code creation / transmission unit, and transmits the QR code to the PC being browsed by the user. Here, it is assumed that the QR code includes an address for downloading the driver's license image transmission application held in the server, and the user captures the QR code with a camera and reads the content of the QR code. It is assumed that the address can be easily accessed.

  When the user accesses the address, the server transmits the held driver's license image transmission application to the user through the application transmission unit 105.

  When the user transmits a license image through the driver's license image transmission application, the server uses the license reading unit 106 to read the date of birth of the user from the license image. The operation of the license reading unit 106 will be described later. The server calculates the user's age based on the date of birth read, and if the user's age is over 20 years, the form is generated using the form generation / transmission unit 102, and the communication unit 105 Send to the web page that the user is browsing via the Internet. As will be described later, since the address and name can also be read by the same means as reading the date of birth, if the server reads the address and name and inputs in advance to the form to be generated, This saves the user trouble and improves user convenience.

  When a form in which necessary items are filled in is transmitted by the user, the server receives the form at the form receiving unit 103 and stores it in the order content DB. The person who provides the product with reference to the DB charges the user for the price based on the contents entered in the form and sends the product to the user, thereby completing the shopping.

License reading section 106
The operation of the license reading unit 106 shown in FIG. 17 will be described with reference to the flowchart of FIG.

  First, the server can specify the region where the date of birth is described in the license image by the region specifying means 500. As described above, it has been revealed to the server where the face photograph exists in the license image by the processing so far. Also, the license format is fixed (at least in prefectures). Therefore, the positional relationship between the upper left of the face photograph and the upper left of the region where the date of birth is described, and the size of the rectangular region shown in FIG. 19 are constant. For this reason, the server can easily specify the region in which the date of birth is described from the position of the photo region.

  In this way, it is possible to easily specify the area because not only the server corrects the correction in the present embodiment but also corrects the aspect ratio using the face photograph area. is there. If tilt correction is not performed, considering that the aspect ratio is not corrected, even if the position of the face photo is known, the region where the date of birth is described is not easily known.

  Subsequently, in the character cutout unit 501, the server cuts the image included in the specified area into images of each character. Further, the character collating unit 502 refers to the character DB, and determines which character each character cut out by pattern matching is close to.

Effect of First Embodiment The first embodiment has been described so far. As described in the prior art, it is impossible to correct the tilt while maintaining an accurate aspect ratio unless the distance from the subject (identification card) and the angle between the license and the camera are unclear. On the other hand, in the present embodiment, for a license that is a fixed format, it has been shown that tilt correction with an accurate aspect ratio can be automatically performed by using the size information of the fixed format. Furthermore, among the standard formats, attention was paid to the face and the face photograph because the face photograph used for a license is extremely suitable for face detection. Furthermore, the technique described in Non-Patent Document 2 shown in the present embodiment is a technique that is currently widely used in digital cameras and the like, and the recognition accuracy is extremely high. For these reasons, the face detection described in the license can be detected with extremely high accuracy.
Also, with this configuration, it is possible to perform accurate tilt correction on the driver's license, so it is possible to very well identify the area where the description items of the license are written from the face position.
Although the present invention has been described only with respect to a driver's license, the present invention can be applied to any identification card that has a fixed format and a face photograph attached thereto.

Second embodiment of the present invention (only face detection is performed with a mobile phone)
In the ID card photographing system described in the second embodiment, the system operation viewed from the user side and the system operation viewed from the server side are substantially the same as the ID card shooting system described in the first embodiment. However, the difference is that processing such as face detection and tilt correction is implemented on the server side. Accordingly, the operation of the driver's license photographing application and the operation of the license reading unit, which are different from the first embodiment, will be described.

Operation of Driver's License Shooting Application FIG. 20 shows the operation of the driver's license application. When the user starts shooting, an image is acquired by a camera attached to the mobile phone (step 200). When shooting is started, it is desirable that the application automatically sets the camera to the macro mode, as in the first embodiment. Further, in this embodiment, since it is not known at all whether the image taken by the user is suitable for adult identification, the application displays the outline of the license on the display of the mobile phone at the time of shooting. The user should be encouraged to shoot according to the shape. By doing so, the user can know in which direction the license should be placed on the screen and what size should be taken. Then, since the photographed image has relatively favorable conditions, the accuracy of the following processing is increased. The captured image is transmitted to the server (step 201).

Operation of license reading part (Fig.17-106)
Basically the same as in the first embodiment, but the processing from face detection (step 101) to aspect ratio correction (step 106), which was the processing of the application on the camera side in the first embodiment, is performed on the server. It differs in that it is performed (see FIG. 4). Most of the processing is the same as that described with reference to FIG. 4, but only the photo region specification (step 105) is slightly different. FIG. 21 is a flowchart of the photograph area specifying process. In the determination means 404, in the first embodiment, if the maximum count is 80% or more, the application is assumed to be a photograph area candidate. Although the configuration is such that the user selects it, in this embodiment, the server immediately determines the photo area when the maximum count is reached. Such a process is performed because the process is performed on the server and an interactive operation from the user is impossible.

Effects of Second Embodiment As described above, in the second embodiment, recognition accuracy is reduced because there is no interactive operation such as the user selecting a photo area candidate as in the first embodiment. However, since processing on a mobile phone having generally poor processing capability is kept to a minimum, it operates at high speed and the convenience of the user is enhanced.

Third Embodiment of the Present Invention The ID card photographing system described in the third embodiment has substantially the same configuration as the ID card photographing system described in the first embodiment, but the tilt correction shown in FIG. The operation of (step 104) is different. In the tilt correction shown in FIG. 10 of the first embodiment, edge detection processing, Hough conversion, and intersection detection processing are performed on the entire image. On the other hand, in the present embodiment, the range to be subjected to the processing is limited to the range surrounded by the thick line in FIG. Note that the bold dotted line is the face range detected by face detection, and the area to be processed is a range that is doubled vertically and horizontally without changing the face range and the center of gravity.

  In the tilt correction shown in FIG. 10 of the first embodiment, processing is performed on the entire surface in the hope that four corners of the license can be obtained. However, in this embodiment, four corners of the photographic area are displayed. Processing is performed with the expectation that it will be removed. It is better to correct the tilt by looking at a wide range (4 corners of the license) empirically, but the narrow range (4 corners of the photographic area) is seen as in this embodiment. Thus, the amount of processing can be greatly reduced by performing tilt correction.

As described above, in each of the embodiments so far, the identification of the identity is performed by linking the mobile phone and the server. However, either the mobile phone or the server may be responsible for each component. . For example, the mobile phone can be provided with all functions for performing these processes. For example, it is also possible to provide a user as identification software and sell a predetermined item only when authentication is performed by the software.
Similarly, there is no problem if the server does all processing. It is also possible to send an image of the ID card sent from the user to the server, and to process the image to perform identification and sell a predetermined article.

  Hereinafter, the function and effect of each claim will be described.

  According to the invention of claim 1 and the like, by detecting a region known to be a fixed form of a photograph of a driver's license and performing the aspect ratio correction after correcting the tilt using the width and height of the region, Accurate correction can be performed, and the accuracy when reading the contents of the license is improved.

  According to the invention of claim 2 and the like, the face detection process is performed on the tilt-corrected image, so that the accuracy of face detection is improved and the accuracy when reading the contents of the license is improved.

  According to the invention of claim 3 and the like, since face detection is performed and it is determined whether or not the photographed image is suitable for reading, since relatively heavy tilt correction is performed, until an image suitable for reading is obtained. The heavy processing is not repeated many times, and the convenience for the user is improved.

  According to the invention of claim 4 or the like, since the face direction in which face detection can be performed is constant, there is no problem that age authentication cannot be performed by photographing with the wrong arrangement, and convenience for the user is improved. .

  According to the invention of claim 5 and the like, by allowing the user to select from a plurality of face photo area candidates, the face photo area can be specified accurately without imposing a load on the user, so that an accurate aspect ratio can be corrected.

  According to the invention of claim 6 or the like, reading processing is performed for a small image or an image in which a necessary area is cut off by performing reading only when the position and size of the photograph are suitable for adult authentication. Therefore, the user's convenience is improved because reading is completed without performing processing such as prompting the user to take another image.

  According to the invention of claim 7 and the like, since the macro mode is automatically set at the same time as the shooting of the driver's license is started, a focused image can be obtained even if the shooting is sufficiently close to the license. Improves reading accuracy.

  According to the invention of claim 8 and the like, since OCR processing, which is relatively heavy processing, is performed by a terminal having a rich processing capability such as a server, the processing system of the mobile phone is not required to have excessive processing capability, and the user For this, the threshold for system introduction is lowered.

  According to the invention of claim 9 and the like, since the range for performing the correction for the tilt correction from the face area obtained by the face detection means can be significantly reduced, the processing system of the cellular phone is excessively processed. The threshold for system introduction is lowered for the user.

  According to the invention of claim 10 or the like, the outline of the identification card appears on the shooting screen at the time of shooting, and it is only necessary to take a picture of the identification card according to the outline. The user convenience is improved because it is intuitively understood whether or not it should be done.

  Each embodiment described above is a preferred embodiment of the present invention, and various modifications can be made without departing from the gist of the present invention.

JP 2003-030726 A Japanese Patent Laid-Open No. 2001-084365 JP 09-289600 A JP 2001-177716 A

Claims (28)

  An imaging device that shoots an identification card, a face detection unit that determines whether or not a face is included in an image, a photo detection unit that detects a photo area including a face, and a tilt correction that corrects a tilt of the entire image And when the face detection unit detects a face in the image obtained by the imaging device, the photo detection unit detects a photo area, and the photo detection unit An ID card photographing system that corrects the aspect ratio of an image after tilt correction using the size.   2. The identification card photographing system according to claim 1, wherein face detection processing is performed on the image after tilt correction.   2. The identification card photographing system according to claim 1, wherein face detection processing is performed on an image before tilt correction.   4. The identity according to claim 2, wherein the face detection means is capable of detecting a face only when the image is picked up by an image pickup apparatus and is substantially in a predetermined direction. Certificate shooting system.   5. The display device according to claim 4, further comprising display means for displaying the detected face photograph area candidates, and when there are a plurality of candidates, the display means displays a plurality of results and allows the user to select one. Identification card taking system.   6. The identification card photographing system according to claim 5, wherein it is determined whether or not reading of the description items of the identification card is started according to the size and position of the photo area.   The identification image capturing system according to claim 1, wherein the image capturing apparatus automatically enters a macro mode during image capturing.   2. The identification card according to claim 1, further comprising an OCR unit that reads items described in the identification card, wherein the OCR unit is mounted on hardware different from the imaging device. Shooting system.   2. The identification card photographing system according to claim 1, wherein the tilt correction means corrects the tilt according to the photograph area obtained by the photograph detection means.   2. The identification card photographing system according to claim 1, wherein the outline of the identification card is displayed on the display means at the time of photographing, and the user is encouraged to photograph according to the outline.   An imaging device that shoots an identification card, a face detection unit that determines whether or not a face is included in an image, a photo detection unit that detects a photo area including a face, and a tilt correction that corrects a tilt of the entire image An identification image photographing method in an identification image photographing system, wherein a face is detected by the photo detection means when a face is detected by the face detection means in an image obtained by the imaging device. A method of taking an identification card, comprising: detecting and correcting an aspect ratio of an image after tilt correction using a size of a photograph obtained by the photograph detecting means.   The identification image capturing method according to claim 11, wherein face detection processing is performed on the image after tilt correction.   The identification image capturing method according to claim 11, wherein face detection processing is performed on an image before tilt correction.   14. The identity according to claim 12, wherein the face detection means is capable of detecting a face only when the image is picked up by an image pickup apparatus and is substantially in a predetermined direction. Certificate shooting method.   The identification card photographing system further includes a display unit, and displays the detected face photo area candidates. When there are a plurality of candidates, the display unit displays a plurality of results and allows the user to select one. 15. The identification card photographing method according to claim 14.   16. The method of photographing an identification card according to claim 15, wherein it is determined whether or not to start reading the identification item according to the size and position of the photo area.   The identification image capturing method according to claim 11, wherein the image capturing apparatus automatically enters a macro mode at the time of capturing.   12. The identification card according to claim 11, further comprising an OCR unit that reads items described in the identification card, and the OCR unit is mounted on hardware different from the imaging device. Shooting method.   12. The method of photographing an identification card according to claim 11, wherein the tilt correcting means corrects the tilt according to the photograph area obtained by the photograph detecting means.   12. The method of photographing an identification card according to claim 11, wherein the outline of the identification card is displayed on the display means at the time of photographing, and the user is prompted to photograph according to the outline.   When a face is detected by a face detection unit that acquires an image obtained by an imaging device that shoots an identification card in a computer and determines whether or not a face is included in the image, a photo area including the face Detecting a photograph area by a photograph detecting means for detecting the image, and executing a process for correcting the aspect ratio of the image after tilt correction using the size of the photograph obtained by the photograph detecting means. program.   The program according to claim 21, wherein face detection processing is performed on an image after tilt correction.   The program according to claim 21, wherein face detection processing is performed on an image before tilt correction.   24. The program according to claim 22 or 23, wherein the face detection means can detect a face only when the image taken by the imaging apparatus is substantially in a predetermined direction. .   25. The program according to claim 24, further displaying the detected face photo area candidates, and when there are a plurality of candidates, the plurality of results are displayed on the display means to allow the user to select.   26. The program according to claim 25, wherein it is determined whether or not to start reading the description items of the identification card according to the size and position of the photo area.   23. The program according to claim 21, wherein the tilt correction unit performs a tilt correction according to a photograph area obtained by the photograph detection unit.   22. The program according to claim 21, wherein the outline of the identification card is displayed on the display means at the time of shooting, and the user is prompted to take a picture according to the outline.

Images