JP2022075264A - Image processing system - Google Patents

Abstract

To provide a configuration capable of correcting distortion of a captured image without performing camera calibration for each camera in advance.SOLUTION: The distortion of an image captured by an imaging unit 23 is corrected by an image correction process on the basis of a distortion coefficient calculated by a distortion coefficient calculation process, and a face feature point extraction processing using the corrected image is performed. In the distortion coefficient calculation process, a distortion coefficient is calculated on the basis of the pixel coordinates that are the coordinates of each of coordinate comparison cells that are at least a part of the plurality of cells in the image obtained by capturing an authentication code C in which a plurality of cells are arranged two-dimensionally by the imaging unit 23, and the three-dimensional position coordinates of each cell of the authentication code C corresponding to the coordinate comparison cell.SELECTED DRAWING: Figure 7

Description

The present invention relates to an image processing apparatus that corrects distortion of a captured image and performs processing using the corrected corrected image.

In an image captured by a fisheye camera or the like that uses a fisheye lens, the imaged object is imaged so that it is distorted. Therefore, depending on the processing using the image, it may be necessary to correct the distortion in the image. .. As a technique for correcting the distortion of the captured image as described above, for example, the calibration system disclosed in Patent Document 1 below is known. In this calibration system, the calibration device acquires the data of the image in which the chart for calibration is taken by the image pickup device, and performs the calibration calculation based on the data, so that the distortion correction coefficient of the camera of the image pickup device and the distortion correction coefficient can be obtained. It is configured to derive camera parameters such as internal parameters and external parameters. By using the camera parameters derived in this way, it is possible to perform correction processing so as to suppress distortion of the image captured by the image pickup device.

Japanese Patent No. 6764533

By the way, since the unique camera parameters may vary due to individual differences of the cameras, the generated variations become large, and therefore, camera calibration for calculating the distortion correction coefficient etc. may be required for each camera. be. That is, it may be necessary to calibrate the camera for each camera in advance, such as before shipping the product, and there is a problem that it is difficult to shorten the process work time. Such a problem related to camera calibration may occur not only in a fisheye camera using a fisheye lens or a wide-angle camera using a wide-angle lens, but also in a camera using a general image pickup lens.

The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to provide a configuration capable of correcting distortion of a captured image without performing camera calibration for each camera in advance. To do.

In order to achieve the above object, the image processing apparatus (20) according to claim 1 of the claims is
An image pickup unit (23) having an image pickup lens and an image pickup element for capturing an image obtained by the image pickup lens, and an image pickup unit (23).
The distortion coefficient calculation unit (21) for calculating the distortion coefficient for correcting the distortion of the captured image of the imaging unit, and the distortion coefficient calculation unit (21).
A distortion correction unit (21) that corrects the distortion of the captured image of the image pickup unit based on the distortion coefficient calculated by the distortion coefficient calculation unit.
A processing unit (21) that performs processing using the corrected image whose distortion has been corrected by the distortion correction unit, and
Equipped with
The strain coefficient calculation unit captures a two-dimensional code (C) in which a plurality of cells are arranged by the image pickup unit, and the coordinates of each of the coordinates comparison cells that are at least a part of the plurality of cells. It is characterized in that the strain coefficient is calculated based on the pixel coordinates to be obtained and the three-dimensional position coordinates of each of the cells of the two-dimensional code corresponding to the coordinate comparison cell.
The reference numerals in the parentheses indicate the correspondence with the specific means described in the embodiments described later.

In the invention of claim 1, the distortion of the image taken by the imaging unit is corrected by the distortion correction unit based on the distortion coefficient calculated by the distortion coefficient calculation unit, and the processing unit performs processing using this corrected image. In the strain coefficient calculation unit, the pixel coordinates, which are the coordinates of the coordinate comparison cells that are at least a part of the plurality of cells in the captured image obtained by capturing the two-dimensional code in which a plurality of cells are arranged, are used. The strain coefficient is calculated based on the three-dimensional position coordinates of each cell of the two-dimensional code corresponding to the coordinate comparison cell.

Since the strain coefficient is calculated using the captured two-dimensional code in this way, the strain coefficient can be easily calculated without preparing a dedicated calibration chart or the like. For this reason, it is possible to capture a two-dimensional code and calculate the distortion coefficient each time processing using the captured image is performed. Therefore, the calculated distortion coefficient can be calculated without performing camera calibration for each camera in advance. It is possible to correct the distortion of the captured image based on.

Even in a configuration in which the image pickup unit functions as a fisheye camera by adopting a fisheye lens as an image pickup lens as in the invention of claim 2, a two-dimensional code captured every time a process using a fisheye image is performed is used. By calculating the distortion coefficient, it is possible to correct the distortion of the fisheye image based on the calculated distortion coefficient.

In the invention of claim 3, a decoding unit is provided that performs a decoding process for decoding the two-dimensional code captured image captured by the imaging unit, and the corrected image is generated using the decoding result by the decoding unit. The processing used is performed by the processing unit. As a result, the distortion coefficient can be calculated from the captured image of the two-dimensional code captured in order to obtain the decoding result, so that the two-dimensional code that does not need to be decoded is not imaged just for calculating the distortion coefficient, so that the processing efficiency is high. Can be achieved.

In the invention of claim 4, the two-dimensional code records the face feature point information regarding the face feature points extracted from the user's face image, and the processing unit records the face feature points obtained as the decoding result by the decoding unit. The user authentication process is performed using the information and the facial feature points extracted from the user's face image imaged by the imaging unit and corrected by the distortion correction unit. As a result, not only the distortion coefficient can be calculated from the captured image of the two-dimensional code captured to obtain the facial feature point information, but also the distortion of the user's face image from which the facial feature point is extracted based on the distortion coefficient can be corrected. Therefore, the authentication accuracy of the authentication process can be improved.

In the invention of claim 5, since the information regarding the cell size of the two-dimensional code is recorded in the two-dimensional code, the actual shape of the captured two-dimensional code can be accurately grasped. This makes it easier to accurately calculate the three-dimensional position coordinates of each cell of the two-dimensional code corresponding to the coordinate comparison cell, so that the calculation accuracy of the distortion coefficient, that is, the distortion correction accuracy of the captured image can be improved. Can be done.

In the invention of claim 6, the strain coefficient calculation unit calculates the strain coefficient based on the pixel coordinates and the three-dimensional position coordinates of the two-dimensional code captured a plurality of times. This makes it easier to suppress the effects of camera shake during imaging, as compared to the case where the distortion coefficient is calculated based on the pixel coordinates of the two-dimensional code and the three-dimensional position coordinates from one captured image. It is possible to improve the calculation accuracy of the coefficient, that is, the distortion correction accuracy of the captured image.

In the invention of claim 7, since the two-dimensional code is displayed on the screen, the two-dimensional code itself is not distorted at the time of imaging because it is printed on paper or the like, so that the calculation accuracy of the distortion coefficient, that is, the imaging is performed. The distortion correction accuracy of the image can be improved.

It is explanatory drawing which shows schematic structure of the authentication system provided with the image processing apparatus which concerns on 1st Embodiment. It is explanatory drawing which illustrates a part of the part where the face feature point is extracted from the photographed face image data. It is a block diagram which shows roughly the electric structure of an image processing apparatus. It is a block diagram which shows the electric composition of a mobile terminal schematicly. It is a part of a flowchart illustrating the flow of authentication processing by an image processing device. It is a part of a flowchart illustrating the flow of authentication processing by an image processing device. It is explanatory drawing explaining the relationship between a camera coordinate system and a pixel coordinate system.

[First Embodiment]
Hereinafter, an embodiment embodying an authentication system including an image processing device according to the present invention will be described with reference to the drawings.
The authentication system 10 shown in FIG. 1 mainly includes an image processing device 20, and is a two-dimensional code for authentication in which facial feature points, personal information, etc. extracted from a user's face image are recorded (hereinafter, authentication). It is configured as a system for personally authenticating the user by using the code C).

In the present embodiment, the authentication system 10 is adopted, for example, as an entrance / exit management system in which a user who has succeeded in face recognition is allowed to enter a specific room. The user carries the mobile terminal 30 as an information code recording medium including the authentication code C, and the image processing device 20 installed at the entrance / exit to the specific room is displayed on the screen of the mobile terminal 30. By performing an operation of imaging the authentication code C, entry into the specific room is permitted when the authentication is successful. Specifically, the image processing device 20 transmits the authentication success information to the lock / unlock control device that controls the lock / unlock of the electric lock provided at the door of the doorway, so that the electric lock is unlocked by the lock / unlock control device. By becoming, entry to the above specific room is permitted.

The authentication code C is configured as a two-dimensional code in which a plurality of light-colored cells and dark-colored cells are arranged, specifically, a QR code (registered trademark), and is used by a user who applies for personal authentication. It is generated by a predetermined code generator that has acquired the face image data. This code generator extracts facial feature points from face image data acquired from the user, and in addition to the face feature point information and the user's personal information regarding the face feature points, information regarding the cell size (hereinafter, simply referred to as "simply"). , Also referred to as cell size) and the like are coded to generate an authentication code C. In the present embodiment, as facial feature points, items such as the size and shape of each facial organ such as eyes, eyebrows, nose, ears, and mouth, contours, and placement positions of facial organs are adopted. Then, in response to the face feature point extraction process for extracting known face feature points performed by the code generator, for example, the face image data D acquired as illustrated in FIG. 2 has features for each item. For example, data such as "D1: cut length of eyes" and "D2: large mouth" are calculated and extracted as facial feature points, respectively. Then, a decryption key (encryption key) is used based on the information about the facial feature points extracted as described above, personal information, cell size, etc. according to the known information code generation process performed by the code generation device. The authentication code C is generated by encrypting the information so that it can be decrypted.

Next, the image processing apparatus 20 will be described with reference to FIGS. 1 and 3.
The image processing device 20 is configured as an authentication device that authenticates the user by using the authentication code C displayed on the screen of the mobile terminal 30, and is an entrance / exit to the specific room, specifically, an entrance / exit to the specific room. , It is installed at the height of the user's line of sight on the wall surface near the door provided at the doorway. As shown in FIG. 3, the image processing device 20 includes a control unit 21, a storage unit 22, an image pickup unit 23, a display unit 24, a lighting unit 25, an operation unit 26, a communication unit 27, and the like.

The control unit 21 is mainly composed of a microcomputer and performs overall control of the image processing device 20 and various operations, and functions to execute authentication processing described later. The storage unit 22 is composed of known storage media such as a ROM, RAM, HDD, and non-volatile memory, and an application program for executing an authentication process, a predetermined database, or the like can be used in advance by the control unit 21. It is stored. Further, the storage unit 22 stores in advance a decryption key for decrypting the authentication code C.

The image pickup unit 23 is configured as a camera having an image pickup lens and an image pickup element (for example, a C-MOS area sensor, a CCD area sensor, etc.) for capturing an image by the image pickup lens, and has a reading window (imaging window). The image data of the captured image captured through the image is output to the control unit 21. In the present embodiment, the image pickup unit 23 is configured as a fisheye camera capable of capturing a wider range by adopting a fisheye lens as the image pickup lens.

As will be described later, the control unit 21 corrects the distortion of the user's face image captured by the image pickup unit 23, and then performs face feature point extraction processing on the corrected image in the same manner as in the above code generation device. By doing so, facial feature points are extracted. Further, the control unit 21 decodes the authentication code C encrypted by using the decryption key stored in the storage unit 22 with respect to the image captured by the image pickup unit 23 for the authentication code C. By performing the processing, the face feature point information, the cell size, and the like recorded in the authentication code C can be obtained as the decoding result.

The display unit 24 is, for example, an LED, which is arranged in a substantially square ring shape so as to surround the reading window of the image pickup unit 23 as shown in FIG. 1, and is controlled by the control unit 21 to extract facial feature points. It is configured so that the lighting / blinking state changes according to the success or failure of the screen and the result of the authentication process. Due to such an arrangement configuration, the image pickup unit 23 is arranged at a position within the visual field range of the user who is viewing the display unit 24.

The illumination unit 25 includes an illumination light source, an illumination lens, and the like, and is controlled by the control unit 21 to irradiate the illumination light toward the image pickup range of the image pickup unit 23. The operation unit 26 is configured to output an operation signal corresponding to the input operation to the control unit 21, and the control unit 21 receives the operation signal and performs processing according to the input operation. There is. The communication unit 27 is configured as a communication interface for performing data communication with an external device such as the lock / unlock control device and a server for entry / exit management, and performs communication processing in cooperation with the control unit 21. It is configured to do.

Next, the mobile terminal 30 will be described with reference to FIGS. 1 and 4.
The mobile terminal 30 shown in FIGS. 1 and 4 is, for example, a smartphone, and is a touch panel whose display contents are controlled by a control unit 31 including a CPU, a memory 32, an image pickup unit 33 configured as a camera, and a control unit 31. 34, an operation unit 35 that outputs a signal to a control unit 31 for a touch operation or a key operation on the touch panel 34, a communication unit 36 for communicating with an external device or the like, and the like are provided.

The authentication code C generated by the above-mentioned code generator is stored in advance in the memory 32 of the mobile terminal 30 configured in this way. Then, the authentication code C read from the memory 32 is displayed on the touch panel 34 on the screen in response to a predetermined operation or the like on the operation unit 35 by the user (see FIG. 1).

Next, the authentication process performed by the image processing device 20 will be described in detail with reference to the flowcharts shown in FIGS. 5 and 6.
When the authentication process is started by the control unit 21 of the image processing device 20, the image pickup process shown in step S101 of FIG. 5 is performed. In this process, the image pickup unit 23 takes an image in a state where the illumination unit 25 irradiates the illumination light through the reading window. Subsequently, in the determination process of step S103, it is determined whether or not a two-dimensional code whose size occupies a specified value (for example, 30% of the imaging region) or more in the captured image is captured. Here, if the two-dimensional code is not imaged, or if the size of the two-dimensional code is less than the above-mentioned specified value even if it is imaged, it is determined as No in step S103, and the process from step S101 is performed.

On the other hand, when a two-dimensional code whose size occupies the captured image is equal to or larger than the above specified value is captured (Yes in S103), it is assumed that the two-dimensional code is the authentication code C, and the captured image is captured. The decoding process is performed using the above-mentioned decoding key (S105). Then, when the authentication code C is successfully decoded (Yes in S107), the captured image successfully decoded is stored (stored) in the storage unit 22. (S109). The control unit 21 that performs the decoding process may correspond to an example of the “decoding unit”.

Subsequently, in the determination process shown in step S111, it is determined whether or not the number of captured images stored in the storage unit 22 is a specified number (for example, 10 images), and the number of stored captured images is the above. If the specified number has not been reached (No in S111), the process from step S101 is performed.

Since the user holds the authentication code C displayed on the screen of the mobile terminal 30 close to the reading window, the authentication code C occupying the captured image is continuously equal to or larger than the above specified value. When the image is captured and the number of times the captured image is stored reaches the above-mentioned specified number (Yes in S111), the face feature point information acquisition process shown in step S113 is performed. In this process, face feature point information, cell size, and the like are acquired based on the decoding result of the decoding process.

Subsequently, the pixel coordinate acquisition process shown in step S115 is performed. In this process, each dark-colored cell constituting the authentication code C is selected as a coordinate comparison cell for the captured images of the specified number of authentication codes C stored in the storage unit 22, and the image plane is displayed. In the reference pixel coordinate system, the process for two-dimensionally calculating the center position of each coordinate comparison cell in the captured image as the pixel coordinates (image coordinates) m _2i (u, v) is applied to each captured image. It is done against.

Next, the three-dimensional position coordinate acquisition process shown in step S117 is performed. In this process, for each captured image of the authentication code C, in the camera coordinate system with reference to the image pickup unit 23, the center position of the actual cell corresponding to the coordinate comparison cell of the authentication code C is three-dimensionally positioned and coordinated. Processing is performed to calculate three-dimensionally as x _i (x, y, z). Specifically, the exact shape of the authentication code C can be grasped by using the cell size acquired as described above with one of the four corners of the authentication code C as the origin, so that the above three-dimensional position can be grasped. The coordinates x _i (x, y, z) can be calculated accurately.

When the pixel coordinates m _2i (u, v) and the three-dimensional position coordinates x _i (x, y, z) are calculated and acquired as described above, the strain coefficient calculation process shown in step S119 is performed. In this process, as shown in FIG. 7, the three-dimensional position coordinates x _i (x, y, z) of the authentication code C and the pixel coordinates m when the authentication code C is imaged are imaged using the Zhang method. The pixel coordinates m _3i (u, v) calculated from the three-dimensional position coordinates _xi (x, y, z) and the pixel coordinates by collecting the pairs for each of the coordinates comparison cells with _2i (u, v). The strain coefficient (camera parameter p) is calculated so that the error with m _2i (u, v) becomes small. When the function for calculating the pixel coordinates m _3i (u, v) is F and the camera parameters are collectively p, it can be expressed as the following equation (1).

ｒ^２＝ｙ’^２＋ｘ’^２ ・・・（６）
ｕ＝ｆ_ｘｘ”＋ｃ_ｘ ・・・（７）
ｖ＝ｆ_ｙｙ”＋ｃ_ｙ ・・・（８）
ここで、（ｘ’，ｙ’）は、ｚ＝１に画像平面があると仮定したときの座標比較用セルを二次元座標画像平面に映し込んだ時の座標（理想）に相当する。また、（ｘ”，ｙ”）は、（ｘ’，ｙ’）に対して歪みを考慮した場合の三次元位置座標に相当する。また、ｋ_１～ｋ_６，ｐ_１～ｐ_２，ｓ_１～ｓ_４は、歪み係数（歪みパラメータ）であり、ｆ_ｘ，ｆ_ｙは、焦点距離であり、ｃ_ｘ，ｃ_ｙは、カメラ（撮像部２３）の光軸中心である。 With respect to the above-mentioned function F, the coordinate comparison cell in _xi (x, y, z) when viewed from the camera coordinate system uses the following equations (2) to (8) to obtain pixel coordinates m _3i (u). , V).
x'= x / z ... (2)
y'= y / z ... (3)

r ² = ^y'2 + ^x'2 ... (6)
u = f _x x "+ c _x ... (7)
v = f _{y y} "+ cy ... (8)
Here, (x', y') corresponds to the coordinates (ideal) when the coordinate comparison cell assuming that the image plane is at z = 1 is projected on the two-dimensional coordinate image plane. Further, (x ", y") corresponds to three-dimensional position coordinates when distortion is taken into consideration with respect to (x', y'). Further, k ₁ to k ₆ , p ₁ to p ₂ , s ₁ to s ₄ are distortion coefficients (distortion parameters), f _x and f _y are focal lengths, and c _x and _cy are cameras. It is the center of the optical axis of (imaging unit 23).

By solving the optimization problem of the above equation (1) so as to use the above equations (2) to (8), the strain coefficients k ₁ to k ₆ , p ₁ to p ₂ , and s ₁ to s ₄ are calculated. be able to. In the optimization problem, various methods can be used, for example, the LM (Levenberg-Marquardt) method and the least squares method can be used. The control unit 21 that performs the distortion coefficient calculation process may correspond to an example of the “distortion coefficient calculation unit”.

When the strain coefficient is calculated as described above, the face imaging process shown in step S121 of FIG. 6 is performed, and the image pickup unit 23 performs a process for imaging the user's face image through the reading window. Subsequently, the image correction process shown in step S123 is performed, and the captured image captured in step S121 is corrected so that the distortion is suppressed according to the distortion coefficient calculated in step S119. The control unit 21 that performs the image correction process may correspond to an example of the “distortion correction unit”.

Subsequently, the face feature point extraction process shown in step S125 is performed, and a process for extracting face feature points from the captured image (corrected image) corrected as described above is performed. If the facial feature points are not extracted from the corrected image (No in S127), the process from step S121 is performed, and the process of extracting the facial feature points from the corrected image that has been imaged again and corrected for distortion is repeated. The control unit 21 that performs the face feature point extraction process can correspond to an example of a "processing unit" that performs a process using the corrected image.

Then, when the face feature points are successfully extracted from the corrected image (Yes in S127), the collation process shown in step S129 is performed, and the decoding result of the face feature points extracted from the corrected image and the authentication code C decoding process (S105) is performed. Using the facial feature point information obtained as, it is collated whether or not all the items in the facial feature points match.

Then, in the determination process shown in step S131, when it is determined that the authentication is successful because all the items at the facial feature points match (Yes in S131), the authentication success information is transmitted to the lock / unlock control device. This completes the authentication process (S133). In the above determination process, it may be determined that the authentication is successful when it is considered that a predetermined number of items at the facial feature points match.

As described above, the authentication success information is transmitted from the image processing device 20, and the lock / unlock control device that receives the authentication success information unlocks the electric lock, so that the user can move to the specific room. You can enter the room.

As described above, in the image processing apparatus 20 according to the present embodiment, the distortion of the image captured by the image pickup unit 23 is corrected by the image correction process (S123) based on the distortion coefficient calculated by the distortion coefficient calculation process (S119). Then, the face feature point extraction process (S125) using this corrected image is performed. In the strain coefficient calculation process, each of the coordinate comparison cells that are at least a part of the plurality of cells in the image captured by the image pickup unit 23 for the authentication code C in which the plurality of cells are two-dimensionally arranged. Distortion coefficient based on the pixel coordinates m _2i (u, v) as coordinates and the three-dimensional position coordinates x _i (x, y, z) of each cell of the authentication code C corresponding to the coordinate comparison cell. Is calculated.

In this way, since the distortion coefficient is calculated using the imaged authentication code C, the strain coefficient can be easily calculated without preparing a dedicated calibration chart or the like. Therefore, since the authentication code C can be imaged and the distortion coefficient can be calculated every time the processing using the captured image is performed, the distortion coefficient is based on the calculated distortion coefficient without performing camera calibration for each camera in advance. It is possible to correct the distortion of the captured image.

Even in a configuration in which the image pickup unit 23 functions as a fisheye camera by adopting a fisheye lens as an image pickup lens, the authentication code C captured every time the face feature point extraction process using the fisheye image is performed is used. By calculating the distortion coefficient, it is possible to correct the distortion of the fisheye image based on the calculated distortion coefficient. The image pickup unit 23 is not limited to being configured as a fisheye camera that employs a fisheye lens, and may be configured as a wide-angle camera that employs a wide-angle lens having a narrower angle of view than the fisheye camera, and is generally used. It may be configured as a camera that employs an image pickup lens. Even with this configuration, distortion of wide-angle images and the like can be corrected.

In particular, a decoding process for decoding the authentication code C is performed on the image captured by the authentication code C captured by the image pickup unit 23, and a face feature point extraction process using the corrected image is performed using the decoded result. Will be. As a result, the distortion coefficient can be calculated from the image captured by the authentication code C captured in order to obtain the decoding result, and the two-dimensional code that does not need to be decoded is not imaged just for calculating the distortion coefficient. Efficiency can be improved.

Further, the face feature point information regarding the face feature points extracted from the user's face image is recorded in the authentication code C, and the face feature point information obtained as the decoding result and the image captured by the image pickup unit 23 are captured. The user authentication process is performed using the facial feature points extracted from the user's face image whose distortion has been corrected by the correction process. As a result, not only the distortion coefficient can be calculated from the image captured by the authentication code C captured in order to obtain the face feature point information, but also the distortion of the user's face image that extracts the face feature point based on the distortion coefficient is corrected. Therefore, the authentication accuracy of the authentication process can be improved.

Further, since the information regarding the cell size of the authentication code C is recorded in the authentication code C, the actual shape of the imaged authentication code C can be accurately grasped. This makes it easier to accurately calculate the three-dimensional position coordinates of each cell of the authentication code C corresponding to the coordinate comparison cell, so that the calculation accuracy of the distortion coefficient, that is, the distortion correction accuracy of the captured image is improved. be able to. The authentication code C may be generated with a predetermined cell size, so that it is not necessary to record the cell size.

Further, in the strain coefficient calculation process, the pixel coordinates m _2i (u, v) of the authentication code C captured a plurality of times according to the specified number and the three-dimensional position coordinates x _i (x, y, z) are used. Based on this, the strain coefficient is calculated. As a result, it becomes easier to suppress the influence of camera shake during imaging as compared with the case where the distortion coefficient is calculated based on the pixel coordinates of the authentication code C and the three-dimensional position coordinates from one captured image. It is possible to improve the calculation accuracy of the distortion coefficient, that is, the distortion correction accuracy of the captured image.

Further, since the authentication code C is displayed on the screen on the mobile terminal 30, the authentication code C itself is not distorted at the time of imaging because it is printed on paper or the like, so that the calculation accuracy of the distortion coefficient, that is, , It is possible to improve the distortion correction accuracy of the captured image. The authentication code C is not limited to being displayed on the screen and used, and may be printed on, for example, a print medium such as an employee ID card or a membership card that is not easily distorted.

[Other embodiments]
The present invention is not limited to the above embodiments and modifications, and may be embodied as follows, for example.
(1) In the authentication process performed by the control unit 21 of the image processing device 20, the distortion coefficient is not limited to being calculated every time the authentication code C held by the user is imaged, but is already calculated within a predetermined period. The processing from the process of step S101 to the process of step S119 may be abolished on the premise of diversion of the strain coefficient.

(2) The coordinate comparison cells to be calculated for the pixel coordinates m _2i (u, v) and the three-dimensional position coordinates x _i (x, y, z) are set in each dark color cell constituting the authentication code C. For example, it may be set in some dark-colored cells or may be set in each light-colored cell. Further, the pixel coordinates m _2i (u, v) and the three-dimensional position coordinates x _i (x, y, z) are not limited to being set at the center position of the coordinate comparison cell, and are, for example, one of the four corners. It may be set.

(3) The present invention is not limited to being adopted in an image processing device 20 that performs authentication processing using a face image, and for example, a predetermined state using a decoding result of a two-dimensional code and a corrected image corrected with an captured image. It may be adopted in the image processing apparatus which performs the processing of. Further, the present invention may be adopted, for example, in an image processing device that uses a corrected image corrected based on a distortion coefficient calculated from a captured image of a two-dimensional code that does not use the decoding result.

(4) The authentication code C is not limited to being configured as a QR code, but is a two-dimensional code of another code type in which a plurality of dark-colored cells and light-colored cells are arranged, for example, a data matrix code. It may be configured as a maxi code or the like, or may be configured as a two-dimensional code in which a plurality of color cells are arranged in a two-dimensional manner.

10 ... Authentication system 20 ... Image processing device 21 ... Control unit (distortion coefficient calculation unit, distortion correction unit, processing unit, decoding unit)
23 ... Imaging unit 30 ... Mobile terminal C ... Authentication code (two-dimensional code)

Claims (7)

An image pickup unit having an image pickup lens and an image pickup element for capturing an image obtained by the image pickup lens, and an image pickup unit.
A distortion coefficient calculation unit for calculating a distortion coefficient for correcting distortion of an image captured by the imaging unit, and a distortion coefficient calculation unit.
A distortion correction unit that corrects the distortion of the captured image of the imaging unit based on the distortion coefficient calculated by the distortion coefficient calculation unit, and a distortion correction unit.
A processing unit that performs processing using the corrected image whose distortion has been corrected by the distortion correction unit, and a processing unit.
Equipped with
The strain coefficient calculation unit is a pixel that is each coordinate of a coordinate comparison cell that is at least a part of the plurality of cells in an image captured by the image pickup unit in which a two-dimensional code in which a plurality of cells are arranged is captured. An image processing apparatus characterized in that the strain coefficient is calculated based on the coordinates and the three-dimensional position coordinates of each of the cells of the two-dimensional code corresponding to the coordinate comparison cell. The image processing apparatus according to claim 1, wherein the image pickup unit functions as a fisheye camera by adopting a fisheye lens as the image pickup lens. A decoding unit that performs a decoding process for decoding the two-dimensional code with respect to the captured image of the two-dimensional code captured by the imaging unit is provided.
The image processing apparatus according to claim 1 or 2, wherein the processing unit performs processing using the corrected image using the decoding result of the decoding unit. In the two-dimensional code, facial feature point information regarding facial feature points extracted from the user's face image is recorded.
The processing unit is a face extracted from the face feature point information obtained as a result of decoding by the decoding unit and the face image of the user imaged by the imaging unit and corrected for distortion by the distortion correction unit. The image processing apparatus according to claim 3, wherein the user authentication process is performed by using the feature points. The image processing apparatus according to claim 3 or 4, wherein the two-dimensional code records information regarding the size of the cell of the two-dimensional code. One of claims 1 to 5, wherein the strain coefficient calculating unit calculates the strain coefficient based on the pixel coordinates of the two-dimensional code and the three-dimensional position coordinates captured a plurality of times. The image processing apparatus according to paragraph 1. The image processing apparatus according to any one of claims 1 to 6, wherein the two-dimensional code is displayed on a screen.

Images