JP2016122947A - Image processing apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus that can perform alignment and reverse distortion correction to a reference image to perform image processing effectively using the reference image and improve image quality.SOLUTION: An image processing apparatus comprises: imaging means; acquisition means that acquires a reference image from the outside; specification means that specifies, on the basis of at least a difference between a photographed image obtained by the imaging means, an angle of view of the reference image and lens aberration information on the imaging means, a reference pixel or a reference area of the reference image corresponding to a pixel of interest or an area of interest of the photographed image; and correction means that corrects the pixel of interest or the area of interest of the photographed image on the basis of an image of the reference pixel or the reference area.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing apparatus that processes an image captured using a reference image and a control method thereof.

  An image captured using an external reference image in an image processing apparatus such as a digital camera, a digital video camera, a camera-equipped mobile phone, or an in-vehicle camera, which is an image capturing apparatus that acquires an image using an image sensor such as a CCD or CMOS. Various techniques have been developed for processing the above (see Patent Document 1). Further, it is known that a captured image is distorted due to the influence of distortion aberration characteristics of the imaging lens.

  When using an external reference image, depending on the order of image processing, a captured image before distortion correction may be processed in consideration of distortion aberration characteristics of the imaging lens. For this reason, it is conceivable that the reference image is used for image processing after performing inverse distortion correction using the distortion correction technique described in Patent Document 2, Patent Document 3, and the like.

JP 2007-324887 A JP 2012-65263 A JP 2002-94860 A

However, with the techniques disclosed in Patent Document 2 and Patent Document 3, when the reference image is an image photographed from a different angle of view and angle from the captured image, the reference image cannot be correctly used only by distortion correction. There is a possibility, and various countermeasures against it have been desired.
An object of the present invention is to provide an image processing apparatus that can perform image processing using a reference image effectively and perform image quality improvement by performing alignment and inverse distortion correction on the reference image. There is.

In order to achieve the above object, an image processing apparatus of the present invention provides:
Imaging means;
An acquisition means for acquiring a reference image from outside;
The reference image corresponding to the pixel of interest or the region of interest of the captured image based on at least the difference in angle of view between the captured image obtained by the imaging unit and the reference image and lens aberration information of the imaging unit A specifying means for specifying a reference pixel or a reference area of
Correction means for correcting the pixel of interest or the region of interest of the captured image based on the image of the reference pixel or the reference region;
It is characterized by providing.

  According to the present invention, it is possible to provide an image processing apparatus that can perform image processing that effectively uses a reference image by performing alignment and inverse distortion correction on the reference image, and can improve image quality. realizable.

1 is a block diagram of an imaging apparatus according to a first embodiment of the present invention. It is an image figure of communication between cameras in the 1st example of the present invention. It is a block diagram of the image processing part in the 1st example of the present invention. It is a figure for demonstrating distortion correction coordinate calculation in 1st Example of this invention. It is an image figure figure in the 1st example of the present invention. It is a flowchart of the crack correction process in 1st Example of this invention. It is a flowchart of NR processing in the 1st example of the present invention. It is a flowchart of the gamma correction process in 1st Example of this invention. It is a block diagram of the image processing part in the 2nd example of the present invention. It is a flowchart of the defect correction process in 2nd Example of this invention. It is a flowchart of the NR process in the 2nd Example of this invention. It is a flowchart of the gamma correction process in 2nd Example of this invention.

[Example 1]
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. The image processing apparatus according to the present embodiment will be described as a part of the imaging apparatus. Although the imaging apparatus is described as a digital camera, an imaging apparatus such as a digital video camera, a camera-equipped mobile phone, and an in-vehicle camera can be employed.

  FIG. 1 is a block diagram showing an example of the configuration of the digital camera according to the first embodiment. Hereinafter, FIG. 1 will be described. The imaging element 100 is a light receiving element such as a CCD or CMOS sensor that converts received light into an electrical signal to create image data. The A / D converter 101 converts the analog output signal of the image sensor 100 into a digital signal.

  Reference data 102 is image data scheduled to be taken by the imaging apparatus of the present embodiment and image data of the same subject, landscape, and the like. Note that the reference data 102 may be image data taken at the same time or at different times. Furthermore, it may be image data taken from different angles and angles of view. The image processing unit 103 performs an acquisition unit 104, an identification unit 105, a correction unit 106, a data transfer unit 107, a recognition unit 116, a plurality of image processing units (not shown), an image buffer memory, and image data compression processing and decompression processing. It comprises a compression / expansion unit, a display control unit for displaying on a monitor, a storage processing unit for storing in a recording medium, and the like.

  The acquisition unit 104 is an acquisition unit that acquires reference data 102, position and orientation information of the digital camera, and the like. As shown in FIG. 2, reference data, position and orientation information, and the like are acquired by communicating with the digital camera 201 and the external digital camera 200 or a server (not shown). Alternatively, it can be obtained from a recording medium (not shown). The position / orientation information uses position information obtained by GPS and attitude information obtained by a gyroscope or inertial navigation sensor as position / orientation information of the digital camera. However, the position / orientation information may be acquired using another means.

  The specifying unit 105 aligns the reference data 102 so that the angle of view and the position are the same as those of the digital camera, and further performs reverse distortion correction in consideration of lens characteristics of the digital camera. Details will be described later. The correction unit 106 performs image processing such as scratch correction, noise reduction (hereinafter referred to as NR), and gamma correction. Details will be described later.

  The data transfer unit 107 includes WRDMAC and RDDMAC of a plurality of Direct Memory Access controllers that perform data transfer (not shown). The image data is output to the bus 109 by WRDMAC and temporarily stored in the memory 111 via the memory control unit 110. The image data temporarily stored in the memory 111 is output from the memory 111 to the bus 109 by RDDMAC, and the data is output to the image processing unit 103.

  The recognition unit 116 authenticates face recognition, human body recognition, and also a person registered in the nonvolatile memory 113 or an external memory (not shown) and a subject person. A bus 108 is a system bus, and a bus 109 is an image data bus. In this embodiment, the system bus and the image data bus are separated, but the same bus may be used.

  The memory control unit 110 writes data into the memory 111 or reads data from the memory 111 in accordance with an instruction from the system control unit (CPU) 114 or the data transfer unit 107. Note that output data from the A / D converter 101 may be directly written into the memory 111. The memory 111 is a storage device having a storage capacity sufficient to store a predetermined number of still images, moving images for a predetermined time, data such as sound, constants for operating the system control unit 114, programs, and the like.

  The non-volatile memory control unit 112 writes data to the non-volatile memory 113 or reads data from the non-volatile memory 113 in accordance with an instruction from the system control unit 114. The nonvolatile memory 113 is an electrically erasable / recordable memory, and for example, an EEPROM or the like is used. The nonvolatile memory 113 stores constants, programs, and the like for operating the system control unit 114.

  The system control unit 114 includes a microcomputer that controls the operation of the digital camera, and performs various instructions to the functional blocks constituting the digital camera and executes various control processes. The system control unit 114 controls the image processing unit 103, the data transfer unit 107, the memory control unit 110, the nonvolatile memory control unit 112, the operation unit 115, and the image sensor 100 connected via the bus 108. Execution of the microcomputer realizes each process of the present embodiment by executing the program recorded in the nonvolatile memory 113 described above. The operation unit 115 includes switches and buttons operated by the user, and is used for operations such as power ON / OFF and shutter ON / OFF.

  FIG. 3 is a block diagram illustrating the specifying unit 105, the correction unit 106, and the data transfer unit 107 according to the first embodiment. Hereinafter, FIG. 3 will be described. In addition, about the same part as FIG. 1, detailed description is abbreviate | omitted by attaching | subjecting the same code | symbol.

  The alignment unit 300 calculates the address before conversion (reference data 102) based on the coordinates after conversion (processed image 310) so that the reference data 102 has the same angle of view as the processed image 310. For the calculation of the address, the position / orientation information of the digital camera 201 and the external digital camera 200 is acquired from the acquisition unit 104, and is calculated by projective transformation based on the information. Next, based on the converted coordinates, the reference data 102 is corrected and alignment is performed. Note that other methods may be used for alignment. For the correction, the same interpolation as the distortion correction described later is used.

  Since the distortion correction unit 301 distorts the image in the same manner as the processed image 310, it calculates reverse distortion correction coordinates in consideration of lens information that matches the angle of view. Although the calculation of the correction coordinates will be described with reference to FIG. 4, other methods may be adopted as long as aberration correction is possible.

  FIG. 4 is an image of a two-dimensional orthogonal coordinate system in which the output pixel of the image sensor 100 defines the horizontal axis along the scanning direction and the vertical axis orthogonal to the scanning direction. It is the figure which showed the pixel position 402 of this and the pixel position 401 of the image after correction | amendment. It is known that the correction amount C for correcting the pixel position 402 of the uncorrected image is obtained from the image height R representing the distance from the center of the image and the characteristics of the optical system. The pixel position of the uncorrected image is calculated by converting the angle θ between the vertical direction of the center 400 of the image and the direction of the pixel position 401 of the corrected image from the center 400 of the image and the correction amount C. Further, the pixel position 402 of the pre-correction image is not necessarily a pixel unit of the image sensor, and may be moved between pixels of the image sensor, and the decimal point is calculated.

  The corrected reference data 307 is created by interpolating using the peripheral pixels using an image buffer memory (not shown) around the calculated distortion correction coordinates. For the interpolation, bilinear, bicubic interpolation or the like is used, but other methods may be adopted. The image data 306 is an image of the reference data 102. Further, the image data 308 is an image of the corrected reference data 307 that has been aligned and corrected by the specifying unit. In this embodiment, the description has been made in the order of alignment and distortion correction. However, after the distortion correction unit 301 performs distortion correction, the alignment unit 300 may perform alignment.

  The image data 309 is an image image of the processed image 310 obtained by converting the data captured by the image sensor 100 by the A / D converter 101. Further, the image data 309 may be an image after image processing (not shown) of the image processing unit 102 is appropriately performed, for example, white balance, black level correction, and shading correction. The scratch correction unit 302 corrects the processed image 310 using the correction reference data 307 and creates a processed image 311.

  The NR unit 303 corrects the processed image 311 using the corrected reference data 307 and creates a processed image 312. The gamma correction unit 304 corrects the processed image 312 using the corrected reference data 307 and creates a processed image 313. The scratch correction unit 302, the scratch correction unit 302, and the gamma correction unit 304 will be described later with reference to flowcharts.

  The distortion correction unit 305 corrects the processed image 313 by the same method as the distortion correction unit 301. The image data 314 is an image after the processed image 313 is corrected by the distortion correction unit 305. The distortion correction unit 301 and the distortion correction unit 305 may be exclusively used as one block. Note that the order of image processing does not matter. In addition, reading and writing are performed in the memory every time image processing is performed, but each processing unit in the image processing unit 102 may be directly connected to perform image processing.

  FIG. 5 is an image diagram showing a reference image 500 and a captured image 501 processed by the specifying unit 105. The reference image 500 is an image in which the face of the person B is partially blocked by the arm of the person A. The captured image 501 is also an image diagram illustrating use-prohibited areas 502 and 503 that prohibit use of the reference image in image processing to be described later. The recognition unit 116 recognizes the persons A and B in the captured image 501 and then recognizes the reference image 500. At this time, only the person A is recognized in the reference image 500, the face of the person B is not recognized, and the human body is recognized. Based on the recognition information of the captured image 501 and the reference image 500, the use prohibited areas 502 and 503 are determined.

  FIG. 6 is a flowchart showing a flaw correction processing flow. In step 600, the system control unit 114 reads the flaw address information held in the nonvolatile memory 113 and the use-prohibited area address held in the memory 111 and sets them in the flaw correction unit 302. In step 601, the scratch correction unit 302 determines whether the scratch address is a prohibited area. If it is a use prohibition area, the process proceeds to step 602, where the defect correcting unit 302 performs defect correction. If it is not a prohibited area, the process proceeds to step 603. In step 603, the defect correction unit 302 reads the corrected reference image 307 from the memory 111.

  In step 604, the defect correcting unit 302 replaces the pixel at the corresponding address in the processed image 310 with the corrected reference image 307 based on the defect address information. Further, after the replacement, processing such as a smoothing filter and a moving average filter may be performed centering on the scratch address. When the process of step 602 or step 604 is completed, the process proceeds to step 605. In step 605, the defect correction unit 302 determines whether all the processes have been completed for the defect address of the processed image 310. If the process has not been completed, the process returns to step 601. If the scratch correction unit 302 determines that all the scratch corrections have been completed, the scratch correction process is terminated.

  FIG. 7 is a flowchart showing the NR processing flow. In step 700, the system control unit 114 reads out the prohibited area address held in the memory 111 and sets it in the NR unit 303. In step 701, the NR unit 303 determines whether the pixel or area to be processed is a prohibited area. If it is determined in step 701 that the area is a use-prohibited area, the process proceeds to step 706. If it is determined that the area is not a use-prohibited area, the process proceeds to step 702. In step 702, the corrected reference image 307 is read from the memory 111 and edge determination is performed. If it is determined that there is no edge, the process proceeds to step 706, and if it is determined that there is an edge, the process proceeds to step 703.

  In step 703, the NR unit 303 reads the processed image 311 from the memory 111 and performs edge determination. If it is determined that there is no edge, the process proceeds to step 704. If it is determined that there is an edge, the process proceeds to step 706. In step 704, the NR unit 303 acquires image data corresponding to a region determined to have no edge in the processed image 311 from the corrected reference image 307. In step 705, the NR unit 303 replaces the data acquired in step 704 with the data of the processed image 311. In step 706, NR processing is performed by applying filter processing such as a median filter and a Gaussian filter in consideration of edge information.

  Further, processing other than the median filter and Gaussian filter may be applied to the NR processing. In step 707, the NR unit 303 determines whether to end the process. If it is determined that the process has not ended, the process returns to step 700 to continue the process. If it is determined to end, the NR process ends.

  FIG. 8 is a flowchart showing a gamma correction processing flow. In step 800, the system control unit 114 reads out the use-prohibited area address held in the memory 111 and sets it in the gamma correction unit 304. In step 801, the gamma correction unit 304 determines whether the pixel or area to be processed is a prohibited area. If it is determined in step 801 that the area is a use-prohibited area, the process proceeds to step 802. If it is determined that the area is not a use-prohibited area, the process proceeds to step 803. In step 802, the gamma correction unit 304 performs gamma correction on the processed image 312.

  In step 803, the gamma correction unit 304 reads the corrected reference image 307 corresponding to the processing target area from the memory 111. Subsequently, in step 804, the gamma correction unit 304 performs light / dark determination on the region in step 803. In step 805, the gamma correction unit 304 performs gamma correction on the processed image 312 based on the brightness result in step 804. In step 806, the gamma correction unit 304 determines whether or not to end the process. If it is determined that the process is not ended, the process returns to step 800 to continue the process. If it is determined to end, the gamma correction processing is ended.

  As described above, according to the first embodiment of the present invention, by using the alignment unit 300 and the distortion correction unit 301 of the specifying unit 105, the reference image data has the same position as the captured image. Thus, it is possible to perform alignment and to apply reverse distortion correction based on the lens information. Therefore, when processing the image data before distortion correction, the reference image data can be used for flaw correction, NR, gamma correction, etc. of the captured image data, and high image quality can be achieved.

  The present invention has been specifically described above based on the first embodiment. However, the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the scope of the present invention. Needless to say.

[Example 2]
Next, a second embodiment of the present invention will be described. Here, only portions different from the first embodiment described above will be described, and the same portions will be denoted by the same reference numerals and detailed description thereof will be omitted.

  FIG. 9 is a block diagram illustrating the specifying unit 105, the correction unit 106, and the data transfer unit 107 of the image processing unit 103 according to the second embodiment. Hereinafter, FIG. 9 will be described. 1 and 3 are denoted by the same reference numerals, and detailed description thereof is omitted. The alignment coordinate calculation unit 900 calculates an address before conversion (reference data 102) based on coordinates after conversion (processed image 310) so that the reference data 102 has the same angle of view as the processed image 310. For the calculation of the address, the position / orientation information of the digital camera 201 and the external digital camera 200 is acquired from the acquisition unit 104, and is calculated by projective transformation based on the information.

  The distortion correction coordinate calculation unit 901 calculates correction coordinates based on the coordinates calculated by the projective transformation of the alignment coordinate calculation unit 900 in order to distort the same as the processed image 311. The calculation of the correction coordinates is calculated in the same manner as in the first embodiment. Further, the distortion correction coordinate calculation unit 901 can be used as a partial function of the distortion correction units 301 and 305. The scratch correction unit 902 corrects the processed image 310 using the reference data 102 and creates a processed image 311. The NR unit 903 corrects the processed image 311 using the corrected reference data 102 and creates a processed image 312. The gamma correction unit 904 corrects the processed image 312 using the reference data 102 and creates a processed image 313.

  The scratch correction unit 902, the NR unit 903, and the gamma correction unit 904 will be described later with reference to flowcharts. Note that the order of image processing does not matter. In addition, reading and writing are performed in the memory every time image processing is performed, but each processing unit in the image processing unit 102 may be directly connected to perform image processing.

  In FIG. 5 described in the first embodiment, the reference image 500 is an image after being processed by the specifying unit 105. However, in the present embodiment, the reference image 500 is calculated before being processed by the specifying unit 105, that is, using the reference image data 102 as the reference image 500. Alternatively, based on the coordinate information obtained by the specifying unit 105, the reference image data 102 can be interpolated by the distortion correction unit 305 to obtain the reference image 500. In this case, the reference image 500 is equivalent to that in the first embodiment.

  FIG. 10 is a flowchart showing a flaw correction processing flow. In step 1000, the system control unit 114 reads out the flaw address information held in the nonvolatile memory 113 and the use-prohibited area address held in the memory 111 and sets them in the flaw correction unit 902. In step 1002, a flaw address is set in the specifying unit 105 to acquire the address of the reference image data 102. In step 1002, the scratch correction unit 902 determines whether the scratch address is a prohibited area. If it is a use prohibition area, the process proceeds to step 1003, and the defect correction unit 902 performs defect correction.

  If it is not a prohibited area, the process proceeds to step 1004. In step 1004, the scratch correction unit 902 reads the reference image data 102 from the memory 111. In step 1005, the defect correction unit 902 interpolates the reference image data 102 based on the address information acquired in step 1001 and replaces the pixel at the corresponding address in the processed image 310. Further, after the replacement, processing such as a smoothing filter and a moving average filter may be performed centering on the scratch address. When the processing of step 1003 or step 1005 is completed, the process proceeds to step 1006.

  In step 1006, the defect correcting unit 902 determines whether all the processes have been completed for the defect address of the processed image 310. If the process has not been completed, the process returns to step 1000. If the scratch correction unit 902 determines that all the scratch corrections have been completed, the scratch correction processing is terminated.

  FIG. 11 is a flowchart showing the NR processing flow. In step 1100, the system control unit 114 reads the use-prohibited area address held in the memory 111 and sets it in the NR unit 903. In step 1101, the NR unit 903 determines whether the pixel or area to be processed is a prohibited area. If it is determined in step 1101 that the area is a prohibited area, the process proceeds to step 1107. If it is determined that the area is not a prohibited area, the process proceeds to step 1102. In step 1102, an edge determination area is set in the specifying unit 105 to acquire the address of the reference image data 102.

  In step 1103, the reference image data 102 is read from the memory 111, and edge determination is performed after interpolation. If it is determined that there is no edge, the process proceeds to step 1107. If it is determined that there is an edge, the process proceeds to step 1104. In step 1104, the NR unit 903 reads the processed image 311 from the memory 111, and performs edge determination. If it is determined that there is no edge, the process proceeds to step 1105, and if it is determined that there is an edge, the process proceeds to step 1107. In step 1105, the NR unit 903 acquires from the reference image data 102 image data corresponding to a region determined to have no edge in the processed image 311.

  In step 1106, the NR unit 903 replaces the data acquired in step 1105 with the data of the processed image 311 after interpolation. In step 1107, NR processing is performed by applying filter processing such as a median filter and a Gaussian filter in consideration of edge information. Further, processing other than the median filter and Gaussian filter may be applied to the NR processing. In step 1108, the NR unit 903 determines the end of the process. If it is determined that the process is not ended, the process returns to step 1100 to continue the process. If it is determined to end, the NR process ends.

  FIG. 12 is a flowchart showing a gamma correction processing flow. In step 1200, the system control unit 114 reads out the use-prohibited area address held in the memory 111 and sets it in the gamma correction unit 904. In step 1201, the gamma correction unit 904 determines whether the pixel or area to be processed is a prohibited area. If it is determined in step 1201 that the area is a use-prohibited area, the process proceeds to step 1202. If it is determined that the area is not a use-prohibited area, the process proceeds to step 1203. In step 1202, the gamma correction unit 904 performs gamma correction on the processed image 312.

  In step 1203, the processing area is set in the specifying unit 105 and the address of the reference image data 102 is acquired. In step 1204, the gamma correction unit 904 reads the reference image data 102 corresponding to the processing target area from the memory 111 and corrects it. Subsequently, in step 1205, the gamma correction unit 904 performs light / dark determination on the area in step 1204. In step 1206, the gamma correction unit 904 performs gamma correction on the processed image 312 based on the brightness result in step 1205. In step 1207, the gamma correction unit 904 determines whether or not to end the process. If it is determined that the process is not ended, the process returns to step 1200 to continue the process. If it is determined to end, the gamma correction processing is ended.

  As described above, according to the second embodiment of the present invention, by using the alignment coordinate calculation unit 900 and the distortion correction coordinate calculation unit 901 of the specifying unit 105, the captured image is obtained with respect to the reference image data. The reverse distortion correction coordinates based on the same position and lens information can be obtained. Therefore, when processing the image data before distortion correction, it can be used for defect correction, NR, gamma correction, etc. of the captured image data based on the reference image data and the address information output by the specifying unit 105. High image quality can be achieved. Furthermore, the correction is not performed on all the pixels with respect to the reference image data, but only the portions that need to be corrected with the captured image data can be made, so that the memory access bandwidth can be saved.

  The present invention has been specifically described above based on the second embodiment. However, the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the scope of the present invention. Needless to say.

DESCRIPTION OF SYMBOLS 100 Image sensor, 101 A / D converter, 102 Image processing part, 103 Imaging process part,
104 acquisition unit, 105 identification unit, 106 correction unit, 116 recognition unit,
107 data transfer unit, 108 system bus, 109 data bus,
110 memory control unit, 111 memory, 112 non-volatile memory control unit,
113 nonvolatile memory, 114 system control unit, 115 operation unit,
300 Alignment unit, 301 Distortion correction unit, 302 Scratch correction diagram, 303 NR unit,
304 gamma correction unit, 305 distortion correction unit, 900 alignment coordinate calculation unit,
901 distortion correction coordinate calculation unit, 902 scratch correction diagram, 903 NR unit,
904 Gamma correction unit

Claims (11)

Imaging means;
An acquisition means for acquiring a reference image from outside;
At least the difference in the angle of view between the captured image obtained by the imaging unit and the reference image and the reference image corresponding to the target pixel or the target region of the captured image based on the lens aberration information of the imaging unit. A specifying means for specifying a reference pixel or a reference region;
Correction means for correcting the pixel of interest or the region of interest of the captured image based on the image of the reference pixel or the reference region;
An image processing apparatus comprising: The specifying means is:
Alignment means for specifying and aligning a reference pixel or a reference area of the reference image based on a difference in angle of view between the captured image and the reference image;
Distortion correction means for performing distortion correction by specifying a reference pixel or a reference region of the reference image based on lens aberration information of the imaging means;
The image processing apparatus according to claim 1, further comprising: The specifying means is:
Alignment coordinate calculation means for specifying a reference pixel or a reference area of the reference image based on a difference in angle of view between the captured image and the reference image and calculating coordinates for alignment;
Distortion correction coordinate calculation means for calculating coordinates for correcting distortion by specifying a reference pixel or a reference area of the reference image based on lens aberration information of the imaging means;
The image processing apparatus according to claim 1, further comprising: The acquisition means includes
The image processing apparatus according to any one of claims 1 to 3, further comprising position / orientation information acquisition means for acquiring position / orientation information obtained by capturing the reference image and position / orientation information of the image processing apparatus. . Recognizing means for recognizing and authenticating a human face and human body with respect to the captured image and the reference image;
The recognition unit recognizes the captured image and the reference image, and sets a region where there is a difference between recognition results of the captured image and the reference image as a prohibited region. 5. The image processing device according to any one of 4. The correction means includes
Flaw correction means for correcting flaw pixels or flaw areas of the picked-up image using flaw address information unique to the image pickup device and a corrected reference image obtained by aligning the reference image by the specifying means and correcting the distortion;
The image processing apparatus according to claim 1, further comprising: The correction means includes
Edge determination means for performing edge determination of the corrected reference image obtained by aligning the reference image by the specifying means and correcting distortion, and edge determination of the captured image;
Edge use determination means for determining whether to use an edge of the corrected reference image;
Noise removing means for removing noise by replacing an edge region of the reference image with respect to the captured image according to a result of an edge use determining means;
The image processing apparatus according to claim 1, further comprising: The correction means includes
Brightness / darkness discrimination means for discriminating the brightness of an image in a certain area with respect to the corrected reference image obtained by aligning the reference image by the specifying means and correcting the distortion;
Gamma correction means for performing gamma correction on at least one region of the captured image according to the result of the light / dark discrimination means;
The image processing apparatus according to claim 1, further comprising: The correction means includes
Based on the flaw address information specific to the image sensor and the flaw address information, using a partially corrected reference image obtained by correcting a part of the reference image from the alignment coordinates and distortion correction coordinates calculated by the specifying unit,
Flaw correction means for correcting flaw pixels or flaw areas of the captured image;
The image processing apparatus according to claim 1, further comprising: The correction means includes
Edges for performing edge determination of a partially corrected reference image obtained by correcting a part of the reference image based on alignment coordinates and distortion correction coordinates calculated by the specifying unit and edge determination of the captured image based on address information of a certain area A determination means;
Edge use determination means for determining whether to use an edge of the partially corrected reference image,
Noise removing means for removing noise by replacing an edge region of the reference image with respect to the captured image according to a result of an edge use determining means;
The image processing apparatus according to claim 1, further comprising: The correction means includes
Brightness / darkness determination that determines the brightness of an image in a region with respect to a partially corrected reference image obtained by correcting a part of the reference image based on alignment coordinates and distortion correction coordinates calculated by the specifying unit based on address information of a certain region Means,
Gamma correction means for performing gamma correction on at least one region of the captured image according to the result of the light / dark discrimination means;
The image processing apparatus according to claim 1, further comprising: