KR20160065751A

KR20160065751A - Control apparatus, image processing apparatus, lens apparatus, image processing system, control method and image processing method

Info

Publication number: KR20160065751A
Application number: KR1020150166897A
Authority: KR
Inventors: 쿠니히로 시라이
Original assignee: 캐논 가부시끼가이샤
Priority date: 2014-12-01
Filing date: 2015-11-27
Publication date: 2016-06-09
Also published as: JP6573386B2; CN105657248A; CN105657248B; JP2016110622A; KR101862643B1

Abstract

A control apparatus (103) comprises a memory part (103b) which memorizes first data including coefficient data, and a determining part (103c) which determines second data less than the first data, from the first data memorized in the memory part, based on the information of an image processing apparatus for performing an image process on an image photographed by using an optical system. The optical transfer function of the optical system can represent the coefficient data as the coefficient of an approximation function with a preset dimension. So, proper OTF data can be provided to the image processing apparatus.

Description

TECHNICAL FIELD [0001] The present invention relates to a control apparatus, an image processing apparatus, a lens apparatus, an image processing system, a control method, and an image processing method.

The present invention relates to an image processing system for correcting a deteriorated image by an imaging optical system with high resolution and high quality.

The subject image photographed through the imaging optical system can not converge light generated from one point to another point due to the influence of diffraction or aberration generated in the imaging optical system and has a small diffusion. This slightly diffused distribution is called a point-like intensity distribution function (PSF). Due to the influence of such an imaging optical system, the photographed image is blurred as the PSF is formed by being folded on the object, and the resolution is deteriorated.

In recent years, it has been common to store photographed images as electronic data, and a technique for correcting image degradation caused by an optical system by image processing has been proposed. Japanese Patent No. 4337463 discloses an image processing method of storing filter coefficients for correcting image deterioration and performing image processing. Japanese Patent Laying-Open No. 2013-33496 discloses an image processing method for storing filter coefficients of a predetermined approximate function for correcting image deterioration to compensate for image deterioration.

However, in the image processing method disclosed in Japanese Patent No. 4337463, when performing the deterioration correction of the photographed image, it is necessary to store the information (OTF data) of the optical transfer function for creating the image recovery filter for each pixel. Since the OTF data is calculated on the basis of the respective information of the image pickup element and the imaging optical system, it becomes a large amount, and it is difficult to store all the OTF data in each apparatus accordingly. In the image processing method disclosed in JP-A-2013-33496, it is possible to reduce OTF data, but even when approximation is appropriately performed for a specific apparatus, there is a possibility that a good correction effect can not be obtained in other apparatuses.

The present invention provides a control device, a lens device, an image processing system, and a control method capable of providing appropriate OTF data in accordance with an image processing apparatus that performs image restoration processing.

The present invention further provides an image processing apparatus, an image processing system, and an image processing method capable of storing appropriate OTF data for performing image restoration processing.

A control device as one aspect of the present invention includes: a storage unit that stores first data including a plurality of coefficient data; and a control unit that, based on information of an image processing apparatus that performs image processing on an image photographed using the optical system, And a determination unit that determines second data having a data amount smaller than that of the first data from the first data stored in the storage unit, wherein the optical transfer function of the optical system is a function of transferring the plurality of coefficient data As the coefficient of the approximate function having the degree of the "

A lens apparatus as another aspect of the present invention includes an optical system for forming a subject image and the control device.

An image processing system as another aspect of the present invention includes the control device and an image processing device that performs image restoration processing using the second data.

According to another aspect of the present invention, there is provided a control method including: inputting information of an image processing apparatus that performs image processing on an image photographed using an optical system; And determining second data having a smaller data amount than the first data, from the first data including the coefficient data of the first coefficient data, wherein the optical transfer function of the optical system converts the plurality of coefficient data into a predetermined As a coefficient of an approximate function having a degree.

An image processing apparatus according to another aspect of the present invention includes an input unit for receiving first data including a plurality of coefficient data, and an output unit for outputting, from the first data, second data having a smaller data amount than the first data And a processing unit for performing image processing on an image photographed using an optical system by using the second data, wherein the optical transfer function of the optical system is a function of converting the plurality of coefficient data into a predetermined degree As the coefficient of the approximate function having the function of the function.

An image processing system as another aspect of the present invention includes the image processing apparatus and a control device for outputting the first data.

An image processing method as another aspect of the present invention includes the steps of: receiving first data including a plurality of coefficient data; determining, from the first data, second data having a smaller data amount than the first data And performing image processing on an image photographed by using an optical system by using the second data, wherein the optical transfer function of the optical system is a function of converting the plurality of coefficient data into a plurality of coefficient data having a predetermined degree Can be represented by using as an approximate function coefficient.

Further features and aspects of the present invention will become apparent from the following description of embodiments with reference to the accompanying drawings.

1A is a configuration diagram of an image processing system according to the first embodiment.
Fig. 1B is a block diagram of the control device in the first embodiment. Fig.
1C is a block diagram of an image processing apparatus according to the first embodiment.
Fig. 2 is a flowchart showing data transfer processing of OTF data in the first embodiment.
3 is an example of a table showing the relationship between the connected device (image pickup device) and the maximum degree in the first embodiment.
4 is a configuration diagram of an image processing system in the second embodiment.
Fig. 5 is a sequence diagram of the image processing system in the second embodiment. Fig.
6A to 6D are examples of the OTF data in the second embodiment.
7A to 7C are graphs showing the MTF and the maximum gain in the second embodiment.
8A to 8D are examples of OTF data according to the aperture stop in the second embodiment.
9A and 9B are diagrams of a reference table and a maximum gain table in the second embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

First, a general image recovery method will be described. (X, y), a point-like intensity distribution function (PSF) is denoted by h (x, y), and a degraded image is denoted by g (x, y) , The following expression (1) is satisfied.

g (x, y) =? f (X, Y) h (x-X, y-Y) dXdY ... (One)

When the Fourier transform is performed on the equation (1) to convert from the actual space (x, y) to the frequency space (u, v), the following equation (2) is satisfied.

G (u, v) = F (u, v) * H (u, v) ... (2)

As a result of Fourier transform of f (x, y), F (u, v) is a result of Fourier transform of g (x, y) ) Is the result of the Fourier transform of h (x, y). According to the equation (2), the following equation (3) is satisfied.

F (u, v) = G (u, v) / H (u, v) ... (3)

The equation (3) shows the result F (u, v) of the Fourier transform of the degraded image g (x, y) in the frequency space as a result H (x, y) of the Fourier transform h (u, v) to obtain the Fourier transform result F (u, v) of the non-degraded image f (x, y). Therefore, by performing Fourier inverse transform on F (u, v), an image f (x, y) which has not been degraded can be obtained.

However, in reality, when such an image f (x, y) obtained by performing such processing is obtained, the noise generated by the image pickup device is amplified, and thus a good image can not be obtained.

In order to solve this problem, it is known to use a Wiener filter W (u, v) expressed by the following equation (4) as an image recovery method for suppressing the amplification of noise.

1 / H (u, v) * (| H (u, v) | 2 / (| H (u, v) | 2 + Γ)) ... (4)

In equation (4), the symbol H (u, v) is an optical transfer function (OTF), and the symbol Γ is an integer for reducing the amount of amplification of noise.

By integrating the equation (4) into the OTF having the frequency information and the phase information of the imaging optical system, the phase of the PSF generated due to the diffraction or aberration of the optical system becomes zero, and as the frequency characteristic is amplified, Can be obtained. In order to effectively use the equation (4), it is necessary to obtain accurate OTF information of the imaging optical system. When the design value information of the imaging optical system is available as the method of obtaining the OTF information, the OTF information can be obtained by calculation based on the design value information. Alternatively, OTF information can be obtained by photographing a point light source and performing Fourier transform on the point intensity distribution function (PSF). Generally, in an imaging optical system used in a camera, its optical performance (F value and aberration) fluctuates largely depending on the image height. Thus, in order to correct the deterioration of the object, the equation (4) can not be directly calculated in the frequency space, and instead, the equation (4) is converted into a filter on the actual space for each image height, .

The optical image (object image) formed (formed) by the imaging optical system is electrically sampled by the imaging element. Since the optical image having the original continuous value is converted into the discrete value, the optical image has the frequency signal of the period of the sampling frequency in the frequency space. Due to this periodicity, when the frequency signal is distributed over one-half of the sampling frequency, the accurate signal can not be reproduced as the frequency signal overlaps. A value of one half of this sampling frequency is called a Nyquist frequency. The Nyquist frequency is represented by fn = 1 / (2 * b), where b is the pixel pitch of the image pickup device.

The spatial frequency characteristic of the optical image immediately before the image pickup element is represented by the OTF of the imaging optical system. It is desirable to match the size of one tap of the image recovery filter with the size of one pixel of the image pickup device and also to reflect the aperture characteristics of the image pickup device. The former corresponds to cutting the OTF to a spatial frequency at which the Nyquist frequency of the image pickup element is the maximum, and the latter corresponds to the case where the low-pass filter is applied to the OTF by the image pickup element. Therefore, the OTF information used for creating the image recovery filter is not solely determined by the imaging optical system alone, but depends on the imaging element.

(Embodiment 1)

First, the image processing system according to the first embodiment of the present invention will be described with reference to FIG. 1A. Fig. 1A is a configuration diagram (overall view) of an image processing system in the present embodiment. 1A, reference numeral 101 denotes an information processing apparatus for calculating and storing an optical transfer function (OTF) of an imaging optical system based on a design value or a measured value. The information processing apparatus 101 is provided by a provider that provides optical transfer function data (OTF data) for correction of a shot image. OTF data created by the information processing apparatus 101 can be managed on the network 102. [

Next, a method of creating data of OTF data created by the information processing apparatus 101 will be described in detail. In this embodiment, a method of generating and storing a coefficient by approximating an OTF (design value or measured value) of an imaging optical system by fitting processing to a predetermined function will be described. As a function used in fitting processing, the present embodiment uses a Legendre polynomial. However, the present embodiment is not limited to this, and another orthogonal function such as a Chebyshev polynomial may be used. The Rajendor polynomial is expressed by the following equation (5). In this equation (5), the symbol [x] is the maximum integer that does not exceed the value of x

The OTF is expressed in the form of z = f (x, y). For this reason, in the present embodiment, it is necessary to calculate the coefficient a _ij in the following expression (6).

Equation (6) is an orthogonal function (approximate function), and the value of the coefficient a _ij is determined without depending on the order used in the fitting process. As described above, the storage amount of necessary data can be reduced by approximating the OTF of the imaging optical system by fitting processing to a predetermined function and creating a coefficient. Further, by using the characteristic of the orthogonal function expressed by the equation (6), the fitting process of the OTF can be stopped with a low degree to the extent that the fitting process can be performed with high accuracy with a high degree, .

The real part of the OTF is symmetrical with respect to each of the meridional direction and the sagittal direction. The imaginary part of the OTF is symmetric with respect to the meridional direction, although its sign (plus or minus) is opposite, and symmetrical with respect to the sideward direction. According to this symmetry, as the data of the OTF used for the fitting, information of a symmetric area such as at least a 1/4 (1/4) area in the entire domain is sufficient. In this embodiment, for this reason, the 1/4 region of the entire region from the OTF to the real part and the imaginary part is cut so as to include the DC component to perform the high-precision fitting processing of the OTF.

If a certain precision is required, if the fitting process is stopped with a low order, the original OTF can not be reconstructed, and accordingly, there is a possibility that appropriate correction can not be performed. Therefore, the proper order differs depending on the shape of the original OTF. In other words, when the provider creates the OTF data, it is necessary to perform the approximation using a sufficient degree (that is, a sufficiently high number of orders). It is desirable that the created OTF data is managed on the network 102 so that it can be always used by the user.

With respect to the OTF data created by this method, the user can access the information processing apparatus 101 through the network 102 from the information processing apparatus 103 possessed by the user, and obtain only necessary optical system information. In this embodiment, the user acquires the OTF data stored in the information processing apparatus 101 via the network 102 or records the OTF data in a recording medium (storage medium) such as a CD-R and a DVD You may.

The user can acquire the OTF information (OTF data) of the optical system (imaging optical system) to be corrected using the information processing apparatus 103 possessed by the user. The image processing apparatus of the present embodiment can be applied to, for example, an information processing apparatus 103 owned by a user (an image processing application installed in the information processing apparatus 103 (not shown)) or an image capturing apparatus 104 , 105, 106). There is a high possibility that the accuracy of the OTF data required for the correction of each image processing apparatus is different from each other. Accordingly, in order to appropriately correct the photographed image with a small amount of OTF data (OTF information amount), it is necessary to transmit appropriate (more preferably optimal) OTF data (OTF information) for each image processing apparatus. In this embodiment, it is possible to transmit appropriate OTF data to each of the image processing application installed in the information processing apparatus 103 and the image pickup apparatuses 104, 105, and 106 provided as an image processing apparatus. For example, the above-described image processing application can be applied to any OTF data by changing the program. On the other hand, the image processing units (image processing apparatuses in this embodiment) provided in each of the image capturing apparatuses 104 to 106 are generally constituted by hardware because the processing speed is prioritized, and the amount of input data is limited in many cases . For this reason, in particular, in a cost-effective and inexpensive imaging apparatus, the approximation coefficient of the OTF data may be limited.

The present embodiment provides a method of setting OTF data of a desired optical system in each of the imaging apparatuses 104 to 106. [ For example, the information processing apparatus 103 acquires the OTF data stored in the storage medium installed in the information processing apparatus 101 via the network 102. [ Then, OTF data acquired by the information processing apparatus 101 is transferred to the imaging apparatuses 104 to 106 via USB or communication (wired communication or wireless communication). In this embodiment, the information processing apparatus 103 transmits appropriate OTF data in accordance with the device (for example, any one of the imaging apparatuses 104 to 106) to which the information processing apparatus 103 is connected.

Next, an outline of the information processing apparatus 103 (control apparatus) will be described with reference to Fig. 1B. Fig. 1B is a block diagram of the information processing apparatus 103. Fig. The information processing apparatus 103 (control apparatus) includes an input circuit (input section) 103a, a memory circuit (memory or storage section) 103b, a decision circuit (decision section or decision section) 103c, ) 103d. The input circuit 103a and the output circuit 103d are, for example, a Wi-Fi (wireless fidelity) communication module. The connection module is not limited to the communication circuit for wireless communication, but may alternatively be a communication circuit for wired communication. The memory circuit 103b is a memory such as a ROM, and the decision circuit 103c is a processor such as a CPU.

The input circuit 103a inputs information of the image processing apparatus (for example, the image processing application installed in the information processing apparatus 103 and the image pickup apparatuses 104 to 106, respectively). The storage circuit 103b stores first data (e.g., OTF data obtained through the network 102) including data relating to the first optical transfer function (OTF), that is, a plurality of coefficient data . The determination circuit 103c determines second data having a smaller data amount than the first data from the first data stored in the storage circuit 103b. The second data is data relating to the second optical transfer function, that is, OTF data suitable for each image processing apparatus. The output circuit 103d outputs the second data to the image processing apparatus. The optical transfer function OTF of the optical system can be represented by using a plurality of coefficient data as a coefficient of an approximate function having a predetermined order (m-th order). Preferably, the plurality of coefficient data is determined by approximating an optical transfer function of the optical system to an approximate function having a predetermined degree (predetermined degree). Preferably, the second data is coefficient data corresponding to an order (n-th order (n <m)) smaller than the predetermined order of the approximate function.

Next, with reference to Fig. 2, a processing flow for transmitting the optimum OTF data according to the device to which the information processing apparatus 103 is connected will be described. 2 is a flow chart showing data transfer processing of OTF data. Each step of FIG. 2 is executed by a control unit (CPU) of the information processing apparatus 103 based on an instruction of a program of an application installed in the information processing apparatus 103.

First, in step S201, the information processing apparatus 103 (control section) transmits OTF data (appropriate OTF data) of the imaging optical system used for acquiring the photographed image, that is, the correction target optical system, from the information processing apparatus 101 (Acquired) through the network 102. For example, an application for data registration owned by the user is installed in the information processing apparatus 103, and the user can select a required optical system (imaging optical system) by using this application. The OTF data acquired in step S201 is stored in a memory (storage circuit) provided in the user's information processing apparatus 103. [

Next, in step S202, the information processing apparatus 103 (an application installed in the information processing apparatus 103) is connected to a device (for example, the imaging apparatuses 104 to 106, etc.) connected to the information processing apparatus 103 Of the image processing apparatus). Specifically, the information processing apparatus 103 determines the degree that can be used for the image processing block that performs correction using the OTF data of the image pickup apparatuses 104 to 106. [

Subsequently, in step S203, the information processing apparatus 103 reduces the order (some data) of the OTF data based on the connected device information (maximum degree) acquired in step S202 from the OTF data acquired in step S201 , And creates OTF data for transmission. In other words, based on the first data (OTF data acquired in step S201) stored in the storage circuit 103b, the determination circuit 103c generates second data (data for transfer) smaller than the first data Of the OTF data). In this case, the information processing apparatus 103 can employ a method of directly receiving the maximum available degree from the connected devices (the imaging apparatuses 104 to 106) as a method of determining the required degree (degree). The information processing apparatus 103 also stores a table indicating the relationship between the connected device and the maximum degree in the storage circuit 103b provided in the information processing apparatus 103, .

3 is an example of a table showing the relationship between connected devices (cameras A to C as image pickup devices) and the maximum degree. As shown in FIG. 3, the maximum degree of the advanced camera A is 20 (that is, the camera A is applicable up to 20), and the maximum degree of the cheap camera C is five (that is, It is possible).

After the maximum degree (applicable maximum degree) is determined for each device (imaging device) connected in steps S202 and S203, the flow proceeds to step S204. In step S204, the information processing apparatus 103 extracts up to the maximum degree of the connected devices among the orders of the OTF data acquired in step S201, and transfers the obtained OTF data to each device. In this case, when the image restoration processing is performed by the image processing application installed in the information processing apparatus 103, the image restoration processing can be performed by adopting the maximum degree of the application. When an effect equivalent to that of the image pickup apparatus is required, the maximum degree obtained at the time of photographing may be recorded in the image file, and the correction processing may be performed based on the value (that is, the recorded maximum degree).

According to the flow shown in Fig. 2, individual data according to the characteristics of each imaging apparatus can be determined based on one original data, and the individual data can be transferred to the corresponding imaging apparatus. Therefore, it is possible to perform appropriate correction for each image pickup apparatus (image processing apparatus) without storing the original data of the OTF data in each image pickup apparatus.

As described above, in the first embodiment, a configuration for generating OTF data for transmission by reducing the order of OTF data based on information of an image processing apparatus that is a device connected to the information processing apparatus 103 is described as an example The present invention is not limited to this.

1C is a block diagram showing a configuration of an image processing apparatus for image restoration processing. An image sensing apparatus 104 that is an example of an image processing apparatus includes an input circuit (input section) 104a, a determination circuit (a determiner or a determination section) 104b, a storage circuit (memory or storage section) 104c, Processor or processing unit) 104d. The input circuit 104a is, for example, a Wi-Fi communication module. The communication module is not limited to the communication circuit for wireless communication, or may be a communication circuit for wired communication. The determination circuit 104b and the processing circuit 104d are constituted by a processor such as a CPU and the memory circuit 104c is a memory such as a ROM.

For example, the information processing apparatus 103 directly transfers the acquired OTF data to the image capturing apparatus 104, and the image capturing apparatus 104 receives the OTF data through the input circuit 104a. The decision circuit 104b reduces the order of the received OTF data and is stored in the storage circuit 104c. The processing circuit 104d generates a recovery filter using the stored OTF data and performs an image recovery process.

Alternatively, the information processing apparatus 103 can divide the acquired OTF data according to the degree and transmit the divided OTF data to the image capturing apparatus 104. [ The determination circuit 104b may store only applicable OTF data of the OTF data received by the input circuit 104a in the storage circuit 104c.

(Second Embodiment)

Next, an image processing system according to a second embodiment of the present invention will be described with reference to FIG. Fig. 4 is a configuration diagram of the image processing system according to the present embodiment, and shows an image processing system applicable to the image pickup apparatus of the interchangeable lens system.

Reference numeral 401 denotes an interchangeable lens (lens device) having an optical system (imaging optical system), which can be mounted on various imaging apparatuses (for example, imaging apparatuses 402, 403, and 404). The interchangeable lens 401 includes an input circuit (input section) 401a, a memory circuit (memory or storage section) 401b, a decision circuit (decision section or decision section) 401c and an output circuit (output section) Respectively. These elements have functions equivalent to those of the respective elements of the information processing apparatus 103 of the first embodiment described with reference to Fig. 1B. The ROM (storage circuit 40 lb) of the interchangeable lens 401 stores OTF data (OTF information) for correcting deterioration (optical deterioration) caused by the optical system. For example, the ROM of the interchangeable lens 401 stores the OTF data including the degree (degree) sufficient to reproduce the characteristics in the format described in the first embodiment.

The interchangeable lens 401 can be mounted on each of the image sensing apparatuses 402, 403, and 404, and particularly, the image sensing apparatus 402 is an advanced apparatus, the image sensing apparatus 403 is an intermediate stage, and the image sensing apparatus 404 is a distributor. Generally, the high-end device (imaging device 402) has a high-spec hardware configuration as compared with the intermediate device (imaging device 403) and the air supply device (imaging device 404) high.

In the present embodiment, similar to the information processing apparatus 103 of the first embodiment, the interchangeable lens 401 is configured so that the OTF data stored in the ROM (storage circuit) And transmits the selected order. In this case, there is a possibility that sufficient correction can not be performed in a specific region of the interchangeable lens 401 by using OTF data of the maximum degree allowed by the inexpensive imaging apparatus 404 (air supply unit). This phenomenon will be described with reference to Figs. 6A to 6D. 6A to 6D are examples of OTF data in this embodiment.

As described in the first embodiment, there is a real part and an imaginary part in the OTF data, and the inverse Fourier transform is performed on the combination of the real part and the imaginary part, thereby making it possible to reconstruct the PSF which is the diffusion function of the origin. 6A and 6B show the real part (the real part of OTF in the raw data) and the imaginary part (the imaginary part of OTF in the original data) of the OTF data in a specific photographing condition of the interchangeable lens 401, respectively. FIGS. 6C and 6D show the real part (the real part of the reconstructed OTF) and the imaginary part (the imaginary part of the reconstructed OTF) of the reconstructed OTF data based on the OTF data with reduced orders, respectively, as actual data.

6A and 6C, and FIGS. 6B and 6D, the shape of each function (OTF data) is changed. In other words, the reconstructed PSF based on Figs. 6C and 6D is changed from the original PSF. When the OTF after reconstruction is different from the OTF before reconstruction, since the correction is performed with characteristics different from those of the optical system (imaging optical system) used for imaging, there is a possibility that an unexpected effect appears in the corrected image. Concretely, the edges are vibrated to reveal a plurality of edges such as ringing or a black level depression in which edges of the edges are largely immersed. Accordingly, in the case of an inexpensive imaging apparatus 404 which can not maintain a sufficient degree to perform appropriate correction, it is necessary to take measures to weaken the correction amount in order to reduce the occurrence of the above-mentioned inconveniences.

As an example of a countermeasure for weakening the correction amount, there is a method of reducing the maximum gain. First, the maximum gain will be described. In creating the image recovery filter, it is necessary to create a filter in consideration of the noise term such as Γ, instead of using the inverse of the simple OTF in the creation of the filter, as shown in Equation (4) or the filter. By controlling the value (function) of?, It is possible to provide the maximum gain in the frequency domain. The description will be made with reference to Figs. 7A to 7C.

7A to 7C are graphs showing the MTF and the maximum gain. 7A shows a graph of the MTF of the optical system to be corrected. The relationship between the OTF and the MTF can be expressed by the following equation (7).

As described above, the MTF (Modulation Transfer Function) is the absolute value of the OTF, and the PTF (Phase Transfer Function) is the phase difference which is a function of the spatial frequency.

The image restoration process is a process of restoring the MTF by taking the inverse number of the MTF as a filter (i.e., multiplying) by the intensity of the recovery. In this case, in the high frequency region, the signal is greatly reduced as shown in Fig. 7A. Therefore, if the image is multiplied by the reciprocal of the MTF, a large gain is required, and as a result, it is not preferable as an output image. Therefore, as described above, a method of creating a filter by inserting a term such as the formula (4) or a term for reducing the gain of high frequency like a filter is generally adopted. It is possible to adjust the method of reducing the high frequency gain by taking measures against the function of? In other words, by adjusting the term of?, The gain can be controlled as a parameter that determines the degree of recovery (i.e., how much the image is to be restored).

For example, with respect to the imaging apparatus 402 that has a relatively limited hardware restriction and can perform a relatively aggressive recovery process, the term of? Is set so that the value of the maximum gain becomes Max_b, as shown in Fig. 7B. On the other hand, with respect to the imaging device 404, which has a large hardware restriction and can not actively correct, the term of? Is set so that the maximum gain value becomes a value of Max_c smaller than Max_b, as shown in FIG. As a result, the image restoration effect can be reduced, and it becomes possible to reduce adverse effects such as ringing and black level depression. Accordingly, the imaging device 404 having a large hardware restriction can perform appropriate image recovery processing even when the number of usable orders of the imaging device 404 is small.

As described above, when the constraint is applied to the image pickup apparatus 404 by using a constant maximum gain for any condition, the gain is reduced in the region where the OTF data can be sufficiently reproduced with the approximation of low dimension, There is a possibility of reducing the correction effect. This specific example will be described with reference to Figs. 8A to 8D.

8A to 8D are examples of OTF data according to the aperture stop in this embodiment. FIGS. 8A and 8B show OTF data (reconstruction) of the real part (the real part of the OTF in the open state) and the imaginary part (the imaginary part of the OTF in the open state) of the OTF under the condition that the aperture stop of a certain interchangeable lens is in the open state Quot; data "). FIGS. 8C and 8D are graphs showing OTF (imaginary part of the OTF in the small iris state) and imaginary part (imaginary part of the OTF in the small iris state) of the OTF in the condition that the aperture stop of the interchangeable lens is in the small iris state. It is an example of data (data before reconstruction).

In a common interchangeable lens, in many cases, various aberrations occur near the open state as shown in Figs. 8A and 8B, and accordingly, the OTF has a complicated shape. On the other hand, if the aperture of the aperture stop is narrowed, the influence of diffraction becomes large, and the influence of other aberrations is buried. Accordingly, as shown in Figs. 8C and 8D, the OTF has a simple shape (i.e., aberration is a simple shape). In other words, for the OTF in the open state, it is necessary to reproduce the original OTF with a higher order approximation function to perform the approximation, and in the small aperture state, a lower order approximation function can be used to sufficiently reproduce the original OTF .

In accordance with this phenomenon, a method for obtaining the maximum correction effect even in an imaging apparatus with a large hardware restriction will be described with reference to Fig. 5 is a sequence diagram of the image processing system according to the present embodiment, which shows a data sequence when the interchangeable lens 401 is attached to any one of the image pickup devices 402 to 404.

When the interchangeable lens 401 is mounted in any one of the image pickup apparatuses (image processing apparatuses), in step S501, the interchangeable lens 401 requests information on the type of the image pickup apparatus to the connected image pickup apparatus do. For example, the interchangeable lens 401 requests information as to whether or not the connected imaging apparatus can perform correction processing (that is, image restoration processing is possible). Then, the interchangeable lens 401 determines whether or not the connected imaging apparatus is capable of image restoration processing, and transmits the determination result to the connected imaging apparatus. When the connected imaging apparatus is capable of image restoration processing, the interchangeable lens 401 requests information on the maximum order (maximum order information) for the connected imaging apparatus.

Subsequently, in step S502, in response to a request from the interchangeable lens 401 in step S501, the image capturing apparatus obtains information about the maximum degree allowed by the image capturing apparatus Maximum degree information) to the interchangeable lens 401. In this case, the interchangeable lens 401 inputs the maximum order information (information of the image pickup apparatus) transmitted from the image pickup apparatus via the input circuit 401a. In step S503, the interchangeable lens 401 extracts the maximum gain table from the reference table stored in the ROM provided in the interchangeable lens 401, in accordance with the maximum order information obtained from the image pickup apparatus in step S502, . In other words, the determination circuit 401c determines the second data in accordance with information on the maximum degree allowed by the image processing apparatus. Preferably, the determination circuit 401c determines information on the maximum gain (correction strength) used to generate the image recovery filter, according to the information on the maximum degree. Preferably, the determination circuit 401c determines the second data based on the photographing condition information determined to photograph the image.

Referring to Figs. 9A and 9B, the reference table stored in the ROM provided in the interchangeable lens 401 and the maximum gain table extracted from the reference table will be described. 9A and 9B are diagrams showing a reference table and a maximum gain table, and Figs. 9A and 9B show an example of a reference table and a maximum gain table, respectively.

The reference table shown in FIG. 9A is a table for determining a gain table to be transmitted according to the order of the connected image pickup apparatus, that is, a table that makes the degree of the image pickup apparatus and the gain table associated with each other. 9A shows that, for example, the image pickup apparatus whose maximum order is 10 orders transmits the gain table of "Table B ".

The maximum gain table shown in Fig. 9B is a table showing the value of the maximum gain according to the aperture diaphragm. As described above, in the case of using an image pickup apparatus whose maximum order is 10th, "Table B" in FIG. 9A is referred to. For this reason, for the image photographed at the aperture value F5.6, the filter is generated under the condition that the maximum gain is 4. Particularly, when the imaging apparatus has a large hardware restriction, "Table C" in Fig. 9A is referred to. In this case, the maximum gain is 2 in the open state of the aperture stop, and the maximum gain is 5 in the small aperture state. In other words, as described with reference to Figs. 8A to 8D, since the positive correction can not be performed in a region where a higher order number is required for reproduction of the OTF, the maximum gain is set to be doubled to reduce the occurrence of the above-mentioned malfunction. On the other hand, in the small iris state, since the OTF can be reproduced by the low order, active correction is possible. Although the maximum gain table for the aperture stop is described in this embodiment, a table may be created in consideration of other parameters (other shooting condition information such as the shooting distance and the focal distance) depending on the characteristics of the optical system.

Next, returning to Fig. 5, the operation at the time of photographing will be described. In step S504, at the timing (S2) when the user presses the shutter button at the time of photographing, the image capturing apparatus transmits the photographing condition information such as the focal length, the photographing distance, and the aperture stop to the interchangeable lens 401. When photographing condition information is determined for photographing an image, OTF is specified. Therefore, the interchangeable lens 401 transmits appropriate OTF data from the OTF data (OTF coefficient data) stored in the ROM provided in the interchangeable lens 401 to the image pickup apparatus. In this case, the OTF data is transmitted up to the data of the maximum degree of the imaging apparatus acquired in step S502. The imaging device that receives the OTF data determines the maximum gain based on the reference table and the maximum gain table obtained when the interchangeable lens 401 is mounted (step S503), and generates an image recovery filter to perform image recovery. As described above, by preparing one kind of OTF data in the ROM provided in the interchangeable lens 401, appropriate image restoration processing can be performed in accordance with various mountable image pickup apparatuses.

The interchangeable lens 401 may transmit the OTF data stored in the ROM to the image pickup device as it is, and the image pickup device may adjust the degree included in the OTF data to determine the maximum gain based on the reference table and the maximum gain table.

(Other Embodiments)

The embodiment (s) of the present invention may also be used to read computer-executable instructions (e.g., one or more programs) recorded on a storage medium (more fully, also referred to as a 'non-transitory computer readable storage medium' (E.g., an application specific integrated circuit (ASIC)) for performing one or more functions of the embodiment (s) described above and / or performing one or more functions of the above- (E. G., By reading and executing the computer-executable instructions from the storage medium to perform one or more functions of the embodiment (s) and / or < / RTI > / RTI > by the computer having the system or the device by controlling the one or more circuits that perform one or more functions of the above-described embodiment (s) It can be realized by the method to be. The computer may comprise one or more processors (e.g., a central processing unit (CPU), a microprocessor unit (MPU)) or may be a separate computer or a separate processor for reading and executing computer- A network may be provided. The computer-executable instructions may be provided to the computer from, for example, a network or the storage medium. The storage medium may be, for example, a hard disk, a random access memory (RAM), a read only memory (ROM), a storage of a distributed computing system, an optical disk (compact disk (CD), digital versatile disk Ray disk (BD) ^TM, etc.), a flash memory device, a memory card, and the like.

In each of the embodiments described above, in the image processing system for correcting the deteriorated image (captured image) by reconstructing the optical transfer function OTF, that is, for performing the image recovery process, an appropriate OTF Data (approximate data) can be provided. According to the embodiments, it is possible to provide a control device, an image processing system, a lens device, an image processing system, a control method, and an image processing method capable of providing appropriate OTF data in accordance with an image processing apparatus that performs image restoration processing. Further, according to each embodiment, it is possible to provide a control device, an image processing device, a lens device, an image processing system, and an image processing method capable of storing appropriate OTF data for performing image restoration processing.

While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all modifications, equivalent structures and functions.

Claims

A storage unit for storing first data including a plurality of coefficient data; And
Second data having a smaller amount of data than the first data is read out from the first data stored in the storage unit on the basis of information of an image processing apparatus that performs image processing on an image photographed using the optical system And a determination unit for determining,
And the optical transfer function of the optical system can be expressed by using the plurality of coefficient data as a coefficient of an approximate function having a predetermined order.

The method according to claim 1,
Wherein the plurality of coefficient data is determined by approximating an optical transfer function of the optical system to an approximate function having the predetermined order.

The method according to claim 1,
Wherein the approximate function is an orthogonal function.

The method according to claim 1,
And the second data is coefficient data corresponding to an order smaller than the predetermined order of the approximate function.

5. The method of claim 4,
Wherein the determination unit determines the second data in accordance with information on a maximum degree allowed by the image processing apparatus.

6. The method of claim 5,
Wherein the determining unit determines information on the maximum gain used to generate the image recovery filter according to the information on the maximum degree.

The method according to claim 1,
Wherein the determining unit determines the second data based on photographing condition information determined to photograph the image.

8. The method of claim 7,
Wherein the photographing condition information includes information on a focal length, a photographing distance and an aperture stop.

The method according to claim 1,
Wherein the storage unit is capable of acquiring the first data through a network.

The method according to claim 1,
An input unit for inputting information of the image processing apparatus; And
And an output unit outputting the second data to the image processing apparatus.

An optical system for forming a subject image;
A storage unit for storing first data including a plurality of coefficient data; And
Second data having a smaller amount of data than the first data is read out from the first data stored in the storage unit on the basis of information of an image processing apparatus that performs image processing on an image photographed using the optical system And a determination unit for determining,
Wherein the optical transfer function of the optical system can be expressed by using the plurality of coefficient data as a coefficient of an approximate function having a predetermined order.

An image processing device;
A storage unit for storing first data including a plurality of coefficient data; And
Second data having a smaller amount of data than the first data is read out from the first data stored in the storage unit on the basis of information of an image processing apparatus that performs image processing on an image photographed using the optical system And a determination unit for determining,
The image processing apparatus performs image restoration processing using the second data,
Wherein the optical transfer function of the optical system can be represented by using the plurality of coefficient data as a coefficient of an approximate function having a predetermined degree.

An input unit for receiving first data including a plurality of coefficient data;
A determining unit that determines, from the first data, second data having a smaller data amount than the first data; And
And a processing unit that performs image processing on an image photographed using the optical system by using the second data,
Wherein the optical transfer function of the optical system can be represented by using the plurality of coefficient data as a coefficient of an approximate function having a predetermined order.

14. The method of claim 13,
Wherein the plurality of coefficient data is determined by approximating an optical transfer function of the optical system to an approximate function having the predetermined order.

14. The method of claim 13,
Wherein the approximate function is an orthogonal function.

14. The method of claim 13,
Wherein the second data is coefficient data corresponding to an order smaller than the predetermined order of the approximate function.

17. The method of claim 16,
Wherein the determination unit determines the second data according to information on a maximum degree allowed by the processing unit.

18. The method of claim 17,
Wherein the determination section determines information on a maximum gain used to generate an image recovery filter in accordance with the information on the maximum degree.

14. The method of claim 13,
Wherein,
Generating an image recovery filter using the second data,
And performs image processing using the image recovery filter.

An input unit for receiving first data including a plurality of coefficient data;
A determining unit that determines, from the first data, second data having a smaller data amount than the first data;
A processing unit that performs image processing on an image photographed using the optical system by using the second data; And
And a control device for outputting the first data,
Wherein the optical transfer function of the optical system can be represented by using the plurality of coefficient data as a coefficient of an approximate function having a predetermined degree.

Inputting information of an image processing apparatus that performs image processing on an image photographed using an optical system; And
And second data having a smaller data amount than the first data, from the first data including a plurality of coefficient data stored in the storage unit based on the information of the image processing apparatus,
Wherein the optical transfer function of the optical system can be represented by using the plurality of coefficient data as a coefficient of an approximate function having a predetermined order.

Receiving first data including a plurality of coefficient data;
Determining second data having a smaller data amount than the first data from the first data; And
And performing image processing on an image photographed using the optical system by using the second data,
Wherein the optical transfer function of the optical system can be expressed by using the plurality of coefficient data as a coefficient of an approximate function having a predetermined degree.