JPH06343159A - Image recording and reproducing system provided with image stabilizing function - Google Patents

Abstract

PURPOSE:To more accurately perform the restoration of source images by an image processing in a system composed of a photographing device for recording a subject in a prescribed recording medium and a reproducing device for reproducing recorded images provided with a function for image stabilization by the image processing on the side of the reproducing device. CONSTITUTION:A camera detects the shake and records the shake information on a film. On the other hand, in the reproducing device, the images recorded on the film 81 are converted to electric signals by an area sensor 83 and inputted to an arithmetic part 86. The arithmetic part 86 corrects the shake of subject images based on the shake information read by an information read part 89. At this time, prescribed image information which can not be obtained from the photographed subject images is outputted from an information storage part 91 to the arithmetic part 86. Since the prescribed image information is inputted from the information storage part 91 other than the information of the photographed subject images and the information of the shake to the arithmetic part 86, at the time of restoring the source images by mathematical operations, it can be prevented that a part of the solution is undefined.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION The present invention relates to a photographing device such as a camera,
The present invention also relates to an image recording / playback system including a playback device that plays back an image captured by the image capturing device. More specifically, the present invention relates to a system in which even when a photograph is taken in a camera shake state, information about blurring is detected on the side of the photographing device and the blurring of the reproduced image is corrected on the side of the reproducing device based on the blurring information.

[0002]

2. Description of the Related Art Conventionally, many devices for correcting blur have been proposed for silver halide cameras and video cameras. They detect camera vibrations (camera shake) using an angular velocity sensor or the like, and drive the photographing optical system based on the detected blur information to prevent the subject image from blurring. Also, in the video camera, a method of shifting the image reading range of the image sensor according to the direction and speed of the blur has been proposed instead of driving the photographing optical system. According to this method, when a subject moves during image capturing with a video camera, the subject image is formed by shifting out of a predetermined image reading range within the image capturing area, and thus the shifted image is within the image capturing area of the image sensor. In some cases, the captured image is read by shifting the image read range by the above-mentioned shifted amount.

However, all of these blur correction devices are provided in the camera. When the camera shake is provided with the shake correction device, an optical system for correction and a drive device therefor must be provided, which causes a problem of increasing the size and weight of the camera. Further, even in the above method of shifting the image reading range of the image sensor, it is necessary to detect the shift amount and control the change of the reading range according to the shift amount in order to correct the blurring. There was a problem of price reduction.

As a method for solving the above-mentioned problems, the applicant of the present application, in the application filed on the same date as the present application, detects the blur on the camera side and records the blur information, and on the reproducing device side, the blur information recorded on the camera side. Based on this, we propose a system that corrects the blurring of reproduced images by image processing. According to this method, it is not necessary to provide a camera shake correction device on the camera side, and it is possible to prevent the camera from becoming large and expensive.

As an example of the blur correction by the above image processing, a function (hereinafter,
There is a method of correcting an image deteriorated due to blur by using a deterioration function). However, in many cases, the input of the image processing is only the image data of the blurred image and the deterioration function, and the solution of the original image (image before the blur) desired to be mathematically determined is indefinite. Then, a new problem arises that a part of the indefinite solution must be assumed or approximated to a certain value.

[0006]

SUMMARY OF THE INVENTION Therefore, the present invention solves the problems of the above-mentioned conventional example, that is, the camera side is not provided with the shake correction device but only the shake detection device is provided, and the playback device side is provided with the image processing. It is an object of the present invention to make the restoration of an original image by the above image processing more accurate in an image recording / reproducing system provided with a device for correcting the above.

[0007]

In order to achieve such an object, in the image recording / reproducing system of the present invention, a blur detecting means for detecting information on blur and a predetermined predetermined image information are provided. It is characterized in that the blur of the reproduced image is corrected based on the output means for outputting and the blur information and the predetermined image information outputted from these means. (Claim 1) Further, in the invention of claim 2, the predetermined image information output by the output means is a certain specific value.

According to a third aspect of the invention, the predetermined image information output by the output means is obtained from the image information of the subject image.

In the invention of claim 4, the camera side is provided with the first recording means for recording the subject image on the recording medium and the second recording means for recording the information about the subject image separately from the first recording means. On the other hand, the reproducing device side is provided with a correcting unit that corrects the blur of the reproduced image based on the blur information detected by the camera side and the information about the subject image recorded by the second recording unit.

[0010]

According to the first aspect of the present invention, in the image processing for blur correction, the predetermined image information output by the output means is input instead of the input data that cannot be obtained from the captured image. , It is possible to prevent the solution from becoming indefinite in the mathematical operation for restoring the original image.

According to the second aspect of the invention, since the predetermined image information output by the output means has a specific constant value, a structure for obtaining the image information is provided on the photographing device side or the image is reproduced on the reproducing device side. It is possible to realize a system having a simple configuration without the need to perform a calculation for obtaining information.

According to the third aspect, since the predetermined image information output by the output means is obtained from the image information of the subject image, it is more accurate than performing the image processing by inputting a specific constant value as described above. The original image can be restored.

According to the fourth aspect, the second recording means is provided on the side of the photographing device in addition to the first recording means for recording the subject image. The second recording means is provided for recording data used for image processing for blur correction. The second recording unit can obtain information on the subject before blurring (information on the original image) when the subject is photographed by the first recording unit. By inputting the information of this original image to the image processing of the blur correction, as compared with the case where a specific constant value is input as described above or the value obtained from the image information of the blurred image is input, The original image can be restored more accurately.

[0014]

Embodiments of the present invention will be described below with reference to the drawings. In the present embodiment, a silver-salt camera is used as the photographing device, and a film-video player that reproduces an image recorded on the film by the silver-salt camera on a television monitor is used as the reproducing device. To do.

FIG. 1 is an external view of an image recording / reproducing system according to the present invention. Reference numeral 1 denotes a camera having a blur detection function, and the detected blur information is recorded on a film together with a captured image. The details of this blur information will be described later. The film which has been photographed is developed in a developing room and then returned to the photographer in the form of being housed in the cartridge 2. When a cartridge 2 is loaded, a reproducing device 3 reads an image recorded on a film by photoelectrically converting it with an area sensor such as a CCD (Charge Coupled Device) and reads the image converted into an electric signal on a television. Play back on monitor 4. The reproducing device 3 has a function of correcting blur, and performs blur correction by image processing based on the blur information recorded on the film by the camera 1. Therefore, even if the image is taken in a camera shake state, a clear image with the shake corrected is reproduced on the television monitor 4.

FIG. 2 is an external view of the camera 1 having the blur detection function according to the present invention. The camera 1 includes an imaging unit for reading an image of a subject for detecting blur information. The image pickup unit includes a blur detection sensor 14 that photoelectrically converts the optical image of the subject into an electrical signal and reads the optical signal, and an optical system 13 that forms a subject image on the imaging surface of the blur detection sensor 14. The optical system 13 is provided above the photographing lens 11 that forms a subject image for photographing on the film surface 12,
The blur detection sensor 14 is provided at an appropriate rear position on the optical axis of the optical system 13.

The shutter button 15 is an operation button for instructing photometry / distance measurement and release, and switches S1 and S2 to be described later are turned on in conjunction with the operation of the shutter button 15.
・ Turns off.

When the switch S1 is turned on, photometry and distance measurement are performed, and AF (Auto Focus) control and exposure control value setting are performed. When the switch S2 is turned on, shooting (exposure control) is performed based on the set exposure control value, and during the exposure control period, the blur detection sensor 14 calculates blur information for the captured image. The details of the calculation of the blur information will be described later.

FIG. 3 is a block diagram showing the main part of the camera 1. A main CPU 21 controls the entire camera. The photometric sensor 22 and the photometric unit 23, which are SPDs (Silicon Photo Diodes), measure the subject brightness. The brightness signal detected by the photometric sensor 22 is input to the photometric unit 23, and the photometric unit 23 uses the brightness signal. The photometric value is calculated based on. The result of this calculation is the main CPU 2
Input to 1. The taking lens 11 is a focus adjustment drive unit 24.
Focus adjustment is performed through. The photographic lens circuit 25 stores the F-number of the lens unique to the photographic lens 11, the focal length f, and the like. The shutter blades 26 are driven by the shutter drive unit 27. The film winding unit 28 winds and rewinds the film loaded in the camera. The AF module 29 performs a focus detection operation to detect the distance to the subject.

The switch S1 is the first shutter button 15
It is turned on with a stroke to start photometry and AF operation.
The switch S2 is turned on by the second stroke of the shutter button 15 to start the release operation. The exposure correction amount input unit 30 is a main CPU based on the photometric result of the photometric unit 23.
When the exposure control value set by 21 is corrected, the correction amount is input.

The blur detection unit 32 detects the amount of movement (image blur amount) of the subject image formed on the film by the taking lens 11. As will be described later, the blur detection unit 32 detects the image forming position of the subject image a plurality of times at every predetermined time during the exposure time, and determines each image forming position and the image forming position of the subject image immediately after the start of the photographing (initial image forming position). ) Is detected to detect the amount of movement of the subject image every predetermined time. Each moving amount is calculated as a distance of the image forming position after a predetermined time elapses from the initial image forming position, and the calculation result is recorded on the film by the information writing unit 33 as blurring information of the captured image.
The information writing unit 33 records the shooting information such as the date and the exposure value on the film in addition to the shake information detected by the shake detecting unit 32. In this case, it may be optically imprinted on the film, or may be magnetically recorded on the film by including a magnetic head or the like. Since the technique of recording information optically or magnetically on a film is already known, a description thereof will be omitted.

Next, the contents of the blur detection unit 32 will be described with reference to FIG. The blur detection sensor 14 includes an image pickup unit 41 (hereinafter, referred to as CCD 41) including a CCD, an output amplifier 42 and a C for amplifying an image signal output from the CCD 41.
The clock generator 45 includes an illuminance monitor circuit 43 that monitors the illuminance of the subject image formed on the CD 41, and controls the imaging operation of the CCD 41, that is, the charge accumulation time (integration time). ing. The CCD 41 includes a spectral filter 64 that splits the incident light into three primary colors of RGB, and converts each color component of RGB into an electric signal and outputs the electric signal. The blur detection sensor 14 also includes a sensor data memory 65. The memory 65 stores data such as the number of pixels of the CCD 41, a pixel pitch, an imaging range, and the like, and data such as the focal length of the blur detection optical system 13. These data are recorded on the film from the control CPU 63 through the information writing unit 33, and are used for the blur correction calculation by the reproducing device described later.

The blur detection optical system 13 is a single focus lens having a focal length fd, and is an optical system for forming a subject image on the CCD 41. The blur detection optical system 13 is
The CCD 41 covers the same area as the subject image recorded on the film.
The image may be formed on the upper side, or only one portion (for example, only the central portion) of the subject image recorded on the film may be enlarged and formed. By doing so, the number of pixels for imaging the same range is relatively increased, so that the resolution of the CCD 41 with respect to the subject image is increased,
The shake detection can be performed more finely.

The clock generator 45 is the CCD 4
1, D / A converter 48, sensitivity variation correction memory 4
9. A clock for driving the dark output correction memory 50 and the A / D converter 51 is generated. Further, the clock generator 45 uses the CCD 4 based on the signal relating to the illuminance of the subject image input from the illuminance monitor circuit 43.
A charge accumulation time control signal of 1 is generated.

The differential amplifier 46 corrects a dark output level of a signal (hereinafter, referred to as a pixel signal) received by each pixel forming the CCD 41, which is output from the blur detection sensor 14. Gain control amplifier 47
Is for correcting the error of the pixel signal due to the sensitivity variation of the pixel. Data for correcting an error in pixel signals due to sensitivity variations (hereinafter referred to as sensitivity variation correction data) is stored in the sensitivity variation correction memory 49, and data for correcting the dark output level (hereinafter referred to as dark output correction data). Are stored in the dark output correction memory 50. The D / A converter 48 converts the dark output correction data read from the dark output correction memory 50 from a digital signal to an analog signal and operates the operational amplifier 4 described above.
6 is input.

When the image pickup of the subject image by the CCD 41 is completed, each pixel signal forming the subject image is sequentially read out to the differential amplifier 46, and the corresponding dark output correction data is synchronized with this and the dark output correction data is corrected. The data is read from the memory 50, D / A converted by the D / A converter 48, and input to the differential amplifier 46. And the differential amplifier 4
In step 6, the dark output level is corrected by subtracting the dark output correction level from the pixel signal level.

After the dark output level is corrected, each pixel signal is input to the gain control amplifier 47. The sensitivity variation correction data is read from the sensitivity variation correction memory 49 in synchronization with the input of each pixel signal, and the gain of the gain control amplifier 47 is set by this correction data. Then, each pixel signal is level-adjusted by the gain control amplifier 47 with a predetermined gain, so that an error due to sensitivity variation is corrected.

The pixel signal output from the gain control amplifier 47 is converted from an analog signal to a digital signal by the A / D converter 51, and the reference section memory 5 is provided.
2 or the reference unit memory 53. Reference part memory 5
Reference numeral 2 denotes a memory that stores pixel data that forms an image (an image captured at the start of image capturing, hereinafter referred to as a reference image) that serves as a reference for detecting the amount of movement of a subject image during image capturing. The reference unit memory 53 is a memory in which an image to be compared with the reference image is stored, and pixel data forming an image (hereinafter referred to as a reference image) of each subject image captured at predetermined time intervals after the start of image capturing is updated. Be remembered.

The standard memory 52 and the reference section memory 53.
Has a storage area (a storage area for a total of three screens) for storing images of respective colors of RGB. The address generator 54 generates address data when the pixel data of the standard image and the reference image are stored in the standard memory 52 and the reference memory 53, respectively.

The calculator 55 calculates the vertical contrast data, the horizontal contrast data, and the correlation data in the blur information calculation, and the subtractor circuit 56 and the absolute value circuit 5
7, an adder circuit 58 and a register 59. The correlation data memory 60, the vertical contrast memory 61, and the horizontal contrast memory 62 store the correlation data, the vertical contrast data, and the horizontal contrast data, respectively. The details of the vertical / horizontal contrast data and the correlation data will be described later.

The control CPU 63 centrally controls the operation of each constituent member of the blur detection unit 32.
The control CPU 63 controls the drive of the clock generator 45 to capture a subject image for detecting blur information, and controls the drive of the address generator 54 to reference the image data of the captured standard image and reference image. The partial memory 52 and the reference memory 53 are stored. Further, the control CPU 63 calculates blur information using various data stored in the correlation data memory 60, the vertical contrast memory 61, and the horizontal contrast memory 62.

Next, a method for detecting blurring will be described. In the present invention, the subject image is decomposed into RGB components and the blur is detected for each color component, but in the following description of the blur detection method, only the R component will be described.
With respect to the remaining G and B components, blurring is detected by the same method as that for the R component, and thus description thereof is omitted.

The R component of the subject image extracted by the spectral filter 64 is converted into two-dimensional image data by the CCD 41. Then, by using the image data, the amount of movement (blurring amount) of the subject image over time is calculated to detect the blur of the R component subject image.

The blur detection sequence can be roughly divided into three parts: (1) block selection, (2) correlation calculation, and (3) interpolation calculation. First, the block selection is a process of dividing the image pickup area of the CCD 41 into a plurality of small area blocks and selecting a plurality of blocks suitable for blur detection from the blocks. The subject image formed on the image pickup surface of the CCD 41 has a portion suitable for blur detection and a portion not suitable for blur detection. In the block selection process, if the CCD 41 is an area sensor having I × J pixels for the R component, this imaging area is M
An area that is divided into × N small area blocks, and among these blocks, a plurality of blocks including an image with high contrast are suitable for shake detection (hereinafter referred to as shake detection area)
Is selected as.

I for the R component of the CCD 41
For a pixel array of × J, the standard portion memory 52 and the reference portion memory 53 have a capacity of I × J words for the R component, and the vertical contrast memory 61 and the horizontal contrast memory 62 respectively for the R component. It has a capacity of M × N words. Further, each of the memories 52, 53, 61, 62
Has the same capacity for the other color components G and B.

Next, in the correlation calculation, the standard image and the reference image at the same position as the standard image and at a plurality of positions separated from the same position by a predetermined number of pixels are compared to determine the degree of difference between the two images at each position. Is calculated. The smaller the difference between the standard image and the reference image, the smaller the correlation value, and the position at which the correlation value becomes the minimum is estimated to be the imaging position of the subject image at the time of detecting the blur. Therefore, the distance (pixel pitch × number of pixels) from the same position as the reference image to the position where the correlation value is the minimum is estimated as the amount of movement of the subject image at the time of blur detection.

The above correlation calculation is performed on the image of each selected blur detection area. Assuming that each block includes K × L pixels, the standard image and the reference image at the same position are compared by comparing (K × L) pixel signals with the pixel signals that form the standard image and the pixel signals that form the standard image. This is performed by comparing the pixel signal with the pixel signal forming the reference image at the same position.

Further, the comparison between the reference image and the reference image at a position separated by a predetermined number of pixels ± h (h = 1, 2, ...) From the reference image is performed when the reference image is separated in the row direction and in the column direction. This is done when you leave.

For example, when the reference image is separated from the same position as the standard image by + m pixels in the row direction, (K−m) ×
The L pixel signals are compared by comparing the pixel signals forming the standard image with the pixel signals forming the reference image at positions + m pixels away from the same position as the pixel signal in the row direction. Similarly, if the reference image is separated from the same position as the standard image by + n pixels in the column direction, K ×
For (L−n) pixel signals, the pixel signals forming the standard image are compared with the pixel signals forming the reference image at positions + n pixels away from the same position as the pixel signal in the column direction. Be done.

In the correlation calculation as described above, the number of pixels h =
Correlation values at positions separated by 0, ± 1, ± 2, ... Are calculated. When the number of calculated correlation values is H, the correlation data memory 60 has a capacity of H × H words for each of the RGB color components.

Further, the interpolation calculation is to interpolate the movement amount of the subject image calculated by the correlation calculation. As described above, in the correlation calculation, if the correlation value between the reference image and the reference image at the position separated by the predetermined number of pixels h from the same position as the reference image is the minimum, this position is estimated as the moving position of the subject image. Therefore, if the minimum position of the true correlation value is within the range of one pixel pitch of the estimated position, an error will be generated by this amount. The interpolation calculation is to estimate the minimum position of the true correlation value by using the plurality of correlation values calculated by the correlation calculation, and reduce the error.

The blur detection procedure for the R component will be described below in order by dividing it into the above sections (1) to (3).

In this embodiment, I = 68, J = 58,
The case of K = L = 8, M = 8, N = 6 and H = 5 will be described. Further, the resolution of the A / D converter 51 is 8 bits, the register 59 is 14 bits, and each word of the correlation data memory 60, the vertical contrast memory 61, and the horizontal contrast memory 62 is composed of 14 bits. To do.

FIG. 5 is a schematic view of the image pickup surface for the R component of the CCD 41. In the figure, each of the smallest squares represents a pixel, and the square surrounded by a thick line represents a block. In the figure, the lower left pixel is P _1,1 and the upper right pixel is P _68,52 , the lower left block is B _1,1 and the upper right block is B _8,6 . Each block B _{k, l} (k = 1
˜8, l = 1 to 6) is 8 × 8 pixels P _{i, j} (i = 8k−5)
.About.8k + 2, j = 8l-5 to 8l + 2).

The reason why the peripheral portion for two pixels is provided outside the block is to enable the correlation calculation for the pixels in the portion in contact with the peripheral portion of each block. In the present embodiment, since the reference image shifted by ± 2 pixels at the maximum is compared with the standard image in the correlation calculation, the peripheral portion of 2 pixels is provided around the block.
Generally, assuming that the maximum shift amount of the reference image is ± h, a peripheral portion of at least h pixels is provided around the block.

(1) Block selection In this block selection operation, the contrast of the subject image is calculated for each block, and four blocks with a large vertical contrast and four blocks with a large horizontal contrast are calculated. Eight blocks are selected as the blur detection area.

First, the output of the CCD 41 (image data)
Are stored in both the standard memory 52 and the reference memory 53. The vertical contrast and the horizontal contrast of each block are calculated using this data.

Each block B _{k, l} (k = 1 to 8, l = 1
The procedure of calculating the contrast in the vertical direction in (6) is as follows.

The register 59 is cleared. The pixel data _{P 8k-5,8l-5} stored in the reference unit memory 52 to the input of one of the subtractor circuit 56, the other pixel data stored in the reference unit memory 53 to the input _P 8k- 5,8l- _Four
Is generated by the address generator 54. The subtraction circuit 56 subtracts the two input pixel data, and the output thereof is converted into an absolute value by the absolute value circuit 57. The absolute valued data is added to the contents of the register 59 by the adder circuit 58.

After that, the address generator 54 generates appropriate address data, so that the above-mentioned operations are repeated while sequentially shifting pixels. By repeating this operation, the register 59 block B _k, contrast VC k longitudinal _{l, l} is stored. The vertical contrast VC _{k, l} of the block B _{k, l} is expressed by the following equation.

[0051]

[Equation 1]

The VC stored in the register 59
_{k, l} is transferred to a predetermined address of the vertical contrast memory 61 and stored as the vertical contrast of the block B _{k, l} . By repeating the above-mentioned operations (1) to (5), the vertical contrasts of all blocks are calculated one after another and stored in the vertical contrast memory 61.

Next, the horizontal contrast of each block is calculated. The contrast in the horizontal direction is obtained by the same operation as that in the vertical direction by the address generator 54 sequentially generating appropriate address data.
The register 59 stores the horizontal contrast of the block B _{k, l} ,

[0054]

[Equation 2]

Is stored. After that, HC _{k, l} is transferred to a predetermined address of the lateral contrast memory 62 and stored as the lateral contrast of the block B _{k, l} .

The control CPU 63 selects a block by referring to the vertical and horizontal contrasts of each block obtained by the above procedure. From the 8 × 6 blocks, four blocks are selected in descending order of contrast in each of the vertical and horizontal directions. The selected eight blocks are designated as B1 to B8.

After completion of block selection, control CPU
The reference numeral 63 reads the pixel data of the reference image stored in the reference memory 52 and outputs it to the information writing section 33. The information writing unit 33 records the pixel data of the input reference image on the film as initial image data when the film is wound. This initial image data is
This is input as data when performing blur correction calculation, and by inputting this initial image data, the reproducing apparatus can perform more accurate blur correction. As the initial image data, all the pixel data of the reference image stored in the reference unit memory 52 may be recorded on the film, or only the pixel data of a predetermined part of the reference image may be extracted and recorded. You may. The recording of the initial image data will be described later in detail.

(2) Correlation calculation In the reference section memory 52, the pixel data forming the reference image used for the contrast calculation is left as it is.
In the reference unit memory 53, new data output one after another from the CCD 41 is stored as pixel data forming a reference image. Every time new data is stored in the reference unit memory 53, the pixel data forming the standard image and the pixel data forming the reference image are compared, and the spatial shift (movement amount) between the two images is a blur of the subject image. To be detected. The amount of movement between images is calculated by the correlation calculation / interpolation calculation described below.

Correlation calculation result of block B _{k, l} (correlation value)
Is defined by the following formula.

[0060]

[Equation 3]

Here, S _{i, j} is pixel data corresponding to the pixel P _{i, j} at the (i, j) position stored in the reference part memory 52, and R _{i, j} is stored in the reference part memory 53 ( i, j)
It is pixel data corresponding to the pixel P _{i, j} at the position. Then, the correlation values corresponding to the eight blocks B1 to B8 selected in the block selection operation of (1) above are respectively set to C1 (m,
n) to C8 (m, n), and the total is

[0062]

[Equation 4]

It is defined by

In order to perform the above correlation calculation, the subtraction circuit 56
Pixel P stored in the reference unit memory 52
_{i, j} of the pixel data S _{i, j,} the other pixel P _{i + m,} which is stored in the reference unit memory 53 to the _{input, j + n} pixel data R
_{i + m, j + n} are input. The address generator 54 sequentially generates appropriate address data so that the register 5
C (m, n) is stored in 9 and transferred / stored in a predetermined address of the correlation result memory 60. The above process is performed for all m and n, and a total of 5 × 5 correlation values C
(M, n): (m, n = 0, ± 1, ± 2) is stored in the correlation result memory 60.

(3) Interpolation Calculation FIG. 6 shows C (m, n) obtained by the above correlation calculation.
It is a figure which shows the distribution of. Correlation value C (m, n) is C (0,
0) = 0, and the larger the distance from this point, the larger the value. If the reference image is displaced from the standard image by m ₀ pixels to the right and n ₀ pixels above, C (m ₀ ,
n ₀ ) = 0, and has a larger value as the distance from this point increases.

However, the image shift is not always an integral multiple of the pixel. FIG. 6 shows the correlation value C (m,
It shows the state of the distribution of n). Referring to FIG. 6, the values are actually obtained only on the grid points in the figure, but it is assumed that the values also exist between the grids, and the correlation values have the same magnitude (height). ) Are connected by contour lines. The coordinates of the center MP of the contour line are (x ₀ , y ₀ ), and it is assumed that C (x ₀ , y ₀ ) = 0.

The control CPU 63 stores the correlation value C (m, n) stored in the correlation result memory 60: (m, n =
0, ± 1, ± 2) is used for interpolation calculation, and C (x ₀ ,
The point MP at which y ₀ ) = 0, that is, the movement amount (x ₀ , y ₀ ) of the reference image with respect to the standard image is calculated. The calculation procedure will be specifically described below.

First, the minimum value of C (m, n) C (m ₀ ,
find n ₀ ). The point (m ₀ , n ₀ ) is considered to be the closest point to the point (x ₀ , y ₀ ) among the lattice points in FIG.

Next, in order to obtain x _{0 in the} horizontal direction, two correlation values C (m ₀ -1, n ₀ ) and C (m ₀ +) are obtained.
The distribution state of the correlation values in the horizontal direction is classified into the following (a) to (d) according to the magnitude relationship of (1, n ₀ ). 7A to 7D are diagrams corresponding to the cases (a) to (d) below, respectively.

(A) m ₀ = 0 or ± 1 and C (m ₀ −
In the case of 1, n ₀ ) ≧ C (m ₀ + 1, n ₀ ), it is determined that x ₀ is within the range of m ₀ ≦ x ₀ <m ₀ +1. Then, the point (m ₀ -1, C (m ₀ -1, n ₀ )), the straight line u connecting the point (m ₀ , C (m ₀ , n ₀ )), and the point (m ₀ +
1, C (m ₀ + 1, n ₀ )), and the m coordinate of the intersection point of the straight line u with a slope and the straight line v with the opposite sign is x ₀ . Note that x ₀ thus obtained is calculated by the following equation.

[0071]

[Equation 5]

(B) m ₀ = 0 or ± 1 and C (m ₀ −
In the case of 1, n ₀ ) <C (m ₀ +1, n ₀ ), it is determined that x ₀ is within the range of m ₀ −1 ≦ x ₀ <m ₀ . Then, the point (m ₀ + 1, C (m ₀ + 1, n ₀ )), the straight line u ′ connecting the point (m ₀ , C (m ₀ , n ₀ )), and the point (m ₀ −
1, C (m ₀ -1, n ₀ )), and the m coordinate of the intersection of the straight line u ′ with a slope and the straight line v ′ with the opposite sign is x ₀ . In addition,
The x ₀ thus obtained is calculated by the following equation.

[0073]

[Equation 6]

(C) When m ₀ = -2, and (d) m ₀
In case of = 2, it is determined that the blur exceeding the maximum expected blur amount has occurred, and the blur amount cannot be detected.

By the above procedure, the horizontal blur amount x ₀
Is calculated. Similarly, two correlation values C (m ₀ , n ₀ -1)
And the distribution of the correlation values in the vertical direction are classified according to the magnitude relation between C (m ₀ , n ₀ +1) and the vertical blur amount y ₀ is calculated.

By the operations (1) to (3), the horizontal blur amount x ₀ and the vertical blur amount y ₀ for the R component are obtained. With respect to the G and B components, the blur amounts in the vertical / horizontal directions can be obtained in the same manner as above.

Next, the blur detection timing and its recording will be described. FIG. 8 shows the relationship between the movement (blur) of the subject image in the R component and the camera shake detection timing. The horizontal axis t represents time, and the vertical axis X represents the position of the subject image in the horizontal direction. It should be noted that here, for simplicity, only the lateral blur will be described.
The vertical direction is exactly the same as the horizontal direction, so description thereof will be omitted.

In the figure, t ₀ , t ₁ , t ₂ , t ₃ , ... Are CCDs 4.
The integration start time of 1 is t ₀ ′, t ₁ ′, t ₂ ′, t ₃ ′, ...
Indicates the integration end time. Curve 70 shows the position of the lateral object image on the CCD 41, the point Px _s is the position at the exposure start time t = t _s.

The points Px ₀ , Px ₁ , Px ₂ , Px ₃ , ... Are integrated periods t _{0 to} t ₀ ′, t _{1 to} t ₁ ′, t ₂ to t ₂ ′, t ₃ ˜.
The average position of the subject image on the CCD 41 at t ₃ ', ... In this case, the central time of the integration period is approximately t _m0 = (t ₀ + t ₀ ') / 2, t _m1 = (t ₁
+ T ₁ ') / 2, t _m2 = (t ₂ + t ₂ ') / 2, t _m3 =
The position of the subject image at (t ₃ + t ₃ ') / 2, ...

In the camera of this embodiment, the first integration (integration period t _{0 to} t ₀ ') of the CCD 41 is performed immediately after the exposure is started at the time t _s . The first output image data (initial image data) is stored in the standard memory 52 and the reference memory 53. Then, the above-described contrast calculation and block selection are performed using the stored initial image data.

The second and subsequent integrations (integration periods t _{1 to} t ₁ ',
_{t 2 ~t 2 ', t 3} ~t 3', ...) image data output by the stored one after another reference unit memory 53. The reference part memory 52 stores the initially stored initial image data as it is. Each time the iterative integration is performed and the image data is output, the amount of movement of the subject image in the lateral direction: Px ₁ -Px ₀ , Px ₂ is calculated using both the image data stored in the standard memory and the reference memory. -Px ₀ , Px ₃ -P
x ₀ , ..., And the amount of movement of the subject image in the vertical direction (not shown):
_{Py 1 -Py 0, Py 2 -Py} 0, Py 3 -Py 0, ... is detected.

The amount of movement of the subject image is the distance between the position of the subject image at the exposure start time ts and the average position of the subject image at each integration time, so the amount of movement of the subject image at each integration time is accurate. The data should be as shown in Equation 7 below, but in the present embodiment, tm ₀ ≈t _s ,
It is approximated by Px ₀ ≈Px _s and Py ₀ ≈Py _s, and is represented by the following formula 8. The blur data 0 is always 0 by the above approximation.

[0083]

[Equation 7]

[0084]

[Equation 8]

The time data t for the R component, the horizontal blur amount data X, and the vertical blur amount data Y are obtained by the method described above. Also, for other G and B components, time data and vertical / horizontal blur amount data are obtained by the same method. These blur data are recorded on the film by the information writing unit 33 when the film is wound up. As for the initial image data read out after the block is selected, the initial image data for each of RGB is recorded when the film is wound.

Next, the control of the integration time of the CCD 41 will be described. FIG. 9A is a circuit configuration diagram of the illuminance monitor circuit. The illuminance monitor circuit 43 integrates the light receiving element 431 formed of SPD, the current output from the light receiving element 431 and outputs the integrated value, and the reset switch 433 that resets the integrated value of the integrating circuit 432. It consists of

The light receiving element 431 receives the reflected light from the subject, photoelectrically converts it into a current proportional to the illuminance of the subject, and outputs it to the integrating circuit 432. The integrating circuit 432 integrates the current input from the light receiving element 431, and the integrated value is output to the clock generator 45. The output current of the integrating circuit 432 is proportional to the amount of light incident on the light receiving element 431,
The output current per unit time is proportional to the illuminance of the subject image.

The illuminance monitor circuit monitors the illuminance of the subject in order to keep the charge storage amount of the CCD 41 constant even when the subject is illuminated by an AC light source such as a fluorescent lamp. When the subject under illumination of the AC light source is imaged, if the integration time of the CCD 41 is constant,
The amount of accumulated charge changes every time. (This will be described in detail later.) Therefore, in order to stabilize the charge storage amount of the CCD 41, it is necessary to adjust the integration time according to the illuminance of the subject. In this embodiment, the CCD
Simultaneously with the start of the integration of 41, the integration of the integration circuit 432 of the illuminance monitor circuit 43 is started, and the integration end timing of the CCD 41 is controlled based on the output current of the integration circuit 432. Note that FIG. 9B is a modification example of FIG. 9A.

Next, the operation of the integration time control when the CCD 41 is used as the shake detection sensor will be described with reference to FIG. First, the reset circuit is turned on to reset the integrating circuit, and at the same time, the CCD 41
Clear the charge in the storage part of. Then, when the integration of the CCD 41 is started, the reset circuit is turned off, and the photometric circuit starts the integration.

FIG. 10 is a diagram showing the temporal change of the photometric circuit output. The horizontal axis t represents the integration time, and the vertical axis I represents the magnitude of the output of the photometric circuit. When the subject is illuminated by the AC light source, the output of the photometric circuit rises in a curved line as shown in the figure. The photometric output is set to an appropriate reference value I _0.
In comparison, when the photometric output I reaches I ₀ , the integration circuit is reset and the integration of the CCD 41 is terminated. C
The output of the CD 41 is read and it is determined whether the exposure amount is appropriate. When the exposure amount is appropriate, the reference value I ₀ is used for the subsequent exposures. When the exposure amount is inappropriate, for example, the reference value is multiplied by k, I ₀ × k is set as a new reference value, and the subsequent exposure is performed.

The camera sequence including the above-described blur detection operation will be described with reference to the flowcharts of the main CPU and the blur detection control CPU. First, the sequence of the main CPU 21 will be described with reference to FIG. In a loop of step # A5 (hereinafter abbreviated as step), the main CPU is in a standby state and waits for the switch S1 to be turned on by the first stroke of the shutter button. Switch S1 is ON
Then, the AF completion flag AFEF and various signals are reset (# A10), and the photometry unit 23 and the AF module 2
9. The necessary circuits such as the shake detection unit 32 are powered on (# A15). Next, the timer built in the control CPU 63 is reset and then started (# A2
0), data relating to the taking lens 11 is input from the taking lens circuit 25 (# A25), and photometry is performed (#A).
30).

In # A35, it is determined whether or not the AF completion flag AFEF is 1, and if it is 1, the program is # A50.
Jump to # 1, and if not 1, go to # A40. # A40
Then, the distance to the subject is detected, and the AF completion flag AFEF is set to 1 in # A45. # A50
Photometric value, film sensitivity, and # A4 obtained in # A30
AE calculation is performed based on the distance information obtained at 0. #
At A55, the exposure amount of the CCD 41 is set.

At # A60, it is determined whether or not the switch S2 is turned on by the second stroke of the shutter button. If the switch S2 is ON, the program is # A8
5. If not ON, jump to # A65. #
At A65, it is determined whether the switch S1 is ON. S
If 1 is ON, the program jumps to # A20. If S1 is not ON, since the immersing operation of the shutter button has been released, it is determined at # A70 by the timer incorporated in the control CPU 63 whether or not a predetermined time has elapsed. If a certain time has elapsed, the photometric unit 23, AF
The power supply for the circuits such as the module 29 and the shake detection unit 32 is turned off.
Then, the program jumps to the standby state of # A5 (# A80). If the fixed time has not elapsed, the AF completion flag AFEF is set to 0 (# A75), and the program proceeds to # A65.

When S2 is turned on, the photographing lens 11 is driven to the focus position (# A85), the blur detection signal is set to the H level (# A90), and the blur detection sequence of the control CPU 63 is started. In # A95,
The shutter blades 26 are opened and exposure is started. When the shutter blades 26 are closed and the exposure is completed, the blur detection signal is set to the L level, and the control CPU 63 is informed that the exposure is completed (# A100). After the exposure, the shooting information (date data, exposure value, etc.) in the exposure is output to the information writing unit 33 (# A103).

The film is wound up at # A105,
At the same time, in # A110, the information writing unit 33 records the blur information such as the blur amount and the shooting information such as the date on the film. # Return the shooting lens to the initial position with A115,
After waiting for S2 to be turned off at A120, the program jumps to # A65 to prepare for the next shooting.

Next, referring to FIG. 12, the control CPU
The operation of will be described. In # A15 of FIG. 11,
The power of the shake detection unit is turned on and the sequence is started.

In # B5, each flag and output signal are reset, and in # B10, data relating to the blur detection sensor 14 such as the number of pixels of the CCD 41 is output to the information writing section 33. In # B15, the shake detection signal from the main CPU 21 is waited for the H level, and when it becomes the H level, the shake detection sequence is started. When the blur detection signal becomes H level and the blur detection is started, the integration of the CCD 41 is once reset and then started (# B20). # B2
Waiting for the integration to end in 5, the pixel data of each of the RGB images read after the integration is completed is stored in the reference unit memory 5
2 and the reference unit memory 53 are dumped to a predetermined storage area (# B30). After dumping pixel data, again CCD4
It is started after the integration of 1 is reset (# B35).

In # B40, contrast calculation is performed for each of the RGB color components based on the pixel data dumped in # B30, and block selection is performed based on the calculated contrast value (# B45). # B5
At 0, the pixel data stored in the reference unit memory 52 is output to the information writing unit 33 as initial image data.

At # B55, the completion of the integration of the CCD 41 is awaited, and the pixel data of each image of RGB read after the completion of the integration is dumped into a predetermined storage area of the reference memory 53 (# B60). . After dumping the pixel data,
The integration of the CCD 41 is reset and started again (#
B65).

In # B70 and # B75, the above correlation calculation and interpolation calculation are performed for each of the RGB color components. In # B80, the central time of the integration period is calculated, and in # B85, the RGB time data and the vertical / horizontal blur amount data are output to the information writing unit 33. In # B90, it is determined whether the blur detection signal from the main CPU 21 is at L level. If it is at the L level, shooting has ended, and the program jumps to # B10 to end blur detection. If it is not at the L level, the program jumps to # B55 in order to carry out blur detection subsequently.

The above is the camera sequence performed by the main CPU 21 and the control CPU 63.

Next, a reproducing apparatus of the image recording / reproducing system according to the present invention will be described. FIG. 13 is a block diagram showing a main part of the reproducing apparatus 3 according to the present invention. In the figure, reference numeral 81 denotes a developed film housed in the cartridge 2 described above, and the reproducing apparatus of the present invention converts an image recorded on the developed film 81 into an electric signal and reproduces it on the television monitor 4. Is. 80 is a film 81
Is an optical system for forming an image on a film on an area sensor 83 which is an image pickup element. The area sensor 83 converts the image formed on the imaging area into electrical signals (image signals) of RGB color components and outputs the electrical signals. Area sensor 8
3 includes a sensor memory 94. In the sensor memory 94, the number of pixels and the pixel pitch of the area sensor 83,
Alternatively, data such as the focal length of the image pickup optical system 82 is stored. RGB output by the area sensor 83
The image signal of is converted into digital data by the A / D conversion unit 84 and then stored in the memory 85.

The information reading section 89 reads the photographing information such as the blur information and the date data recorded on the film together with the image by the camera. The conversion unit 90 converts the blur information read by the information reading unit 89 into a format suitable for the reproducing device. Specifically, the blur amount data recorded by the camera as described above is the blur amount on the CCD 41 on the camera side. Therefore, for example, even if the camera side records the blur amount data "moved to the right on the CCD 41 to the X pixel to the Y pixel" on the film, the image data output by the area sensor 83 on the reproducing device side having a different number of pixels or the like. It cannot be used as is for blur correction. The same applies to the initial image data, and the camera side records the imaged data of each pixel of the CCD 41, and the area sensor 83 on the reproduction device side.
No corresponding pixel. The conversion unit 90, based on the data (number of pixels, pixel pitch, etc.) regarding the CCD 41 recorded by the camera side and the data regarding the area sensor 83 on the reproducing device side (input from the sensor memory 94), stores the data on the CCD 41 on the camera side. The blur information is converted into a format on the area sensor 83 on the reproducing device side.

To briefly explain an example of this conversion,
The p pixel on the blur detection sensor 14 is the area sensor 83.
In the above, there are p × P ₁ · β ₁ × β ₂ / P ₂ pixels. Where β ₁
= (Focal length f of photographing lens) / (blurring detection optical system 13)
F2), β ₂ = magnification of the imaging optical system 82, P ₁ is the pixel pitch of the CCD 41, and P ₂ is the pixel pitch of the area sensor 83.

The data read by the information reading section 89 is subjected to the above conversion for the blur information by the conversion section 90, and then stored for the other photographing information as it is in the information storage section 91.

The arithmetic section 86 inputs the blur information stored in the information storage section 91, and corrects the blur by performing image processing on each of the RGB image signals stored in the memory 85. The method of this blur correction will be described later in detail. Further, the calculation unit 86 also performs image processing for color balance and the like based on the other shooting information stored in the information storage unit 91. After blurring correction and other corrections such as color balance are performed, the RGB image signal output from the calculation unit 86 is stored in the memory 87. Then, the output processing unit 88 converts the RGB signal into an NTSC signal, and outputs the NTSC signal to the television monitor through the output terminal 93. The reproduction CPU 92 controls the entire reproduction apparatus 3 including the above-described sequence for blur correction.

Next, a method of blur correction by image processing performed by the arithmetic unit 86 will be described. In the present invention, it is assumed that the original image before blurring is estimated and restored using the blurring image recorded on the film and the blurring information recorded together with the image by the camera. Note that the calculation unit 86 performs blur correction for each of the RGB color components, but for simplicity, the blur correction for the R component will be described below. Therefore, the image (pixel) data, blur data, deterioration function, etc. described below are all related to the R component. Since blur correction is performed for the G and B components in the same manner as for the R component, description thereof will be omitted.

Letting g (x, y) be an image affected by blurring (so-called blurry image) and f (x, y) be the original image,
It is known that the following relationships are established between these images.

[0109]

[Equation 9]

The above equation (9) is represented by the following equation (10) in digital image processing.

[0111]

[Equation 10]

In the equation (10), g (i, j) is pixel data at the position (i, j) of the area sensor, and f (i, j) is f.
(K, l) is the (k, l) of the area sensor to be restored
It is the pixel data at the position (where i ≠ k, j ≠ l).

Equation (9) shows that the image g (x, y) at the (x, y) position of the blurred image is the image f (x, y) at the (x, y) position of the original image. , y) positions other than the (x ', y') image f (x of ', y') indicates that it is produced under the influence of, the degradation function h _xy is the original image the image f (x ' , Y ′) indicates the ratio of the influence of the image f (x, y). Similarly, in the equation (10), the pixel data g (i, j) of the blurred image is the pixel data f (i, j) of the original image at the position (k, l) other than (i, j). The deterioration function h _ij indicates that the pixel data f (k, l) is generated by being influenced by f (k, l).
It shows the ratio of the effect on j).

When the area sensor 83 has m × n pixels, i = k = 1 to m and j = 1 to 1 to n, and the deterioration function h _ij (i−k, j−). l) is a matrix with m rows and n columns. In the present invention, the RGB image data digitally converted by the A / D conversion unit 84 is input to the calculation unit 86, and therefore the following description will be given using Equation (10).

First, the time data and the vertical / horizontal blurring amount data input from the information storage unit 91 are calculated as (ts, Xs, Ys).
(However, s = 0, 1, ... P, and Xs and Ys are vertical / horizontal blurring amount data at time ts, respectively.)
And The deterioration function h _ij is obtained using the blur amount data (ts, Xs, Ys). At this time, assuming that the initial value of h _ij is 0, the deterioration function h _ij is obtained by the following formula.

[0116]

[Equation 11]

Now, g (i, which is the pixel data of the blurred image)
j) is known because the image on the film 81 is converted into an electric signal by the area sensor 83 and input to the calculation unit 86, and the deterioration function h _ij can also be obtained by the above equation (11). By inputting these g (i, j) and _hij into the above equation (10), m * n simultaneous equations are obtained.

By the way, pixel data g (i,
In j), the pixel data f (i, j) is generated in the original image under the influence of the surrounding pixel data f (k, l). Therefore, the number of pixel data forming the original image is The number of pixels is larger than the number of pixel data forming a blurred image. That is, the size of the original image is larger than the size of the blurred image. Therefore, m × n obtained from the above equation (10)
Each simultaneous equation contains pixel data of an area of the original image that does not overlap with the blurred image (hereinafter referred to as unknown pixel data) as unknowns, and the total number of unknowns is m × n or more. You cannot solve the equation.
Therefore, it is assumed that a predetermined dummy data is assumed for the unknown pixel data and the simultaneous equations are corrected to simultaneous equations of m × n elements to obtain a solution.

As the dummy data of the unknown pixel data, a specific constant value may be set uniquely, or the outermost pixel data g (i, i, which constitutes the periphery of the blurred image may be set.
j) or the average value of pixel data. In the former case, the constant value is stored in advance in the information storage unit 91,
In the latter case, it is calculated from the pixel data of the blurred image input by the calculation unit 86.

Next, the operation of restoring the original image by the above calculation will be described using a specific example. For simplicity, it is assumed that the image deterioration due to blur is uniform in each pixel on the area sensor 83.

FIG. 14 is a diagram showing an example of the original image. Here, for the sake of simplicity, the number of pixels of the area sensor 83 is set to 10 × 10 as shown in the figure. FIG. 15 is a diagram showing a state in which the original image of FIG. 14 is deteriorated due to blurring. FIG. 15 shows a case where the photographing region is moved to the lower left by one pixel each, that is, the subject image is moved to the upper right by one pixel between the start and the end of the exposure of the camera. In this case, the pixel data g of the blurred image
As (i, j), the pixel data of the photographing area at the end of the exposure, which is surrounded by the thick line, is obtained. In this embodiment, as shown in FIG. 15, the coordinates of the lower left pixel of the photographing area at the end of exposure are set to (1, 1), and the right direction and the upper direction of the image are set to the positive directions.

Now, it is assumed that blur detection is performed 5 times during the exposure period and the blur data (ts, Xs, Ys) obtained is as follows.

[0123]

[Equation 12]

In these data, at a position (i, j), the image at the position (i, j) of the original image is exposed at time t = 0.0, 0.1, 0.2, 0.5, and at time t = 0.3, 0.4 means that the image at the position (i-1, j-1) of the original image has been exposed. From these data, the above equation (1
Substituting in 1)

[0125]

[Equation 13]

[0126] Therefore, the deterioration function h _ij is as follows.

[0127]

[Equation 14]

Substituting the deterioration function h _ij obtained by the above calculation into the above equation (10) gives the following.

[0129]

[Equation 15]

When the pixel data g (i, j) of the blurred image obtained by the image pickup of the area sensor 83 is actually input to the equation (15), the following is obtained and 10 × 10 simultaneous equations are obtained.

[0131]

[Equation 16]

In the above simultaneous equations, {10 × 10 + (1
0 + 10-1)} = 119 unknowns. Here, the (10 + 10-1) unknowns are the pixel data f (i,
0) (i = 0 to 9) and f (0, j) (j = 1 to 9)
Then, it corresponds to 19 pixel data located on the lower side and the left side of the photographing area at the start of exposure in FIG. As it is, this equation cannot be solved. Therefore, if a specific constant value (for example, 100) is set as dummy data for the pixel data f (0-9,0) and f (0,1-9), the above equation Is

[0133]

[Equation 17]

Therefore, the number of unknowns can be set to 10 × 10. By solving this simultaneous equation, the pixel data f (i, j) of the original image can be obtained.

The above is the blur data (ts, Xs, Ys).
Is used to obtain the pixel data f (i, j) of the original image from the pixel data g (i, j) of the blurred image. In the above description, the method of restoring the original image of the R component has been described, but the original image can be restored in the same manner for the G and B components.

In the above embodiment, it is assumed that all unknown pixel data (pixel data in the area outside the blurred image area in the original image) is 100. However,
The original image can be more accurately restored by approximating the unknown pixel data by a value obtained from the pixel data of the blurred image including the information of the subject, rather than by unequivocally setting a specific value. As another embodiment, an example in which the pixel data of the unknown area is approximated by the value obtained from the pixel data of the blurred image will be described below.

Second Embodiment In the second embodiment, the unknown pixel data is replaced with pixel data g (i, j) of a blurred image at a position adjacent to the position corresponding to the unknown pixel data. . More specifically, the pixel data of the blurred image as described below is substituted into the unknown pixel data f (0 to 9,0) and f (0,1 to 9) in the simultaneous equation (16).

[0138]

[Equation 18]

After the above substitution, the simultaneous equations may be solved for 10 × 10 unknowns as in the first embodiment.
According to the present embodiment, since the pixel data of the blurred image including the information of the subject image is substituted as the unknown pixel data, it is possible to substitute a specific constant value uniquely as in the first embodiment. The original image can be accurately restored. Moreover, in a normal subject, adjacent pixels often have pixel data that are close to each other, and by substituting pixel data of adjacent blurred images, the original image can be restored more accurately.

In addition to the pixel data of the adjacent blurred images, for example, the average value of the pixel data of the blurred images may be substituted.

[Third Embodiment] In the third embodiment, in order to more accurately restore the original image, initial image data recorded on the camera side is substituted as unknown pixel data. The recording method of the initial image data has already been described.

FIG. 16 shows a case where the original image shown in FIG. 14 is picked up by the camera shake detection sensor 14. Generally, the camera shake detection sensor has a smaller number of pixels and a lower resolution than an area sensor on the playback device side.
In this embodiment, for the sake of simplicity, the number of pixels of the blur detection sensor is 5 × 5, the coordinates of the lower left pixel are (1, 1), and the right and upper directions are the positive directions. Each pixel data shown in FIG. 16 is converted into e (k, l) (k = 1
˜5, l = 1 to 5), and this e on the camera side
(K, l) shall be recorded on the film as initial image data.

First, on the reproducing apparatus side, the initial image data e (k, l) recorded on the film is read by the information reading section 89, and then converted into the reproducing apparatus side format by the converting section 90. The pixel data of the initial image converted into the format on the reproduction device side is e ′ (i, j) (i =
1-10, j = 1 to 10), this conversion is performed by the following arithmetic expression, for example.

[0144]

[Formula 19]

When the above conversion by the conversion unit 90 is completed, the pixel data e ′ (i, j) of the initial image is stored in the information storage unit 91.
Memorized in. In the present embodiment, the pixel data e ′ (i, j) of the initial image thus obtained is substituted as the unknown pixel data. Specifically, the unknown pixel data f in the simultaneous equations (16)
The following pixel data are substituted into (0 to 9,0) and f (0,1 to 9).

[0146]

[Equation 20]

After performing the above substitution, the simultaneous equations may be solved for 10 × 10 unknowns as in the first and second embodiments. According to this embodiment, as the unknown pixel data,
The initial image data recorded on the camera side is substituted.
Since the initial image data is pixel data recorded immediately after the start of exposure, it has a value very close to the pixel data f (i, j) of the original image. For this reason, the original image is more accurate than when the specific constant value is substituted as in the first embodiment or when the value obtained from the blurred image g (i, j) is substituted as in the second embodiment. Can be restored.

In this embodiment, the pixel data of the entire surface of the blur detection sensor 14 is recorded on the film as the initial image data. However, as can be seen from the above calculation, since the unknown region is in the peripheral portion of the shooting range, the pixel data in the central portion is not used. Therefore, only the pixel data of the peripheral portion of the blur detection sensor may be recorded on the camera side. By doing so, the number of data recorded on the film can be reduced and the recording area can be saved.

Further, as can be seen from the above calculation, for example, when the subject image is moved one pixel to the upper right, the unknown region is the range of the leftmost column and the lowermost line. Similarly, if one pixel is moved to the left, for example, the rightmost column is an unknown area, and if it is moved downward by two pixels, the upper two rows are an unknown area. Therefore, the camera may first detect the blur amount data and record only a necessary portion on the film as initial image data according to the blur amount. By doing so, the number of data recorded on the film can be further reduced.

[Fourth Embodiment] In the fourth embodiment, the camera-shake blur detection sensor 14 images only the central portion of the photographing range, and pixel data of only the central portion is recorded on the film as initial image data. I shall. FIG. 17 shows a case where the blur detection sensor 14 images only the central portion of the original image shown in FIG. The number of pixels of the blur detection sensor 14 is set to 4 × 4 for simplicity. In this case, the resolution is higher and more accurate initial image data can be obtained as compared with the case where the entire imaging range is imaged (as in the case of the third embodiment). In this embodiment, the camera side of FIG.
It is assumed that the pixel data shown in is recorded on the film as initial image data.

On the reproducing apparatus side, the initial image data recorded on the film is read, converted by the conversion unit 90, and then stored in the information storage unit 91. Then, at the time of calculation of blur correction by the calculation unit 86, the simultaneous equations (16)
To. Specifically, the pixel data f (i,
The pixel data of the initial image is substituted into f (4 to 7, 4 to 7) of j).

In this case, since dummy data is not substituted for the unknown pixel data, the solution (particularly the peripheral portion) of part of the original image becomes indefinite. However, the central part (f
In (4-7, 4-7)), the original image can be accurately obtained by substituting the initial image data. In normal photography, the main subject to be photographed is often located in the center. Therefore, it is very effective to accurately restore the central original image. In addition, in the peripheral portion where the solution is indefinite, it may be restored by the method described in the first and second embodiments. The above is the blur correction method by image processing performed by the calculation unit 86.

The sequence of the reproducing apparatus including the blur correction operation as described above will be described with reference to the flowchart of FIG. In addition, the flowchart of FIG.
The operation from the time when the cartridge 2 is loaded and the reproduced image is set at a predetermined position until the image is reproduced on the television monitor 4 is shown.

In # C5, first, the photographing information recorded on the film on the camera side is read by the information reading section 89, and it is determined in # C10 whether or not there is blur information. If no blur information is recorded, the program is #
The process jumps to C38 and the shooting information other than the blur information is stored in the information storage unit 91. When the blur information is recorded, the conversion unit 90 first inputs the blur detection sensor information (# C15). The conversion unit 90 is based on the input blur detection sensor information (number of pixels, image capturing range, etc.), and the blur data and initial image data recorded on the film by the camera are data in a format in which the arithmetic unit 86 can perform blur correction. (# C20, # C25). The blur data and the initial image data that have been converted are stored in the information storage unit 91 (# C30, # C35). When the blur information is stored, other shooting information is stored in the storage unit 91 in # C38.

From # C40, the reproduction image pickup operation is started. First, the light source 80 for illuminating the set image is turned on (# C40), and then the area sensor 83 converts the image into RGB image data (# C45).
The image data output from the area sensor 83 is A / D
It is digitally converted by the conversion unit 84 and stored in the memory 85 (# C50, # C55).

Next, in # C60, it is determined whether or not the blur information is present. If the blur information is not recorded, the program jumps to # C80. On the other hand, when the blur information is recorded, the calculation unit 86 causes the information storage unit 91 to
The blur data and the initial image data stored in are input (# C65, # C70). The calculation unit 86 performs blur correction of the image data stored in the memory 85 based on the input blur data and initial image data (# C75).
Note that the blur correction method has already been described.
When blur correction is performed for each of the RGB color components, image processing other than blur correction such as color balance is then performed (# C80).

The image data subjected to the image blur correction and other image processing is stored in the memory 87 (# C8
5) is input to the output processing unit 88. The image data input to the output processing unit 88 is converted from an RGB signal into an NTSC signal (# C90) and output to the television monitor 4 through the output terminal 93.

By the above operation, the image recorded on the film 81 is reproduced on the television monitor 4. In addition,
When the reproduction of the set image is completed, the film 81 is fed by one frame and the image of the next frame is set. When the image of the next frame is set, the same operation is performed again from # C5, and the image is reproduced.

The above is the embodiment of the image recording / reproducing system of the present invention. In this embodiment, the blur information detected by the camera is recorded on the film.
A recording medium different from the film such as the C card may be provided and recorded on the recording medium, and the reproducing device may read the blur information from the IC card.

Although a silver salt camera is used as a photographing device, the present invention is not limited to this, and a blur is similarly detected in, for example, a video camera or a still video, and the blur information is displayed in a predetermined area (for example, an audio track) of a video tape or a floppy disk. May be recorded. In the video deck serving as the reproducing device, the recorded blur information may be read and the blur may be corrected by the method described above.

Further, although a device for displaying an image on a television monitor is used as a reproducing device, the present invention is not limited to this, and can be applied to, for example, a printer for printing image data converted into an electric signal.

[0162]

As described above, the image recording of the present invention
The reproduction system is based on a blur detection unit that detects and outputs blur information, an output unit that outputs predetermined predetermined image information, and a blur information based on the blur information and the predetermined image information output from these units. It has a correction means for correcting the blur of the reproduced image. The predetermined image information output by the output unit is input in place of image data that cannot be obtained from the captured image in the image processing for blur correction by the correction unit. Therefore, when mathematically restoring the original image by the above image processing, it is possible to prevent a part of the solution from becoming indefinite. (Claim 1) Further, in the invention of Claim 2, since the predetermined image information output by the output means is a specific constant value, for recording data that cannot be obtained from a photographed image. There is no need to provide a structure on the camera side. In addition, the reproducing device side does not require a calculation or the like for obtaining the image information output by the output means. Therefore, it is possible to realize an image / recording system having a simple structure.

Further, in the third aspect of the invention, the predetermined image information output by the output means is obtained from the image information of the photographed subject image. Therefore, the output image information includes data related to the subject although the deterioration due to blurring has occurred. Therefore, for example, the original image can be more accurately restored as compared with the case of inputting the specific constant value described above.

According to the invention of claim 4, in addition to the first recording means for recording the subject image on the photographing device side, the second recording means for recording the image information used for blur correction is provided. Since the second recording means can record the image information directly from the subject on the photographing device, it is possible to obtain the data of the original image before the blur occurs. On the reproducing device side, the image information recorded by the second recording means is input to the image processing for blur correction.

Since the image information recorded by the second recording means includes the information of the original image as described above, the photographed image which is deteriorated due to blurring when inputting the above-mentioned specific constant value The original image can be more accurately restored as compared with the case of inputting image information from.

[Brief description of drawings]

FIG. 1 is an external view of an image recording / reproducing system according to the present invention.

FIG. 2 is an external view of a camera according to the present invention.

FIG. 3 is a block diagram showing a main part of a camera according to the present invention.

FIG. 4 is a block diagram showing the inside of the blur detection unit of the camera according to the present invention.

FIG. 5 is a schematic view of a light receiving surface of a CCD included in a camera according to the present invention.

FIG. 6 is a graph showing a calculation result of a correlation calculation performed by the camera of the present invention.

FIG. 7 is a graph showing a calculation result of a correlation calculation performed by the camera of the present invention.

FIG. 8 is a graph showing the relationship between the movement of a subject image and the timing of blur detection.

FIG. 9 is a circuit diagram of a circuit that controls an integration time of a CCD included in a camera according to the present invention.

FIG. 10 is a graph showing a temporal change in the output of the photometric circuit that controls the integration time of the CCD.

FIG. 11 is a flowchart showing the operation of the main CPU of the camera of the present invention.

FIG. 12: CPU for camera shake detection according to the present invention
3 is a flowchart showing the operation of FIG.

FIG. 13 is a block diagram showing a main part of a reproducing apparatus according to the present invention.

FIG. 14 is a diagram showing an original image to be restored by blur correction according to the present invention.

FIG. 15 is a diagram showing a blurred image when the original image is moved by one pixel in the upper right direction.

FIG. 16 is a diagram showing a case where a blur detection sensor having 5 × 5 pixels captures an original image.

FIG. 17 is a diagram showing a case where a blur detection sensor having 4 × 4 pixels images only a central portion of an original image.

FIG. 18 is a flowchart showing an operation of the reproducing apparatus according to the present invention.

[Explanation of symbols]

 1 Camera 3 Playback Device 4 Television Monitor 14 Blur Detection Sensor 21 Main CPU 32 Blur Detection Section 33 Information Writing Section 52 Standard Section Memory 53 Reference Section Memory 55 Computing Unit 63 Control CPU 81 Developed Film 83 Area Sensor 86 Computing Section 89 Information Reading unit 90 Conversion unit 91 Information storage unit 92 Reproduction CPU

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁵ Identification number Office reference number FI Technical display location H04N 5/91 J 4227-5C 5/92 Z 4227-5C (72) Inventor Katsuyuki Namba Osaka City Central 2-3-13 Azuchi-cho, Tokyo Osaka International Building Minolta Camera Co., Ltd.

Claims (4)

[Claims] 1. An image recording / reproducing system comprising a photographing device for recording a subject image in a predetermined recording medium and a reproducing device for reproducing the image recorded in the recording medium, wherein the subject image is used as image information in the predetermined recording medium. Image recording means for recording on the recording medium, blur detection means for detecting and outputting information on blurring, image reproducing means for reproducing the image information recorded on the recording medium, and outputting predetermined predetermined image information. Output means, correction means for correcting the blur of the image reproduced by the image reproduction means, based on the information on the blur output by the blur detection means and the predetermined image information output by the output means; An image recording / reproducing system characterized by having. 2. The image recording / reproducing system according to claim 1, wherein the predetermined image information output by the output means is a specific constant value. 3. The image recording / reproducing system according to claim 1, wherein the predetermined image information output by the output means is obtained from image information of a subject image recorded on the recording medium. 4. An image recording / reproducing system comprising a photographing device for recording a subject image on a predetermined recording medium and a reproducing device for reproducing the image recorded on the recording medium, wherein the subject image is recorded on the predetermined recording medium. An image pickup apparatus having a first recording unit, a second recording unit that separately records information about a subject image, and a blur detection unit that detects and outputs information about blurring, and the recording medium. Image reproduction means for reproducing the subject image recorded in the image recording means, and reproduction by the image reproduction means on the basis of the information about the blur output by the blur detection means and the information about the subject image recorded by the second recording means. An image recording / reproducing system, comprising: a reproducing apparatus having a correcting unit that corrects a blur of an image to be recorded.