JPH11224324A - Image signal processing method and device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily provide landscape photographs on which cars or human beings are not photographed by taking out the image components of high correlation from plural image data and generating new images from which objects other than still objects are eliminated. SOLUTION: When the second sheet of the image data are read by a consecutive photographing operation, a memory controller 16 adds input data and data stored in a first memory 18, rewrites the result and rewrites and updates the data contents of the first memory 18. When the prescribed number of times of addition is completed, the data are written to the first memory 18 and a second memory 20. At the time, the data stored in the second memory 20 are outputted to a liquid crystal monitor 28. Moving objects such as the human being and the car, etc., are eliminated from a screen and only the still objects such as buildings and roads, etc., remain. When the pressurization of a release button 32 is released, the image data stored in the second memory 20 are recorded in a record medium 30.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to an image signal processing method and apparatus, and more particularly to a still image processing technique applied to an electronic still camera or the like.

[0002]

2. Description of the Related Art Conventional techniques for processing an image signal include, for example, Japanese Patent Application Laid-Open Nos.
JP-A-116770 and JP-A-6-261249. JP-A-7-177426 discloses that a difference process is performed between a plurality of images acquired via an imaging unit, and this difference information is stored in a memory, and a high-definition image is obtained from a reference image and the difference information. Means for obtaining are disclosed.

[0003] Also, Japanese Patent Application Laid-Open No. Hei 9-116770 discloses that
There is disclosed an image processing apparatus for obtaining a soft focus image by combining data (through signal) of an image read by a photoelectric conversion element such as a CCD and blurred image data obtained by filtering the image data. The image processing apparatus described in Japanese Patent Application Laid-Open No. 6-261249 has processing means for soft-focusing an input image, and performs or does not perform soft-focus processing on one image as a whole. Is switched.

As disclosed in these publications, techniques for digitizing an image signal and processing the image by digital signal processing are widely known.

[0005]

In recent years, digitization of map information has been rapidly promoted, and accordingly, data of a photographed image showing a scene of an actual intersection and the like, and map data created graphically. Linking with is desired. When photographing such actual intersections, it is originally desirable to obtain only images of buildings and roads.However, usually, cars and people enter the screen and only landscapes are photographed. Was extremely difficult in nature.

In a conventional silver halide camera, a bright lens having a long focal length had to be used when focusing on a main subject and blurring the background during portrait photography. It was expensive, large, and could not easily take portrait shots. The present invention has been made in view of such circumstances, and an object of the present invention is to provide an image processing method and apparatus capable of easily acquiring a landscape photograph without a car or a person. Another object of the present invention is to provide an image processing method and apparatus capable of obtaining a portrait photograph with a blurred background even with a small and inexpensive single focus camera.

[0007]

In order to achieve the above object, according to the first aspect of the present invention, image data for a plurality of screens taken through an image pickup means are continuously taken in, and a correlation is obtained from the plurality of image data. It is characterized by extracting an image component having a high image quality and generating a new image excluding subjects other than the still subject.

According to the present invention, an image part of a common still object included in each image data is extracted by extracting a highly correlated image component from a plurality of image data successively acquired, and a moving object or the like is extracted. The image portion of the subject that is not commonly included in each image data disappears.
With this, even when a moving subject (unnecessary subject) such as a person or a car is inevitably captured in a static subject such as a building in an actual photographing, such as a scenery of an intersection where people and cars come and go, By using the image signal processing method of the present invention, it is possible to easily obtain a landscape photograph that includes only a stationary subject such as a building or a road and excludes unnecessary subjects such as people and cars.

According to a second aspect of the present invention, image data for a plurality of screens taken through an image pickup means is taken in, and a correlation process for obtaining a correlation between the plurality of image data is performed. It is characterized in that, after performing different filtering processes on a part having no correlation, a new image is generated by combining the two. When the present invention is applied to portrait photography, a portrait is photographed against a background, and an image including the background and the portrait is captured.
Remove the person from the screen and capture the image with only the background. Note that an image including a person may be captured after capturing an image of only the background first.

Then, a correlated portion and a non-correlated portion are distinguished from these two screen images by a correlation process. Of the two screens, a common scenery portion is extracted as a correlated portion, and a person portion is detected as an uncorrelated portion. The signal processing (soft focus processing) that gives a blur effect is applied only to the correlated part (background), and the uncorrelated part (person part) is synthesized without adding any special processing. You can get a portrait photo.

According to a third aspect of the present invention, there is provided an apparatus embodying the method of the first aspect of the present invention, wherein an image pickup means for picking up an image of a subject and converting the image into image data representing a subject image, and a photographing start signal. In response to the input, the continuous shooting control means for continuously taking in image data for a plurality of screens from the image pickup means, and a highly correlated image component is extracted from the plurality of image data to eliminate subjects other than the still subject. It is characterized by comprising signal processing means for generating new image data, and an output unit for outputting the image data generated by the signal processing means.

As a specific form of the signal processing means,
According to a fourth aspect of the present invention, it is possible to use a signal processing unit including an adding unit for adding a plurality of image data for each pixel and an arithmetic unit for averaging the addition result according to the number of additions.
According to a fifth aspect of the present invention, there is provided an apparatus embodying the method of the second aspect of the present invention, wherein an imaging means for imaging a subject and converting the image into image data representing a subject image, and an input of a shooting start signal. Accordingly, a photographing control unit that captures image data from the imaging unit, a correlation processing unit that calculates a correlation between image data of a plurality of screens, and an image component of a correlated portion based on a processing result of the correlation processing unit. And a filter means for performing different predetermined filter processing on the image components of the uncorrelated portions, respectively, and generating new image data by combining the two image components subjected to different processing by the filter means It is characterized by comprising a combination processing means and an output unit for outputting the image data generated by the combination processing means.

As one form of the filter means, it is possible to use a means for performing a soft focus process only on the image component of the correlated portion. According to a seventh aspect of the present invention, the image signal processing apparatus according to the third or fifth aspect is applied to an electronic still camera.
Alternatively, the image signal processing apparatus according to the present invention is mounted, and recording means for recording the image data generated by the signal processing means on a recording medium is provided.

Since the electronic still camera according to the present invention utilizes the correlation between a plurality of image data, it is preferable that the camera is fixed by a tripod or the like. Means may be provided to ensure the sameness of the shooting range between the image data of a plurality of screens. By doing so, it is also possible to perform hand-held shooting.

According to a ninth aspect of the present invention, the image signal processing apparatus according to the fourth aspect is applied to an electronic still camera. The electronic still camera according to the present invention includes an adding section and an averaging section. An image signal processing device having signal processing means is mounted, and setting means for setting the number of additions by the adding unit; and an addition result each time the addition operation of the number of additions set by the setting means is completed. Signal processing means for averaging, image display means for displaying an image indicated by the image data generated by the signal processing means, and recording means for recording the image data generated by the signal processing means on a recording medium. , Is provided.

According to a tenth aspect of the present invention, in the electronic still camera according to the eighth aspect, during the period in which the photographing instruction member is pressed, the capture of image data by continuous photographing is continued, and the setting is performed. When the averaging process of the number of additions set by the means is completed, the averaged data is newly input to the adding unit as the first image data, and control for repeatedly executing the averaging process of the image data is performed. A recording unit for recording the image data when the pressing of the photographing instruction member is released.

According to the present invention, a camera capable of photographing only a landscape such as a building or a road is realized.

[0018]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of an image signal processing method and apparatus according to the present invention will be described below in detail with reference to the accompanying drawings. FIG. 1 shows an example in which the present invention is applied to an electronic still camera. This electronic still camera 10
Are mainly an image pickup means 12, an A / D converter 14, a memory controller 16, a first memory (memory A in the figure)
18, a second memory (denoted as a memory B in the figure) 20, a central processing unit (CPU) 22, a D /
It comprises an A converter 24, an image compression and recording control circuit 26, a liquid crystal monitor 28, a recording medium 30, and the like.

Although the detailed configuration of the image pickup means 12 is not shown, the image pickup means 12 is composed of a photographing lens, a solid-state image pickup device such as a CCD, and an analog signal processing circuit. The image of the subject formed
The signals are photoelectrically converted in the CCD and sequentially read as video signals. Then, the video signal read from the CCD is sampled and held for each pixel by a CDS circuit, amplified by a gain control amplifier (not shown), and then point-sequentially converted to R, G, and B by the A / D converter 14. And is applied to the memory controller 16.

The memory controller 16 includes a digital image processing circuit such as a luminance signal generation circuit and a color difference signal generation circuit, and performs various operations while utilizing first and second memories corresponding to storage means. The image data output from the A / D converter 14 is stored in the memory controller 1
In step 6, the arithmetic processing is appropriately performed. The details of the data processing will be described later.

The CPU 22 controls and controls each circuit of the camera, and controls the imaging means 12 for performing photographing such as focusing based on a signal input from a release button (corresponding to a photographing instruction member) 32. Do,
It also controls the memory controller 16. Also, CP
U22 performs various calculations such as an exposure value and a focus position in accordance with a predetermined algorithm, and performs controls such as automatic exposure control, auto focus, auto strobe, and auto white balance. That is, the CPU 22 controls the electronic shutter value of the CCD of the imaging unit 12 according to the exposure value calculated based on the image signal to set the exposure value, and controls the gain control amplifier according to the calculated RB gain value to control the white balance. Set.

The CPU 22 calculates a focus evaluation value indicating the sharpness of the subject image from the image signal, and calculates a focus position based on the focus evaluation value. Then, the focus system of the photographing lens is controlled according to the calculated focus position, and the focus position is set. Note that the autofocus unit is not limited to the above-described embodiment, and a known distance measuring unit such as an AF sensor may be used.

The image data output from the memory controller 16 is converted into an analog signal by a D / A converter 24 and guided to a liquid crystal monitor 28 as a monitor output signal. Thus, the image captured by the imaging unit 12 is displayed on the liquid crystal monitor 28. This LCD monitor 28
Not only a still image shot by the release operation but also a video (moving image or intermittent image) before the release operation can be displayed.

The image data output from the memory controller 16 is processed by the image compression and recording control
The data is compressed in a predetermined format such as PEG and recorded on the recording medium 30. The recording medium 30 can take various forms, such as a flash memory with a built-in camera, a smart media, a magneto-optical disk, and an IC card. Next, a method of processing an image signal in the electronic still camera configured as described above will be described.

After the camera is fixed on a tripod or the like, when the release button 32 is pressed and a shooting start signal is applied to the CPU 22, the camera starts a continuous shooting operation, Are continuously taken in.
That is, data of an image photographed via the imaging means 12 is:
First, the data is stored in the first memory 18 via the memory controller 16. At this time, the second memory 20 is cleared.

When the second image data is read by the continuous shooting operation, the register in the memory controller 16 stores the following expression (1).

[0027]

## EQU1 ## The input data + the data of the first memory 18 → the data of the first memory 18... (1) That is, the input data and the first
An addition operation is performed on the data stored in the memory 18, the result of the addition is written back to the first memory 18, and the data content of the first memory 18 is rewritten and updated.

During the continuous shooting operation, 2 ⁿ
(N = 1, 2, 3 ...) times. The number of additions can be freely set by a user through a setting input unit (not shown), and the CPU 22 sets ²ⁿ times of additions to the memory controller 16 according to the user's setting. When the addition of 2 ⁿ times is completed, the data shifted by ⁿ bits is written to the first memory 18 and the second memory 20. At this time, the data stored in the second memory 20 is transmitted to the liquid crystal monitor 28 via the D / A converter 24.
Output to

On the other hand, the first memory 18 stores the n bits
The shifted data (averaged data) is newly added to the first
The data input by the next continuous shooting is cumulatively added according to the same sequence as described above, using the data as the second data. While the photographer keeps holding down the release button 32, the above-described image data capturing and averaging processing are repeatedly executed. That is, each time 2 ⁿ additions are completed (each sequence), the preview screen displayed on the LCD monitor 28 is updated, and as the release button 32 is kept pressed for a longer time, the movement of a person or a car from the screen moves. The moving subject is deleted, and only the still subject such as a building or a road remains.

The photographer releases the pressing of the release button 32 when the unnecessary subject is eliminated while checking the image displayed on the liquid crystal monitor 28. When the release button 32 is released, the second memory 2
The image data stored in 0 is recorded on the recording medium 30 via the image compression recording control circuit 26. in this way,
Utilizing the two memories of the first memory 18 and the second memory 20, while outputting the image data to the liquid crystal monitor 28, the continuous photographed image is continuously taken in and the addition processing is continued. The processing results of the second and subsequent sequences can be obtained in a short time.

If the continuous shooting is completed at the time when the set 2 ⁿ addition processes are completed and the addition process sequence is completed only once, the second memory 20 may be omitted. It is. According to the present embodiment, unnecessary subjects, such as a person and a car, which are inevitably captured when capturing a landscape can be deleted from a plurality of continuously shot images by signal processing. Further, since data of a plurality of continuously shot images are cumulatively added and averaged, there is an advantage that the S / N of the image is improved.

A photographing mode for photographing a photograph in which an unnecessary object is eliminated by signal processing of a continuous photographed image as described above;
A shooting mode (so-called normal shooting mode) in which a shot image is recorded as it is in response to the operation of the release button is provided in one electronic still camera, and the shooting mode can be freely switched by the mode selection means. It is more preferable to configure the above.

Next, a second embodiment of the present invention will be described. FIG. 2 is a block diagram showing a configuration of the electronic still camera according to the second embodiment of the present invention. In the figure, the same parts as those in the block diagram shown in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted. The electronic still camera shown in FIG. 2 is provided with a filter circuit 34 for performing a blurring process (soft focus process) on image data input via the imaging unit 12. Filter circuit 34
Is a low-pass filter (LP
F) or a filter means for averaging the data of several neighboring pixels (for example, four adjacent pixel data) is used.

In the electronic still camera, a portrait button 36 corresponding to an external operation means for selecting whether or not to execute the soft focus processing by the filter circuit 34, and the soft focus processing is obtained. A recording button 38 corresponding to an operation means for selecting whether to save an image (portrait photographed image with a blurred background) in the recording medium 30 is provided.

The procedure of portrait photographing in the electronic still camera configured as described above is as follows. First, the camera is fixed on a tripod or the like, and the release button 32 is operated to photograph a screen including both a person and a background at the same time as when photographing with a normal camera, and the image data is stored in the first memory 18. .

Next, the subject person is removed from the screen, only the background is photographed at the same position, and this image data is stored in the second memory 20. Next, when a predetermined operation member (for example, the portrait button 36) is pressed, the memory controller 16 detects the correlation between the data in the first memory 18 and the data in the second memory 20, and detects the correlated data (corresponding part). ) Is recorded in the first memory 18, and the first data component of the uncorrelated data (unmatched portion) is recorded in the second memory 20.

Then, only the data stored in the first memory 18 is input to the filter circuit 34,
After applying a predetermined process to give an image a blurring effect, the data after the process is written back to the first memory 18. Subsequently, the data in the first memory 18 and the second memory 20 are combined in the memory controller 16, and the combined data is output to the liquid crystal monitor 28 via the D / A converter 24.

When the photographer confirms the monitor output and presses the recording button 38, the composite image is recorded on the recording medium 30 via the image compression recording control circuit 26. As a result, even with a single-focus wide-angle camera, portrait shooting with a blurred background, such as using a telephoto lens, becomes possible. In addition, instead of the above-described filter circuit 34 that provides the soft focus effect, a processing circuit (filter circuit) that replaces a correlated portion with one color of a specific color (for example, white) is used, so that the background is left while leaving a person. Can be completely deleted. As a result, it is possible to easily take an ID photo in which the background is not mixed.

In each of the embodiments described above, the camera is fixed to a tripod or the like in consideration of camera shake. However, by using a well-known camera shake correction technique, hand-held shooting is also possible. Next, an example of the camera shake correction unit will be described. FIG. 3 shows a configuration example of the electronic camera shake correction unit. The electronic image stabilizing means 40 mainly includes
It comprises a motion detection circuit 42, a field memory 44, and a memory control circuit 46, and is provided after the A / D converter 14 in the block diagram described with reference to FIG.

The A / D converter 14 converts an analog signal input from the imaging means 12 into a digital signal, and outputs the digital signal to the motion detection circuit 42 and the field memory 44. The field memory 44 stores the image data for one field, and outputs the image data with a delay of one field period.
Thus, the readout area (screen cutout frame) in one field to be output is controlled.

The motion detection circuit 42 adds the image data in the motion detection area of the current field to the image data added via the field memory 44 and the memory control circuit 46.
The image data in the motion detection area of the field before the field is compared with the image data, and the direction amount of motion between fields (difference motion vector) is obtained. This difference motion vector is obtained from the shift amount (the amount of parallel movement of the screen) at which the correlation value is maximized by performing a correlation operation on each image data in the motion detection areas of the preceding and following fields.

Then, the difference motion vector between the fields obtained in this manner is integrated from the start of the camera shake correction, thereby obtaining the translation amount of the current image with respect to the reference image at the start of the camera shake correction, that is, the integrated motion vector. (Hereinafter simply referred to as a motion vector). The memory control circuit 46 controls the cutout frame of the screen output from the field memory 44 so as to cancel the screen motion based on the motion vector detected by the motion detection circuit 42. That is, with respect to the preceding output screen, the cutout frame of the subsequent output screen is controlled in accordance with the detected motion vector, whereby the parallel movement is performed in the direction opposite to the movement due to the camera shake, and the camera shake is corrected. You. Note that the output screen is set to a predetermined size smaller than the input screen so that the edge of the screen is not cut off during the parallel movement (so that part of the screen is not lost).

Further, at the subsequent stage of the field memory 44,
An image correction circuit (not shown) that performs electronic zoom so that the output screen read from the field memory 44 has the same size as the input screen is provided, and the input signal is enlarged by performing an interpolation operation. It is configured to output an output screen of the same size as the input screen. The image data read from the field memory 44 in this manner is applied to the memory controller 16 described with reference to FIG. The camera shake correcting means 4 is controlled by the memory controller 16 shown in FIG.
It is also possible to use the memory control circuit 46 of “0” also.

The camera shake correcting means 40 constructed as shown in FIG.
By comparing predetermined regions of image data between fields obtained from the imaging means 12 and determining the correlation between the two, the amount of parallel translation of the succeeding image with respect to the preceding image should be detected and output. The image is shifted in the reverse direction by this amount of translation. Accordingly, camera shake can be prevented, and good image capturing is possible even in hand-held image capturing without fixing the camera to a tripod or the like.

Note that the camera shake correcting means is not limited to the embodiment shown in FIG. 3, and other types of camera shake correcting means such as a mechanical camera shake correcting means may be used. In the above-described embodiment, an example in which the image signal processing method and apparatus according to the present invention is applied to an electronic still camera has been described. However, the image signal processing method and apparatus according to the present invention can be used to capture and process still images. It can be widely applied to processing equipment.

[0046]

As described above, according to the image signal processing method and apparatus according to the present invention, a portion having no correlation is deleted from a plurality of continuous images by signal processing. When it is desired to take a picture of a landscape, it is possible to take a picture without unnecessary cars or persons.

Further, according to the present invention, a correlated portion and a non-correlated portion are distinguished from each other by a correlation process from a plurality of images, and different signal processes are applied to both portions, and then these are combined. Therefore, by using the present invention, even in a single-focus wide-angle camera, it is possible to perform portrait photographing with a blurred background, such as using a telephoto lens.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an electronic still camera according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of an electronic still camera according to a second embodiment of the present invention.

FIG. 3 is a block diagram showing a configuration example of a camera shake correction unit added to the electronic still camera shown in FIG. 1 or 2;

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 12 ... Image pick-up means 14 ... A / D converter 16 ... Memory controller 18 ... 1st memory 20 ... 2nd memory 22 ... Central processing unit (CPU) 26 ... Image compression recording control circuit 28 ... Liquid crystal monitor 30 ... Recording medium 34 ... Filter circuit 40: Camera shake correction means 42: Motion detection circuit 44: Field memory

Claims (10)

[Claims] 1. A new image in which image data for a plurality of screens photographed via an image pickup means is continuously taken in, an image component having a high correlation is taken out of the plurality of image data, and a subject other than a still subject is excluded. Generating an image signal. 2. An image data for a plurality of screens photographed via an image pickup means is taken in, a correlation process for obtaining a correlation between the plurality of image data is performed, and a correlated portion and a non-correlated portion are processed. An image signal processing method characterized by performing a different filtering process and then combining them to generate a new image. 3. An image capturing means for capturing an image of a subject and converting the image data into image data representing a subject image, and continuous shooting for continuously capturing image data for a plurality of screens from the image capturing means in response to input of a shooting start signal. Control means; signal processing means for extracting image components having high correlation from the plurality of image data to generate new image data excluding a subject other than a stationary subject; and outputting image data generated by the signal processing means. An image signal processing device, comprising: 4. The signal processing means according to claim 3, wherein said signal processing means comprises an adding section for adding a plurality of image data for each pixel, and a calculating section for averaging the addition result according to the number of additions. The image signal processing device according to any one of the preceding claims. 5. An image capturing means for capturing an image of a subject and converting the image data into image data representing a subject image; a photographing control means for capturing image data from the image capturing means in response to a photographing start signal; Correlation processing means for obtaining a correlation between data; and performing, based on a processing result of the correlation processing means, different predetermined filter processing for image components of a correlated portion and image components of a non-correlated portion, respectively. A filter unit, a combination processing unit that combines the two image components subjected to different processing by the filter unit to generate new image data, and an output unit that outputs the image data generated by the combination processing unit An image signal processing device, comprising: 6. The image signal processing apparatus according to claim 5, wherein said filter means is a means for performing soft focus processing only on the image component of the correlated portion. 7. An electronic still camera, comprising: the image signal processing device according to claim 3; and recording means for recording image data generated by the signal processing means on a recording medium. 8. A system according to claim 7, further comprising a camera shake correction unit, wherein a shift in a shooting range due to camera shake does not occur between image data of a plurality of screens obtained via said imaging unit. An electronic still camera as described. 9. An electronic still camera on which the image signal processing device according to claim 4 is mounted, wherein: a setting unit for setting the number of additions by the adding unit; and an addition operation of the number of additions set by the setting unit. Every time is completed, a signal processing means for averaging the addition result, an image display means for displaying an image indicated by the image data generated by the signal processing means, and an image data generated by the signal processing means An electronic still camera, comprising: a recording unit that records on a recording medium. 10. While the photographing instructing member is being pressed, the capture of image data by continuous photographing is continued,
When the averaging process of the number of additions set by the setting unit is completed, the averaged data is newly input to the adding unit as first image data, and the averaging process of the image data is repeatedly executed. The electronic still camera according to claim 9, further comprising a control unit configured to cause the recording unit to record the image data when the pressing of the photographing instruction member is released.