JPH11261797A - Image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a processing method to obtain a high-definition picture satisfactorily reproducing subjects, by composing plural picture data photographed at different focal distances to generate one picture data through the use of picture data on pictures focused on the subjects so as to focus on the plural subjects. SOLUTION: Plural pictures obtained by photographing to a film F by the image pickup device of a camera, a digital camera photographing pictures and photographing the same scene at different focal distances, so called picture data on a multistage focus picture, are sent to a processor 14. A picture composing part 44 extracts subjects on a focal point in the respective pictures of different focal distances concerning respective supplied pictures. Then the part 44 composes one picture of this scene from the supplied multistage focus pictures through the use of picture data of a picture obtained by focusing on at least a noticed subject according to information on the supplied subject to be noticed and outputs it to a display 20 or a printer 16.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention belongs to the technical field of image processing using a multi-stage focus image.

[0002]

2. Description of the Related Art At present, an image photographed on a photographic film (hereinafter referred to as a film) such as a negative film or a reversal film is printed on a photosensitive material (photographic paper) by projecting an image of the film onto the photosensitive material. Surface-exposing the photosensitive material,
So-called direct exposure (analog exposure) is the mainstream.

On the other hand, in recent years, a printing apparatus using digital exposure, that is, an image recorded on a film is photoelectrically read, the read image is converted into a digital signal, and then various image processing is performed. 2. Description of the Related Art Digital photo printers have been put to practical use, in which image data (latent images) are recorded by scanning and exposing a photosensitive material with recording light modulated in accordance with the image data, and the resulting images are printed.

In a digital photo printer, exposure conditions at the time of printing can be determined by image data processing using an image as digital image data, so that image skipping and blurring caused by backlight, strobe photography, and the like can be corrected, and sharpness can be reduced. By suitably performing (sharpening) processing or the like, a high-quality print that cannot be obtained by conventional direct exposure can be obtained. In addition, image synthesis and division, character synthesis, and the like can also be performed by image data processing, and prints that have been freely edited / processed according to applications can be output. Moreover, according to the digital photo printer, not only can the image be output as a print (photo), but also the image data can be supplied to a computer or the like or stored on a recording medium such as a floppy disk. Can be used for various purposes other than photography. Further, according to the digital photo printer, in addition to the image photographed on the film, an image (image data) photographed by an imaging device such as a digital camera or a digital video camera can be output as a print.

[0005] Such a digital photo printer basically includes a scanner (image reading device) for photoelectrically reading an image recorded on a film, and image data (exposure conditions) for processing the read image and outputting the image. An image input device having an image processing device, a printer (image recording device) for scanning and exposing a photosensitive material in accordance with image data output from the image input device to record a latent image, and an exposed photosensitive material And an image output device having a processor (developing device) that performs a development process on the image to produce a print.

In a scanner, reading light emitted from a light source is incident on a film to obtain projection light carrying an image photographed on the film, and the projection light is transmitted to an image sensor such as a CCD sensor by an imaging lens. The image is read by forming an image and performing photoelectric conversion, and after performing various types of image processing as necessary, the image is sent to the image processing apparatus as image data (image data signal) of the film. The image processing apparatus performs predetermined image processing on image data read by a scanner or image data supplied from a digital camera or the like, and sends the image data to a printer as output image data (exposure conditions) for image recording. In a printer, for example, if the apparatus uses light beam scanning exposure, the light beam is modulated in accordance with image data sent from the image processing apparatus, and the light beam is deflected in the main scanning direction and the main scanning direction. The photosensitive material is exposed (printed) by a light beam carrying an image by transporting the photosensitive material in a sub-scanning direction orthogonal to the direction.
To form a latent image, and then perform a development process or the like in accordance with the photosensitive material in a processor to obtain a reproduced print (photograph) of the image photographed on the film.

[0007]

By the way, when a certain scene is optically photographed, it is impossible to focus on all objects (subjects) existing in the scene, and the photographer ( Depending on the distance from the camera, for example, the person is in focus, but the building behind it is out of focus, or the background is in focus, but the For example, a person may be out of focus. Therefore, even if the photographer focuses on a subject at a different distance, in the example described above, both the person and the background, and desires a print that is well reproduced, such a print cannot be created.

An object of the present invention is to use a multi-step focus image of the same scene photographed at different focal lengths and digital image processing to focus on a plurality of subjects at different distances from a photographer (camera). An object of the present invention is to provide an image processing method capable of obtaining a high-quality image (print) in which the subject is well reproduced.

[0009]

In order to achieve the above object, the present invention relates to a method for storing image data of a plurality of images obtained by photographing the same scene at different focal lengths and information of a subject to be noted in the scene. At least for the subject to be focused on, image data of a plurality of images photographed at different focal lengths is synthesized using image data of an image in which the subject is in focus, and An image processing method for generating image data is provided.

It is preferable that information on a subject to be noted in the scene is obtained from at least one of photographing focal length information and in-scene area information.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an image processing method according to the present invention will be described in detail based on preferred embodiments shown in the accompanying drawings.

FIG. 1 is a block diagram showing an example of a digital photo printer using the image processing method of the present invention. The digital photo printer (hereinafter, referred to as a photo printer 10) shown in FIG. 1 basically includes a scanner (image reading device) 12 that photoelectrically reads an image captured on a film F, and read image data ( An image processing device 14 for performing the image processing method of the present invention, which performs image processing of the image information and the overall operation and control of the photo printer 10, and light modulated according to image data output from the image processing device 14. And a printer 16 that performs image exposure of a photosensitive material (photographic paper) with the beam, develops the material, and outputs it as a (finished) print. The image processing apparatus 14 includes an operation system 18 having a keyboard 18a and a mouse 18b for inputting (setting) various conditions, selecting and instructing processing, and inputting instructions such as color / density correction.
And a display 2 for displaying an image read by the scanner 12, various operation instructions, a condition setting / registration screen, and the like.
0 is connected.

The scanner 12 is a device for photoelectrically reading an image photographed on a film F or the like one frame at a time.
A variable aperture 24, a diffusion box 28 for making the reading light incident on the film F uniform in the surface direction of the film F, an imaging lens unit 32, R (red), G (green), and B
An image sensor 34 having a line CCD sensor corresponding to each image reading of (blue), an amplifier 36,
An A / D (analog / digital) converter 38 is provided.

In the photo printer 10, the type and size of a film such as a new photographic system (Advanced Photo System) or a 135-size negative (or reversal) film, the form of a film such as strips and slides, and the like are determined. In addition, a dedicated carrier that can be mounted on the main body of the scanner 12 is prepared, and by changing the carrier, it is possible to cope with various films and processes. An image (frame) photographed on a film and provided for printing is conveyed to a predetermined reading position by this carrier. When reading an image photographed on the film F in such a scanner 12, the reading light emitted from the light source 22 and the light amount of which is adjusted by the variable aperture 24 is transferred to the film F positioned at a predetermined reading position by the carrier. , And is transmitted therethrough to obtain projection light carrying an image photographed on the film F.

The illustrated scanner 12 reads an image photographed on a film by slit scanning. The film F is positioned at a reading position by a carrier 30 and is transported in a longitudinal direction (sub-scanning direction). The reading light is incident. The projection light transmitted through the film F is regulated by a slit disposed at the reading position and extending in a main scanning direction orthogonal to the sub-scanning direction. As a result, the film F is two-dimensionally slit-scanned by the slit extending in the main scanning direction, and the image of each frame captured on the film F is read.

The reading light is transmitted through the film F held by the carrier 30 to become projection light for carrying an image. The projection light is imaged on the light receiving surface of the image sensor 34 by the imaging lens unit 32. The image sensor 34 has R
A so-called three-line color CCD sensor having a line CCD sensor for reading an image, a line CCD sensor for reading a G image, and a line CCD sensor for reading a B image. Extending. The projection light of the film F is separated into three primary colors of R, G and B by the image sensor 34 and read photoelectrically. The output signal of the image sensor 34 is amplified by the amplifier 36, converted into a digital signal by the A / D converter 38, and sent to the image processing device 14.

The photo printer 1 utilizing the present invention
In 0, the scanner is not limited to the one using the slit scanning, and may use a surface exposure that reads the entire surface of an image of one frame at a time. In this case, for example, using an area CCD sensor, providing means for inserting R, G, and B color filters between the light source and the film F, inserting a color filter, and reading an image with the area CCD sensor. Is sequentially performed by each of R, G, and B color filters, and the image photographed on the film is separated into three primary colors, and sequentially performed.

Further, a photo printer 1 utilizing the present invention
0, a scanner for reading an image of a reflection original, an imaging device such as a digital camera or a digital video camera, a computer communication such as the Internet, a floppy disk or an MO disk (magneto-optical disk) in addition to the image of the film F read by the scanner 12. It is also possible to receive image data from an image data supply source R such as a recording medium) and create a print by reproducing the image data.

As described above, the digital signal output from the scanner 12, the digital camera, etc.
4 (hereinafter, referred to as a processing device 14). The processing device 14 includes a data processing unit 40, an image processing unit 42, and an image synthesizing unit 42 that performs the image processing method of the present invention. The processing device 14 further includes a CPU for controlling and managing the entire photo printer 10 including the processing device 14, a memory for storing information necessary for the operation of the photo printer 10, and the like. , The operation system 18 and the display 20 are connected to each part via the CPU or the like (CPU bus).

The R, G, and B digital signals output from the scanner 12 are subjected to predetermined data processing such as darkness correction, defective pixel correction, and shading correction in a data processing unit 40, and are further Log-converted. Then, the image data is converted into digital image data (density data). When image data is supplied from an image data supply source R such as a digital camera, the image data is supplied to the data processing unit 4.
At 0, the image data is converted into image data corresponding to the photo printer 10, and necessary processing is performed.

The image processing section 42 performs predetermined image processing on the digital image data processed by the data processing section 40, and further converts the image-processed image data into 3D (three-dimensional) data.
-Convert using an LUT or the like to obtain image data corresponding to image recording by the printer 16 or display on the display 22. The image processing performed by the image processing unit 42 is not particularly limited, and various types of known image processing are exemplified. For example, gray balance adjustment using a LUT (look-up table), gradation correction, and density Adjustment, correction of shooting light source type by matrix (MTX), adjustment of image saturation, and other uses such as low-pass filter, adder, LUT, MTX, etc., and averaging process and interpolation calculation by appropriately combining these. Examples include electronic scaling processing, dodging processing (compression / expansion of density dynamic range), and sharpness (sharpening) processing. The image processing unit 42
The various processing conditions in are, for example, performed before the main reading (main scanning) for obtaining the image data for output,
What is necessary is just to set using the image data of the pre-scan which reads an image roughly, or the image data which thinned out the image data corresponding to the image data for output to the printer 16.

The image data processed by the image processing section 42 is output to the image synthesizing section 44 when the image processing method of the present invention is performed, and is output to the printer 16 and the display 20 otherwise.

When the image processing method of the present invention is implemented in the photo printer 10, the same scene captured by an image capturing device such as a camera or a digital camera that captures an image on the film F is stored in the processing unit 14. Image data of a plurality of images photographed at different focal lengths, so-called multi-stage focus images, is sent. In the present invention, a multi-stage focus image photographed by a digital camera is preferable in that the photoelectric reading by the scanner 12 is unnecessary and the alignment at the time of image composition is easy.

A camera for photographing an image may be provided with a multi-stage focus mode for automatically photographing a plurality of images having different focal lengths. The image data of the other images may be output as difference data from the reference image, or the difference data may be encoded and output.

When the image processing method of the present invention is performed, the processing device 14 stores image data of a multi-stage focus image, for example, image data N1 having a short focal length, as schematically shown in FIG. , Image data N having a middle focal length
2, and three types of image data having different focal lengths in the same scene, of image data N3 having a long focal length,
After being subjected to the above-described predetermined processing in the data processing unit 40 and the image processing unit 42, the data is supplied to the image synthesizing unit 44.

The image synthesizing unit 44 detects the sharpness of each of the supplied images on a pixel-by-pixel basis by using the spatial frequency, the sharpness by differential filter processing, or the quantification of edge strength, and the like. For each subject, the subject with the highest sharpness (so-called edge) compared to the other images, that is, focus on each image having a different focal length
Is extracted (or an area) where In the illustrated example, the image data N1 of the short distance focus is focused on the flower, the image data N2 of the middle distance focus is focused on the person, and the image data N3 on the long distance focus is focused on the background mountain. These are extracted in the image.

Next, the image synthesizing unit 44 uses at least image data of an image in which the subject of interest is in focus according to the supplied information of the subject to be focused (hereinafter referred to as the subject of interest). One image of this scene is synthesized from the supplied multi-stage focus image, and the display 2
0 and output to the printer 16. FIG. 2 shows a synthesis example.
In the image a in FIG. 2, a person and a flower are designated as the subject of interest, and in the image b, a person and the mountain are designated as the subject of interest. Both the subject of interest are focused and reproduced sharply, and the other subjects are blurred.

The subject of interest can be entered by the operator using the keyboard 18a in response to a request from a customer (a print requester).
The input may be performed by a known method using the mouse or the mouse 18b. A preferable method is, for example, a method of giving an instruction based on the focal length of an image in which the subject of interest is in focus. For example, when the subject of interest is a person and a flower as in the image a, the operator operates the medium distance focus corresponding to the image data N2 focused on the person, and the image data N2 focused on the flower.
The short-distance focus corresponding to 1 may be instructed and input as information on the subject of interest. In addition, the correspondence between each image data of the multi-stage focus image and the focal length may be such that the operator inputs the focal length (short distance, middle distance, long distance, etc.) for each image, and the image data is stored in a digital camera or the like. In the case of using an imaging device, information on the focal length may be recorded in the header of an image file. In the case of using a film F of a new photographic system, a magnetic recording medium formed on the film may be used. And record the focal length information in
This may be read.

As another preferred method of inputting information on the subject of interest, there is also exemplified a method of cutting out a certain area of the image using the mouse 18b or the like and designating this as information on the subject of interest. These means may be used in combination.

The method for synthesizing images is not particularly limited, and various known methods for synthesizing a plurality of images (image data) can be used. For example, a method of incorporating (replacing) an image (image data) of a focused subject of interest taken in another image into a single image in which the focused subject is focused is used as a reference image. Is done. Alternatively, as described above, when the subject of interest is designated by the focal length, all (the sharpest) subjects focused by the image of the designated focal length are incorporated in the same reference image. Good.

If there is an unnecessary subject such as an unrelated person or dust in the scene, and this is focused in the synthesized image, the unnecessary subject (for example, using the mouse 18b) is used. An area including the image may be designated, and the image may be synthesized using the unfocused image in the multi-stage focus image. Further, at the time of image synthesis, in order to make the created image a natural image, for example, image data is weighted at an edge portion (end portion) of each subject, and the images are synthesized, and the image of each subject is synthesized. The seam may be finished smoothly and naturally.

Here, since the multi-stage focus images have different focal lengths even in the same scene, the size of each subject in the images is actually slightly different. Therefore, in order to prevent image shift when combining images and omission of an image due to the image shift, the reference image is based on an image having a focused subject in the scene and having the longest focal length. It is preferred that For example, in the example shown in FIG. 2, when a person and a flower are instructed as a subject of interest as shown in an image a, a medium-range focus image (image data N2) where the person is in focus is taken. As a reference image, a flower or an in-focus subject of an image (image data N1) of a short-distance focus is incorporated. Further, as shown in the image b, when a person and a mountain are designated as the subject of interest, the image of the middle distance focus is determined based on the image of the long distance focus (image data N3) where the mountain is in focus. The person or the focused object of (image data N2) is incorporated.

Further, from the sharpest subject in each image of the multi-stage focus image, the photographing distance of each subject in the scene and the distance between the subjects are obtained in pixel units, and information on the virtual binocular eyes is obtained by using the information of this distance. A stereoscopic image may be created by creating a visual image.

The image processing section 42 extracts a main part (main subject) of the image as necessary and performs image processing according to the extracted main part. However, a multi-stage focus image is supplied. In this case, information on the focused object (region) extracted for each image may be used for extracting a main part of the image.

For example, in the image shown in FIG. 3A, when the face of a person is extracted as a main part and the color of the ground (background) of the area where the flower exists is close to the skin color, In the conventional main part extraction, an area including a flower may be erroneously extracted as a face together with the face.

Here, the in-focus subject in each of the multi-stage focus images is almost the same distance from the photographer. Further, in the multi-stage focus image, the photographer usually photographs one image focused on the most important subject. In this method, in each image of the multi-stage focus image, the main part is extracted by focusing on the focused object, and the consistency of the extraction result is checked. In a photograph, even if there are a plurality of main parts, the distance from the photographer of each main part is usually similar. For example, FIG.
When a main face is extracted from the in-focus subject of each image in the multi-stage focus image as shown in FIG. 3B, it is appropriate that the main part exists only in one of the three images. If a main part is extracted from a plurality of images having different focal lengths, one of them can be determined to be an erroneous extraction. Therefore,
As shown in FIG. 3 (B), a face is extracted from both the image with the middle focus (image data N2) and the image with the short focus (image data N1). If the image is of a medium focus image, the main part extracted from the short focus image can be determined to be inappropriate and can be canceled. That is, erroneous extraction of the main part can be prevented.

As another face extraction judging method, after applying face extraction processing to the entire screen, the distribution of the focusing distance for each pixel in each extraction area is checked, and consistency is checked to reduce erroneous extraction. . For example, in FIG. 3A, an area extracted by superimposing a flower and a mountain is divided into areas having greatly different focusing distances, so that it is determined that the area is not a face. Since the distances are almost the same and the same area, the face is correctly determined. Alternatively, erroneous extraction can be reduced by dividing the screen by a set of pixels for each predetermined focusing distance range and applying face extraction processing to each of the divided areas. For example, when the face extraction processing is applied to each of the in-focus areas of each of the multi-stage focus images N1 to N3 in FIG. 3B, the face area is correctly extracted only from the in-focus area of the image N2.

In a portrait or the like, it is preferable to appropriately finish not only the face but also the body of the person. However, since the body of the person is physically close to the face from the photographer, the face is in focus. By extracting the torso from the current image, the accuracy and efficiency of the extraction can be improved. In addition, usually, in the vicinity of a person in a captured scene, the distance from the photographer is close to the face only in the body, so if the body that is the in-focus subject is known, the area extracted as a face candidate The ratio of the background portion occupying the periphery can also be used for the consistency evaluation of face extraction. Alternatively, the imaging magnification may be obtained from the information on the focal length, and this may be compared with the extracted size of the focused main part, the body, or the like, and used for evaluation of the consistency of face extraction. Furthermore, it analyzes the shooting scenes such as close-up shooting, portrait, landscape photography, etc. from the focal length and shape of the in-focus subject, and controls exposure, sharpness enhancement and blurring for each subject according to the type of scene. , Image processing unit 42
May be controlled.

In the above method, the method of extracting the main part is not particularly limited, and various known methods can be used. For example, face extraction for extracting a region assumed to be human skin from the color / density and continuity of an image (pixel),
Japanese Patent Application Laid-Open No. 184925/1992 discloses that an image is binarized using a threshold value, for example, is separated into a high-density region and a low-density region to perform region extraction, and then contour extraction of each region is performed.
A method of extracting a subject that uses this to extract a hair part, a body, a face internal structure (eye part), a background part, and the like is exemplified. Further, in addition to this, for example,
No. 346332, No. 4-346333, No. 4-346
No. 334, No. 5-100328, No. 5-158164
Nos. 5-165119, 5-165120, 6-67320, 6-160992, 6-16
No. 0993, No. 6-160994, No. 6-16099
No. 5, No. 8-122944, No. 9-80652, No. 9-101579, No. 9-138470, No. 9-1
Examples of various subject extraction methods disclosed in each publication of No. 38471 are exemplified.

As described above, the image (image data) processed by the image processing unit 42 or the image synthesizing unit 44
Is output to the display 20 and the printer 16. The printer 16 prints a latent image by exposing a photosensitive material (photographic paper) in accordance with the supplied image data, and performs a predetermined process on the exposed photosensitive material and outputs it as a print. And a processor (developing device). In the printer, for example, after cutting the photosensitive material into a predetermined length according to the print, the back print is recorded, then, R exposure according to the spectral sensitivity characteristics of the photosensitive material,
The three types of light beams of G exposure and B exposure are modulated in accordance with the image data output from the processing device 14 to be deflected in the main scanning direction, and to convey the photosensitive material in a sub scanning direction orthogonal to the main scanning direction. Thereby, the photosensitive material is two-dimensionally scanned and exposed with the light beam to record a latent image and supply the latent image to the processor. The processor that received the photosensitive material developed the color,
A predetermined wet development process such as bleach-fixing or washing is performed, dried to form a print, sorted into a predetermined unit such as one film, and accumulated.

Although the image processing method of the present invention has been described in detail above, the present invention is not limited to the above-described embodiments, and various improvements and modifications may be made without departing from the gist of the present invention. Of course.

[0042]

As described above in detail, according to the present invention, objects at different distances from the photographer are focused together by using a multi-stage focus image obtained by photographing the same scene at different focal lengths. Thus, an image reproduced well can be created, and a high-quality print with higher added value can be created.

[Brief description of the drawings]

FIG. 1 is a block diagram of a digital photo printer using an image processing method of the present invention.

FIG. 2 is a conceptual diagram illustrating an image processing method according to the present invention.

FIGS. 3A and 3B are conceptual diagrams for explaining a main subject extraction method using a multi-stage focus image.

[Explanation of symbols]

 10 (Digital) Photo Printer 12 Scanner 14 (Image) Processing Device 16 Printer 18 Operation System 20 Display 22 Light Source 24 Variable Aperture 28 Diffusion Box 30 Carrier 32 Imaging Lens Unit 34 Image Sensor 36 Amplifier 38 A / D Converter 40 Data Processing Unit 42 image processing unit 44 image synthesis unit

Claims (2)

[Claims] An image data of a plurality of images obtained by photographing the same scene at different focal lengths and information on a subject to be noted in the scene are obtained. An image processing method comprising: combining image data of a plurality of images photographed at different focal lengths using image data of a focused image to generate image data of one image. 2. The image processing method according to claim 1, wherein information of a subject to be noted in the scene is obtained by at least one of photographing focal length information and in-scene area information.