JP2005228104A - Image processing device, method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To keep quality of each image and to create a composite image giving a natural impression in creation of the composite image consisting of a plurality of images. <P>SOLUTION: In an image processing part 70, data analysis is carried out on a first image inputted by a control part 7, and an image area of a principal object serving as a composite object area is extracted. A defocus area, which is a band-shaped area within a predetermined area from the principal object in the vicinity of a boundary between the principal object and its back ground, that is, in the vicinity of the principal object, is extracted from the first image. When the defocus area is extracted, a second image of the composite object inputted by a control part 7 and an image of the principal object area are combined together, and based on image information of the defocus area, image processing is carried out for creating the composite image. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program that perform composite processing of a plurality of images to create a composite image.

  Today, digital image data taken with a photographing device such as a digital camera is recorded on various recording media such as a CD-R (Compact Disk-Recoderble), a floppy (registered trademark) disk (hereinafter referred to as FD), a memory card, and the like via the Internet. Etc., and display output (soft copy) on display means such as CRT (Cathode Ray Tube), LCD (Liquid Crystal Display), and plasma display, and printing means such as electrophotographic printers, inkjet printers, thermal printers, etc. The output methods are diversified, such as printing on a recording medium (hard copy).

  Also, when outputting digital image data, various adjustments such as gradation adjustment, brightness adjustment, color balance adjustment, sharpness enhancement, etc. are performed so that a desired image quality can be obtained according to the device characteristics on the output device. Image processing is generally performed, and the development of the image processing technology is remarkable.

  In recent years, with the diversification of image output methods and the development of image processing technology, general individual users have obtained digital images taken with digital cameras, etc., and film images taken with silver halide cameras, using scanners. Opportunities for using digital images are increasing, such as performing image processing and image processing on such digital images and posting them on homepages on the Internet or using them for printing on postcards.

  There are various types of image processing, and one of them is a composition process in which an image of a main subject in an image and another background image are combined to create one image. Usually, when the image area of the main subject is simply cut out from the image and pasted on the background image, there is no sense of unity between the main subject and the background, giving an unnatural impression that the main subject has been pasted. This is because an image in which both the foreground (pointing to the main subject) and the background are in focus is used, and a sense of depth and stereoscopic effect are not felt. Accordingly, various image processing methods have been developed in order to eliminate such unnaturalness.

For example, blur processing called defocus is one of them. In contrast to an unnatural image in which both the foreground and the background are in focus, the background image is blurred (defocused) to give a sense of distance and depth and make it appear natural (for example, Non-patent document 1). Also, by creating a non-sharp image in which a part of the original image has been blurred, and combining this unsharp image with the original image that has not been blurred, a lens with a shallow depth of field can be used. There has also been proposed an image processing method capable of obtaining a natural image with a blurred background such as a photographed image (see, for example, Patent Documents 1 and 2).
JP-A-9-172600 JP-A-9-116770 "Defocusing device for background synthesis" IEICE Technical Report (Hayashi, Shimoda, Kinji), IE88-10, pp.69-75, 1988

  According to recent research, the main subject placed in three dimensions and the background structure are compressed and drawn in two dimensions, but the image has no sense of incongruity because there is a blurred area around the main subject. It was found that this was due to the blur in this area. However, in the conventional image processing methods according to the above documents 1 to 3, the distance between the main subject and the synthesized background can be balanced, but when the main subject is extracted, only the clear main subject region is extracted and the background is extracted. Since it is combined with the image, there is no continuity near the boundary between the main subject and the background, leaving an unnatural impression of pasting. Further, if the background is blurred to give a sense of distance, the sharpness of the background image is lost, and the use of the composite image is limited.

  An object of the present invention is to create a composite image that retains the image quality of each image and gives a natural impression when creating a composite image of a plurality of images.

The invention according to claim 1 is an image processing apparatus,
Image acquisition means for acquiring a first image and a second image to be combined;
A synthesis target area extracting means for extracting a synthesis target area from the acquired first image;
Defocus area extracting means for extracting a defocus area related to the extracted synthesis target area from the first image;
Combining processing means for generating a combined image using the extracted combining target region, the defocused region, and the acquired second image;
It is characterized by providing.

The invention according to claim 2 is the image processing apparatus according to claim 1,
The defocus area extracting unit extracts, as a defocus area, an image area within a predetermined range from an edge portion of the synthesis target area around the synthesis target area extracted by the synthesis target area extraction unit. .

The invention according to claim 3 is the image processing apparatus according to claim 1,
The defocus area extraction means includes a composition target area extracted by the composition target area extraction means under a prescribed extraction condition, and an allowable area extracted from the first image by an extraction condition that relaxes the prescribed extraction condition. The difference is extracted as a defocus area.

According to a fourth aspect of the present invention, in the image processing apparatus according to the first aspect,
The defocus area extraction unit extracts a background area of a synthesis target area from the first image, and extracts an image area obtained by subtracting the background area and the synthesis target area from the first image area as a defocus area. It is characterized by doing.

The invention according to claim 5 is the image processing apparatus according to claim 1,
An operation means for designating an image area in the first image;
The defocus area extracting unit extracts an image area designated through the operation unit as a defocus area.

The invention according to claim 6 is the image processing apparatus according to any one of claims 1 to 5,
The composition processing unit creates a composite image in which images of the composition target area and the defocus area are arranged at positions designated on the second image.

The invention according to claim 7 is the image processing apparatus according to any one of claims 1 to 5,
The synthesis processing means performs a defocus process on a second image area corresponding to a defocus area image whose position is specified on the second image, and the second image area surrounded by the defocused area is subjected to the defocus process. The composition target region image is arranged to create a composite image.

The invention according to claim 8 is the image processing apparatus according to claim 7,
The defocus processing is unsharpening processing,
The synthesis processing unit calculates a frequency characteristic of the defocus area, and determines a degree of unsharpening by the unsharpening process according to the calculated frequency characteristic in the defocus area.

The invention according to claim 9 is the image processing apparatus according to claim 7, wherein the defocusing process is a process of mixing the second image and the defocused area image at a predetermined ratio. .

The invention according to claim 10 is an image processing method,
An image acquisition step of acquiring a first image and a second image to be combined;
A synthesis target area extracting step of extracting a synthesis target area from the acquired first image;
A defocus area extracting step of extracting a defocus area related to the extracted synthesis target area from the first image;
A synthesis processing step of creating a synthesized image using the extracted synthesis target area, the defocus area, and the acquired second image;
It is characterized by including.

The invention according to claim 11 is the image processing method according to claim 10,
In the defocus area extraction step, an image area within a predetermined range is extracted as a defocus area from an edge portion of the synthesis target area around the synthesis target area extracted in the synthesis target area extraction step. .

The invention according to claim 12 is the image processing method according to claim 10,
In the defocus area extraction step, a synthesis target area extracted under a specified extraction condition in the synthesis target area extraction step and an allowable area extracted from the first image under an extraction condition that relaxes the specified extraction condition The difference is extracted as a defocus area.

The invention according to claim 13 is the image processing method according to claim 10,
In the defocus area extraction step, a background area of a synthesis target area is extracted from the first image, and an image area obtained by subtracting the background area and the synthesis target area from the first image area is extracted as a defocus area. It is characterized by doing.

The invention according to claim 14 is the image processing method according to claim 10,
In the defocus area extracting step, the image area designated via the operation means is extracted as a defocus area.

The invention according to claim 15 is the image processing method according to any one of claims 10 to 14,
In the synthesizing process step, a synthesized image is created in which the images of the synthesis target area and the defocus area are arranged at positions designated on the second image.

The invention according to claim 16 is the image processing method according to any one of claims 10 to 14,
In the composition processing step, a defocus process is performed on a second image area corresponding to a defocus area image whose position is specified on the second image, and the second image area surrounded by the defocused area is The composition target region image is arranged to create a composite image.

The invention according to claim 17 is the image processing method according to claim 16,
The defocus processing is unsharpening processing,
In the synthesizing process step, the frequency characteristic of the defocus area is calculated, and the degree of unsharpening by the unsharpening process is determined according to the calculated frequency characteristic in the defocus area.

The invention according to claim 18 is the image processing method according to claim 16,
The defocus process is a process of mixing the second image and the defocus area image at a predetermined ratio.

The invention according to claim 19 is
On the computer,
An image acquisition function for acquiring a first image and a second image to be combined;
A synthesis target area extraction function for extracting a synthesis target area from the acquired first image;
A defocus area extraction function for extracting a defocus area related to the extracted synthesis target area from the first image;
A composition processing function for creating a composite image using the extracted composition target region, the defocus region, and the acquired second image;
It is an image processing program for realizing the above.

The invention according to claim 20 is the image processing program according to claim 19,
The defocus area extraction function extracts an image area within a predetermined range from an edge portion of the synthesis target area as a defocus area around the synthesis target area extracted by the synthesis target area extraction function. .

The invention according to claim 21 is the image processing program according to claim 19,
The defocus area extraction function includes: a synthesis target area extracted under a specified extraction condition by the synthesis target area extraction function; and an allowable area extracted from the first image according to an extraction condition that relaxes the specified extraction condition. The difference is extracted as a defocus area.

The invention according to claim 22 is the image processing program according to claim 19,
The defocus area extraction function extracts a background area of a synthesis target area from the first image, and extracts an image area obtained by subtracting the background area and the synthesis target area from the first image area as a defocus area. It is characterized by doing.

The invention according to claim 23 is the image processing program according to claim 19,
The defocus area extraction function is characterized in that an image area designated via the operation means is extracted as a defocus area.

The invention described in claim 24 is the image processing program according to any one of claims 19-23,
The composition processing function creates a composite image in which images of the composition target area and the defocus area are arranged at positions designated on the second image.

The invention described in claim 25 is the image processing program according to any one of claims 19-23,
The composition processing function performs a defocus process on a second image area corresponding to a defocus area image whose position is specified on a second image, and applies the defocus process to the second image area surrounded by the defocused area. The composition target region image is arranged to create a composite image.

The invention described in claim 26 is the image processing program according to claim 25,
The defocus processing is unsharpening processing,
The synthesis processing function calculates a frequency characteristic of the defocus area, and determines a degree of unsharpening by the unsharpening process according to the calculated frequency characteristic in the defocus area.

The invention described in claim 27 is the image processing program according to claim 25,
The defocus process is a process of mixing the second image and the defocus area image at a predetermined ratio.

  According to the first aspect of the present invention, since the synthesized image is created using the defocus area related to the synthesis target area, an unsharp area can be created around the synthesis target area in the synthesized image. Therefore, continuity is maintained at the boundary portion between the synthesis target area and the second image area as the background, and a synthesized image with a natural impression can be obtained. In addition, since it is not necessary to unsharp the entire second image, the image quality of the original image can be maintained.

  According to the second aspect of the present invention, as a method for extracting a defocus area, a method for extracting an image area within a predetermined range from an edge portion of the composition target area as a defocus area can be applied.

  According to the third aspect of the present invention, as a method for extracting the defocus area, the difference between the synthesis target area extracted under the specified extraction condition and the allowable area extracted under the extraction condition with the specified condition relaxed is calculated. A method of extracting as a defocus area can be applied.

  According to the fourth aspect of the present invention, as a method for extracting the defocus area, a method for extracting the image area obtained by subtracting the background area and the synthesis target area from the first image area as the defocus area is applied. be able to.

  According to the fifth aspect of the present invention, as a method for extracting the defocus area, a method for extracting the area designated via the operation means as the defocus area can be applied.

  According to the sixth aspect of the present invention, a method of arranging the composition target region image and the defocus region image at the position designated in the second image can be applied as a method of creating the composite image.

  According to the seventh aspect of the present invention, as a method of creating a composite image, the second image region that is defocused on the second image region corresponding to the defocused region image and is surrounded by the defocused region It is possible to apply the method of arranging the composition target region image to the above. Therefore, only the blurring feeling of the defocus area can be transplanted to the composite image, and a more natural composite image can be created.

  According to the eighth aspect of the invention, since the second image area corresponding to the defocus area image is subjected to the non-sharpening process according to the frequency characteristic of the defocus area, the blur feeling originally possessed by the defocus area is synthesized. Can be reproduced in the image.

  According to the ninth aspect of the present invention, it is possible to apply a method of creating a composite image by mixing the defocus area image and the second image area in the second image area corresponding to the defocus area image. . Therefore, it is possible to transplant the blurring feeling of the defocus area into the composite image.

  In the present embodiment, a print lab system is assumed, and an example of a print creation system in which an image recording apparatus and an image processing apparatus according to the present invention are integrated will be described.

First, the configuration will be described.
FIG. 1 shows an appearance of a print creation system 1 according to the present embodiment.
As shown in FIG. 1, the print creation system 1 includes a main body unit 2 and a magazine loading unit 3. In the main body 2, an exposure processing unit 4 that exposes a photosensitive material that is a recording medium, a print creation unit 5 that develops the exposed photosensitive material and dries to create a print, and a print creation system 1 are integrated. The control part 7 which controls automatically is provided. In addition, a tray 6 for discharging the created print is provided on the right side surface of the main body 2.

  A CRT 8 and an operation unit 11 are provided on the upper part of the main body unit 2 for an operator's print operation. In addition, it is good also as providing a touch panel with CRT8. Further, a film scanner unit 9 which is a transparent document reading device is disposed on the left side of the CRT 8, and a reflective document input device 10 such as a flood bed scanner is disposed on the right side.

  An original read from the film scanner unit 9 or the reflection original input device 10 includes a photographic photosensitive material. As the photographic photosensitive material, a color negative film, a color reversal film, a black and white negative film on which a frame image taken by an analog camera is recorded, Examples include black and white reversal films. The film scanner unit 9 reads the frame images recorded on these photographic photosensitive materials and converts them into digital information to obtain frame image data. When the photographic photosensitive material is color paper such as photographic paper, the frame image data can be obtained by reading the frame image with the reflective original input device 10.

  An image reading unit 14 is provided on the front surface of the main body unit 2. The image reading unit 14 includes a PC card adapter 14a and an FD adapter 14b, and the PC card 13a and the FD 13b can be inserted therein. The PC card 13a has a memory in which a plurality of frame image data is stored by photographing with a digital camera. The FD 13b stores, for example, a plurality of frame image data obtained by photographing with a digital camera.

  1 shows an example in which the operation unit 11, the CRT 8, the film scanner unit 9, the reflective document input device 10, and the image reading unit 14 are configured integrally with the main body unit 2, but are provided separately. Also good.

  Similarly, an image writing unit 15 is provided on the front surface of the main body unit 2. The image writing unit 15 is provided with an FD adapter 15a, an MO (Magneto Optical disk) adapter 15b, and an optical disk adapter 15c. The created image data such as a composite image and other various data can be written to each recording medium. The image reading unit 14 and the image writing unit 15 may be configured to apply other recording media such as a multimedia card, a memory stick, an MD (Mini Disk), and a CD (Compact Disk) in addition to the above recording media. Good.

Further, the control unit 7 is provided with a communication unit (not shown in FIG. 1; the communication unit will be described later), and the captured image is directly taken from another computer in the facility or a distant computer via the Internet. It is also possible to function as a so-called network printer device.
Also, part or all of the image data generated by the image processing method according to the present invention can be sent to the user via the Internet or the like as part of a commercial service.

Next, the functional configuration of the print creation system 1 will be described with reference to FIG.
As shown in FIG. 2, the print creation system 1 reads document information by the film scanner unit 9, the reflective document input device 10, and the image reading unit 14 in accordance with an instruction from the operation unit 11. After the image data processing according to 7, various prints P 1, P 2, and P 3 are created by the exposure processing unit 4 and the print creation unit 5.

  The main unit 2 includes a control unit 7, an exposure processing unit 4, a print creation unit 5, a communication unit 32, and a data storage unit 71. The communication unit 32 includes a communication interface such as a network interface card or a modem, and transmits / receives information to / from an external device on the communication network. The data storage unit 71 stores and sequentially accumulates a data group of image data and print instruction information thereof (information such as a frame number of the image instructed to be printed and its print size).

The flow of print processing in the print creation system 1 will be described below with reference to FIG.
First, image data is input by the film scanner unit 9, the reflective document input device 10, the image reading unit 14, or the communication unit 32. In the case of the film scanner unit 9, frame image data from the developed negative film N obtained by developing the negative film taken by the analog camera Q1 is input. In the case of the reflective original input device 10, frame image data from a print P obtained by developing a frame image on a photographic printing paper is input. In the case of the image reading unit 14, image data obtained by photographing with the digital camera Q2 is recorded on the recording media 13a and 13b, and image data is read from the recording media 13a and 13b inserted in the adapters 14a and 14b. Input. In the case of the communication unit 32, work instruction information such as image data and printing is received directly from a distant computer via a communication network such as the Internet.

  The input image data is output to the control unit 7. Under the control of the control unit 7, the input image data is temporarily stored in the data storage unit 71, and this image data is displayed and output on the CRT 8. Then, image data processing is performed by the image processing unit 70 in accordance with an instruction input via the operation unit 11.

  Next, when printing image data, the control unit 7 creates exposure image data and outputs it to the exposure processing unit 4. In the exposure processing unit 4, the photosensitive material is exposed based on the input exposure image data. When the exposure processing is completed, the photosensitive material is output to the print creating unit 5. In the print creating unit 5, the exposed photosensitive material is developed and dried, and prints P1, P2, and P3 are created. The print P1 is a service size, a high-vision size, a panorama size, etc., the print P2 is an A4 size print, and the print P3 is a business card size print.

  On the other hand, when recording image data on a recording medium, the image processing unit 70 outputs the image data subjected to image processing to the image writing unit 15, and the image writing unit 15 receives the input image data as each adapter 15a. To the recording medium inserted in .about.15c.

  The control unit 7 includes an image processing unit 70 therein, and performs data processing on the image data input by the image processing unit 70. In addition, the control unit 7 reads out the image composition processing program stored in the data storage unit 71 and causes the image processing unit 70 to execute a composite image process (see FIG. 4) described later according to the composite image processing program.

FIG. 3 shows an internal configuration of the image processing unit 70.
As shown in FIG. 3, the image processing unit 70 includes an image data processing unit 70a, a film scan data processing unit 70b, a reflection original scan data processing unit 70c, an image data format decoding processing unit 70d, a CRT specific processing unit 70f, and a printer specific. A processing unit 70g and an image data format creation processing unit 70h are provided.

  The image data processing unit 70a is optimal on various output media such as CRT8 and print paper for the image data input by the film scan data processing unit 70b, the reflection original scan data processing unit 70c, and the image data format decoding processing unit 70d. Image processing such as color conversion processing and gradation conversion processing so as to obtain a satisfactory image quality, and processed images to the CRT specific processing unit 70f, printer specific processing unit 70g, image data format creation processing unit 70h, and data storage unit 71 Send data. Various image processing is performed on the image data in accordance with an instruction input via the operation unit 11. Examples of various image processing include color conversion processing, gradation conversion processing, trimming processing, rotation processing, enlargement / reduction processing, and the like. It is done.

Further, the image data processing unit 70a executes an image composition process to be described later in accordance with an instruction from the control unit 7. In the image composition processing, data analysis of the first image input by the control unit 7 is performed to extract the image area of the main subject that is the composition target area. As the extraction method, the following known methods can be applied.
(1) A method of detecting discontinuity information of an image by a line process method and extracting a specific area based on the discontinuity information (Japanese Patent Laid-Open Nos. 2000-201358 and 2000-207569)
(2) An image feature amount obtained from a captured image that is an extraction target of the specific region is input to a neural network that is preliminarily constructed from the image feature amount obtained from the specific region as teacher data, and the obtained output value Method for extracting a specific region based on the method (Japanese Patent Laid-Open Nos. 7-234856 and 8-87589)
(3) Method of extracting partial parts and combining them to determine as a main subject region candidate (Japanese Patent Laid-Open No. 11-316845)
(4) A method of extracting an image region that is likely to be a region corresponding to the main subject based on the result of weighting the results of a plurality of extraction methods in which weighting factors are individually set based on the weighting factor (Japanese Patent Laid-Open No. Hei 9). No. 1387470)
Note that the extraction method mentioned here is an example, and other methods can be applied. For example, the image area of the main subject specified by the operator via the mouse 12 or the operation unit 11 may be extracted, or this may be used in combination with the extraction method described above as an auxiliary means. Further, extraction may be performed using main subject position information, main subject region information, and the like included in tag information represented by Exif.

  A defocus area is extracted from the first image. The defocus area is a band-like area within a predetermined range from the main subject near the boundary between the main subject and the background thereof, that is, around the main subject. When the defocus area is extracted, the second image to be combined input by the control unit 7 and the image of the main subject area are combined, and the defocus area extracted around the main subject area is combined, or A defocus process is performed based on the image information of the defocus area to create a composite image. Here, the defocus processing refers to processing that creates a non-sharp area around the main subject area, that is, a defocus area, in the composite image. As the defocus processing, for example, non-sharpening processing for reducing sharpness, processing for mixing image information of unsharp areas at a predetermined ratio, and the like can be applied.

  The film scan data processing unit 70b performs a calibration operation specific to the film scanner unit 9, negative / positive reversal in the case of a negative document, dust flaw removal, gray balance adjustment, contrast adjustment, and granular noise on the image data input from the film scanner unit 9. A person who performs various image processing such as removal and sharpening enhancement. In addition, the film size, the negative / positive type, the information related to the main subject, the information related to the shooting conditions (for example, the description information content of APS), etc. recorded on the film are acquired and output to the image data processing unit 70a together with the image data.

  The reflection original scan data processing unit 70c performs a calibration operation unique to the reflection original input device 10, negative / positive reversal in the case of a negative original, dust flaw removal, gray balance adjustment, and contrast adjustment for the image data input by the reflection original input device 10. Image processing such as noise removal and sharpening enhancement is performed, and the image data subjected to the image processing is output to the image data processing unit 70a.

  The image data format decoding processing unit 70d performs, for the image data input by the image reading unit 14 or the communication unit 32, decompression of the compression code and conversion of the color signal expression method according to the data format, if necessary. The data is converted into a data format suitable for calculation in the image processing unit 70 and output to the image data processing unit 70a. Further, when shooting information such as shooting conditions is acquired from header information represented by, for example, Exif standard, tag information, or the like of image data, the image data is output to the image data processing unit 70a together with the shooting information.

  Although the designation about the size of the output image is input from the operation unit 11, the designation information about the size of the output image is received via the communication unit 32, or the image reading unit 14 acquires the designation. When the designation information about the size of the output image is included in the header information and tag information of the image data, the image data format decoding processing unit 70d detects the designation information and outputs it to the image data processing unit 70a.

  The CRT specific processing unit 70f performs processing such as changing the number of pixels, color matching, and brightness adjustment as necessary on the image data received from the image data processing unit 70b, and synthesizes the information with information that needs to be displayed such as control information. The displayed signal is sent to the CRT 8.

  The printer-specific processing unit 70f performs exposure processing for the image data input from the image data processing unit 70a as necessary by performing printer-specific calibration processing, color matching, pixel number change, and the like. It is sent to the exposure processing unit 4. When another external printer device such as an ink jet printer is further connected to the print creation system 1, a printer specific processing unit 70f is provided for each connected printer device, and an appropriate calibration process or color corresponding to each device characteristic is provided. Matching, changing the number of pixels, etc. are performed.

  The image data format creation processing unit 70h converts the image data input from the image data processing unit 70a into various general-purpose image formats typified by JPEG, TIFF, Exif, and the like as necessary. The image data is output to the writing unit 15 and the communication unit 32.

  The film scan data processing unit 70a, the reflection original scan data processing unit 70c, the image data format decoding processing unit 70d, the image data processing unit 70a, the CRT specific processing unit 70f, the printer specific processing unit 70g, and the image data format creation processing unit 70h. The configuration unit indicates a functional configuration of the image processing unit 70 of the present embodiment, and is not necessarily realized as a physically independent device. For example, a processing step in software processing in a single CPU It may be realized as a section of

Next, the operation in the present embodiment will be described.
FIG. 4 is a flowchart for explaining image composition processing executed by the print creation system 1. In the following description, an example in which the image area of the main subject included in the first image is extracted as the composition target area and the image area of the main subject is combined with the background image that is the second image will be described.

  In the image composition processing shown in FIG. 4, first, data of a first image to be synthesized is acquired from the data storage unit 71 (step S1). The first image is an image including an image area of a main subject (hereinafter referred to as a main subject area). When the first image is acquired, the image area of the main subject is extracted from the first image (step S2). The extracted main subject area is an area surrounded by an edge portion with high sharpness.

  Next, a defocus area is extracted from the first image (step S3). The defocus area refers to a band-like area within a predetermined range from the accommodated subject in the vicinity of the boundary between the main subject and its background, that is, around the main subject. This area has low sharpness, and visually, the vicinity of the edge of the main subject appears to be blurred.

As a method for extracting the defocus area, a publicly known method can be applied, but the following describes four methods with specific examples.
(1) A method of extracting an image area within a predetermined range from a main subject area as a defocus area.
In this method, for example, as shown in FIG. 5, when the first image a1 is acquired and the main subject area a2 is extracted from the first image a1, for example, three pixels from the edge portion of the main subject area a2, etc. A region a3 within a predetermined range surrounding the main subject region a2 is extracted as a defocus region.

(2) A method of extracting a difference between a main subject area extracted under a specified extraction condition and an allowable area extracted by relaxing the specified extraction condition as a defocus area.
In this method, for example, as shown in FIG. 6, when the first image b1 is acquired and the main subject region b3 is extracted from the first image b1 with the specified extraction condition, the specified subject from which the main subject region b3 is extracted. The allowable area b2 is extracted from the first image b under the extraction condition that relaxes the extraction condition. The extracted allowable area b2 is estimated to have a larger area range than the main subject area b3 because the extraction conditions are relaxed. Next, a difference between the extracted shared area b2 and main subject area b3 is obtained as a defocus area b4.

  For example, in the line process method, a color difference between adjacent pixels is detected, and if the color difference exceeds a predetermined range, it is determined as a discontinuous boundary portion having different color information, and the discontinuous portion is connected. This is a technique for detecting an area as a specific area having the same color information. For example, if a specified extraction condition by the line process method has a color difference of ± 5% or less between adjacent pixels, it is determined as an area having the same color information. If the condition is, the main subject area is extracted under this default condition, and if the color difference is within ± 10%, it is determined as an area having the same color information. Extract regions.

  Note that the main subject region is extracted from the same first image by applying different extraction methods, the region extracted by one of the extraction methods is the first main subject region, and the other is the second main subject region. Then, the difference between the first main subject area and the second main subject area may be obtained as the defocus area.

(3) A method of extracting a background region of the main subject region and extracting a region not extracted as a background region around the main subject region as a defocus region.
In this method, for example, as shown in FIG. 7, when the first image c1 is acquired and the main subject region c3 is extracted from the first image c1 by, for example, the line process method, the first image is similarly obtained by the line process method. The background area c2 is extracted from c1. Since only the areas with high sharpness are extracted from the background area and the main subject area, the boundary area between the main subject area and the background area is unsharp and the images of the background area and the main subject area are gradually mixed. Is not extracted as a background area or a main subject area. Accordingly, an area obtained by subtracting the background area c2 and the main subject area c3 from the first image c1 is obtained as the defocus area c4.

(4) A method of extracting an area designated by the operator as a defocus area.
In this method, for example, as shown in FIG. 8, when the first image d1 is acquired, a pointer d2 for designating a defocus area is displayed on the CRT 8 together with the first image d1. When the pointer d2 is operated by the operator via the mouse 12 and a defocus area around the main subject area is designated, the designated area is extracted as a defocus area. Instead of leaving everything to the operator's designation operation, the area information designated by the operator may be combined with the methods as described in the above (1) to (3) as auxiliary information.

  The defocus area extraction method has been described above with specific examples. However, other extraction methods may be used, and the method is not particularly limited to the above method.

  When the defocus area is extracted in this way, data of the background image that is the second image to be combined is acquired from the data storage unit 71 (step S4). Next, a composite image in which the main subject region and the defocus region are combined with the acquired second image is created (step S5), and this process ends.

As a synthesis method, publicly known and publicly available methods can be applied, but three methods will be described below with specific examples.
(1) A method of arranging and synthesizing images of the main subject area and the defocus area at designated positions on the second image.
In this method, for example, as shown in FIG. 9, when the second image e1 is acquired and the main subject area e2 and the defocus area e3 are extracted, the operator arranges the main subject area e2 on the second image e1. When the position is designated, the main subject area e2 is arranged at the designated position, the defocus area e3 is arranged in the peripheral area of the main subject area e2, and the composite image e4 is created.

(2) A method of synthesizing an image of the main subject area with an image obtained by unsharpening (defocusing processing) an image area corresponding to the defocus area in the second image.
In this method, for example, as shown in FIG. 10, when the second image f1 is acquired and the main subject region f2 and the defocus region f3 are extracted, the operator first arranges the main subject region f2 in the second image f1. When the position is designated, the position of the defocus area f3 is determined according to the position of the designated main subject area f2. Next, in the second image f4 in which the position of the defocus area f3 is determined, the image area of the second image corresponding to the position of the defocus area f3 is unsharpened. Next, the main subject area f2 is arranged at a position designated in advance in the background processed image f5 in which the area corresponding to the defocus area f3 is unsharpened, and a composite image f6 is created. That is, the defocus area is not pasted and synthesized as it is, but only the area information is used.

  In the synthesis method of (2) above, the frequency characteristic of the defocus area extracted from the first image is obtained, and the defocus area in the second image is set to have the same frequency characteristic as the frequency characteristic of the defocus area It is also possible to unsharp the image area corresponding to.

(3) A method of mixing the second image and the image of the defocus area at a predetermined ratio in the image area corresponding to the defocus area in the second image (defocus processing), and synthesizing the main subject area image with the image .
In this method, as shown in FIG. 11, first, when the arrangement position of the main subject region g2 is designated by the operator in the second image g1, the defocus region g3 is changed according to the designated position of the main subject region g2. The position is determined. Next, in the second image g4 in which the position of the defocus area g3 is determined, the pixel information (RGB information) of the image area of the second image corresponding to the position of the defocus area g3, the pixel information of the defocus area g3, Are mixed by 1: 1, that is, an average is obtained, and the average value is set in the image area of the second image corresponding to the defocus area g3. Next, the main subject region g2 is arranged at a position designated in advance in the background processed image g4 in which the pixel information is mixed at a ratio of 1: 1, and a composite image g5 is created.

  In addition, although the example in which the pixel information of the second image and the defocus area image is mixed at a 1: 1 mixing ratio has been described here, the mixing ratio is not limited to this. Further, instead of mixing pixel information, the pixels of the second image and the defocus area image may be mixed at a predetermined ratio.

  While the synthesis method has been described with specific examples, other synthesis methods may be used, and the method is not particularly limited to the above method.

  The generated composite image data is stored in the data storage unit 71, and prints of the composite image, data writing to a recording medium, or the like is performed in accordance with an instruction.

  As described above, in the present embodiment, when creating a composite image using the first image and the second image, the main subject area to be synthesized is extracted from the first image, and the area around the main subject area is extracted. A defocus area is also extracted, and a composite image of the main subject area image and the second image is created using the information on the defocus area, so that the focus is on the periphery of the main subject area image with high sharpness. Unsharp areas can be created and the blurring feeling of the first image can be transplanted.

  Therefore, the continuity of the boundary between the main subject region and the background image can be maintained in the composite image, and a composite image with a natural impression can be created. In addition, since the background image is not unsharpened more than necessary, the image quality of the original image to be synthesized can be maintained.

  Note that the description in this embodiment is a preferred example of a print creation system to which the present invention is applied, and is not limited to this.

  For example, in the above-described example, the example in which the image of the main subject and the background image are combined has been described. However, the image of the region to be combined is not particularly limited. For example, the image of the main subject and the image of the main subject are combined. Alternatively, the component in the image and the background image may be synthesized.

  In addition, the detailed configuration and detailed operation of the print creation system in the present embodiment can be changed as appropriate without departing from the spirit of the present invention.

1 is an external view of a print creation system 1 in the present embodiment. 1 is a diagram illustrating a functional configuration of a print creation system 1. FIG. 2 is a diagram illustrating a functional configuration of an image processing unit 70. FIG. 6 is a flowchart illustrating image composition processing executed by the print creation system. It is a figure which shows an example of the extraction method of a defocus area | region. It is a figure which shows an example of the extraction method of a defocus area | region. It is a figure which shows an example of the extraction method of a defocus area | region. It is a figure which shows an example of the extraction method of a defocus area | region. It is a figure which shows an example of the synthetic | combination method. It is a figure which shows an example of the synthetic | combination method. It is a figure which shows an example of the synthetic | combination method.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Print production system 2 Main body part 4 Exposure processing part 5 Print production part 7 Control part 70 Image processing part 71 Data storage part 70a Image data processing part 8 CRT
DESCRIPTION OF SYMBOLS 9 Film scanner part 10 Reflective original input device 11 Operation part 12 Mouse 14 Image reading part 15 Image writing part 32 Communication part

Claims (27)

Image acquisition means for acquiring a first image and a second image to be combined;
A synthesis target area extracting means for extracting a synthesis target area from the acquired first image;
Defocus area extracting means for extracting a defocus area related to the extracted synthesis target area from the first image;
Combining processing means for generating a combined image using the extracted combining target region, the defocused region, and the acquired second image;
An image processing apparatus comprising:   The defocus area extracting unit extracts, as a defocus area, an image area within a predetermined range from an edge portion of the synthesis target area around the synthesis target area extracted by the synthesis target area extraction unit. The image processing apparatus according to claim 1.   The defocus area extraction means includes a composition target area extracted by the composition target area extraction means under a prescribed extraction condition, and an allowable area extracted from the first image by an extraction condition that relaxes the prescribed extraction condition. The image processing apparatus according to claim 1, wherein the difference is extracted as a defocus area.   The defocus area extraction unit extracts a background area of a synthesis target area from the first image, and extracts an image area obtained by subtracting the background area and the synthesis target area from the first image area as a defocus area. The image processing apparatus according to claim 1, wherein: An operation means for designating an image area in the first image;
The image processing apparatus according to claim 1, wherein the defocus area extracting unit extracts an image area designated through the operation unit as a defocus area.   6. The composition processing unit according to claim 1, wherein the composition processing unit creates a composite image in which images of the composition target area and the defocus area are arranged at positions designated on the second image. The image processing apparatus according to item.   The synthesis processing means performs a defocus process on a second image area corresponding to a defocus area image whose position is specified on the second image, and the second image area surrounded by the defocused area is subjected to the defocus process. The image processing apparatus according to claim 1, wherein a composite image is created by arranging composite region images. The defocus processing is unsharpening processing,
The synthesis processing unit calculates a frequency characteristic of the defocus area, and determines a degree of unsharpening by the unsharpening process according to the calculated frequency characteristic in the defocus area. Item 8. The image processing device according to Item 7.   The image processing apparatus according to claim 7, wherein the defocus process is a process of mixing the second image and the defocus area image at a predetermined ratio. An image acquisition step of acquiring a first image and a second image to be combined;
A synthesis target area extracting step of extracting a synthesis target area from the acquired first image;
A defocus area extracting step of extracting a defocus area related to the extracted synthesis target area from the first image;
A synthesis processing step of creating a synthesized image using the extracted synthesis target area, the defocus area, and the acquired second image;
An image processing method comprising:   In the defocus area extraction step, an image area within a predetermined range is extracted as a defocus area from an edge portion of the synthesis target area around the synthesis target area extracted in the synthesis target area extraction step. The image processing method according to claim 10.   In the defocus area extraction step, a synthesis target area extracted under a specified extraction condition in the synthesis target area extraction step and an allowable area extracted from the first image under an extraction condition that relaxes the specified extraction condition The image processing method according to claim 10, wherein the difference is extracted as a defocus area.   In the defocus area extraction step, a background area of a synthesis target area is extracted from the first image, and an image area obtained by subtracting the background area and the synthesis target area from the first image area is extracted as a defocus area. The image processing method according to claim 10.   The image processing method according to claim 10, wherein, in the defocus area extraction step, an image area designated via an operation unit is extracted as a defocus area.   15. The composition process according to claim 10, wherein the composition processing step creates a composite image in which the image of the composition target area and the defocus area is arranged at a position designated on the second image. The image processing method according to item.   In the composition processing step, a defocus process is performed on the second image area corresponding to the defocus area image whose position is specified on the second image, and the second image area surrounded by the defocused area is The image processing method according to claim 10, wherein a composite image is created by arranging composite region images. The defocus processing is unsharpening processing,
In the synthesis processing step, a frequency characteristic of the defocus area is calculated, and a degree of unsharpening by the unsharpening process is determined according to the calculated frequency characteristic in the defocus area. Item 17. The image processing method according to Item 16.   The image processing method according to claim 16, wherein the defocusing process is a process of mixing the second image and the defocused area image at a predetermined ratio. On the computer,
An image acquisition function for acquiring a first image and a second image to be combined;
A synthesis target area extraction function for extracting a synthesis target area from the acquired first image;
A defocus area extraction function for extracting a defocus area related to the extracted synthesis target area from the first image;
A composition processing function for creating a composite image using the extracted composition target region, the defocus region, and the acquired second image;
An image processing program for realizing   The defocus area extraction function extracts an image area within a predetermined range from an edge portion of the synthesis target area as a defocus area around the synthesis target area extracted by the synthesis target area extraction function. The image processing program according to claim 19.   The defocus area extraction function includes: a synthesis target area extracted under a specified extraction condition by the synthesis target area extraction function; and an allowable area extracted from the first image according to an extraction condition that relaxes the specified extraction condition. The image processing program according to claim 19, wherein the difference is extracted as a defocus area.   The defocus area extraction function extracts a background area of a synthesis target area from the first image, and extracts an image area obtained by subtracting the background area and the synthesis target area from the first image area as a defocus area. The image processing program according to claim 19, wherein:   The image processing program according to claim 19, wherein the defocus area extraction function extracts an image area designated via an operation unit as a defocus area.   The composite processing function creates a composite image in which images of the composition target region and the defocus region are arranged at positions designated on the second image. The image processing program according to item.   The composition processing function performs a defocus process on a second image area corresponding to a defocus area image whose position is specified on a second image, and applies the defocus process to the second image area surrounded by the defocused area. The image processing program according to any one of claims 19 to 23, wherein a composition target region image is arranged to create a composite image. The defocus processing is unsharpening processing,
The composition processing function calculates a frequency characteristic of the defocus area, and determines a degree of unsharpening by the unsharpening process according to the calculated frequency characteristic in the defocus area. Item 26. The image processing program according to Item 25.   The image processing program according to claim 25, wherein the defocusing process is a process of mixing the second image and the defocused area image at a predetermined ratio.

Images