JP2011210204A - Difference image generation device, program and difference image generation method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To certainly segment only an object desired by a user.SOLUTION: When a set of images composed of a first image P1(d) and a second image P2 (a) is selected, the second image P2 is displayed on a monitor 4 as shown in (a). If a touch point by a pen of a user on the monitor 4 is detected, the touch point T is displayed on the monitor 4 where the second image P2 is displayed as shown in (b). Furthermore, a coupled line L2 obtained by sequentially coupling adjacent touch points T around a main object C which the second image P1 has is displayed as shown in (c). Block definition is performed on an image part PP2 surrounded by the coupled line L2 of the second image P2 shown in (c) and the image part PP2 surrounded by a coupled line L1 at the same position and in the same shape as those of the coupled line L2 of the first image P1 shown in (d). A difference image RP is segmented in divided images PP1, PP2 in the defined block (e).

Description

  The present invention relates to a difference image generation apparatus, a program, and a difference image generation method for generating a difference image between a photographed first image and a second image.

  Conventionally, a technique has been devised for cutting out a difference image between a first image taken together and a second image in which a subject to be cut out exists in the first image (see Patent Document 1). If this technique is used, a first image of only the background is captured, and then a second image in which the main subject to be extracted exists in the background is generated, thereby generating an image in which only the main subject is cut out. Can do.

Japanese Patent Laid-Open No. 10-21408

  However, the background at the time of shooting is not necessarily fixed, and there may be a moving subject such as a tree that swings in the wind or a clock with a minute hand in the background. In this case, when the second image in which the main subject is present in the background is taken after taking the first image of only the background, not only the main subject but also a tree swung by the wind or rotated. A moving background subject such as a minute hand exists as a difference image between the two images.

  Therefore, when the difference image is cut out from the first image and the second image, the moving background subject is cut out together with the main subject, and there is a problem that it is not possible to cut out only the target main subject.

  The present invention has been made in view of such conventional problems, and an object of the present invention is to provide a differential image generation device, a program, and a differential image generation method that can reliably cut out only a subject desired by a user in an image. To do.

  In order to solve the above-mentioned problem, the invention according to claim 1 is characterized in that a first image captured together and an acquisition unit that acquires a second image in which a subject to be cut out exists in the first image. , Area specifying means for specifying an area based on the second image, and clipping means for cutting out a difference image between the first image and the second image in the area specified by the area specifying means. It is characterized by providing.

  According to a second aspect of the present invention, in the first aspect of the invention, the area designating unit enlarges the first image and the second image at the same magnification, and the angle of view at the time of photographing. Image processing means for generating a first enlarged image and a second enlarged image having a smaller angle of view, and at the time of photographing the first enlarged image and the second enlarged image generated by the image processing means Designating means for designating, as the area, an angle of view smaller than the angle of view of the first and second enlarged images in the area designated by the designating means. And a difference image is cut out.

  According to a third aspect of the present invention, in the second aspect of the present invention, a display unit that displays the second image on a screen and an instruction unit that instructs the image processing unit to perform the enlargement process in response to an operation. And a display control means for enlarging the second image displayed on the screen in response to an instruction from the instruction means.

  According to a fourth aspect of the invention, there is provided the display device according to the first aspect, further comprising display means for displaying the second image on a screen, wherein the area designating means is the screen on which the second image is displayed. The region is designated in accordance with a designation operation for.

  According to a fifth aspect of the present invention, in the fourth aspect of the invention, the area designating unit designates a plurality of points on the screen of the display unit on which the second image is displayed. It is characterized by comprising: designation means; and second designation means for designating a range surrounded by a line sequentially connecting a plurality of points designated by the first designation means as the region.

  The invention described in claim 6 is an acquisition means for acquiring a first image taken together with a computer of the device and a second image in which a subject to be cut out exists in the first image, An area designation unit that designates an area based on the second image, and a cutout unit that cuts out a difference image between the first image and the second image in the area designated by the area designation unit.

  According to a seventh aspect of the present invention, there is provided an acquisition step of acquiring a first image taken together and a second image in which a subject to be cut out exists in the first image, and the second image An area designating step for designating an area based on the area, and a step of cutting out a difference image between the first image and the second image in the area designated by the area designating step. .

  According to the present invention, it is possible to reliably cut out only a subject desired by a user in an image.

1 is a block diagram of an electronic camera to which an embodiment of the present invention is applied. It is a flowchart which shows the process sequence of 1st Embodiment. It is explanatory drawing which shows the process sequence in this Embodiment. It is a flowchart which shows the process sequence of 2nd Embodiment. It is explanatory drawing which shows the process sequence in this Embodiment.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a circuit block diagram showing an electrical configuration of a digital camera 1 common to each embodiment of the present invention. The digital camera 1 has general functions such as AE (Auto Exposure), AWB (Auto White Balance), and AF (Auto Focus). That is, the lens block 11 includes an optical system such as a zoom lens 111 and a focus lens (not shown), and a drive mechanism for driving the optical system. The zoom lens 111 is driven in the optical axis direction by a zoom motor 12 provided in a drive mechanism.

  A motor driver 16 is connected to a central processing unit (CPU) 13 that controls the entire digital camera 1 via a bus 14 and a timing generator (TG) 15. The motor driver 16 drives the zoom motor 12 based on a timing signal generated by the timing generator 15 in accordance with an instruction from the CPU 13 according to a user operation. As a result, the zoom lens 111 is driven to change its lens position.

  At this time, the CPU 13 stores and manages instructions generated by itself. Therefore, the CPU 13 can acquire and manage the lens position of the zoom lens 111 that changes according to the timing signal generated by the timing generator 15 according to the command generated by the CPU 13 by storing and managing the command generated by the CPU 13. it can.

  Although not shown in the drawings, actually, a focus motor and a motor driver for driving the focus lens, a mechanical shutter, a mechanical aperture, and a drive mechanism having a motor for driving them are provided.

  The digital camera 1 also has a CCD (Charge Coupled Device) 18 as an image sensor. The CCD 18 is disposed on the optical axis of the lens block 11. The subject is imaged on the light receiving surface of the CCD 18 by the lens block 11. The CCD 18 is driven by a vertical and horizontal driver 19 based on a timing signal generated by the timing generator 15 in accordance with a command from the CPU 13, and outputs an analog imaging signal corresponding to the optical image of the subject to the unit circuit 20. The unit circuit 20 is a CDS (Correllated Double Sampling) circuit that removes noise included in the output signal of the CCD 18 by correlated double sampling, or an A / D (Analog / Digital) that converts an imaging signal from which noise has been removed to a digital signal. ) Consists of a converter and the like, and outputs a digital signal to a DSP (Digital Signal Processor) 21.

  The DSP 21 performs image processing such as pedestal clamp and auto white balance on the digital signal output from the unit circuit 20 to generate image data including a luminance (Y) signal and a color difference (UV) signal. The image data converted by the DSP 21 is sequentially stored in an SDRAM (Synchronous Dynamic Random Access Memory) 22 and a video signal every time one frame of data (image data) is accumulated in the REC (Record) live view mode. And is sent to the liquid crystal monitor 4 and displayed on the screen as a live view image.

  In the still image shooting mode, using the shutter button operation as a trigger, the CPU 13 switches the live view image shooting mode to the still image shooting mode for the CCD 18, the vertical and horizontal drivers 19, the unit circuit 20, and the DSP 21. The image data obtained by the shooting processing in the still image shooting mode and temporarily stored in the SDRAM 22 is compressed by the CPU 13 and finally recorded in the external memory 25 as a still image file of a predetermined format. The still image file recorded in the external memory 25 is read out and expanded by the CPU 13 in accordance with the user's selection operation in the PLAY mode, and expanded as image data on the SDRAM 22, and then an image based on the image data is displayed on the monitor 4. Is displayed.

  The flash memory 26 stores various programs for causing the CPU 13 to control the above-described units, for example, programs for controlling AE, AF, and AWB, and programs for executing processing shown in flowcharts to be described later. Has been.

  The digital camera 1 also has a key input unit 27 including a plurality of operation keys and switches such as a shutter button, a zoom button, and an OK button. When any operation key is operated by the user, the key input unit 27 supplies an operation signal corresponding to the operation content to the CPU 13.

  A touch panel 23 is stacked on the monitor 4. The touch panel 23 detects the touch position of the user on the monitor 4 and supplies a signal indicating the detected touch position to the CPU 13.

(First embodiment)
Next, the operation of the digital camera 1 according to the above configuration in the first embodiment will be described. FIG. 2 is a flowchart showing a processing procedure in the difference image generation mode in the present embodiment. When the difference image generation mode is set by the user's operation on the key input unit 27, the CPU 13 executes the processing shown in this flowchart according to the program.

  That is, the CPU 13 determines whether or not the shooting mode in the difference image generation mode is set (step S101). If the shooting mode is set, it is determined whether or not the first shutter button operation has been detected (step S102). If the first shutter button operation is detected, the photographing process is executed and the image is recorded as the first image (step S103). Specifically, the CPU 13 compresses the image data stored in the S10 DRAM 22 when the shutter button is pressed, and records the compressed image data in the external memory 25 as a first image.

  Next, it is determined whether or not a second shutter button operation has been detected (step S104). If the second shutter button operation is detected, the photographing process is executed and the image is recorded as the second image in the external memory 25 in association with the first image recorded in step S103 (step S105). ).

  Subsequently, it is determined whether or not the cut-out mode in the difference image generation mode has been shifted, and the processing in steps S101 to S106 is repeated until the shift is made. Therefore, during this period, the user captures the background by the first shutter button operation, and captures the image in which the main subject to be cut out in the background exists by the second shutter button operation. Multiple sets of images are recorded in the external memory 25 with the image and the second image as a set.

  When the user sets the extraction mode in the difference image generation mode to synthesize the difference image, the process proceeds from step S106 to step S107, and any one of the plurality of sets of images recorded in the external memory 25 is one set. It is determined whether an image is selected. If a set of images composed of the associated first image and second image is selected from the external memory 25, the second image in the selected set of images is displayed on the monitor 4 ( Step S108). Therefore, if a set of images consisting of the first image P1 shown in FIG. 3A and the second image P2 shown in FIG. 3B is selected, as shown in FIG. The second image P2 is displayed on the monitor 4.

  As shown in FIG. 3A, in this example, the first image P1 is an image composed of a background in which trees exist as background subjects A and B. As shown in FIG. 3B, the second image is an image in which a main subject C to be cut out is present in a background where similar trees exist as background subjects A and B.

  Next, the CPU 13 determines whether or not an operation of a zoom button provided in the key input unit 27 is detected (step S109). If the operation of the zoom button by the user is detected, the second image P2 is enlarged in the SDRAM 22 and the image displayed on the monitor 4 is enlarged (step S110). Subsequently, it is determined whether or not an OK button operation has been detected (step S111), and the processing from step S109 is repeated until an OK button operation is detected.

  Accordingly, whenever the user operates the zoom button during this time, the second image P2 is enlarged in the SDRAM 22, and the image displayed on the monitor 4 is enlarged as shown in FIG. Step S110). As a result, the background subjects A and B are outside the screen of the monitor 4 as shown in FIG. As a result, in the screen of the monitor 4, only the enlarged partial image P3 having a small angle of view including the main subject C and including the background subjects A and B is displayed.

  In this way, if the image P3 in which the main subject C exists except for the background subjects A and B is displayed on the monitor 4, the user operates the OK button. Then, the CPU 13 advances the processing from step S111 to step S112, and calculates the magnification of the second image, that is, the magnification of the enlarged partial image P3 being displayed.

  Here, the enlargement magnification is the magnification of the enlarged partial image P3 with the angle of view of the screen of the monitor 4 calculated using the second image P2 having the angle of view at the time of shooting as a reference (1 time).

  When the enlargement magnification is calculated, the first image P1 is enlarged in the SDRAM 22 with the calculated enlargement magnification (step S113). Therefore, if the first image P1 is also displayed on the monitor 4, the background subjects A and B are enlarged to a size outside the screen of the monitor 4. As a result, the SDRAM 22 stores the first image P1 and the second image P2 that have been enlarged at the same magnification.

  Subsequently, the CPU 13 designates a region within the angle of view displayed on the monitor 4 of the enlarged second image P2 and the corresponding angle of view of the enlarged first image (step S114). ). Then, a difference image between the first image P1 and the second image P2 that has been enlarged is cut out within the designated region (step S115).

  At this time, trees are present as background subjects A and B within the angle of view of the captured first image P1 and second image P2, as shown in FIGS. Therefore, after the first image P1 having only the background is captured and then the second image P2 in which the main subject C is present in the background is captured, if the background subjects A and B swing due to the wind, the second In this image P2, not only the main subject C but also background subjects A and B that have been swung by the wind exist as differential images in both images. Therefore, if the difference image is cut out using the first image P1 and the second image P2 as they are, not only the main subject C but also the changed portions of the background subjects A and B are cut out in the difference image. End up.

  However, in the present embodiment, the background subjects A and B are moved outside the area in accordance with the enlargement process, and then the difference image between the first image P1 and the second image P2 is cut out within the area ( Step S115). Therefore, even if there is a subject other than the main subject C in the first and second images P1 and P2, as shown in FIG. 3D, only the main subject C desired by the user is included. The difference image RP can be reliably cut out.

  In addition, in the present embodiment, the second image P2 is displayed on the monitor 4, and the image displayed on the monitor 4 is enlarged in accordance with the user's operation so that the area is designated. Therefore, the user can easily exclude unnecessary background subjects A and B from the cutout target area while viewing the image displayed on the monitor 4.

  When the difference image RP of the main subject C is cut out as described above, the difference image RP is recorded in the external memory 25 in association with or without association with the first and second images (step S116). . Therefore, the user can generate a composite image by reading out the differential image RP consisting only of the main subject C from the external memory 25 and pasting it on another background image at a later date.

(Second Embodiment)
FIG. 4 is a flowchart showing a processing procedure in the second embodiment of the present invention.
When the difference image generation mode is set by the user's operation on the key input unit 27, the CPU 13 executes the processing shown in this flowchart according to the program.

  That is, the CPU 13 determines whether or not the shooting mode in the difference image generation mode is set (step S201). If the shooting mode is set, it is determined whether or not the first shutter button operation has been detected (step S202). If the first shutter button operation is detected, the photographing process is executed and the image is recorded as the first image (step S203). Specifically, the CPU 13 compresses the image data stored in the S20 DRAM 22 when the shutter button is pressed, and records the compressed image data in the external memory 25 as a first image.

  Next, it is determined whether or not a second shutter button operation has been detected (step S204). If the second shutter button operation is detected, the photographing process is executed and the image is recorded as the second image in the external memory 25 in association with the first image recorded in step S203 (step S205). ).

  Subsequently, it is determined whether or not the cut-out mode in the difference image generation mode has been shifted, and the processes in steps S201 to S206 are repeated until the shift is made. Therefore, during this period, the user captures the background by the first shutter button operation, and captures the image in which the main subject to be cut out in the background exists by the second shutter button operation. Multiple sets of images are recorded in the external memory 25 with the image and the second image as a set.

  When the user sets the extraction mode in the difference image generation mode to synthesize the difference images, the process proceeds from step S206 to step S207, and any one of the plurality of sets of images recorded in the external memory 25 is one set. It is determined whether an image is selected. If a set of images composed of the associated first image and second image is selected from the external memory 25, the second image in the selected set of images is displayed on the monitor 4 ( Step S208). Therefore, if a set of images consisting of the first image P1 shown in FIG. 5D and the second image P2 shown in FIG. 5A is selected, as shown in FIG. The second image P2 is displayed on the monitor 4.

  As shown in FIG. 5D, in this example, the first image P1 is an image composed of a background in which trees exist as background subjects A and B. Further, as shown in FIG. 5A, the second image is an image in which a main subject C to be cut out is present in a background where similar trees exist as background subjects A and B.

  Next, the CPU 13 determines whether or not a point that is a touch position of the user's pen on the monitor 4 is detected by the touch panel 23 (step S209). If the touch point is detected, the touch point is displayed on the monitor 4 on which the second image P2 is displayed (step S210). Subsequently, it is determined whether or not an OK button operation has been detected (step S211), and the processing from step S209 is repeated until an OK button operation is detected.

  Therefore, during this time, when the user touches the main subject C around the main subject C so as to remove the background subjects A and B on the monitor 4 on which the second image P2 is displayed, the touch point display processing is performed in step S210 each time. Thus, a plurality of touch points T are displayed around the main subject C as shown in FIG.

  In this way, when a plurality of touch points T surrounding the main subject C excluding the background subjects A and B are displayed on the monitor 4, the user operates the OK button. Then, the CPU 13 advances the processing from step S211 to step S212, and displays the adjacent touch points on the monitor 4 by connecting them sequentially with a line (step S212).

  Therefore, as shown in FIG. 5C by the process in step S212, the monitor 4 has a connecting line L2 that sequentially connects adjacent touch points T around the main subject C included in the second image P2. Is displayed.

  Subsequently, the CPU 13 designates the area of the image portion surrounded by the connecting line of the second image and the corresponding image portion of the first image (step S213). Therefore, by the processing in step S213, as shown in FIG. 5C, the image portion PP2 surrounded by the connection line L2 of the second image P2, and the connection line as shown in FIG. The region of the image portion PP1 of the first image P1 surrounded by the connecting line L1 having the same position and the same shape as L2 is designated. Then, a difference image is cut out from the designated area, that is, from the partial image PP1 and the partial image PP2 (step S214).

  At this time, trees are present as background subjects A and B within the angle of view of the captured first image P1 and second image P2, as shown in FIGS. Therefore, after the first image P1 having only the background is captured and then the second image P2 in which the main subject C is present in the background is captured, if the background subjects A and B swing due to the wind, the second In this image P2, not only the main subject C but also background subjects A and B that have been swung by the wind exist as differential images in both images. Therefore, if the difference image is cut out using the first image P1 and the second image P2 as they are, not only the main subject C but also the changed portions of the background subjects A and B are cut out in the difference image. End up.

  However, in the present embodiment, in step S214, a difference image is cut out from the partial images PP1 and PP2 that are within the area designated by the user. Therefore, even if there is a subject other than the main subject C in the first and second images P1 and P2, as shown in FIG. 5E, only the main subject C desired by the user is included. The difference image RP can be reliably cut out.

  Moreover, in the present embodiment, the second image P2 is displayed on the monitor 4 and is displayed on the monitor 4 according to the user's operation so that the user can specify the area. Therefore, the user can easily exclude unnecessary background subjects A and B from the cutout target area while viewing the image displayed on the monitor 4.

  When the difference image RP of the main subject C is cut out as described above, the difference image RP is recorded in the external memory 25 in association with or without association with the first and second images (step S215). .

  In the present embodiment, the present invention is applied to a digital camera. However, the present invention is not limited to a digital camera, and may be applied to various devices having a shooting function such as a mobile phone. If it is a device capable of capturing from the outside the first image taken without the image and the second image in which the subject to be cut out exists in the first image, it should be applied to other electronic devices. Is possible.

1 Digital Camera 4 Monitor 11 Lens Block 12 Zoom Motor 13 CPU
18 CCD
19 Horizontal driver 20 Unit circuit 21 DSP
22 SDRAM
23 Touch Panel 25 External Memory 26 Flash Memory 27 Key Input Unit 111 Zoom Lens A Background Subject C Main Subject RP Difference Image T Touch Point

Claims (7)

Acquisition means for acquiring a first image taken together and a second image in which a subject to be cut out exists in the first image;
Area designating means for designating an area based on the second image;
Clipping means for cutting out a difference image between the first image and the second image in the area designated by the area designation means;
A difference image generation apparatus comprising: The area specifying means includes:
Image processing for enlarging the first image and the second image at the same magnification, and generating a first enlarged image and a second enlarged image having an angle of view smaller than the angle of view at the time of shooting Means,
Designating means for designating the first enlarged image generated by the image processing means and the second enlarged image within the angle of view smaller than the angle of view at the time of shooting as the region;
2. The difference image generation apparatus according to claim 1, wherein the cutout unit cuts out a difference image between the first enlarged image and the second enlarged image within an area designated by the designation unit. Display means for displaying the second image on a screen;
Instructing means for instructing the enlargement processing to the image processing means according to an operation
3. The difference image generating apparatus according to claim 2, further comprising display control means for enlarging the second image displayed on the screen in response to an instruction from the instruction means. Display means for displaying the second image on a screen;
The difference image generating apparatus according to claim 1, wherein the region specifying unit specifies the region in accordance with a specifying operation on the screen on which the second image is displayed. The area specifying means includes a first specifying means for specifying a plurality of points on the screen of the display means on which the second image is displayed;
5. The difference image generating apparatus according to claim 4, further comprising: a second specifying unit that specifies, as the region, a range surrounded by a line that sequentially connects a plurality of points specified by the first specifying unit. The computer that the device has
Acquisition means for acquiring a first image taken together and a second image in which a subject to be cut out exists in the first image;
Area designating means for designating an area based on the second image;
Cutting means for cutting out a difference image between the first image and the second image in the area designated by the area designating means;
A program characterized by functioning as An acquisition step of acquiring a first image taken together and a second image in which a subject to be cut out exists in the first image;
An area designating step of designating an area based on the second image;
A step of cutting out a difference image between the first image and the second image in the region specified by the region specifying step;
The difference image generation method characterized by including.

Images