JP2004173300A - Image pickup device and picked-up image working method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image pickup device and an image working method appropriate for cutting a desired portion in a picked-up image or pasting the cut-out image on the other image. <P>SOLUTION: The image pickup device is provided with an image pickup means for picking up an image of an object, a frame image storage means for storing a frame image to be composed of a picked-up image (A) which is picked up by the image pickup means, a frame image reproducing means for reproducing desired one of frame images (B) stored in the frame image storage means as a reproducing frame image, and a composition means for producing a synthetic image (C) composing the picked-up image and the reproducing frame image. <P>COPYRIGHT: (C)2004,JPO

Description

  The present invention relates to an image pickup apparatus and an image pickup image processing method for picking up an image of a subject, and is particularly suitable for application to a digital still camera, but is not limited thereto.

  2. Description of the Related Art In recent years, a digital camera has been developed which converts a subject image converted into an electrical signal by an image pickup device such as a CCD into digital data (hereinafter referred to as image data) and records it on a recording medium, and can reproduce the image as desired by a user. Recording / reproducing devices for personal computers and image input devices such as personal computers. Many of such digital cameras include a display device that displays a subject image at the time of shooting and also displays a reproduced image at the time of reproduction.

  In order to process the captured image data stored in the digital camera, the captured image data is stored in an information processing device such as a personal computer, and the stored captured image data is processed using an application program for image processing. Are doing.

  However, when processing captured image data using an existing application program for image processing, the processing operation is extremely complicated. For example, a desired portion in a captured image is cut out and combined with another image. I couldn't do it easily.

  An object of the present invention is to provide an imaging apparatus and an image processing method suitable for cutting out a desired portion in a captured image and pasting the cut-out image to another image.

  In order to achieve the above object, an image pickup apparatus according to a first aspect of the present invention includes an image pickup means for picking up an image of a subject, a frame image storage means for storing a frame image combined with a picked up image picked up by the image pickup means, and a frame image storage Frame image reproducing means for reproducing a desired one of the frame images stored in the means as a reproduced frame image, and synthesizing means for generating a synthesized image obtained by synthesizing the captured image and the reproduced frame image. Features.

  Further, the imaging apparatus according to the first aspect of the invention includes a captured image storage unit that stores a captured image captured by the imaging unit, and a desired image among the captured images stored in the captured image storage unit as a reproduced image. An image reproducing means for reproducing the image may be provided, and the synthesizing means may be configured to generate a synthetic image obtained by synthesizing the reproduced captured image reproduced by the captured image reproducing means and the reproduced frame image.

  Further, each of the imaging devices according to the first invention may further include a display unit that displays the composite image.

  Further, each of the imaging devices according to the first aspect of the invention may be configured to further include an adjusting unit that adjusts a relative position of the captured image in the composite image.

  Each of the imaging devices according to the first invention includes a mask image storage unit that stores a mask image that masks the captured image, and a generation unit that combines the captured image with the mask image to generate a frame image. You may make it provide further.

  Each of the imaging devices according to the first aspect of the present invention may further include a composite image storage unit that stores a composite image.

  Further, an imaging device according to a second aspect of the present invention is an imaging device for imaging a subject and outputting a captured image, a captured image storage device for storing a captured image output from the imaging device, and a pattern storage device for storing a plurality of patterns. Pattern selecting means for selecting a desired pattern from a plurality of patterns stored in the pattern storing means, and a captured image stored in the captured image storing means using the pattern selected by the pattern selecting means And a cut-out image generating means for generating a cut-out image by cutting out a corresponding part of the above, and an image synthesizing means for synthesizing the cut-out image generated by the cut-out image generating means with a desired image.

  Further, an imaging device according to a third aspect of the invention is an imaging device that captures an image of a subject and outputs a captured image, a captured image storage device that stores a captured image output from the imaging device, and a captured image storage device. Pattern generating means for generating a pattern from a stored image, pattern storing means for storing a pattern generated by the pattern generating means, and imaging stored in a captured image storing means using the pattern stored in the pattern storing means The image processing apparatus includes a cutout image generating unit that cuts out a corresponding portion of the image to generate a cutout image, and an image combining unit that combines the cutout image generated by the cutout image generating unit with a desired image.

  In the imaging apparatus according to the third aspect, the pattern generation unit generates a pattern by extracting a contour in the captured image.

  Further, in the imaging apparatus according to the third aspect of the present invention, the image capturing apparatus further includes a designating unit that designates a photographing distance, and the pattern generating unit extracts a pattern in the captured image by extracting an image portion corresponding to the photographing distance designated by the designating unit. May be generated.

  In the imaging device according to the third aspect, the pattern generation unit may generate the pattern by binarizing the captured image.

  In the imaging apparatus according to the third aspect, the pattern generation means may generate the pattern by calculating a luminance component of the captured image.

  According to a fourth aspect of the present invention, there is provided an image capturing apparatus, comprising: an image capturing unit configured to capture an image of a subject; a captured image storing unit configured to store a captured image output from the image capturing unit; a drawing data input unit configured to input drawing data; Pattern generation means for generating a pattern from input drawing data, pattern storage means for storing a pattern generated by the pattern generation means, and storage in the captured image storage means using the pattern stored in the pattern generation means Clipping image generating means for generating a clipped image by clipping a corresponding portion of the captured image, and image synthesizing means for synthesizing the clipped image generated by the clipped image generating means with a desired image. .

  Further, in the imaging device according to the fourth aspect, the drawing means performs drawing using a cursor or a pointing device.

  According to a fifth aspect of the present invention, there is provided an imaging apparatus for imaging a subject and outputting a captured image, a display unit for displaying a captured moving image output from the imaging unit, and an image clipping pattern storage unit for storing an image clipping pattern. Display control means for combining and displaying the image clipping pattern stored in the image clipping pattern storage means on the captured moving image displayed on the display device; key input means; A captured still image storage means for storing an image and a captured still image stored in the captured still image storage means when the key input means is operated while the captured moving image and the cutout image pattern are displayed on the display means by the display control means. And storage control means for storing

  In the imaging device according to the fifth aspect, the storage control unit generates a cutout still image by cutting out a corresponding portion of the shot still image using the image cutout pattern displayed on the display unit. Is stored in the captured still image storage means.

  Further, the imaging apparatus according to the fifth aspect of the present invention further includes a reproducing unit that reproduces the captured still image stored in the captured still image storage unit, and an image cutout pattern stored in the image cutout pattern storage unit. The image processing apparatus includes: a cutout image generation unit that cuts out a corresponding portion of the captured still image reproduced by the reproduction unit to generate a cutout image; and an image synthesis unit that synthesizes the cutout image generated by the cutout image generation unit with a desired image. You may do so.

  According to a sixth aspect of the invention, there is provided an imaging unit for imaging a subject and outputting a captured image, a captured still image storage unit for storing a captured still image output from the imaging unit, and displaying a captured moving image output from the imaging unit. Display means, pattern storage means for storing a pattern, and cutout image generation for generating a cutout image by cutting out a corresponding portion of the captured image stored in the captured image storage means using the pattern stored in the pattern storage means Means, display control means for synthesizing the clipped image generated by the clipped image generation means on the picked-up moving image displayed on the display device, and key input means; and imaging on the display means by the display control means. When the key input unit is operated while the moving image and the cutout image are displayed, a storage control unit that stores the captured still image in the captured still image storage unit. And having a, the.

  Further, in the imaging device according to the sixth aspect, the memory control unit stores the captured still image and the cutout image displayed on the display unit in the captured still image storage unit.

  Further, the imaging apparatus according to the sixth aspect of the present invention may further include a movement instruction unit for instructing a movement of the cutout image displayed on the captured moving image to a desired position in a display screen by a display control unit. You may.

  An imaging apparatus according to a seventh aspect of the present invention includes an imaging unit that captures an image of a subject and outputs a captured image, a captured image storage unit that stores a captured image output from the imaging unit, and a captured image stored in the captured image storage unit Display means for displaying a pattern, pattern storage means for storing a pattern, display control means for displaying a pattern stored in the pattern storage means on a captured image displayed on the display means, and display control means for displaying the image by the display control means. Moving means for moving a pattern displayed on an image to a desired position in a display screen, and cutout image generation for cutting out a corresponding portion of a captured image using the pattern moved by the moving means to generate a cutout image Means, and an image synthesizing means for synthesizing the clipped image generated by the clipped image generating means with a desired image.

  According to a ninth aspect of the present invention, there is provided a captured image processing method for capturing a subject to obtain a captured image, storing the captured image, generating a pattern from the stored captured image, and storing the pattern. And a step of generating a cutout image by cutting out a corresponding portion of the stored captured image using the stored pattern, and combining the generated cutout image with a desired image. And

  According to the imaging apparatus of the first aspect, a frame image having a frame is registered in advance in a storage unit such as a flash memory, and the captured image is combined with a desired one of the registered frame images. Thus, a composite image with a frame can be easily obtained. Further, since a frame image can be displayed as a guide frame at the time of photographing, the size of the image can be made substantially constant, and a photograph for certification or the like having a predetermined size can be easily photographed (claim 1).

  Further, the imaging device of the first aspect of the invention can be provided with a storage unit such as a flash memory for storing the captured image and a reproducing unit for reproducing the stored captured image. Composite processing can be easily performed (claim 2).

  Further, the imaging device of the first invention can be provided with a display means for displaying a composite image, so that the user can view the composite result in real time, and can perform a redo, a recording, or an external output. The determination as to whether or not to do so can be made easily and quickly (claim 3).

  In addition, the imaging device of the first aspect of the present invention can be provided with an adjusting means for adjusting the relative position of the captured image in the composite image, so that the position of the captured image and the frame image can be easily adjusted while looking at the display screen. (Claim 4).

  Further, the imaging apparatus according to the first aspect of the invention can include a mask image storage unit that stores a mask image, and a generation unit that generates a frame image by combining the captured image and the mask image. In addition to the previously registered frame images, the user can synthesize and register a desired frame image by himself / herself (Claim 5). Further, since the image pickup apparatus according to the first aspect of the present invention can be provided with a composite image storage unit for storing a composite image, a favorite composite image can be easily recorded and stored (claim 6).

  According to the imaging apparatus of the second aspect, the pattern is registered in advance in a storage means such as a flash memory, and a desired captured image is combined with a desired one in the registered pattern to cut out a combined portion. By generating the cutout image, a desired image can be easily obtained by combining the cutout image with the desired image. Thus, image processing such as simply pasting an image in the form of a certain pattern to another image can be performed by an extremely simple operation as compared with a conventional image processing apparatus such as a personal computer. Item 7).

  According to the imaging device of the third invention and the captured image processing method of the eighth invention, a pattern is generated from a storage image stored in a storage unit such as a flash memory by imaging, and a desired captured image and the generated pattern are generated. Can be synthesized to generate a cutout image in which a synthesized portion is cut out, and an image obtained by synthesizing the cutout image with a desired image can be easily obtained. This allows the user to take out a pattern from the captured image, synthesize it with a desired image, cut it out, and paste it into another image with extremely simple operations compared to a conventional image processing device. (Claims 8 and 22).

  Further, the image pickup apparatus of the third invention can be provided with a means for extracting a contour in a picked-up image and generating a pattern. In this case, the pattern can be easily extracted (claim 9).

  Further, in the imaging apparatus according to the third aspect of the present invention, since a designating means for designating a photographing distance can be provided, a pattern can be generated by extracting an image portion corresponding to the photographing distance (claim 10).

  Further, in the imaging device according to the third aspect of the present invention, since a binarized pattern of the captured image can be generated, mask processing when cutting out another image is simplified (claim 11).

  In the imaging device according to the third aspect, the pattern can be generated by calculating the luminance component of the captured image. This makes it possible to easily obtain a clipped image with a blurred boundary.

  An imaging apparatus according to a fourth aspect of the present invention generates a pattern by drawing input, combines a desired captured image with the generated pattern, generates a cutout image in which a combined portion is cut out, and combines the cutout image with a desired image. The obtained image can be easily obtained. With this, image processing such as combining a pattern drawn by the user himself with a desired image, cutting out the image, and pasting the image easily into another image can be performed with an extremely simple operation as compared with a conventional image processing apparatus. (Claim 13).

  In the imaging apparatus according to the fourth aspect of the present invention, since the drawing can be performed using the cursor or the pointing device, the input operation can be easily performed for the user. No feeling is produced (claim 14).

  An imaging apparatus according to a fifth aspect of the present invention combines and displays a through image displayed in an imaging mode and an image clipping pattern, and flashes a through image displayed as a moving image as a still image by performing a key operation at a desired timing. It can be stored in storage means such as a memory. That is, according to the present invention, since the image clipping pattern is displayed on the through image, it is possible to perform shooting by properly adjusting the angle to fit the subject desired by the user in the clipping pattern (claim 15). ).

  Further, in the imaging apparatus according to the fifth aspect of the present invention, a corresponding portion of the captured still image can be cut out using the image cutout pattern displayed at the time of shooting to generate / record a cutout still image. It is possible to obtain a new effect that the clipped image in a state where it fits well in the pattern can be stored, which is not provided in the above-described conventional image processing apparatus.

  Further, since means for reproducing an image stored in the image pickup apparatus of the fifth invention can be provided, a cut-out image is generated by cutting out a pattern storing the reproduced image, and the cut-out image is generated in a desired image. Can be easily obtained. This makes it possible to play back the recorded image after shooting and recording in a state in which the subject desired by the user fits well in the cutout pattern and to cut out using the cutout pattern. (Claim 17).

  In the imaging device according to the sixth aspect of the invention, the stored image can be cut out using the stored pattern to generate a cut-out image, and the cut-out image can be synthesized with the displayed through image. By adjusting the angle, the user can take a picture when the positional relationship between the displayed cut-out image and the through-the-lens image becomes a desired one. (Claim 18).

  Further, in the imaging apparatus according to the sixth aspect of the present invention, since the still image of the cut-out image and the through image which are synthesized and displayed at the time of shooting can be stored, the positional relationship between the cut-out image and the through image becomes desired. A composite image of the state can be recorded (claim 19).

  Further, in the imaging apparatus according to the sixth aspect, since the cut-out image displayed on the through image can be moved by the movement instructing means, the alignment at the time of synthesis can be easily performed (claim 20).

  The imaging apparatus of the seventh invention can move the stored pattern on the captured image displayed on the screen, so that a desired image portion is cut out by the pattern to generate a cutout image, and the desired image It is possible to easily obtain an image in which a clipped image is combined with a desired portion (claim 21).

<Circuit configuration example>
FIG. 1 is a block diagram illustrating a configuration example of a digital camera as an embodiment of an imaging apparatus according to the present invention. A digital camera 100 includes a lens block 41 connected to a camera body 42 via a connection unit 43. I have.

  The lens block 41 is configured to supply an imaging lens 1, a charge coupled device (CCD) 2, and an amplifier (digital data) to a timing generator.

  The timing generator 12 controls the drive circuit 5 via the terminal unit 32 to drive the CCD 2. The timing generator 12 also writes the image data supplied from the A / D conversion circuit 4 to the DRAM 13. Image data written to the DRAM 13 has yellow (Ye), cyan (Cy), and green (Gr) color components.

  In addition to an area for temporarily storing image data, the DRAM 13 has a work area (work area) for image processing when executing the image processing of the present invention. The image data written in the DRAM 13 is read out at a constant cycle and supplied to the signal generator 16 when one frame of image data is ready. The image data written in the DRAM 13 is compressed by the compression / expansion circuit 15 and written into the flash memory 14 (captured image storage means) and stored (recorded).

  In the flash memory 14, in addition to an area for recording image data of a photographed image, an area for a cutout image memory 14A (cutout image storage means) for recording a cutout image, and a mask pattern memory 14B (mask pattern memory) for recording a mask pattern Storage area). Further, the area for the cutout image memory 14A and the area for the mask pattern memory 14B may be provided in another recording medium, for example, a ROM or a memory card.

The signal generator 16 takes out one frame of image data from the DRAM 13 at a constant cycle, sequentially performs white balance adjustment and color calculation processing, and performs YUV data (luminance signal Y, color difference signals RY, BY). Is generated and stored in the VRAM 17.
The signal generator 16 also reads out the YUV data stored in the VRAM 17 and outputs it to the D / A conversion circuit 18, performs D / A conversion, and then outputs it to the liquid crystal display 20 via the amplifier 19 to display an image. The video signal output from the amplifier 19 can be output to an external device (not shown).

  With the above configuration, the liquid crystal display (LCD) 20 can display a subject image via the CCD 2 at the time of photographing (through image display) and can display a reproduced recorded image at the time of image reproduction. The subject image displayed at the time of shooting is displayed as a moving image through the imaging lens 1 (that is, a monitor image (hereinafter, referred to as a through image) according to the movement of the subject or the digital camera 100 is displayed. 20 can also be used as a finder.

  In FIG. 1, a transparent position detection sensor (not shown) may be provided on the screen of the liquid crystal display unit 20. As the position detection sensor, for example, a light detection element for detecting specific light or a touch electrode can be used. When a light detection element is used as the position detection sensor, the user can point on the screen using an input pen that emits specific light (for example, infrared light) as an input means, thereby detecting light at the pointed position. The device becomes active. The CPU 21 detects the coordinates of the light detecting element in the active state, and can know the position and range (shape) of the pointed image. When a touch electrode is used, when the user touches the screen with a touch pen or a fingertip, the electrode of the touched portion is turned on. The CPU 21 can detect the coordinates of the touched electrode that is turned on, and thereby know the position and range (shape) of the pointed image.

The CPU 21 executes various processes according to a program stored in the ROM 22 or the flash memory 14.
The RAM 23 appropriately stores programs and data necessary for the CPU 21 to execute various processes.
The key input unit 24 includes various input keys and buttons as shown in FIG. 2. When operated by the user, the key input unit 24 sends a corresponding key or button status signal to the CPU 21. In addition to the key input unit 24, a pointing device such as an input pen or a touch pen may be added.

  The I / O port 25 executes data input / output interface processing between the CPU 21 and an external device.

  Further, a distance measuring unit (not shown) that can measure a distance to a subject may be provided in the lens block 41 or the camera body 42. As the distance measuring means, besides mechanical means, for example, when the digital camera has an autofocus mechanism, the focal length f of the imaging lens 1 obtained by the autofocus operation, the distance a between the imaging lens 1 and the CCD 2, and It can be configured as a means for obtaining the distance b to the subject from the moving distance of the imaging lens 1. Further, the distance measuring means can be constituted by using an infrared light receiving / emitting device.

<Configuration example of key input unit>
FIG. 2 is a diagram showing a typical key or button arrangement example of the key input unit 24 provided in the digital camera to which the present invention is applied. In FIG. 2, the frame button 61 is operated by the user when reading out the cut-out image from the cut-out image memory 14A, and the move key 62 is operated when moving the cursor displayed on the liquid crystal display 20 in the up, down, left and right directions. The button 63 is operated when recording a captured image in the flash memory 14, and the play button 64 is used to store a stored image (a captured image, a pre-registered image, a monochrome image, and an IC card) recorded in the flash memory 14. This operation is performed to set a reproduction mode for reproducing an image or the like that is being reproduced.

  A record button 65 is operated when setting a recording mode for recording a captured image in the flash memory 14, a mask button 66 is operated when reproducing a mask pattern (image), and an up / down key 67 is operated on a liquid crystal display. The direct key 68 is operated when the cursor displayed on the cursor 20 is moved up or down, or when an instruction to start execution of a function specified by a program is given, and the direct key 68 is used to start drawing input using a cursor or a pointing device (eg, a touch pen). This operation is performed when the CPU 21 notifies the CPU 21 or the like.

<Processing mode>
FIG. 3 is an explanatory diagram showing a configuration example of the processing mode of the digital camera in FIG. 1. The processing mode of the digital camera is roughly classified into a normal shooting mode including a recording mode and a playback mode and a special shooting mode such as close-up shooting. Switching between the normal mode and the special photographing mode is performed by switching a processing mode switch (not shown).

  When the processing mode is switched to the normal mode, first, the mode transits to the recording mode. The recording mode includes an imaging mode for photographing a subject and displaying a through image, a through image combining mode, and a recording and saving mode for recording a desired through image or a combined image on a recording medium (the flash memory 14 in the embodiment). Contains.

  When the play button 64 is pressed in the normal mode, the mode is first shifted to the play mode. The reproduction mode includes a reproduction / display mode for reproducing and displaying a recorded image recorded in the flash memory 14, a recorded image composition mode, and image data (composited image data) transmitted to an external device via the I / O port 25. Is included in the output mode. It should be noted that the mode determination means (configured by a program in the embodiment) determines which mode has been changed by a key operation or button operation of the processing mode changeover switch and the key input unit 24 by the operator. The mode determining means checks the key and button state signals sent from the key input unit 24 to the CPU 21 and transits to the corresponding mode processing circuit or program.

<Operation of digital camera under each mode>
Next, the operation of the digital camera in FIG. 1 in each mode will be described.

[Operation under recording mode]
When the processing mode is set to the recording mode, first, the mode is changed to the imaging mode. In the image capturing mode, one frame of image data periodically taken in from the CCD 2 is displayed on the liquid crystal display 20 as a through image. When the mask button 66 or the frame button 61 is pressed in the imaging mode, a through image synthesis mode is set, and a desired image is used as a mask pattern, an image is synthesized and displayed with a desired mask pattern or a cutout image, and image synthesis is performed. be able to. When the up / down key 67 is pressed, a shooting guide frame can be displayed. In addition, when the release button 63 is pressed, the recording mode is set, and a desired through image or a combined image can be recorded in the flash memory 14.

(1) Display of Captured and Through Images When the user presses the record button 65 in the recording mode, the CPU 21 sends a photographing instruction signal to control the timing generator 12 to capture the subject image. When the timing generator 12 receives the photographing instruction signal from the CPU 21, it drives the drive circuit 5 via the terminal 32 to take in the image signal converted by the CCD 2. Since an optical image of a subject to which the user points the imaging lens 1 is formed on the CCD 2 via the imaging lens 1, the driving circuit 5 drives the CCD 2 to convert the image signal photoelectrically converted by the CCD 2 into an amplifier. The signal is input to the A / D conversion circuit 4 through the A / D conversion circuit 3. The A / D conversion circuit 4 A / D converts the input image signal into image data, and supplies the image data to the timing generator 12 via the terminal 31. The timing generator 12 stores the input image data (Ye, Cy, Gr) in the DRAM 13. The CPU 21 reads out the image data once stored in the DRAM 13 and supplies it to the signal generator 16.
The signal generator 16 adjusts the white balance of the image data, generates video data (YUV data) by encoding, and temporarily stores the video data (YUV data) in the VRAM 17. The signal generator 16 also reads out the video data drawn in the VRAM 17 and supplies the video data to the D / A conversion circuit 18.
The D / A conversion circuit 18 D / A converts the video data and outputs the video data to the liquid crystal display 20 via the amplifier 19 for display. In this way, an image (through image) monitored by the user via the imaging lens 1 is displayed on the liquid crystal display 20. The user can confirm the subject by looking at the image displayed on the liquid crystal display 20.
Note that video data for a through image displayed on the liquid crystal display 20 can be easily generated as described above. For example, since luminance data and chrominance data are generated using only the Ye component of the image data stored in the DRAM 13, an image that changes every moment through the imaging lens 1 (ie, , Through image) can be quickly updated and displayed on the liquid crystal display 20.

(2) Through image synthesis mode When the user presses the mask button 66 or the frame button 61 in the shooting mode, the camera enters the through image synthesis mode.
When the mask button 66 is pressed, a desired portion as a mask pattern is cut out from the through image and registered in the mask pattern memory 14B, or a desired portion of the through image is combined with a desired mask pattern to synthesize the desired pattern. Can be created. When the frame button 61 is pressed, a desired cut-out image recorded in the cut-out image memory 14A can be pasted on the through image and synthesized. The operation of the digital camera 100 in the through image synthesis mode will be described later (FIG. 4, Examples 1 to 3, 7, 8 to 10).

(3) Display of guide frame When the up / down key 67 is pressed in the image capturing mode, a guide frame for photographing such as a driver's license or a passport photograph can be displayed. The display operation of the guide frame will be described later (seventh embodiment).

(4) Recording and Saving of Image On the other hand, when the recording button is operated by operating the release button 63 in the recording mode, the CPU 21 causes the image data stored in the DRAM 13 (the combined image data of the combined image data combined in the image combining mode). Is supplied to the signal generator 16 to generate (encode) video data more accurately by using all of the Ye component, the Cy component, and the Gr component, and display the video data on the liquid crystal display 20 as a still image. Further, the CPU 21 supplies the video data to the compression / decompression circuit 15 to execute a compression process by, for example, a JPEG (Joint Photographic Experts Group) method. The compressed video data is recorded (written) in the flash memory 14.
At the time of recording and saving, information relating the mask pattern to the composite image and its background image (in this case, the recorded image) on the flash memory 14 (for example, the image number of the composite image and the background image, the type and image of the mask pattern) The numbers may be written in the link table 50 provided on the flash memory 14 in association with each other. The storage position of each image or mask pattern may be written instead of the image number.

[Operation in playback mode]
When the play button 64 is pressed in the normal mode, the mode is set to the play / display mode, in which the recorded image recorded in the flash memory 14 is read, reproduced, and displayed on the liquid crystal display 20. Further, each time the play button is pressed, the next recorded image is read and reproduced and displayed on the liquid crystal display 20, so that the user can obtain a desired reproduced image.
The display image may be fast-forwarded when the play button is kept pressed, and a fast-forward mode in which the image at that time is displayed when the finger is released from the play button may be added.
When the mask button 66 or the release button 63 is pressed in the reproduction / display mode, a recording image synthesizing mode is set, and a mask pattern is generated from a desired reproduction image in the recording image, or a desired mask pattern or a cutout image is superimposed on the recording image. It can be displayed or a desired cut-out image can be pasted and combined with the recorded image.

(1) Playback and Display of Recorded Image When the user operates the playback button 64 of the key input unit 24 to set the playback / display mode, the CPU 21 writes the video data of one image written in the flash memory 14. The data is read out and supplied to the compression / decompression circuit 15.
The compression / decompression circuit 15 decompresses the video data and supplies it to the signal generator 16.
The signal generator 16 adds a synchronization signal or the like to the received video data, draws the video data once in the VRAM 17, reads out the video data drawn in the VRAM 17, and supplies it to the D / A conversion circuit 18.
The D / A conversion circuit 18 performs D / A conversion on the video data, and outputs the video data to a liquid crystal display 20 via an amplifier 19 for display. Thus, the image recorded in the flash memory 14 can be displayed on the liquid crystal display 20. As described above, since this display image is an image captured by performing more accurate processing, unlike a through image, it is displayed as a clearer image than the through image.

(2) Synthesis of Recorded Image In the playback / display mode, when the mask button 66 or the release button 63 is pressed, the recording image is synthesized. When the mask button 66 is pressed, a desired mask pattern is generated from the recorded image as a mask pattern. It is possible to superimpose and display a desired pattern on a recorded image or paste and combine a desired recorded image and a desired pattern. When the release button 63 is pressed, the mask pattern generated from the recorded image is registered in the mask pattern memory 14B, the generated cutout image is registered in the cutout image memory 14A, and the composite image is registered in the flash memory 14. Can be performed. If the link table 50 is provided and information for associating the mask pattern or the cut-out image with the composite image and its background image is recorded, the information is updated at the time of recording and saving. The operation in the recording image synthesis mode will be described later (FIG. 4, Embodiments 4 to 6, Embodiments 9 and 10).

<Image processing means>
The image processing means 100 executes image processing in the normal mode. Although the image processing means 100 can be constituted by a hardware circuit, in this embodiment, the image processing means 100 is constituted by a program. Some of the modules of the image processing means 100 may be configured by a hardware circuit, and the other modules may be configured by a program. Further, each module of the image processing means 100 constituted by a program is recorded in the ROM 22 or the flash memory 14 and is executed and controlled by the CPU 21 under the control of the control program, thereby realizing various image processing of this embodiment. FIG. 4 is a block diagram illustrating a configuration example of the image processing unit 100. The image processing unit 100 includes a through-image synthesis processing unit 110, a recorded image synthesis processing unit 120, and a guide frame display processing unit 130.

[Through image synthesis processing means]
The through-image synthesis processing unit 110 includes a processing method determination unit 111 that determines an image processing method when the shooting mode is switched from the imaging mode to the through-image synthesis mode in the recording mode, and a mask that extracts a desired mask pattern from the through image and performs masking. A pattern generation / registration unit 112 to be registered in the pattern memory 14B, a through image and a desired mask pattern are superimposed and displayed, and a desired cutout image is generated from the superimposed and displayed through image and the desired pattern to generate a cutout image memory 14A. And a through-image synthesizing unit 114 that superimposes and displays a desired cut-out image on a through-image and performs a pasting-and-synthesizing process between the through-image and the desired cut-out image.

  The processing method determining unit 111 is activated when the mode determining unit transits to the live view image combining processing unit 110, and the CPU 21 determines the operation result (state signal) of the user received from the key input unit 24, and performs the operation of the user. If the result is that the mask button 66 is depressed, the pattern generation / registration means 112 is activated. If the frame button 61 is depressed, the cutout image generation / registration means 113 is activated. If the release button is depressed, the through-image synthesis means 114 is activated. to start.

  As an option, information registered in the link table 50 when the link table 50 is provided in the flash memory 14, such as an image number indicating the relationship between the background image and the composite image, and the relationship between the background image and the cutout image or the mask pattern. May be incorporated in the subsequent stage of the through-image synthesizing means.

  The pattern generation / registration unit 112 includes a binarized pattern generation unit 1121 and a luminance component processing pattern generation unit 1122. The pattern generation / registration unit 112 allows the user to find a desired subject image as a mask pattern in the imaging mode or to draw a pattern to be registered as a mask pattern on paper or a board (for example, FIG. 8A). When the user presses the mask button 66 at a desired angle while viewing the image, the processing method determination unit 111 starts the operation, the through image (pattern candidate image) is taken into the DRAM 13, and the binarization pattern generation unit 1121 is started. Then, the captured pattern candidate image is binarized and the binarized pattern (FIG. 8B) is displayed on the liquid crystal display 20. In this case, when the up / down key 67 is pressed, the luminance component processing pattern generating means 1122 is activated, the luminance component of the pattern candidate image is fetched with a predetermined bit (8 bits in the embodiment), and the luminance component is changed. The displayed pattern (FIG. 8 (B ′)) is displayed on the liquid crystal display 20. Here, when the release button 63 is pressed, the extracted pattern is registered (recorded) in the mask pattern memory 14B (Example 1).

  Further, the pattern generation / registration unit 112 may be configured to include a pattern generation unit 1125 that performs a contour extraction process of detecting and extracting a contour of a subject and extracting a pattern of a desired subject image as a mask pattern.

  Further, the user moves the cursor on the screen of the liquid crystal display unit 20 using the movement key 62, traces the contour of the desired subject image as a mask pattern, and includes a pattern generation unit 1125 for extracting the pattern. The generation / registration unit 112 may be configured. In FIG. 1, a transparent position detection sensor (not shown) is provided on the screen of the liquid crystal display unit 20 as input means in addition to the keys and buttons of the key input unit 24 shown in FIG. When a pointing device (not shown) or a finger can be used to specify the pattern, the pattern generating unit 1125 may be configured so that the contour of the subject image in the through image can be traced and extracted as a mask pattern. (Example 8).

The mask pattern to be registered is a figure, a pattern, or a frame (for example, a photographing frame indicating the size of an image, such as a photographing frame having a size of a photograph for a car driving license or a photographing frame having a size for a passport photograph). May be.
Various general subjects such as objects, animals, plants, buildings, signboards, and other attached facilities, landscapes, and natural phenomena may be used, as long as they can be photographed and can be extracted as mask patterns. Note that, for a plurality of subjects included in such a general image, a portion corresponding to the distance can be extracted at the time of reproduction to generate a mask pattern. In this case, at the time of shooting, the pattern generation / registration unit 112 is configured to include a distance data storage unit 1126 that obtains the distance from the distance measurement unit to the subject at a plurality of points in the shot image and stores the distance together with the image data (implementation). Example 9). It is desirable that the measured distance data be written in a link table 50 described later.

  Further, the pattern generation unit 1125 may be configured so that the user moves the cursor on the screen of the liquid crystal display unit 20 using the movement key 62 to draw a locus, and cuts out the drawn figure as a mask pattern. When a transparent position detection sensor (not shown) is provided on the screen of the liquid crystal display unit 20 and can be designated by a pointing device (not shown) such as a touch pen or a finger, the user However, the pattern generation unit 1125 may be configured to draw a graphic while pointing on the screen of the liquid crystal display unit 20 and cut out the drawn graphic as a mask pattern. When the position detection sensor is constituted by a contact type detection sensor such as a touch electrode, it is also possible to cut out a figure drawn by a user touching the screen with a finger (Example 10).

  The cutout image generation / registration unit 113 cuts out a desired portion of the through image with the desired mask pattern registered in the mask pattern memory 14B and registers (stores) the cutout image in the cutout image memory 14A (second embodiment).

  In this case, the user selects a desired one from the mask patterns displayed on the screen. When a desired portion of the through image is cut out using the selected mask pattern, the mask pattern and the through image are combined on a screen and displayed on the liquid crystal display 20 to position the through image (or the mask pattern). However, since such a screen display (hereinafter, referred to as a preview screen display) takes time, the cut-out image generation / registration unit 113 includes a display coordinate determination unit that performs preview display at high speed in order to speed up the preview display. (Example 11).

  The through-image synthesizing unit 114 combines the cut-out image registered in the cut-out image memory 14A with a desired image among the through-images in the imaging mode, and registers (records) it in the flash memory 14 (third embodiment). In this case, the user can move the camera to move the through image so that the cutout images overlap at a desired angle. Alternatively, the displayed cut-out image can be moved without moving the camera so as to overlap a desired portion on the through image.

  In this case, the user selects a desired one from the cutout images displayed on the screen. In addition, when the selected cutout image is embedded in a desired portion of the through image and combined, the cutout image and the through image are displayed on the screen and positioned. In order to speed up such preview screen display, the through-image synthesis 114 can be configured to include a display coordinate determination unit that performs preview display at high speed, similarly to the cut-out image generation / registration unit 113 (Example 11). ).

  Further, the through-image synthesizing unit 114 is configured to include a resolution determining unit that automatically determines the resolution when the synthesized image is compressed and registered in the flash memory 14 from the resolution of the cut-out image, the resolution of the through-image, and the like. (Example 12).

[Recorded image synthesis processing means]
The recording image synthesizing processing unit 120 extracts a mask pattern from a recording image, and a processing method determining unit 121 that determines an image processing method when the mode is switched from the reproduction / display mode to the recording image synthesis mode in the reproduction mode. And a mask pattern generating / registering means 122 for registering the desired pattern in the mask pattern memory 14B, and performing a process of generating a cutout image by superimposing and displaying a desired pattern on the recorded image and clipping the superimposed displayed image with a desired mask pattern. It includes a cut-out image generation / registration unit 123 and a recorded image synthesizing unit 124 that superimposes and displays a recorded image and a desired cut-out image and superimposes the desired cut-out image on the desired recorded image.
Optionally, information to be registered in the link table 50 when the link table 50 is provided in the flash memory 14, such as an image number indicating the relationship between the background image and the composite image, or the relationship between the background image and the cutout image or the mask pattern. May be incorporated in the subsequent stage of the recorded image synthesizing means 124.

  The processing method determining means 121 is activated when the mode determining means transits to the recorded image synthesizing processing means 120, and the CPU 21 determines the operation result (state signal) of the user received from the key input unit 24, and performs the operation of the user. If the result is that the mask button 66 is depressed, the pattern generation / registration means 122 is activated. If the frame button 61 is depressed, the cutout image generation / registration means 123 is activated. If the release button is depressed, the recorded image synthesis means 124 is activated. I do.

  The pattern generation / registration unit 122 includes a binarized pattern generation unit 1221 and a luminance component processing pattern generation unit 1222. Here, since the binarized pattern generation unit 1221 and the luminance component processing pattern generation unit 1222 have the same configuration and function as the above-described binarization pattern generation unit 1121 and luminance component processing pattern generation unit 1122, they are used as common modules. When the pattern generation / registration unit 112 or the pattern generation / registration unit 122 is activated, the process branches to the module, and returns to the pattern generation / registration unit 112 or the pattern generation / registration unit 122 as soon as the processing is completed. It is desirable to configure.

  The pattern generation / registration unit 122 performs processing by the processing method determination unit 121 when the user finds an image desired as a mask pattern from the recorded images reproduced and displayed in the reproduction / display mode and presses the mask button 66. When started, the image is written into the work area provided in the DRAM 13 and branched to the binarized pattern generating means 1221. When the processing is completed, the binarized pattern is returned and the extracted binarized pattern (FIG. 16C) is returned. The information is displayed on the liquid crystal display 20. In this case, when the up / down key 67 is pressed, the process branches to the luminance component processing pattern generating means 1222, and upon completion of the processing, returns to display the extracted pattern (FIG. 16C ') on the liquid crystal display 20. Here, when the release button 63 is pressed, the extracted pattern is registered (recorded) in the mask pattern memory 14B (Example 4).

Similarly to the above-described pattern generation / registration unit 112, the pattern generation unit 1225 includes a pattern generation unit 1225 that performs a contour extraction process of detecting and extracting a contour of a subject to extract a desired pattern of a subject image as a mask pattern. The generation / registration unit 122 may be configured. In addition, the pattern generation unit 1225 may be configured to generate a mask pattern by extracting a matching image portion based on stored distance data when a user specifies a distance during reproduction (embodiment). 9).
Similarly to the pattern generation / registration unit 112 described above, the user moves the cursor on the screen of the liquid crystal display unit 20 using the movement key 62 to draw a locus, and extracts the drawn figure as a mask pattern. You may comprise so that the pattern generation means 1225 may be included. Similarly, when a transparent position detection sensor is provided on the screen of the liquid crystal display unit 20 as an input means and is configured to be able to be designated by a pointing device, the user can use the screen of the liquid crystal display unit 20. The pattern generation unit 1225 may be configured to draw a figure while pointing up, and cut out the drawn figure as a mask pattern. In the case where the position detection sensor is configured by a contact-type detection element, the user can touch the screen with his / her finger to cut out a drawn figure (Embodiment 10).

  The mask pattern to be registered is a figure, a pattern, or a frame (for example, a photographing frame of a size for a car driver's license or a photographing frame of a size for a passport photograph), as in the case of the pattern generation / registration unit 112 described above. For example, a photographing frame indicating the size of an image, an object, an animal, a natural phenomenon, etc. may be used as long as it can be extracted as a mask pattern.

The cutout image generation / registration unit 123 registers a desired portion of a desired image in the recorded image reproduced with the mask pattern registered in the mask pattern memory 14B in the cutout image memory 14A (Embodiment 5).
In this case, the user specifies and selects a desired pattern from the mask patterns displayed on the screen. When a desired portion of the print image is cut out with the selected mask pattern, the mask pattern and the print image are combined and displayed on a screen, and positioning is performed. The cutout image generation / registration unit 123 can be configured to include a display coordinate determination unit that performs preview display at high speed, like the cutout image generation / registration unit 113 in order to speed up the preview screen display. (Example 11).

  The recording image synthesizing unit 124 synthesizes a desired clipped image registered in the clipped image memory 14A with a desired portion of a desired image of the reproduced recorded images, and records the resultant in the flash memory 14 (Embodiment 6). ).

  Further, the user can move the displayed cutout image so as to overlap a desired portion on the reproduced image serving as the background. Conversely, the reproduced image serving as the background can be moved so that the cut-out images overlap at a desired angle. In this case, the user selects a desired one from the cutout images displayed on the screen. Further, when the selected cutout image is pasted on a desired portion of the recorded image and combined, the cutout image and the recorded image are displayed on the screen to perform positioning. In order to speed up such preview screen display, the recorded image synthesizing unit 124 can be configured to include a display coordinate determining unit that performs preview display at high speed, similarly to the through image synthesizing unit 114 (Embodiment 11). .

  Further, the recording image synthesizing unit 124 is configured to include a resolution determining unit that automatically determines the resolution when the synthesized image is compressed and registered in the flash memory 14 from the resolution of the cut-out image, the resolution of the recording image, and the like. (Example 12).

[Guidance frame display processing means]
The guide frame display processing means 130 displays a guide frame pattern such as a guide frame for an ID photograph registered in the mask pattern memory on the liquid crystal display 20 in the imaging mode, and displays an image of a subject when taking an ID photo. The photographer adjusts the camera angle so that it fits within the displayed guide frame, and performs photographing / recording, so that an image of a desired size can be obtained. It is desirable that the size information is recorded in the link table 50 in association with the image at the time of recording so that the desired size can be printed at the time of printing.

<Link table>
FIG. 5 is an explanatory diagram showing a configuration example of the link table. The link table 50 is information for associating the composite image and its background image (in this case, the recorded image) on the flash memory 14, that is, the image of the composite image and the background image. A number, a pattern type, an image number, an image number of a cutout image, and the like can be recorded.
In FIG. 5, (A) shows an example of a link table in which the combined image, the background image, and the cutout image are associated with each other when the combined image is recorded in the flash memory 14 after pasting and combining. The image number of the image, the image number of the background image in the image number column 512, and the number of the clipped image stored in the clipped image memory 14A are recorded in the clipped image number column 513.

  (B) shows an example of a link table for registering information for associating a synthesized image with a background image and a cutout image without recording a synthesized image after pasting and synthesizing, and a cutout image number column 513-1 to 513. In −n (n ≧ 1), the number of the cut-out image synthesized with the background image of the image number recorded in the image number column 512 is recorded.

  (C) shows an example of a link table in which information for associating a clipped image, a background image (a combined image may be used as a background image), and a mask pattern image used for generating a clipped image is registered. In the mask pattern number column 514, the number of the mask pattern used when generating the cutout image of the cutout image number recorded in the cutout image number column 513 among the mask patterns recorded in the mask pattern memory 14B is recorded. ing. Also, in the composite position column 515, position information (coordinate values) of the background image in which the mask pattern is positioned on the background image when the cutout image is generated is recorded.

  When the pattern generation / registration unit 122 is configured to include the image distance data storage unit 1226, a plurality of distance columns are provided for one image, and distance data to a designated subject can be recorded. The configuration is as follows. Further, a print size field for designating the size of a print image at the time of image printing may be provided, and a value of the print size or a code corresponding to the print size may be recorded.

<Example>
FIG. 6 is an explanatory diagram of an example of combining an image and a cutout image (paste combining process) in the through image combining mode or the recorded image combining mode. The image in FIG. 6A is a through image displayed on the liquid crystal display 20 in the imaging mode or a recorded image reproduced in the reproduction mode. It is also assumed that a clipped image as shown in FIG. 6B is stored in the DRAM 13 or registered in the clipped image memory 14A in advance. In this case, in the pasting / synthesizing process, the image of FIG. 6A and the clipped image of FIG. 6B are synthesized to generate a synthesized image of FIG. Hereinafter, embodiments of the through image synthesis mode and the recorded image synthesis mode will be described.

[Example 1]: Pattern generation / registration processing (mask pattern generation / registration from through image)
This embodiment is an embodiment of the pattern generation / registration unit 112 in which a desired mask pattern is extracted from the through image by background processing in the through image synthesis mode and registered in the mask pattern memory 14B. FIG. 7 is a flowchart showing one embodiment of the pattern generation / registration processing operation, and FIG. 8 is an explanatory diagram of the pattern generation / registration process based on the flowchart of FIG. In FIG. 7, an image (a heart mark drawn on paper or a board in this example) desired by the user as a mask pattern in the imaging mode as shown in FIG. And a through image is displayed (T1).

  Next, when the camera is moved and the mask button 66 is operated at a desired angle while viewing the through image, the CPU 21 writes the image data once stored in the DRAM 13 into the work area of the DRAM 13 (T2).

  Here, the CPU 21 checks whether or not the user has operated the up / down key 67. If the up / down key 67 has been operated, the process proceeds to T7 (T3), and if not, the CPU 21 writes the data in the work area. The subject image (image data) thus cut is subjected to background processing and cut out, and then binarized (T4).

  After the binarization process, the signal generator 16 converts the binarized image data into video data and draws the video data on the VRAM 17, reads out the video data drawn on the VRAM 17, outputs the video data to the liquid crystal display 20, and displays the video data. As a result, a through image is displayed on the liquid crystal display 20 as a result of processing the patterning candidate image monitored by the user via the imaging lens 1 into a binary pattern. The user can check the pattern (FIG. 8B) by viewing the image displayed on the liquid crystal display 20 (T5).

  Here, the CPU 21 checks whether the user has operated the release button 63. If the release button 63 has been pressed, the process proceeds to T12, and if not, the process returns to T3 (T6). When the up / down key 67 is operated at T3, it is checked whether or not the luminance component changing operation has been performed. The presence or absence of the luminance component change operation is determined by whether the plus and minus up-down keys are pressed simultaneously or only one of them, and if they are pressed simultaneously, it is determined that the change operation has not been performed, and the process proceeds to T9. If it is pressed, it is determined that a change operation has been performed, and the process proceeds to T8 (T7).

  At T7, when the up / down key 67 (+) is pressed, the luminance value is shifted by a predetermined value (in this embodiment, one by one) to a higher value, and when the up / down key 67 (-) is pressed, the luminance value is low. Is shifted by a predetermined value (in this embodiment, one by one). Therefore, the luminance value of the pattern can be changed by operating the up-down key 67 (+) or the up-down key 67 (-) (T8). The luminance component of the pattern candidate image written in the work area of the DRAM 13 is fetched and processed by 8 bits to form a pattern (T9).

  Each time the up / down key 67 is operated once, the signal generator 16 converts the brightness-processed image data into video data and draws it on the VRAM 17, reads out the video data drawn on the VRAM 17, outputs the video data on the liquid crystal display 20, and displays it. . As a result, a through image is displayed on the liquid crystal display 20 as a result of processing the patterning candidate image monitored by the user via the imaging lens 1 into a luminance processing pattern. The user can confirm the pattern (FIG. 8B ′) by looking at the image displayed on the liquid crystal display 20 (T10).

  Here, the CPU 21 checks whether or not the user has operated the release button 63. If the release button 63 has been operated, the process proceeds to T12, and if not, the process returns to T7 (T11). When the release button 63 is pressed at T6 or T11, that is, when a desired mask pattern (FIG. 8B or 8B ') is obtained, the CPU 21 displays the pattern displayed at that time. Is extracted as a mask pattern and registered in the mask pattern memory 14B (T12). The luminance component processing pattern image obtained in T10 becomes a pattern in which the periphery of the pattern is blurred as shown in FIG. 8B ′, and a blurred cutout as shown in FIG. An image (cutout image) can be obtained.

  According to the present embodiment, a drawn picture can be photographed and extracted / registered as a mask pattern, so that registration of the mask pattern can be extremely easily performed. In addition, since a favorite pattern can be designed and registered, an image based on the design pattern, for example, a prototype of a trademark or a company emblem can be easily produced by a synthesis process described later. In the above description, an example was described in which a drawn picture was photographed and extracted / registered as a mask pattern. However, the present invention is not limited to this, and figures and images in towns and cities, posters, signboards, magazines, and the like can be used. It is also possible to take an image and extract and pattern those which the user wants to pattern from among them, or to pattern a figure directly drawn and input.

As an example of the former,
(1) an example of extracting a desired subject image pattern as a mask pattern by performing a contour extraction process of detecting and extracting a contour of a subject,
(2) An example in which a cursor is moved on a screen and a pattern is extracted by tracing a contour of a desired image as a mask pattern;
(3) An example in which a transparent position detection sensor is arranged on the screen, and a pattern is extracted by tracing the outline of the through image with a pointing device such as a touch pen (Example 8). , As an example of the latter,
(4) An example in which a user moves a cursor on a screen and inputs a drawn figure and extracts the figure as a mask pattern (Embodiment 9);
(5) An example in which a transparent position detection sensor is provided on a screen and a figure is drawn on the screen with a pointing device such as a touch pen and extracted as a mask pattern can be given, but not limited thereto.

[Example 2]: Cutout image generation / registration processing (generation / registration of cutout image from through image and mask pattern)
This embodiment is an embodiment of the image generation / registration unit 113. In this example, a through image and a registered mask pattern are appropriately combined in a through image combining mode to form a cutout image, which is then cut out and registered. is there. FIG. 9 is a flowchart showing one embodiment of the cutout image generation / registration processing operation. FIGS. 10 and 11 are explanatory diagrams of the cutout image generation / registration process based on the flowchart of FIG. Hereinafter, a case will be described as an example where the user uses a subject image as shown in FIG. 10A as a cutout image.

  In FIG. 9, a subject as shown in FIG. 10A, which is desired by the user as a cut-out image in the imaging mode, is photographed, and the subject image is once taken into the DRAM 13 and displayed as a through image (U1). Here, the CPU 21 checks whether or not the mask button 66 has been operated (S2).

  When the user presses the mask button 66, the CPU 21 reads the mask pattern stored in the mask pattern memory 14B, draws the mask pattern on the VRAM 17 via the signal generator 16, reads the mask pattern drawn on the VRAM 17, and performs D / A conversion. The information is displayed on the liquid crystal display 20 via the circuit 18 and the amplifier 19. Thereby, for example, a mask pattern as shown in FIG. 10B is superimposed on a through image as shown in FIG. 10A and displayed on the liquid crystal display 20 as shown in FIG. 10C. In this case, the mask pattern is positioned at the center of the screen (U3).

  Here, when the mask button 66 is further pressed, the next mask pattern is read out and superimposed on the through image, so that the user determines whether or not the mask pattern displayed on the liquid crystal display 20 is a desired one. If it is not the desired mask pattern, the mask button 66 is pressed until the desired mask pattern appears, and the mask pattern and the through image are combined and superimposed and displayed (U4).

  When a desired mask pattern is displayed in U4, the user looks at the through image superimposed on the mask pattern in U4 (for example, a desired portion of the through image is superimposed on the mask pattern in the center of the screen). If it is determined that the position of the through image needs to be corrected, the user moves the camera and adjusts the angle so that the desired portion of the through image overlaps the mask pattern in the center of the screen, If it is not necessary to correct the position of the through image, the release button is pressed (U5). The CPU 21 checks whether or not the user has pressed the release button 63 (U6). If the release button 63 has been pressed, the CPU 21 cuts out an image portion (FIG. 10D) displayed on the liquid crystal display 20. It is registered in the cutout image memory 14A (U7). In step U5 of FIG. 9, the user moves the camera to correct the position of the through image so that a desired portion of the through image overlaps the mask pattern at the center of the screen. However, the user moves step U5. The mask pattern may be moved by operating the key 62 (or an input means such as a pointing device).

  When the cut-out image created by the user himself is registered in the cut-out image memory 14A as described above, the later-described synthesizing process (embodiments 3 and 5) can be performed. FIG. 10B shows a binarized pattern. In this case, the cutout image has a clear boundary as shown in FIG. 10D. On the other hand, when the cutout image shown in FIG. 11D is generated by using the luminance processing pattern shown in FIG. 11B in the above embodiment, a cutout image with a blurred boundary can be obtained. Further, in step U7 of FIG. 9, the image portion displayed on the liquid crystal display 20 is cut out and registered. You may do so.

[Embodiment 3] Paste synthesis processing (synthesis of a through image and a clipped image)
This embodiment is an example of the live view image combining processing unit 114, and is an example of combining the live view image and the cutout image registered in the cutout image memory 14A in the live view image combining mode.

  FIG. 12 is a flowchart showing one embodiment of the pasting and combining processing operation of the digital camera of FIG. 1, and FIGS. 13 and 14 are explanatory diagrams of the pasting and combining process based on the flowchart of FIG. In FIG. 12A, the user takes a desired subject in the imaging mode, takes in the subject image into the DRAM 13, and displays a through image (V1).

  Next, when the user presses the frame button 61 (V2), the CPU 21 reads out the first cutout image from the cutout images recorded in the cutout image memory 14A, supplies the first cutout image to the signal generator 16, and causes the VRAM 17 to draw. . Since the through image captured at V1 has already been drawn in the VRAM 17, as a result, an image obtained by combining the through image and the reproduction cutout image is drawn in the VRAM 17. The synthesized image is read by the signal generator 16 and D / A converted by the D / A conversion circuit 18, output to the liquid crystal display 20 via the amplifier 19, and set at a predetermined position of the liquid crystal display 20 (in the embodiment, (Center). Thereby, for example, a composite image as shown in FIG. 13A is obtained. In FIG. 13A, reference numeral 21 denotes a through image, reference numeral 22 denotes a clipped image, and reference numeral 23 denotes a background image (V3).

  The user looks at the composite image 20 of FIG. 13A displayed on the liquid crystal display 20 and determines whether or not the displayed clipped image 22 is a desired clipped image. If the displayed clipped image 22 is not the desired clipped image, the process returns to V2. The user keeps pressing the frame button 61 until a composite image with the desired cutout image is displayed. As a result, the CPU 21 reads out the next clipped image recorded in the clipped image memory 14A, and displays the composite image at a predetermined position on the liquid crystal display 20 using V3 as described above. When the composite image with the desired cutout image is displayed, the process proceeds to V5 (V4).

  The user needs to correct the position of the through image 21 by looking at the image displayed in the above V4 (for example, when the layout of the pasted image 22 is not preferable as shown in FIG. 13A due to the composition). If it is determined that there is, the user moves the camera and adjusts the angle so that a desired portion of the through image 21 overlaps the cutout image 22 at the center of the screen as shown in FIG. . If it is not necessary to correct the position of the through image 21, the release button is pressed (V5).

  When a desired composite image is obtained in V5, the user selects whether or not to record it in the flash memory 14 (V6). As a result of the selection, if the composite image is to be recorded in the flash memory 14, when the user further operates the release button 63 (V7), the CPU 21 reads the composite image data currently drawn in the VRAM 17 and reads the composite image data. Is recorded as one of the recorded images (V8). If the synthesized image is not recorded in the flash memory 14 as a result of the selection in V6, the synthesized image is simply displayed on the liquid crystal display 20 in this case.

  In step V5 of FIG. 12A, the user corrects the position of the through image 21 by moving the camera so that a desired portion of the through image overlaps the pasted image 22. The user may operate the cursor movement key 62 (or an input means such as a pointing device) to move the cutout image. In this case, if it is necessary to correct the position of the cut-out image in step V5 as shown in FIG. 12B, the user operates a moving key corresponding to a predetermined direction among the moving keys 62 (V5). '), The CPU 21 moves / displays the cutout image in the direction corresponding to the operated moving key 62 and returns to V5 (V5 "). As a result of the selection in V5, there is no need to correct the cutout image. (Including the case where the result of the correction operation by V5 ', V5 "is OK), the process can be shifted to V6.

  FIGS. 13A and 13B show an example in which the cutout image is moved. In FIG. 13A ', the position of the cutout image 22 at the center overlaps with the background and the background is not visible. This is an example in which the cutout image 22 is moved to the upper left as shown in FIG.

  Further, in the present embodiment, as described in V8 above, the combined image data is recorded in the flash memory 14 as one of the recorded images. However, the present invention is not limited to this, and the link table 50 (FIG. 5) described above may be flashed. It may be provided in the memory 14 so that, when a composite image is recorded, the information is registered in the link table 50 with a meaning different from the recorded image. Instead of recording the composite image, only the information is registered in the link table 50, and when displaying the composite image or outputting to the external device, the composite image is reproduced based on the information registered in the link table 50. May be configured. The cutout image 22 used in FIG. 13 is a composite image with the binarized pattern. In this case, the boundary of the cutout image 22 is clear. On the other hand, in the above-described embodiment, if the cutout image 22 ′ formed by the synthesized image with the luminance processing pattern as shown in FIG. 14 is used, a synthesized image with a blurred boundary with the cutout image can be obtained.

[Embodiment 4]: Pattern generation / registration processing (mask pattern generation / registration from recorded image)
In the first embodiment, a desired mask pattern is extracted from a through image and registered, but a mask pattern can be extracted and registered from a recorded image. This embodiment is an embodiment of the pattern generation / registration unit 122, in which a desired portion is extracted as a mask pattern from a reproduced desired recorded image and registered in the mask pattern memory 14B in the recorded image combining mode. It is.

  FIG. 15 is a flowchart showing one embodiment of the pattern generation / registration processing operation, and FIG. 16 is an explanatory diagram of the pattern generation / registration process based on the flowchart of FIG. In FIG. 15, the user operates the play button 64 to set the play mode, and plays and displays the first recorded image. The user can operate the mask button 66 to reproduce an image including a subject to be used as a mask pattern and display the reproduced image on the liquid crystal display 20. For example, a reproduced image (in this example, an image of a picture in which a pattern is drawn) as shown in FIG. 16A is displayed on the liquid crystal display 20. In addition, the CPU 21 displays a frame cursor 171 at a predetermined position on the screen by a predetermined operation (W1).

  Next, when the user positions the cursor 171 so as to surround the desired image 172 while viewing the reproduced image and operates the mask button 66, the CPU 21 moves the cursor from the video data (compressed data) recorded in the flash memory 14 to the cursor. The data corresponding to the portion surrounded by the frame 171 is read and supplied to the data compression / decompression circuit 15. Then, the decompressed video data (FIG. 16B) is written into the work area of the DRAM 13 (W2). When the desired image is larger than the cursor 171 at W1, the cursor 171 can be enlarged by pressing the up / down key 67 (+) to surround the desired image. If the desired image is too small compared to the cursor 171, the size of the frame can be adjusted by pressing the up / down key 67 (−) to reduce the cursor 172.

  Here, the CPU 21 checks whether or not the user has operated the up / down key 67 (W3). If the up / down key 67 has been operated, the process proceeds to W7, and if not, the CPU 21 writes in the work area. The clipped subject image (image data) is cut out and binarized (W4). After the binarization process, the signal generator 16 converts the binarized image data into video data and draws the video data on the VRAM 17, reads out the video data drawn on the VRAM 17, outputs the video data to the liquid crystal display 20, and displays the video data. As a result, the liquid crystal display 20 displays the result of processing the reproduced patterning candidate image into a binary pattern. The user can confirm the pattern (FIG. 16C) by looking at the image displayed on the liquid crystal display 20 (W5).

  Here, the CPU 21 checks whether or not the user has operated the release button 63. If the release button 63 has been operated, the process proceeds to W12, and if not, the process returns to W3 (W6). If the up / down key 67 is operated in W3, it is checked whether or not the luminance component changing operation is performed. If not, the process proceeds to W9, and if it is performed, the process proceeds to W8 (W7). When the luminance component changing operation is performed at W7, when the up / down key 67 (+) is pressed, the luminance value is shifted by a predetermined value (in this embodiment, by 1) to a higher luminance value, and the up / down key 67 (+) is pressed. When-) is pressed, the luminance value is shifted by a predetermined value (in this embodiment, one by one) toward the lower side. Therefore, by operating the up / down key 67 (+) or the up / down key 67 (-), the luminance value of the pattern can be changed (W8).

  The luminance component of the pattern candidate image written in the work area of the DRAM 13 is fetched and processed by 8 bits to form a pattern (W9). Each time the up / down key 67 is operated once, the signal generator 16 converts this luminance component processed image data into video data and draws the video data on the VRAM 17, reads out the video data drawn on the VRAM 17, outputs it to the liquid crystal display 20, and displays it. Let it. As a result, the result of processing the desired patterning candidate image read from the flash memory 14 by the user into the luminance component processing pattern is displayed on the liquid crystal display 20. The user can confirm the pattern (FIG. 16C ′) by looking at the image displayed on the liquid crystal display 20 (W10).

  Here, the CPU 21 checks whether or not the user has operated the release button 63. If the release button 63 has been operated, the process proceeds to W12, and if not, the process returns to W7 (W11). When the release button 63 is pressed by the above W6 or W11, that is, when a desired mask pattern (FIG. 16 (C) or (C ′)) is obtained, the CPU 21 masks the displayed pattern image. The pattern is registered in the mask pattern memory 14B (W12).

  According to the present embodiment, the luminance component processing pattern image obtained in W10 has a pattern in which the periphery of the pattern is blurred as shown in FIG. 16 (C ′), and the pattern is blurred as shown in FIG. Can be obtained. Further, according to the present embodiment, a recorded image can be reproduced and extracted / registered as a mask pattern, so that a photographed, recorded and saved image can be taken out at a desired time and a mask pattern can be created by a simple operation. .

Further, in the present embodiment, the frame-shaped cursor 171 as shown in FIG. 16A is displayed in step W1, and the portion surrounded by the cursor 171 in the reproduced image is extracted in W2, but the present invention is not limited to this. Absent. In such an alternative embodiment,
(1) A point cursor is displayed in step W1, and in W2, the user operates the movement key 62 to move the cursor on the screen to trace the contour of a desired portion of the reproduced image, and pattern the traced closed area. Examples extracted,
(2) An example in which a transparent position detection sensor is provided on the screen, and a contour of a desired portion of a reproduced image is traced with a pointing device such as a touch pen, and a traced closed area is extracted as a pattern (Eighth Embodiment) ).
(3) Distance data of a plurality of subjects included in the image is recorded together with the image data, the image is reproduced, a distance is specified, a certain portion in the reproduced image is specified, and the portion is extracted. Example (Example 9),
(4) An example in which a user moves a cursor on a screen using a movement key to trace a contour of a subject image to perform a contour extraction process to extract a pattern.

[Embodiment 5]: Recording image synthesis processing (Synthesis of recording image and cutout image)
In the present embodiment, in the recorded image combining mode, the recorded image combining unit 124 combines the recorded image recorded in the flash memory 14 and the clipped image with the frame registered in the clipped image memory 14A in advance. This is one embodiment.

  FIG. 17 is a flowchart showing one embodiment of the recording image composition processing operation of the present invention, and FIG. 18 is an explanatory diagram of a composite image generation process based on the flowchart of FIG. In FIG. 17, the user reads the recorded image to be synthesized from the flash memory 14 and displays the image on the liquid crystal display 20. This operation is performed by operating the reproduction button 64 to set the reproduction mode, reproducing and displaying the first recorded image, and operating the up / down key 67 to reproduce / display a desired image (reproduction). /Display mode). Thus, for example, a reproduced image as shown in FIG. 18A is displayed on the liquid crystal display 20 (S1).

  Next, when the user operates the frame button 61 (S2), the CPU 21 reads out a predetermined cutout image from the cutout images recorded in the cutout image memory 14A, supplies the readout cutout image to the signal generator 16, and causes the VRAM 17 to draw. . Since the recorded image reproduced in S1 is already drawn on the VRAM 17, as a result, an image obtained by combining the reproduced image and the reproduced clipped image is drawn. The synthesized image is read out by the signal generator 16, D / A converted by the D / A conversion circuit 18, output to the liquid crystal display 20 via the amplifier 19, and displayed. Thereby, for example, a composite image as shown in FIG. 18B is obtained (S3).

  The user looks at the composite image displayed on the liquid crystal display 20 and selects whether or not the displayed image is a desired cutout image. If the displayed image is not the desired cutout image, the process returns to S2, and the frame button 61 is operated again. As a result, the CPU 21 reads out the next cut-out image recorded in the cut-out image memory 14A and displays the composite image on the liquid crystal display 20 in S3 in the same manner as described above. The user keeps pressing the frame button 61 until the composite image with the desired clipped image is displayed. When the composite image with the desired clipped image is displayed, the process proceeds to S5 (S4).

  The user looks at the composite image and selects whether or not it is necessary to correct the position of the image within the frame (frame). As a result of the selection, when it is necessary to correct the image in the frame, the process proceeds to S6, and otherwise, the process proceeds to S8 (S5). If it is necessary to correct the position of the image in the frame in S5, the user operates a moving key corresponding to a predetermined direction among the moving keys 62 (S6), and the CPU 21 operates the operated moving key. The reproduced image is moved / displayed in the direction corresponding to 62, and the process returns to S5 (S7). For example, when the movement key 62 is operated to instruct the movement in the left and down directions while the composite image of FIG. 18B is displayed, as shown in FIG. The displayed image moves to the lower left. As described above, when the user operates the movement key 62, the background image (the reproduced recorded image) can be moved within the frame and positioned at a desired position. Note that the cutout image may be moved instead of the background image.

If it is determined that the image in the frame does not need to be corrected as a result of the selection in S5 (including the case where the position correction (S6, S7) results in OK), the user can now use the liquid crystal display. The user selects whether or not to record the composite image displayed on the flash memory 20 in the flash memory 14 (S8). As a result of the selection, if the composite image is to be recorded in the flash memory 14, the process proceeds to S9; otherwise, the process ends. In this case, the composite image is simply displayed on the liquid crystal display 20.
As a result of the selection in S8, if the composite image is to be recorded in the flash memory 14, when the user further operates the release button 63 (S9), the CPU 21 reads the composite image data currently drawn in the VRAM17. Then, it is recorded as one of the recorded images in the flash memory 14 (S10).

  In the present embodiment, the composite image data is recorded as one of the recording images as described in S10 above, but the present invention is not limited to this, and the above-described link table 50 (FIG. When recording a composite image, the information may be registered in the link table 50 with a meaning different from the recorded image, and the link table 50 may be used instead of recording the composite image. Only the information may be registered, and the composite image may be reproduced based on the information registered in the link table 50 when the composite image is displayed or output to an external device.

[Sixth Embodiment]: Cutout image generation / registration processing (generation / registration of cutout image from recorded image and mask pattern)
In the synthesizing process, the frame-attached cut-out image previously recorded in the cut-out image memory 14A is appropriately read and a desired image is selected from a plurality of cut-out images. It is also possible to generate a cut-out image using the recorded image that is present. In the present embodiment, a cut-out image generation / registration unit 123 generates and registers a cut-out image by registering a cut-out image by appropriately synthesizing a print image and a pre-registered mask pattern in a print image synthesis mode. FIG.

  FIG. 19 is a flowchart showing one embodiment of the cutout image generation / registration processing operation, and FIG. 20 is an explanatory diagram of the cutout image generation / registration process based on the flowchart of FIG. In FIG. 19, the user operates the playback button 64 to set the playback mode, plays back the first recorded image and displays it, and operates the up / down key 67 to play back the image desired to be a cut-out image. It is displayed on the display 20. Thereby, for example, a recorded image as shown in FIG. 20A is displayed on the liquid crystal display 20 (S21).

  Next, when the user operates the mask button 66, the CPU 21 reads the mask pattern stored in the mask pattern memory 14B, draws it on the VRAM 17 via the signal generator 16 (S22), and writes the mask pattern drawn on the VRAM 17 The data is read out and displayed on the liquid crystal display 20 via the D / A conversion circuit 18 and the amplifier 19. Thereby, for example, a mask pattern as shown in FIG. 20B is displayed on the liquid crystal display 20. At this time, the CPU 21 displays a cursor so as to be superimposed on the mask pattern (S23).

  Here, the user selects a desired one from the mask patterns displayed on the liquid crystal display 20. That is, a cursor (a frame-shaped cursor in the example in the figure) as shown in FIG. 20B is operated on the desired mask pattern by operating the movement key 62. Then, the mask button 66 is operated again to confirm the selection (S24). The CPU 21 sends only the mask pattern selected in S24 to the VRAM 17, and draws the mask pattern superimposed on the reproduced image selected in S21. Thereby, for example, as shown in FIG. 20C, the reproduced image of FIG. 20A and the selected mask pattern of FIG. 20B are superimposed and displayed on the liquid crystal display 20 (S25).

  Next, when the user determines that it is necessary to correct the position of the mask pattern by looking at the image superimposed and displayed in S25, the process proceeds to S27, and otherwise, proceeds to S29 (S26). If it is determined in S26 that the position of the mask pattern needs to be corrected, the user operates the move key 62 (S27) to move the mask pattern. For example, when an upward key among the move keys 62 is operated in the state of FIG. 20C, the mask pattern is moved upward as shown in FIG. 20D (S28). When a composite image obtained by combining the desired mask pattern and the recording image is obtained in S26 or S28, the user operates the release button 63 to register the composite image as a cut-out image in the flash memory (S29). . At this time, the CPU 21 cuts out the composite image (the image in FIG. 20D) displayed on the liquid crystal display 20 and records it as a cutout image (pattern frame + image) in the cutout image memory 14A (S30).

  When the cut-out image created by the user is registered in the cut-out image memory 14A as described above, the cut-out image synthesizing process is performed by the same operation (see the fifth embodiment) as described in the flowchart of FIG. it can. That is, when the frame button 61 is operated a predetermined number of times in S2 of FIG. 17, the image shown in FIG. 20D is read from the clipped image memory 14A and reproduced, and is reproduced on the liquid crystal display 20 in S3. Therefore, if the recorded image to be combined with the cutout image is reproduced and displayed in advance in S1, the cutout image created by the user can be combined with another recorded image as shown in FIG. In addition, the combined image can be recorded in the flash memory 14 by operating the release button 63 as described in S9.

  As described above, in the fifth and sixth embodiments, the image recorded in the flash memory 14 and the clipped image registered in the clipped image memory 14A in advance can be combined, and the clipped image with frame is already recorded. It can be newly generated by appropriately combining the existing image and the mask pattern. As described above, the recorded image can be easily processed (combined).

[Example 7]
FIG. 21 is a flowchart showing an example of the process of displaying a guide frame of the digital camera shown in FIG. 1 by taking an ID photo frame using the ID photo frame, and FIG. 22 is a flowchart showing the ID photo shooting process based on the flowchart of FIG. FIG.
In FIG. 21, when the user operates the up / down key 67, the CPU 21 reads the mask pattern stored in the mask pattern memory 14B, draws the mask pattern on the VRAM 17 via the signal generator 16, and draws the guide frame pattern drawn on the VRAM 17. Is read out and displayed on the liquid crystal display 20 via the D / A conversion circuit 18 and the amplifier 19. Thereby, for example, the information is displayed on the liquid crystal display 20 as a guide frame 81 (in this example, a guide frame for a driver's license photograph) as shown in FIG. In this case, the guide frame 81 is positioned at the center of the screen (X1).

Here, when the up / down key 67 is further operated, the next guide frame is read out and displayed, so that the user determines whether or not the guide frame displayed on the liquid crystal display 20 is the desired guide frame. If not, the guide frame is displayed by operating the up / down key 67 until the desired guide frame appears (X2).
When a desired guide frame is displayed in X2, the user views the through image 82 ′ (FIGS. 22C and 22D) of the photographed person 82 (FIG. 22B) in the center of the screen. Check whether it is properly in the guide frame (X3). If it is properly in the guide frame, press the release button 63. Otherwise, adjust the camera angle to display the desired through image on the screen. Make sure it fits in the center guide frame (X4).

The CPU 21 checks whether or not the user has pressed the release button 63 (X5). When the release button 63 has been pressed, the subject image 83 displayed on the liquid crystal display 20 (FIG. 22E). Is cut out and recorded in the flash memory 14 as a photographed image (X6).
It should be noted that a through image may be recorded as it is without being clipped at the time of photographing, and then may be clipped from a reproduced image later (that is, used as a guide for photographing).
In the above description, the guide frame is for photographing, but may be any of mask patterns. Further, the mask pattern and the cutout image may be registered in advance in a recording medium such as a flash memory by a maker or the like.
As a result, the guide frame can be displayed at the time of photographing, so that the size of the image can be made substantially constant, or a photograph for certification or the like having a predetermined size can be easily taken.

  [Eighth Embodiment] Example of Outline Extraction by Pointing Device or Cursor FIG. 23 is an explanatory diagram showing an example of mask pattern generation by outline extraction.

FIG. 23A shows a reproduced recorded image, which corresponds to the reproduced image selected by the user in step W1 of the flowchart in FIG. FIG. 23B corresponds to step W2 of the flowchart in FIG. 15. After a desired image 91 (a bear in this example) is displayed, the outline of the face of the image 91 is traced with a touch pen as indicated by a symbol 92. It shows the state where it was turned on. When the specification of the extraction range with the touch pen is completed and the user operates the release button 63, the CPU 21 obtains a closed space from the coordinates (xi, yj) of the locus of the touch pen. Then, the image data stored in the work area is offset in a portion other than the closed region 93 (W3), and then binarized (W4).
FIG. 23C corresponds to step W5 in the flowchart of FIG. 15, and displays the result of processing into the binarized pattern displayed on the liquid crystal display 20. When the user operates the release button 63, the pattern image 95 is registered as a mask pattern in the mask pattern memory 14B as a desired mask pattern (FIG. 23D) (W12).

  In the present embodiment, the reproduced recorded image is traced with the touch pen. However, even if the cursor movement key 62 is operated to trace the outline of the face of the image 91 with the cursor, the mask pattern is operated in substantially the same manner as described above. Can be obtained. In this embodiment, the outline is extracted by tracing the outline of the reproduced image. However, by extracting the outline of the through image based on the flowchart of FIG. 7, the desired mask pattern can be obtained as in the case of the recorded image. Can be obtained.

Ninth Embodiment Example of Pattern Generation Based on Distance Designation There is a case where a user looks at a signboard in a townscape and wants to make a pattern. The number of signboards in the field of view is not limited to one, and the signboard may be viewed from the front to the back in a narrow field of view. In such a case, many signboards enter the same field of view of the camera at a distance.

  The present embodiment is an example in which subjects at different distances within the same field of view are used as patterning candidates to form a single recorded image, which is reproduced to enable a mask patterning operation for each subject.

When the digital camera has an autofocus mechanism, the autofocus mechanism operates so that the camera focuses on a point of interest (in most cases, the center of the viewfinder), and the focus is determined. Here, if the distance between the subject and the imaging lens 1 is a, the distance between the imaging lens 1 and the surface of the CCD 2 is b, and the focal length of the lens is f, 1 / a + 1 / b = 1 / f holds.
Here, when the imaging lens 1 is moved ± Δx (Δx <b << a),
1 / (a−Δx) + 1 / (b + Δx) = 1 / f or
1 / (a + Δx) + 1 / (b−Δx) = 1 / f
However, since b << a, it can be considered that 1 / (A + ΔX) ≒ 1 / a of the first term on the left side, and 1 / a + 1 / (b ± Δx) = 1 / f (Formula 1) holds. .

  Since the moving distance Δx of the lens at the time of focusing by the auto focus control is known, the distance a between the subject and the imaging lens 1 can be calculated from the above equation (1). Therefore, if the camera is finely moved and the camera is finely moved so that the pattern candidate in the same field of view becomes the attention point of the viewfinder in order, the distance between each pattern candidate and the lens can be obtained. Can be. Further, the distance between each pattern candidate and the lens can be obtained with an imaging device having a distance measuring mechanism without having an autofocus mechanism.

  FIG. 24 is a flowchart showing an example of the pattern generation operation based on the distance designation. FIG. 24A shows the operation of the distance data storage unit 1126 at the time of photographing, and FIG. 24B shows the operation at the time of reproduction (patterning). The distance aj (j = 1 to n) between the subject which is a patterning candidate in the same visual field and the imaging lens 1 is obtained by the above-described method, and is temporarily stored in the distance data storage area of the DRAM 13 (Y1).

If the user presses the release button 63 to take a picture, the process proceeds to Y3, otherwise returns to Y1 (Y2).
When the photographing is performed at Y2, each distance data of the image is read out from the distance data storage area of the DRAM 13 and stored in the distance column of the link table 50 simultaneously with the recording of the image data (Y3).
At the time of reproduction, a desired reproduced image is displayed (Y4), and distance data of the image is extracted from the link table 50 and displayed together with the reproduced image (Y5).
When the user operates the movement key 62 to specify one of the distances indicated by the cursor (Y6) and presses the release button 63 (Y7), the subject image at that distance is extracted (Y8) and displayed. (Y9).

  When the user presses the mask button 66 (Y10), the displayed subject image is subjected to mask patterning processing (T3 to T11 in FIG. 7) such as binarization processing and contour extraction processing (Y11), and the mask pattern is displayed. (Y12), but if the displayed subject image includes another image at the same distance in addition to the pattern candidate, the movement key 62 is operated (Y13) to obtain the above-described contour. An extraction process (Eighth Embodiment) can be performed (Y14).

  When the contour extraction processing has been performed, the image other than the pattern candidates is offset and the process proceeds to the mask patterning processing (Y11) (Y15). Also, by repeating such an operation, a plurality of mask patterns can be obtained from the same image. Note that a distance may be designated at the time of displaying a through image, a subject image at that distance may be extracted, and registered as a mask pattern.

  [Tenth Embodiment] Example of Drawing Input Using Cursor or Pointing Device When the direct button 68 is pressed in the through-image combining mode or the recorded image combining mode, a locus drawn on the screen by the cursor or the pointing device (or a designated point is connected). ) Can be input as a pattern, and can be registered as a mask pattern after the extraction processing.

FIG. 25 is a flowchart showing an operation example of the pattern generation unit 1125, and FIG. 26 is an explanatory diagram of mask pattern formation of a pattern drawn by a cursor.
When the direct button 68 is pressed in the through image synthesizing mode (Z1), the CPU 21 controls the timing generator 12 to cut off the supply of image data to the signal generator 16, and the liquid crystal display 20 displays FIG. A drawing input start instruction message and a cursor 310 as shown in (A) are displayed (Z2).

  When the user operates the move key 62 to continuously move the cursor, the locus 311 of the cursor movement is displayed (FIG. 26B) (Z3), and the coordinates (xi, yj) on the locus are sequentially stored in the DRAM 13 (Z4). At this time, if the cursor is moved by pressing the down key 67 (-), the trajectory of the cursor is not displayed, and the trajectory coordinates during that time are not taken into the work area (Z5). In this case, if the cursor is moved so as to overlap the displayed locus, the line segment can be deleted (FIG. 26C) (Z6). When the up key 67 (+) is pressed, the display of the trace of the cursor is started, and the capture of the trace coordinates into the work area is restarted (Z7).

  After the drawing figure 300 composed of the closed areas connected by the continuous lines is formed by repeating the above Z5 and Z6, when the release button 63 is pressed (Z8), the linear shaping for linearly shaping the linear continuous lines is performed. The icon 301, the icons 302 and 303 for shaping the curve which gives smoothness to the curved continuous line, and the icons for fixed figures (for example, a circle 304, a rectangle 305, an ellipse 306, a triangle 307,...). This is shown together with the drawn figure (FIG. 26E) (Z9).

  The user operates the movement key 62 to designate the section PQ {(xa, yb), (xc, yd)} on the drawing 300 with the cursor (FIG. 26 (E)), and then selects the desired icon. Is designated, the section is shaped in the form of the designated icon (FIG. 26 (F)). If a standard graphic is desired, the user can point to an internal area (one point in both closed areas) of the graphic 300 and then point to a desired icon to obtain a standard graphic (Z10).

  When the up / down key 67 is operated after the shaping, the graphic is enlarged / reduced. FIG. 26G shows a reduced pattern 300 ′ (Z11, Z12). When the user presses the release button 63, the graphic on the screen is extracted as a pattern (Z13, Z14). When the pattern is extracted, a mask patterning process (see steps T3 to T11 in FIG. 7) is performed (Z15), and after the mask patterning process is completed, it is registered in the mask pattern memory 14B (Z16).

[Example 11] Example of preview image display (example in which boundary of mask pattern is blurred)
In the present embodiment, the registered mask pattern is displayed (U3 in FIG. 9, X1 in FIG. 21), the moving display of the cutout image (step V3 in FIG. 12, step S3 in FIG. 17, step S21 in FIG. 19), This is an example of speeding up preview display performed before the combining process, such as display of a background image (T1 in FIG. 7, U1 in FIG. 9, step V1 in FIG. 12, step W1 in FIG. 15, step S1 in FIG. 17).

  In image synthesis, a storage area (work area) for one screen (one frame) is provided in the DRAM 13. At the time of screen display, image data of one image to be displayed at that time is read from a memory (a mask pattern memory 14B for a mask pattern, a clipped image memory 14A for a clipped image, and a flash memory 14 for a recorded image). It is issued and stored in the work area.

  Here, assuming that the coordinates of the upper left position of the screen of the liquid crystal display 20 are (0, 0), first, data corresponding to the screen coordinates (0, 0) is written at the position (0, 0) of the work area. , Data corresponding to the screen coordinates (xi, yj: i = 0 to n−1, j = 0 to m−1) are sequentially written to the position (xi, yj) of the work area. Data is written.

Here, the coordinates of each data written in the work area are defined as follows for explanation.
Coordinates of mask pattern data: M (xi, yj)
Coordinates of cutout image data: C (xi, yj)
Coordinates of background image data: B (xi, yj)
Display coordinates (coordinates of data in the work area after the combining process): W (xi, yj)
Suppose that data is represented by 8 bits.

In this case, for example, when a mask pattern is displayed on a cut-out original image (background image) in order to select a mask pattern, the background image of the selection candidate mask pattern and the cut-out original image is synthesized. At this time, the coordinates of the display position of the mask pattern are calculated as
W (xi, yj) = M (xi, yj) × B (xi, yj)
+ (255-M (xi, yj) * C (xi, yj)) / 255
Is determined. Here, the numerical value 255 is luminance.

When a cut-out image is generated by cutting out a background image as a cut-out original image using a mask pattern, a cut-out candidate image is generated / displayed by synthesizing the mask pattern and the background image while moving the background image. Can be cut out (extracted) as a cut-out image when a desired one is selected from the cut-out candidate images. In this case, the display coordinates of the cut-out candidate image are determined by the movement amount in the coordinate calculation of the image synthesis processing. Is p bits wide and q bits high,
W (xi, yj) = ｛M (xi−p, yj−q) × B (xi, yj)
+ (255-M (xi-p, yj-q))
× C (xi-p, yj-q)｝ / 255
As can be determined.

  In this embodiment, the display coordinate determination module is incorporated in the pattern generation unit 1125 (or 1225) or the cutout image generation registration unit 113 (123) to calculate the display coordinates. In order to display the image at high speed, the display coordinate determination module is configured to perform the coordinate calculation for each image in each case as described below.

(1) When selecting a mask pattern, set the display coordinates of the mask pattern to
When the luminance of M (xi, yj) is smaller than a certain threshold,
W (xi, yj) = C (xi, yj)
When the luminance of M (xi, yj) is larger than a certain threshold,
W (xi, yj) = B (xi, yj)
This process is performed for all the areas of the work area for one screen (i = 0 to n−1, j = 1 to m−1). Based on the result, the mask pattern is displayed on the liquid crystal display 20 by the pattern generation / registration unit 112 (122) and the cutout image generation / registration unit 113 (123). In this embodiment, the threshold value is set to 128. According to the above coordinate calculation, the mask pattern can be displayed several times faster than the synthesis speed (coordinate calculation speed) during normal image synthesis processing. In this case, the boundary between the mask pattern and the background becomes clear, but there is no problem even if the boundary is clear for preview display, not actual synthesis.

(2) When selecting a cropped image, the display coordinates of the cropped image are
When the luminance of M (xi, yj) is smaller than a certain threshold,
W (xi, yj) = C (xi + p, yj + q)
When the luminance of M (xi, yj) is larger than a certain threshold,
W (xi, yj) = B (xi, yj)
This process is performed for all the areas of the work area for one screen (i = 0 to n−1, j = 1 to m−1). Based on the result, the cut-out image is displayed on the liquid crystal display 20 by the pattern generation registration unit 112 (122). Here, p and q are vertical and horizontal movement amounts (the number of dots). In this embodiment, the threshold value is set to 128. According to the above coordinate calculation, the cut-out image can be displayed several times faster than the synthesis speed (coordinate calculation speed) at the time of normal image synthesis processing.

(3) When displaying the background image, the display coordinates of the background image (through image or reproduced recorded image) are
The raw data, that is, the coordinates of the through image or the reproduced recorded image is written in the work area as it is. That is,
W (xi, yj) = B (xi, yj)
This process is performed for the entire area of the work area for one screen (i = 0 to n-1, j = 1 to m-1). Based on this result, the background image is displayed on the liquid crystal display 20 by the pattern generation / registration unit 112 (122), the cut-out image generation / registration unit 113 (123), the through-image synthesis unit 114, or the recorded image synthesis unit 124.

(4) When selecting the paste position (when moving the image), specify the display coordinates of the cropped image,
Assuming that the movement amount is p dots vertically and q dots horizontally, and the movement amounts from the center of the pasting position are r and s,
When the luminance of M (xi-r, yj-p) is smaller than a certain threshold,
W (xi, yj) = C (xi + pr, yj-q + s)
When the luminance of M (xi-r, yj = p) is larger than a certain threshold,
W (xi, yj) = B (xi + pr, yj-q + s)
This process is performed for all the areas of the work area for one screen (i = 0 to n−1, j = 1 to m−1). Based on this result, the cut-out image is positioned at the pasting position by the cut-out image generation / registration means 113 (123) and displayed on the liquid crystal display 20. In this embodiment, the threshold value is set to 128. According to the above coordinate calculation, the cut-out image can be displayed several times faster than the synthesis speed (coordinate calculation speed) at the time of normal image synthesis processing.

[Example 12] Example of Automatic Determination of Resolution of Synthetic Image Since the data capacity of the image is large and the capacity of the memory is limited, when the image is recorded in the memory, the image (data) is subjected to JPEG compression processing. The recorded image (data) is subjected to decompression processing and reproduced at the time of recording and reproduction.
In the image compression processing, when the resolution is increased, the compression rate is increased, but the processing time is longer, so that the recording speed is slower. When the resolution is lower, the compression rate is lower, but the processing time is shorter, so the recording speed is faster.
Digital cameras are usually configured so that the user can select the resolution, but the resolution of the cropped image and background image of the image to be processed depends on the use of the product, so the resolution of the composite image Options are diverse.

  In this embodiment, the resolution is adjusted to the lowest resolution in order to record the synthesized image quickly, and the resolution of the cutout image memory 14A and the resolution of the image recorded in the background image buffer (DRAM 13) are compared by the automatic resolution determination means. The resolution is determined as the resolution of the combined image, and the combined image is compressed by the compression / expansion circuit 15 based on the determined resolution and recorded in the flash memory 14.

Also, the coordinate calculation at the time of combining the cutout image and the background image is different from the display coordinates in the case of the preview display. That is, the coordinates at the time of the synthesizing process are as follows: the moving amount is vertical p dots and the horizontal q dots, and the moving amounts from the center of the pasting position are r and s.
W (xi, yj) = M (xi−r, yj−s) × B (xi, yj)
+ (255-M (xi-r, yj-s)
× C (xi + pr, yj + q−s)) / 255
The processing is performed over the entire area of the work area for one screen (i = 0 to n−1, j = 1 to m−1) to perform synthesis.
Although the embodiments of the present invention have been described above, the present invention is not limited to the above embodiments, and it goes without saying that various modifications can be made.

FIG. 1 is a block diagram illustrating a configuration example of a digital camera as one embodiment of a photographing device of the present invention. FIG. 2 is a diagram illustrating a configuration example of a key input unit illustrated in FIG. 1. FIG. 2 is an explanatory diagram of a processing mode of the digital camera shown in FIG. 1. FIG. 3 is a block diagram illustrating a configuration example of an image processing unit. FIG. 4 is an explanatory diagram illustrating a configuration example of a link table. FIG. 9 is an explanatory diagram of a synthesis example of a captured image and a clipped image. 6 is a flowchart illustrating an example of a pattern generation / registration processing operation. FIG. 8 is an explanatory diagram of a pattern generation / registration process based on the flowchart of FIG. 7. 6 is a flowchart illustrating an example of a cut-out image generation / registration processing operation. FIG. 10 is an explanatory diagram of a cutout image generation / registration process based on the flowchart shown in FIG. 9. FIG. 10 is an explanatory diagram of a cutout image generation / registration process based on the flowchart shown in FIG. 9. 3 is a flowchart illustrating an example of a cut-out image synthesis processing operation of the digital camera illustrated in FIG. FIG. 13 is an explanatory diagram of a cut-out image synthesizing process based on the flowchart shown in FIG. 12. FIG. 13 is an explanatory diagram of a cut-out image synthesizing process based on the flowchart shown in FIG. 12. 6 is a flowchart illustrating an example of a pattern generation / registration processing operation. FIG. 16 is an explanatory diagram of a pattern generation / registration process based on the flowchart shown in FIG. 15. 3 is a flowchart illustrating an example of a pasting and combining process operation of the digital camera illustrated in FIG. 1. FIG. 18 is an explanatory diagram of a composite image generation process based on the flowchart shown in FIG. 17. 9 is a flowchart illustrating an example of a cutout image generation / registration processing operation. FIG. 20 is an explanatory diagram of a cutout image generation / registration process based on the flowchart shown in FIG. 19. 2 is a flowchart illustrating an example of a guide frame display processing operation of the digital camera illustrated in FIG. 1, which takes an ID photo frame using an ID photo frame. FIG. 22 is an explanatory diagram of an ID photographing process based on the flowchart shown in FIG. 21. FIG. 9 is an explanatory diagram illustrating an example of generating a mask pattern by contour extraction. 9 is a flowchart illustrating an example of a pattern generation operation based on a designated distance. 6 is a flowchart illustrating an operation example of a pattern generation unit based on a drawn pattern. FIG. 26 is an explanatory diagram of mask patterning of a pattern drawn based on the flowchart shown in FIG. 25.

Explanation of reference numerals

1 imaging lens (imaging means)
2 CCD (imaging means)
14. Flash memory (captured image storage means, captured still image storage means)
14A cutout image memory (frame image storage means, image cutout pattern storage means)
14B Mask pattern memory (mask image storage means, pattern storage means)
16 signal generator (display control means, storage control means)
20 Liquid crystal display (display means)
21 CPU (display control means, storage control means)
24 key input section (key input means)
112, 122 Pattern generation / registration means (pattern generation means)
113, 123 Cutout image generation / registration means (cutout image generation means)
114 Through image combining means (image combining means, display control means)
124 Recorded image synthesizing means (image synthesizing means, display control means)

Claims (22)

Imaging means for imaging a subject;
A frame image storage unit that stores a frame image to be combined with a captured image captured by the imaging unit;
A frame image reproducing unit for reproducing a desired one from among the frame images stored in the frame image storing unit as a reproduced frame image,
Synthesizing means for generating a synthesized image obtained by synthesizing the captured image and the reproduction frame image,
An imaging device comprising:   A captured image storage unit configured to store a captured image captured by the imaging unit; and an imaging playback unit configured to play back a desired one of the captured images stored in the captured image storage unit as a playback image, 2. The imaging apparatus according to claim 1, wherein the synthesizing unit generates a synthesized image obtained by synthesizing the reproduced captured image reproduced by the captured image reproducing unit and the reproduced frame image.   The imaging apparatus according to claim 1, further comprising a display unit configured to display the composite image.   The imaging apparatus according to claim 2, further comprising an adjustment unit configured to adjust a relative position of the captured image in the composite image.   2. The image processing apparatus according to claim 1, further comprising: a mask image storing unit configured to store a mask image for masking the captured image; and a generating unit configured to combine the captured image and the mask image to generate the frame image. The imaging device according to any one of claims 1 to 4.   The imaging device according to claim 1, further comprising a composite image storage unit configured to store the composite image. Imaging means for imaging a subject and outputting a captured image;
A captured image storage unit that stores a captured image output from the imaging unit;
Pattern storage means for storing a plurality of patterns;
Pattern selection means for selecting a desired one from a plurality of patterns stored in the pattern storage means,
A cutout image generation unit that cuts out a corresponding portion of the captured image stored in the captured image storage unit using the pattern selected by the pattern selection unit to generate a cutout image,
An image synthesizing unit, comprising: a cutout image generated by the cutout image generating unit; Imaging means for imaging a subject and outputting a captured image;
A captured image storage unit that stores a captured image output from the imaging unit;
Pattern generation means for generating a pattern from a stored image stored in the captured image storage means,
Pattern storage means for storing the pattern generated by the pattern generation means,
A cutout image generation unit that cuts out a corresponding portion of the captured image stored in the captured image storage unit using a pattern stored in the pattern storage unit to generate a cutout image,
An image synthesizing unit, comprising: a cutout image generated by the cutout image generating unit;   The imaging apparatus according to claim 8, wherein the pattern generation unit generates a pattern by extracting a contour in the captured image. There is a designating means for designating the shooting distance,
9. The imaging apparatus according to claim 8, wherein the pattern generation unit generates a pattern by extracting an image portion in the captured image corresponding to a shooting distance specified by the specification unit.   The imaging apparatus according to claim 8, wherein the pattern generation unit generates a pattern by binarizing the captured image.   The imaging apparatus according to claim 8, wherein the pattern generation unit generates a pattern by calculating a luminance component of the captured image. Imaging means for imaging a subject;
A captured image storage unit that stores a captured image output from the imaging unit;
Drawing data input means for inputting drawing data;
Pattern generation means for generating a pattern from the drawing data input by the drawing data input means,
Pattern storage means for storing the pattern generated by the pattern generation means,
A cutout image generation unit that cuts out a corresponding portion of the captured image stored in the captured image storage unit using a pattern stored in the pattern generation unit to generate a cutout image,
An image synthesizing unit, comprising: a cutout image generated by the cutout image generating unit;   14. The imaging apparatus according to claim 13, wherein the drawing unit performs drawing using a cursor or a pointing device. Imaging means for imaging a subject and outputting a captured image;
Display means for displaying a captured moving image output from the imaging means;
Image cutout pattern storage means for storing an image cutout pattern;
Display control means for combining and displaying the image clipping pattern stored in the image clipping pattern storage means on the captured moving image displayed on the display device,
Key input means,
A captured still image storage unit that stores a captured still image output from the imaging unit,
A storage control unit that stores the captured still image in the captured still image storage unit when the key input unit is operated while the captured moving image and the cutout image pattern are displayed on the display unit by the display control unit; When,
An imaging device comprising: The storage control means,
Using the image clipping pattern displayed on the display unit, a corresponding portion of the captured still image is cut out to generate a cutout still image, and the cutout still image is stored in the captured still image storage unit. The imaging device according to claim 15. Playback means for playing back the captured still image stored in the captured still image storage means;
A cutout image generation unit that cuts out a corresponding portion of the captured still image reproduced by the reproduction unit using the image cutout pattern stored in the image cutout pattern storage unit to generate a cutout image,
Image synthesizing means for synthesizing the cut-out image generated by the cut-out image generating means with a desired image,
The imaging device according to claim 15, comprising: Imaging means for imaging a subject and outputting a captured image;
A captured still image storage unit that stores a captured still image output from the imaging unit,
Display means for displaying a captured moving image output from the imaging means;
Pattern storage means for storing a pattern,
A cutout image generation unit that cuts out a corresponding portion of the captured image stored in the captured image storage unit using a pattern stored in the pattern storage unit to generate a cutout image,
Display control means for combining and displaying the cutout image generated by the cutout image generation means on the captured moving image displayed on the display device,
Key input means,
A storage control unit that stores the captured still image in the captured still image storage unit when the key input unit is operated while the captured moving image and the cutout image are displayed on the display unit by the display control unit; ,
An imaging device comprising:   19. The imaging apparatus according to claim 18, wherein the storage control unit stores the captured still image and the clipped image displayed on the display unit in the captured still image storage unit.   19. The imaging apparatus according to claim 18, further comprising a movement instruction unit configured to instruct the cutout image displayed on the captured moving image to a desired position in a display screen by the display control unit. . Imaging means for imaging a subject and outputting a captured image;
A captured image storage unit that stores a captured image output from the imaging unit;
Display means for displaying a captured image stored in the captured image storage means,
Pattern storage means for storing a pattern,
Display control means for displaying a pattern stored in the pattern storage means on a captured image displayed on the display means,
Moving means for moving the pattern displayed on the captured image by the display control means to a desired position in a display screen;
A cutout image generating unit that cuts out a corresponding portion of the captured image using the pattern moved by the moving unit to generate a cutout image,
Image synthesizing means for synthesizing the cut-out image generated by the cut-out image generating means with a desired image,
An imaging device comprising: A step of capturing a subject to obtain a captured image, and a step of storing the captured image,
Generating a pattern from the stored captured image, storing the pattern, and generating a cutout image by cutting out a corresponding portion of the stored captured image using the stored pattern. ,
Combining the cutout image with a desired image.

Images