JP5266701B2 - Imaging apparatus, subject separation method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus, a subject separating method, and a program capable of separating, from a background, a subject that secures a space with a depth. <P>SOLUTION: When a photographer designates an address position of a club head of a subject 51 as an A point (focus position closest to the photographer) and designates focus positions thereafter in order from the side closest to the photographer like B point and C point as illustrated in Fig. 2(b), a digital camera segments a space 52 between A and C, extracts an edge portion for separating the subject 51 surrounded with the focus positions A, B and C and separates a background and the subject 51. <P>COPYRIGHT: (C)2009,JPO&amp;INPIT

Description

  The present invention relates to a focus area automatic adjustment and subject separation technique in an imaging apparatus such as a digital camera.

  There are known techniques for transmitting or inputting an image taken with a conventional digital camera to a personal computer and capturing the captured image on the personal computer side, and then processing the image or transforming the entire image. In addition, a technique for separating a main subject from a background of a captured image captured on a personal computer and a technique for adding a picture or a character to an image are also known.

  In addition, as a technique for capturing an image with a digital camera and extracting a main subject from the background of the captured image, a low-frequency filter is provided between the CCD and the signal processing unit that processes the output signal of the CCD, thereby eliminating the background of the image. Can be separated from the main subject (see, for example, Patent Document 1).

  In Patent Document 1, the aperture is opened, the shutter speed is set so as to obtain an appropriate exposure, the desired subject is focused, and the obtained image signal is passed through a low-frequency cutoff filter. A digital camera configured to obtain the image (signal) is disclosed, and an imaging unit that configures the digital camera includes a lens, a diaphragm, a shutter, and the like, and projects input light onto the imaging unit. The imaging unit is a CCD and converts an optical signal into an electrical signal. The low frequency cutoff filter is between the two and blocks the low frequency component of the optical signal. Accordingly, a technique is disclosed in which image information of a blurred area is cut using a low-frequency cutoff filter, and a main subject (a focused subject) is cut out from the background.

  However, the technique disclosed in Patent Document 1 has a problem that a new hardware (low-frequency cutoff filter) must be provided in the digital camera and that a cut-out image does not always include only the main subject. It was.

  On the other hand, as a technique for taking an image with a digital camera without providing hardware such as a low-frequency cutoff filter and extracting a main subject from the background of the taken image, when the extraction means performs subject extraction, The imaging condition is controlled by the condition control means, and an image suitable for subject extraction is acquired from the imaging means (the extraction means extracts only the focused subject in the image through attention area extraction and contour extraction). At this time, depending on the extraction result of the extraction means, further feedback is applied to change the imaging condition. A technique for obtaining a new composite photograph by combining an extracted subject image with a second image as a background is disclosed.

The point of the invention described in Patent Document 2 is that, as described in the paragraph “0049” of Patent Document 2, “the imaging condition control means 102 controls the imaging conditions in order to obtain an optimal image for subject extraction. Furthermore, depending on the extraction result, the imaging condition can be changed again by feedback. By changing the imaging condition, an image can be easily obtained for subject extraction. As described in paragraphs “0050” and “0051” of Patent Document 2, “Generally, when the aperture is opened on the telephoto side, the depth of field becomes shallower and the distance of the focus is short. In an image with a deep depth of field, if you open the aperture and reduce the depth of field, the depth of focus decreases, and only the subject is focused. "If it is such an image, it will be easy to extract the subject from the background with a relatively small amount of calculation, that is, it can be implemented even with scarce hardware such as a digital camera." Is. "
JP-A-10-260462 JP 2006-128754 A

  However, in the technique described in Patent Document 2, the above-described paragraphs “0050” and “0051” of Patent Document 2 and “Image capturing apparatus according to Claim 1” of Claim 3 of Patent Document 2 are described. The imaging conditions include an aperture, a shutter speed, and sensitivity, and the imaging apparatus is characterized in that control is performed so that the depth of field of an image obtained by photographing these three parameters is shallow. As described above, since the depth of field of the subject to be extracted is controlled to be shallow, there is a problem in that a deep space cannot be secured for the subject to be extracted.

  The present invention relates to an imaging apparatus capable of adjusting a depth of field by specifying a plurality of focus positions at the time of shooting and separating a subject image securing a deep space from a background, and a subject separation method by automatically adjusting a focus area And for the purpose of providing a program.

In order to solve the above-mentioned problem, the invention described in claim 1 is a through-displayed image in an imaging apparatus having an imaging system that captures a subject image, converts it into image data, and displays the subject image through during shooting. a focus position specifying means specifying multiple portions of the desired object as a focus position from among the among the plurality of focus positions specified by the focus position specifying means, the focus position closer to the imaging element in the position of the focus lens In order to detect the edges of the portions corresponding to a plurality of focus positions in sequence, the focus lens is moved in order from the focus lens, and an aperture adjustment unit that adjusts the aperture value is output in accordance with the adjustment by the aperture adjustment unit. that detects the edge is a high-frequency component, has a plurality of specified by the focus position specifying means follower A depth of field obtaining means for obtaining a depth of field that can detect the edge of all Kas position, when detecting the photographing instruction, all designated as captured photographed images from the imaging system by the focus position specifying means Image recording means for associating a focus position and recording in a memory; and image reproduction means for reproducing and displaying an image recorded in the memory and retrieving a focus position associated with the image when a reproduction instruction is detected; A filtering unit that identifies a subject image based on the extracted focus position and acquires an edge by scanning through an image displayed by the image reproducing unit; and an area outside the edge acquired by the filtering unit. As the low frequency component region, the background and the subject image And boundary display means for displaying defining a field, characterized by comprising a.

According to a second aspect of the present invention, in the first aspect of the invention, when an instruction from the outside is detected, a subject image separated from the background is stored in association with the captured image stored in the memory. And a recording control means for controlling the recording to be performed.

According to a third aspect of the present invention, in an imaging apparatus having an imaging system that captures a subject image, converts it into image data, and displays the subject image through at the time of shooting, a desired subject is selected from the through-displayed images. A focus position designating step for designating a plurality of parts as a focus position, and moving the focus lens in order from the focus position closer to the image sensor at the focus lens position among the plurality of focus positions designated by the focus position designating step. In order to detect the edges of the portions corresponding to a plurality of focus positions in sequence, an adjustment step for adjusting the aperture value and an edge that is a high-frequency component output in accordance with the adjustment in the adjustment step are detected. , All the focus positions specified by the focus position specifying step A depth of field obtaining step of obtaining depth of field capable of detecting the edge for, when detecting the photographing instruction, all of the focus position designated by the focus position designating step and the captured photographed images from the imaging system An image recording step for associating and recording in the memory; an image reproduction step for reproducing and displaying the image recorded in the memory and detecting a focus position associated with the image when a reproduction instruction is detected; and image reproduction For the image displayed by the step, a subject image is specified based on the extracted focus position, and a filtering step for acquiring an edge by scanning by filtering, and a region outside the edge acquired by the filtering step is a low frequency region. As the component region, the back Characterized in that it comprises a boundary display step of displaying to define the boundaries of the object image with.

According to a fourth aspect of the present invention, there is provided a program for controlling a computer of an image pickup apparatus having an image pickup system that takes in a subject image, converts it into image data, and displays the subject image through during shooting. , the focus position specifying means for specifying multiple portions of the desired object as a focus position from among the images being through display, among the plurality of focus positions specified by the focus position specifying means, the image pickup element at the position of the focus lens In order to detect the edges of the portions corresponding to a plurality of focus positions, the aperture adjustment means for adjusting the aperture value in order to move the focus lens in order from the focus position closest to the position, and the adjustment by the aperture adjustment means Detecting an edge which is a high-frequency component output in response to the focus position finger Depth of field acquisition means for acquiring a depth of field that can detect the edge of all the plurality specified focus position by means, upon detecting the photographing instruction, by the the captured photographed images from the imaging system focus position specifying means Image recording means for associating all designated focus positions and recording them in the memory. When a reproduction instruction is detected, the image recorded in the memory is reproduced and displayed, and the focus position associated with the image is extracted. An image reproducing unit, a filtering unit that identifies a subject image based on the extracted focus position and acquires an edge by scanning by filtering, and an outside of the edge acquired by the filtering unit. As the low frequency component area, Display means for displaying to define the boundaries of the serial background and the object image, to function as a.

According to the present invention, at the time of shooting, a plurality of focus areas are specified, and the subject's distance is automatically adjusted in the range of the farthest position from the side closest to the camera to ensure the depth of field of the subject. The subject can be recognized as a space having a depth, the edges of the subject and the background can be detected, and the subject image can be separated from the background. It can also be applied to moving images and continuous shooting.
For example, when shooting a golf swing as shown in FIG. 2, it is difficult to recognize the entire subject due to the background or surrounding people, but applying the present invention ensures the depth of field of the subject. Since a subject image with a deep space can be separated, the subject image can be raised by separating (cutting out) from the background.

  1 is a block diagram illustrating a configuration example of an electronic circuit of a digital camera used as an example of an imaging apparatus according to an embodiment of the present invention. In FIG. 1, the digital camera 100 is in focus by moving the zoom lens 12-1 and the focus lens 12-2 for moving the zoom lens 12-1 and performing the optical zoom operation in the photographing mode which is the basic mode. An AF driving unit 11-2 that performs an operation, a lens optical system 12 that constitutes an imaging lens including a zoom lens 12-1 and a focus lens 12-2, a CCD 13 that is an imaging element, a timing generator (TG) 14, and a vertical driver 15. Sample hold circuit (S / H) 16, A / D converter 17, color process circuit 18, DMA (Direct Memory Access) controller 19, DRAM interface (I / F) 20, DRAM 21, control unit 22, VRAM controller 23, VRAM 24, digital video encoder 25, display unit 26, JPEG circuit 7, storage memory 28, and a subject separation unit 29 and the key input unit 30,.

  In the monitoring state in the photographing mode, when there is an optical zoom instruction, the zoom drive unit 11-1 drives a zoom lens drive motor (not shown) based on a control signal from the control unit 22 to light the zoom lens 12-1. By moving back and forth along the axis, the magnification itself of the image formed on the CCD 13 is changed.

  The AF drive unit 11-2 drives a focus lens drive motor (not shown) to move the focus lens 12-2.

  Then, the CCD 13, which is an image pickup device disposed behind the photographing optical axis of the optical system 12 constituting the image pickup lens, is scanned and driven by a timing generator (TG) 14 and a vertical driver 15, and images are formed at regular intervals. A photoelectric conversion output corresponding to the optical image is output for one frame.

  The CCD 13 is a solid-state imaging device that captures a two-dimensional image of a subject, and typically captures an image of several tens of frames per second. The imaging element is not limited to a CCD, and may be a solid-state imaging device such as a CMOS (Complementary Metal Oxide Semiconductor).

  The photoelectric conversion output is appropriately gain-adjusted for each primary color component of RGB in the state of an analog value signal, sampled and held by a sample hold circuit (S / H) 16, and digital data by an A / D converter 17. Color process processing including image interpolation processing and γ correction processing is performed in the color processing circuit 18 to generate a luminance signal Y and color difference signals Cb and Cr, and the DMA (Direct Memory Access). ) Is output to the controller 19.

  The DMA controller 19 uses the luminance signal Y and the color difference signals Cb and Cr output from the color process circuit 18 by using the composite synchronization signal, the memory write enable signal, and the clock signal from the color process circuit 18 as well as a DRAM interface. DMA transfer is performed via the (I / F) 20 to the DRAM 21 used as a buffer memory.

  The control unit 22 is responsible for the overall control operation of the digital camera 100, and is executed by the CPU or MPU (hereinafter referred to as CPU) and the CPU including a subject separation program with multiple focal point designation as described later. A program storage memory storing programs and the like, and a RAM used as a work memory, and after the DMA transfer of the luminance and color difference signals to the DRAM 21 are completed, the luminance and color difference signals are transferred from the DRAM 21 via the DRAM interface 20. Read and write to the VRAM 24 via the VRAM controller 23.

  The control unit 22 also extracts a processing program and menu data corresponding to each mode stored in a program storage memory such as a flash memory in response to the status signal from the key input unit 30, and Execution control of each function, for example, subject separation operation by multiple focal point designation according to the present invention, operation control of the zoom lens at the time of optical zoom, electronic zoom, through display, automatic focusing, shooting, recording, and recording The execution control such as reproduction / display of the selected image, the display control of the function selection menu when selecting the function, the display control of the setting screen, and the like are performed.

  For example, the control unit 22 cuts out the space between the deepest focus area from the frontmost focus area from the focus area (focus area) designated by the user, and specifies the focus area while focusing on the front focus area. The depth of field is secured so that the edges can be detected in all the focus areas.

  The digital video encoder 25 periodically reads the luminance and color difference signals from the VRAM 24 via the VRAM controller 23, generates a video signal based on these data, and outputs the video signal to the display unit 26.

  As described above, the display unit 26 functions as a monitor display unit (electronic finder) in the shooting mode. By performing display based on the video signal from the digital video encoder 25, the display unit 26 receives from the VRAM controller 23 at that time. The image based on the stored image information is displayed on the liquid crystal monitor screen in real time.

  In response to the focus instruction, the controller 22 immediately stops the path from the CCD 13 to the DRAM 21 after the completion of the DMA transfer of the luminance and color difference signals for one frame that are being fetched from the CCD 13 to the DRAM 21 at that time. Transition to saved state.

  In this stored recording state, the control unit 22 transmits the luminance and color difference signals for one frame written in the DRAM 21 through the DRAM interface 20 to each of Y, Cb, and Cr components of 8 pixels × 8 pixels horizontally. Are read in units called basic blocks and written into a JPEG (Joint Photograph Cording Experts Group) circuit 27. The JPEG circuit 27 uses ADCT (Adaptive Discrete Cosine Transform), a Huffman code which is an entropy coding system. Data compression is performed by processing such as conversion. The obtained code data is read out from the JPEG circuit 27 as a data file of one image, and is recorded and stored in the storage memory 28. In addition, the controller 22 activates the path from the CCD 13 to the DRAM 21 again with the completion of the compression processing of the luminance and color difference signals for one frame and the writing of all the compressed data to the storage memory 28.

  In the playback mode, which is the basic mode, the control unit 22 selectively reads out the image data recorded in the storage memory 28, and is compressed by a procedure completely opposite to the procedure of data compression in the image shooting mode by the JPEG circuit 27. The expanded image data is decompressed, the decompressed image data is expanded and stored in the VRAM 24 via the VRAM controller 23, and then periodically read out from the VRAM 24 to generate a video signal based on the image data. The display unit 26 reproduces and outputs.

  The JPEG circuit 27 supports a plurality of compression ratios, and a mode for storing data corresponding to the compression ratio includes a mode corresponding to a high resolution (generally called high definition, fine, normal, etc.) with a low compression ratio. There is a low-resolution (commonly called economy) mode with a high compression rate. It also supports high to low pixel counts.

  The storage memory 28 includes a recording medium such as a built-in memory (flash memory), a hard disk, or a removable memory card, and stores and records image data, photographing information, and the like.

  The subject separation unit 29 extracts an edge (high frequency component) from the image data of the subject image formed by the imaging lens, that is, from the output of the color process circuit 18 using a high-pass filter. When the control unit 22 secures a depth of field at which edges can be detected in all the focus areas designated by the user, an edge portion for separating the background and the subject image is extracted, and the boundary area is defined as a line ( Display as a line). For example, an edge is searched from the edge of the screen toward the center, and an area outside the detected edge is set as a low-frequency component to be erased or edited. The subject separation unit 29 adjusts so that an edge (high frequency component) can be detected according to the subject by changing the filter table in which the threshold value of the high-pass filter is registered in advance. As the filter table, for example, a filter table described in JP-A-2001-215404 can be employed.

  The key input unit 30 generates the operation signal of the mode dial 2, the cursor key 5, the SET key 6, the zoom button, the shutter key 8, and the power button 9, and the operated key when the key is operated. 22 includes a key processing unit (not shown) to be sent to 22.

  The shutter key 8 performs a release operation at the time of shooting, has a stroke of two steps, and performs auto focus (AF) and automatic exposure (AE) by the first step operation (half-pressed state). And a shooting instruction signal for performing a shooting process in the second-stage operation (fully pressed state). When the “subject separation mode” is selected, the focus instruction signal is not generated even if the first stage operation (half-pressed state) is performed.

<Embodiment 1>
(When separating subject images in real time during shooting)
In this embodiment, it is assumed that the digital camera 100 stores a plurality of filter tables used for filtering by the subject separation unit 29 in the program storage memory or the storage memory 100. A high-pass filter value corresponding to the aperture value is set in the filter table. The set value is different for each filter table.

  In the present embodiment, when the user designates a plurality of focus points at the time of shooting, that is, when a through image is displayed, the digital camera 100 secures a depth of field that can detect an edge at all of the designated focus points, and subjects are separated. The unit 29 extracts an edge portion for separation from the background, and displays the boundary area in a line in real time. For example, an edge is searched from the edge of the screen toward the center, and an area outside the detected edge is set as a low-frequency component to be erased or edited.

FIG. 2 is an explanatory diagram of a subject separation method based on designation of multiple focal points according to the present invention, taking a golf swing as an example.
In FIG. 2A, the photographer designates the address position of the club head of the subject 51 as point A (the closest focus position (focusing point)), and hereinafter, as shown in FIG. In this order, point B (grip position when the club is raised), point C (club head position when the club is raised), and focus position are designated. In the example shown in the figure, the point C is the most probable point, but the designated focus position is not limited to three points. That is, at least two points of the foremost part and the rearmost part are indispensable for securing the depth, but a plurality of points of two or more points can be designated.

  In the example of FIG. 2, point A is the foremost focus position, point C is the foremost focus position, and when the photographer performs a focus position designation end operation (for example, pressing the SET key 6), the digital camera 100 A space 52 between A and C is cut out. A cut-out space 52 between A and C indicates the depth of field.

  The focus position is set so that when the through image is displayed in the subject separation mode, the portion that the photographer wants to specify as the focus position is within the focusing frame displayed at the center of the camera. This can be specified by half-pressing the shutter key 8. Further, pressing down the SET key 6 can be a focus position designation end operation. The focus position is specified in order from the most likely part of the subject image to be separated from the background (point A in the example of FIG. 2), and the most likely part (point C in the example of FIG. 2). When the designation of the focus position is finished, the focus position designation end operation is performed, and in this case, the focus instruction signal is not generated even if the shutter key 8 is pressed halfway. This occurs when the shutter key 8 is pressed halfway after that.

Note that the method for specifying the focus position is not limited to the above-described method (that is, a method for designating in order from the most likely part of the subject to be separated from the background and specifying the most likely part).
For example, on the assumption that the subject is located in the center of the camera, the focus area may be designated in order from the low frequency area in the peripheral portion.

  In the embodiment, as described above, the focus position is specified by half-pressing the shutter key 8 and the SET6 key is pressed by the focus position specifying end operation. However, the present invention is not limited to this. For example, the focus position designation may be the operation of the cursor key 5, the designation confirmation may be a depression operation of the SET key 6, and the focus position designation end operation may be a half-press operation of the shutter key 8 (at the same time, a focusing instruction signal is generated).

Further, when the focus position is designated, it is desirable that the designated portion is surrounded by a frame or highlighted so that the photographer can easily understand the designated focus position.
Furthermore, when the focus position is designated, the focus lens 12-2 is moved and an evaluation value is acquired in the same manner as when performing a normal AF operation, and among the designated focus positions displayed on the liquid crystal monitor screen 4, It is also possible to display whether or not the depth of field is within the same range by displaying a focusing frame of substantially the same color for each focus position where the focusing point is approximate.

  Note that if the color of the frame surrounding the background and the focus position is substantially the same, the frame cannot be seen, so the frame color may be different from the background color. A publicly known method for changing the color depending on the background, for example, a method of obtaining the degree of approximation of the color of the focusing frame set in advance by design and the color of the background, and changing the color when they are close is used. be able to. Since the color is determined by the color component of the data, it is possible to identify the color. For example, the average value of the color of the pixels around the frame line is compared with the color of the frame line, and if it is closer than the predetermined value, it is changed to the second color can do.

  FIG. 3 is a flowchart illustrating an operation example of the control unit 22 and the subject separation unit 29 under the control of the subject separation program by specifying a plurality of in-focus points when the subject image is separated at the time of shooting (through image display). Hereinafter, a description will be given based on FIGS. As the filter table used in step S5, a standard table in which standard values are set is used in the first loop, and a table specified by the control unit 22 is used from the next loop.

  In FIG. 3, when the photographer operates the power button 9 to turn on the main power supply and operates the mode dial 1 to set the photographing mode, the control unit 22 causes the digital camera 100 to start displaying a through image. Control each component to display a through image and display a menu (not shown) for selecting a shooting mode type to prompt the user to select a shooting mode. When the photographer selects “Subject Separation Mode”, a step Proceeding to S2, when a photographing mode other than the “subject separation mode” is selected, the processing proceeds to the selected photographing mode (step S1).

  The photographer designates the range of the space to be separated (cut out) from the background in the through image by designating a plurality of focus positions by half-pressing the shutter key 8 as described above (see FIG. 2). Since the focus position designation end operation is performed after the end, the control unit 22 sequentially stores the designated focus position (for example, coordinate values on the screen) in the DRAM 20, and proceeds to step S3 when the focus position designation end operation is detected. At this time, the control unit 22 controls the display unit 26 so as to clearly indicate the designated focus position by surrounding it with a frame (step S2).

  When the focus position designation end operation is detected, the control unit 22 controls the AF driving unit 11-2 to drive the focus lens 12-2 based on the foremost focus position stored in the primary storage memory. A focus value is focused and an aperture value is acquired (step S3).

  Next, based on the acquired aperture value, for each focus position acquired in step S3, the operation of detecting edges (high frequency components) while automatically adjusting the aperture value from the front to the back is repeated and specified in step S2. When the edge detection of all focus positions is completed, the depth of field that can detect the edge detection of all focus positions is secured, and the process proceeds to step S5. Otherwise, the operation of this step is repeated (step S4).

  The control unit 22 controls the subject separation unit 29 to extract the edge portion for separating the subject surrounded by the focus position designated as the background in the through image display state, and separates the background and the subject. The subject separation unit 29 searches for an edge from the edge of the screen toward the center by filtering based on the values in the filter table, and acquires the boundary portion between the background and the subject as a low frequency component in the area outside the detection edge (step S5) The boundary area between the background and the subject image is displayed as a line in real time (so that the contour of the subject is traced with a line) (step S6).

  When the photographer separates the subject image from the background by looking at the displayed boundary, the photographer performs a background erasure instruction operation (for example, depressing the SET key 6), so the control unit 22 receives the signal from the key input unit 30. If a background erase instruction operation is performed, the process proceeds to step S8, and if not, the process proceeds to step S9 (step S7).

  When there is a background erasure instruction operation, the control unit 22 controls the subject separation unit 29 to hold an image obtained by erasing an area outside the boundary from the captured image as a low frequency component region in a predetermined region of the DRAM 20 (temporary storage). ) And then proceed to step S9. Thereby, a desired subject image can be separated from the background (step S8).

  The control unit 22 displays a message asking whether or not the subject image to be separated may be an image shown in the boundary area, that is, a message asking whether or not the boundary area needs to be changed, and prompts the photographer to give instructions. If the photographer performs a boundary area change instruction operation (for example, depressing the SET key 6), the process proceeds to step S10, and if the shutter key 8 is fully depressed, the process proceeds to step S11 (step S9).

  When the photographer performs a boundary area change instruction operation, the control unit 22 changes the filter table used by the subject separation unit 29 and returns to step S5 (step S10).

  When the shutter key 8 is fully pressed, the photographing process is executed, and the photographed image and the image (the subject image separated from the background) held in the predetermined area of the DRAM 20 in step S8 are associated and recorded in the storage memory 28, The photographing of one image is finished (step S11).

  With the operation shown in the flowchart of FIG. 3, the digital camera 100 can recognize the subject as a space having a depth in real time, detect the edge of the subject image and the background image, and separate the subject image. In addition, the subject separation method by specifying multiple in-focus points described in the present embodiment can be applied to moving images and continuous shooting.

<Embodiment 2>
(When separating the subject image after shooting)
In the first embodiment, the subject image is separated from the background at the time of shooting (when the through image is displayed). However, the subject image can also be separated from the background of the shot image after shooting. In this case, only step S5 and the subsequent steps in the flowchart of FIG. 3 are different, and the others are the same as those in the first embodiment. Hereinafter, description will be made based on the flowchart of FIG.

  FIG. 4 is a flowchart illustrating an operation example of the control unit 22 and the subject separation unit 29 under the control of the subject separation program by specifying a plurality of focal points when the subject image is separated from the photographed image after photographing. Hereinafter, a description will be given based on FIGS. As the filter table used in step T7, a standard table in which standard values are set is used in the first loop, and a table specified by the control unit 22 is used from the next loop.

  In FIG. 4, when the photographer operates the power button 9 to turn on the main power supply and operates the mode dial 1 to set the photographing mode, the control unit 22 of the digital camera 100 starts to display a through image. Control each component to display a through image and display a menu (not shown) for selecting a shooting mode type to prompt the user to select a shooting mode. When the photographer selects “Subject Separation Mode”, a step Proceeding to T2, when a photographing mode other than the “subject separation mode” is selected, the processing proceeds to the processing of the selected photographing mode (step T1).

  The photographer designates the range of the space to be separated (cut out) from the background in the through image by designating a plurality of focus positions by half-pressing the shutter key 8 as described above (see FIG. 2). Since the focus position designation end operation is performed at the end, the control unit 22 sequentially stores the designated focus position (for example, coordinate values on the screen) in the DRAM 20, and proceeds to step T3 when the focus position designation end operation is detected. At this time, the control unit 22 controls the display unit 26 so as to clearly indicate the designated focus position with a frame (step T2).

  When the focus position designation end operation is detected, the control unit 22 controls the AF driving unit 11-2 to drive the focus lens 12-2 based on the foremost focus position stored in the primary storage memory. A focus value is focused and an aperture value is acquired (step T3).

  Next, based on the acquired aperture value, for each focus position acquired in step T3, the operation of detecting an edge (high frequency component) while automatically adjusting the aperture value from the front to the back is repeated, and specified in step T2. When the edge detection of all focus positions is completed, the depth of field that can detect the edge detection of all focus positions is secured, and the process proceeds to step T5. Otherwise, the operation of this step is repeated (step T4).

  When the shutter key 65 is fully pressed, the photographing process is executed, and the control unit 22 associates the photographed image with the focus position held in the DRAM 20 and records it in the storage memory 28 to finish photographing one image. (Step T5).

  Next, when the photographer operates the mode dial 1 to set the playback mode, the control unit 22 reads the captured image recorded in the storage memory 28 in step T4, and sends the image data subjected to the playback process to the display unit 30. Display. The focus position recorded in association with the image is sent to the subject separation unit 29 (step T6).

  In addition, the control unit 22 controls the subject separation unit 29 to extract an edge portion for separating the subject surrounded by the recorded focus position and separate the background and the subject. The subject separation unit 29 searches for an edge from the edge of the screen toward the center by filtering based on the values in the filter table, and acquires the boundary portion between the background and the subject as a low frequency component in the area outside the detection edge (step T7) The boundary area between the background and the subject image is displayed in a line in real time (so that the contour of the subject is traced with a line) (step T8).

  When the photographer separates the subject image from the background by looking at the displayed boundary, the photographer performs a background erasure instruction operation (for example, depressing the SET key 6), so the control unit 22 receives the signal from the key input unit 30. If a background erase instruction operation is performed, the process proceeds to step T10, and if not, the process proceeds to step T11 (step T9).

  When there is a background erasure instruction operation, the control unit 22 controls the subject separation unit 29 to hold an image obtained by erasing an area outside the boundary from the captured image as a low frequency component region in a predetermined region of the DRAM 20 (temporary storage). ) And then proceed to Step T13. Thereby, a desired subject image can be separated from the background (step T10).

  The control unit 22 displays a message asking whether or not the subject image to be separated may be an image shown in the boundary area, that is, a message asking whether or not the boundary area needs to be changed, and prompts the photographer to give instructions. If the photographer performs a boundary area change instruction operation (for example, pressing the SET key 6), the process proceeds to step T12, and if not, the process proceeds to step T13 (step T11).

  When the photographer performs a boundary area change instruction operation, the control unit 22 changes the filter table used by the subject separation unit 29 and returns to step T7 (step T12).

  The original image of the reproduced image recorded in the storage memory 28 and the image (subject image separated from the background) held in the predetermined area of the DRAM 20 in step T10 are associated with each other and stored in the storage memory 28 (step T13). .

  When the next image is reproduced, the user performs the next image reproduction instruction operation (for example, the cursor key 5 is operated to the right (→) or the lower (↓) and then the SET key 6 is pressed). The signal from the key input unit 30 is checked, and if there is a next image reproduction instruction operation, the process returns to the separated step T7 to finish photographing one image (step T14).

  With the operation shown in the flowchart of FIG. 4, the digital camera 100 can reproduce a captured image even in a captured image (still image) and recognize the subject as a space having a depth, and the subject image and the background edge can be recognized. It is possible to detect and separate the subject image.

  In step S8 in FIG. 3 and step T10 in FIG. 4, the area outside the boundary is erased as a low frequency component area from the captured image. However, instead of erasing the low frequency component area, the high frequency component area is erased. It may be specified as a subject image area and extracted.

1 is a block diagram illustrating a configuration example of an electronic circuit of a digital camera used as an example of an imaging apparatus according to an embodiment of the present invention. It is explanatory drawing of the to-be-photographed object separation method by multiple focal point designation | designated based on this invention. 3 is a flowchart illustrating an operation example at the time of shooting of the digital camera according to the first embodiment. 10 is a flowchart illustrating an operation example at the time of shooting of the digital camera according to the second embodiment.

Explanation of symbols

6 SET key 8 Shutter key 22 Control unit 29 Subject separation unit 100 Digital camera (electronic camera)

Claims (4)

In an imaging apparatus having an imaging system that captures a subject image, converts it into image data, and displays the subject image through during shooting,
Focus position designation means for designating a plurality of parts of a desired subject as focus positions from the through-displayed image;
Of the plurality of focus positions designated by the focus position designation means, the focus lens is moved in order from the focus position closer to the image sensor at the focus lens position, and the focus positions are sequentially corresponded to. Aperture adjustment means for adjusting the aperture value to detect the edge of the part,
An object field for detecting an edge that is a high-frequency component output in accordance with the adjustment by the diaphragm adjusting unit, and acquiring a depth of field at which the edge can be detected for all the focus positions specified by the focus position specifying unit Depth acquisition means;
Detecting an imaging instruction, an image recording means for associating and recording in a memory a captured image captured from the imaging system and all focus positions designated by the focus position designation means;
Detecting a reproduction instruction, reproduces and displays the image recorded in the memory, and retrieves a focus position associated with the image;
Filtering means for specifying an object image based on the extracted focus position for an image displayed by the image reproducing means, and scanning to obtain an edge by filtering;
Boundary display means for demarcating and displaying the boundary between the background and the subject image, with the area outside the edge acquired by the filtering means as a low frequency component area;
An imaging apparatus comprising: And a recording control unit that controls to store a subject image separated from the background in association with a captured image stored in the memory when an instruction from the outside is detected. Item 2. The imaging device according to Item 1. In an imaging apparatus having an imaging system that captures a subject image, converts it into image data, and displays the subject image through during shooting,
A focus position designating step of designating a part of a desired subject as a focus position from the through-displayed image;
Of the plurality of focus positions designated in the focus position designation step, the focus lens is moved in order from the focus position closest to the image sensor at the focus lens position, and the focus is sequentially handled. An adjustment step to adjust the aperture value to detect the edge of the part;
Depth of field for detecting an edge that is a high-frequency component output in accordance with the adjustment in the adjustment step, and acquiring a depth of field that can detect the edge for all of the focus positions specified in the focus position specification step An acquisition step;
Detecting a shooting instruction, an image recording step of associating and recording in a memory a captured image captured from the imaging system and all the focus positions specified in the focus position specifying step;
When a reproduction instruction is detected, an image reproduction step of reproducing and displaying the image recorded in the memory and extracting a focus position associated with the image;
A filtering step of identifying an object image based on the extracted focus position and acquiring an edge by scanning by filtering for the image displayed by the image reproduction step;
A boundary display step for defining and displaying a boundary between the background and the subject image, using a region outside the edge acquired by the filtering step as a low frequency component region;
An object separation method comprising: A program for controlling a computer of an imaging apparatus having an imaging system that captures a subject image, converts it into image data, and displays the subject image through during shooting,
The computer,
Focus position designation means for designating a plurality of parts of a desired subject as focus positions from the through-displayed image;
Of the plurality of focus positions designated by the focus position designation means, the focus lens is moved in order from the focus position closer to the image sensor at the focus lens position, and the focus positions are sequentially corresponded to. Aperture adjustment means for adjusting the aperture value to detect the edge of the part,
An object field for detecting an edge that is a high-frequency component output in accordance with the adjustment by the diaphragm adjusting unit, and acquiring a depth of field at which the edge can be detected for all of the focus positions specified by the focus position specifying unit. Depth acquisition means,
An image recording unit that records a captured image captured from the imaging system and all the focus positions specified by the focus position specifying unit in association with each other when a shooting instruction is detected;
An image reproducing means for reproducing and displaying the image recorded in the memory and extracting a focus position associated with the image upon detecting a reproduction instruction;
Filtering means for specifying an object image on the basis of the extracted focus position for an image displayed by the image reproducing means, and scanning to obtain an edge by filtering;
Display means for defining and displaying a boundary between the background and the subject image, with a region outside the edge acquired by the filtering means as a low frequency component area;
Program to function as.

Images