JP4246240B2 - Imaging apparatus and imaging program - Google Patents

Description

  The present invention relates to an imaging apparatus and an imaging program, and more particularly to a technique for performing image processing on a desired object (subject) from image information obtained by imaging.

  When it is desired to obtain a still image obtained by performing image processing on a desired object such as a person from an image captured by a digital camera, the processing is conventionally performed by an external device such as a personal computer using dedicated processing software. It was.

  However, in the above-described prior art, since image processing is performed by an external device such as a personal computer, it is not possible to perform image processing on a desired object in real time, and on the spot where the image-processed still image is taken. There was a problem that it could not be confirmed.

  The present invention has been made to solve the above-described conventional problems, and an object thereof is to provide an imaging device and an imaging program capable of performing image processing on a desired object in real time.

The present invention relates to a moving image compression image encoding unit that compresses and encodes a captured image as a moving image, and moving image information obtained in the compression encoding process. Means for extracting information about at least one subject contained therein;
Display means for displaying the captured image; and means for specifying a desired subject from the displayed images while displaying information on the extracted subject on the display means, the designated The photographing condition is set for the subject, and a still image is photographed.

  Preferred embodiments of the present invention are as follows.

  (1) The image display means displays a marker corresponding to the designated subject, and the means for designating the desired subject corresponds to the desired subject among the markers displayed on the image display means. The subject is specified by selecting the marker to be used.

  (2) Furthermore, an image processing unit is provided that performs desired image processing on the designated subject based on the information about the extracted subject.

  (3) The image processing means can perform a plurality of types of image processing, and can select the type of image processing for each subject.

  (4) Still image compression encoding means for compressing and encoding the captured still image, and a moving image output from the moving image compression encoding means or a still image output from the still image compression encoding means Is recorded on a recording medium.

  (5) It further includes means for generating a moving image at the position of the still image from moving images positioned before and after the still image in time series, and outputting the moving image to the moving image compression encoding means.

  (6) The generation of the moving image at the position of the still image is generated by interpolating the moving image frame using the correlation information of the moving image frames before and after the moving image frame dropped when shooting as the still image. The video will not drop.

  Further, an imaging program according to the present invention includes an imaging procedure for performing imaging with an imaging device, a moving image compression encoding procedure for compressing and encoding a captured image as a moving image, and a process of performing the compression encoding. From the obtained moving image information, a procedure for extracting information on at least one subject included in the moving image, a procedure for displaying the captured image, and information on the extracted subject are displayed and displayed. And a procedure for specifying a desired subject from the captured images and a procedure for setting a shooting condition for the specified subject and shooting as a still image.

  Preferred embodiments of the present invention are as follows.

(1) The image display procedure displays a marker corresponding to the designated subject,
In the procedure for designating the desired subject, the subject is designated by selecting a marker corresponding to the desired subject from the markers displayed in the image display procedure.

  (2) Further, an image processing procedure for performing desired image processing on the designated subject based on the information on the extracted subject is provided.

  (3) The image processing procedure can perform a plurality of types of image processing, and the type of image processing can be selected for each subject.

  (4) At least a still image compression encoding procedure for compressing and encoding the captured still image, a moving image output by the moving image compression encoding procedure, or a still image output by the still image compression encoding procedure And a procedure for recording one on a recording medium.

  (5) The method further includes a procedure of generating a moving image at the position of the still image from moving images positioned before and after the still image in time series and sending the moving image to the moving image compression encoding procedure.

  (6) The procedure for generating the moving image at the position of the still image is generated by interpolating the moving image frame using correlation information of moving image frames before and after the moving image frame that has been dropped as a still image. , Make the video without frame dropping.

  According to the present invention, object information is extracted in the process of compression encoding of a moving image, and image processing is selectively performed on the extracted object, so that efficient image processing can be performed in real time, It becomes possible to quickly and easily confirm a still image that has undergone desired image processing.

  Embodiments of the present invention will be described below with reference to the drawings.

(Overview)
FIG. 1 is a diagram for explaining an outline of an imaging apparatus according to an embodiment of the present invention.

  The digital camera captures moving images at a predetermined rate (for example, 30 frames / second) (FIG. 1A). The captured moving image is compressed by a predetermined compression encoding process (in this case, MPEG compression process), and in the moving image recording mode, the compressed moving image data is recorded on a recording medium such as a hard disk. In MPEG compression processing, image compression is performed using correlation information between frames and within frames, etc., but information on objects (subjects) included in a moving image is obtained from information obtained in the process of compression processing (see FIG. In the example of FIG. 1, information such as an airplane, a car, and a background can be extracted (FIG. 1B).

  By using the extracted object information, the following processing can be performed.

  The processing shown in FIGS. 1C and 1D is for extracting only a designated object and setting shooting conditions, etc., and the outline is as follows.

  The photographer selects an object extraction mode, and selects a desired object from one or more objects (each object is extracted from information obtained during the MPEG compression process) in the moving image displayed on the LCD screen. Specify an object. For example, a desired object on the LCD screen may be specified using a marker. By designating a desired object and then pressing the release switch, shooting conditions such as AE and AF are automatically set for the designated object, and only the designated object is photographed as a still image. The obtained still image data is compressed by a predetermined compression encoding process (here, JPEG compression process), and the compressed still image data is recorded on a recording medium such as a hard disk.

  The process shown in FIG. 1E performs different image processing for each object, and the outline is as follows.

  The photographer selects an editing processing mode and designates desired processing such as background blurring, adaptive tone curve correction, adaptive edge enhancement, and skin color correction described later for each object. Each object can be designated by a marker, and a desired process is designated for each designated object. After performing such a designation operation, by pressing the release switch, an image subjected to the designated process for each designated object is photographed as a still image. The obtained still image data is compressed by a predetermined compression encoding process (JPEG compression process), and the compressed still image data is recorded on a recording medium such as a hard disk.

  FIG. 2 schematically shows processing operations on the CCD when performing the above-described processes in still image shooting. As shown in the figure, the specified process is applied only to the specified object (automobile in the figure), and such processing is performed for the other parts (background part in the figure). Absent.

  Although the outline of the embodiment of the present invention has been described above, as described above, information related to an object is extracted from moving image information obtained in the process of compression encoding such as MPEG compression processing and selected for the object. Thus, since desired image processing (signal processing) is performed, it is possible to efficiently perform image processing only on a desired object in real time. Therefore, it is not necessary to perform image processing using an external device such as a personal computer as in the prior art, and it is possible to easily confirm on the spot where a still image subjected to desired image processing is taken.

(Constitution)
FIG. 3 is a block diagram showing the configuration of the main part of the imaging apparatus according to the embodiment of the present invention.

  The basic configuration of the imaging unit is the same as that of a normal digital camera, and a lens unit 11 (comprising a photographic lens, a diaphragm, a shutter, and the like) and a CCD (imaging) that photoelectrically converts a subject image formed by the lens unit 11. Element 12, an analog processing circuit 13 that performs amplification and A / D conversion of a photoelectric conversion signal from the CCD 12, a driving circuit 14 that drives the lens unit 11 and the CCD 12, and the like.

  Each unit of the digital camera is controlled by a CPU 15, and a switch unit 16 including a release switch, a switch for designating an object, a switch for designating various image processes, and the like is connected to the CPU 15.

  The output of the analog processing circuit 13 is connected to the digital processing circuit 17, and various digital processes are performed by the digital processing circuit 17. The digital processing circuit 17 is connected to a memory unit 18, an MPEG processing unit 19, a JPEG processing unit 20, and an object detection unit 21.

  The memory unit 18 has a function as a buffer memory, and temporarily stores image data obtained by the imaging unit, and is also used as a work area for various image processing. The MPEG processing unit 19 performs compression coding processing for moving images represented by MPEG2 and MPEG4, and performs image coding, compression, and the like using interframe and intraframe correlation. The JPEG processing unit 20 performs compression encoding processing for still images, and performs image encoding, compression, and the like using intra-frame correlation of still images. The object detection unit 21 extracts object information included in a moving image from information obtained in the course of compression processing by the MPEG processing unit 19.

  By transmitting and receiving various types of information to / from the memory unit 18, MPEG processing unit 19, JPEG processing unit 20, and object detection unit 21 described above, the digital processing circuit 17 performs various processes as described below. .

  That is, the digital processing circuit 17 performs various processes such as a process of extracting an object and an adaptive tone curve correction based on the object information extracted in the compression process by the MPEG processing unit 19. The digital processing circuit 17 also performs signal processing such as luminance level and color tone conversion, white balance, and gamma correction, and processing for decompressing compressed image data. The digital processing circuit 17 feeds back information such as the luminance and frequency component of the selected object to the lens unit 11 via the CPU 15 and the like. Based on the fed back information, shooting conditions such as exposure and focus for the selected object are optimized. Further, as will be described later, the digital processing circuit 17 obtains it during the compression process in order to prevent the moving image from dropping and causing an unnatural display when both the moving image and the still image are captured and recorded. A process of interpolating frames with dropped frames using the correlation information of the obtained moving image is also performed.

  The output of the digital processing circuit 17 is connected to the D / A conversion circuit 22, and an image is displayed on the screen of the LCD 23 based on the signal D / A converted by the D / A conversion circuit 22. On the screen of the LCD 23, it is possible to display either an image obtained by performing signal processing by thinning out pixels of a captured moving image, or an image obtained by performing decompression processing on a compressed moving image. The LCD 23 also displays a marker for designating an object.

  The digital processing circuit 17 can transmit and receive information to and from the HDD 24 and other recording media 25 (FD, memory card, etc.), and image data can be selected by selecting a compression rate and a recording medium. Is recorded in the HDD 24 or other recording medium 25. Image data with a large amount of data such as moving images and images with a low compression rate is mainly recorded in the HDD 24, and image data with a small amount of data such as a still image is mainly recorded in the other recording medium 25. By recording image data on a predetermined recording medium according to the type of image (according to the type of moving image, still image, image processing, etc.), different types of image data can be managed for each recording medium.

(Operation)
FIG. 4 is a diagram showing an outline of the operation in the present embodiment.

  Normally, moving images are captured at a predetermined rate (for example, 30 frames / second), moving images are displayed on the LCD screen (finder screen), and moving image data compressed by MPEG processing is recorded on a recording medium such as a hard disk. To be recorded. Further, object information included in the moving image is extracted from information obtained in the course of the MPEG processing, and information based on the extracted object information is stored in the lens unit (photographing lens) so that the designated object can be imaged under the optimum conditions. , Aperture, shutter, etc.).

  When acquiring a still image, imaging is performed at a desired timing by a release operation, and desired signal processing (signal processing such as object extraction, background blurring, adaptive tone curve correction) is performed on the captured image. Then, a JPEG-processed still image is recorded on the recording medium.

  When shooting a still image, shooting conditions that are different from the shooting conditions for the movie are set (for example, the shooting conditions are optimized for the specified object). It cannot be used as it is as one frame of a moving image corresponding to the image capturing timing. Therefore, in order to prevent a moving image display with dropped frames, the frame dropped portion is interpolated using correlation information obtained in the course of MPEG processing. Specifically, the video frames dropped are interpolated using the correlation information of the video frames before and after the dropped frames. With such a process, even if a still image is captured while a series of moving images is being captured, a frame-dropped portion of the moving image is interpolated and a natural moving image can be obtained.

  FIG. 5 is a diagram illustrating a processing flow of the operation of the imaging apparatus according to the present embodiment.

  When the power is turned on and the CCD is driven (S1), in order to obtain object information from MPEG information, moving images are continuously captured (S2). The obtained moving image data is subjected to MPEG compression processing (S3), and the compressed moving image data is recorded on a recording medium such as a hard disk (S4). The moving image data is subjected to a process of expanding the compressed data or a pixel thinning process (S5), and the moving image is displayed on the viewfinder (LCD screen) (S6). In addition, since the object is specified from the object information obtained by the MPEG processing and the marker is displayed on the screen corresponding to each object, the photographer can select a desired object by a predetermined switch operation. (S7).

  When the still image shooting mode is selected (S8), the following processing is performed.

  In the normal mode, general image editing processing is performed, and brightness, color tone, white balance, gamma correction, and the like are performed (S9).

  In the special editing imaging mode, various processes are performed on the object designated in step S7. In the case of object extraction photography, only the selected object is extracted, and image processing is performed only on the object (S10). In the case of background blur shooting, filter processing is performed only on the selected background image (S11). When adaptive tone curve correction photographing is performed, a tone curve is selected for each selected object (S12), and each object is processed using the selected tone curve (S13).

  The image data of the still image obtained in this way is subjected to JPEG compression processing (S14), or is recorded in the selected recording medium without being compressed in the non-compression mode (S15). , S16).

  FIG. 6 is a diagram showing an outline of the flow of processing in various modes (object extraction mode, background blur mode, and adaptive tone curve correction mode) regarding the operation of the present embodiment.

  In the object extraction mode, based on the object information obtained by MPEG processing, digital processing is performed only for the signal of the specified object among the image data of the captured still image, and other signals are discarded without being processed. It is done. Predetermined digital processing is performed on the designated object, and display on the LCD screen or recording on a recording medium is performed based on the signal subjected to the digital processing.

  In the background blur mode, the blur process is performed only on the background signal among the image data of the captured still image based on the object information obtained by the MPEG process. The background signal subjected to the blur processing is subjected to predetermined digital processing together with signals other than the background, and display on the LCD screen or recording to the recording medium is performed based on the signal subjected to the digital processing.

  In the adaptive tone curve correction mode, a correction process using predetermined tone curves (tone curves A to C) for each object (objects A to C) extracted based on the object information obtained by MPEG processing. Is done. The image data of the still image that has been subjected to tone curve correction for each object is subjected to predetermined digital processing, and is displayed on the LCD screen or recorded on a recording medium based on the digitally processed signal. Is called.

  Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the present invention. Furthermore, the above embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining the disclosed constituent elements. For example, even if several constituent requirements are deleted from the disclosed constituent requirements, the invention can be extracted as an invention as long as a predetermined effect can be obtained.

The figure for demonstrating the outline | summary of embodiment of this invention. The figure which showed typically the processing operation on CCD at the time of performing various processes in still image photography in connection with embodiment of this invention. 1 is a block diagram showing a configuration of a main part of an imaging apparatus according to an embodiment of the present invention. The figure which showed the outline | summary of operation | movement of the imaging device which concerns on embodiment of this invention. The figure which showed the flow of the process in the imaging device which concerns on embodiment of this invention. The figure which showed the outline | summary of the flow of the process in various modes about the imaging device which concerns on embodiment of this invention.

Explanation of symbols

11 ... Lens unit 12 ... CCD
13 ... Analog processing circuit 14 ... Drive circuit 15 ... CPU
DESCRIPTION OF SYMBOLS 16 ... Switch part 17 ... Digital processing circuit 18 ... Memory part 19 ... MPEG processing part 20 ... JPEG processing part 21 ... Object detection part 22 ... D / A conversion circuit 23 ... LCD
24 ... HDD
25. Other recording media

Claims (14)

In an imaging device that performs imaging with an imaging element,
Moving image compression encoding means for compressing and encoding the captured image as a moving image;
Means for extracting information on at least one subject contained in the moving image from the moving image information obtained in the compression encoding process;
Display means for displaying the captured image;
Displaying information on the extracted subject on the display means, and means for designating a desired subject from the displayed images,
An imaging apparatus, wherein a shooting condition is set for the designated subject and a still image is shot. The image display means displays a marker corresponding to the designated subject,
2. The imaging according to claim 1, wherein the means for designating the desired subject designates the subject by selecting a marker corresponding to the desired subject from the markers displayed on the image display means. apparatus.   The imaging apparatus according to claim 1, further comprising an image processing unit that performs desired image processing on the designated subject based on the information about the extracted subject.   The imaging apparatus according to claim 3, wherein the image processing unit can perform a plurality of types of image processing, and can select a type of image processing for each subject.   Still image compression encoding means for compressing and encoding the captured still image, and at least one of a moving image output from the moving image compression encoding means or a still image output from the still image compression encoding means The image pickup apparatus according to claim 1, wherein the image pickup device is recorded on a recording medium.   6. The apparatus according to claim 5, further comprising a unit that generates a moving image at a still image position from a moving image positioned before and after the still image in time series, and outputs the generated moving image to the moving image compression encoding unit. The imaging device described in 1.   The moving image is generated at the position of the still image by interpolating the moving image frame using the correlation information of the moving image frames before and after the dropped moving image frame when shooting as the still image, and there is no frame dropping. The imaging apparatus according to claim 6, wherein the imaging apparatus is a moving image. On the computer,
An imaging procedure for imaging with an imaging element;
A moving image compression encoding procedure for compressing and encoding a captured image as a moving image;
A procedure for extracting information on at least one subject included in the moving image from the moving image information obtained in the compression encoding process;
A procedure for displaying the captured image;
Displaying information about the extracted subject and specifying a desired subject from the displayed images;
An imaging program for setting a shooting condition for the designated subject and shooting a still image. The image display procedure displays a marker corresponding to the designated subject,
9. The imaging according to claim 8, wherein the step of designating the desired subject designates the subject by selecting a marker corresponding to the desired subject from the markers displayed in the image display procedure. program.   The imaging program according to claim 8, further comprising an image processing procedure for performing desired image processing on the designated subject based on information on the extracted subject.   The imaging program according to claim 10, wherein the image processing procedure is capable of a plurality of types of image processing, and the type of image processing can be selected for each subject.   Recording at least one of a still image compression encoding procedure for compressing and encoding the photographed still image and a moving image output by the moving image compression encoding procedure or a still image output by the still image compression encoding procedure The imaging program according to claim 8, further comprising a procedure for recording on a medium.   13. The method according to claim 12, further comprising a step of generating a moving image at a still image position from a moving image positioned before and after the still image in time series, and sending the generated moving image to the moving image compression encoding procedure. The imaging program described.   The procedure for generating the moving image at the position of the still image is generated by interpolating the moving image frame using the correlation information of the moving image frames before and after the dropped moving image frame when shooting as the still image. The imaging program according to claim 13, wherein the moving image has no image.

Images