JP4639965B2 - Imaging apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an image pickup apparatus such as a digital camera, and more particularly to an image pickup apparatus suitable for use when performing a panning shot using a moving body as a subject, and an image processing method and program used for the image pickup apparatus.

  For example, when shooting a moving body such as a train or a car, a so-called “panning” technique is used. This is because the photographer pans (moves) the camera in accordance with the movement of the moving object while the shutter speed is set slower than normal, thereby obtaining an image that emphasizes the sense of speed of the moving object by flowing the background. Is the method.

Conventionally, proposals have been made for effectively obtaining such a panning image. For example, in Patent Document 1, in an electronic camera having a camera shake correction function, when a photographer performs a panning shot, the image processing for camera shake correction is changed according to the direction of the panning shot to It is disclosed to obtain an image that naturally flows.
JP 2002-305682 A

  However, panning requires advanced techniques and it is difficult to obtain the image intended by the photographer. In this case, even if a technique for changing camera shake correction according to the direction of panning is used as in Patent Document 1, if the movement of the moving body does not match the movement of the camera, the moving body will be blurred. Or a non-main background will be shot without blurring.

  Therefore, an object of the present invention is to provide an imaging apparatus, an image processing method, and a program that can easily obtain a panning image in which the movement of the moving body is emphasized without accurately capturing the camera in accordance with the movement of the moving body. Is to provide.

According to a first aspect of the present invention, an imaging device that captures a subject continuously to acquire a plurality of image data and a moving body are identified by comparing the plurality of image data acquired by the imaging unit. A selecting unit that selects image data including the moving object specified by the specifying unit and image data not including the moving object among the plurality of image data acquired by the photographing unit; and the selection Synthesis means for extracting a partial image corresponding to the moving object from the image data including the moving object selected by the means, and synthesizing the extracted partial image with image data not including the moving object. It is characterized by .

Further, the invention according to claim 2 is characterized in that the synthesizing unit synthesizes the extracted partial image at the center of the image data not including the moving body.

The invention according to claim 3 further includes image processing means for performing a blurring process on the image data that does not include the moving object selected by the selection means, and the synthesizing means includes the extracted partial image. Is synthesized with image data that does not include the moving object subjected to the blurring process by the image processing means .

Further, the invention according to claim 4 further includes a detecting unit that detects a moving direction of the moving body specified by the specifying unit, and the image processing unit performs the processing on the image data not including the moving body. It is characterized by performing a blurring process in the moving direction of the moving body.

In addition, the invention according to claim 5 further includes detection means for detecting a moving speed of the moving body specified by the specifying means, and the image processing means performs the processing on the image data not including the moving body. A blurring process is performed in an amount corresponding to the moving speed of the moving object.

According to a sixth aspect of the present invention, there is provided an imaging apparatus for capturing a plurality of image data by continuously capturing a subject and moving each subject portion by comparing the plurality of image data acquired by the photographing unit. A detecting means for detecting a speed; a selecting means for selecting a subject part having the slowest moving speed among a plurality of subject parts whose moving speed is detected by the detecting means; and a subject part selected by the selecting means. And an image processing means for performing a sharpening process.

The invention according to claim 7 further includes mode selection means for selecting a camera follow mode and a camera fixed mode as an operation mode at the time of panning shooting, and the camera follow mode is set in the selection means. In this case, the subject portion having the slowest moving speed is selected, and when the camera fixing mode is set, the subject portion having the fastest moving speed is selected, and the image processing means is set to the camera following mode. If the camera fixing mode is set, the subject portion selected by the selection means is subjected to a blurring process. It is characterized by giving.

Further, the invention according to claim 8 further performs shooting by setting the shutter speed to a predetermined value or less when the camera following mode is set, and shutter speed when the camera fixing mode is set. Is set to a predetermined value or more, and shooting is performed.

An image processing method according to claim 9 of the present invention specifies a moving body by comparing a plurality of image data acquired by the imaging step of acquiring a plurality of image data by continuously imaging a subject and the imaging step. A selecting step for selecting image data including the moving body specified by the specifying step and image data not including the moving body among the plurality of image data acquired by the photographing step; Extracting a partial image corresponding to the moving object from the image data including the moving object selected in the selection step, and combining the extracted partial image with image data not including the moving object. It is characterized by.

According to a tenth aspect of the present invention, there is provided an image processing method in which a subject is continuously photographed to obtain a plurality of image data, and a plurality of pieces of image data obtained by the photographing step are compared with each other. A detecting step for detecting a moving speed, a selecting step for selecting a subject part having the slowest moving speed among a plurality of subject parts for which the moving speed has been detected by the detecting step, and a subject part selected by the selecting step And an image processing step for performing a sharpening process on the image.

According to an eleventh aspect of the present invention, there is provided a program for capturing a plurality of image data obtained by continuously photographing a subject and obtaining a plurality of image data by using a computer mounted on the imaging device. Specifying means for specifying a moving body by comparison of the above, image data including the moving body specified by the specifying means among a plurality of image data acquired by the photographing means, and image data not including the moving body Extracting a partial image corresponding to the moving body from the selection means for selecting and the image data including the moving body selected by the selecting means, and combining the extracted partial image with image data not including the moving body. It is characterized by functioning as a synthesis means.

According to a twelfth aspect of the present invention, there is provided a program for capturing a plurality of image data obtained by continuously photographing a subject and obtaining a plurality of image data by using a computer mounted on the imaging apparatus. Detecting means for detecting the moving speed of each subject part by comparison, selecting means for selecting a subject part having the slowest moving speed among a plurality of subject parts whose moving speed is detected by the detecting means, and the selection The image processing means is configured to function as a sharpening process on the subject portion selected by the means.

  As described above, according to the present invention, when shooting a moving body as a main subject, a panning image in which the movement of the moving body is emphasized can be taken without accurately capturing the camera with the movement of the moving body. Can be easily obtained.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
FIG. 1 is a diagram showing an external configuration when a digital camera is taken as an example of the imaging apparatus according to the first embodiment of the present invention. FIG. 1 (a) is mainly a front configuration, and FIG. It is a perspective view which mainly shows the structure of a back surface.

  The digital camera 1 has a photographing lens 3, a self-timer lamp 4, an optical finder window 5, a strobe light emitting unit 6, a microphone unit 7 and the like on the front surface of a substantially rectangular thin plate-like body 2 on the upper surface (for the user). On the right end side, a power key 8 and a shutter key 9 are provided.

  The power key 8 is a key operated every time the power is turned on / off, and the shutter key 9 is a key for instructing a photographing timing at the time of photographing.

  Also, on the back of the digital camera 1, a shooting mode (R) key 10, a playback mode (P) key 11, an optical viewfinder 12, a speaker unit 13, a macro key 14, a strobe key 15, a menu (MENU) key 16, a ring A key 17, a set (SET) key 18, a display unit 19, and the like are provided.

  The shooting mode key 10 is operated automatically from the power-off state to automatically turn on the power and shift to the still image shooting mode. On the other hand, by repeatedly operating from the power-on state, the still image mode and the moving image mode are switched. Set cyclically. The still image mode is a mode for photographing a still image. The moving image mode is a mode for shooting a moving image.

  The shutter key 9 is commonly used for these photographing modes. That is, in the still image mode, a still image is taken at the timing when the shutter key 9 is pressed. In the moving image mode, shooting of a moving image is started at a timing when the shutter key 9 is pressed, and shooting of the moving image is ended when the shutter key 9 is pressed again.

  When the playback mode key 11 is operated from the power-off state, the playback mode key 11 is automatically turned on to enter the playback mode.

  The macro key 14 is operated when switching between normal shooting and macro shooting in the still image shooting mode. The strobe key 15 is operated when switching the light emission mode of the strobe light emitting unit 6. The menu key 16 is operated when selecting various menu items including the panning mode. The ring key 17 is integrally formed with item selection keys in the up, down, left, and right directions, and the set key 18 located in the center of the ring key 17 sets the item selected at that time. To operate.

  The display unit 19 is composed of a color liquid crystal panel with a backlight, and displays a through image on the monitor as an electronic viewfinder in the photographing mode, and reproduces and displays the selected image and the like in the reproduction mode.

  Further, the digital camera 1 is provided with an optical zoom function, and the enlargement ratio of the image can be changed by physically changing the focal length by operating the zoom keys 20a and 20b. Of the zoom keys 20a and 20b, one zoom key 20a is for the telephoto end and is used when the zoom magnification is changed to the telephoto side. The other zoom key 20b is for the wide end and is used when the zoom magnification is changed to the wide angle side.

  Although not shown, the digital camera 1 has a memory card slot for attaching / detaching a memory card used as a recording medium, a serial interface connector for connecting to an external personal computer, etc., for example, USB (Universal). Serial Bus) connector and the like are provided.

  FIG. 2 is a block diagram showing an electronic circuit configuration of the digital camera 1.

  In the digital camera 1, a lens optical system 22 including a focus lens and a zoom lens (not shown) constituting the photographing lens 3 is provided so as to be movable within a predetermined range in the optical axis direction by driving a motor 21. . A CCD (charge coupled device) 23 that is an image sensor is disposed behind the optical axis of the lens optical system 22. The CCD 23 receives light from each part of the subject input through the photographing lens 3 and outputs an electrical signal corresponding to the intensity of the light.

  In the recording mode, which is the basic mode, the CCD 23 is scanned and driven by a timing generator (TG) 24 and a driver 25, and outputs a photoelectric conversion output corresponding to a light image formed at regular intervals for one screen. The photoelectric conversion output of the CCD 23 is appropriately gain-adjusted for each primary color component of RGB in the state of an analog value signal, sampled and held by the sample hold circuit 26, and converted into digital data by the A / D converter 27. The

  Then, the image processing circuit 28 performs image processing including pixel interpolation processing and γ correction processing to generate a digital luminance signal Y and color difference signals U and V (Cb, Cr), and DMA (Direct Memory Access). ) Output to the controller 29.

  The DMA controller 29 once uses the luminance signal Y and the color difference signals U and V output from the image processing circuit 28 by using the composite synchronization signal, the memory write enable signal, and the clock signal from the image processing circuit 28 once. And the DMA transfer to the DRAM 31 used as the buffer memory via the DRAM interface (I / F) 30.

  The control unit 32 controls the entire digital camera 1 and is constituted by a microcomputer including a CPU, a ROM storing an operation program executed by the CPU, and a RAM used as a work memory 32a. Is done.

  After the DMA transfer of the luminance and color difference signals to the DRAM 31, the control unit 32 reads the luminance and color difference signals from the DRAM 31 via the DRAM interface 30 and writes them to the VRAM 34 via the VRAM controller 33.

  The digital video encoder 35 periodically reads the luminance and color difference signals from the VRAM 34 via the VRAM controller 33, generates a video signal based on these data, and outputs the video signal to the display unit 19.

  As described above, the display unit 19 functions as a monitor display unit (electronic finder) at the time of shooting. By performing display based on the video signal from the digital video encoder 35, the display unit 19 captures from the VRAM controller 33 at that time. An image based on the image information is displayed in real time.

  As described above, when the image at that time is displayed in real time as the monitor image on the display unit 19, for example, when the shutter key 9 is pressed at a timing at which still image shooting is desired, a trigger signal is generated.

  In response to the trigger signal, the control unit 32 immediately stops the path from the CCD 23 to the DRAM 31 immediately after the DMA transfer of the luminance and color difference signals for one screen captured from the CCD 23 to the DRAM 31 is completed. , Transition to the record storage state.

  In this recording and storage state, the control unit 32 outputs the luminance and color difference signals for one frame written in the DRAM 31 to 8 pixels × 8 pixels for each of Y, Cb, and Cr components via the DRAM interface 30. The data is read out in units called basic blocks, written in a JPEG (Joint Photographic Coding Experts Group) circuit 37, and this JPEG circuit 37 uses an ADCT (Adaptive Discrete Cosine Transform), which is an entropy coding system. Data compression is performed by processing such as conversion.

  The obtained code data is read out from the JPEG circuit 37 as a data file of one image and written in the recording memory 38. The memory 38 includes a memory card that is detachably mounted as a recording medium in addition to an internal memory such as a flash memory built in the main body in advance. With the compression processing of the luminance and color difference signals for one frame and the completion of writing all the compressed data to the memory 38, the control unit 32 activates the path from the CCD 23 to the DRAM 31 again.

  The control unit 32 is further connected with an audio processing unit 39, a USB interface (I / F) 40, and a strobe driving unit 41.

  The sound processing unit 39 includes a sound source circuit such as a PCM sound source, digitizes the sound signal input from the microphone unit (MIC) 7 when recording sound, and performs a predetermined data file format such as MP3 (MPEG-1 audio layer). 3) Data compression is performed in accordance with the standard to create an audio data file and send it to the memory 38. On the other hand, at the time of audio reproduction, the audio data file read from the memory 38 is uncompressed and converted into an analog signal. The sound is output through a speaker unit (SP) 13 provided on the back side.

  The USB interface 40 performs communication control when image data and other information are transmitted / received to / from another information terminal device such as a personal computer connected by wire via a USB connector. The strobe drive unit 41 charges a strobe capacitor (not shown) at the time of shooting, and then drives the strobe light emitting unit 6 to flash based on control from the control unit 32.

  In addition to the shutter key 9 described above, the key input unit 36 includes a power key 8, a shooting mode key 10, a playback mode key 11, a macro key 14, a strobe key 15, a menu key 16, a ring key 17, and a set key. 18, zoom keys 20 a and 20 b and the like, and signals accompanying these key operations are sent directly to the control unit 32.

  Further, when shooting a moving image instead of a still image, when the shutter key 9 is pressed, the above-described JPEG circuit 37 compresses the captured moving image using a technique such as motion-JPEG (Joint Photographic Experts Group). To the memory 38. When the shutter key 9 is operated again, the recording of the moving image data is finished.

  On the other hand, in the playback mode which is the basic mode, the control unit 32 selectively reads out the image data recorded in the memory 38 and is compressed by a procedure completely opposite to the procedure of data compression in the recording mode by the JPEG circuit 37. Decompress image data. The decompressed image data is held in the DRAM 31 via the DRAM interface 30, and then the content held in the DRAM 31 is stored in the VRAM 34 via the VRAM controller 33. The image data is periodically read out from the VRAM 34. A video signal is generated and reproduced and output by the display unit 19.

  If the selected image data is not a still image but a moving image, the multiple frames of still image data that make up the moving image data are played back and displayed sequentially in chronological order, and all the still image data is played back. For example, the top still image data is displayed until the next playback instruction is given.

  Next, the operation of the first embodiment of the present invention will be described.

  Assume that a panning is performed with a moving object such as an automobile as a main subject. Usually, when performing panning, an advanced technique is required such as pressing the shutter key while panning the camera in accordance with the movement of the moving object. The present invention proposes a method for easily obtaining a panning image in which the movement of a moving object is emphasized without requiring such an advanced technique.

  The method will be briefly described with reference to FIGS.

  (1) First, as shown in FIG. 3, the digital camera 1 is fixed to a moving body (automobile in this example) with a tripod or the like, and a through image is displayed.

  FIG. 4 shows an example of image data of each frame sequentially obtained when the moving body moves in one direction (in this example, from right to left). Note that P1 to P4 in the figure indicate the positions of the moving bodies, and P2 is the front position of the digital camera 1. These image data are displayed on the monitor 19 as a through image on the monitor.

  (2) The image data of each frame sequentially obtained during the through image display is compared, the moving body is specified from the change of the luminance and the color signal, and the moving direction of the moving body is detected.

  (3) Here, when the moving body comes to the front of the digital camera 1, that is, at the timing when the photographer presses the shutter key 9 when it comes to the position P2, as shown in FIG. An image [A] including a moving object is acquired.

  (4) The moving body is monitored while performing comparison between frames even after the image [A] is recorded. During this time, when it is confirmed that the moving body has advanced to the position P4 and out of the frame, an image [B] of only the background is acquired as shown in FIG.

  (5) As post-processing, as shown in FIG. 6, an image of the moving body part [C] is extracted from the image [A]. Further, the blurring process is performed on the image [B], which is the background image, in the moving direction of the moving object.

  (6) Finally, the image [D] after the blurring process and the image of the moving body part [C] are synthesized. Thereby, the synthesized image [E] can be obtained as a panning image.

  Hereinafter, specific processing for creating a panning image by such a method will be described in detail. Note that the processes shown in the following flowcharts are executed when the control unit 32, which is a microcomputer, reads a predetermined program recorded in a recording medium such as a ROM or a RAM.

  7 and 8 are flowcharts showing the panning shooting process of the digital camera 1 in the first embodiment. This panning shooting process is executed, for example, when the panning mode is set by operating the menu key 16.

  When the panning mode is set, the control unit 32 provided in the digital camera 1 first displays the image data of each frame sequentially obtained from the CCD 23 as well as captures these images and temporarily stores them in the work memory 32a. (Steps A11 and A12). Then, the control unit 32 takes out the image data of the immediately preceding two frames from the work memory 32a and compares the pixels (step A13), thereby moving the pixel portion in which the luminance or color signal changes to a predetermined value or more to the moving object. (Step A14), and the moving direction of the moving body is detected based on the pixel change information (step A15).

  When there are a large number of pixel portions where the luminance and color signals change to a predetermined value or more, the pixel portion that occupies the largest proportion among them is specified as a moving object.

  Here, when the photographer presses the shutter key 9 while confirming the moving object in the image (Yes in Steps A16 and A17), the control unit 32 sets a preset shutter speed (exposure). The captured image data is taken in accordance with (time) and is recorded in the memory 38 as an image [A] (steps A18 and A19). As shown in FIG. 5, the image [A] is image data including a moving body that is a main subject.

  In the same manner as described above, the control unit 32 sequentially captures the image data of each frame during the through image display into the work memory 32a (steps A20 and A21), and compares the pixels of the image data of the immediately preceding two frames. Then, the presence or absence of a moving body is determined (steps A22 to A24). When it is confirmed that the moving body is out of the frame and disappears from the image data (No in Step A25), the control unit 32 is photographed in accordance with a preset shutter speed (exposure time) in the same manner as in Step A18. The captured image data is captured and recorded in the memory 38 as an image [B] (step A26). As shown in FIG. 5, the image [B] is image data of only the background.

  When the image [A] and the image [B] are obtained in this way, the control unit 32 extracts the data of the moving body part [C] from the image [A] (step A27). The data extraction of the moving body portion [C] is performed by comparing the image [A] and the image [B].

  Subsequently, the control unit 32 performs an image blurring process in the moving direction of the moving body on the image [B] that is the background image to create an image [D] (step A28). The blur amount at this time may be a fixed amount, or may be arbitrarily set by the user (photographer) on the blur amount selection screen 51 as shown in FIG. In the example of FIG. 9, the blurring amount can be arbitrarily set between levels 1 to 3 by selecting the items 52 to 54 with a cursor or the like. In this case, if the amount of blur increases in the order of levels 1 to 3, the greater the blur amount, the higher the background sink effect, and the moving body appears to move faster.

  Next, the control unit 32 creates an image [E] by superimposing the data of the moving body part [C] on the image [D] after the blurring process (step A29), and the image [E] is panned. An image is recorded in the memory 38 (step A30). As shown in FIG. 6, the image [E] is a composite of a moving object and a background image, and the background image is subjected to blurring processing in the moving direction.

  As described above, according to the first embodiment of the present invention, even if the camera is not accurately matched to the movement of the moving object that is the main subject, the background is flowed to emphasize the movement of the moving object. A captured image can be easily obtained.

  In the first embodiment, each of the image [A] (image including a moving object), the image [B] (background image), and the image [E] (panning image) is recorded in the memory 38. However, it is not always necessary to record these images. For example, only the image [A] obtained at the shutter timing is recorded in the memory 38, and the images [B] and [E] are temporarily stored in the work memory 32a and displayed on the screen as necessary. Then, it may be recorded selectively in the memory 38 in accordance with a user instruction.

  The user has pressed the shutter key 9 to determine the position of the moving body (main subject). For example, it is determined that the moving body has come to the center position of the image, and the image at that time is displayed as the image [A]. It is good also as a structure which acquires automatically as.

(Second Embodiment)
Next, a second embodiment of the present invention will be described.

  In the first embodiment, the blur amount of the background image is fixed or arbitrarily specified by the user. However, in the second embodiment, an optimal blur amount is determined according to the moving speed of the moving object. It is characterized by.

  FIG. 10 and FIG. 11 are flowcharts showing the panning shooting process of the digital camera 1 in the second embodiment. This panning shooting process is executed, for example, when the panning mode is set by operating the menu key 16.

  The basic processing flow is the same as that in the first embodiment (FIGS. 7 and 8). That is, when the panning mode is set, the control unit 32 provided in the digital camera 1 first displays the image data of each frame sequentially obtained from the CCD 23 as well as taking in these images to obtain a work memory 32a. Is temporarily stored (steps B11 and B12). Then, the control unit 32 takes out the image data of the immediately preceding two frames from the work memory 32a and compares the pixels (step B13), so that a portion where the luminance or color signal changes to a predetermined value or more is used as a moving body. Specify (step B14).

  When there are a large number of pixel portions where the luminance and color signals change to a predetermined value or more, the pixel portion that occupies the largest proportion among them is specified as a moving object.

  Here, in the second embodiment, the moving direction of the moving body is detected based on the information on the pixel change, and further, the moving body is determined based on the information on the angle of view and the distance to the main subject by AF (automatic focusing). Is calculated (step B15).

  If the photographer presses the shutter key 9 while confirming the moving body in the image (Yes in steps B16 and B17), the control unit 32 follows the preset shutter speed (exposure time). The captured image data is taken in and recorded in the memory 38 as an image [A] (steps B18 and B19). As shown in FIG. 5, the image [A] is image data including a moving body that is a main subject.

  In the same manner as described above, the control unit 32 sequentially captures the image data of each frame during the through image display into the work memory 32a (steps B20 and B21), and compares the pixels of the image data for the immediately preceding two frames. Then, the presence / absence of a moving body is determined (steps B22-24). Then, when it is confirmed that the moving body is out of the frame and disappears from the image data (No in Step B25), the control unit 32 is photographed according to a preset shutter speed (exposure time) in the same manner as in Step B18. The obtained image data is taken in and recorded as image [B] in the memory 38 (step B26). As shown in FIG. 5, the image [B] is image data of only the background.

  When the image [A] and the image [B] are obtained in this way, the control unit 32 extracts the data of the moving body part [C] from the image [A] (step B27). The data extraction of the moving body portion [C] is performed by comparing the image [A] and the image [B].

  Here, the control unit 32 refers to the blur control table 61 as shown in FIG. 12 and selects an optimal blur amount according to the moving speed of the moving object (step B28). In this blur control table 61, an optimum blur amount with respect to the moving speed is set in advance. The blur control table 61 is provided in, for example, a RAM in the control unit 32. Then, the control unit 32 performs a blurring process on the image [B], which is the background image, according to the moving direction of the digit moving body and the selected blurring amount to create the image [D] (step B29). .

  Next, the control unit 32 creates the image [E] by superimposing the data [C] of the moving body part on the image [D] after the blurring process (step B30), and flows the image [E]. The captured image is recorded in the memory 38 (step B31). As shown in FIG. 6, the image [E] is a composite of a moving object and a background image, and the background image is subjected to blurring processing in the moving direction.

  As described above, according to the second embodiment of the present invention, not only the moving direction of the moving object serving as the main subject but also the background image including the moving speed element is blurred, thereby making the moving object feel more speedy. An effectively expressed panning image can be obtained.

  Here, the actual moving speed of the moving object is calculated based on the angle of view and the distance to the subject. However, the moving speed of the main subject portion on the image data may be simply calculated based on the pixel change information. good.

  Further, as described in the first embodiment, the image [A] (image including a moving body), the image [B] (background image), and the image [E] (panning shot image) are necessarily recorded. For example, only the image [A] obtained at the shutter timing may be recorded in the memory 38, and the image [B] and the image [E] may be selectively recorded in the memory 38 according to a user instruction.

  Even if the user does not press the shutter key 9 to determine the position of the moving object (main subject), for example, it is determined that the moving object has come to the center position of the image, and the image at that time is displayed as the image [A]. It is good also as a structure which acquires automatically as.

(Third embodiment)
Next, a third embodiment of the present invention will be described.

  In the first and second embodiments, shooting is performed with the camera fixed. In the third embodiment, shooting is performed including shooting when panning without fixing the camera. We propose a method that can easily obtain a panning image intended by a person.

  FIG. 13 is a diagram illustrating an example of a panning mode selection screen used in the third embodiment. For example, when the panning mode is set by operating the menu key 16, a panning mode selection screen 71 as shown in FIG. 13 is displayed. The panning mode selection screen 71 includes items 72 to 74 for selecting any one of “camera following mode”, “camera fixed mode”, and “free mode” as the panning mode.

  The “camera follow-up mode” is a mode that assumes a case where the photographer pans the camera in accordance with the movement of the moving body and shoots. The “camera fixed mode” is a mode that assumes a case where the camera is fixed and the moving body is photographed, as in the first and second embodiments. The “free mode” is a mode that assumes a case where a moving body is photographed by normal camera operation without fixing the camera.

  FIG. 14 and FIG. 15 are flowcharts showing a panning shooting process of the digital camera 1 in the third embodiment. This panning shooting process is executed, for example, when the panning mode is set by operating the menu key 16. With the setting of the panning mode, a panning mode selection screen 71 as shown in FIG. 13 is displayed. The photographer selects any one of “camera following mode”, “camera fixed mode”, and “free mode” on the panning mode selection screen 71, and shoots the moving object by the photographing method corresponding to the selected mode. Shall be performed.

  The control unit 32 provided in the digital camera 1 first determines the mode selected on the panning mode selection screen 71 (step C11). In the “camera follow-up mode”, the exposure condition is determined so that the shutter speed becomes a predetermined value or less (step C12). If it is “camera fixed mode”, the exposure condition is determined so that the shutter speed becomes equal to or higher than a predetermined value (step C13). In the “free mode”, the shutter speed set by the photographer is used.

  When shooting is started, the control unit 32 displays the image data of each frame sequentially obtained from the CCD 23 in accordance with the determined exposure condition, and displays a predetermined number (for example, two frames) of the latest one. The information is stored cyclically in the memory 32a (step C14). When the photographer presses the shutter key 9 (Yes in Step C15), the control unit 32 additionally stores a predetermined number of image data including the image data obtained at the shutter timing in the work memory 32a ( Step C16), the following processing for specifying the main subject is performed.

  That is, first, the control unit 32 compares the image data of each frame stored in the work memory 32a, and extracts a region portion in which pixel values and their arrangement patterns are continuously similar over a predetermined area. (Step C17). Then, the movement vector (speed and direction) of each pixel in the image of these area portions is calculated (step C18), and the area portions having different movement vectors (velocity and direction) on the image are actually separated with different movement speeds. (Step C19).

  Subsequently, a main subject among these subject portions is specified. The main subject is a moving body that the photographer is paying attention to, and the movement of the moving body varies depending on the shooting state of the camera. In other words, when shooting with the camera fixed, the moving object that is the main subject moves the fastest on the image, whereas when shooting while panning the camera, It will be moving the slowest.

  Therefore, the control unit 32 determines the type of the panning mode that is currently set (step C20). As a result, if the “camera following mode” is set, the image of the subject portion with the slowest moving speed is determined as the main subject (step C21), and sharpening processing is performed on the main subject portion. Determine (step C22). On the other hand, if the “camera fixed mode” is set, the image of the subject portion with the fastest moving speed is determined as the main subject (step C23), and the main subject portion is determined to be blurred. (Step C24).

  If the “free mode” is set, the control unit 32 determines the image of the subject portion closest to the target area (center of the screen) in the finder as the main subject (step C25), and the main subject portion. It is determined that sharpening processing is performed on the image and blurring processing is performed on other portions (step C26). Further, in the “camera fixed mode” and the “free mode”, the direction and intensity (blurring amount) of the portion to be blurred are determined based on the moving direction and moving amount of the main subject portion (step C27). .

  Next, the control unit 32 searches the image data of each frame in the work memory 32a for image data in which the determined main subject (moving body) is present at the center (step C28). If there is corresponding image data (Yes in Step C28), the control unit 32 adopts it as an original image (Step C29). The original image referred to here is an image that is a source of the panning image, and corresponds to the image [A] shown in FIG.

  If there is no corresponding image data (No in Step C28), an original image including the main subject (moving body) in the center is created as follows.

  That is, the control unit 32 selects image data that does not include the main subject from the image data of each frame in the work memory 32a as a background image (step C30), and image data that includes the entire main subject. Then, the image of the main subject portion is cut out (step C31), and the cut out main subject portion is combined with the center of the selected image data as the background image, and this combined image is used as the original image (step S31). C32).

  When the original image including the main subject (moving body) in the center is acquired, the control unit 32 performs image processing (sharpening) determined according to the mode type for the main subject portion and other portions of the original image. Conversion processing / blurring processing) are performed separately (step C33).

  That is, in the “camera following mode”, the control unit 32 performs a certain sharpening process on the main subject portion of the original image. As a result, when the photographer actually shoots while panning the camera in accordance with the movement of the moving body, even if the moving body is blurred and photographed because the camera does not match the movement of the moving body, the moving body The image blur is automatically corrected. As a result, it is possible to obtain a captured image as if the moving body is stationary. Although the sharpening process is constant, the sharpening process may be changed according to the moving direction and the moving amount of the moving body as in the blurring process.

  In the “camera fixed mode”, the control unit 32 performs a blurring process on the background portion other than the main subject portion of the original image according to the moving direction and the moving amount of the moving object. As a result, as in the first and second embodiments, it is possible to obtain a panning image in which an image of the background portion is flowed only by photographing the moving body while fixing the camera.

  In the “free mode”, the control unit 32 performs sharpening processing on the main subject portion of the original image and blurring processing on the other portions according to the moving direction and moving amount of the moving object. As a result, when the photographer freely photographed the moving object, the moving object was photographed because the camera did not match the movement of the moving object, or the background other than the moving object was photographed without blurring. However, these are automatically corrected. As a result, it is possible to obtain a panning image in which the moving body is stopped and a part other than the moving body is flowed in the moving direction.

  In other words, whether the main subject portion is sharpened or the background portion is blurred, the amount of blur or blur between the main subject portion and other background portions is relatively different. Thus, a composite image in which the movement of the moving body is emphasized can be obtained.

  Various methods for the sharpening process and the blurring process are known as well-known techniques, and any method may be used for each, although details are not described.

  Also, when the blurring process or sharpening process is performed based on the specified blurring direction or amount, the specified information is specified when the blurring or blurring direction or amount of the image is specified in advance. There is also known a technique for restoring to a state without blurring and blurring with higher accuracy. For example, the PSF (Point Spred Function) method is one of the techniques that can restore a state without blurring or blurring more accurately based on information on the direction and amount of blurring and blurring. .

  In this way, the panning image generated according to each mode is displayed on the screen, for example, and then recorded and stored in the memory 38 in accordance with a user instruction.

  As described above, according to the third embodiment of the present invention, a panning image can be easily obtained in a state where the camera is fixed as in the first and second embodiments. Even when a moving object is photographed without being fixed, it is possible to easily obtain a panning image in which the camera is accurately matched to the movement of the moving object.

  In each of the above-described embodiments, the digital camera has been described as an example. However, the present invention is not limited to this. For example, in addition to a digital movie camera, a cellular phone or a PDA (Personal Digital Assistant) having a similar imaging function. ).

  In short, the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying constituent elements without departing from the scope of the invention in the implementation stage. Various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the respective embodiments. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

FIG. 1 is a diagram showing an external configuration when a digital camera is taken as an example of the imaging apparatus according to the first embodiment of the present invention. FIG. 2 is a block diagram showing an electronic circuit configuration of the digital camera in the embodiment. FIG. 3 is a diagram for explaining a moving body photographing method in the embodiment. FIG. 4 is a diagram showing an example of a frame image during through image display in the embodiment. FIG. 5 is a diagram illustrating an example of a captured image and a background image including a moving object in the embodiment. FIG. 6 is a diagram illustrating an example of an image of a moving body portion, a background image after blurring processing, and a panning image that is a composite image thereof according to the embodiment. FIG. 7 is a flowchart showing a panning shooting process of the digital camera in the embodiment. FIG. 8 is a flowchart showing a panning shooting process of the digital camera in the embodiment. FIG. 9 is a diagram showing an example of the blur amount selection screen in the embodiment. FIG. 10 is a flowchart showing a panning shooting process of the digital camera according to the second embodiment of the present invention. FIG. 11 is a flowchart showing a panning shooting process of the digital camera in the embodiment. FIG. 12 is a diagram showing an example of a blur control table in the embodiment. FIG. 13 is a diagram showing an example of a panning mode selection screen in the third embodiment of the present invention. FIG. 14 is a flowchart showing panning shooting processing of the digital camera in the embodiment. FIG. 15 is a flowchart showing a panning shooting process of the digital camera in the embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Digital camera, 2 ... Body, 3 ... Shooting lens, 4 ... Self-timer lamp, 5 ... Optical finder window, 6 ... Strobe light emission part, 7 ... Microphone part, 8 ... Power key, 9 ... Shutter key, 10 ... Photographing Mode key, 11 ... Playback mode key, 12 ... Optical viewfinder, 13 ... Speaker unit, 14 ... Macro key, 15 ... Strobe key, 16 ... Menu (MENU) key, 17 ... Ring key, 18 ... Set key, DESCRIPTION OF SYMBOLS 19 ... Display part, 20a, 20b ... Zoom key, 21 ... Motor, 22 ... Lens optical system, 23 ... CCD, 24 ... Timing generator (TG), 25 ... Driver, 26 ... Sample hold circuit (S / H), 27 A / D converter 28 Image processing circuit 29 DMA controller 30 DRAM interface (I / F) 31 DRAM 3 Control unit 32a Work memory 33 VRAM controller 34 VRAM 35 Digital video encoder 36 Key input unit 37 JPEG circuit 38 Memory 39 Audio processing unit 40 USB interface ( (I / F), 41... Strobe drive unit, 51... Blurring amount selection screen, 61... Blurring control table, 71.

Claims (12)

Photographing means for continuously photographing a subject and obtaining a plurality of image data;
A specifying means for specifying a moving body by comparing a plurality of image data acquired by the photographing means;
Selecting means for selecting image data including the moving body specified by the specifying means and image data not including the moving body from among a plurality of image data acquired by the photographing means;
Synthesizing means for extracting a partial image corresponding to the moving object from the image data including the moving object selected by the selecting means, and synthesizing the extracted partial image with image data not including the moving object;
An imaging apparatus comprising: The imaging apparatus according to claim 1, wherein the synthesizing unit synthesizes the extracted partial image with a center of image data not including the moving body. Image processing means for performing a blurring process on image data not including the moving object selected by the selection means;
The imaging apparatus according to claim 1, wherein the synthesizing unit synthesizes the extracted partial image with image data that does not include the moving object subjected to the blurring process by the image processing unit. A detecting means for detecting a moving direction of the moving body specified by the specifying means;
The imaging apparatus according to claim 3, wherein the image processing unit performs blurring processing in a moving direction of the moving body on image data that does not include the moving body. A detecting means for detecting a moving speed of the moving body specified by the specifying means;
The imaging apparatus according to claim 3, wherein the image processing unit performs blurring processing of an amount corresponding to a moving speed of the moving body on image data that does not include the moving body. Photographing means for continuously photographing a subject and obtaining a plurality of image data;
Detecting means for detecting the moving speed of each subject portion by comparing a plurality of image data acquired by the photographing means;
A selection means for selecting a subject portion having the slowest moving speed among a plurality of subject portions whose moving speed is detected by the detecting means;
Image processing means for performing sharpening processing on the subject portion selected by the selection means;
An imaging apparatus comprising: Mode selection means for selecting a camera follow mode and a camera fixed mode as an operation mode at the time of panning shooting,
The selection means selects the subject portion having the slowest moving speed when the camera tracking mode is set, and selects the subject portion having the fastest moving speed when the camera fixing mode is set. ,
The image processing means performs a sharpening process on the subject portion selected by the selection means when the camera tracking mode is set, and the selection when the camera fixing mode is set. The imaging apparatus according to claim 6, wherein blurring processing is performed on a subject portion selected by the means. When the camera tracking mode is set, shooting is performed with a shutter speed set to a predetermined value or less, and when the camera fixing mode is set, shooting is performed with a shutter speed set to a predetermined value or more. The imaging apparatus according to claim 6, wherein: A shooting step of continuously shooting a subject to obtain a plurality of image data;
A specifying step of specifying a moving body by comparing a plurality of image data acquired by the photographing step;
A selection step of selecting image data including the moving body specified by the specifying step and image data not including the moving body from among a plurality of image data acquired by the photographing step;
Extracting a partial image corresponding to the moving object from the image data including the moving object selected in the selecting step, and combining the extracted partial image with image data not including the moving object;
An image processing method comprising: A shooting step of continuously shooting a subject to obtain a plurality of image data;
A detection step of detecting a moving speed of each subject portion by comparing a plurality of image data acquired in the photographing step;
A selection step of selecting a subject portion having the slowest moving speed among a plurality of subject portions whose moving speed is detected by the detecting step;
An image processing step of performing a sharpening process on the subject portion selected by the selection step;
An image processing method comprising: The computer installed in the imaging device
Photographing means for continuously photographing a subject and obtaining a plurality of image data;
A specifying means for specifying a moving body by comparing a plurality of image data acquired by the photographing means;
Selecting means for selecting image data including the moving body specified by the specifying means and image data not including the moving body from among a plurality of image data acquired by the photographing means;
Synthesizing means for extracting a partial image corresponding to the moving object from the image data including the moving object selected by the selecting means, and synthesizing the extracted partial image with image data not including the moving object;
A program characterized by making it function. The computer installed in the imaging device
Photographing means for continuously photographing a subject and obtaining a plurality of image data;
Detecting means for detecting the moving speed of each subject portion by comparing a plurality of image data acquired by the photographing means;
A selection means for selecting a subject portion having the slowest moving speed among a plurality of subject portions whose moving speed is detected by the detecting means;
Image processing means for performing sharpening processing on the subject portion selected by the selection means;
A program characterized by making it function.

Images