JP5352377B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing method.

  In a video camera that mainly captures moving images, video is recorded even during a zoom operation, and the subject is captured while zooming in or out, and the data is recorded. The zoom of the video camera being recorded is required to operate smoothly (the change in zoom speed is small) so that the viewer does not feel uncomfortable when playing back a moving image including zoom-in or zoom-out. .

  On the other hand, a camera such as a digital single-lens reflex camera with interchangeable lenses does not record an image during a zoom operation. Although there are exceptional shooting operations such as zoom between exposures, zoom operation is not performed during normal shooting. The zoom lens employs a mechanism for manually rotating the zoom ring so that the zoom magnification desired by the photographer can be quickly reached in still image shooting.

  For example, Patent Literature 1 and Patent Literature 2 are disclosed as techniques relating to zooming. In Patent Document 1, when the optical zoom is operated using a motor, a gear, or the like, a point that there is a time lag between the user's operation and the operation of the zoom mechanism is improved. For this reason, the time lag is compensated for by changing the angle of view (so-called digital zoom). Japanese Patent Application Laid-Open No. 2004-228561 is a technique that enables manual zoom to be switched to electric zoom.

JP 2008-53769 A JP 2007-108373 A

  By the way, in recent years, even a camera that has been conventionally used for still image shooting, such as a digital single-lens reflex camera with interchangeable lenses, has been able to capture a moving image in the same manner as a video camera. However, in a digital single-lens reflex camera that can capture a moving image, the user needs to manually perform a zoom operation. Therefore, there is a problem that the zoom speed becomes non-uniform compared to a video camera that performs zoom operation electrically.

  Therefore, as in Patent Document 2, it is possible to configure the lens so that the zoom mechanism of the interchangeable lens can be electrically operated. However, the manufacturing cost of the lens increases, and the entire lens becomes larger or heavier. There is a problem. In addition, as an advantage of digital single-lens reflex cameras, there are cases where past lens assets can be used, but since most conventional lenses do not have an electric zoom mechanism, it is natural that zooming is performed electrically when these lenses are attached. Impossible. Patent Document 1 does not disclose that the user manually performs the zoom operation.

  Therefore, the present invention has been made in view of the above problems, and an object of the present invention is to generate a moving image having a smooth zoom change when a user manually performs a zoom operation to shoot. It is an object of the present invention to provide a new and improved image processing apparatus and image processing method that can be performed.

  In order to solve the above-described problem, according to one aspect of the present invention, a plurality of images captured by using a zoom lens for enlarging or reducing a subject image by a user's manual operation, and a plurality of images A recording unit that records a focal length at the time of shooting corresponding to an image, and a focal length calculation unit that calculates a focal length for smoothing based on the recorded focal length so that a temporal change in the focal length becomes smooth A smoothing focal length, a recorded focal length, and an image generation unit that generates a smoothing image having an angle of view corresponding to the smoothing focal length based on the recorded image. Is provided.

  With this configuration, the focal length for smoothing is calculated based on the focal length recorded in the recording unit so that the temporal change of the focal length is smooth, and the smoothing image corresponds to the focal length for smoothing. It is generated based on the focal length for smoothing, the focal length recorded in the recording unit and the image so as to have an angle of view. As a result, it is possible to generate a moving image having a smooth zoom change even when the user manually performs a zoom operation using a zoom lens.

  When the image generation unit generates the smoothing image, the focal length calculation unit calculates the smoothing focal length so that the smoothing image is enlarged with respect to the recorded image. With this configuration, when a smoothing image is generated, an image recorded at the same time can be used without fail, so that a moving image having a smooth zoom change can be reliably generated.

  The focal length calculation unit calculates the focal length for smoothing so that the temporal change of the focal length is smooth while the recorded focal length is changing in one direction of increasing direction or one direction of decreasing direction. To do. With this configuration, only the period in which the recorded focal length changes are the same is taken into account in the calculation of the smoothing focal length.

  In order to solve the above-described problem, according to another aspect of the present invention, a plurality of images photographed by using a zoom lens for enlarging or reducing a subject image by a user's manual operation, Based on the recorded focal length, the focal length for smoothing is calculated so that the temporal change of the focal length is smooth based on the step of recording the focal length at the time of shooting corresponding to a plurality of images in the recording unit. Generating a smoothing image having an angle of view corresponding to the smoothing focal length based on the step, the smoothing focal length, the recorded focal length, and the recorded image. An image processing method is provided.

  As described above, according to the present invention, it is possible to generate a moving image having a smooth zoom change when a user manually performs a zoom operation to shoot.

1 is a block diagram illustrating an imaging apparatus 100 according to a first embodiment of the present invention. 4 is a flowchart showing a moving image shooting operation of the imaging apparatus 100 according to the embodiment. 4 is a flowchart showing a moving image shooting operation of the imaging apparatus 100 according to the embodiment. 4 is a flowchart showing a smoothing operation of a memory temporarily recorded video of the imaging apparatus 100 according to the embodiment. It is a graph which shows the change characteristic of a focal distance. It is a graph which shows the change characteristic of a focal distance. It is a graph which shows the change characteristic of a focal distance. It is a block diagram which shows the image processing apparatus 200 which concerns on the 2nd Embodiment of this invention. It is explanatory drawing which shows the data structure of the moving image file applicable to the image processing apparatus 200 which concerns on the embodiment. 4 is a flowchart showing a zoom smoothing operation of the image processing apparatus 200 according to the embodiment. It is a graph which shows the change characteristic of a focal distance. It is a flowchart which shows the moving image shooting operation | movement of the conventional imaging device.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

<1. First Embodiment>
[Configuration of imaging device]
First, the configuration of the imaging apparatus 100 according to the first embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram illustrating an imaging apparatus 100 according to the present embodiment.

  The imaging apparatus 100 is a digital single-lens reflex camera capable of taking a still image and a moving image and capable of exchanging lenses. The imaging apparatus 100 includes, for example, an interchangeable lens 102, an imaging element 110, a preprocessing unit 112, an MPU 120, an image memory 122, a memory card 126, a still image processing unit 132, a moving image processing unit 134, a JPEG processing unit 136, and a display unit 140. The operation unit 150 and the like. The imaging apparatus 100 is not limited to a single-lens reflex camera, and may be a compact camera to which an interchangeable lens can be attached.

  The interchangeable lens 102 includes a lens 103, an AF driving unit 104, and the like. Although not shown, the lens 103 includes, for example, a zoom lens, a diaphragm, and a focus lens. The interchangeable lens 102 is provided with, for example, a zoom ring so that the user can manually change the zoom. By rotating the zoom ring, the subject image formed on the imaging surface can be enlarged or reduced.

  The interchangeable lens 102 is an optical system that forms an image of external light information on the image sensor 110, and transmits light from the subject to the image sensor 110. The zoom lens is a lens that changes the angle of view by changing the focal length. The diaphragm is a mechanism that adjusts the amount of light transmitted. The focus lens moves in the optical axis direction to focus the subject image on the imaging surface of the image sensor 110. The focus lens is driven by the AF driving unit 104.

  The image sensor 110 is an example of a photoelectric conversion element, and includes a plurality of elements capable of performing photoelectric conversion that converts optical information that has passed through the interchangeable lens 102 and entered into an electrical signal. Each element generates an electrical signal corresponding to the amount of light received. As the image sensor 110, a charge coupled device (CCD) sensor, a complementary metal oxide semiconductor (CMOS) sensor, or the like can be applied.

  In addition, in order to control the exposure time of the image sensor 110, a mechanical shutter (not shown) can be applied so that the light is blocked during non-shooting and the light is applied only during shooting. Further, the present invention is not limited to this, and an electronic shutter may be applied. The mechanical shutter or electronic shutter is operated by a switch of the shutter button (operation unit 150).

  The image sensor 110 further includes a CDS / AMP unit and an A / D conversion unit. The CDS / AMP unit (correlated double sampling circuit / amplifier) removes reset noise and amplifier noise included in the electrical signal output from the image sensor 110, and converts the electrical signal into an arbitrary signal. Amplify to level. The A / D conversion unit digitally converts the electrical signal output from the CDS / AMP unit to generate a digital signal. The A / D conversion unit outputs the generated digital signal to the preprocessing unit 112.

  The preprocessing unit 112 performs processing on the digital signal output from the A / D conversion unit, and generates an image signal that can be processed. The preprocessing unit 112 performs processes such as pixel defect correction, black level correction, and shading correction of the image sensor 110, for example. The preprocessing unit 112 outputs the generated image signal to the still image processing unit 132, for example. Further, the preprocessing unit 112 controls reading / writing of image data from / to the image memory 122.

  An MPU (Micro Processor Unit) 120 functions as an arithmetic processing device and a control device according to a program, and controls processing of each component provided in the imaging device 100. For example, the MPU 120 outputs a signal to the AF driving unit 104 to drive the focus lens of the interchangeable lens 102. Further, the MPU 120 controls each component of the imaging device 100 based on a signal from the operation unit 150. In the present embodiment, the MPU 120 is composed of only one, but the MPU 120 may be composed of a plurality of CPUs, for example, a signal system command and an operation system command are executed by separate CPUs.

  The image memory 122 is, for example, an SDRAM (synchronous DRAM), and temporarily stores image data of captured images. The image memory 122 has a storage capacity capable of storing image data of a plurality of images. Reading and writing of images to and from the image memory 122 is controlled by the preprocessing unit 112. In the present embodiment, the image memory 122 is an example of a recording unit, and a plurality of images captured by using a zoom lens by a user's manual operation and a focal length at the time of shooting corresponding to the plurality of images. Can record.

  The memory card 126 writes image data or reads recorded image data and setting information. The memory card 126 is a recording medium such as a magnetic disk or a semiconductor storage medium, and records captured image data. As long as it is a recording medium, it is not limited to the memory card 126, and may be an optical disk (CD, DVD, Blu-ray disk, etc.), a magneto-optical disk, or the like. The memory card 126 may be configured to be detachable from the imaging device 100.

  The still image processing unit 132 receives the video signal from the preprocessing unit 112 and converts it into a luminance signal and a color signal. The still image processing unit 132 generates an image signal subjected to image processing based on the WB control value, the γ value, the edge (contour) enhancement control value, and the like. The still image processing unit 132 sends the generated image signal to the moving image processing unit 134 and the JPEG processing unit 136.

  The moving image processing unit 134 receives image data of a still image from the still image processing unit 132, and generates moving image data by coding (encoding) processing. The moving image processing unit 134 performs encoding processing in, for example, the MPEG format, and generates stream data in which a plurality of image frames are collected as one file. The generated moving image data is recorded on the memory card 126 or the like.

  The moving image processing unit 134 further includes a focal length calculation unit, and calculates the smoothing focal length based on the focal length recorded in the image memory 122 so that the temporal change of the focal length becomes smooth. The moving image processing unit 134 is an example of an image generation unit, and smoothing is performed based on the focal length for smoothing, the focal length recorded in the image memory 122, and the image recorded in the image memory 122. A smoothing image having a field angle corresponding to the focal length is generated.

  When generating the smoothing image, the focal length calculation unit calculates the smoothing focal length so that the smoothing image is enlarged with respect to the recorded image. Further, the focal length calculation unit is configured to smooth the focal length for smoothing so that the temporal change of the focal length is smooth while the recorded focal length is changing in one direction of increasing direction or one direction of decreasing direction. Is calculated.

  The JPEG processing unit 136 performs compression coding processing on the image data from the still image processing unit 132 by a still image coding method such as JPEG (Joint Photographic Experts Group) standard. The generated compression-encoded data is recorded on the memory card 126 or the like. Further, decompression decoding processing is performed on the encoded data of the still image supplied from the memory card 126.

  The display unit 140 receives image data from, for example, a VRAM and displays an image on the screen. The display unit 140 is provided in the main body of the imaging device 100, for example. The image displayed on the display unit 140 is, for example, an image before shooting (live view display) read from the VRAM, various setting screens of the imaging apparatus 100, an image recorded and recorded, and the like. The display unit 140 is, for example, a liquid crystal display or an organic EL display.

  A VRAM (video RAM) is a memory for image display and has a plurality of channels. The VRAM can simultaneously input image data for image display from the image memory 122 and output image data to the display unit 140. The resolution and the maximum number of colors of the display unit 140 depend on the VRAM capacity.

  The operation unit 150 is, for example, an up / down / left / right key, a power switch, a mode dial, or a shutter button provided in the imaging apparatus 100. The operation unit 150 sends an operation signal to the MPU 120 or the like based on an operation by the user. For example, the shutter button can be pressed halfway (S1 operation), fully pressed (S2 operation), and released by the user. When the shutter button is half-pressed, it outputs an operation signal for starting focus control, and when the half-press is released, the focus control ends. Further, when the shutter button is fully pressed, an operation signal for starting shooting is output.

  Note that a series of processing in the imaging apparatus 100 may be processed by hardware or may be realized by software processing by a program on a computer.

[Operation of imaging device]
2A and 2B are flowcharts illustrating the moving image shooting operation of the imaging apparatus 100 according to the present embodiment. FIG. 11 is a flowchart showing the moving image shooting operation of the conventional imaging apparatus.

  Compared with the conventional imaging apparatus, the imaging apparatus 100 according to the present embodiment performs step S111 first, and performs various processes depending on whether or not the zoom operation is being performed after the YCC processing in step S115. Different. In the conventional image pickup apparatus, regardless of whether or not the zoom operation is being performed, the video coding (step S15), the video stream card recording (step S16), and the like are performed after the YCC conversion process in step S14, and the shooting is completed. To do.

Hereinafter, the moving image shooting operation of the imaging apparatus 100 according to the present embodiment will be described.
When the moving image shooting operation of the imaging apparatus 100 is started, first, a temporary recording flag (Flag) indicating whether or not image data during the zoom operation is recorded in the image memory 122 is set to OFF (step S111). Then, automatic exposure (AE) and automatic focus (AF) are set (steps S112 and S113). The shooting data is read out as RAW image data from the image sensor 110, and YCC processing is performed by the still image processing unit 132 (step S115).

  In addition, it is determined whether shooting by the imaging apparatus 100 is in the zoom operation or the zoom operation is stopped (step S116). Whether or not the zoom operation is being performed determines whether or not the focal length at the time of shooting changes in one direction of increase or one direction of decrease. When the direction of change in the focal length is opposite or constant, it is determined that the zoom operation has stopped.

  The determination as to whether or not the zoom operation is being performed may be performed for each frame or may be performed at predetermined time intervals such as every second. However, if the determination as to whether or not the zoom operation is being performed is performed in units of a short time such as every frame, the period during which the zoom is determined to be continued may be shortened depending on how the zoom operation is performed. As a result, a large number of short-time files are generated, so it is preferable to perform the file at a frequency of about once per second.

  If it is determined that the zoom operation is not being performed, it is then determined whether the temporary recording flag is ON or OFF (step S117). If the zoom operation has not been performed before, the temporary recording flag is OFF. If it is determined that the temporary recording flag is OFF, moving image coding is performed on the image data being shot (step S118). For example, the image data is encoded in MPEG format or the like. Thereafter, the coded image data (moving image stream) is recorded on a recording medium such as the memory card 126 (step S119).

  That is, when the zoom operation has not been performed before and when the zoom operation is not being performed, the data being shot is recorded on the recording medium as it is, as in the past.

  When it is determined that the zoom operation is not being performed and the card recording of the moving image stream (step S119) is completed, it is determined whether or not shooting has been completed (step S126). If shooting has not ended, the process returns to step S112. When shooting is completed, the process proceeds to step S127.

  On the other hand, if it is determined in step S116 that shooting by the imaging apparatus 100 is in the zoom operation, first, the temporary recording flag indicating whether or not the video is temporarily recorded is switched ON (step S120). Then, the image data (moving image stream) during the zoom operation that has been converted to YCC in step S115 is sequentially recorded in the image memory 122 (step S121). Further, the focal length at the time of shooting corresponding to the image data being zoomed is simultaneously recorded in the image memory 122.

  The image data is recorded in units of frames, but the determination of whether or not the zoom operation is being performed in step S116 may be, for example, every second as described above. In this case, the previous status is continued in a frame in which the determination in step S116 is not performed. For example, if the previous status is zoom operation, it is determined that the zoom operation is also performed for a frame for which it is not determined whether the zoom operation is being performed. The process proceeds to steps S120 and S121, and image data during the zoom operation is recorded. Is done.

  When the temporary recording of the moving image stream (step S121) is completed during the zoom operation, it is determined whether or not the shooting is finished (step S126). If shooting has not ended, the process returns to step S112. If shooting has been completed during the zoom operation, the process proceeds to step S127.

  If the photographing has not been completed, when the zoom operation is stopped, it is determined in step S116 that the zoom operation is stopped, and in step S117, it is determined that the temporary recording flag is ON. And it transfers to step S122 and switches a temporary recording flag to OFF. Thereafter, the smoothing process is performed on the image data during the zoom operation temporarily recorded in the image memory 122 in step S121 (step S123). The smoothing process is performed while the recorded focal length is changing in one direction of increase or one direction of decrease. As a result, an image that has been manually zoomed by the user and is captured is converted into a smooth image.

  The smoothed image data is then subjected to video coding (step S124). For example, the image data is encoded in MPEG format or the like. Thereafter, the coded data (moving image stream) sequentially output is recorded on a recording medium such as the memory card 126 (step S125). The operations in steps S122 to S123 are performed in parallel with the moving image coding (step S118) and the moving image stream card recording (step S119) in shooting without the zoom operation and the zoom operation.

  When it is determined that the zoom operation is not being performed and the smoothed video stream card recording (step S125) is completed, it is determined whether or not shooting has been completed (step S126). If shooting has not ended, the process returns to step S112. If shooting has been completed, the process proceeds to step S127.

  In step S127, it is determined whether the temporary recording flag is ON or OFF (step S127). When shooting is completed during the zoom operation, the temporary recording flag is ON, and the process proceeds to step S128. On the other hand, in other cases, the temporary recording flag is OFF, and the process proceeds to step S132.

  When shooting is completed during the zoom operation, the smoothing processing of the temporarily recorded image data during the zoom operation such as step S123 described above is not performed. Therefore, first, the temporary recording flag is switched to OFF (step S128). Thereafter, the smoothing process is performed on the image data during the zoom operation temporarily recorded in the image memory 122 in step S121 (step S129). As a result, an image that has been manually zoomed by the user and is captured is converted into a smooth image.

  The smoothed image data is then subjected to video coding (step S130). Thereafter, the coded data (moving image stream) sequentially output is recorded on a recording medium such as the memory card 126 (step S131). The operations in steps S128 to S131 are performed after shooting is completed.

  Thereafter, from the start of shooting to the end of shooting, the data that has been smoothed and the data that did not require smoothing are recorded separately on the memory card 126, so the files are combined into a series of video files and finalized. Process (step S132). As a result, even if a manual zoom operation is performed by a user during a single shooting, smoothed shooting data is recorded instead of shooting data in which the zoomed image is manually recorded as it is. can do.

  FIG. 6 is a graph showing the change characteristic of the focal length, and shows the change of the focal length from the start of shooting to the end of shooting. In the example shown in FIG. 6, there are two periods in which the user has performed the zoom operation and the zoom movement area A and the zoom movement area B, and the period in which the user has performed the zoom operation without performing the zoom operation is zoomed. There is one time between the movement area A and the zoom movement area B. In such a case, there are a total of three files that have been smoothed and unsmoothed, so these files are combined and finalized.

[About smoothing processing]
Next, an operation for performing the smoothing process on the image data during the zoom operation temporarily recorded in the image memory 122 will be described. FIG. 3 is a flowchart showing the smoothing operation of the memory temporarily recorded video of the imaging apparatus 100 according to the present embodiment.

In the image memory 122, image data during a zoom operation is recorded in which the zoom lens is used by a manual operation by a user. Further, the focal length f ₀ at the time of shooting corresponding to the image data is simultaneously recorded in the image memory 122. Therefore, the image data of each frame during the zoom operation and the focal length f ₀ at the time of shooting are acquired, and for example, the temporal change characteristic of the focal length f ₀ at the time of shooting as shown by the solid line in FIGS. Is created (step S141). 4 and 5 are graphs showing the focal length change characteristics. FIG. 4 shows the zoom-in operation, and FIG. 5 shows the zoom-out operation.

Next, a time change characteristic of the smoothing focal length fs is created (step S142). In step S142, first, Δf ₀ / Δt is calculated based on the temporal change characteristic of the focal length f ₀ at the time of photographing, and the temporal change of Δf ₀ / Δt is acquired. Then, a point at which the change in Δf ₀ / Δt starts to decrease is acquired and stored. In the examples shown in FIGS. 4 and 5, the points are indicated by white dots. Then, using the stored point, the zoom operation start point, and the zoom operation end point, for example, quadratic spline interpolation is performed, for example, at each time as shown by the broken lines in FIGS. The smoothing focal length fs is calculated.

The smoothing focal length fs may be calculated by processing other than the interpolation method described above. When the smoothing focal length fs is calculated, it may be calculated in a direction in which the image corresponding to the smoothing focal length is enlarged as compared with the image data at the time of shooting. That is, in the example shown in FIGS. 4 and 5, the smoothing focal length fs may be calculated closer to the telephoto side than the focal length f ₀ at the time of shooting so that the broken line is positioned above the solid line.

Then, in order to create smoothing image data from the image data at the time of shooting, the original image frame and the trimming size to be subjected to digital zoom are determined (step S143). In the example shown in FIGS. 4 and 5, the image frame to be subjected to the digital zoom is extracted as a frame in which the solid line and the broken line do not overlap, that is, a frame having a different focal length f ₀ and a smoothing focal length fs. To decide. Trimming size is determined by calculating the enlargement ratio for the original frames based on the focal length fs for the focal length f ₀ and smoothing at the time of photographing.

Incidentally, in the calculation of smoothing for focal length fs, depending on the variation characteristics of the interpolation method and the focal length, there is a case where more of the focal length f ₀ of the time of shooting is greater than the focal length fs for smoothing calculated. In this case, when calculating the enlargement ratio for the original frame, the size of the image frames for smoothing becomes a direction to reduce the original frames based on the focal length f ₀ and smoothing for focal length fs at the time of shooting . In this case, since the trimming cannot be performed, the original frame is used as the smoothing image frame. Alternatively, when the image is picked up larger than the size of the original frame on the image pickup surface of the image sensor 110, a reduction process may be performed on the original frame using an extra imaged portion. .

  Next, a trimming process and an enlargement process are performed using the calculated enlargement ratio and the image frame at the time of shooting, thereby generating a smoothing image frame (step S144). Then, the generated smoothing image frame is replaced with the original image frame at the time of shooting (step S145). Thus, the operation for smoothing the image data during the zoom operation temporarily recorded in the image memory 122 is completed.

  The smoothing process is performed after a series of photographing is completed or after a zoom operation by the user is completed.

As shown in FIG. 10, when the change in the focal length f ₀ during shooting during the zoom operation is smooth and there is no point at which the change in Δf ₀ / Δt starts to decrease, the zoom operation start point and the zoom When interpolation processing is performed using the operation end point, for example, a broken line in FIG. 10 is obtained. However, manual zoom operation is smooth in the first place, and if smoothing processing is performed, trimming processing and enlargement processing occur and resolution changes. Therefore, image data at the time of shooting is adopted, and smoothing processing is not performed. That is, the points at which the change in Δf ₀ / Δt starts to decrease are counted, and if the count number is 0, steps S143 to S145 are skipped and the smoothing process is omitted.

<2. Second Embodiment>
Next, an image processing apparatus 200 according to the second embodiment of the present invention will be described.
In the above-described embodiment, the case has been described in which the zoomed image is temporarily stored without coding in real time at the time of shooting and the smoothing process is performed after the zoom process. In the second embodiment, smoothing processing is not performed at the time of shooting, but smoothing processing is performed using an information processing device (image processing device) such as a personal computer based on the image data at the time of shooting and the focal length at the time of shooting. Perform after shooting.

  FIG. 7 is a block diagram showing an image processing apparatus 200 according to the second embodiment of the present invention. The image processing apparatus 200 includes, for example, an MPU 220, a memory 222, an HDD 226, a display unit 240, an operation unit 250, and the like. The MPU 220 includes a focal length calculation unit 232, an image generation unit 234, and the like.

  An MPU (Micro Processor Unit) 220 functions as an arithmetic processing device and a control device according to a program, and controls processing of each component provided in the image processing device 200. The memory 222 stores a program executed by the MPU 220 and the like. An HDD (Hard Disk Drive) 226 is an example of a storage medium and its control device. An image taken by an imaging device is recorded, and an image recorded during the smoothing process is read out. In addition, the smoothed image generated in the present embodiment is newly recorded in the HDD 226. The storage medium is not limited to a hard disk, and may be another storage medium such as an optical disk.

  The display unit 240 is a liquid crystal display, an organic EL display, or the like, for example, and displays an image or the like on the screen. The operation unit 250 is, for example, a keyboard, a mouse, and the like, receives an operation by a user, and sends an operation signal to the MPU 220. The HDD 226, the display unit 240, and the operation unit 250 may be provided integrally with the image processing apparatus 200, or may be configured to be detachable from the image processing apparatus 200.

  The focal length calculation unit 232 calculates the smoothing focal length so that the temporal change of the focal length becomes smooth based on the focal length recorded together with the image. Based on the focal length for smoothing, the focal length recorded together with the image, and the image read from the HDD 226, the image generation unit 234 has the angle of view corresponding to the focal length for smoothing. Is generated. Also, the MPU 220 performs image data decoding processing and smoothing image moving image coding processing. Further, the MPU 220 replaces the smoothed image with the original image, and performs a merge process between the smoothed image and the non-smoothed image.

  In order to realize the second embodiment, first, focal length information is recorded in association with each recorded frame at the time of photographing. The imaging apparatus records moving image data and focal length information in a moving image file format as shown in FIG. 8, for example. FIG. 8 is an explanatory diagram showing a data structure of a moving image file applicable to the image processing apparatus 200 according to the second embodiment of the present invention.

The moving image file includes, for example, a moving image data header 10, moving image data 20, a moving image information data header 30, and moving image information data 40.
The moving image data header 10 describes identification information of the moving image data 20, the entire moving image data size, and the like. The moving image data 20 is identified by the identification information of the moving image data 20.
The moving image data 20 is stored in a plurality of blocks, for example, chunk 01 (Chunk01), chunk 02 (Chunk02), etc. In this embodiment, for example, one chunk is composed of a collection of frame data for one second.
The moving picture information data header 30 records the vertical and horizontal pixel sizes, the frame rate, the format type, and the like of the moving picture, and further records the number of frames of each chunk, address information for access, and the like.
The focal length information of each frame is stored in the area of the moving image information data 40.

Next, the zoom smoothing operation of the image processing apparatus 200 according to the present embodiment will be described.
FIG. 9 is a flowchart showing the zoom smoothing operation of the image processing apparatus 200 according to the present embodiment.

  First, the focal length information 42 of each frame in the moving image information data 40 is acquired (step S241). Then, from the acquired focal length information, a frame photographed by performing a zoom operation, that is, a frame in which the focal length changes with time is searched, and a chunk including the searched frame is specified. Thereafter, the identified chunk is extracted (step S242).

  Next, the extracted chunk (moving picture stream composed of a plurality of frames) is decoded (step S243). Then, smoothing processing is performed on the image data during the zoom operation (step S244). The smoothing process is the same operation as the process described in the first embodiment. By the smoothing process, an image shot by manually zooming is converted into a smooth image.

  Then, the smoothed image data is subjected to video coding (encoding) in units of chunks. For example, the image data is encoded in MPEG format or the like. Thereafter, the coded data (moving image stream) sequentially output is recorded on a recording medium such as a hard disk drive (HDD 226) (step S245).

  Thereafter, the chunk before smoothing processing of the original moving image file is replaced with the smoothed chunk, and the smoothed chunk is merged with the original moving image file (step S246). At this time, in the moving image file, the data is also rewritten so that the moving image data header 10 and the moving image information data header are adapted to the new chunk. As a result, even if a user performs a manual zoom operation on the way of one shot, the zoomed image can be converted from the shooting data recorded as it is to the smoothed shooting data. .

  As described above, according to the first and second embodiments of the present invention, a moving image is smoothed using an image frame during zoom operation, a focal length corresponding to the image frame, and a smoothing focal length. Trim the image frame to zoom. At this time, the focal length for smoothing may be calculated so as to be on the telephoto side with respect to the focal length at the time of shooting.

  As a result, it is possible to prevent the zoom speed from becoming non-uniform when shooting a moving image with an imaging apparatus having a manual zoom mechanism such as a digital single-lens reflex camera. Further, it is possible to generate a moving image having a smooth zoom change without separately preparing an interchangeable lens having an electric zoom function.

  The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field to which the present invention pertains can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that these also belong to the technical scope of the present invention.

DESCRIPTION OF SYMBOLS 100 Imaging device 102 Interchangeable lens 103 Lens 104 AF drive part 110 Imaging element 112 Pre-processing part 120,220 MPU
122 image memory 126 memory card 132 still image processing unit 134 moving image processing unit 136 JPEG processing unit 140, 240 display unit 150, 250 operation unit 200 image processing device 222 memory 226 HDD
232 Focal length calculation unit 234 Image generation unit

Claims (4)

A recording unit that records a plurality of images captured by using a zoom lens for enlarging or reducing a subject image by a user's manual operation, and a focal length at the time of shooting corresponding to the plurality of images; ,
A focal length calculation unit that calculates a focal length for smoothing based on the recorded focal length so that a temporal change in the focal length is smooth;
An image generation unit that generates a smoothing image having an angle of view corresponding to the smoothing focal length based on the smoothing focal length, the recorded focal length, and the recorded image; An image processing apparatus comprising:   When the image generation unit generates the smoothing image, the focal length calculation unit calculates the smoothing focal length so that the smoothing image is enlarged with respect to the recorded image. The image processing apparatus according to claim 1, wherein:   The focal length calculation unit is configured to make the focal length for smoothing smooth so that the temporal change of the focal length is smooth while the recorded focal length is changing in one direction of increasing direction or one direction of decreasing direction. The image processing apparatus according to claim 1, wherein the image processing apparatus calculates an image. A plurality of images that are photographed by using a zoom lens for enlarging or reducing a subject image by a user's manual operation and a focal length at the time of photographing corresponding to the plurality of images are recorded in a recording unit. Steps,
Based on the recorded focal length, calculating a smoothing focal length so that a temporal change in focal length is smooth;
Generating a smoothing image having an angle of view corresponding to the smoothing focal length based on the smoothing focal length, the recorded focal length, and the recorded image. An image processing method.

Images