WO2022019197A1

WO2022019197A1 - Imaging device, lens tube device, imaging method, transmission method

Info

Publication number: WO2022019197A1
Application number: PCT/JP2021/026496
Authority: WO
Inventors: 真己斗大田; 淳司島田
Original assignee: ソニーグループ株式会社
Priority date: 2020-07-21
Filing date: 2021-07-14
Publication date: 2022-01-27
Also published as: JPWO2022019197A1

Abstract

In this imaging device, processing is performed for generating, as metadata associated with a captured image, correction information including: a first camera shake correction value for a first camera shake correction function by which the positional relationship between an optical image incident via a lens and an output captured image is corrected; and a second camera shake correction value for a second camera shake correction function provided to a lens tube comprising a lens.

Description

Imaging device, lens barrel device, imaging method, transmission method

This technology relates to an image pickup device and its image pickup method, a lens barrel device and its transmission method, and particularly to the processing of image stabilization data.

For example, as an interchangeable lens type camera, there is known an image pickup device that is composed of a camera body (camera body) and a lens barrel and can record moving images on a recording medium.
In such an image pickup device, there are those that perform image stabilization by mechanically moving the image stabilization function in the camera body and those that perform image stabilization by mechanically moving the image stabilization function in the lens barrel. Further, there is also known an electronic image stabilization function that changes the reading range of an image signal from an image sensor according to blurring and changes the cropping range of an image in image signal processing.

Patent Document 1 below discloses a configuration in which image stabilization is performed on the lens barrel side and the camera body side, respectively.

WO2018 / 025639 Gazette

Nowadays, there is an environment in which a user can easily perform various image imaging and image adjustment using a mobile terminal such as a smartphone or tablet, a camera itself, a personal computer, or the like. In this case, the influence of camera shake during imaging may be removed with high accuracy, or conversely, the image may be positively shaken to obtain an effect.
Therefore, in the present disclosure, it is assumed that the shaking of the image is added or removed from the moving image after imaging, and a technique for storing appropriate information in the imaging device for this purpose is proposed.

The image pickup apparatus according to the present technology includes a first blur correction value related to a first blur correction function for correcting the positional relationship between an optical image incident via a lens and an output captured image, and a lens mirror provided with the lens. It is provided with a control unit that generates correction information including a second blur correction value related to the second blur correction function provided in the cylinder as metadata associated with the captured image.
The first blur correction function is a function of correcting the positional relationship between the optical image and the output image captured image (the image finally output from the image pickup device) on the main body side. The second image stabilization function is a function provided on the lens barrel side. In this way, when the lens barrel side and the camera body side each have a mechanical or electronic image stabilization function, the correction information including the respective image stabilization values on the lens barrel side and the camera body side can be obtained. To be associated with the video being captured as metadata. For example, metadata is recorded on a recording medium in a state associated with image data.

In the image pickup apparatus according to the present technology described above, it is conceivable that the control unit uses the second blur correction value received from the lens barrel as the correction information.
The second image stabilization value related to the second image stabilization function on the lens barrel side is transmitted from the lens barrel to the camera body. The control unit includes this in the metadata.

In the image pickup apparatus according to the present technology described above, it is conceivable that the first blur correction function performs the blur correction of the high frequency component, and the second blur correction function performs the blur correction of the low frequency component.
The shaking caused by camera shake, etc. applied to the image pickup device is divided into high-frequency components and low-frequency components, and the high-frequency components are corrected on the main unit side and the low-frequency components are corrected on the lens barrel side. ..

In the image pickup apparatus according to the present technology described above, the lens barrel is configured to be removable from the camera body, and the control unit has a communication speed between the mounted lens barrel and the lens barrel. Based on the determination, it is conceivable to control the first blur correction function to perform blur correction of the high frequency component and the second blur correction function to perform blur correction of the low frequency component.
For example, when a lens barrel with a slow communication speed with the main body is attached, the shaking caused by camera shake etc. applied to the image pickup device is divided into high frequency components and low frequency components, and the main body side performs image stabilization. Therefore, the image stabilization of low frequency components is performed on the lens barrel side.

In the image pickup apparatus according to the present technology described above, in the lens barrel, the second image stabilization value at each sample timing is stored in the storage unit together with the time information, and the second image stabilization value stored in the storage unit at a predetermined time point. The image stabilization value and the time information are transmitted to the control unit, and the control unit converts the second image stabilization value received from the lens barrel into a frame of a moving image based on the time information. It is conceivable to perform processing with the corresponding metadata.
The lens barrel side does not sequentially transmit the second image stabilization value to the main body side, but stores it in the storage unit and transmits it at a certain point in time.

In the image pickup apparatus according to the present technology described above, the control unit is generated by an interpolation process using the second image stabilization value received from the lens barrel and the received second image stabilization value. It is conceivable that the second blur correction value is used as the correction information.
For example, when the second blur correction value transmitted from the lens barrel side has a low sampling rate, the second blur correction value by interpolation processing can also be generated.

In the image pickup apparatus according to the present technology described above, in the control unit, the blur correction of the high frequency component is performed by the first blur correction function, and the blur correction of the low frequency component is performed by the second blur correction function. It is conceivable to set the frequency range of the low frequency component based on the communication speed with the lens barrel side.
When blur correction of low frequency components is performed on the lens barrel side, the frequency range for the low frequency components is set according to the communication speed.

In the image pickup apparatus according to the present technology described above, in the control unit, the blur correction of the high frequency component is performed by the first blur correction function, and the blur correction of the low frequency component is performed by the second blur correction function. It is conceivable to set the frequency range of the low frequency component based on the storage capacity of the storage unit.
When blur correction of a low frequency component is performed on the lens barrel side, the frequency range of the low frequency component is determined based on the storage capacity of the storage unit, for example, the storable capacity of the second blur correction value. Set.

In the image pickup apparatus according to the present technology described above, it is conceivable that the control unit performs a process of recording the communication speed information between the lens barrel and the camera body as metadata on a recording medium.
In addition to the first image stabilization value and the second image stabilization value, the communication speed information on the lens barrel side and the camera body side is also recorded as metadata.

The lens barrel device according to the present technology has a blur correction function for displaced the blur correction lens, a storage unit, a process for detecting a blur correction value related to the blur correction function at each sample timing, and the detected blur correction value. A control unit that stores the image in the storage unit together with the time information, and a control unit that transmits the image stabilization value and the time information stored in the storage unit to the camera body unit to which the lens is mounted. To prepare for.
When the lens barrel side is equipped with a mechanical image stabilization function, the image stabilization value is temporarily stored together with the time information and then transmitted to the camera body side.

In the lens barrel device according to the present technology described above, the control unit performs a process of transmitting the blur correction value and time information stored in the storage unit at intermittent timings during the moving image imaging period. Is possible.
The control unit on the lens barrel side stores the image stabilization value in the storage unit during the period during video recording, and transmits the image stabilization value to the camera body, for example, periodically or irregularly.

In the lens barrel device according to the present technology described above, the control unit performs a process of transmitting the blur correction value and the time information stored in the storage unit to a time point after the moving image imaging is completed. Is possible.
The control unit on the lens barrel side transmits the image stabilization value stored in the storage unit to the camera body side after the moving image recording is completed.

In the lens barrel device according to the present technology described above, the control unit selects whether to transmit the blur correction value detected at each sample timing to the camera body unit or store it in the storage unit. The blur correction value stored in the storage unit is stored in the storage unit together with the time information, read from the storage unit at a predetermined time point, and transmitted to the camera main body unit together with the time information. It is conceivable to do.
The control unit on the lens barrel side transmits the image stabilization value even during video recording, but if transmission is inappropriate for some reason, it is stored in the storage unit and transmitted at a later point in time.

The imaging method according to the present technology includes a first image stabilization value related to a first image stabilization function for correcting the positional relationship between an optical image incident through a lens and an output image image, and a lens mirror provided with the lens. The image pickup apparatus performs a process of generating correction information including a second blur correction value related to the second blur correction function provided in the cylinder as metadata associated with the captured image.
This makes it possible to refer to the image stabilization values on both the lens barrel side and the main body side from the metadata when capturing an image.

The transmission method according to the present technology includes a process in which a lens barrel device provided with a blur correction function for displacement of a blur correction lens and a storage unit detects a blur correction value related to the blur correction function at each sample timing. A process of storing the detected image stabilization value together with time information in the storage unit, and a process of transmitting the image stabilization value and the time information stored in the storage unit at a predetermined time point to the camera body unit to which the lens is mounted. And do. As a result, information is appropriately transmitted to the camera body side regardless of the communication speed.

It is explanatory drawing of the image pickup apparatus and image processing apparatus of embodiment of this technique. It is explanatory drawing of the data flow of the image pickup apparatus and the image processing apparatus of embodiment. It is a block diagram of the configuration example of the image pickup apparatus of an embodiment. It is explanatory drawing of the correction process of the image pickup apparatus of embodiment. It is explanatory drawing of the metadata of embodiment. It is explanatory drawing of the function of the image processing apparatus of embodiment. It is a flowchart of the setting processing example which concerns on the blur correction of an embodiment. It is a flowchart of the processing example which concerns on the blur correction of 1st Embodiment. It is a flowchart of the processing example which concerns on the blur correction of 1st Embodiment. It is a flowchart of recording of IMU data of embodiment. It is a flowchart of the processing example which concerns on the blur correction of 2nd Embodiment. It is a flowchart of the processing example which concerns on the blur correction of 2nd Embodiment. It is a flowchart of the processing example which concerns on the blur correction of 3rd Embodiment. It is a flowchart of the processing example which concerns on the blur correction of 4th Embodiment. It is a flowchart of recording of communication speed information of 5th Embodiment. 6 is a flowchart of recording communication speed information according to the sixth embodiment.

Hereinafter, embodiments will be described in the following order.
<1. Shake change by image pickup device and image processing device>
<2. Imaging device configuration and metadata>
<3. Functions of image processing equipment>
<4. Setting process related to image stabilization>
<5. First Embodiment>
<6. Second Embodiment>
<7. Third Embodiment>
<8. Fourth Embodiment>
<9. Fifth Embodiment>
<10. 6th Embodiment>
<11. Summary and modification>

<1. Shake change by image pickup device and image processing device>
FIG. 1 shows an example of the image pickup apparatus 1 of the embodiment and an image processing apparatus (5, 6) for acquiring an image file MF imaged by the image pickup apparatus 1.

The figure shows an example in which the mobile terminal 7 and the personal computer 8 function as the

image processing devices

5 and 6. Although not shown, various other devices such as an image editing dedicated device, a cloud server, a television device, and a video recording / playback device are assumed as the

image processing devices

5 and 6. These devices can function as any of the

image processing devices

5 and 6.

The image processing device 5 is a device that temporarily performs a shake change process on the image data acquired from the image pickup device 1.
On the other hand, the image processing device 6 is a device that secondarily performs the shaking change processing on the image data that has already been shake-changing processing by another image processing device.

Note that "shake" refers to a shake between frames (interframe shake) of images constituting a moving image. Vibration components that occur between frames, such as image shake caused by camera shake in the image captured by the image pickup device 1 and shake intentionally added by image processing (image fluctuation between frames). Is broadly referred to.
The term "camera shake" is also used when referring to "shaking" caused by camera shake or the like during imaging with the image pickup apparatus 1. "Image stabilization" is used for corrections performed in the image pickup device 1 to reduce image shake due to camera shake, etc. (including vibrations applied when the image pickup device 1 is fixedly placed without being held by hand). It is distinguished from the process of "shaking change" in the

image processing devices

5 and 6.

"Interframe shake modification" refers to changing the state of shaking in an image, such as reducing the shaking occurring in the image or adding shaking to the image.
This "shake change" shall include the following "interframe shake reduction" and "interframe shake production".

"Shake removal" refers to eliminating or reducing (totally removing) or reducing (partially removing) shaking that occurs in an image due to camera shake during imaging in the

image processing devices

5 and 6. ..

The “shaking effect” refers to changing the shaking state of the image with the

image processing devices

5 and 6. In some cases, the shaking is reduced as this shaking effect, and in that sense, it may be the same as "sway removal". However, in the present embodiment, the amount of change in the shaking is instructed by the user's operation or automatic control. It means changing the shaking state of the image according to the instruction.
For example, reducing or increasing the shaking generated at the time of imaging according to a user instruction or the like, or adding new shaking corresponds to "shaking effect".
As an example of the purpose of the shaking effect, it is assumed that the image is intentionally shaken in order to give power to the moving image scene.

The image pickup device 1 in FIG. 1 is a so-called digital still camera or a digital video camera, and is capable of at least moving image.

The camera body of the image pickup apparatus 1 is shown as the camera body 2.
The lens barrel 3 functions as a so-called interchangeable lens, and is detachable from the camera body (camera body 2) of the image pickup apparatus 1. The user can replace and use the lens barrel 3 according to the use case.
In the embodiment, such an interchangeable lens type image pickup apparatus 1 is assumed, but the technique of the present disclosure can be applied to a type in which the lens barrel 3 cannot be removed from the camera body 2.

The image pickup device 1 can perform moving image imaging, and transfer the image file MF obtained by the moving image imaging to a mobile terminal 7 as an image processing device 5, a personal computer 8 or the like via wired communication or wireless communication. Alternatively, the image pickup device 1 may record the image file MF on a recording medium such as a memory card, and the mobile terminal 7 or the personal computer 8 may be able to read the image file MF from the memory card.
Further, the image file MF includes not only the image data as a moving image but also the metadata which is the additional information corresponding to the image data.

FIG. 2 shows a state of information transmission in the image pickup apparatus 1, the image processing apparatus 5, and the image processing apparatus 6.
The image data VD1 and the metadata MTD1 are transmitted from the image pickup device 1 to the image processing device 5 via wired communication, wireless communication, or a recording medium.
The image data VD1 and the metadata MTD1 are information transmitted as, for example, an image file MF.
In the case of the present embodiment, the metadata MTD1 includes, for example, information regarding camera shake correction at the time of imaging.

The image processing device 5 can receive the image data VD1 and the metadata MTD1 and perform various processing.
For example, the image processing apparatus 5 can perform the shake change processing for the image data VD1 by using the information related to the camera shake correction included in the metadata MTD1.
As described above, the shake change is a process of canceling the image stabilization and returning to the original image with shake, performing more advanced shake removal, and adding shake to the image for production. ..

The image processing device 5 can transfer the image data VD2 that has undergone the shake change processing and the like and the metadata MTD2 to another image processing device 6.
In this case, since the metadata MTD2 is added with information related to the shaking change processing and the like, the image processing apparatus 6 can also perform various shaking changes.

In the present embodiment, on the assumption of such information transmission, at least the shaking change in the image processing apparatus 5 can be appropriately performed. For this purpose, we will focus on the metadata recording at the time of imaging in the imaging device 1.

<2. Imaging device configuration and metadata>
FIG. 3 shows a configuration example of the image pickup apparatus 1 and the lens barrel 3.

A lens system 10 having a plurality of optical components is formed on the lens barrel 3. For example, the lens system 10 includes a zoom lens 10a, an aperture mechanism 10b, an image stabilization lens mechanism 10c, a focus lens 10d, and the like.
The image stabilization lens mechanism 10c is a mechanism that reduces the shaking that occurs in an image by mechanically driving the lens against camera shake.

The light (incident light) from the subject is focused on the image sensor unit 12 via the lens system 10 and the shutter 11 in the camera body 2.

The image sensor unit 12 includes, for example, an image sensor (image sensor) such as a CMOS (Complementary Metal Oxide Semiconductor) type or a CCD (Charge Coupled Device) type.
The image sensor unit 12 executes, for example, CDS (Correlated Double Sampling) processing, AGC (Automatic Gain Control) processing, and the like on the electric signal obtained by photoelectric conversion of the light received by the image sensor, and further performs A / D (A / D). Analog / Digital) Performs conversion processing. Then, the image pickup signal as digital data is output to the camera signal processing unit 13 in the subsequent stage.

An image pickup surface camera shake correction unit 30 is provided for the image pickup element unit 12.
The image stabilization unit 30 is a mechanism for correcting image shake by mechanically moving the image sensor in response to camera shake or the like.

The camera signal processing unit 13 is configured as an image processing processor by, for example, a DSP (Digital Signal Processor) or the like. The camera signal processing unit 13 performs various signal processing on the digital signal (image pickup image signal) from the image pickup element unit 12. For example, as a camera process, the camera signal processing unit 13 performs preprocessing, simultaneous processing, YC generation processing, various correction processing, resolution conversion processing, codec processing, and the like.

In the preprocessing, a clamping process for clamping the black level of R, G, B to a predetermined level, a correction process between the color channels of R, G, B, etc. are performed on the captured image signal from the image sensor unit 12. conduct.
In the simultaneous processing, a color separation processing is performed so that the image data for each pixel has all the color components of R, G, and B. For example, in the case of an image sensor using a Bayer array color filter, demosaic processing is performed as color separation processing.
In the YC generation process, a luminance (Y) signal and a color (C) signal are generated (separated) from the image data of R, G, and B.
In the resolution conversion process, the resolution conversion process is executed for the image data to which various signal processes have been performed.

FIG. 4 shows an example of various correction processes performed from the lens system 10 to the camera signal processing unit 13. FIG. 4 exemplifies the correction processing performed by the camera signal processing unit 13 together with the optical image stabilization performed by the camera shake correction lens mechanism 10c and the image stabilization unit 30 according to the execution order. There is.

As the optical image stabilization of the processing F1, lens vibration isolation by the image stabilization lens mechanism 10c and body vibration isolation by the image pickup surface image stabilization unit 30 are performed.
For example, camera shake correction as lens vibration isolation by shifting the yaw direction and pitch direction of the camera shake correction lens mechanism 10c, and body vibration isolation by shifting the image sensor yaw direction and pitch direction by the image stabilization unit 30. By performing the image stabilization, the image of the subject is formed on the image sensor in a state where the influence of the camera shake is physically canceled.
In some cases, this lens vibration isolation and body vibration isolation are performed only in one case, and in other cases, both are performed.
In addition to the above optical image stabilization, electronic image stabilization may be performed as the image stabilization.

In the camera signal processing unit 13, processing from processing F2 to processing F6 is performed by spatial coordinate conversion for each pixel.
In the process F2, lens distortion correction is performed.
In the process F3, focal plane distortion correction is performed as one element of electronic image stabilization. It should be noted that this corrects the distortion when the rolling shutter method is read out by, for example, a CMOS type image sensor.

Roll correction is performed in the process F4. That is, the roll component is corrected as one element of the electronic image stabilization.
In the process F5, trapezoidal distortion correction is performed for the trapezoidal distortion amount generated by the electronic image stabilization. The trapezoidal distortion caused by electronic image stabilization is perspective distortion caused by cutting out a place away from the center of the image.
In the process F6, shift and cutout in the pitch direction and the yaw direction are performed as one element of the electronic image stabilization.
For example, camera shake correction, lens distortion correction, and trapezoidal distortion correction are performed by the above procedure.
It is not essential to carry out all of the processes listed here, and the order of the processes may be changed as appropriate.

In the codec processing in the camera signal processing unit 13 of FIG. 3, for example, coding processing for recording and communication and file generation are performed on the image data subjected to the above various processing. For example, an image file MF is generated as an MP4 format used for recording MPEG-4 compliant video / audio. It is also conceivable to generate files in formats such as JPEG (Joint Photographic Experts Group), TIFF (Tagged Image File Format), and GIF (Graphics Interchange Format) as still image files.

Although the audio processing system is not shown in FIG. 3, it actually has an audio recording system and an audio processing system, and the image file MF includes the audio data as well as the image data as a moving image. May be good.

The camera control unit 18 is composed of a microcomputer (arithmetic processing unit) provided with a CPU (Central Processing Unit).
The memory unit 19 stores information and the like used for processing by the camera control unit 18. As the illustrated memory unit 19, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), a flash memory, and the like are comprehensively shown.

The RAM in the memory unit 19 is used for temporarily storing data, programs, and the like as a work area for various data processing of the CPU of the camera control unit 18.
The ROM and flash memory (nonvolatile memory) in the memory unit 19 include an OS (Operating System) for the CPU to control each unit, content files such as image files, application programs for various operations, and firmware. It is used for memory of etc.
The memory unit 19 may be a memory area built in the microcomputer chip as the camera control unit 18, or may be configured by a separate memory chip.

The camera control unit 18 controls the entire image pickup apparatus 1 and the lens barrel 3 by executing a program stored in the ROM of the memory unit 19, the flash memory, or the like.
For example, the camera control unit 18 controls the shutter speed of the image sensor unit 12, gives instructions for various signal processing in the camera signal processing unit 13, captures and records operations according to user operations, reproduces recorded image files, and lenses. The operation of each necessary part is controlled with respect to the operation of the lens system 10 such as zoom, focus, and aperture adjustment in the lens barrel 3, the user interface operation, and the like.

The camera control unit 18 also performs various processing and output control on the image data processed by the camera signal processing unit 13.
The camera control unit 18 can execute the electronic image stabilization process on the image data by the electronic image stabilization control unit 35.
Further, the camera control unit 18 causes the image stabilization metadata processing unit 36 to execute metadata generation related to image stabilization. The camera control unit 18 also controls to generate metadata including various information including information related to image stabilization and record it as information related to the image file MF.
Further, the camera control unit 18 communicates with the lens control unit 20 on the lens barrel 3 side via the communication control unit 33.

In FIG. 3, the electronic image stabilization control unit 35 and the image stabilization metadata processing unit 36 are shown in separate blocks from the camera control unit 18, but these are realized by the microcomputer constituting the camera control unit 18. It can be thought of as a function. Therefore, for the sake of explanation, the camera control unit 18, the electronic image stabilization control unit 35, and the image stabilization metadata processing unit 36 are collectively referred to as a “control unit 40”. However, these may be configured by a separate arithmetic processing unit.

The recording control unit 14 records and reproduces, for example, a recording medium using a non-volatile memory. The recording control unit 14 performs a process of recording an image file MF such as moving image data or still image data, a thumbnail image, or the like on a recording medium, for example.
The actual form of the recording control unit 14 can be considered in various ways. For example, the recording control unit 14 may be configured as a flash memory built in the image pickup device 1 and a write / read circuit thereof, or a recording medium that can be attached to and detached from the image pickup device 1, such as a memory card (portable flash memory, etc.). ) May be in the form of a card recording / playback unit that performs recording / playback access. Further, it may be realized as an HDD (Hard Disk Drive) or the like as a form built in the image pickup apparatus 1.

The display unit 15 is a display unit that displays various displays to the user, and is a display device such as a liquid crystal panel (LCD: Liquid Crystal Display) or an organic EL (Electro-Luminescence) display arranged in the housing of the image pickup device 1, for example. It is said to be a display panel or viewfinder.
The display unit 15 causes various displays to be executed on the display screen based on the instructions of the camera control unit 18.
For example, the display unit 15 displays a reproduced image of image data read from a recording medium by the recording control unit 14.
Further, the display unit 15 is supplied with image data of the captured image whose resolution has been converted for display by the camera signal processing unit 13, and the display unit 15 is based on the image data of the captured image according to the instruction of the camera control unit 18. May be displayed. As a result, a so-called through image (subject monitoring image), which is an captured image during composition confirmation, is displayed.
Further, the display unit 15 causes various operation menus, icons, messages, etc., that is, display as a GUI (Graphical User Interface) to be executed on the screen based on the instruction of the camera control unit 18.

The output unit 16 performs data communication and network communication with an external device by wire or wirelessly.
For example, the image data (still image file or moving image file) is transmitted and output to an external display device, recording device, playback device, or the like.
Further, assuming that the output unit 16 is a network communication unit, it communicates with various networks such as the Internet, a home network, and a LAN (Local Area Network), and transmits / receives various data to / from a server, a terminal, etc. on the network. You may do so.

The operation unit 17 collectively shows input devices for the user to perform various operation inputs. Specifically, the operation unit 17 shows various controls (keys, dials, touch panels, touch pads, etc.) provided in the housing of the image pickup apparatus 1.
The operation unit 17 detects the user's operation, and the signal corresponding to the input operation is sent to the camera control unit 18.

The shutter drive unit 31 drives the shutter 11 based on the instruction of the camera control unit 18.

The correction unit drive unit 32 drives the image stabilization unit 30 based on the instruction of the camera control unit 18, and displaces the image sensor in the image sensor unit 12 for optical image stabilization.

The blur detection unit 34 shows a sensor that detects the vibration applied to the camera body 2. As the blur detection unit 34, for example, an IMU (inertial measurement unit) is mounted. For example, an angular velocity is detected by a three-axis angular velocity (gyro) sensor of pitch-, yaw, and roll, and acceleration is detected by an acceleration sensor. Can be detected.
The blur detection unit 34 may include a sensor capable of detecting camera shake during imaging, and does not need to include both a gyro sensor and an acceleration sensor.

In the lens barrel 3, for example, a lens control unit 20 by a microcomputer is mounted.
When the lens barrel 3 is attached to the camera body 2, the camera control unit 18 and the lens control unit 20 can communicate with each other via the communication control units 27 and 33.
The communication control units 27 and 33 are connected by wire when the lens barrel 3 is attached to the camera body 2 to perform communication. However, both may be configured to perform wireless communication.

The lens control unit 20 and the camera control unit 18 constantly exchange bidirectional data communication at a certain communication speed.
For example, the camera control unit 18 instructs the lens control unit 20 to drive the zoom lens 10a, the focus lens 10d, the aperture mechanism 10b, and the camera shake correction lens mechanism 10c. The lens control unit 20 executes the operation of the lens system 10 in response to these drive instructions.
Further, the lens control unit 20 transmits the distortion correction information of the lens, the focal length information, the position of the correction lens of the camera shake correction lens mechanism 10c, and the like to the camera control unit 18.

The lens barrel 3 drives, for example, a zoom drive unit 21 having a motor and a motor driver for driving the zoom lens 10a, an aperture drive unit 22 having a motor and a motor driver for driving the aperture mechanism 10b, and a camera shake correction lens mechanism 10c. A correction lens drive unit 23 having a motor and a motor driver, and a focus drive unit 24 having a motor and a motor driver for driving the focus lens 10d are provided.
The zoom drive unit 21, the aperture drive unit 22, the correction lens drive unit 23, and the focus drive unit 24 apply a drive current to the corresponding motor in response to an instruction from the lens control unit 20 based on an instruction from the camera control unit 18. .. As a result, a zoom operation, an aperture opening / closing operation, an optical image stabilization operation, and a focus operation are executed.

The memory unit 26 stores information and the like used for processing by the lens control unit 20. The memory unit 26 comprehensively shows, for example, a ROM, a RAM, a flash memory, and the like. The memory unit 26 may be used for temporarily storing information transmitted by the lens control unit 20 to the camera control unit 18.

The blur detection unit 25 indicates a sensor that detects vibration in the lens barrel 3, and it is assumed that an IMU is mounted in the same manner as the blur detection unit 34 on the camera body 2 side, for example. It is also assumed that the blur detection unit 34 is not mounted in the lens barrel 3.

Subsequently, the contents of the image file MF and the contents of the metadata transmitted from the image pickup apparatus 1 to the image processing apparatus 5 will be described.
FIG. 5A shows the data included in the image file MF. As shown in the figure, the image file MF includes various data as "header", "sound", "movie", and "metadata".

In the "header", information such as a file name and a file size as well as information indicating the presence or absence of metadata are described.
"Sound" is audio data recorded with a moving image. For example, 2-channel stereo audio data is stored.
The "movie" is moving image data, and is composed of image data as each frame (# 1, # 2, # 3 ...) constituting the moving image.
As the "metadata", additional information associated with each frame (# 1, # 2, # 3, ...) Constituting the moving image is described.

An example of the content of the metadata is shown in FIG. 5B. For example, IMU data, coordinate conversion parameter HP, timing information TM, and camera parameter CP are described for one frame. It should be noted that these are part of the metadata content, and other information may be possible. In addition, any of the information shown in the figure may not be included.
Further, FIG. 5B shows a case where the communication speed information is included in the metadata by the broken line, which is the case where the processing of the fifth and sixth embodiments described later is performed.

As the IMU data, a gyro (angular velocity data), an accelerator (acceleration data), and a sampling rate are described.
The IMU mounted on the image pickup apparatus 1 as the

blur detection units

34 and 25 outputs the angular velocity data and the acceleration data at a predetermined sampling rate. Generally, this sampling rate is higher than the frame rate of the captured image, so that many IMU data samples can be obtained in one frame period.

Therefore, as the angular velocity data, n samples are associated with each frame, such as gyro sample # 1, gyro sample # 2, ... Gyro sample # n shown in FIG. 5C.
Also, as acceleration data, m samples are associated with each frame, such as accelerator sample # 1, accelerator sample # 2, ... accelerator sample # m.
In some cases, n = m, and in some cases, n ≠ m.
Although the metadata is described here in the example of being associated with each frame, for example, the IMU data may not be completely synchronized with the frame. In such a case, for example, the time information associated with the time information of each frame is provided as the IMU sample timing offset in the timing information TM.

The coordinate conversion parameter HP is a general term for parameters used for correction accompanied by coordinate conversion of each pixel in an image. It also includes non-linear coordinate transformations such as lens distortion.
The coordinate conversion parameter HP is a term that can include at least a lens distortion correction parameter, a trapezoidal distortion correction parameter, a focal plane distortion correction parameter, an electronic camera shake correction parameter, and an optical camera shake correction parameter.

The lens distortion correction parameter is information for directly or indirectly grasping how the distortion such as barrel aberration and pincushion aberration is corrected and returning to the image before the lens distortion correction.

The trapezoidal distortion correction parameter is a correction amount for correcting trapezoidal distortion caused by shifting the cutout area from the center by electronic image stabilization, and is also a value corresponding to the correction amount for electronic image stabilization.

The focal plane distortion correction parameter is a value indicating the amount of correction for each line with respect to the focal plane distortion.

Regarding electronic image stabilization and optical image stabilization, it is a parameter that indicates the amount of correction in each axial direction of yaw, pitch, and roll.

Here, in the present embodiment, as described in FIG. 3, as the optical image stabilization mechanism, the image stabilization lens mechanism 10c and the image stabilization unit 30 are provided. Therefore, as the correction information indicating the correction amount of the optical image stabilization, for example, the body side image stabilization value and the lens side image stabilization value are recorded as shown in FIG. 5D.
The body-side image stabilization value is an image stabilization value in the image stabilization unit 30.
The lens-side image stabilization value is an image stabilization value in the camera shake correction lens mechanism 10c.

It is assumed that these image stabilization values are the correction execution values of the actual image pickup surface image stabilization unit 30 and the image stabilization lens mechanism 10c. This correction effective value is the position information that changes due to the actual correction, which is detected by the position sensor provided in the image stabilization unit 30 and the camera shake correction lens mechanism 10c, and the position information from the previous frame. It is a value representing the actual displacement performed as optical image stabilization, such as the amount of displacement of.

Alternatively, these blur correction values are a correction instruction value output by the camera control unit 18 to the correction unit drive unit 32, or a correction instruction value transmitted by the camera control unit 18 to the correction lens drive unit 23 via the lens control unit 20. It may be. This is because the image pickup surface camera shake correction unit 30 and the camera shake correction lens mechanism 10c are driven so that the position and the displacement amount of the position correspond to these correction instruction values.

The parameters of lens distortion correction, trapezoidal distortion correction, focal plane distortion correction, and electronic camera shake correction are collectively referred to as coordinate conversion parameters, but these correction processes are performed by each of the image sensors of the image sensor unit 12. This is because it is a correction process for the image formed on the pixel and is a parameter of the correction process accompanied by the coordinate conversion of each pixel.
Also, for the sake of explanation, the correction information for optical image stabilization is also one of the coordinate conversion parameters, because in optical image stabilization, the correction of the fluctuation of the inter-frame component is a process that involves coordinate conversion of each pixel. be.
That is, if the reverse correction is performed using these parameters, the image data to which the lens distortion correction, the trapezoidal distortion correction, the focal plane distortion correction, the electronic camera shake correction, and the optical camera shake correction are performed is performed before each correction processing, that is, It is possible to return to the state when the image was formed on the image sensor of the image sensor unit 12.

In addition, each parameter of lens distortion correction, trapezoidal distortion correction, and focal plane distortion correction is distortion correction processing for the case where the optical image itself from the subject is an image captured in an optically distorted state, and each is optical distortion. Since it is intended for correction, it is collectively referred to as an optical distortion correction parameter.
That is, if the reverse correction is performed using these parameters, the image data to which the lens distortion correction, the trapezoidal distortion correction, and the focal plane distortion correction have been performed can be returned to the state before the optical distortion correction.

The timing information TM in the metadata includes each information of exposure time (shutter speed), exposure start timing, readout time (curtain speed), number of exposure frames (long exposure information), IMU sample offset, and frame rate.
These are mainly used to associate the line of each frame with the IMU data.
However, even if the image sensor 12a is a CCD or a global shutter type CMOS, if the exposure center of gravity shifts using an electronic shutter or mechanical shutter, the exposure start timing and curtain speed are also used to match the exposure center of gravity. Correction is possible.

The angle of view (focal length), zoom position, and lens distortion information are described as the camera parameter CP in the metadata.

<3. Functions of image processing equipment>
In the image processing device 5, the shake change processing can be performed on the image file MF generated by the image pickup by the image pickup device 1.
The functional configuration of the image processing apparatus 5 for this purpose is shown in FIG. It is assumed that the image processing device 6 also has a similar configuration.

The image processing device 5 may be an information processing device such as the mobile terminal 7 or the personal computer 8 shown in FIG. 1. In that case, the function as shown in FIG. 6 is formed by the application program.
For example, the image processing device 5 has functions as a correction canceling unit 51, a shaking effect unit 52, a processing setting unit 53, and a user interface unit 54. The "user interface" is also referred to as "UI", and the user interface unit 54 is hereinafter referred to as "UI unit 54".

The correction canceling unit 51 and the shaking effecting unit 52 are functions for changing the shaking of the image.

The correction canceling unit 51 cancels the optical image stabilization and the electronic image stabilization applied by the image pickup apparatus 1, and the image stabilization is not applied, that is, the image is shaken due to the original image stabilization. It is a function to change the shaking like returning to.
Alternatively, the correction canceling unit 51 performs a process of canceling only the correction by the camera shake correction lens mechanism 10c, a process of canceling only the correction by the image pickup surface camera shake correction unit 30, a process of canceling only the electronic camera shake correction, and the like. You can also.

The shaking effect unit 52 is a function of performing a process of changing the shaking state of the image data according to a parameter or an instruction input by the user.
The shaking effecting unit 52 may add or remove shaking to, for example, the image data VD1 that has not been canceled by the correction canceling unit 51, or the image data VD1 that has been canceled by the correction canceling unit 51. Directing processing can be performed.
As the shaking effect processing, it is assumed that the shaking of the image is reduced, the shaking is removed with higher accuracy than the camera shake correction of the image pickup apparatus 1, or the shaking is added to the image.

The UI unit 54 is a function that causes the user to present an operator regarding correction cancellation and shaking change, and also performs a process of acquiring operation information by the operator.

The processing setting unit 53 sets the processing parameters for correction cancellation based on the metadata MTD1 and executes the processing of the correction cancellation unit 51. Further, the processing setting unit 53 sets the processing parameters for changing the shaking according to the user operation detected by the UI unit 54 and the metadata MTD1, and causes the processing of the shaking effect unit 52 to be executed.

With such an image processing device 5, the user can make desired fluctuation changes to the image file MF obtained by the image pickup device 1.
In particular, since the metadata MTD1 contains information on optical image stabilization, it is possible to recognize the correction by the image stabilization lens mechanism 10c and the correction by the image stabilization unit 30, and perform processing according to each. It becomes. For example, it is possible to cancel the image stabilization and add an appropriate amount of shaking.

<4. Setting process related to image stabilization>
In the image pickup apparatus 1 of the present embodiment, camera shake correction is performed by the camera shake correction lens mechanism 10c in the lens barrel 3 and the camera shake correction unit 30 on the image pickup surface in the camera body 2. By simultaneously functioning the image stabilization on both the lens barrel 3 side and the camera body 2 side, there are merits such as wide range of movement.
Also, when capturing a moving image, by saving the motion data log as metadata in a state where the moving data is matched with the image in time series, it is possible to create a moving image in which the metadata can be used later. For example, it is conceivable to store IMU data.
In addition, as shown in FIG. 5D as the body side image stabilization value and the lens side image stabilization value, by recording the information related to the image stabilization on the lens barrel 3 side and the camera body 2 side in the metadata. Even for videos recorded while applying image stabilization, it is possible to cancel the image stabilization later, or to remove even larger image stabilization that could not be removed by the lens or body by cutting out.

Here, when recording a moving image on the camera body 2 side, the camera control unit 18 can easily save the body side image stabilization value as metadata as it is, but the lens side image stabilization value is the same as that of the lens control unit 20. It will be received by communication between them and will be used as metadata.

In such an image pickup apparatus 1, it is conceivable that the camera shake correction of the low frequency component is performed on the lens barrel 3 side and the high frequency component is corrected on the camera body 2 side.
When performing image stabilization on the lens barrel 3 side and the camera body 2 side together, it is conceivable to simply move the lens barrel 3 side and the camera body 2 side by half the amount of movement. It will be divided for each frequency component.
The side that corrects the high frequency component often has a small amount of movement, and the side that corrects the low frequency component often has a large amount of movement. Therefore, if we think simply, we can think of a way of allocating the high frequency component to the narrow range of motion and the low frequency component to the wide range of motion, but in this embodiment, it is movable. The lens barrel 3 side is responsible for the low frequency component, not the circumstances of the range. This is also related to the fact that the lens-side image stabilization value is sent to the camera control unit 18 by communication between the lens control unit 20 and the camera control unit 18 (hereinafter, also referred to as “lens-body communication”).

By correcting the low frequency component of camera shake on the lens barrel 3 side, the change in the movement of the camera shake correction lens mechanism 10c that moves per unit time becomes smaller than when the high frequency component is corrected. The small change in motion means that the data as the lens-side image stabilization value is small even if it is intermittent to some extent. Therefore, it is possible to reduce the sampling rate required to accurately express the movement of the camera shake correction lens mechanism 10c when the low frequency component is corrected. Then, as the lens-side image stabilization value transmitted from the lens control unit 20 to the camera control unit 18, it is possible to send data with sufficient accuracy even at a low sampling rate. This has the advantage of reducing the amount of communication data (even if communication is high speed, it can be transmitted with a small amount of data), and even when communication is low speed, it is possible to send the lens side blur correction value without reducing the accuracy. There are advantages.

In order to allocate such frequency components, for example, the camera control unit 18 performs the setting process as shown in FIG. 7 for moving image imaging.

In step S1 of FIG. 7, the camera control unit 18 branches the process depending on whether or not metadata recording is performed. For example, one of them is selected according to the user's operation or the setting information from the outside. When the mode is set to not record metadata, the subsequent processing is not executed.

In the mode of performing metadata recording, the camera control unit 18 is set in step S2 to perform metadata recording with moving image imaging.
In step S3, the camera control unit 18 branches the process depending on whether or not the image stabilization is turned on. For example, one of them is determined according to the setting operation of the user. When the camera shake correction is not performed, the process of FIG. 7 is completed.

When performing image stabilization, the camera control unit 18 branches the process in step S4 depending on whether or not the camera body 2 performs image stabilization. This is selected according to, for example, a user setting or an application program specification.

When performing camera shake correction on the camera body 2 side, the camera control unit 18 is set to record the body side shake correction value as metadata in step S5.

In step S6, the camera control unit 18 branches the process depending on whether or not the camera shake correction is performed on both the camera body 2 side and the lens barrel 3 side. This is also selected according to, for example, a user setting or an application program specification.
Further, the mounted lens barrel 3 may be a lens barrel that does not have the camera shake correction lens mechanism 10c. In that case, image stabilization is selected only on the camera body 2 side.
If the camera shake correction is not performed on the lens barrel 3 side, the process of FIG. 7 is completed.

When performing camera shake correction on both the camera body 2 side and the lens barrel 3 side, the camera control unit 18 determines in step S7 whether or not to share each frequency component.
The following examples can be considered as this determination.

First, when performing camera shake correction on both the camera body 2 side and the lens barrel 3 side, it is conceivable to always share the frequency components.
As a result, the above-mentioned effective operation can always be performed regardless of the circumstances of the lens-body communication.
Further, it may be decided according to the user's selection operation.

Further, it is conceivable to determine whether or not to share the frequency components according to the model, model number, type, year of manufacture, software version, etc. of the lens barrel 3.
That is, when the communication performance is determined from the model of the lens barrel 3 and the communication speed performance is equal to or higher than a predetermined value, the frequency components are not shared and the communication speed performance is not higher than the predetermined value, that is, the communication speed is slow. Is to share the frequency components.

Further, the camera control unit 18 measures the communication speed (for example, bps: bits per second) as a result of communicating with the lens control unit 20 with the lens barrel 3 connected, and the communication speed is equal to or higher than a predetermined value. If this is the case, the frequency component may not be shared, and if the communication speed performance is not equal to or higher than a predetermined value, that is, the communication speed is slow, the frequency component may be used for sharing.

Although it is described that whether or not the frequency component is shared is determined by the setting process of FIG. 7, whether or not the frequency component is shared may be arbitrarily switched. For example, it may be possible to switch during video recording.

If the frequency component is not shared, the camera control unit 18 proceeds to step S9.
When sharing the frequency components, the camera control unit 18 sets the frequency range in step S8. That is, after setting the range of the blur frequency as the low frequency component in charge of the lens barrel 3 side and the range of the blur frequency as the high frequency component in charge of the camera body 2, the process proceeds to step S9.

It is conceivable that the low frequency / high frequency separation frequency in step S8 is determined as follows.
First, it is conceivable to make a decision according to the user's designated operation.
Further, it is conceivable to determine the frequency according to the model, model number, type, year of manufacture, software version, etc. of the lens barrel 3. That is, an appropriate frequency is determined according to the communication speed determined from these, the movable range of the camera shake correction lens mechanism 10c, the frequency response characteristic, and the like.
Further, it may be determined according to the actually measured communication speed between the lens and the body.
Further, in the case of the processing of the second and third embodiments described later, it is also appropriate to determine the frequency according to the capacity of the memory unit 26 on the lens barrel 3 side and the recordable time of the lens side image stabilization value. It becomes.

In step S9, the camera control unit 18 selects whether or not to set the operation mode in which the data of the lens-side image stabilization value is temporarily stored in the memory unit 26 on the lens barrel 3 side. This operation will be described in the second and third embodiments, but the lens control unit 20 temporarily stores the acquired lens-side image stabilization value in the memory unit 26, and the camera control unit 18 at a predetermined time point. It is intended to be sent.
Whether or not to select such an operation mode may be determined according to the user's operation, the communication speed determined from the model, model number, type, year of manufacture, software version, etc. of the lens barrel 3, or It may be determined based on the measured value of the communication speed.

If such an operation mode is not set, the camera control unit 18 proceeds to step S13. When performing such a transmission operation via the memory unit 26, the camera control unit 18 sets the transmission timing in step S10.
Although described in the second and third embodiments, it is assumed that the transmission timing includes periodic transmission, irregular transmission, timing according to the storage capacity of the memory unit 26, and the end of video recording. As such a transmission timing, it is necessary to set at what timing the transmission is performed.

In step S13, the camera control unit 18 receives the lens-side image stabilization value by lens-body communication and sets it to be recorded as metadata. That is, in this case, both the body-side image stabilization value and the lens-side image stabilization value are included in the metadata as correction information in combination with the setting in step S5.

If it is determined in step S4 that camera shake correction is not performed on the camera body 2 side, the camera control unit 18 is set to perform camera shake correction only on the lens barrel 3 side in step S11. Then, in step S12, the camera control unit 18 selects whether or not to set the mode for temporarily storing the lens-side image stabilization value in the memory unit 26, proceeds to step S10 when the mode is set, and when the mode is not set. The process proceeds to step S13.

<5. First Embodiment>
8 and 9 show a processing example of the control unit 40 (mainly the camera control unit 18 and the image stabilization metadata processing unit 36) during video recording as the first embodiment.
In this example, the camera control unit 18 is set to perform camera shake correction of high frequency components on the camera body 2 side and camera shake correction of low frequency components on the lens barrel 3 side.
Further, on the lens barrel 3 side, it is assumed that the lens control unit 20 transmits the lens side image stabilization value in real time without temporarily storing it in the memory unit 26.

FIG. 8 is an example of processing related to image stabilization executed by the control unit 40 at each frame timing during video recording. FIG. 9 is an example of processing according to reception from the lens control unit 20.

In step S101, the control unit 40 detects the amount of blurring. For example, from the IMU data by the blur detection unit 34, the amount of shaking generated between the timings of the previous frame and the current frame is detected. In this case, the IMU data of the blur detection unit 25 in the lens barrel 3 may be referred to.

In step S102, the control unit 40 extracts the high frequency component and the low frequency component of the blur according to the setting of the frequency component sharing (setting of the shared frequency range) in step S8 of FIG. 7.

In step S103, the control unit 40 calculates the body-side image stabilization amount. The body-side image stabilization amount is a correction amount executed by the image pickup surface camera shake correction unit 30, and is a correction amount corresponding to the high-frequency component of the blur extracted in step S102.
Further, in step S104, the control unit 40 calculates the lens-side image stabilization amount. The lens-side image stabilization amount is a correction amount executed by the camera shake correction lens mechanism 10c, and is a correction amount corresponding to the low frequency component of the blur extracted in step S102.
For example, the control unit 40 selects the image stabilization unit 30 and the image stabilization lens mechanism 10c based on the high-frequency component and the low-frequency component of the amount of blur added to the image pickup device 1 detected in step S101. Calculate whether blur correction can be performed by operating only, and set the blur correction amount for each.

In step S105, the control unit 40 transmits a correction instruction value according to the blur correction amount.
That is, the position and the position displacement for instructing the correction amount for the high frequency blur are transmitted to the correction unit drive unit 32 to indicate the position and the position displacement amount, and the position and the position displacement for executing the correction for the low frequency blur. The correction instruction value indicating the amount is transmitted to the lens control unit 20.
As a result, the correction unit drive unit 32 drives the image pickup surface camera shake correction unit 30 corresponding to the blur of the high frequency component.
Further, the lens control unit 20 transmits a correction instruction value to the correction lens drive unit 23, and the correction lens drive unit 23 drives the camera shake correction lens mechanism 10c corresponding to the blur of the low frequency component.
As a result, optical image stabilization is performed on both the lens barrel 3 side and the camera body 2 side, and it functions to reduce image shake due to camera shake or the like applied to the image pickup apparatus 1.

In step S106, the control unit 40 performs a process of using the body-side image stabilization value as metadata. This body-side image stabilization value is recorded corresponding to the current frame.
On the other hand, regarding the lens-side image stabilization value, the control unit 40 waits for reception from the lens control unit 20.
In step S121 of FIG. 9, reception of the lens-side image stabilization value from the lens control unit 20 is waited for, and if there is reception, processing is performed in step S122 to use the lens-side image stabilization value as metadata.

The body-side image stabilization value and the lens-side image stabilization value set as metadata in the above steps S106 and S122 are individually or simultaneously associated with the current frame, and are recorded as metadata in the recording control unit 14. Is recorded on a recording medium.

By recording the metadata as described above, for example, in the image processing device 5, the image information of each frame is subjected to image stabilization, that is, the amount of correction indicated by the body-side image stabilization value, and the lens. The amount of correction indicated by the side shake correction value can be offset. Also, even when distortion correction is performed by image processing, the information in the metadata MTD1 is used to create a state when distortion correction is not performed, and an image when blur correction or distortion correction is not performed is created. Can be created.

Then, using the IMU data recorded as shown in FIG. 5C, the amount of blurring of the camera body 2 was recalculated from the information of the gyro sensor and the acceleration sensor, and the above-mentioned blurring correction and distortion correction were not performed. It is possible to perform blur correction and distortion correction again for the image in the state of the case. That is, it is possible to realize image stabilization later on the moving image data captured in the past.

Note that FIG. 10 shows the processing of IMU data by the control unit 40.
The control unit 40 takes in the IMU data detected by the blur detection unit 34 in step S161 at each predetermined sampling timing, and performs a process for recording as metadata in step S162. As described with reference to FIG. 5C, the IMU data is not necessarily synchronized with the frame timing of the moving image, and therefore, for example, it is controlled so that a plurality of IMU data are recorded as metadata per frame.
As a result, the time-series IMU data and the frame of the moving image are associated with each other.
It is assumed that such recording of IMU data as metadata is performed even in the case of each embodiment described below.

<6. Second Embodiment>
As a second embodiment, an example in which the lens control unit 20 transmits the lens side image stabilization value while temporarily storing it in the memory unit 26 will be described. From now on, the same step numbers will be assigned to the above-mentioned processes, and detailed duplicate explanations will be avoided.

The body-side image stabilization value and the lens-side image stabilization value may be finally recorded as metadata of the moving image. Therefore, it is conceivable that the lens control unit 20 does not send the lens-side image stabilization value in real time, but temporarily stores it in the memory unit 26 and sends it when there is a margin in the lens-body communication. ..

In this case, if the low frequency component blur correction is performed on the lens barrel 3 side, the change in the movement of the camera shake correction lens mechanism 10c that moves per unit time is smaller than that when the high frequency component is corrected. Therefore, the accuracy is maintained even if the amount of data of the lens-side image stabilization value is thinned out to some extent according to the frequency.
Further, for this reason, the lens side image stabilization value can be stored in the form of thinning out the data according to the low frequency component moved on the lens barrel 3 side, and the memory unit 26 also has the advantage of temporarily reducing the storage capacity.

In addition, the lens-body communication is very slow in the first place, and if the movement of the camera shake correction lens mechanism 10c is sampled at that communication interval, the change in movement per time that is lost is large, and the accuracy as data is maintained. Even when the lens-side image stabilization value cannot be sent to the camera body 2 side in real time, the process of temporarily storing it in the memory unit 26 and transmitting it later is useful.

From this point of view, the lens control unit 20 performs the processing of FIG. 11 during moving image imaging and the like.
In step S201, the lens control unit 20 determines whether or not it is the transmission timing. This transmission timing is the timing set in step S10 of FIG.
In the process of FIG. 7, the camera control unit 18 sets periodic transmission, irregular transmission, timing according to the storage capacity of the memory unit 26, the end of video recording, and the like, and notifies the lens control unit 20. .. The lens control unit 20 sets the transmission timing of the lens-side image stabilization value according to the notified setting content, and in step S201, determines whether or not it is the current timing.

If it is not the transmission timing, the lens control unit 20 proceeds to step S202 and detects a lens-side image stabilization value as a correction execution value in the camera shake correction lens mechanism 10c. Then, in step S203, the lens-side image stabilization value detected this time is stored in the memory unit 26 in association with the time stamp indicating the current time (time corresponding to the current frame).
This is because it is necessary to set the time stamp and transmit it on the camera body 2 side in order to associate the lens side image stabilization value with the frame of the moving image.

Even when the lens-side image stabilization value is sent in real time as in the first embodiment described above, there may be some delay. Therefore, even in the case of the first embodiment, it is desirable that the lens-side image stabilization value is transmitted as a set with the time stamp.
Further, in the case of transmission after temporary storage as in the second embodiment, it is necessary to send the lens-side image stabilization value together with the time stamp.

When the transmission timing is reached, the lens control unit 20 proceeds to step S204 and performs a process of transmitting the lens-side image stabilization value recorded in the memory unit 26 to the camera control unit 18 together with the time stamp.

When the transmission timing is set by the process of FIG. 11, the lens-side image stabilization values of the plurality of samples stored in the memory unit 26 are collectively transmitted to the camera control unit 18.
For example, as a periodic timing, batch transmission is performed at regular intervals. In addition, as an irregular timing, batch transmission is performed in response to some trigger. By performing batch transmission periodically or irregularly, the transmission opportunity can be reduced as compared with the case where the lens-side image stabilization value is sequentially transmitted in real time, and the communication load can be reduced.
In addition, batch transmission may be performed at a timing according to the storage capacity of the lens-side image stabilization value in the memory unit 26. As a result, it is possible to avoid a situation in which the lens-side image stabilization value cannot be stored in the memory unit 26.
In addition, it may be transmitted all at once after the video imaging is completed. As a result, transmission can be performed when there is a margin in communication.
These transmission timings may be used together.

The control unit 40 of the camera body 2 performs the process of FIG. 12 in addition to the process of FIG. 8 described above when recording a moving image.
The control unit 40 waits for the lens side image stabilization value to be received from the lens control unit 20 in step S140 of FIG.
When the lens-side image stabilization value is received from the lens control unit 20, in step S141, a process of using the lens-side image stabilization value as metadata is performed. At this time, a time stamp is transmitted corresponding to each lens-side image stabilization value. Therefore, the control unit 40 determines which frame of the moving image corresponds to the lens-side image stabilization value for each received lens-side image stabilization value, and corresponds to the determined frame. It is metadata.

The body-side image stabilization value and the lens-side image stabilization value set as metadata in step S106 of FIG. 8 and step S141 of FIG. 12 are individually or simultaneously associated with the current frame and are metadata. Is recorded on the recording medium in the recording control unit 14.

It should be noted that the user can select whether to transmit the lens-side image stabilization value in real time as in the first embodiment or to temporarily store and transmit the lens-side image stabilization value at a set timing as in the second embodiment. May be.
While temporarily storing and transmitting later has an advantage in terms of communication load as described above, by transmitting in real time, the memory unit 26 on the lens barrel 3 side can be reduced or transmitted later. There are also merits such as reduction of transmission time and metadata processing time. Therefore, it is also appropriate to let the user select according to the situation.

<7. Third Embodiment>
The third embodiment is an example in which real-time transmission and transmission after temporary storage are used in combination.
FIG. 13 shows the processing of the lens control unit 20.

In step S201A, the lens control unit 20 determines whether or not it is the batch transmission timing of the lens-side image stabilization values stored in the memory unit 26.
If it is not the transmission timing, the lens control unit 20 proceeds to step S202 and detects a lens-side image stabilization value as a correction execution value in the camera shake correction lens mechanism 10c.

In step S230, the lens control unit 20 determines whether or not there is a transmission margin at present. For example, it is determined whether or not there is a transmission margin based on the current communication speed, the amount of data to be transmitted, and the like.
If there is a transmission margin, the lens control unit 20 proceeds to step S231 and transmits the lens-side image stabilization value detected this time to the camera control unit 18 together with the time stamp.
On the other hand, if there is no transmission margin, in step S232, the lens-side image stabilization value detected this time is stored in the memory unit 26 in association with a time stamp indicating the current time (time corresponding to the current frame).

When the transmission timing is reached in step S201, the lens control unit 20 proceeds to step S204 and performs a process of transmitting the lens-side image stabilization value recorded in the memory unit 26 to the camera control unit 18 together with the time stamp.

By the process of FIG. 13, real-time transmission and batch transmission after temporary storage are used together for the lens-side image stabilization value. By pairing both with the time stamp, the control unit 40 can appropriately associate it with the frame of the moving image and convert it into metadata.
By using the real-time transmission together, there is an advantage that the capacity of the memory unit 26 can be reduced.

<8. Fourth Embodiment>
As a fourth embodiment, an example in which the control unit 40 interpolates the lens-side image stabilization value will be described.
The lens-side image stabilization value transmitted from the lens control unit 20 to the camera control unit 18 has a slow sampling rate, and may be missing in time series when it is linked to a moving image. In other words, it is expected that all frames will not be supported. Therefore, the lens-side image stabilization value is generated by interpolation processing.

FIG. 14 shows a processing example of the control unit 40. The control unit 40 performs the processing of FIG. 14 for each frame of the moving image during the moving image recording or after the moving image recording is completed.
In step S130, the control unit 40 determines whether or not the lens-side image stabilization value corresponding to the currently targeted frame has been received.
If the corresponding lens-side image stabilization value does not exist, the process proceeds to step S131, and the lens-side image stabilization value corresponding to the target frame is generated by interpolation processing. For example, interpolation is performed by linear interpolation processing using the lens-side image stabilization value in the front and rear frames.

In step S121, the received lens-side image stabilization value or the lens-side image stabilization value generated by the interpolation processing is used as metadata.

By doing so, even when the sampling rate is lowered, the lens-side image stabilization value corresponding to each frame can be left as metadata.
In particular, the camera shake correction lens mechanism 10c corrects camera shake of low-frequency components, and since there are few small changes in position, the lens-side image stabilization value obtained by linear interpolation or the like is also relatively accurate. Using this, the image processing device 5 can also realize an appropriate shake change.

<9. Fifth Embodiment>
The fifth embodiment makes it possible to show the reliability of the lens-side image stabilization value when both the body-side image stabilization value and the lens-side image stabilization value are recorded as metadata.
As described above, the control unit 40 needs to wait for the notification from the lens control unit 20 regarding the lens-side image stabilization value.

Also in the fifth embodiment, the correction amount of the camera shake correction lens mechanism 10c of the lens barrel 3 is also set and corrected on the camera control unit 18 side as in the first to fourth embodiments described above. The indicated value may be transmitted to the lens control unit 20, but on the lens barrel 3 side, the lens control unit 20 may set the correction amount of the camera shake correction lens mechanism 10c.
That is, the camera control unit 18 sets the correction amount of the image stabilization unit 30 according to the detection value of the blur detection unit 34, executes the optical camera shake correction on the camera body 2 side, and the lens control unit 20 performs the blur correction. This is an example in which the correction amount of the camera shake correction lens mechanism 10c is set according to the detection value of the detection unit 25, and the optical camera shake correction on the lens barrel 3 side is executed.

In any case, the control unit 40 uses the body-side image stabilization value and the lens-side image stabilization value transmitted from the lens control unit 20 as metadata.
In this case, the control unit 40 performs the processing as shown in FIG. 15A, and the image processing apparatus 5 performs the processing as shown in FIG. 15B.

FIG. 15A is an example in which the control unit 40 performs a process of recording communication speed information (for example, bps) as metadata.
In step S401, the control unit 40 detects and holds the communication speed of the lens-body communication, that is, the communication speed information between the lens control unit 20 and the camera control unit 18.
In step S402, the control unit 40 records the communication speed information in the metadata together with the image data (see the broken line portion in FIG. 5B).

If the cause of the fluctuation of the communication speed is dominated by the type of the lens barrel 3, for example, the communication speed may be recorded once for one moving image.
If there is a factor that causes the communication speed to fluctuate during video recording, it is conceivable to record the communication speed as metadata in association with each frame.

The image processing apparatus 5 that processes the image file MF performs the processing of FIG. 15B.
The image processing apparatus 5 acquires the image data VD1 and the metadata MTD1 as the image file MF in step S501. For example, the image file MF is read from the recording medium. Alternatively, the image file MF transmitted from the image pickup apparatus 1 is received.

In step S502, the image processing device 5 extracts the communication speed information included in the metadata MTD1 and compares it with the threshold value. This threshold value is a threshold value for determining whether the communication speed between the lens control unit 20 and the camera control unit 18 is in a fast state or a slow state. In other words, this threshold value is a threshold value for determining whether or not the lens-side image stabilization value included in the metadata MTD1 is suitable for use in the shake change processing.

If the communication speed is higher than the threshold value, the image processing device 5 performs high-speed communication between the lens control unit 20 and the camera control unit 18 at the time of image pickup in the image pickup device 1 of the image file MF, and the lens side blurring occurs. Assuming that the reliability of the correction value is maintained, the process proceeds to step S503 to generate a supportable flag.
In that case, the image processing apparatus 5 sets the processing based on the available flag in step S505. Specifically, the setting is such that the shake change processing can be performed using both the body-side image stabilization value and the lens-side image stabilization value.

On the other hand, if the communication speed is not higher than the threshold value, the image processing device 5 has a low communication speed between the lens control unit 20 and the camera control unit 18 at the time of imaging in the image pickup device 1 of the image file MF, and the lens side. Assuming that the reliability of the blur correction value is not maintained, the process proceeds to step S504 to generate a non-correspondence flag.
In that case, the image processing apparatus 5 sets the processing based on the incompatibility flag in step S505. For example, the lens-side image stabilization value is set so that it cannot be used for the image stabilization process.

By doing so, it is possible to prevent the image processing apparatus 5 from performing the shake change processing using an inappropriate lens-side image stabilization value.

<10. 6th Embodiment>
A processing example of the control unit 40 of the sixth embodiment is shown in FIG. 16A, and a processing example of the image processing apparatus 5 is shown in FIG. 16B. This is another processing example having the same purpose as in FIGS. 15A and 15B.

FIG. 16A is an example in which the control unit 40 performs a process of recording flag information as communication speed information.
In step S401, the control unit 40 detects and holds the communication speed (bps) between the lens barrel 3 and the camera body 2, that is, between the lens control unit 20 and the camera control unit 18.

In step S410, the control unit 40 compares the communication speed with the threshold value. This threshold value is the same as the threshold value described in FIG. 15B, and is a threshold value for determining whether the communication speed between the lens control unit 20 and the camera control unit 18 is in a fast state or a slow state.

If the communication speed is higher than the threshold value, the control unit 40 proceeds to step S412 and generates a supportable flag.
If the communication speed is not higher than the threshold value, the control unit 40 proceeds to step S413 and generates a non-correspondence flag.
Then, in step S414, the control unit 40 records the communication speed information in the metadata together with the image data (see the broken line portion in FIG. 5B). In this case, the supportable flag or the non-supportable flag is recorded as communication speed information.

The image processing apparatus 5 that processes the image file MF performs the processing of FIG. 16B.
The image processing apparatus 5 acquires the image data VD1 and the metadata MTD1 as the image file MF in step S501.

In step S510, the image processing apparatus 5 sets the processing based on the available flag or the incompatible flag recorded in the metadata MTD1. That is, if the available flag can be confirmed, the image file MF this time is set so that the shake change processing using both the body side image stabilization value and the lens side image stabilization value can be performed.
On the other hand, if the incompatible flag is confirmed, the lens-side image stabilization value is set so that it cannot be used for the shake change processing for the image file MF this time.

By doing so, it is possible to prevent the image processing apparatus 5 from performing the shake change processing using an inappropriate lens-side image stabilization value.
As the speed information included in the metadata MTD1, for example, 1-bit flag information may be used, which is advantageous for reducing the amount of data in the metadata MTD1.
For example, for a specific bit in the metadata MTD1, for example, "0" may be a supportable flag and "1" may be a non-correspondence flag.

<11. Summary and modification>
According to the above embodiment, the following effects can be obtained.
As described in the first to sixth embodiments, in the control unit 40, in the control unit 40, the body-side image stabilization value by the image stabilization unit 30 (first image stabilization function) of the image pickup surface of the camera body 2. Generates correction information including (first image stabilization value) and lens-side image stabilization value (second image stabilization value) by the camera shake correction lens mechanism 10c (second image stabilization function) in the lens barrel 3. , Performs processing to convert the correction information into metadata associated with the captured image.
This makes it possible to change the state of image shake at a later point in time, for example, in the image processing device 5. For example, both the image stabilization by the image stabilization lens mechanism 10c and the image stabilization by the image stabilization unit 30 can be canceled, or the image stabilization of only one of them can be canceled. Further, when the blur correction performed by the image pickup device 1 is canceled, the image processing device 5 can perform more precise shake correction or perform a shake effect that intentionally adds shake.
In the embodiment, the mechanical image stabilization function by the image pickup surface image stabilization unit 30 and the image stabilization lens mechanism 10c has been focused on, but the technique of the present disclosure is used when the electronic image stabilization function is adopted. It can also be applied. For example, when image stabilization is performed by the image stabilization lens mechanism 10c on the lens barrel 3 side and electronic image stabilization is performed on the camera body 2 side, the electronic image stabilization function is used as the first image stabilization function, the hand. The image stabilization lens mechanism 10c may be considered as the second image stabilization function.

In the embodiment, the control unit 40 uses the lens-side image stabilization value received from the lens barrel 3 side as the correction information.
As a result, the camera control unit 18 can acquire the image stabilization value on the lens side by the camera shake correction lens mechanism 10c of the lens barrel 3 and record it as metadata. In particular, since the lens-side image stabilization value is the correction execution value, highly accurate correction information can be recorded as metadata.

In the embodiment, an example is described in which the image stabilization unit 30 performs image stabilization for high-frequency components, and the image stabilization lens mechanism 10c performs image stabilization for low-frequency components.
The camera control unit 18 receives the lens-side image stabilization value by communication with the lens control unit 20, and records the lens-side image stabilization value as metadata. In this case, by correcting the low frequency component on the lens side, the change in the movement of the lens that moves in a short time becomes smaller than when the high frequency component is corrected, and the sampling rate at which accuracy can be maintained can be lowered. ..
Further, in particular, when the camera control unit 18 receives the lens-side image stabilization value in communication with the lens control unit 20 and uses it as metadata, the amount of communication between the camera control unit 18 and the lens control unit 20 is reduced. There are advantages that it can be reduced and that the information accuracy is not reduced even when the communication speed is low.

In the embodiment, the lens barrel 3 is detachable from the camera body 2, and the control unit 40 takes an image based on the determination of the communication speed with the mounted lens barrel 3. An example has been described in which the face camera shake correction unit 30 performs blur correction of high frequency components, and the camera shake correction lens mechanism 10c controls blur correction of low frequency components.
That is, the camera control unit 18 determines the communication speed of the mounted lens barrel 3 according to the type, communication trial, etc. at the time of step S7 in FIG. 7, and distributes the blur correction according to the frequency component. Can be set whether or not to perform.
As a result, when the lens barrel 3 having a slow communication speed is attached, it is possible to perform blur correction of low frequency components on the lens barrel 3 side.
When the lens barrel 3 having a sufficiently high communication speed is attached, by not performing such image stabilization distribution, it is not necessary to extract the components in step S102 of FIG. 8 for distribution. ..
However, even in the case of the lens barrel 3 having a high communication speed, sorting may be performed according to the frequency component, and in that case, the effect of reducing the communication amount can be obtained.

In the second and third embodiments, on the lens barrel 3 side, the lens side image stabilization value at each sample timing is stored in the memory unit 26 together with the time stamp (time information), and at a predetermined time point, the memory unit 26 stores the image stabilization value. An example of performing a process of transmitting the stored lens-side image stabilization value and the time stamp to the camera control unit 18 has been given. In this case, the control unit 40 performs a process of converting the lens-side image stabilization value received from the lens barrel 3 into metadata corresponding to the frame of the moving image based on the time stamp. The time stamp may be an absolute time, but for example, it may be the elapsed time from the start of recording the moving image. In any case, any time information that can be associated with the frame of the moving image is sufficient.
By doing so, the camera control unit 18 can acquire the lens side image stabilization value at the time corresponding to each frame regardless of the communication speed between the lens control unit 20 and the camera control unit 18. Therefore, it is possible to record an accurate image stabilization value as metadata regardless of the communication speed. It is also possible to reduce the communication processing load when recording a moving image.
Further, since the camera shake correction lens mechanism 10c is in charge of low frequency shake correction, the sampling rate of the lens side shake correction value can be lowered, which means that the amount of data of the lens side shake correction value stored in the memory unit 26 is small. It leads to what you can do. Therefore, the capacity of the memory unit 26 does not become excessive. Further, the amount of transmission data when the lens-side image stabilization value stored in the memory unit 26 is collectively transmitted to the camera control unit 18 can be prevented from becoming excessive.

In the fourth embodiment, the control unit 40 uses the lens-side image stabilization value received from the lens barrel 3 and the lens-side image stabilization value generated by interpolation processing using the received lens-side image stabilization value. An example of using correction information has been described.
As a result, not only the sampling rate of the lens-side image stabilization value but also the lens-side image stabilization value corresponding to each frame of the moving image can be obtained and recorded as metadata.

In the embodiment, the control unit 40 has described an example in which the frequency range of the low frequency component assigned to the lens barrel 3 side is set based on the communication speed with the lens barrel side.
When the communication speed between the camera control unit 18 and the lens control unit 20 is slower, it is more advantageous for the lens barrel 3 side to take charge of lower frequency image stabilization. Therefore, the low frequency range of the shake to be executed by the camera shake correction lens mechanism 10c is determined according to the communication speed. As a result, even when the communication speed is slow, it is possible to easily secure the accuracy of the lens-side image stabilization value as the metadata.

In the embodiment, the control unit 40 has described an example in which the frequency range of the low frequency component assigned to the lens barrel 3 side is set based on the storage capacity of the memory unit 26.
When the lens control unit 20 stores the lens-side image stabilization value in the memory unit 26 on the lens barrel 3 side and transmits it to the camera control unit 18 at a predetermined time point, low-frequency image stabilization is performed to determine the sampling rate. The lower the value, the larger the amount of lens-side image stabilization value that can be stored in the memory unit 26. Therefore, based on the storage capacity of the memory unit 26, the low frequency range of the shaking to be executed by the image stabilization lens mechanism 10c is determined. As a result, the amount of data stored in the memory unit 26 can be reduced and communication opportunities can be reduced. In addition, it is possible to obtain an effect such as reducing the amount of data to be transmitted even in a batch transmission at the end of a moving image.

In the example described in the fifth and sixth embodiments, the control unit 40 uses the communication speed information between the lens barrel 3 (lens control unit 20) and the camera body 2 (camera control unit 18) as metadata. It was decided to perform a process of recording on a recording medium.
As a result, it is possible to confirm from the metadata whether or not the information of the lens-side image stabilization value is affected by the communication speed. For example, if the communication speed is low and the lens-side image stabilization value is not sufficient, it is possible not to cancel the correction using it.

In the fifth embodiment, the communication speed information is a value indicating the communication speed (for example, bps). This makes it possible to confirm from the metadata whether or not the information on the lens-side image stabilization value is delayed information, or the degree of delay.

In the sixth embodiment, the communication speed information is the result information (possible flag / non-correspondence flag) in which the communication speed is compared with a predetermined value.
This makes it possible to easily confirm from the metadata whether or not the information on the lens-side image stabilization value is delayed information.

The lens barrel 3 in the embodiment includes a camera shake correction lens mechanism 10c that displaces the shake correction lens, a memory unit 26, and a lens control unit 20. Then, in the second and third embodiments, the lens control unit 20 stores the process of detecting the lens-side image stabilization value at each sample timing and the detected lens-side image stabilization value in the memory unit 26 together with the time stamp. An example of processing and processing of transmitting the lens-side image stabilization value and the time stamp stored in the memory unit 26 to the camera control unit 18 of the camera body 2 to be mounted at a predetermined time point has been given.
As a result, the camera control unit 18 can appropriately acquire the lens-side image stabilization value corresponding to the time of the frame regardless of the communication speed between the lens control unit 20 and the camera control unit 18. Therefore, the accuracy of the correction information in the metadata MTD1 can be improved.

In the second and third embodiments, the lens control unit 20 gives an example of performing a process of transmitting the lens-side image stabilization value and the time stamp stored in the memory unit 26 at intermittent timings during the moving image imaging period. rice field.
The lens control unit 20 can transmit the lens-side image stabilization value to the camera control unit 18 during video recording, depending on circumstances such as the capacity of the memory unit 26. For example, periodic transmission may be performed, or transmission may be considered every time the amount of data stored in the memory unit 26 reaches a predetermined value.

In the second and third embodiments, the lens control unit 20 gives an example of performing a process of transmitting the lens-side image stabilization value and the time stamp stored in the memory unit 26 to a time point after the moving image imaging is completed. rice field.
The lens-side image stabilization value can be transmitted to the camera control unit 18 when there is a margin in communication after the video recording is completed.
The lens control unit 20 may collectively transmit all the lens-side image stabilization values during video recording to the camera control unit 18 after the video is finished, or may transmit periodically or irregularly during video recording. After the video recording is completed, the remaining lens-side image stabilization values that have not been sent may be collectively transmitted. In any case, there is an advantage that the video is transmitted during a period in which there is a margin for communication after the video recording is completed.

In the third embodiment, the lens control unit 20 selects whether to transmit the blur correction value detected at each sample timing to the camera main body unit or to store it in the memory unit 26, and stores it in the storage unit. The blur correction value to be set is stored in the memory unit 26 together with the time stamp, read from the memory unit 26 at a predetermined time, and transmitted to the camera body 2 side together with the time stamp.
That is, by transmitting when there is a margin for transmission, storing it when it is expected to be delayed, and transmitting it later, the delay in communication is eliminated and the capacity required for the memory unit 26 is not excessive. can do.

In the embodiment, the control unit 40 describes an example of recording the information (IMU data) detected by the blur detection unit 34 as metadata.
By recording the IMU data as metadata as the information of the shaking actually applied to the image pickup apparatus 1, the shaking affecting the actual image can be determined from the metadata. This makes it possible to change the shaking in various ways.

It should be noted that the effects described in the present specification are merely examples and are not limited, and other effects may be obtained.

The present technology can also adopt the following configurations.
(1)
A first image stabilization value related to a first image stabilization function that corrects the positional relationship between an optical image incident through a lens and an output image image, and a second image stabilizer provided on a lens barrel provided with the lens. An image pickup device provided with a control unit that generates correction information including a second blur correction value related to the blur correction function as metadata associated with the captured image.
(2)
The image pickup apparatus according to (1) above, wherein the control unit uses the second blur correction value received from the lens barrel as the correction information.
(3)
The image pickup apparatus according to (1) or (2) above, wherein the first blur correction function performs blur correction of a high frequency component, and the second blur correction function performs blur correction of a low frequency component.
(4)
The lens barrel is configured to be removable from the camera body.
The control unit performs blur correction of high-frequency components by the first blur correction function based on the determination of the communication speed with the mounted lens barrel, and is low by the second blur correction function. The image pickup apparatus according to any one of (1) to (3) above, which controls to perform blur correction of frequency components.
(5)
In the lens barrel, the second blur correction value at each sample timing is stored in the storage unit together with the time information, and at a predetermined time point, the second blur correction value stored in the storage unit and the time information are controlled. Perform the process of sending to the department
Described in (1) to (3) above, the control unit performs a process of converting the second blur correction value received from the lens barrel into metadata corresponding to a frame of a moving image based on the time information. Imaging device.
(6)
The control unit uses the correction information of the second image stabilization value received from the lens barrel and the second image stabilization value generated by interpolation processing using the received second image stabilization value. The image pickup apparatus according to any one of (1) to (5) above.
(7)
The control unit
The first blur correction function controls the blur correction of the high frequency component, and the second blur correction function controls the blur correction of the low frequency component.
The image pickup apparatus according to any one of (1) to (6) above, wherein the frequency range of the low frequency component is set based on the communication speed with the lens barrel side.
(8)
The control unit
The first blur correction function controls the blur correction of the high frequency component, and the second blur correction function controls the blur correction of the low frequency component.
The image pickup apparatus according to (5) above, wherein the frequency range of a low frequency component is set based on the storage capacity of the storage unit.
(9)
The control unit
The image pickup apparatus according to any one of (1) to (8) above, which performs a process of recording the communication speed information between the lens barrel and the camera body as metadata on a recording medium.
(10)
The image stabilization function that displaces the image stabilization lens and
Memory and
A process of detecting a blur correction value related to the blur correction function at each sample timing, a process of storing the detected blur correction value in the storage unit together with time information, and a process of storing the detected blur in the storage unit at a predetermined time point. A lens barrel device including a control unit that performs a process of transmitting a correction value and the time information to a camera body unit to which the lens is mounted.
(11)
The lens barrel device according to (10) above, wherein the control unit performs a process of transmitting the blur correction value and time information stored in the storage unit at intermittent timings during the moving image imaging period.
(12)
The lens barrel device according to (10) or (11) above, wherein the control unit performs a process of transmitting the blur correction value and time information stored in the storage unit to a time point after the moving image imaging is completed.
(13)
The control unit
For the image stabilization value detected at each sample timing, select whether to send it to the camera body or store it in the storage unit.
The blur correction value stored in the storage unit is stored in the storage unit together with the time information, and at a predetermined time point, it is read from the storage unit and transmitted to the camera main body unit together with the time information. The lens barrel device according to any one of 10) to (12).
(14)
A first image stabilization value related to a first image stabilization function that corrects the positional relationship between an optical image incident through a lens and an output image image, and a second image stabilizer provided on a lens barrel provided with the lens. An imaging method in which an image pickup apparatus performs a process of generating correction information including a second image stabilization value related to an image stabilization function as metadata associated with an image image stabilization.
(15)
The image stabilization function that displaces the image stabilization lens and
Memory and
As a transmission method in a lens barrel device equipped with
The process of detecting the image stabilization value related to the image stabilization function at each sample timing,
A process of storing the detected image stabilization value in the storage unit together with time information,
A process of transmitting the image stabilization value and the time information stored in the storage unit to the camera body unit to which the camera is mounted at a predetermined time point.
How to send.

1 Image pickup device 2 Camera body 3

Lens lens barrel

5, 6 Image processing device 10 Lens system

10a Zoom lens

10b Aperture mechanism 10c Image stabilization lens mechanism 10d Focus lens 11 Shutter 12 Image pickup element unit 13 Camera signal processing unit 14 Recording control unit 15 Display unit 16 Output unit 17 Operation unit 18 Camera control unit 19 Memory unit 20 Lens control unit 21 Zoom drive unit 22 Aperture drive unit 23 Correction lens drive unit 24 Focus drive unit 25 Blur detection unit 26 Memory unit 27 Communication control unit 30 Imaging surface Camera shake correction unit 31 Shutter drive unit 32 Correction unit drive unit 33 Communication control unit 34 Shake detection unit 35 Electronic camera shake correction control unit 36 Shake correction metadata processing unit 40 Control unit

Claims

A first image stabilization value related to a first image stabilization function that corrects the positional relationship between an optical image incident through a lens and an output image image, and a second image stabilizer provided on a lens barrel provided with the lens. An image pickup device provided with a control unit that generates correction information including a second blur correction value related to the blur correction function as metadata associated with the captured image.
The image pickup apparatus according to claim 1, wherein the control unit uses the second blur correction value received from the lens barrel as the correction information.
The image pickup apparatus according to claim 1, wherein the first blur correction function performs blur correction of a high frequency component, and the second blur correction function performs blur correction of a low frequency component.
The lens barrel has a structure that allows it to be attached to and detached from the camera body.
The control unit performs blur correction of high-frequency components by the first blur correction function based on the determination of the communication speed with the mounted lens barrel, and is low by the second blur correction function. The image pickup apparatus according to claim 1, wherein the image stabilization of a frequency component is controlled so as to be performed.
In the lens barrel, the second blur correction value at each sample timing is stored in the storage unit together with the time information, and at a predetermined time point, the second blur correction value stored in the storage unit and the time information are controlled. Perform the process of sending to the department
The image pickup apparatus according to claim 1, wherein the control unit performs a process of converting the second blur correction value received from the lens barrel into metadata corresponding to a frame of a moving image based on the time information.
The control unit uses the correction information of the second image stabilization value received from the lens barrel and the second image stabilization value generated by interpolation processing using the received second image stabilization value. The imaging device according to claim 1.
The control unit
The first blur correction function controls the blur correction of the high frequency component, and the second blur correction function controls the blur correction of the low frequency component.
The image pickup apparatus according to claim 1, wherein the frequency range of a low frequency component is set based on the communication speed with the lens barrel side.
The control unit
The first blur correction function controls the blur correction of the high frequency component, and the second blur correction function controls the blur correction of the low frequency component.
The image pickup apparatus according to claim 5, wherein the frequency range of the low frequency component is set based on the storage capacity of the storage unit.
The control unit
The image pickup apparatus according to claim 1, wherein a process of recording the communication speed information between the lens barrel and the camera body as metadata on a recording medium.
The image stabilization function that displaces the image stabilization lens and
Memory and
A process of detecting a blur correction value related to the blur correction function at each sample timing, a process of storing the detected blur correction value in the storage unit together with time information, and a process of storing the detected blur in the storage unit at a predetermined time point. A lens barrel device including a control unit that performs a process of transmitting a correction value and the time information to a camera body unit to which the lens is mounted.
The lens barrel device according to claim 10, wherein the control unit performs a process of transmitting the blur correction value and time information stored in the storage unit at intermittent timings during a moving image imaging period.
The lens barrel device according to claim 10, wherein the control unit performs a process of transmitting the blur correction value and time information stored in the storage unit at a time point after the end of moving image imaging.
The control unit
For the image stabilization value detected at each sample timing, select whether to send it to the camera body or store it in the storage unit.
A claim that the blur correction value stored in the storage unit is stored in the storage unit together with time information, and at a predetermined time point, is read from the storage unit and transmitted to the camera main body unit together with the time information. 10. The lens barrel device according to 10.
A first image stabilization value related to a first image stabilization function that corrects the positional relationship between an optical image incident through a lens and an output image image, and a second image stabilizer provided on a lens barrel provided with the lens. An imaging method in which an image pickup apparatus performs a process of generating correction information including a second image stabilization value related to an image stabilization function as metadata associated with an image image stabilization.
The image stabilization function that displaces the image stabilization lens and
Memory and
As a transmission method in a lens barrel device equipped with
The process of detecting the image stabilization value related to the image stabilization function at each sample timing,
A process of storing the detected image stabilization value in the storage unit together with time information,
A process of transmitting the image stabilization value and the time information stored in the storage unit to the camera body unit to which the camera is mounted at a predetermined time point.
How to send.