JP5359797B2 - Interchangeable lens type camera system integrated with an image sensor and moving image generation method thereof - Google Patents

Description

  The present invention is a camera system that includes a lens unit integrated with an image sensor and a main unit that can be interchangeably connected, and generates and records a moving image based on video signals and audio data exchanged between the units. The present invention relates to a system and a moving image generation method thereof.

  In recent years, the use of digital cameras as image capturing devices has become widespread. Many digital cameras have been commercialized, from professional single-lens reflex cameras to compact cameras that take ordinary snapshots. Furthermore, some mobile phones are equipped with the functions of a digital camera, and digital cameras have become common. As the number of users who use this digital camera increases, the number of users who are particular about photographs is expanding. Under such circumstances, interchangeable lens cameras are also gaining popularity.

  There are two types of this interchangeable lens camera. The camera system (normal single-lens reflex type) in which only the lens unit can be replaced with a built-in image sensor on the main unit, the lens and the image sensor are integrated. There is a camera system that can replace the lens unit.

  An example of a camera system capable of exchanging a lens unit having the latter imaging element is disclosed in Patent Document 1.

  In a normal single-lens reflex camera system, the functions of a speaker unit that generates a shutter sound and a microphone unit that enables voice input are installed on the main unit side because they do not depend on the lens unit. This is because the microphone unit is placed away from the lens unit to prevent the zoom and focus driving sounds from being mixed into the audio input.

  In the normal single-lens reflex type camera system, since the main unit has a sensor (imaging device) that acquires a video signal, it is possible to acquire audio and video in the same housing.

  Also in a camera system having an image sensor on the lens unit side, it is desirable to place the microphone unit on the main unit side away from the lens unit in order to prevent zoom and focus drive sounds from being mixed into the audio input.

  However, since there is a sensor (imaging device) for acquiring a video signal on the lens unit side, when trying to generate a moving image consisting of a video signal and an audio signal, the video signal (image data) is generated on the lens unit side, Since the audio signal (audio data) is generated on the main unit side, synchronization between the video data and the audio data becomes a problem.

  The present invention solves the above-described problems of the prior art, and is a combination of a difference in processing speed, a difference in frame rate, and the like between a lens unit having an image sensor and a main body unit to which the lens unit is interchangeably connected. It is an object of the present invention to provide an imaging lens-integrated interchangeable lens type camera system that generates and records a moving image in which a video signal (image data) and audio data are synchronized, and a moving image generation method thereof .

  In order to achieve the above object, a camera system according to claim 1 of the present invention forms an image on an image sensor through an optical system including a photographing lens, and converts the image data into electronic data by photoelectric conversion. A camera system comprising a lens unit in which an image pickup device for outputting a converted video signal is integrated, and a body unit in which the lens unit can be interchangeably connected and image processing is performed on output data from the lens unit and recorded on a memory card. The lens unit includes synchronization signal output means for outputting a synchronization signal synchronized with the video signal of the image pickup device, and the main body unit receives the synchronization signal output from the audio input means for acquiring the audio data and the synchronization signal output means. In the case of recording a video signal in the main unit, the synchronization signal input means is provided. It is synchronized with the synchronizing signal input to and obtains audio data.

  With this configuration, a moving image in which a video signal (image data) and audio data are synchronized can be generated and recorded.

  Further, in the camera system according to claim 1, the video signal is output from the lens unit to the main unit in synchronization with the synchronization signal, and the video signal is the synchronization signal. Output from the lens unit to the main unit asynchronously, and the period of the video signal output from the lens unit is adjusted from the video signal acquired by the imaging unit of the lens unit according to the processing capability or setting of the main unit. Is characterized by having a long or short period.

  With this configuration, the video signal (image data) and the audio data can be reliably synchronized in units of frames, and the video signal (image data) and the audio data can be synchronized without being restricted by the synchronization signal from the lens unit. It can be synchronized and a video can be generated and recorded.

  According to a fifth aspect of the present invention, there is provided a camera system that forms an image on an image sensor through an optical system including a photographing lens and outputs an image signal converted into image data converted into electronic data by photoelectric conversion. In a camera system comprising a body unit and a body unit that can be interchangeably connected to the lens unit and that processes output data from the lens unit and records the data in a memory card, the body unit acquires audio data. When a video signal is recorded in the main unit with audio input means, the frame rate of the video signal output from the lens unit and the frame rate of the video signal recorded in the main unit are independent. It can be set.

  With this configuration, it is possible to record a moving image of a video signal as the frame rate of the main unit without being limited by the frame rate of the image sensor.

  According to the sixth and seventh aspects of the present invention, in the camera system of the fifth aspect, when the video signal is recorded in the main unit, the video signal output from the lens unit and the audio data acquired by the audio input means The synchronization signal for synchronizing the video signal is generated by the timer means included in the main unit, or when the video signal is recorded in the main unit, the video signal output from the lens unit and the audio data acquired by the audio input unit are used. The synchronization signal to be synchronized is generated at a transfer timing for outputting a video signal from the lens unit.

  With this configuration, the video signal can be recorded as a frame rate of the main unit without being restricted by the frame rate of the image sensor, which can be generated by the timer means and the transfer timing.

  According to another aspect of the present invention, there is provided a moving image generation method for a camera system, which forms an image on an image pickup device through an optical system including a photographing lens and outputs a video signal converted into image data converted into electronic data by photoelectric conversion. A method for generating a moving image of a camera system comprising a lens unit integrated with an image pickup device and a main body unit in which the lens unit can be interchangeably connected and image processing is performed on output data from the lens unit and recorded on a memory card. A synchronization signal output step of outputting a synchronization signal synchronized with the video signal of the image sensor, an audio input step of acquiring audio data by the audio input means of the main unit, and a synchronization signal output in the synchronization signal output step to the main unit A sync signal input process, and when recording video signals in the main unit, In synchronization with the synchronization signal input by, and acquires voice data by the voice input step.

  By this method, a moving image in which a video signal (image data) and audio data are synchronized can be generated and recorded.

  The invention described in claims 9 to 11 is the moving image generation method according to claim 8, wherein the video signal is output from the lens unit to the main unit in synchronization with the synchronization signal, and the video signal is Asynchronously with the synchronization signal, the lens unit outputs the signal to the main unit, and the period of the video signal output from the lens unit is acquired by the imaging unit of the lens unit in accordance with the processing capability or setting of the main unit. The period is longer or shorter than the video signal.

  By this method, the video signal (image data) and the audio data can be reliably synchronized on a frame basis, and the video signal (image data) and the audio data can be synchronized without being restricted by the synchronization signal from the lens unit. It can be synchronized and a video can be generated and recorded.

  A moving image generating method according to a twelfth aspect of the invention includes an image pickup device that outputs an image signal that is imaged on an image pickup device through an optical system including a photographing lens and converted into image data converted into electronic data by photoelectric conversion. A method for generating a moving image of a camera system comprising an integrated lens unit and a main body unit in which the lens unit can be interchangeably connected and image processing is performed on output data from the lens unit and recorded on a memory card. A voice input step of acquiring audio data by the voice input means, and when the video signal is recorded in the main unit, the video signal frame rate output from the lens unit and the video recording is performed in the main unit. The frame rate of the video signal can be set independently.

  By this method, it is possible to record a moving image of a video signal as the frame rate of the main unit without being restricted by the frame rate of the image sensor.

  Further, the invention described in claims 13 and 14 is the moving image generation method according to claim 12, wherein when the moving image recording of the video signal is performed by the main unit, the video signal output from the lens unit and the audio input means are obtained. The synchronization signal for synchronizing the audio data is generated by the timer means included in the main unit, or when the video signal is recorded in the main unit, the video signal output from the lens unit and acquired by the audio input unit. The synchronization signal for synchronizing the audio data is generated at a transfer timing for outputting a video signal from the lens unit.

  According to this method, the video signal can be recorded as a frame rate of the main unit without being limited by the frame rate of the image sensor, which can be generated by the timer means or the transfer timing.

  According to the present invention, in a camera system including a lens unit integrated with an image sensor and a main body unit to which the image sensor can be interchangeably connected, restrictions on combinations due to differences in processing speed, frame rates, and the like between the units. The moving image based on the video signal (image data) and the audio data can be generated and recorded without being received.

FIG. 1 is an external view showing examples (a) and (b) of a camera system in an embodiment of the present invention. Functional block diagram showing a main unit and a lens unit constituting the camera system in the present embodiment Functional block diagram showing another example of a lens unit constituting the camera system in the present embodiment Functional block diagram showing still another example of the lens unit constituting the camera unit in the present embodiment The figure explaining the flow (a) of the image data at the time of video recording in this embodiment, and the flow (b) of another image data The figure which shows the image data control of the camera system of Example 1 in this embodiment. The figure which shows the image data control of the camera system of Example 2 in this embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIGS. 1A and 1B are external views illustrating an example of a camera system (imaging device) according to an embodiment of the present invention. 1A includes a main unit 1 and a lens unit 2 (lens unit 3) that can be attached to and detached from the main unit 1. FIG. In the camera system, the lens unit 2 and the main unit 1 are integrated to exhibit a function as an imaging device. The lens unit 2 is appropriately selected from various types by the user and attached to the main unit 1. For example, as the lens unit 2 (2 ′), the lens unit 3 of a single focus lens or the lens unit 2 (FIG. 1). There is a lens having a lens unit 3 ′ equipped with the optical zoom of (b).

  Next, the functional configuration of the camera system in the present embodiment will be described with reference to FIGS. FIG. 2 is a diagram showing functional blocks of the main body unit and the lens unit constituting the camera system in the present embodiment.

  2 includes a lens group 107 having a focus lens, an image sensor 108 that converts a subject image received through the lens group 107 from an optical signal to an electrical signal, and outputs the image. An AFE (analog front end) 109 that converts the signal (analog image data) output from the element 108 into digital image data and amplifies the signal, and converts the converted digital image data into YUV data A CPU 103 with a microcomputer built in an image engine capable of performing predetermined image processing such as processing, JPEG compression processing, and RAW data generation processing.

  The lens unit 2 also has a joint connector 116 that constitutes the inter-unit interface 5 that is electrically connected to the main unit 1, and a bidirectional bus 123 for transferring image data to the main unit 1 via the joint connector 116. , A control signal 121, a serial interface signal 122, and a vertical synchronization signal 124, respectively.

  The lens unit 2 also has a motor driver 111 that controls a motor 110 used for extending and storing the lens barrel of the lens group 107. The motor driver 111 is controlled by a control signal 121 received from the main unit 1. With this mechanism, depending on the type of the interchangeable lens, various operation controls such as storing the lens barrel when the power of the camera system is turned off and performing a zooming operation by pressing a button (not shown) can be performed.

  The lens unit 2 detects the DC-DC converter 101 that generates various types of power necessary for the operation of the lens unit 2 from the power 120 supplied from the main unit 1 and the power 120 supplied from the main unit 1. A sub CPU 102 that controls the DC-DC converter and a detection circuit 113 that detects a teleconverter lens and a wide converter lens that can be mounted outside the lens unit 2 are provided.

  The lens unit 2 includes a gyro sensor 106 that detects the tilt of the camera system (imaging device), an acceleration sensor 112 that detects acceleration applied to the imaging device, and the tilt and acceleration sensor 112 detected by the gyro sensor 106. A coil 105 that drives the lens group 107 by acceleration and a Hall element 104 that detects the driving amount of the coil 105 are provided.

  The gyro sensor 106, the acceleration sensor 112, the coil 105, and the hall element 104 can exhibit a camera shake prevention function.

  In the lens unit 2, software for performing image processing and operation control processing is stored in a flash ROM 114, and the CPU 103 uses the RAM 115 as a work area to perform processing by this software. And processing control.

  2 includes a joint connector 201 that constitutes an inter-unit interface 5 that is electrically connected to the lens unit 2, and image data received from the lens unit 2 via the joint connector 201. A bi-directional bus 223 for transferring the image data to the CPU 208, which is a so-called image engine, and image data received via the bi-directional bus 223, conversion processing to YUV data, compression processing in JPEG format, and decompression processing from JPEG format The CPU 208, which is a so-called image engine that appropriately performs RAW data generation processing, the control signal 221 line connected to the control signal 121 line of the lens unit 2, and the serial interface signal 122 line of the lens unit 2 are connected. Serial interface A line of scan signals 222, and a line of the vertical synchronization signal 224 to be connected to the lines in the vertical synchronizing signal of the lens unit 2.

  The main unit 1 also includes a focus & release switch 211 that is a switch for starting a shooting operation of the camera system by a predetermined pressing operation and the like, and a cross used for selection setting such as an operation mode of the camera system set in the main unit 1. A switch 206 composed of a key and the like, and an input of the switch 206 is detected to perform a predetermined setting process and the power supplied from the lithium ion battery 204 is controlled using the DC-DC converter 203. In addition, a sub CPU 205 that controls a power switch 202 that is a switch for supplying power to the lens unit 2 is also provided.

  The main unit 1 also includes a flash ROM 219 in which software for performing image processing and operation control processing is stored, and the CPU 208 uses the RAM 220 as a work area to perform processing by this software. It is comprised so that control may be performed.

  The main unit 1 also includes an audio codec 216, a microphone 218 that inputs an audio signal to the audio codec 216, a speaker 217 that outputs sound from the audio codec 216, a USB interface connector 214, an AV output connector 213, HDMI signal output interface 212, SD card 215 as a removable storage means for storing captured image data, strobe 207 also serving as a connection circuit for attaching an external strobe to main unit 1, and a switch An LCD 210 and an EVF (Electronic View Finder) 209 are provided as display means for monitoring and displaying a subject image during a focusing operation by an operation 211 and displaying captured image data when performing a photographing operation.

  FIG. 3 is a functional block diagram showing another example of the lens unit constituting the camera system in the present embodiment. In FIG. 3, the lens unit 2 'has substantially the same configuration as the lens unit 2 shown in FIG. A different configuration is that the lens group 307 includes a zoom lens and has a zoom motor 310 for moving the zoom lens. The lens unit 2 ′ is configured to cause the focus lens and the zoom lens provided in the lens group 307 to perform predetermined operations by operating a zoom switch provided in the main body unit 1 (not shown).

  FIG. 4 is a functional block diagram showing still another example of the lens unit constituting the camera unit in the present embodiment. 4, the lens unit 4 has substantially the same configuration as the lens unit 2 shown in FIG. 2, and a different configuration includes a large image sensor 408, so that a hall element 104 and a coil 105 that perform a camera shake prevention operation are provided. The configuration corresponding to the gyro sensor 106 is omitted.

  According to the camera system having the above-described configuration, the bus between the bidirectional bus 123 of the lens unit 2 and the bidirectional bus 223 of the main unit 1 is set according to the setting of the operation mode and the setting of the image recording size and the image recording format. The width setting can be changed. Thus, the bus width can be changed at any time according to the amount of data transmitted and received. Further, according to the camera system of the present embodiment, a plurality of image processes for image data (video signal) according to the amount of data transmitted / received between the lens unit 2 and the main unit 1 that change according to the above-described shooting mode setting. Can be distributed, and the range of image processing shared between the lens unit 2 and the main unit 1 can be changed. Can be exchanged. The data communication method performed between the lens unit 2 and the main unit 1 may be appropriately selected from various transfer methods such as DMA transfer and packet transfer.

  In addition, the camera system according to the present embodiment can perform the above-described characteristic processing using any of the lens units 2, 2 ′, and 4. In addition, the moving image generation method according to the present invention can be performed by any camera system having any of the lens units. Therefore, in the following description, an example using the lens unit 2 will be described unless otherwise specified. The operation of the camera system is controlled by software stored in the flash ROM 114 of the lens unit 2 and the flash ROM 219 of the main unit 1.

  Here, the flow of image data during moving image shooting will be described with reference to FIGS. 2 and 5A. First, an electrical signal corresponding to the subject image captured by the image sensor 108 is output via the lens group 107 provided in the lens unit 2 (S1), and the electrical signal is converted into digital data (S2), and RAW data is obtained. Is generated. Next, YUV conversion processing is performed on the RAW data (S3), and the YUV data obtained thereby is compressed and converted in JPEG format (S4), and sequentially transferred to the main unit 1 (S5). The flow up to the transfer process is continuously performed while the photographing operation in the main unit 1 is continued.

  When the main unit 1 sequentially receives JPEG data from the lens unit 2 via the inter-unit interface 5 (joint connectors 116 and 201) (S6), it adds header information such as predetermined metadata as a single image file. Then, an image file in JPEG format is generated. The generated JPEG image file is sent to the storage means and stored in a predetermined recording medium (SD card 215). Further, the received JPEG data is expanded into YUV data (S7), and is output to display means such as the LCD 210.

  Further, another flow of image data during moving image shooting will be described with reference to FIGS. 2 and 5B. First, an electrical signal corresponding to a subject image captured by the image sensor 108 is output from the image sensor 108 via the lens group 107 provided in the lens unit 2 (S1), and the electrical signal is converted into digital data (S2). ), RAW data is generated. Next, YUV conversion processing is performed on the RAW data (S3), and the YUV data obtained thereby is transferred to the main unit 1 (S5).

  When the main body unit 1 sequentially receives YUV data from the lens unit 2 via the inter-unit interface 5 (joint connectors 116 and 201) (S8), the main body unit 1 performs a process of outputting to the display means such as the LCD 210. Further, the image data is compressed and converted in the JPEG format (S9), and header information such as predetermined metadata is added to generate an image file in the JPEG format. The generated JPEG image file is sent to the storage means and stored in a predetermined recording medium (SD card 215).

  In the former flow of image data, since the format of image data transmitted / received via the inter-unit interface 5 is JPEG, the amount of communication data is smaller than when RAW data or YUV data is transmitted / received. However, JPEG compression and JPEG expansion processing take time, and this time is a delay until output to the display means. Therefore, when the throughput of the inter-unit interface 5 is sufficiently large, communication is performed using RAW data or YUV data. The main unit 1 may take a method of displaying and recording the received data after performing necessary subsequent processing.

  In the other flow of the latter image data, the lens unit 2 performs only A / D conversion processing and YUV conversion processing, and transfers YUV data to the main unit 1 without compressing the data into JPEG data. As a result, the main unit 1 can output the received YUV data to the LCD 210 without performing JPEG expansion processing. Therefore, since the JPEG compression process and the JPEG expansion process are not performed before and after the communication process, display delay due to these processes does not occur. However, since the image data is transferred without compression, it is vulnerable to noise, and it is necessary to secure a wide communication band of the inter-unit interface 5.

  Here, the relationship between the communication band in the inter-unit interface 5 and the above-described two image data flows will be described in detail. When the size of the image data transferred from the lens unit 2 to the main unit 1 is the VGA size (640 pixels × 480 pixels) uncompressed YUV422 format, and image data equivalent to 30 images is sent per second The amount of data transferred from the lens unit 2 to the main unit 1 per second is about 18 megabytes (640 × 480 × 2 × 30 = 18,432,000).

  If the communication band of the inter-unit interface 5 is 18.432 (megabytes / second) or more, even if image data in the uncompressed YUV format is transferred, no delay occurs. If the communication band is 18.432 (megabytes / second), image data equivalent to exactly 30 images is sent per second, so the transfer time per image is 1/30 (second) (approximately 33 milliseconds).

  Since the delay time at this time is equal to the time for transferring the image data from the lens unit 2 to the main unit 1, a delay of 1/30 (second) occurs.

  On the other hand, when JPEG data is transferred from the lens unit 2 to the main unit 1, if the compression rate by JPEG compression processing is 1/10, the transfer time is also 1/10, which is 1/300 (seconds). Become.

  Here, if it is assumed that JPEG compression processing and JPEG decompression processing each require 1/30 (second), the transfer time and the sum of these times are display delays. That is, the display delay is about 69.3 milliseconds (1/300 + 1/30 + 1/30 (seconds)). However, if JPEG compression processing and JPEG decompression processing are 1/300 (seconds), the transfer time is about 10 milliseconds (1/300 + 1/300 + 1/300 (seconds)), which is more than the data transfer in the YUV format. The delay time is shortened.

  In general, if the communication bandwidth is the same and the total time required for compression processing and decompression processing is smaller than the difference in communication time due to the difference in transfer amount, the image data is compressed and transferred to the main unit 1 is displayed. The delay is small. Therefore, the format of the image data to be transferred is determined based on the communication bandwidth of the inter-unit interface 5 and the total time required for JPEG compression processing and JPEG decompression processing so that the display delay is reduced. The processing content performed by the main unit 1 may be changed.

  Here, as Example 1 in the present embodiment, audio data that is simultaneously acquired and recorded when moving image recording of a video signal will be described.

  In the above-described flow of image data, audio data is handled by the main unit 1 shown in FIG. For this reason, it is necessary to synchronize the video signal (image data) and the audio data between the lens unit 2 and the main unit 1, and an interruption by the vertical synchronization signal (124, 224) generated when the lens unit 2 completes light reception The unit 1 is propagated through the inter-unit interface 5 and synchronized. In the camera system of the first embodiment, the lens unit 2 generates a vertical synchronization signal (VD) and outputs the vertical synchronization signal 124 from the lens unit 2 to the main unit 1.

  Data control at this time will be described with reference to FIG. Image data is sequentially generated on the lens unit 2 side (S11). One of the triangles shown in FIG. 6 represents one frame of image data. The data format may be the JPEG format shown in FIG. 5 or the YUV format. At this time, the vertical synchronizing signal is propagated to the main unit 1 for each frame. On the main unit 1 side, audio data is generated according to the vertical synchronization signal 224 (S13). At the same time, the image data generated on the lens unit 2 side is transferred to the main unit 1 through the inter-unit interface 5 (S12). The main unit 1 generates moving image data by matching the received image data and the generated audio data with the synchronization information (S14). The image data for each frame received by the main unit 1 is generated in accordance with the period of the vertical synchronization signal 224, and the audio data generated by the main unit 1 is also generated according to the vertical synchronization signals 124 and 224 received from the lens unit 2. Therefore, it is possible to generate moving image data by matching image data and audio data with synchronization information.

  At this time, the transfer timing of the video signal may be synchronized with the vertical synchronization signal or may be asynchronous with the vertical synchronization signal. When the transfer timing is synchronized with the vertical synchronization signal, the synchronization amount between the image data and the audio data is adjusted for the first frame, which is the moving image data on the main unit 1 side, that is, the lens unit 2. If the synthesis is performed so as to eliminate the difference between the timing at which the image data is received from the recording data and the timing at which the recording of the audio data is started, then the synthesis may be performed in accordance with the vertical synchronization signal. However, it is necessary to ensure that each frame process is performed without delay. Here, each frame process is a process performed on the lens unit 2 side until the captured image data is converted into a format that is transferred to the main unit 1 such as a JPEG format or a YUV format.

  Also, if the frame data is transferred at an arbitrary timing, for example, the image data is transferred to the main unit 1 after the completion of each frame processing as asynchronous to the vertical synchronization signal, each frame data is transferred to the main unit 1 side. Although it is necessary to perform timing adjustment with the synchronization signal, the timing for each frame process is not critical.

  If the lens unit 2 and the main unit 1 exchange their capacities when the camera system is started up, and the synchronization frequency of the sensor (imaging device) is notified from the lens unit 2 to the main unit 1 side, the main unit 1 transmits the vertical synchronization signal. By counting the number, for example, the timing per second can be obtained. Therefore, the main unit 1 can synthesize the audio data for a predetermined time and the image data for the predetermined time by counting the number of vertical synchronization signals.

  When a memory with a slow writing speed is attached to the main unit 1, a notification is sent from the main unit 1 to the lens unit 2 at the time of capability exchange in advance, and even if the sensor (imaging device) operates at 30 fps, the cycle is doubled. By sending an image at 15 fps, a moving image can be realized even with a memory card with slow writing.

  As a method of changing the transfer cycle to a cycle longer than the imaging cycle of the lens unit 2, in the image processing in the lens unit 2, simple frame thinning and addition of frames are performed (from 1 frame image data to 2 frame image data). Generated).

  Other reasons for delaying the image data transfer cycle include when the power saving mode is set, when the processing capability of the main unit 1 is slow, and when high-sensitivity shooting with low brightness is required. .

  In the power saving mode, this can be realized by simply reducing the number of frames to be processed by thinning out data.

  Switching is necessary even in a combination of the main body unit 1 having a slow operation and the lens unit 2 on which a high-speed sensor (imaging device) is mounted. If the sensor (imaging device) can output data at 60 fps and the main unit 1 can process up to 30 fps, the lens unit 2 transfers data at 30 fps. Recording can be enabled.

  High sensitivity can be realized by adding frames. By changing the transfer cycle to a cycle shorter than the imaging cycle of the lens unit 2, it is possible to realize a moving image frame rate that exceeds the capability of the sensor (imaging device) that the lens unit 2 can originally output. As this method, a method of sending the same image twice or a method of generating a new frame by performing frame interpolation from the correlation with the next frame can be considered.

  By changing the transfer cycle in this way, moving image recording with a moving image frame cycle can be performed with more variations than the restriction on the screen cycle unique to the sensor (imaging device).

  Therefore, in the camera system in which the lens unit 2 having the image sensor as in the first embodiment can be attached to and detached from the main unit 1, the image sensor varies depending on the lens unit 2 to be mounted, and the drive cycle of the image sensor varies depending on the lens unit 2. Even in this case, moving image recording can be performed at an arbitrary frame rate in accordance with the setting and capability on the main unit 1 side.

  Next, in Example 2 of the present embodiment, as in Example 1, the file is configured and recorded as a moving image by the main unit 1 of FIG. 2, and data control at this time is shown in FIG. Will be described with reference to FIG. Image data is sequentially generated on the lens unit 2 side (S21). One triangle in FIG. 7 indicates one frame of image data, and the data format may be JPEG format or YUV format. At this time, image data is generated on the lens unit side based on the frame rate of the image sensor.

  On the main unit 1 side, audio data is generated in accordance with the frame rate of the moving image to be generated (S23). At the same time, the image data generated on the lens unit 2 side is transferred to the main unit 1 through the inter-unit interface 5 (S22). The main unit 1 generates moving image data by matching the received image data and the generated audio data with the synchronization information (S24). Here, the frame rate of the sensor (imaging device) is 24 fps, and the frame rate of the moving image to be generated is 30 fps. Since the frame rate of image data is lower than that of audio data, moving image data of 30 fps is generated by superimposing one frame of the same image data on six images.

  The synchronization between the image data and the audio data at this time may be generated by a timer means on the main unit 1 side, or may be synchronized with the image data transfer timing from the lens unit 2.

  If synchronization is made by this timer means, if the amount of synchronization between the image data and the audio data is adjusted for the first frame, which is the moving image data on the main unit 1 side, the timer means will be used thereafter. You can combine them together. However, it is necessary to ensure that each frame process is performed without delay.

  In addition, the first frame synchronization of the moving image data is controlled in advance from the main unit 1 side to the lens unit 2 side with the control signal 121, and the time until the result is returned is measured. A time lag between the unit 1 and the lens unit 2 may be measured, and the frame synchronization timing at the time of moving image data synthesis may be adjusted.

  Further, in the configuration shown in FIG. 2 of the present embodiment, the focus & release switch 211 is provided only on the main unit 1 side, but this is connected to the control signal 221 and is simultaneously applied to both the main unit 1 and the lens unit 2. A release signal may be transmitted to synchronize with this.

  Furthermore, if the synchronization between the image data and the audio data is adjusted to the image data transfer timing, it is necessary to adjust the timing by the synchronization signal for each frame data on the main unit 1 side. The timing is not critical.

  If the lens unit 2 and the main unit 1 exchange their capacities when the camera system is started up, and the frame rate of the image data of the sensor (imaging device) is notified to the main unit 1 side, the main unit 1 will read the frame of the image data. By counting the number, for example, the timing per second can be obtained.

  When a memory with a slow writing speed is attached to the main unit 1, a memory card with a slow writing speed can be obtained if the image data sent from the lens unit 2 is thinned out appropriately or added between frames. But you can make a video realizable. Thereby, it is possible to compose a moving image having a frame rate slower than the frame rate of the lens unit 2.

  Further, when a moving image having a frame rate faster than the frame rate of the lens unit 2 is formed, if the same image is used a plurality of times or if frame interpolation is performed based on the correlation with the next frame, the lens unit 2 It is also possible to compose a moving image having a frame rate faster than the frame rate.

  In the camera system as in the present invention, not only the lens unit 2 but also a plurality of main body units 1 having different capabilities exist, and any combination of the lens unit 2 and the main body unit 1 is possible, and an interchangeable lens. The moving image can be generated in accordance with the respective capabilities of the unit 2 and the main unit 1 to which the lens unit 2 is attached.

  An imaging element-integrated interchangeable lens type camera system and a moving image generation method thereof according to the present invention are a camera system including an imaging element-integrated lens unit and a body unit that can be interchangeably connected. You can generate and record a video that synchronizes the video signal (image data) and audio data without being restricted by combinations due to differences in processing speed, frame rate, etc. It is useful as an apparatus and method for obtaining a moving image in which a video signal and audio data are synchronized.

1 Main unit 2, 2 ', 4 Lens unit 3, 3' Lens unit 5 Unit interface 124, 224 Vertical synchronization signal

Japanese Patent No. 3945052

Claims (14)

A lens unit integrated with the image pickup device that forms an image on an image pickup device via an optical system including a photographing lens and outputs image data converted into electronic data by photoelectric conversion; and the lens unit includes: In a camera system comprising a main unit that can be interchangeably connected and that records output data from the lens unit on a memory card after image processing,
The lens unit includes a synchronization signal output means for outputting a synchronization signal synchronized with a video signal of the image sensor;
The main unit includes audio input means for acquiring audio data, and synchronization signal input means for inputting a synchronization signal output from the synchronization signal output means,
2. A camera system according to claim 1, wherein when recording a video signal in the main unit, the audio data is acquired in synchronization with a synchronization signal input via the synchronization signal input means.   The camera system according to claim 1, wherein the video signal is output from the lens unit to the main body unit in synchronization with the synchronization signal.   The camera system according to claim 1, wherein the video signal is output from the lens unit to the main body unit asynchronously with the synchronization signal.   The period of the video signal output from the lens unit is longer or shorter than the video signal acquired by the imaging device of the lens unit according to the processing capability or setting of the main unit. Item 4. The camera system according to Item 3. A lens unit integrated with the image pickup device that forms an image on an image pickup device via an optical system including a photographing lens and outputs image data converted into electronic data by photoelectric conversion; and the lens unit includes: In a camera system comprising a main unit that can be interchangeably connected and that records output data from the lens unit on a memory card after image processing,
The main unit includes voice input means for acquiring voice data,
When the video signal is recorded in the main unit, the frame rate of the video signal output from the lens unit and the frame rate of the video signal to be recorded in the main unit can be set independently. A camera system characterized by being.   When the video signal is recorded in the main unit, a synchronization signal for synchronizing the video signal output from the lens unit and the audio data acquired by the audio input unit is generated by a timer unit included in the main unit. The camera system according to claim 5.   When performing video recording of a video signal in the main unit, the synchronization signal for synchronizing the video signal output from the lens unit and the audio data acquired by the audio input means is a transfer timing for outputting the video signal from the lens unit. The camera system according to claim 5, wherein the camera system is generated by: A lens unit integrated with the image pickup device that forms an image on an image pickup device via an optical system including a photographing lens and outputs image data converted into electronic data by photoelectric conversion; and the lens unit includes: A method for generating a moving image of a camera system comprising a main unit that is connectable in a replaceable manner and that performs image processing on output data from the lens unit and records the data in a memory card,
A synchronization signal output step of outputting a synchronization signal synchronized with a video signal of the image sensor; an audio input step of acquiring audio data by an audio input means of the main unit; and the synchronization signal output in the synchronization signal output step A synchronization signal input step for inputting to the main unit,
A moving image of a camera system, wherein when recording a moving image of a video signal in the main body unit, audio data is acquired by the audio input step in synchronization with the synchronizing signal input by the synchronizing signal input step. Generation method.   9. The moving image generation method for a camera system according to claim 8, wherein the video signal is output from the lens unit to the main body unit in synchronization with the synchronization signal.   9. The moving image generation method for a camera system according to claim 8, wherein the video signal is output from the lens unit to the main body unit asynchronously with the synchronization signal.   The period of the video signal output from the lens unit is longer or shorter than the video signal acquired by the imaging device of the lens unit according to the processing capability or setting of the main unit. Item 11. A moving image generation method for a camera system according to Item 10. A lens unit integrated with the image pickup device that forms an image on an image pickup device via an optical system including a photographing lens and outputs image data converted into electronic data by photoelectric conversion; and the lens unit includes: A method for generating a moving image of a camera system comprising a main unit that is connectable in a replaceable manner and that performs image processing on output data from the lens unit and records the data in a memory card,
A voice input step of acquiring voice data by voice input means of the main unit;
When the video signal is recorded in the main unit, the frame rate of the video signal output from the lens unit and the frame rate of the video signal to be recorded in the main unit can be set independently. A method of generating a moving image of a camera system, characterized in that:   When the video signal is recorded in the main unit, a synchronization signal for synchronizing the video signal output from the lens unit and the audio data acquired by the audio input unit is generated by a timer unit included in the main unit. The moving image generation method of the camera system according to claim 12.   When performing video recording of a video signal in the main unit, the synchronization signal for synchronizing the video signal output from the lens unit and the audio data acquired by the audio input means is a transfer timing for outputting the video signal from the lens unit. 13. The moving image generating method for a camera system according to claim 12, wherein the moving image generating method is performed by the following.

Images