JP7374698B2 - Imaging device, control method and program - Google Patents

Description

The present invention relates to an imaging device, a control method, and a program.

Video cameras in recent years have become capable of capturing images with high resolution and high frame rates. For example, shooting using a video signal called HD (High Definition) such as 4K (1 frame is 4096 x 2160) or 8K (1 frame is 8192 x 4320) where the number of pixels in one frame is 1920 x 1080 pixels or more. It can be performed. Furthermore, the frame rate has shifted to higher frame rates such as 120p, 240p, and 960p, which are higher than 60p. Further, standardization that supports the above-mentioned high resolution and high frame rate photography is also progressing.

Conventionally, time codes could only be expressed up to 60p, but according to the SMPTE ST 12-3 standard, it is now possible to provide time codes of 60p or more. Furthermore, the HDMI (registered trademark) 2.1 standard also enables high refresh rates and variable refresh rate functions for external output terminals. As related technologies, imaging devices disclosed in Patent Document 1 and Patent Document 2 have been proposed. In the imaging device of Patent Document 1, even when an external device records and reproduces a moving image in which the same time code is added to multiple frames, a unique time code can be used for each frame. Further, the imaging device of Patent Document 2 matches the phase of the video synchronization signal and time code input from an external device with the phase of the system clock and time code inside the imaging device.

Japanese Patent Application Publication No. 2015-192227 Patent No. 5854832

When synchronized shooting is performed using a plurality of video cameras, the phases of each frame are aligned using time codes assigned to frames shot by each video camera. Here, one video camera among the plurality of video cameras may shoot at a different frame rate from the other video cameras. In this case, correct time codes may not be assigned to consecutive frames.

An object of the present invention is to assign correct time codes to frames during synchronized shooting.

In order to achieve the above object, the imaging device of the present invention records a first time code that includes subframe information that is an extension of frame information, or records a second time code that does not include the subframe information. a setting means capable of setting whether to record the second time code; and a change means for changing the time code of the time code to be recorded.

According to the present invention, a correct time code can be assigned to a frame during synchronous shooting.

FIG. 1 is a diagram showing an example of an imaging device according to an embodiment. FIG. 3 is a diagram showing an example of a folder structure and a container structure of MXF. It is a figure which shows an example of a time code and an extended time code. FIG. 3 is a diagram showing a timing chart of frame rates and refresh rates of moving image data. It is a flowchart which shows the flow of processing of this embodiment. FIG. 6 is a diagram showing an example of a timing chart of each frame in multi-photography using two video cameras. It is a figure showing an example of a UI screen. FIG. 7 is a diagram showing another example of the UI screen.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. However, the configurations described in each embodiment below are merely examples, and the scope of the present invention is not limited by the configurations described in each embodiment.

FIG. 1 is a diagram showing an example of an imaging device 100 of this embodiment. The imaging device 100 is, for example, a video camera. For example, another imaging device (video camera) is connected to the imaging device 100 as an external device. Furthermore, in the case of a system configuration in which a plurality of imaging devices 100 are connected to a predetermined information processing device, one of the plurality of imaging devices may be the imaging device 100 in FIG. 1 . A video synchronization signal and time code information are input to the imaging device 100 from an external imaging device or a predetermined information processing device via a predetermined terminal. As the predetermined terminal, an SDI (Serial Digital Interface) terminal, an HDMI (High-Definition Multimedia Interface) terminal, etc. can be applied.

The imaging device 100 includes a system control section 101, an imaging section 102, an image sensor section 103, a moving image storage section 104, a display section 105, a display data storage section 106, an input section 107, and a system clock generation section 108. The system control unit 101 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a system timer, and the like. A control program stored in the ROM is expanded into the RAM, and the CPU executes the control program expanded into the RAM, thereby realizing the processing of this embodiment. The CPU of the system control unit 101 operates with a clock generated by the system clock generation unit 108 shown in FIG. The system control unit 101 may be a microcomputer, a programming circuit, or the like. The system control unit 101 corresponds to setting means, changing means, and display control means.

The imaging unit 102 includes optical units such as a focus lens, a zoom lens, an aperture, and a shutter, and performs optical processing on an incident optical image. The image sensor unit 103 performs a process of converting the optical image optically processed by the image capture unit 102 into an electrical signal. The converted electrical signal is output to the imaging signal control section 109. The image sensor unit 103 includes an image sensor such as a CCD (Carry Coupled Device Image Sensor) or a CMOS (Complementary Metal Oxide Semiconductor).

The moving image storage unit 104 stores moving image data and is composed of a memory card, a built-in memory, and the like. The display unit 105 displays a predetermined screen (such as a UI screen) under the control of the display control unit 111. The display data storage unit 106 stores display data displayed on the display unit 105. The input unit 107 is a member that accepts input operations by the user, and for example, a jog dial, a dial switch, etc. can be applied.

Next, the system control unit 101 will be explained. The imaging signal control unit 109 performs image processing such as A/D conversion processing and amplification processing on the electrical signal output by the imaging element unit 103 to generate a video signal. The frame rate (number of frames per second) of the generated video signal differs depending on the recording method. In the case of the NTSC system, the frame rate is 120p, 60p, 60i, 30p, or 24p. In the case of the PAL system, the frame rate is 100p, 50p, 50i, or 25p. The size of the recorded image can be set to "4096x2160", "3840x2160", "2160x1080", "1920x1080", "1280x720", "1440x1080", etc. The following description will be made assuming that the size of the recorded image (recorded image size) is "1920 x 1080" and the frame rate is set to 30p. However, the recorded image size and frame rate may be set arbitrarily. The video signal generated by the imaging signal control section 109 is sent to the moving image control section 110.

The moving image control unit 110 has a function of compressing (encoding) and expanding (decoding) video signals. When the video signal is sent from the imaging signal control unit 109, the video control unit 110 compresses the video signal and stores video data constituted by the compressed video signal in the video storage unit 104. Furthermore, when an instruction is sent from the input control section 113, the moving image control section 110 reads out moving image data according to the instruction from the moving image storage section 104. When the moving image control section 110 reads the moving image data from the moving image storage section 104 , it decompresses the moving image data and sends the moving image data to the display control section 111 . The display control unit 111 controls displaying moving image data on the display unit 105. As a result, the moving image data is displayed on the display unit 105. Further, when the time code is sent from the time code phase control unit 114, the moving image control unit 110 superimposes the time code on the video signal and then performs compression. The moving image control section 110 can send the video signal output by the imaging signal control section 109 and the moving image data read from the moving image storage section 104 to the external output control section 124 .

In this embodiment, it is assumed that the moving image data includes management information such as a time code and audio data recorded in a container structure called MXF (Material eXchange Format). However, the moving image data may be in a file format such as AVCHD (Advanced Video Codec High Definition) or MP4 (MPEG-4 Part 14 or ISO/IEC14496-14:2003).

FIG. 2(a) is a diagram showing the folder structure of MXF recorded in the moving image storage unit 104, and FIG. 2(b) is a diagram showing the container structure of MXF. When the moving image storage unit 104 is initialized by an instruction to the input unit 107 (such as a user instruction), a CONTENTS folder 201 and a clip folder 202 are generated. In FIG. 2(b), the clip folder 202 is indicated as CLIPS001. Furthermore, when recording of moving image data is started in response to a user's instruction to the input unit 107, a stream file (MXF file 204) and editing information (XML file 205) are recorded in one clip. FIG. 2A shows an example in which “A001C001_141120XX.MXF” is recorded as the MXF file 204 and “A001C001_141120XX.XML” is recorded as the XML file 205. As a plurality of clips are recorded, the file names of the MXF files and XML files are changed and the number of MXF files and XML files increases. The index file 203 (INDEX.MIF in FIG. 2A) is a file that collectively manages a plurality of clip information.

Each clip information is collectively managed in an index file 203. Therefore, even if the number of clips increases, there is no need to analyze each clip, and it is sufficient to analyze the index file 203. This can reduce analysis time. Further, information such as frame numbers and time codes that need to be set for each frame is recorded in the MXF file 204.

In FIG. 2(b), a header 210 (Header Metadata) indicates the start of the file. The header 210 records metadata regarding the file. The header 210 records information on the resolution and frame rate when recording moving image data, information on the reproduction refresh rate, and the like. Each of these pieces of information is data that constitutes metadata. Metadata may also include other information. Information on the reproduction refresh rate will be described later. The frame 211 is frame data, and is composed of metadata 213 (System Metadata), image data 214, and audio data 215. Footer 212 (Footer Metadata) is information indicating the end of the file. The frame number is recorded in the frame metadata 213 area, with the start frame being "0". The XML file 205 is a markup language XML (Extensible Markup Language) file, and editing information can be written using tag information.

The external output control section 124 outputs the moving image data sent from the moving image control section 110 to an external device. As described above, the external device includes a predetermined information processing device and other imaging device. For the external output control unit 124, an SDI (Serial Digital Interface) terminal, an HDMI (High-Definition Multimedia Interface) terminal, etc. can be adopted. The moving picture storage unit 104 supports MPEG2 (Moving Picture Experts Group), H. 264, H. It is possible to store moving image data based on a format such as H.265. The moving image storage section 104 can write and read moving image data according to instructions received from the moving image control section 110.

The input control unit 113 transmits the input operation instructions received by the input unit 107 to each unit. The input unit 107 is used when the user operates the imaging device 100. The input control unit 113 sends instructions to the moving image control unit 110 to start recording, stop recording, select a recording frame rate, etc. Further, the input control unit 113 can select desired display data from the display data storage unit 106 by sending an instruction to the display data control unit 112. Further, the input control unit 113 can send an instruction to the time code control unit 115 to set a time code arbitrarily specified by the user.

The display data control unit 112 reads display data from the display data storage unit 106 according to the user's instruction received by the input control unit 113. The display data storage section 106 stores display data, and the display data is read out according to instructions received from the display data control section 112. The display control unit 111 can synthesize display data sent from the display data control unit 112 and information from the moving image control unit 110, time code phase control unit 114, and synchronization signal phase control unit 120, and display display data. Convert to a format. Then, the display control unit 111 performs display control by outputting display data to the display unit 105.

The display unit 105 displays the display data in the display format converted by the display control unit 111. The display unit 105 includes, for example, a liquid crystal screen such as an LCD (Liquid Crystal Monitor) or an EVF (Electronic View Finder). However, the display unit 105 is not limited to a liquid crystal screen. The moving image storage unit 104 and the display data storage unit 106 are configured of storage media such as optical disks such as HDDs (Hard Disk Drives) and DVDs (Digital Versatile Discs), semiconductor memories, and the like.

The time code generation unit 116 generates a time code according to instructions received from the time code control unit 115. Time codes can be generated in units of hours, minutes, seconds, frames, and subframes. The number of frames changes depending on the set recording frame rate. A subframe is extended time code information that is more accurate than a frame. FIG. 3 is a diagram showing an example of a time code and an extended time code. Here, the time code setting values defined in SMPTE ST 12-1 and SMPTE ST 12-2 are expressed in hours (HH301), minutes (MM302), seconds (SS303), and frames (FF304). In HH301, time from “00 to 23” is expressed. In MM302, the minutes “00 to 59” are expressed. SS303 represents seconds from "00 to 59". The FF 304 represents frames “00 to 29”.

Here, if the frame rate is 120p, etc., the time code settings defined in SMPTE ST 12-1 and SMPTE ST 12-2 cannot distinguish between frames, and different frames may have the same time code value. is set. On the other hand, in the time code setting value defined in SMPTE ST 12-3, a subframe (SF305) is added. As shown in the extended time code of FIG. 3, subframes "00 to 31" are expressed in SF305. As a result, a different time code can be added to each frame of 960p (=30×32p) using the FF 304 and SF 305. In other words, even when the frame rate is higher than 60p, each frame can be distinguished because a different time code is added to each frame. Hereinafter, information on the frame (FF304) will be referred to as frame information, and information on the subframe (SF305) will be referred to as subframe information.

Time code control section 115 reads out the time code generated by time code generation section 116 and sends it to time code phase control section 114 . Further, the time code control unit 115 can also change the time code of the imaging device 100 by sending the time code sent from the time code phase control unit 114 to the time code generation unit 116. Further, the time code control unit 115 sends the time code to the time code generation unit 116 in response to an instruction sent from the input control unit 113 (instruction received by the input unit 107).

The time code phase control section 114 performs phase control between the time code sent from the time code control section 115 and the external time code sent from the external time code control section 117. If it is necessary to shift the phase, the time code phase control section 114 sends the amount of phase shift to the synchronization signal phase control section 120. When the time code is sent from the external time code control unit 117, the time code phase control unit 114 sends the time code to the time code control unit 115. The time code control unit 115 changes the time code of the imaging device 100 based on the sent time code. By sharing the time code between the time code phase control unit 114 and the moving image control unit 110, the time code can be superimposed on the video signal and then compressed. Furthermore, the time code can be obtained from the expanded video signal. Further, the time code phase control unit 114 can display the time code on the display unit 105 by sending the phase-controlled time code to the display control unit 111.

When the system clock control unit 123 receives the amount of phase shift of the system clock from the synchronization signal phase control unit 120, it controls the system clock generation unit 108 to absorb the phase shift. In this case, the system clock control unit 123 performs PLL (Phase Lock Loop) control of the system clock generation unit 108 to control the phase matching.

The synchronization signal phase control section 120 calculates the amount of phase shift between the video horizontal synchronization signal and vertical synchronization signal sent from the external synchronization signal control section 121 and the system clock sent from the system clock control section 123. do. The synchronization signal phase control unit 120 sends the calculation result to the system clock control unit 123 and issues a request to shift the phase of the system clock of the imaging device 100. Further, when the internal time code sent from the time code phase control section 114 and the external time code are out of phase, the amount of phase shift is sent to the synchronization signal phase control section 120. The synchronization signal phase control section 120 sends the amount of phase shift to the system clock control section 123. As a result, even if the external operating cycle and the internal operating cycle are different, it is possible to request a shift in the phase of the system clock in consideration of the amount of time code shift. Furthermore, the synchronization signal phase control section 120 sends a notification that the phase alignment is completed to the display control section 111. Thereby, it is possible to display on the display unit 105 that the phase alignment has been completed.

The external input signal detection unit 119 detects whether a signal (input signal) is input from an external device. The external input signal detection unit 119 separates a time code signal (hereinafter referred to as an external time code signal) and a video synchronization signal (synchronization signal) from the input signal. The external time code signal is sent to an external time code signal detection section 118, and the video synchronization signal is sent to an external synchronization signal detection section 122. External time code signal detection section 118 detects an external time code signal. External time code control section 117 detects the frequency of the external time code sent from external time code signal detection section 118 and the presence or absence of subframe information, and notifies time code phase control section 114 of the frequency. The external synchronization signal detection unit 122 detects a video synchronization signal using, for example, an SDI terminal or a synchronization signal input terminal. External synchronization signal control section 121 extracts a horizontal synchronization signal and a vertical synchronization signal from the video synchronization signal, and sends the extracted horizontal synchronization signal and vertical synchronization signal to synchronization signal phase control section 120 .

Next, a timing chart of the frame rate and refresh rate of moving image data will be explained. FIG. 4A is a diagram showing an example of a timing chart of the recording frame rate of moving image data stored in the moving image storage unit 104. In FIG. 4(a), the constant frame rate (CFR) is a fixed frame rate (30p in FIG. 4(a)), and each frame is processed in the order of A1 frame, A2 frame, etc. Photograph and record. In the variable frame rate (VFR) in FIG. 4A, the recording frame rate is variable. In the case of the variable frame rate shown in Figure 4(a), frames A1 to A4 are shot at a shooting frame rate of 30p, frames A5 to A8 are shot at a shooting frame rate of 120p, and frames from A9 onwards are shot at a shooting frame rate of 30p. Shoot at the shooting frame rate and record each frame in the shooting order.

Next, the refresh rate will be explained. FIG. 4(b) is a diagram illustrating an example of a timing chart of refresh rates. The refresh rate is a cycle at which a video read from video data stored in the video storage unit 104 or a video captured by the imaging unit 102 is output to the display unit 105 or the external output control unit 124. In FIG. 4B, Constant Refresh Rate (CRR) shows an example in which video is output at a fixed refresh rate of 30p. In FIG. 4B, Variable Refresh Rate (VRR) indicates that the playback refresh rate is variable. In the variable refresh rate of FIG. 4(b), the video is output at 30p from the A1 frame to the A4 frame, at 120p from the A5 frame to the A8 frame, and at 30p for the A9 frame, A10 frame, and so on.

FIG. 4(c) is a diagram showing a timing chart when moving image data recorded by VFR is outputted as a video using CRR and when video is outputted using VRR. In the following description, it is assumed that the video output destination is the display section 105 and that the moving image data is reproduced on the display section 105. When video data recorded in VFR is played back in CRR, frames from A5 frames to A8 frames that were recorded at 120p are played back as 30p. Therefore, it can be seen that frames A5 to A8 are played back at a slow speed of "1/4". On the other hand, when video data recorded with VFR is played back with VRR, frames recorded at 120p from A5 frames to A8 frames are played back as 120p. From this, it can be seen that frames A5 to A8 are reproduced at the same magnification (120 Hz refresh rate reproduction) as when they were recorded.

For this reason, it is necessary to control the time code during recording when recording moving image data using VFR depending on whether CRR or VRR reproduction is performed. Similarly, it is necessary to change the time code control during recording for video output to the external output control section 124. As shown in Figure 4(c), when playing video data recorded using VFR using CRR, the time code during VFR recording must be controlled to 30p, which is the frame rate when playing back using CRR. . By performing this control, during CRR playback, the portion recorded at 120p during VFR recording is played back at "1/4" slow speed. On the other hand, when playing back with VRR, frames recorded at 120p up to A8 frames are played back at 120p. Even in 120p, it is necessary to perform control to record a time code that includes subframe information so that a correct time code can be assigned. By performing this control, the portion recorded at 120p (portion from A5 frame to A8 frame) during VRR playback can be played back at the same frame rate (120p) as at the time of recording.

In this way, it is necessary to give a time code in consideration of the playback refresh rate. Therefore, in this embodiment, when the playback refresh rate is set to VRR, a time code including subframe information is recorded, and when the playback refresh rate is set to CRR, no subframe information is recorded. Record frame rate. That is, when set to VRR, the setting is to record a time code that includes subframes, and when CRR is set, the setting is to record a time code that does not include subframes. However, if the playback refresh rate is higher than 60p, a time code including subframe information may be recorded.

Next, the flow of processing in this embodiment will be explained. FIG. 5 is a flowchart showing the flow of processing in this embodiment. As described above, in this embodiment, multi-photography using a plurality of video cameras can be performed. The flowchart in FIG. 5 shows a flow in which one imaging device 100 among a plurality of video cameras executes the processing of this embodiment when multiple shooting is performed. The system control unit 101 determines whether the VFR recording mode is set based on an instruction received by the user via the input unit 107 (S501). If the determination in S501 is No, CFR recording mode processing is executed. In other words, the process in FIG. 5 is performed when VFR recording is performed. Here, the VFR recording mode is a recording mode (shooting mode) in which the shooting frame rate can be changed according to a user's instruction while moving image data is being shot. The CFR recording mode is a recording mode (shooting mode) in which the shooting frame rate cannot be changed during shooting, and shooting is performed at the shooting frame rate set before shooting, and the shot moving image data is recorded. Note that each process in the flowchart in FIG. 5 is a process related to this embodiment, and the imaging apparatus 100 can perform other processes in addition to each process in the flowchart in FIG.

The system control unit 101 determines whether S502 exists based on an instruction from the user via the input unit 107 (S502). If the determination in S502 is No, the flow does not proceed to the next step and returns to S502. If there is a request to start recording, the determination in S502 is Yes. In this case, the system control unit 101 determines whether multi-imaging is being performed (S503). The system control unit 101 can determine whether multi-imaging is being performed, based on whether the external synchronization signal detection unit 122 detects input of a video synchronization signal (synchronization signal) from an external device.

If it is determined Yes in S503, the system control unit 101 determines whether the refresh rate setting during playback is a CRR setting or a VRR setting, based on an instruction from the user via the input unit 107. is determined (S504). In this embodiment, as described above, when the playback refresh rate set by the user is CRR, the time code control unit 115 performs control that does not take subframe information into consideration, that is, controls the time code that does not include subframe information. Make a record. On the other hand, when the playback refresh rate set by the user is VRR, the time code control unit 115 performs control that takes subframe information into consideration, that is, records a time code that includes subframe information.

If the playback refresh rate set by the user is CRR, the determination in S504 is Yes. In this case, the system control unit 101 changes the reproduction refresh rate setting from CRR to VRR (S505). That is, even if the reproduction refresh rate is set to CRR by the user, the reproduction refresh rate is changed from CRR to VRR. As a result, even if the playback refresh rate is set to CRR and the time code that does not include subframe information is set to be recorded, the time code control unit 115 performs control that takes subframes into account, that is, subframe information is recorded. Change the settings to record timecode including frame information. When recording moving image data using a single imaging device instead of multiple shooting, there is no need to synchronize videos (and time codes).

Therefore, a time code in a format corresponding to the playback refresh rate set by the user is recorded. However, in the case of multi-photography, it is necessary to align the time codes among the plurality of imaging devices. When moving image data is recorded using VFR and the playback refresh rate is set to CRR, a correct time code may not be assigned to each frame constituting the moving image data. Therefore, in the case of multi-shooting, even if the playback refresh rate is set to CRR, by changing to VRR, the correct time code can be recorded, and the time code can be synchronized between multiple imaging devices. You will be able to do this.

Next, the video control unit 110 executes a process of temporarily holding the playback refresh rate information in order to record the playback refresh rate information in the video storage unit 104 as metadata that is incidental information of the video data. (S506). The reproduction refresh rate setting is automatically switched from CRR to VRR in S505. However, the playback refresh rate set by the user was CRR. Therefore, in S506, the playback refresh rate is changed to VRR, but the video control unit 110 records metadata regarding the playback refresh rate setting (playback refresh rate metadata) as CRR in the header 210. to control. The playback refresh rate metadata may be recorded in the index file 203 instead of the header 210. Furthermore, although not described in the flowchart of FIG. 5, the metadata of the playback refresh rate is also recorded during normal shooting, not during multi-shooting, or when the playback refresh rate is set to VRR during multi-shooting. At this time, the set playback refresh rate is recorded as playback refresh rate metadata.

Next, the moving image control unit 110 opens a moving image file (S507). At this time, the playback refresh rate metadata temporarily held in S506 is recorded in the header 210 of the file. After receiving the instruction to start recording in S502, the moving image control unit 110 captures an image using the image sensor unit 103, processes the image in the image signal control unit 109, and sends the video signal ( frame) and performs encoding (S508). The video control unit 110 multiplexes and adds metadata including the time code sent from the time code control unit 115 to the compressed video signal (frame), and stores the frame in the video storage unit 104. It is stored (S509). The time code generation unit 116 performs time code increment control according to the set playback refresh rate according to instructions from the time code control unit 115 (S510). In the present embodiment, in the case of multiple shooting, the time code generation unit 116 performs increment control of the time code according to the shooting frame rate in the VRR. In the case of normal shooting (with one imaging device) rather than multi-shooting, the playback refresh rate that is set, that is, VRR setting or CRR setting, and time code increment control according to the rate. conduct.

The input control unit 113 determines whether the user operates the input unit 107 to request to stop recording (S511). If the determination in S511 is No, the input control unit 113 determines whether the user has operated the input unit 107 to request switching of the shooting frame rate (S512). If the determination is Yes in S512, the control unit (not shown) switches the imaging frame rate at which the image sensor unit 103 performs imaging (S513). The flow then returns to S508. If the determination in S513 is No, the process in S513 is not performed and the flow returns to S508. If the determination in S511 is Yes, the video control unit 110 closes the video file because an operation requesting to stop recording has been performed (S514).

In the above flow, whether or not multi-photography is being performed and the playback refresh rate setting are determined after a request to start recording is made in S502, and the playback refresh rate is changed in accordance with the results. However, it is also possible to determine whether multi-photography is used or not and the playback refresh rate setting during shooting standby in the VFR recording mode, before accepting a request to start recording.

FIG. 6 is a diagram showing an example of a timing chart of each frame in multi-photography using two video cameras. Camera A of the two video cameras switches the frame rate to 120p while recording at a frame rate of 30p, and then switches the frame rate to 30p. On the other hand, camera B continues to record at a fixed frame rate of 30p. In the following description, 120p corresponds to the first frame rate, and 30p corresponds to the second frame rate. The first frame rate is an arbitrary frame rate using subframe information, and the second frame rate is an arbitrary frame rate using no subframe information. Therefore, the first frame rate is higher than the second frame rate.

As shown in FIG. 6, camera A records frames A1 to A3 at a frame rate of 30p, and the time code is controlled using the reproduction refresh rate as VRR. Therefore, the time code is expressed in the order of hours, minutes, seconds, frames and subframes. The time code of the A1 frame is "00:00:00:00.0". Since the frame rate is 30p, the time code of the A2 frame is "00:00:00:01.0". Similarly, the time code of the A3 frame is "00:00:00:02.0".

Here, assume that while an A3 frame is being recorded, the user operates the input unit 107 to request a frame rate switch from 30p to 120p. In this case, the imaging unit 102 and the imaging signal control unit 109 control the A4 frame to be recorded at a frame rate of 120p. Since frames A4 to A11 are recorded at a frame rate of 120p, which is higher than 60p, time code subframes are used. The time code of the A4 frame is "00:00:00:03.0". The time code of the A5 frame is "00:00:00:03.1". The time code of the A8 frame is "00:00:00:04.0". Here, it is assumed that the user requests the input unit 107 to switch the frame rate from 120p to 30p at the timing when the A9 frame is being recorded.

The time code of the A9 frame is "00:00:00:04.1" and the subframe is "1". When the frame rate is switched from 120p to 30p at this timing, the frame rate becomes 30p from the A10 frame. In this case, when the recorded video data is played back by VRR, the time code changes from "00:00:00:04.1" to "00:00:00:05.0" from A9 frame to A10 frame. , it becomes impossible to maintain real time during VRR playback. Therefore, in the present embodiment, if there is a request to switch the frame rate from 120p to 30p while recording an A9 frame, the moving image control unit 110 controls the frame rate to be recorded at 30p from the A12 frame. In other words, if there is a request to switch to recording using subframe information while recording is being performed without using subframe information, the moving image control unit 110 will frame the frame at the timing when the subframe information becomes zero. Switch rates. This makes it possible to maintain real time during VRR playback. Further, since the frame rate of each frame after the A12 frame is 30p, the subframe of each frame remains zero.

Camera B records at a fixed frame rate of 30p starting from the B1 frame, and the frame rate is not switched. However, even if the playback refresh rate is set to CRR, the time code is controlled so that the playback refresh rate is VRR. Therefore, the time code is expressed in the order of hours, minutes, seconds, frames and subframes. The time code of the B1 frame is "00:00:00:00.0" and the time code of the B2 frame is "00:00:00:01.0." In camera A, frame rate switching occurs from the A4 frame, but in camera B, frame rate switching does not occur. Therefore, B4 frames are recorded at a frame rate of 30p. The time code of the B4 frame is "00:00:00:03.0". At the transition to frame B5, the time code subframe remains zero and the time code frame is incremented.

7 and 8 are diagrams showing examples of UI screens for performing various settings. The UI screen is displayed at 105 on the display unit. The UI screens in FIGS. 7 and 8 are displayed on the display unit 105 under the control of the display control unit 111. At this time, the display control unit 111 superimposes an OSD display on the video 400 of the moving image data input from the imaging unit 102 and causes the display unit 105 to display the superimposed OSD display. FIG. 7(a) shows a UI screen for setting the VFR recording mode. The UI screen includes a remaining recording time 401, an operating status 402, a time code display 403, and a mode selection screen 404. The remaining recording time 401 is a display indicating the remaining recordable time of the SD card when the SD card (registered trademark) is applied to the moving image storage unit 104. The operating status 402 is a display indicating whether the imaging apparatus 100 is performing a recording operation. When the imaging device 100 is performing a recording operation, a predetermined mark is displayed. The time code display 403 is a display showing the time code generated by the time code generation unit 116. The mode selection screen 404 is a first screen for selecting whether to enable the VFR recording mode. In other words, the mode selection screen 404 is a screen for selecting either a mode for recording at a variable frame rate or a mode for recording at a fixed frame rate. In the example of FIG. 7(a), the VFR recording mode can be selected as ON or OFF. When the VFR recording mode is set to ON, recording at a variable frame rate is selected. When the VFR recording mode is set to OFF, recording at a fixed frame rate is selected.

FIG. 7(b) shows a UI screen for setting the playback refresh rate mode. When the VFR recording mode is set to ON as shown in FIG. 7A, the operating status of the imaging device 100 becomes a recording standby state in the VFR recording mode. Therefore, the display control unit 111 changes the display of the operating status 402 to "VFR STBY" indicating the recording standby state. Furthermore, the display control unit 111 displays a mode selection screen 404 on which the reproduction refresh rate mode can be set as either CRR or VRR. Here, as described above, if the setting of the playback refresh rate is automatically changed to VRR in S505, the display control unit 111 may disable selection of the playback refresh rate on the mode selection screen 404. good. That is, when the playback refresh rate setting is automatically changed from CRR to VRR in S505, the user may not be allowed to change the time code setting. For example, the display control unit 111 may control the mode selection screen 404 so that neither CRR nor VRR can be selected, or may control the mode selection screen 404 not to be displayed.

FIG. 8A shows a UI screen for starting VFR recording and setting the frame rate during recording. When VFR recording is being performed, the display control unit 111 changes the display of the operating status 402 to "VFR REC" indicating that VFR recording is in progress, and adds a predetermined mark. In the example of FIG. 8(a), the predetermined mark is indicated by a hatched circle. When VFR recording is started, the time code indicated by time code display 403 advances. Further, when VFR recording is started, the display control unit 111 displays frame rate information 410. When the frame rate during recording is 30p and the playback refresh mode is set to VRR, "30fps/30p" indicating real time is displayed in the frame rate information 410.

The display control unit 111 also displays a frame rate selection screen 411. As shown in FIG. 8A, on the frame rate selection screen 411, it is possible to select any one of four shooting frame rates. The user can select any shooting frame rate using the input unit 107. The selectable shooting frame rates are not limited to four.

FIG. 8(b) shows a UI screen displayed after switching the shooting frame rate from 30p to 120p during VFR recording. The time code displayed on the time code display 403 is in increments. Further, it is assumed that the frame rate during recording is 120p and the playback refresh mode is set to VRR. In this case, the display unit 105 displays "120 fps/120p" indicating real time in the frame rate information 410.

The present embodiment described above can be applied when performing multiple imaging. Therefore, in S506 described above, the moving image control unit 110 records metadata regarding the playback refresh rate setting in the header 210 as a CRR. In this regard, in the case of non-multi-shooting, metadata related to playback refresh rate settings (information indicating that the playback refresh rate is VRR or CRR) may not be recorded in the header 210. That is, if a synchronization signal is not input, the above metadata does not need to be recorded in the header 210. Further, as described above, when the playback refresh rate setting is automatically changed from CRR to VRR in S505, changes in the playback refresh rate, that is, the time code recording method, may not be allowed. It can also be allowed. After the playback refresh rate setting is changed from CRR to VRR in S505, when the time code setting (playback refresh rate setting) is changed from VRR to CRR by the user's operation, the moving image control unit 110 changes the playback refresh rate from CRR to VRR. It is also possible to prohibit recording (stop if recording is in progress).

Although the preferred embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications and changes can be made within the scope of the invention. The present invention provides a system or device with a program that implements one or more functions of each embodiment described above via a network or a storage medium, and one or more processors of a computer in the system or device executes the program. It can also be realized by reading and executing processing. The present invention can also be implemented by a circuit (eg, an ASIC) that implements one or more functions.

100 Imaging device 104 Moving image storage section 110 Moving image control section 111 Display control section 114 Time code phase control section 115 Time code control section 116 Time code generation section 119 External input signal detection section

Claims (11)

a setting means that can set whether to record a first time code that includes subframe information that is an extension of frame information, or to record a second time code that does not include the subframe information;
When recording moving image data, if the second time code is set to be recorded and a synchronization signal is input from an external device, the first time code is changed to be recorded. means and
An imaging device comprising: The setting means sets the first time code to be recorded when the refresh rate during playback is set to the variable refresh rate, and records the first time code when the refresh rate during playback is set to the fixed refresh rate. The imaging device according to claim 1, wherein the imaging device is set to record a second time code. The imaging apparatus according to claim 2, wherein the setting unit sets a refresh rate during playback according to a user operation. When the recording of the second time code is changed to the recording of the first time code, control is performed to record information indicating that playback is performed at a fixed refresh rate as additional information of the video data. The imaging apparatus according to claim 1, further comprising a control means for controlling the image capturing apparatus. The control means controls such that when the synchronization signal is not input, information indicating that playback is performed at a fixed refresh rate or information indicating that playback is performed at a variable refresh rate is not recorded in the additional information. The imaging device according to claim 4, characterized in that: It has a variable frame rate recording mode that allows you to change the shooting frame rate while shooting.
The changing means is configured to record the second time code when the variable frame rate recording mode is set, and when a synchronization signal is input from an external device, the changing means changes the first time code. 5. The imaging device according to claim 1, wherein the imaging device is modified to record a time code of . After the recording of the second time code is automatically changed to the recording of the first time code by the changing means, the recording of the first time code is changed to the recording of the second time code by a user operation. 7. The imaging apparatus according to claim 6, wherein when the mode is changed to recording, recording in the variable frame rate recording mode is prohibited. The imaging device according to any one of claims 1 to 5, characterized in that, when a synchronization signal is not input from an external device, the change means does not perform the change. The setting means has a selection screen for selecting recording of the first time code or recording of the second time code,
When the second time code is changed to the first time code by the changing means, selection of recording of the first time code or recording of the second time code is made on the selection screen. The imaging device according to any one of claims 1 to 5, wherein the imaging device is not allowed. Control of an imaging device having a setting means that can set whether to record a first time code that includes subframe information that is an extension of frame information, or to record a second time code that does not include the subframe information. A method,
When recording moving image data, if the second time code is set to be recorded and a synchronization signal is input from an external device, changing the first time code to be recorded. ,
A control method comprising: A program for causing a computer to execute each means of the imaging apparatus according to any one of claims 1 to 9.

Images