JP2007288581A - Image recorder, imaging apparatus, control method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus by which not only video images but sound are smoothly connected and edited without discontinuity. <P>SOLUTION: A digital video camera has: an image compression and expansion part 103; a sound compression and expansion part 104; a multiplexing processing part 105; an external TTC detection part 106; a CPU 109; a recording medium interface 111; a recording medium 112 and an external synchronization detection part 113. The external synchronization detection part 113 synchronizes an image signal of the digital video camera with an image signal of an external digital video camera based on an external synchronizing signal (Genlock signal) input from the outside. The CPU 109 performs control so that compression coding is performed by setting difference between time information of the head PES of image data and time information of the head PES of sound data to zero in a compression coding operation in a state in which the image signal of the digital video camera is synchronized with the image signal of the external digital video camera. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image recording apparatus, an imaging apparatus, a control method, and a program applied to a digital video camera that records a bit stream encoded by a moving picture experts group (MPEG) system on a recording medium.

  For example, when switching or synthesizing a plurality of images taken by two or more video cameras in real time, the plurality of images cannot be mixed as they are. In general, a video camera expresses a moving image by continuously sending images as about 30 “frames” per second, and a start point is set for each frame. If the frame start points do not match, images from multiple video cameras cannot be switched or combined in real time.

  The GenLock signal used in the video camera is a synchronization signal for designating the start point of the frame, and is one of the signals used as a reference for synchronizing a plurality of video cameras with each other. Genlock (phase matching) is generally performed by passing a composite video signal (a signal sent without separating color signals and luminance signals) of a master source side video camera through a sink generator of the slave side video camera.

  The sync generator of the slave video camera to be phased (locked) is provided with a vertical drive circuit, a horizontal drive circuit, and a circuit for separating subcarriers. By performing the phase matching process, the sync generator of the slave video camera is phase-locked (locked) to the subcarrier separation circuit, vertical drive circuit, and horizontal drive circuit of the master source video camera. As a result, both the master source side and slave side video camera sync generators operate at the same frequency and phase.

As a method for synchronizing the images of a plurality of video cameras using the above-described Genlock signal, the following technique has been proposed (see, for example, Patent Document 1).
JP-A-2-149184

  By the way, in recent video cameras, a method of compressing and recording a moving image using a high-efficiency encoding technique such as an MPEG method has become mainstream. In a video camera having a general MPEG encoder, the video and audio encoding units of the generated MPEG bit stream do not match.

  FIG. 3 is a diagram for explaining the discrepancy between the video PTS interval and the audio PTS interval. As shown in FIG. 3, the PTS (Presentation Time Stamp) interval of each video PES (Packetized Elementary Stream) is different from the PTS interval of each audio PES.

  FIG. 4 is a diagram for explaining that the beginning PTS of video and the beginning PTS of audio differ for each stream. (A) is a diagram showing a video PES and an audio PES of the video camera A, and (b) is a diagram showing a video PES and an audio PES of the video camera B. As shown in FIG. 4, the interval between the PTS of the first video PES and the PTS of the first audio PES of the MPEG bit stream is not always a fixed value. That is, in the example of FIG. 4, the delay from the head of the stream is different between the video camera A and the video camera B.

  Therefore, when trying to connect and edit MPEG bit streams generated by a plurality of video cameras whose videos are synchronized by the above-described GenLock signal, the following problems occur. FIG. 5 is a diagram for explaining that an audio discontinuity occurs when two bit streams in which video is synchronized are connected and edited. (A) is a diagram showing a video PES and an audio PES of the video camera A, (b) is a diagram showing a video PES and an audio PES of the video camera B, and (c) is a diagram showing an edit stream. As shown in FIG. 5, the video of the video camera A and the video of the video camera B can be continuously connected, but the audio may be discontinuous.

  An object of the present invention is to provide an image recording apparatus, an imaging apparatus, a control method, and a program capable of smoothly connecting and editing not only video but also sound without discontinuity.

  In order to achieve the above-described object, the image recording apparatus of the present invention compresses and encodes an image signal and an audio signal and reproduces the compressed image data and audio data for each compression encoding unit. An image recording apparatus having a function of adding time information to perform synchronization, and synchronizing means for synchronizing an image signal of the image recording apparatus with an image signal of an external device based on an external synchronization signal input from outside In the compression encoding operation in a state where the image signal of the image recording apparatus is synchronized with the image signal of the external device by the synchronization means, the time information of the first compression encoding unit of the image data and the start of the audio data Control means for controlling to perform compression encoding by setting the difference from the time information of the compression encoding unit to a prescribed value.

  According to the present invention, when a plurality of image recording devices or imaging devices are synchronized with each other by an external synchronization signal, and the recorded image data and audio data are connected and edited, smooth not only for video but also for audio. It is possible to connect and edit.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
FIG. 1 is a block diagram showing a configuration of a digital video camera as an imaging apparatus according to the first embodiment of the present invention.

  1, the digital video camera includes a lens 100, an image sensor (CCD) 101, a camera signal processing unit 102, an image compression / decompression unit 103, an audio compression / decompression unit 104, a multiplexing processing unit 105, and an external TTC detection unit 106. ing. The digital video camera further includes a microphone 107, an audio signal processing unit 108, a CPU 109, a memory 110, a recording medium interface 111, a recording medium 112, an external synchronization detection unit 113, a clock oscillator 114, and an operation unit (not shown).

  The digital video camera according to the present embodiment corresponds to the so-called HDV system whose specifications are defined by, for example, “Specifications of HDV Recording Version 1.0”. The lens 100 forms an optical image of a subject photographed by the digital video camera on the image sensor 101. The imaging element 101 photoelectrically converts an optical image from the lens 100 formed on the imaging surface into an electrical signal and outputs the electrical signal. The camera signal processing unit 102 performs signal processing so that the electrical signal (photoelectric conversion image signal) output from the image sensor 101 becomes a standard image signal.

  The image compression / decompression unit 103 encodes and compresses the image signal processed by the camera signal processing unit 102 using a method such as MPEG2, and sets the maximum bit length of one frame when the image signal is encoded. Is possible. The audio compression / decompression unit 104 performs audio processing for encoding and compressing an audio signal acquired from around the digital video camera by the microphone 106 using a method such as MPEG Audio encoding. The image signal and the audio signal are compressed and encoded, and then TTC, which will be described later, is added as time information.

  The multiplexing processing unit 105 multiplexes the image data compressed by the image compression / decompression unit 103 and the audio data compressed by the audio compression / decompression unit 104 into a PackUnit format. Further, the multiplexing processing unit 105 generates tape format data compliant with the HDV system from the multiplexed data and stores it in the memory 110. The external TTC detection unit 106 acquires TTC from a TTC (Total Time Code) signal input from another device outside the digital video camera, and allows TTC synchronization with the other device.

  The microphone 107 is used when acquiring sound. The audio signal processing unit 108 performs signal processing so that the audio signal acquired by the microphone 107 becomes a standard audio signal. The CPU 109 controls the operation of the entire digital video camera according to the control program stored in the memory 110, and executes the processing shown in the flowchart of FIG. The memory 110 stores a control program and is also used as a memory for temporarily storing image data and audio data.

  The recording medium interface 111 serves as an interface for recording / reproducing the tape format data to / from the recording medium 112. The recording medium interface 111 transfers the tape format data from the memory 110 storing the tape format data processed by the multiplexing processing unit 105 to the recording medium 112 during recording. On the contrary, the recording medium interface 111 expands the reproduction tape format data from the recording medium 112 to the memory 110 at the time of reproduction. The recording medium 112 stores tape format data recorded / reproduced by a digital video camera, and is configured as a magnetic tape.

  The external synchronization detection unit 113 detects a shift between a signal synchronized with a Genlock signal that is an external synchronization signal and a synchronization signal inside the digital video camera. Further, the external synchronization detection unit 113 performs PLL (Phase Lock Loop) control on the clock oscillator 114 so as to absorb the difference between both signals. The details of the synchronization method using the Genlock signal are described in, for example, the above-mentioned Patent Document 1, and thus the description thereof is omitted. The clock oscillator 114 supplies a clock for operating each part in the block 115 of the digital video camera. The operation unit is used for compressing and encoding image signals and audio signals, and instructing start / end of recording and reproduction of the compressed and encoded image data and audio data.

  Next, video and audio PTS and DTS at the start of recording in the digital video camera of this embodiment will be described with reference to the flowchart of FIG. In this embodiment, a case where recording is started simultaneously by a plurality of digital video cameras is taken as an example. Of the plurality of digital video cameras, one digital video camera is on the master source side, and the other digital video camera is on the slave side.

  FIG. 2 is a flowchart showing processing for determining video and audio PTSs at the start of recording by the digital video camera (slave side).

  In FIG. 2, when the CPU 109 of the digital video camera receives a recording start instruction from the user via the operation unit in step S201, the recording medium interface 111 starts a recording operation on the recording medium 112 in step S202. Next, in step S <b> 203, the CPU 109 determines whether or not the digital video camera is synchronized with the outside using the Genlock signal that is an external synchronization signal, using the external synchronization detection unit 113.

  If the digital video camera is in synchronization with the outside in step S203, the CPU 109 performs the following process in step S204. First, the CPU 109 calculates the leading DTS (Decoding Time Stamp) of the video from the track position (ETN) of the recording medium 112 (magnetic tape) at the start of recording by the following equation.

DTS = (ETN * 3003) / 10 ... (for frame rate 29.97Hz)
DTS = (ETN * 3600) / 12 ... (when frame rate is 25Hz)
Next, the CPU 109 calculates an audio top PTS and a video top PTS by the following equations.

PTS (Video) = DTS + 3003 ... (when frame rate is 29.97Hz)
PTS (Video) = DTS + 3600 ... (when frame rate is 25Hz)
PTS (Audio) = PTS (Video)
Incidentally, as shown in FIG. 5C, audio may not be continuous with data already recorded on the recording medium 112 (magnetic tape). Accordingly, if necessary, 80 tracks of invalid data may be recorded before actual data is recorded on the recording medium 112, and the continuity of the MPEG buffer model may be discarded. In the case of FIG. 5C, Invalid data is recorded between VA3 and VB4.

  On the other hand, if the digital video camera is not synchronized with the outside in step S203, in step S205, the CPU 109 determines the beginning PTS of audio and the beginning PTS of video at the start of recording as follows.

PTS (Video) = Video PES recorded at the end of magnetic tape PTS + 3003
... (when frame rate is 29.97Hz)
PTS (Video) = Video PES PTS recorded at the end of magnetic tape + 3600
... (when frame rate is 25Hz)
PTS (Audio) = Audio PES PTS recorded at the end of magnetic tape + 2160
When the audio top PTS and the video top PTS are determined in step S204 or step S205, the CPU 109 sequentially determines the subsequent PTSs so as to maintain MPEG continuity. In step S206, the CPU 109 continues the process until receiving a recording end instruction from the user via the operation unit.

  By performing the above processing, when recording start is simultaneously instructed by a plurality of digital video cameras synchronized with the outside by the Genlock signal, the data recorded on the recording medium is a frame that has passed the same time from the recording start. When connecting and editing, the result is as follows. The delay (delay amount) between the PTS of the video PES and the PTS of the audio PES at an arbitrary connection editing point matches, so that not only video but also audio can be smoothly connected and edited.

  As described above, according to the present embodiment, two or more digital video cameras are synchronized with each other by the Genlock signal, and the same time has elapsed since the start of recording the encoded bitstream recorded on the recording medium. Connect and edit with frames. In that case, not only video of a plurality of bit streams but also audio can be smoothly connected and edited without discontinuity. That is, it becomes possible to connect and edit bit streams on a recording medium that have been recorded at the same timing by a plurality of digital video cameras at arbitrary points.

[Second Embodiment]
The second embodiment of the present invention is different from the first embodiment described above in the following points. Since the other elements of the present embodiment are the same as the corresponding ones of the first embodiment (FIG. 1) described above, description thereof is omitted.

  In the first embodiment, the case where recording is started simultaneously by a plurality of digital video cameras has been described. In contrast, in the present embodiment, a case will be described in which recording can be started at different timings with a plurality of digital video cameras.

  In the digital video camera of the present embodiment, the relationship between recording time information (particularly TTC) and audio PES and video PES PTS is uniquely determined. For this reason, it is necessary to define in advance the timing at which the TTC matches the PTS of the audio PES and the video PES. In the present embodiment, an example will be described in which the timing at which the TTC is 00: 00: 00: 00 in the BCD format is set as the reference timing at which the PTS of the audio PES and the PTS of the video PES match.

  Next, processing for determining PTS in the digital video camera of the present embodiment will be described with reference to the flowchart of FIG. Since the process of step S201-step S203 in this Embodiment is the same as 1st Embodiment mentioned above, description is abbreviate | omitted.

  When the digital video camera performs recording in synchronization with the outside using the Genlock signal, in step S204, the CPU 109 performs the following processing. First, the CPU 109 adds the PTS so that the PTS of the audio PES and the PTS of the video PES match at TTC = 00: 00: 00: 00. Next, the CPU 109 calculates the start DTS of the video and the PTS of the start video PES from the track position (ETN) of the recording medium 112 (magnetic tape) at the start of recording, using the following equations, as in the first embodiment. To do.

DTS = (ETN * 3003) / 10 ... (for frame rate 29.97Hz)
DTS = (ETN * 3600) / 12 ... (when frame rate is 25Hz)
PTS (Video) = DTS + 3003 ... (when frame rate is 29.97Hz)
PTS (Video) = DTS + 3600 ... (when frame rate is 25Hz)
Next, the CPU 109 calculates the delay (delay amount) between the PTS of the leading video PES and the PTS of the leading audio PES as follows.

(1) Calculate the number of video frames Video_Frames between the reference timing (00: 00: 00: 00) and the TTC set at the start of recording
Video_Frames = (TTC at start of recording-00: 00: 00: 00)
(2) Calculate the number of audio frames Audio_Frame that falls between the reference timing (00: 00: 00: 00) and the TTC set at the start of recording.
Audio_Frame = (Video_Frame * 3003) / 2160
... (for frame rate 29.97Hz): Rounding up the division result
Audio_Frame = (Video_Frame * 3600) / 2160
... (for frame rate 25Hz): Rounding up the division result (3) Audio and video when assuming continuous recording from the reference timing (00: 00: 00: 00) to TTC at the start of recording Calculate Delay
Delay = (Audio_Frame * 2160)-(Video_Frame * 3003)
... (when frame rate is 29.97Hz)
Delay = (Audio_Frame * 2160)-(Video_Frame * 3600)
... (when frame rate is 25Hz)
Finally, the CPU 109 calculates the PTS of the head audio PES as follows.

PTS (Audio) = PTS (Video) + Delay
Since step S205 and subsequent steps are the same as those in the first embodiment described above, description thereof will be omitted.

  By performing the above processing, multiple digital video cameras that synchronize with the outside using the Genlock signal can smoothly connect and edit not only video but also audio at the same TTC timing. .

  As described above, according to the present embodiment, two or more digital video cameras are synchronized with each other by the Genlock signal, TTC is input to synchronize the time code, and the codes recorded on the recording medium, respectively. Edit bitstreams at arbitrary points. In that case, in a frame having the same TTC of a plurality of bit streams, not only video but also audio can be smoothly connected and edited without discontinuity. That is, it becomes possible to connect and edit bitstreams on a recording medium that have been recorded at individual timings by a plurality of digital video cameras at arbitrary points.

[Other embodiments]
In the first embodiment, the digital video camera conforming to the HDV system has been described as an example, but the present invention is not limited to this. Any method may be used as long as video and audio are multiplexed using time information and recorded on a recording medium.

  In the second embodiment, the case where the TTC of the reference timing is set to 00: 00: 00: 00 has been described as an example, but the present invention is not limited to this. The TTC of the reference timing may be a value set in advance in the digital video camera or a value set by the user from the operation unit of the digital video camera.

  In the second embodiment, the case where TTC is used to acquire the reference timing is taken as an example, but the present invention is not limited to this. Any information can be used as long as time information can be obtained. For example, information such as RecTime (recording time) may be used.

  The object of the present invention can also be achieved by supplying a storage medium storing software program codes for realizing the functions of the above-described embodiments to a system or apparatus and performing the following processing. That is, it is also achieved by reading and executing the program code stored in the storage medium by the computer (or CPU, MPU, etc.) of the system or apparatus.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code and the storage medium storing the program code constitute the present invention. .

  As a storage medium for supplying the program code, for example, a floppy (registered trademark) disk, a hard disk, or a magneto-optical disk can be used. Further, optical disks such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD + RW, magnetic tape, nonvolatile memory card, ROM, and the like can be used. Alternatively, the program code may be downloaded via a network.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also the following cases are included. That is, based on the instruction of the program code, an OS (operating system) running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing. Cases are also included.

  Furthermore, after the program code read from the storage medium is written in the memory provided in the function expansion board inserted into the computer or the function expansion unit connected to the computer, the following program code is specified based on the instruction of the next program code. This includes cases where processing is performed. That is, the case where the CPU or the like provided in the extension board or the extension unit performs the part or all of the actual processing and the functions of the above-described embodiments are realized by the processing is included.

It is a block diagram which shows the structure of the digital video camera as an imaging device which concerns on the 1st and 2nd embodiment of this invention. It is a flowchart which shows the process for determining video and audio PTS at the time of the image recording start of a digital video camera. It is a figure explaining the discrepancy between the PTS interval of video and the PTS interval of audio. It is a figure explaining that the head PTS of video and the head PTS of audio differ for every stream. (A) is a diagram showing a video PES and an audio PES of the video camera A, and (b) is a diagram showing a video PES and an audio PES of the video camera B. FIG. 4 is a diagram for explaining that audio discontinuity occurs when two bitstreams in which video is synchronized are connected and edited. (A) is a diagram showing a video PES and an audio PES of the video camera A, (b) is a diagram showing a video PES and an audio PES of the video camera B, and (c) is a diagram showing an edit stream.

Explanation of symbols

103 Image compression / decompression unit 104 Audio compression / decompression unit 105 Multiplexing processing unit 106 External TTC detection unit (synchronization means)
109 CPU (control means)
111 Recording medium interface (recording means)
112 Recording medium (recording means)

Claims (7)

An image recording apparatus having a function of compressing and encoding an image signal and an audio signal and adding time information for reproducing the compressed image data and audio data for each compression encoding unit. And
Synchronizing means for synchronizing the image signal of the image recording device with the image signal of the external device based on an external synchronization signal input from the outside,
In the compression encoding operation in a state where the image signal of the image recording apparatus is synchronized with the image signal of the external device by the synchronization means, the time information of the compression encoding unit at the beginning of the image data and the beginning of the audio data A control means for controlling the difference between the time information of the compression coding unit to be set to a prescribed value and performing compression coding;
An image recording apparatus comprising:   The image recording apparatus according to claim 1, wherein the specified value is zero. A recording means for recording image data and audio data to which information relating to the recording time as the time information is added;
In the compression encoding operation in a state where the image signal of the image recording apparatus is synchronized with the image signal of the external device by the synchronization unit, the control unit is a value of information relating to the recording time at the start of the compression encoding operation. The image recording apparatus according to claim 1, wherein the difference between the time information of the first compression coding unit of the image data and the time information of the first compression coding unit of the audio data is changed according to .   4. The image recording apparatus according to claim 3, wherein the information related to the recording time is selected from a group including a recording time of TTC (Total Time Code), image data, and audio data.   An image pickup apparatus comprising the image recording apparatus according to claim 1. Control of an image recording apparatus having a function of compressing and encoding an image signal and an audio signal, and adding time information for reproducing the compressed image data and audio data for each compression encoding unit A method,
A synchronization step of synchronizing an image signal of the image recording device with an image signal of an external device based on an external synchronization signal input from the outside;
In the compression encoding operation in a state where the image signal of the image recording apparatus is synchronized with the image signal of the external device by the synchronization step, the time information of the first compression encoding unit of the image data and the start of the audio data A control step for controlling the difference from the time information of the compression coding unit to a predetermined value and performing compression coding;
A control method comprising: Control of an image recording apparatus having a function of compressing and encoding an image signal and an audio signal, and adding time information for reproducing the compressed image data and audio data for each compression encoding unit A program for causing a CPU to execute a method,
A synchronization module that synchronizes an image signal of the image recording device with an image signal of an external device based on an external synchronization signal input from the outside;
In the compression encoding operation in a state where the image signal of the image recording apparatus is synchronized with the image signal of the external device by the synchronization module, the time information of the first compression encoding unit of the image data and the beginning of the audio data A control module for controlling the difference between the time information of the compression coding unit to be set to a prescribed value and performing compression coding;
A program comprising:

Images