JP2015012544A - Image processing apparatus and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reproduce video images recording the situation where reproduction video images are appreciated, easily synchronously with recorded video images of persons appreciating the video images when viewing the reproduced video images later.SOLUTION: An image processing apparatus comprises: reproduction means for reproducing moving image data; output means for outputting the moving image data reproduced by the reproduction means to a display device; capture means for capturing moving image data; recording means for recording moving image data, and control means in which, when recording second moving image data by means of the recording means while reproducing first moving image data by means of the reproduction means, in the case where a plurality of frames in the second moving image data are captured by the capture means, respectively, additional information is generated for identifying a frame in the first moving image data being outputted to the display device and the recording means is controlled to record the additional information and the second moving image data together.

Description

  The present invention relates to an image processing apparatus and an image processing method, and more particularly to a technique suitable for use in shooting and recording a moving image during reproduction of the moving image.

  2. Description of the Related Art Conventionally, a system has been proposed in which a video that is used as a model for dancing or the like is played, a video that is danced in accordance with the video is shot with a video camera, and the shot video is combined with a playback video of the model and recorded (for example, Patent Document 1).

JP 2010-279012 A

However, in the conventional example, since the reproduced moving image and the photographed moving image are combined and recorded, there is a problem that the reproduced image and the photographed image are overlapped and are difficult to see.
Therefore, it is also conceivable to record the captured moving image separately from the reproduced moving image and to reproduce these in synchronization later.

However, if you fast-forward or rewind while playing a model video, you may not be able to easily synchronize the sample video and the video you shot later. is there.
In view of the above-mentioned problems, the present invention easily reproduces the reproduced video and the recorded video of the viewers when the video recorded to view the reproduced video is viewed later. The purpose is to be able to.

  An image processing apparatus according to the present invention records a moving image data, a reproducing unit that reproduces the moving image data, an output unit that outputs the moving image data reproduced by the reproducing unit to a display device, an acquisition unit that acquires the moving image data, and the moving image data. When the second moving image data is recorded by the recording unit while the first moving image data is reproduced by the recording unit and the reproducing unit, the plurality of frames are included in the plurality of frames in the second moving image data. Generating additional information for identifying a frame in the first moving image data that has been output to the display device when acquired by the acquiring means, and recording the additional information together with the second moving image data Control means for controlling the recording means.

According to the present invention, the video 1 data already recorded on the medium is reproduced and displayed on the display means. When a person who views the video 1 is photographed and recorded as video 2 data, the information specific to the video 1 and 2 when the video 1 is reproduced and the reproduced frame are recorded in the video 2 recording data. Corresponding information and playback mode of recorded frames were recorded. As a result, the video 2 is played back, and the video 1 corresponding to the video 2 is played back. Even if the person photographed in the video 2 plays the video 1 specially, what kind of display scene of the video 1 is viewed. Can be faithfully reproduced.
Further, according to another feature of the present invention, video 1 is set as different recorded data (separate file) with respect to video 2, so that the display method is free when two videos of video 2 and video 1 are reproduced. The degree can be increased.

1 is a block diagram illustrating a configuration example of an imaging apparatus according to a first embodiment of the present invention. 3 is a flowchart illustrating an operation procedure of the imaging apparatus according to the first embodiment. It is a figure for showing a 1st embodiment and explaining a data recording format. It is a figure for showing a 1st embodiment and explaining a data recording format. It is a figure for showing a 1st embodiment and explaining a data recording format. It is a figure for demonstrating the recording format of additional data. It is a figure for demonstrating the recording format of additional data. It is a block diagram which shows the 2nd Embodiment of this invention and shows the structural example of an imaging device. It is a flowchart explaining the operation | movement procedure of the imaging device of 2nd Embodiment. It is a figure for demonstrating the additional data recording format of 2nd Embodiment. It is a block diagram which shows the structural example of the reproducing | regenerating apparatus explaining embodiment.

(First embodiment)
A first embodiment of the image processing apparatus of the present invention will be described with reference to FIG. FIG. 1 is a block diagram illustrating a configuration example of an imaging apparatus 100 illustrating an embodiment of the present invention.
Reference numeral 101 in FIG. 1 denotes an imaging circuit that captures a subject and outputs moving image data. The camera signal processing circuit 102 performs camera signal processing such as white balance, γ processing, and color separation on the moving image data sent from the imaging circuit 101, and writes it to the memory 104 via the bus 119.

At the time of recording, the memory 104 stores captured moving image data, encoded moving image data, and the like. The memory 104 is also used as a storage memory for storing reproduced moving image data, audio data, and the like during reproduction.
The encoding circuit 103 reads the moving image data stored in the memory 104, encodes it using an encoding method such as MPEG-2, MPEG-4, AVC / H.264, and stores the encoded data in the memory 104.

  The audio input circuit 113 performs AD conversion on an audio signal input from a microphone (not shown), performs signal processing such as level conversion and noise removal on the digitized audio data, and fetches it into the memory 104. Alternatively, externally input audio data is taken into the memory 104.

The audio encoding circuit 114 encodes the audio data captured in the memory 104 as necessary, and stores the encoded audio data in the memory 104.
As will be described later, the recording information generation circuit 105 generates additional information when a moving image shot by the imaging circuit 101 is recorded during reproduction of a moving image from the recording medium 120.
When a predetermined amount of encoded data, audio encoded data, and header information is stored in the memory 104, the recording circuit 106 sets the switch 107 to the terminal a side and records moving image and audio streams on the recording medium 120.

In this embodiment, moving image data and audio data are recorded in a file format based on, for example, QuickTime (registered trademark).
As shown in FIG. 3A, the QuickTime file format is composed of basic units called atoms. The atom is composed of a size field 301a indicating its size (number of bytes), a type field 302a for identifying the type of atom by four character strings, and a data field 303a for storing actual data of atom.

FIG. 3B shows a file configuration example of video data composed of one metadata and one media data. Here, the metadata is information necessary for decoding and reproducing the media data.
In FIG. 3B, a File Type Box (ftyp) 311b stores information indicating a file type (QuickTime, MP4, etc.). Movie Box (moov) 312b is an atom that stores the entire metadata. The Media Data Box (mdat) 313b is an atom that stores the entire media data.

  FIG. 3C is a diagram showing an internal configuration of mdat. The inside of mdat is composed of a series of data units called “chunks”. In Fig. 3 (c), chunks of different media formats are alternately arranged in the order of audio chunk (1) AC1, video chunk (1) VC1, audio chunk (2) AC2, video chunk (2) VC2, ... An example of the configured configuration is shown. Further, each chunk is composed of a plurality of samples as a series of “samples” of the same media type.

  Here, a sample is a minimum encoded data unit that can be divided, and represents a data unit corresponding to one frame of video or audio data. FIG. 3C shows an example in which the video chunk (1) VC1 is composed of a plurality of samples as a series from the video sample (1) VS1 to the video sample (V1) VSV1. Similarly, the audio chunk (1) AC1 shows an example in which a plurality of samples are configured as a series from the audio sample (1) AS1 to the audio sample (A1) ASA1.

  FIG. 4 shows the configuration in the Movie Box (moov) atom. The Moov 312b is made up of Movie Header (mvhd) atom 401a and Track (trak) atoms 402a and 403a. Here, trak is an atom that stores information about each medium constituting the scene of the file.

FIG. 4 shows an example composed of Track (trak) 402a of video data and trak 403a of audio data.
As shown in FIG. 4, a large number of atoms are formed in each trak (the internal structure of the audio trak is the same as that of the video and is omitted).
Hereinafter, functions of Sample To Chunk (stsc) atom 404a, Sample Size (stsz) atom 405a, and Chunk Offset (stco) atom 406a included in each trak will be briefly described. The user data udta 407a will be described later with reference to FIG.

  The stsc 404a shown in FIG. 5A is an atom that stores sample information (number of samples and sample description number) included in each chunk of each medium, and is used when accessing desired data in the mdat. Only one stsc 404a exists in each of the aforementioned trak (402a, 403a).

  The stsz 405a shown in FIG. 5B is an atom that stores size (number of bytes) information of each sample of each medium, and is used when accessing desired data in the mdat. Only one stsz 405a exists in each of the above-described trak (402a, 403a).

The stco 406a shown in FIG. 5C is an atom that stores the file offset value of each chunk of each medium, and only one exists in each of the above-described trak (402a, 403a).
The stco 406a stores the total number of video chunks described above (in FIG. 5C, the total number of video chunks is assumed to be M) CT1, and the head offset values (CF1 to CFM) of M video chunks. .

  Here, the head offset value of the chunk means an offset value to the head byte of the head sample among samples constituting the chunk. That is, the start offset values (CF1 to CFM) of the video chunk described above are the start samples (VS1, VSV1 + 1,...) Of the video chunk (VC1, VC2,...) Described in FIG. ) Is the offset value to the first byte.

Using the information stored in the aforementioned stsc 404a, stsz 405a, and stco 406a, the position and size information of any sample of media in the file can be obtained.
That is, when a certain sample is designated by the sample number, first, the size of the designated sample can be known from the information in the stsz 405a. Next, it is possible to know what number chunk the specified sample belongs to by the above-described stsc 404a.

  The stsz 405a makes it possible to know the size S from the first sample to immediately before the designated sample in the chunk to which the designated sample belongs. Then, the start position T of the chunk to which the designated sample belongs can be acquired by stco 406a. By adding the above-described leading position T and the above-described size S from the leading position T to immediately before the designated sample, the leading position of the designated sample can be acquired.

  As described above, by analyzing stsz, stsc, and stco in each trak (402a, 403a), the position and size of an arbitrary sample on the file can be acquired, and arbitrary sample data can be read from the file. It becomes possible.

The reproduction circuit 108 selects the terminal b of the switch 107, reads out the encoded data recorded in the recording medium 120 (here, recorded using the QuickTime-based file format), and stores it in the memory 104. . The playback circuit 108 of the present embodiment can play back moving image data at a playback speed different from the normal playback speed.
The reproduction information extraction circuit 109 reads and analyzes the reproduction information (ftyp, moov atom) of the encoded data stored in the memory 104.

The decoding circuit 110 decodes the encoded data stored in the memory 104 and stores the decoded data in the memory 104.
When the audio data stored in the memory 104 is encoded, the audio decoding circuit 115 stores the decoded audio data in the memory 104 after decoding.

The display information extraction circuit 111 extracts frame information and playback mode of moving image data displayed via the display circuit 112.
The display circuit 112 sends the moving image data decoded by the decoding circuit 110 to a display device (not shown).

The audio output circuit 116 outputs the audio data stored in the memory 104 to a speaker (not shown) or the like.
The control circuit 117 includes a CPU and the like, and controls each unit of the imaging device 100. The operation input circuit 118 includes operation members such as various switches and buttons, and sends out user operation signals to the control circuit.

  The imaging apparatus 100 of the present embodiment configured as described above has a normal recording mode and a reproduction mode. In addition, the imaging apparatus 100 has a simultaneous recording mode in which the imaging circuit 101 performs shooting while reproducing moving image data from the recording medium 120 and records moving image data and audio data.

  When the user operates the operation input circuit 118 to set the normal recording mode, the control circuit 117 waits for a recording instruction from the user. When the start of recording is instructed, the control circuit 117 controls each unit and starts recording the captured moving image data and audio data as described above. Further, when there is an instruction to stop recording, the control circuit 117 stops the recording of the moving image and the sound.

  When the user operates the operation input circuit 118 and designates a reproduction mode for setting a normal reproduction mode for reproduction at a normal reproduction speed, the control circuit 117 waits for a reproduction instruction from the user. When the user designates a moving image to be reproduced from the moving images recorded on the recording medium 120 and gives an instruction to start reproduction, the control circuit 117 controls each unit to start reproducing the moving image designated from the recording medium 120. . When there is an instruction to stop playback, the control circuit 117 stops playback of the moving image.

Next, the simultaneous recording mode will be described.
In the simultaneous recording mode, as described above, the moving image recorded by the imaging circuit 101 is recorded while reproducing the moving image recorded on the recording medium 120. In the following description, it is assumed that the moving image and audio reproduced in the simultaneous recording mode are video 1 and the moving image and audio simultaneously recorded are video 2.

  When the user operates the operation input circuit 118 to set the simultaneous recording mode, the control circuit 117 displays information prompting an instruction to select a moving image to be reproduced as the video 1 on the display circuit 112. The user operates the operation input circuit 118 to select a moving image file to be reproduced as the video 1 from the moving image files recorded on the recording medium 120. When the user selects a moving image file to be played back as video 1, and starts playback, simultaneous recording is started.

FIG. 2 is a flowchart illustrating processing of the imaging apparatus 100 in the simultaneous recording mode. The processing in FIG. 2 is executed by the control circuit 117 controlling each unit.
First, in S201, the control circuit 117 controls the imaging circuit 101, the camera signal processing circuit 102, and the audio input circuit 113, and starts photographing. The camera signal processing circuit 102 processes the moving image data of the captured video 2 and stores it in the memory 104.

  In S <b> 202, the control circuit 117 reads the encoded data of the video 1 from the recording medium 120 via the reproduction circuit 108 and stores it in the memory 104. The reproduction information extraction circuit 109 extracts reproduction information (fytp, moov, trak atom) of the encoded data from the encoded data of the video 1 (video 1 data) stored in the memory 104. Also, the file name, shooting date / time, file size or encoded data size of video 1 is acquired.

In S203, the control circuit 117 detects the reproduction mode. Here, instructions for fast forward, slow, and playback stop are detected.
In S204, the control circuit 117 observes whether or not the reproduction mode detected in S203 has changed. Here, since the operator has selected and played back video 1, the normal playback mode is maintained, so no is selected, and the process proceeds to S207.

  In step S207, the control circuit 117 instructs the reproduction information extraction circuit 109 to perform normal reproduction, and the reproduction information extraction circuit 109 calculates and acquires the position of the beginning of the encoded video and audio data in the video 1 file, and performs control. Notify the circuit 117.

  In S208, the control circuit 117 obtains the encoded data of video 1 from the memory 104 if the encoded data at the beginning of the encoded data of video 1 is stored in the memory 104, and from the recording medium 120 otherwise. Read and store in memory 104.

In step S209, the decoding circuit 110 and the audio decoding circuit 115 read and decode the encoded data of the video 1 stored in the memory 104, and store the decoded image data and audio data of the video 1 in the memory 104.
In step S <b> 210, the control circuit 117 determines whether the image data and audio data of the decoded video 1 stored in the memory 104 can be displayed to the operator. If it cannot be displayed yet, the process returns to S203.

  If it is determined that it can be displayed, the process proceeds to S211 to display the image data of the video 1 via the display circuit 112, and the audio data is reproduced via the audio output circuit 116 in the case of normal reproduction.

  In S212, the control circuit 117 calculates and calculates the frame number m0 of the displayed video 1 from the playback information (the video chunk in the trak atom and its sample information) extracted by the playback information extraction circuit 109. Further, a playback mode acquisition process is performed to acquire the playback mode fw1 (now normal playback) when the frame is decoded.

Next, an image frame of video 2 captured by the imaging circuit 101 at the time when the frame number m0 of video 1 is displayed in S213 is acquired.
Next, in S214, the control circuit 117 generates the frame number 1 of the image frame of the video 2 acquired in S213.
In step S215, the video 2 data of the video frame of video 2 acquired in step S213 is encoded by the encoding circuit 103 and stored in the memory 104. Also, encoding of audio data is started from a position synchronized with an image frame encoded by the audio encoding circuit 114, and the encoded audio data is stored in the memory 104.

  Next, in S216, the control circuit 117 associates the frame number m0 of the video 1 acquired in S212 and the playback mode fw1 with the frame number 1 of the video 2 acquired in S214, as shown in FIG. Generate additional data. That is, associating the frame image of video 2 with the frame image of video 1 as in frame number 1 of video 2, frame number of video 1: m0, and playback mode of video 1: normal playback (fw1). I do. In addition, the file name, shooting date and time, file size, or encoded data size of the video 1 acquired in S202 is also generated as additional data for recording.

  For example, as shown in FIG. 7A, user data udta atom is prepared, and the file name, shooting date, file size, or encoded data size of video 1 is prepared as user 1 atom in udta atom. . This is used to check whether or not the corresponding video 1 exists or has been edited and changed when the video 2 is reproduced after the video 2 is recorded.

  In step S217, the control circuit 117 determines whether to record on the recording medium 120. If it is determined that a predetermined amount of the image 2 and audio encoded data of the video 2 is stored in the memory 104, yes is selected, and the process proceeds to S218.

In S218, as shown in FIG. 3C, recording of the encoded video and audio data to the recording medium 120 via the recording circuit 106 is started as an mdat atom. Then, after recording, the process returns to S203.
If it is determined in S217 that the predetermined amount has not yet been reached, the process returns to S203.
In S203, the reproduction mode is detected again from the operation information input from the operation input circuit 118, and the process proceeds to S204.

In S204, reproduction mode detection is performed to determine whether or not there has been a change in the previous and reproduction modes. If there is no change in the playback mode, the process proceeds to S207, and the decoding position in the file of the next frame is calculated as the same normal playback as the previous time.
For normal playback (fw1), video 1 is played back as frame number m0 followed by m0 + 1, m0 + 2, m0 + 3,. Then, in the additional data generation process of S216, as shown in FIG. 6, each frame of video 2 is captured as additional data for recording in association with frame numbers 2, 3, 4,. Number them in order and add them. The frame of video 1 associated with each frame of video 2 is designated by an offset number from the first frame of video 1.

If it is determined in S204 that the playback mode has been changed, the process proceeds to S205.
In S205, it is determined whether or not the playback mode is the playback stop of the video 1. If the reproduction is not stopped, the process proceeds to S206.
In S206, the decoding position in the next file is calculated according to the changed playback mode of the video 1. For example, suppose that video 1 is played up to frame number m0 + n0-1, and then 6x playback (fw6) is requested in the forward direction. In this case, the control circuit 117 recalculates the image data position in the video 1 file of frame number m0 + n0 + 6 from the reproduction information (information in the trak atom) in the reproduction information extraction circuit 109. Then, the image data position in the video 1 file is obtained again and supplied to the decoding circuit 110.

  In this case, as shown in FIG. 6, the additional data generated in S216 is the frame number of video 1 with respect to frame numbers n0, n0 + 1, n0 + 2, n0 + 3,. m0 + n0-1, m0 + n0 + 6, m0 + n0 + 12, m0 + n0 + 18, and so on. As a result, the playback mode is forward 6 × speed playback (fw6).

  For example, a case will be described in which the operator performs reverse reproduction (for example, −3 times speed) at the time of reproduction up to frame number m1 of video 1. In this case, as described above, the additional data generated in S216 is as shown in FIG. That is, for frame number n1, n1 + 1, n1 + 2, n1 + 3,... Of image 2, the frame numbers of image 1 are m1, m1-3, m1-6, m1-9,.・ ・. In the playback mode, the playback direction is the reverse direction—3 × speed (rw3).

  In FIG. 6, normal playback is then resumed (n2, n2 + 1,..., M2, m2 + 1,...), And forward slow playback (+ 1/4 × speed) starts from frame number m4 of video 1. . In this case, if the frame number m4 of image 1 is set to the frame number n3 of image 2, m4 + 1 is applied to n3 + 1 to n3 + 4 of image 2, and n3 + 5 to n3 + 8 of image 2 is applied. Thus, the frame number of video 1 is m4 + 2. Playback mode is forward 1 / 4x (fsl 1/4).

In slow playback, the frame number of video 1 is the same for different frame numbers of video 2. For this reason, when the video 1 corresponding to the video 2 is played back, if only the corresponding frame number is viewed, it cannot be distinguished whether it is a pause, skip playback state, or slow playback. Therefore, knowing that the playback mode is slow playback from the playback mode of video 1 when recording video 2, the same playback state as when recording can be reproduced.
Thereafter, in FIG. 6, the normal playback is resumed at the frame number m4 + 10 of the video 1.

  When the control circuit 117 detects a change in the playback mode in S204 and detects the stop of playback of the video 1 in S205, the control circuit 117 ends the recording of the video 2. In that case, the user data udta atom shown in FIG. 7A is generated and recorded in the video trak for the moov atom of the video 2 in FIG.

  The user data (udta atom) information includes, as shown in FIG. 7A, video 2: file name, shooting date and time, and file size (or encoded data size). Corresponding video 1: file name, shooting date and time, and file size (or encoded data size) are user 1 information (user 1 atom). Then, the reproduction information of the video 1 corresponding to the recorded video 2 shown in FIG. 6 is recorded as user 2 information (user 2 atom).

Next, a playback device that plays back moving image data recorded as video 2 in the simultaneous recording mode in synchronization with the moving image of video 1 that was being played back at the time of recording will be described.
FIG. 11 is a block diagram illustrating a configuration example of a playback device. Note that the configuration shown in FIG. 11 can be configured to be included in the imaging apparatus shown in FIG.

The video 2 file recorded on the recording medium 120 is reproduced via the reproduction circuit 1101 and stored in the memory 1103. The reproduction information extraction circuit 1102 extracts reproduction information (ftyp, moov atom) from the reproduction data of the video 2 stored in the memory 1103.
From the reproduction information extracted by the reproduction information extraction circuit 1102, the control circuit 1111 searches for user additional information (udta atom).

Here, in the case of the file recorded in the first embodiment, since there is user additional information shown in FIG. 7A, this information is read. Then, it is confirmed (from the file name) whether or not the corresponding video 1 file is recorded on the recording medium 120.
If the video 1 file is not recorded, the display circuit 1108 or 1109 notifies the fact and only the video 2 stream is reproduced.

  When the file of video 1 is recorded on the recording medium 120, it is compared whether the file size of video 1 (or the size of the encoded data) has not changed since the video 2 was recorded. This can be understood by comparing with the file size registered in the file system. When the encoded data size is recorded, the video 1 file is read from the recording medium 120, and the encoded data size is calculated from the trak information in the moov atom and compared.

Here, if the sizes do not match, the video 1 has been edited or the like and has changed from when the video 2 was recorded, so that only the video 2 is reproduced.
If the sizes match, the video 1 file is also read into the memory 1103 via the playback circuit 1101 to obtain playback information (moov atom).

  The control circuit 1111 extracts user additional information (table information in FIG. 6) recorded in the moov atom of the video 2. Then, in accordance with the reproduction of the video 2, the image frame of the video 2 and the corresponding image frame of the video 1 are read from the recording medium 120 to the memory 1103 via the reproduction circuit.

The image data of video 2 read to the memory 1103 is decoded by the decoding circuit 1 (1104), the image data of video 1 is decoded by the decoding circuit 2 (1105), and the decoded frames of the video 1 and 2 are obtained. Store in the memory 1103. As the audio data, the audio data of the video 2 is read, decoded by the audio decoding circuit 1106, and stored in the memory 1103.
When the decoded image data of video 2, audio data, and video data of video 1 are stored in a predetermined amount in the memory 1103, the control circuit 1111 starts reproduction.

  Here, since the viewer can select the playback method, the viewer instructs the playback method via the operation input circuit 1112. For example, it is assumed that the display is selected to be mixed (displayed by selecting how to mix video 1 and video 2) on one display device. In this case, the control circuit 1111 mixes the decoded image data of the video 1 and video 2 stored in the memory 1103 by the mixing circuit 1107 and displays them on a display device (not shown) via the display circuit 1 (1108). . The decoded audio data stored in the memory 1103 is output via the audio output circuit 1110. When the viewer designates to display video 1 and video 2 on separate display devices, the control circuit 1111 sends the image data of video 1 and video 2 to the display circuit 1 (1108) and the display circuit 2 (1109), respectively. Output.

  As described above, since each frame of video 1 is associated with playback of video 2, playback of video 1 that is exactly the same as when video 2 was recorded can be performed. Also, since the playback mode of video 1 is recorded, not only normal playback that plays back at the normal playback speed but also slow playback, fast forward playback, frame advance playback, etc. can be reproduced and played back. Further, even when the video 2 is specially reproduced, the video 1 can be decoded and reproduced.

  In the present embodiment, as described above, the case where the video 1 already recorded on the recording medium is reproduced, the person being viewed is photographed and recorded as the video 2 is recorded. In such a case, the unique information (file name, shooting date / time, file size, encoded data size, etc.) of video 1 and video 2 when video 1 is played back in the recorded data of video 2 and the played frame Corresponding information and playback mode of recorded frames were recorded. As a result, it is possible to generate information related to the playback speed of the video 1 being played back during the recording of the video 2. Therefore, even when the video 2 is played back, and the video 1 corresponding to the video 2 is played back, and the person photographed in the video 2 is playing back the video 1 specially, what kind of display scene of the video 1 is seen and reacted You can now faithfully reproduce what you are doing.

In addition, since the video 1 is recorded as different recording data (separate file) with respect to the video 2, the degree of freedom of display is increased when the two videos of the video 2 and the video 1 are reproduced.
In addition, information unique to the video 1 and the video 2 when the video 1 is reproduced (file name, shooting date, file size, encoded data size, etc.) is recorded. Thereby, it is possible to know whether the video 1 exists when the video 2 is reproduced or whether the video 1 and the video 2 have been changed (edited) since the video 2 was recorded. It can be judged whether it can be reproduced or not.
Further, the correspondence information between the reproduced frame and the photographed frame and the reproduction mode information shown in FIG. 6 may hold only the change point as shown in FIG. 7B.

(Second Embodiment)
FIG. 8 is a block diagram illustrating a configuration example of an imaging apparatus 800 according to the second embodiment of this invention. In FIG. 8, the same reference numerals are given to the same functions as those in the first embodiment. FIG. 9 is a flowchart for explaining the processing sequence of the imaging apparatus 800. The same sequence portions as those in the first embodiment are denoted by the same reference numerals as the step numbers in the flowchart of FIG.

Hereinafter, only different points from the first embodiment will be described.
When the image data of video 1 is displayed in S211, the process proceeds to S901.
In step S <b> 901, the control circuit 117 encodes the image data of the displayed video 1 in the second encoding circuit 801 and stores the encoded data in the memory 104.

  Next, in S213, image data of video 2 (captured by the imaging circuit 101) captured at the same timing as video 1 displayed in S211 is acquired, and in S215, image data and accompanying audio data are acquired. Is encoded.

  In step S902, additional data and reproduction information are generated. Here, reproduction information is generated so that the image data encoded in S901 and S215 have the same frame number (the same number of frames from the start of recording). It is stored in the mdia atom in the trak atom 1001a in FIG. Here, trak atom 1001a is reproduction information of video 2. The encoded data of video 1 reproduced and displayed is encoded as video 3 and recorded in the trak atom 1003a as reproduction information of video 3 data. A trak atom 1002a is reproduction information of encoded audio data.

  In S902, as shown in FIG. 10B, the file name, shooting date / time, encoded data size (or file size) of video 2, and shooting date / time (recording date / time) of video 3 and encoded data size ( File size) as additional data. And it records on udta atom 1004a and 1005a as user data.

  As described above, in the present embodiment, the video 1 that has already been recorded on the recording medium is reproduced, and the person being viewed is photographed and recorded as the video 2. In such a case, the frame of the video 1 displayed at the timing of each frame of the video 2 to be shot is captured and encoded, and the video 3 is recorded together with the video 2. As a result, when the video 2 is reproduced, the video 3 having the same frame number is reproduced from the reproduction information, so that the reproduction state of the video 1 viewed by the viewer when the video 2 is recorded can be reproduced.

  In addition, when the viewer is fast-forwarding video 1 and performing special playback such as slow motion, the display image is recorded as video 3, so that the playback status of video 1 that the viewer was watching is displayed. It can be reproduced completely.

Since the display video of video 1 is recorded in video 2 as video 3, even if video 1 is moved to another recording medium or edited, the viewing state at the time of recording video 2 is reproduced. it can.
In addition, since the video 3 is recorded as different recorded video data with respect to the video 2, the degree of freedom in display is increased when the two videos, the video 2 and the video 3, are reproduced.
Here, even if the video 3 is recorded as a separate file from the video 2, the same effect can be obtained.

  In the second embodiment, even when the display data of video 1 is encoded as video 3 data and recorded in the file of video 2 (or as a separate file), the playback information of video 2 and the playback information of video 3 are Since it is associated one-to-one, it can be played back as described above.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (computer program) that implements the functions of the above-described embodiments is supplied to a system or apparatus via a network or various computer-readable storage media. Then, the computer (or CPU, MPU, etc.) of the system or apparatus reads out and executes the program.

DESCRIPTION OF SYMBOLS 100 Imaging device 101 Imaging circuit 102 Camera signal processing circuit 103 Encoding circuit 104 Memory 105 Recording information generation circuit 106 Recording circuit 107 Switch 108 Playback circuit 109 Playback information extraction circuit 110 Decoding circuit 111 Display information extraction circuit 112 Display circuit 113 Audio input Circuit 114 audio encoding circuit 115 audio decoding circuit 116 audio output circuit 117 control circuit 118 operation input circuit 119 bus 120 recording medium

Claims (9)

Playback means for playing back video data;
Output means for outputting the moving image data reproduced by the reproducing means to a display device;
An acquisition means for acquiring video data;
Recording means for recording video data;
When the second moving image data is recorded by the recording unit while the first moving image data is reproduced by the reproducing unit, the plurality of frames of the plurality of frames in the second moving image data are respectively obtained by the acquiring unit. Generating additional information for identifying a frame in the first moving image data that has been output to the display device, and recording the additional information together with the second moving image data. An image processing apparatus comprising control means for controlling recording means.   The said control means produces | generates the said additional information containing the information for identifying the said 1st moving image data reproduced when recording the said 2nd moving image data. Image processing apparatus.   The reproduction means can reproduce moving image data at a reproduction speed different from the normal reproduction speed, and the control means reproduces a plurality of frames in the second moving image data during recording of the second moving image data. 2. The recording apparatus according to claim 1, further comprising: generating information related to a reproduction speed of the first moving image data being recorded, and controlling the recording unit to record the additional information including the information related to the reproduction speed. Image processing device.   The reproduction unit can change the reproduction direction of the moving image data between a forward direction and a reverse direction, and the control unit performs the second moving image data on a plurality of frames in the second moving image data. 2. The recording unit is controlled to generate information relating to a reproduction direction of the first moving image data being reproduced during recording, and to record the additional information including information relating to the reproduction direction. The image processing apparatus according to 1.   The image processing apparatus according to claim 1, wherein the reproduction unit reproduces the first moving image data from a recording medium, and the recording unit records the second moving image data on the recording medium. .   The control means starts recording the second moving image data in response to the reproduction start of the first moving image data by the reproducing means, and the second moving image in response to the reproduction stop of the first moving image data. The image processing apparatus according to claim 1, wherein the recording unit is controlled to stop data recording. Playback means for playing back video data;
Output means for outputting the moving image data reproduced by the reproducing means to a display device;
An acquisition means for acquiring video data;
Recording means for recording video data;
When the second moving image data is recorded by the recording unit while the first moving image data is reproduced by the reproducing unit, the third moving image data is generated using the first moving image data and the third moving image data is generated. Additional information for associating the second moving image data with the second moving image data is generated, and the recording means is controlled to record the third moving image data and the additional information together with the second moving image data. An image processing apparatus comprising control means. A playback process for playing back video data;
An output step of outputting the video data reproduced in the reproduction step to a display device;
An acquisition process for acquiring video data;
A recording process for recording video data;
When the second moving image data is recorded in the recording step while the first moving image data is reproduced in the reproducing step, the plurality of frames of the plurality of frames in the second moving image data are respectively obtained in the obtaining step. Generating additional information for identifying a frame in the first moving image data that has been output to the display device, and recording the additional information together with the second moving image data. An image processing method comprising: a control process for controlling a recording process. A playback process for playing back video data;
An output step of outputting the video data reproduced in the reproduction step to a display device;
An acquisition process for acquiring video data;
A recording process for recording video data;
When recording the second moving image data in the recording step while reproducing the first moving image data in the reproducing step, the third moving image data is generated using the first moving image data and the third moving image data is generated. Additional information for associating the second moving image data with the second moving image data is generated, and the recording step is controlled so as to record the third moving image data and the additional information together with the second moving image data. An image processing method comprising a control step.

Images