JP5930807B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus, and more particularly to shooting and recording of moving images.

  2. Description of the Related Art Conventionally, an imaging device such as a digital camera that captures a moving image and records it on a recording medium such as a memory card is known. In recent years, consumer digital cameras that can shoot movies with a large number of pixels have also appeared. In addition, the frame rate (the number of frames per unit time) of a moving image to be shot is high.

JP 2005-101835 A

  When the number of pixels and the frame rate of a moving image to be photographed increase, the amount of data to be processed increases, so that it is necessary to process the moving image data faster than before.

  In order to increase the processing capacity of moving image data, a processing circuit such as a memory having a large storage capacity and high speed access and a microcomputer capable of processing moving image data at a higher speed is required.

  However, using such a high-performance memory or microcomputer leads to an increase in circuit scale and power consumption.

  In particular, in a consumer digital camera, there are demands for limiting size and cost, or minimizing power consumption, and there are cases where a high-performance memory or microcomputer cannot be used.

  For this reason, there is a problem that it is impossible to shoot a moving image with a large number of pixels and frame rate.

  An object of the present invention is to solve such a problem and to provide an apparatus capable of processing a moving image having a large number of pixels and a high frame rate while suppressing a circuit scale and power consumption.

  The imaging apparatus of the present invention is a circuit having an imaging unit, a first CPU, and a first communication unit, acquires moving image data from the imaging unit, processes the acquired moving image data, and processes the first moving image data. A moving image data of a frame different from the frame acquired by the first processing circuit, the circuit having a first processing circuit that is output by one communication unit, a second CPU, and a second communication unit. A second processing circuit that records the processed moving image data from the imaging unit and the moving image data output from the first processing circuit by the first communication unit on a recording medium; The second CPU includes the first processing circuit so that the first processing circuit obtains and processes moving image data of a frame different from the frame processed by the second processing circuit from the imaging unit. By processing circuit Control information for controlling processing timing is transmitted to the first processing circuit by the second communication means, and the first CPU is based on the control information transmitted from the second communication means. The timing at which the first processing circuit processes the moving image data is controlled, and the first processing circuit is configured to capture the moving image data of the frame processed by the first processing circuit in accordance with the controlled timing. Get from.

  According to the present invention, it is possible to process a moving image with a large number of pixels and a high frame rate while suppressing the circuit scale and power consumption.

It is a block diagram which shows the structure of the imaging device in embodiment. It is a figure which shows the structure of the moving image file recorded. 5 is a flowchart showing processing at the time of recording by the processing circuit 200. 4 is a flowchart showing processing at the time of recording by the processing circuit 100. 3 is a flowchart showing an encoding process by a processing circuit 200. 3 is a flowchart showing an encoding process by a processing circuit 100. It is a figure which shows the storage area of the compressed moving image data. 5 is a flowchart showing processing when recording is stopped by a processing circuit 200. 4 is a flowchart showing processing when recording is stopped by the processing circuit 100. 4 is a flowchart showing a moving image file recording process by the processing circuit 200. It is a figure which shows the process timing at the time of the moving image data recording by the processing circuits 100 and 200. FIG. It is a figure which shows the encoding system of the moving image data encoded. 4 is a flowchart showing processing during reproduction by the processing circuit 200. 4 is a flowchart illustrating a moving image data decoding process performed by a processing circuit 200. 4 is a flowchart illustrating moving image data decoding processing by the processing circuit 100. 5 is a flowchart showing display processing by a processing circuit 200. It is a figure which shows the processing timing at the time of reproduction | regeneration of the moving image data by the processing circuits 100 and 200. FIG.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. FIG. 1 is a block diagram illustrating an example of a configuration of an imaging apparatus 500 according to an embodiment of the present invention.

  The imaging apparatus 500 includes two processing circuits 100 (first processing circuit) and a processing circuit 200 (second processing circuit). In the present embodiment, each of these two processing circuits 100 and 200 is constituted by a single microcomputer. In the present embodiment, the processing circuits 100 and 200 are each configured as one integrated circuit (LSI).

  In addition, a dedicated bus 300 for performing communication between the two processing circuits 100 and 200 is provided. The processing circuits 100 and 200 can acquire moving image data from the imaging unit 400 independently of each other. Each of the processing circuits 100 and 200 can process moving image data acquired from the imaging unit 400.

  Next, the configuration of the processing circuits 100 and 200 will be described.

  The processing circuit 100 includes an image processing unit 101, a first CPU (Central Processing Unit) 102, a memory 103, a clock control unit 104, a codec unit 105, a resizing unit 106, a communication unit 107, and a bus 108. In this embodiment, an SDRAM is used as the memory 103. Further, although the memory 103 is built in the processing circuit 100, the memory 103 can be provided outside the processing circuit 100.

  The CPU 102 controls the overall operation of the camera 500 in accordance with a computer program (software) stored in the memory 103. The memory 103 functions as a work area for the CPU 102. The work area of the CPU 102 is not limited to the memory 103, and may be an external recording device such as a hard disk drive. The image processing unit 101 performs image processing such as pixel interpolation processing and color conversion processing on the moving image data acquired from the imaging unit 400. The image processing unit 101 converts moving image data in the RGB color space acquired from the imaging unit 400 into a data format in the YUV color space.

  The imaging unit 400 and the image processing unit 101 are controlled by the CPU 102 to perform autofocus (AF) processing and automatic exposure control (AE) processing. When the CPU 102 instructs to start AF processing and AE processing, the image processing unit 101 performs arithmetic processing using moving image data acquired from the imaging unit 400. The imaging unit 400 and the image processing unit 101 perform TTL (through-the-lens) AF processing and AE processing based on the calculation result.

  When the CPU 102 instructs to start moving image shooting, the imaging unit 400 and the image processing unit 101 execute shooting processing including processing such as exposure processing and development processing. The imaging unit 400 includes an imaging element such as a CCD or CMOS, an AD converter, and the like. The imaging unit 400 converts an analog signal obtained by the imaging element into digital data and outputs the digital data. The moving image data acquired from the imaging unit 400 is stored in the memory 102 as YUV video data. The moving image data stored in the memory 103 is encoded by the codec unit 105 (first encoding unit, first decoding unit), and the data amount of the moving image data is compressed.

  At the time of shooting, the codec unit 105 converts the moving image data acquired from the imaging unit 400 to H.264. It is encoded by a known encoding method such as H.264 (MPEG4 AVC). Further, the codec unit 105 decodes the moving image data read from the recording medium 212 at the time of reproduction as described later. The clock control unit 104 generates various operation clocks for controlling the processing timing in the processing circuit 100. The timing at which moving image data is acquired from the image capturing unit 400 during moving image shooting is controlled by the timing signal from the clock control unit 104.

  The resizing unit 106 (first resizing unit) changes the number of pixels (size) of each frame of the moving image data acquired from the imaging unit 400 to a necessary image size at the time of shooting. The communication unit 107 (first communication unit) transmits and receives moving image data and other necessary commands to and from the processing circuit 200. The communication unit 107 includes a data receiving unit 107a for receiving moving image data, a data transmitting unit 107b for transmitting moving image data, and a message communication unit 107c for transmitting messages such as control commands. Communication in the communication unit 107 is performed via a dedicated bus 300. Each communication can be performed independently.

  In the present embodiment, as will be described later, the moving image data processed by the processing circuit 100 during shooting and the moving image data processed by the processing circuit 100 during reproduction are transmitted to the processing circuit 200 by the communication unit 107.

  The processing circuit 200 includes an image processing unit 201, a second CPU 202, a memory 203, a clock control unit 204, a codec unit 205, a resizing unit 206, a communication unit 207, a display unit 208, an audio control unit 209, a recording unit reproduction unit 210, An operation unit 211 and a bus 213 are included. In the present embodiment, an SDRAM is used as the memory 203. Further, although the memory 203 is built in the processing circuit 100, the memory 203 can be provided outside the processing circuit 200. Each block of the image processing unit 201, the second CPU 202, the memory 203, the clock control unit 204, the codec unit 205, the resizing unit 206, and the communication unit 207 (second communication unit) has the same function as each block in the processing circuit 100. have.

  At the time of shooting, the processing circuit 200 acquires moving image data from the imaging unit 400 and performs encoding processing by the codec unit 205 (second encoding unit, second decoding unit). During recording, the recording / playback unit 210 stores the moving image data encoded by the codec unit 205, the moving image data transmitted from the processing circuit 100, and the audio data generated by the audio processing unit 209 in the recording medium 212. Record. The recording medium 212 is a random access medium such as a memory card. Further, in the present embodiment, mounting and discharging can be easily performed by a mounting and discharging mechanism (not shown). Further, the recording medium 212 may be built in the imaging apparatus 500.

  The CPU 202 controls the recording of moving image data and audio data according to the user's settings for the moving image frame rate and the presence / absence of audio specified by the operation unit 211.

  Further, the recording / playback unit 210 reads the moving image data and audio data of the scene selected by the user from the recording medium 212 as described later during playback. The codec unit 205 decodes the moving image data and audio data read from the recording medium 212 during reproduction.

  The resizing unit 206 (second resizing unit) changes the image size of the moving image data acquired from the imaging unit 400 according to the size of the display unit 208 and stores it in the memory 203 at the time of shooting. Then, the resized data is supplied to the display unit 208 and displayed. At the time of reproduction, the resizing unit 206 changes the size of the reproduced moving image data in accordance with the size of the display unit 208 and stores it in the memory 203. Then, the resized data is supplied to the display unit 208 and displayed.

  The reproduced audio data is supplied to the audio control unit 209 and output. The display unit 208 is configured by a display such as a liquid crystal display. The display unit 208 displays various necessary information in addition to the captured moving image and the reproduced moving image. The CPU 202 generates information to be displayed on the display unit 208 and sends it to the display unit 208.

  The operation unit 211 functions as a user interface for operating the imaging apparatus 500. The operation unit 211 includes a power button, a mode change button, a shutter button, a cross button, a menu button, and the like for operating the imaging apparatus 500, and each button includes a switch, a touch panel, and the like. The CPU 202 controls the imaging apparatus 500 in accordance with a user instruction input via the operation unit 211. When a button on the operation unit 211 is operated by the user, an operation signal corresponding to each button is input from the operation unit 211 to the CPU 202. The CPU 202 analyzes the operation signal input from the operation unit 211 and determines processing corresponding to the operation signal according to the analysis result. The CPU 202 controls each unit of the imaging apparatus 500 so as to execute processing corresponding to the operation signal input from the operation unit 211.

  Next, the format of moving image data recorded in the imaging apparatus 500 will be described. In the present embodiment, the captured moving image and audio data are recorded on the recording medium 212 as a moving image file. In the present embodiment, a moving image file is recorded in accordance with the MOV format that is a general-purpose file format.

  FIG. 2 shows the structure of the MOV file. As shown by 220 in FIG. 2, the MOV file includes an mdat atom 222 in which stream data including encoded moving image data and audio data is stored, and a moov atom 221 in which management information related to the stream data is stored. It is composed of The mdat atom 222 is further composed of a plurality of chunks (chunk cN) as indicated by 225. Each chunk is composed of a plurality of samples (sample sN) as indicated by 226. For example, each sample corresponds to each frame of moving image data encoded.

  As shown in FIG. 2C, the moov atom 221 includes an mvhd atom 223 that is header information in which the creation date and time are recorded, and a trak atom 224 in which management information is stored. The trak atom 224 stores a stsz atom 228 that stores information on the size of each sample. The trak atom 224 stores a stsc atom 229 that stores information on the number of samples in each chunk. The trak atom 224 stores a stco atom 230 that stores information on an offset value (number of bytes) from the beginning of the file to each chunk of the mdat atom 222.

  The amount of data stored in the stsz atom 228, the stsc atom 229, and the stco atom 230 increases with the amount of recorded moving image and audio data, that is, the recording time. For example, when an image of 30 frames per second is stored in one chunk every 15 frames, data of about 1 megabyte is obtained in 2 hours. Further, the contents of the management information stored in the moov atom 221 are not determined until recording is completed. Also, since the size of the moov atom 221 increases according to the recording time, the size of the moov atom 221 is not fixed until the recording is completed. Therefore, the mdat atom 222 is arranged at the head of the file, and the moov atom 221 is arranged behind the mdat atom 222 at the end of recording.

  In the case of playing a moving image file in the MOV format, the moov atom 221 is read first, and each chunk can be accessed from the management information stored in the moov atom 221. For this reason, it is desirable to record in a structure that facilitates access to the moov atom 221 by placing the moov atom 221 at the beginning of the file, rather than placing the moov atom 221 behind the mdat atom 222.

  Next, processing at the time of recording in the imaging apparatus 500 will be described.

  First, the frames processed by the processing circuits 100 and 200 will be described. As described above, in this embodiment, moving image data is converted to H.264. The encoding is performed according to the H.264 system. H. H.264 uses three encoding schemes: intra-frame prediction encoding, forward prediction inter-frame prediction encoding, and bidirectional prediction inter-frame prediction encoding. Here, a frame to be encoded by intra-frame prediction encoding is an I frame, a frame to be encoded by forward prediction inter-frame prediction encoding is a P frame, and a frame to be encoded by bidirectional inter-frame prediction encoding is a B frame. Call.

  In the present embodiment, the two systems of moving image data encoded by the processing circuits 100 and 200 are recorded on the recording medium 212 as one system of moving image data. Thus, in the present embodiment, each processing circuit 100, 200 encodes each frame of moving image data by intraframe prediction encoding. Thereby, it is not necessary to transmit / receive reference frame information between the processing circuits 100 and 200 at the time of encoding. However, not only intra-frame prediction but also inter-frame prediction coding can be used. In that case, it is necessary to transmit / receive information on the reference frame for inter-frame prediction between the processing circuits 100 and 200.

  In FIG. 12, 1201 represents moving image data generated by the processing circuit 200, 1202 represents moving image data generated by the processing circuit 100, and 1203 represents moving image data recorded on the recording medium 212. As described above, the processing circuits 100 and 200 alternately acquire moving image data generated by the imaging unit 400 every other frame and perform encoding processing. That is, the processing circuit 200 processes moving image data of a frame different from the frame processed by the processing circuit 100. Therefore, it is possible to record a moving image having a frame rate that is twice the frame rate of moving image data that can be processed by each of the processing circuits 100 and 200.

  Further, the processing circuit 200 encodes each frame of the moving image data as an IDR (Instantaneous Decoding Refresh) -I frame. H. In H.264 encoding, inter-frame predictive encoding can be performed by skipping I frames. On the other hand, inter-frame prediction that skips IDR-I frames is prohibited. On the other hand, in the processing circuit 100, the first frame after the recording start instruction is encoded as an IDR-I frame, and the second and subsequent frames are encoded as normal I frames. It is necessary to add an ID called idr_pic_id to each IDR-I frame. H. In H.264, the same value cannot be added to idr_pic_id of adjacent IDR-I frames. For example, when the processing circuits 100 and 200 are all encoded from the beginning as an IDR-I frame and the same value is added as idr_pic_id, the moving image data that are combined into one is adjacent. The idr_pic_id value may be the same in the IDR-I frame.

  The data of idr_pic_id is Golomb coded and further described with an indefinite length. Therefore, it is difficult to change the value of idr_pic_id after the moving image data generated by the processing circuits 100 and 200 are combined. Therefore, in this embodiment, the moving image data generated by the processing circuit 100 is encoded as 2 frames and later as I frames, and the idr_pic_id is changed so as not to record the first IDR-I frame as described later. I do not have to.

  Next, processing at the start of recording of moving image data by the processing circuit 200 will be described using the flowchart of FIG. 3 is executed by the CPU 202 of the processing circuit 200 controlling each unit.

  When the power is turned on by the operation unit 211, the CPU 202 sets the imaging device 500 to the shooting mode. Then, the CPU 202 starts generating a timing signal by the clock control unit 204, controls the imaging unit 400 according to the timing signal, and starts a moving image shooting operation by the imaging unit 400. In the present embodiment, the imaging unit 400 outputs moving image data having one frame of horizontal 3840 pixels × vertical 2160 pixels and a frame rate of 30 frames per second (fps). The CPU 202 acquires moving image data from the imaging unit 400 in accordance with the timing signal from the clock control unit 204 and stores it in the memory 203. Then, the size of the moving image data stored in the memory 203 is changed by the resizing unit 206 and stored again in the memory 203. The display unit 208 reads out each frame of the moving image data from the memory 203 and displays it.

  In such a shooting standby state, when there is an instruction to start recording from the operation unit 211, the CPU 202 notifies the processing circuit 100 of a command for recording preparation processing via the message communication unit 207c (S301). The processing of the processing circuit microcomputer 100 will be described later with reference to the flowchart of FIG. Next, the CPU 202 performs a recording preparation process (S302). Specifically, the CPU 202 sets a frame rate of a moving image to be encoded, a target data rate after encoding, an encoding form (all are encoded as IDR-I frames in the processing circuit 200), and the like. In addition, the CPU 202 sets a recording buffer area on the memory 203 for storing each frame of the encoded moving image data. In addition to the recording preparation process command, the CPU 202 notifies the processing circuit 100 of information such as the frame rate of the moving image to be encoded and the target data rate after encoding by the message communication unit 207c.

  Next, when the message communication unit 207c receives notification from the processing circuit 100 that preparation for starting moving image recording has been completed (S303), the CPU 202 performs a frame reading synchronization process with the processing circuit 100 (S304). Specifically, the CPU 202 transmits a control command indicating the processing timing of the moving image data by the processing circuit 100 to the processing circuit 100 from the message communication unit 207c. As a result, the processing circuits 100 and 200 are controlled to alternately process the moving image data obtained by the imaging unit 400 every other frame. Next, the CPU 202 acquires moving image data from the imaging unit 400 according to the timing signal of the clock control unit 204, and starts encoding processing of moving image data (S305).

  Next, processing at the start of recording of moving image data by the processing circuit 100 will be described with reference to the flowchart of FIG. 4 is executed by the CPU 102 of the processing circuit 100 controlling each unit.

  As described above, when the message communication unit 107c receives a command to notify the start of recording from the processing circuit 200 (S401), the CPU 102 performs a recording preparation process of the processing circuit 100 (S402). Specifically, the CPU 102 sets the frame rate of the moving image to be encoded, the target data rate after encoding, the encoding mode (the processing circuit 200 encodes all the second and subsequent frames as I frames), and the like. Set. In addition, the CPU 102 sets a storage area for storing each frame of the encoded moving image data on the memory 103.

  Next, dummy data encoding processing is executed using the codec unit 105 (S403). The CPU 102 acquires one frame of moving image data from the imaging unit 400 and encodes it as an IDR-I frame. The frame to be encoded here is the first frame at 1202 in FIG. The frame encoded here is dummy data and is not actually recorded, so it is not transmitted to the processing circuit 200 but discarded. By discarding the first frame without recording it in this way, it is possible to avoid the occurrence of a time lag to the recorded moving image.

  Next, the CPU 102 transmits a command to the effect that the recording preparation is completed from the message communication unit 107c (S404). Thereafter, when the command indicating the timing of the synchronization processing is received by the message communication unit 107c, the CPU 102 changes the processing timing of the moving image data by the clock control unit 104 so as to process a frame different from the processing circuit 200 ( S405). Next, the CPU 102 acquires moving image data from the imaging unit 400 according to the timing signal from the clock control unit 104 and starts the encoding process (S406).

  In this way, the processing circuits 100 and 200 perform the encoding process by alternately obtaining the moving image data obtained by the imaging unit 400 every other frame.

  Next, a data storage area stored in the memories 103 and 203 of the processing circuits 100 and 200 will be described with reference to FIG.

  701 in FIG. 7 is a diagram illustrating the state of the moving image data stored in the memory 203. In 701, frames 1 and 3 are data encoded by the processing circuit 200, and frames 2 and 4 are data encoded by the processing circuit 100. Data encoded by the processing circuit 100 and the processing circuit 200 is arranged in the storage areas 704 to 707 of the memory 203 without any gap.

  702 in FIG. 7 is a diagram illustrating a storage state in the memory 203 before the encoded moving image data is transmitted from the processing circuit 100. One frame of moving image data encoded by the processing circuit 100 is temporarily stored in the memory 103, read from the memory 103, transmitted to the processing circuit 200, and stored in the memory 203. Reference numeral 703 in FIG. 7 is a diagram illustrating a state of data stored in the memory 103 in the processing circuit 100. In 703, the data of frames 2, 4, and 6 are stored in the storage area 712-714.

  The CPU 102 designates a write address of the memory 203 in the processing circuit 200 and transmits the encoded data to the processing circuit 200 so as to store the moving image data encoded at the specified address. When the CPU 202 receives the encoded moving image data from the processing circuit 100, it sends the received data directly to the memory 103. Then, the CPU 202 writes the received data at the address of the memory 203 designated by the CPU 102. The CPU 202 controls the message communication unit 207c to notify the processing circuit 100 of the write address information designated by the CPU 102.

  Further, before transmitting the encoded data to the processing circuit 200, the CPU 102 informs the CPU 202 of the data amount information of the encoded video data of one frame by the message communication unit 107c. For example, as shown at 703, when the encoding of the moving image data of each frame is completed, the CPU 102 notifies the processing circuit 200 of information indicating the data amount of each frame by the message communication unit 107c.

  When the information on the data amount of frame 2 is transmitted from the processing circuit 100, the CPU 202 reserves the storage area 709 of the memory 203 corresponding to the data amount so as to be continuous with the storage area 708 of frame 1. Then, the CPU 202 stores the data of the frame 3 encoded by the codec unit 202 in an area 710 continuous with the reserved storage area. After the frame 2 storage area is reserved in this way, when the frame 2 moving image data is transmitted from the processing circuit 100, the CPU 202 stores the received frame 2 moving image data in the area 709 of the memory 203. Thus, the CPU 202 transmits the information on the size of the moving image data of each frame encoded by the processing circuit 100 to the processing circuit 200 before starting the transmission of the moving image data, so that the CPU 202 can encode the moving image to be encoded next. Determine the data storage area.

  Next, encoding processing in each processing circuit will be described. FIG. 5 is a flowchart showing the encoding process in the processing circuit 200. The processing in FIG. 5 is executed by the CPU 202 controlling each unit.

  Based on the timing signal from the timing control unit 204, the CPU 202 determines whether or not it is time to acquire moving image data from the imaging unit 400 (S501). If it is not processing timing, the process proceeds to step S507.

  If it is the processing timing of the moving image data, the CPU 202 acquires one frame of moving image data from the imaging unit 400, converts it into YUV color space data by the image processing unit 201, and stores it in the memory 203 (S502). Next, the CPU 202 determines a storage address (storage area) to be stored in the memory 203 after encoding the acquired moving image data of one frame (S503). The area for storing the encoded data is the area secured in the memory 203 in S302 of FIG. The storage state of each frame in the memory 203 is as shown in FIG.

  Next, the CPU 202 encodes the moving image data stored in the memory 203 by the codec unit 205 and stores it in the address of the memory 203 determined in S503 (S505). The CPU 202 notifies the processing circuit 100 of information related to the memory 203 through the message communication unit 207c. The information about the memory notified here is the storage address of the memory 203 for the processing circuit 100 to store the encoded data, and the free capacity of the recording buffer area in the memory 203. When transmitting the encoded data to the processing circuit 200, the CPU 102 designates the address transmitted here.

  When the information on the amount of encoded data is received from the processing circuit 100 (S506), the CPU 202 sets the storage area for the encoded data in the recording buffer area. As described above, the information on the amount of encoded data is notified from the processing circuit 100, so that the address when the processing circuit 200 stores the next frame to be encoded in the recording buffer area can be determined. It becomes.

  Next, the CPU 202 determines whether or not encoded data is transmitted from the processing circuit 100 (S507). When encoded data is transmitted, the data reception unit 207a receives the encoded data ( S508). Note that the processing circuit 100 does not need to periodically transmit the encoded data, and transmits the encoded data to the processing circuit 200 at an arbitrary timing. The processing circuit 200 can acquire data amount information before transmission of one frame of encoded data from the processing circuit 100 is completed. Therefore, the processing circuit 200 can execute the process of the next frame in the microcomputer S200 without waiting for the reception of the encoded data from the processing circuit 100. As a result, the processing load on the processing circuit 200 is reduced. The CPU 202 determines whether or not there is an instruction to stop recording (S509), and when there is an instruction to stop recording, the process ends.

  FIG. 6 is a flowchart showing the encoding process in the processing circuit 100. The process in FIG. 6 is executed by the CPU 102 controlling each unit.

  Based on the timing signal from the timing control unit 104, the CPU 102 determines whether or not it is timing to acquire moving image data from the imaging unit 400 (S601). If it is not processing timing, the process proceeds to step S607.

  When it is the processing timing of the moving image data, the CPU 102 acquires moving image data of one frame from the imaging unit 400, converts it into YUV color space data by the image processing unit 101, and stores it in the memory 103 (S602). Next, the CPU 102 determines a storage address (storage area) to be stored in the memory 103 after encoding the acquired moving image data of one frame (S503). The area for storing the encoded data is the area secured in the memory 103 in S402 of FIG. The storage state of each frame in the memory 103 is as shown in FIG.

  Next, the CPU 102 encodes the moving image data stored in the memory 103 by the codec unit 105 and stores it in the address of the memory 203 determined in S603 (S604). Next, the message communication unit 107c receives memory information from the processing circuit 200 (S605). When the information in the memory 103 is notified from the processing circuit 200, the encoded data can be transmitted from the processing circuit 100 to the processing circuit 200.

  Next, the CPU 102 transmits the size information of the encoded moving image data of one frame to the processing circuit 200 by the message communication unit 107c. Note that the transmission timing of the data amount information of the encoded data to the processing circuit 200 and the transmission timing of the encoded data are irregularly executed under the control of the CPU 102.

  Next, the CPU 102 determines whether or not the information in the memory 203 has been received from the processing circuit 200 (S607). When the information in the memory 203 has been received, the CPU 102 determines whether or not untransmitted encoded data is stored in the memory 103 (S608). If there is untransmitted encoded data, the CPU 102 designates the write address notified in S603, reads out one frame of encoded data that has not been transmitted from the memory 103, and transmits it to the processing circuit 200 by the data transmission unit 207b. (S609).

  When the transmission of the encoded data is completed, the CPU 102 determines whether or not a recording stop instruction has been received from the processing circuit 200 (S610), and ends the process when the recording stop instruction is received. Further, when the information of the write address of the memory 103 is not received in S607, or when the untransmitted encoded data is not stored in the memory 103, the process proceeds to S610.

  Next, a process when recording is stopped will be described. FIG. 8 is a flowchart showing the processing of the processing circuit 200 when recording is stopped. The processing in FIG. 8 is realized by the CPU 202 controlling each unit.

  When there is an instruction to stop recording from the operation member 211, the CPU 202 notifies the processing circuit 100 of the recording stop through the message communication 207c (S801). Next, the CPU 202 stops the encoding process of the codec unit 205 (S802), and determines whether or not all encoded data has been received from the processing circuit 100 (S803). If reception of all the encoded data from the processing circuit 100 is not completed, the CPU 202 receives the encoded data from the processing circuit 100 (S804), and stores the received data in the recording buffer area of the memory 203 (S805). ). Then, when all the encoded data is transmitted from the processing circuit 100, the CPU 202 ends the process.

  FIG. 9 is a flowchart showing processing of the processing circuit 100 when recording is stopped. The processing in FIG. 9 is realized by the CPU 102 controlling each unit.

  When the message communication unit 107c receives a recording stop command from the processing circuit 200, the flow is started, and the CPU 102 stops the encoding process of the codec unit 105 (S901). Next, the CPU 102 determines whether untransmitted encoded data is stored in the memory 103 (S902). If there is untransferred data, the data transmission unit 107b transmits the encoded data to the processing circuit 200. (S904). When the transmission of all the encoded data is completed, the CPU 102 notifies the processing circuit 200 that the transmission of all the encoded data is completed by the message communication unit 107c (S903).

  Next, a moving image file recording process by the processing circuit 200 will be described. FIG. 10 is a flowchart showing a moving image file recording process. The processing in FIG. 10 is executed by the CPU 202 controlling each unit.

  When there is a recording start instruction from the operation unit 211, the CPU 202 controls the recording / reproducing unit 210 to open a new moving image file on the recording medium 212 (S1001). Next, the CPU 202 determines whether or not a predetermined amount of unrecorded moving image and audio data is stored in the recording buffer area of the memory 203 (S1002). In the present embodiment, the CPU 202 generates time code information indicating the elapsed time of each frame from the start of recording, and stores it in the recording buffer area of the memory 203. When a predetermined amount of unrecorded data is stored in the recording buffer area, the CPU 202 controls the recording / playback unit 212 to read unrecorded moving image, audio, and time code data from the memory 203 and record them on the recording medium 212. (S1003). Note that writing to the recording medium 212 may be performed when unrecorded data for a predetermined time is stored in the recording buffer area instead of the predetermined amount. In the present embodiment, it is assumed that the data writing speed to the recording medium 212 is higher than the data rate of the encoded moving image data and audio data.

  When the writing of a predetermined amount of unrecorded data is completed, the CPU 202 stops writing data to the recording medium 212. Then, the CPU 202 determines whether or not there is an instruction to stop recording from the operation unit 211 (S1004). If there is no instruction to stop recording, the recording process is continued. If there is an instruction to stop recording, the CPU 202 reads unrecorded data from the recording buffer area in the memory 203 and records it in the recording medium 212 by the recording / playback unit 210 (S1005). Further, the CPU 202 sequentially generates management information to be stored in the moov atom in FIG. Then, the CPU 202 reads management information stored in the moov atom from the memory 203, records it on the recording medium 212 by the recording / playback unit 210, and closes the moving image file (S1006). By such processing, a moving image file in the MOV format is recorded on the recording medium 212.

  Next, the timing of the moving image data encoding processing by the processing circuits 100 and 200 will be described with reference to FIG.

  In FIG. 11, reference numeral 1101 denotes a frame period corresponding to the frame rate of moving image data to be recorded. In this embodiment, since the frame rate of the moving image data generated by the imaging unit 400 and the frame rate of the recorded moving image are both 30 fps, the frame interval 1101 is 1/30 second. Reference numeral 1102 denotes a frame interval for processing each frame of the moving image data by the processing circuit 200. Reference numeral 1103 denotes a frame interval for processing each frame of the moving image data by the processing circuit 100. Since the processing circuits 100 and 200 alternately process the moving image data output from the imaging unit 400 every other frame, the frame intervals 1102 and 1103 are each twice the frame interval of the recorded moving image.

  1104 is a period during which the processing circuit 200 acquires one frame of moving image data from the imaging unit 400, and 1105 is a period during which the processing circuit 200 encodes one frame of moving image data. Reference numeral 1106 denotes a period during which the processing circuit 100 acquires one frame of moving image data from the imaging unit 400. Reference numeral 1107 denotes a period during which the processing circuit 100 encodes one frame of moving image data. Reference numeral 1108 denotes a period during which encoded data is transmitted from the processing circuit 100 to the processing circuit 200.

  Reference numeral 1109 denotes processing for transmitting information in the memory 203 from the processing circuit 200 to the processing circuit 100. Reference numeral 1110 denotes processing for transmitting information on the amount of encoded data from the processing circuit 100 to the processing circuit 200. Reference numeral 1111 denotes processing for notifying the processing circuit 100 that the transmission of one frame of encoded data has been completed.

  As shown in FIG. 11, when moving image data having a frame rate corresponding to the frame interval 1101 is processed by one of the processing circuits 100 and 200, acquisition and encoding of one frame of moving image data within the period of the frame interval 1101 is performed. Processing must be completed.

  On the other hand, in this embodiment, since the two processing circuits 100 and 200 encode moving image data in parallel, each processing circuit performs acquisition and encoding processing of one frame of moving image data within a frame interval 1101. It does not have to be completed.

  In the present embodiment, when the encoding process of one frame of moving image is completed by the processing circuit 100, information on the amount of encoded data is transmitted to the processing circuit 200. Therefore, the processing circuit 200 can start the encoding process for the next frame before receiving the encoded data from the processing circuit 100.

  Next, the reproduction process will be described.

  FIG. 13 is a flowchart showing processing of the processing circuit 200 during reproduction. The processing in FIG. 13 is realized by the CPU 202 controlling each unit. When the operation unit 211 gives an instruction to switch to the playback mode, the CPU 202 instructs the recording / playback unit 210 to read out a management file for managing each moving image file recorded on the recording medium 212. Then, the CPU 202 displays an index screen of each moving image file recorded on the recording medium 212 on the display unit 208 based on the read management file. When the user operates the operation unit 211 to select one of the moving image files displayed on the display unit 208 and instruct the start of reproduction, the process of FIG. 13 is started.

  The CPU 202 notifies the processing circuit 100 of identification information such as the file name of the moving image file selected by the user via the message communication unit 207c (S1301). Next, the CPU 202 controls the recording / playback unit 210 to read the moov atom data of the selected moving image file and store it in the memory 203 (S1302). Further, the CPU 102 of the processing circuit 100 reads out the moov atom of the selected moving image file from the recording / playback unit 210 by the message communication unit 107c based on the information of the moving image file notified from the processing circuit 200 in S1301. Request. The recording / reproducing unit 210 reads the moov atom of the selected moving image file from the recording medium 212 in accordance with the read request from the CPU 102 and sends it to the processing circuit 100 by the data transmission unit 207a. When receiving the moov atom data of the moving image file selected by the data receiving unit 107a, the CPU 102 stores the data in the memory 103, and when storing the moov atom data in the memory 103, the message communication unit 107c displays information indicating that playback preparation is complete. To the processing circuit 200. As a result, information necessary for the reproduction processing of moving image data can be used between the processing circuits 100 and 200 together. The CPU 102 may not directly request the recording / playback unit 210 to read the moov atom data, but the CPU 202 may transmit the moov atom data stored in the memory 203 to the processing circuit 100.

  In this way, the CPU 202 waits for notification of completion of reproduction preparation from the processing circuit 100 (S1303). When receiving the notification of completion of reproduction preparation, the CPU 202 controls the clock control unit 204 to start outputting a decode clock indicating the decoding timing of the moving image data and a display clock indicating the display timing of the display unit 208 (S1304). . The decode clock is determined based on the information of the moov atom. Further, the CPU 102 controls the clock control unit 104 to start outputting the decode clock. The display clock is determined by the capability of the display unit 208.

  Next, the CPU 202 executes processing for generating a display screen using the decoded moving image data (S1305), and displays the generated display screen on the display unit 208 (S1306). Then, the CPU 202 determines whether or not an instruction to stop reproduction has been given from the operation unit 211, and continues the process until there is an instruction to stop reproduction.

  Each of the processing circuits 100 and 200 executes the moving image data decoding process at a timing corresponding to the decoding clock. As will be described later, the processing circuit 200 generates a display screen by executing a reduction process of the reproduced image at a timing according to the display clock. On the other hand, the processing circuit 100 generates a display screen by executing a reduction process of the reproduced image at a timing independent of the display timing.

  Further, as will be described later, in the present embodiment, the processing circuits 100 and 200 perform decoding processing of moving image data reproduced alternately every other frame. Therefore, in the moving image data, the frame number processed by the processing circuit 100 is notified from the processing circuit 200 to the processing circuit 100 in S1301.

  In the display process of S1306, the generated display screen is displayed on the display unit 208 at a timing corresponding to the display clock. The frame displayed at this time includes a frame generated on the processing circuit 100 side and a frame generated on the processing circuit 200 side. Processing for determining which processing circuit uses the frame generated will be described later.

  Next, display screen generation processing by the processing circuit 200 will be described. FIG. 14 is a flowchart showing display screen generation processing by the processing circuit 200. The processing in FIG. 14 is executed by the CPU 202 controlling each unit.

  The processing circuit 200 decodes one frame of moving image data at a timing corresponding to the decoding clock, reduces the size of the moving image data decoded at a timing corresponding to the display clock, and generates a display screen.

  First, the CPU 202 determines whether or not it is a decoding timing based on a decoding clock (S1401). If it is the decoding timing, the CPU 202 requests the recording / playback unit 210 to read out one frame of moving image data. The CPU 202 uses the codec unit 205 to decode the read one frame of moving image data (S1402), and stores the decoded moving image data in the memory 203 (S1403).

  Next, the CPU 202 determines whether or not it is the decoding timing based on the display clock (S1404). When it is the display timing, the CPU 202 controls the resizing unit 206 to reduce the size of the decoded one-frame moving image data stored in the memory 203 and stores it in the storage area of the display screen of the memory 203. (S1406). The CPU 202 repeats the process of FIG. 14 to generate a display screen until a playback stop instruction is issued.

  Next, display screen generation processing by the processing circuit 100 will be described. FIG. 15 is a flowchart showing display screen generation processing by the processing circuit 100. The processing in FIG. 15 is executed by the CPU 102 controlling each unit.

  The processing circuit 100 decodes one frame of moving image data at a timing according to the decode clock, and subsequently executes a process of reducing the screen size of the decoded moving image data.

  First, the CPU 102 determines whether or not it is a decoding timing based on a decoding clock (S1501). When it is the decoding timing, the CPU 102 requests the recording / reproducing unit 210 to read out one frame of moving image data. The recording / playback unit 210 reads the designated moving image data from the recording medium 211 and sends it to the processing circuit 100 by the data transmission unit 207a. The CPU 202 uses the codec unit 205 to decode the transmitted one frame of moving image data (S1502), and stores the decoded moving image data in the memory 103 (S1503).

  When the decoding process for one frame of moving image data is completed, the CPU 102 controls the resizing unit 106 to reduce the size of the decoded one frame of moving image data stored in the memory 103. Then, the CPU 102 transmits the reduced moving image data to the processing circuit 200 by the data transmission unit 107b and stores it in the storage area of the display screen of the memory 203 (S1505). The CPU 102 generates a display screen until a reproduction stop instruction is issued by repeating the processing of FIG.

  In this way, the processing circuit 100 and the processing circuit 200 can achieve both efficient use of the memory and maintenance of the reproduction speed by making the resizing process different. That is, in the processing circuit 200, the resizing process is performed by the resizing unit 206 in accordance with the display timing of the display unit 208. As a result, the storage area of the memory 203 for storing the resized frame can be minimized and the memory 203 can be used efficiently. Further, in the processing circuit 100 that transfers moving image data through the data bus 300, the decoding circuit, the resizing process, and the transfer process are continuously performed, so that the overhead for communication of the data bus 300 is compensated, and the processing circuit 200 is quickly transmitted. Send the decoded video data. In the present embodiment, a storage area for three frames is prepared as a storage area for the display screen of the memory 203. When the resizing process for one frame is newly completed, the oldest frame data is erased and the new frame data is stored.

  Next, display processing of the decoded moving image data in the processing circuit 200 will be described. FIG. 16 is a flowchart for explaining display processing by the processing circuit 200. The processing in FIG. 16 is executed by the CPU 202 controlling each unit. Further, the process of FIG. 16 is executed according to the display clock.

  First, the CPU 202 updates the frame number of the frame to be displayed (S1601). Here, the frame number is a number indicating the number of frames counted from the top of the moving image. It is assumed that the head is 1, the next is 2, the next is 3, and so on. In this embodiment, the display frame number is updated using the decode clock. That is, if the decode clock has been generated between the last update of the display frame number and step S1601, the display frame number is incremented by 1. Otherwise, the display frame number is not changed.

  Next, the CPU 202 compares the display frame number with the frame number of the frame resized by the processing circuit 200 and stored in the memory 203 (S1602). If these frame numbers match (S1603), the CPU 202 reads out the moving image data of the frame resized by the processing circuit 200 from the memory 203, supplies it to the display unit 208, and displays it (S1604).

  On the other hand, when the display frame number does not match the frame number resized by the processing circuit 200 (1603), the CPU 202 compares the frame number of the frame resized by the processing circuit 200 and stored in the memory 203 (S1602). ). If these frame numbers match (S1606), the CPU 202 reads out the moving image data of the frame resized by the processing circuit 100 and stored in the memory 203 from the memory 203, supplies it to the display unit 208, and displays it. (S1607).

  If the display frame number does not match the frame number resized by the processing circuit 100 (1606), the CPU 202 selects the frame closest to the display frame number from the resized frames stored in the memory 203. To do. Then, the CPU 202 reads the moving image data of the selected frame from the memory 203 and sends it to the display unit 208 for display.

  FIG. 17 is a timing chart showing the decoding and display processing of moving image data by the processing circuits 100 and 200 during reproduction. As shown in FIG. 17, each of the processing circuits 100 and 200 decodes one frame of moving image data at a timing corresponding to the decode clock. The processing circuit 100 executes a resizing process subsequent to the decoding process of the moving image data, and transmits the resized frame data to the processing circuit 200 following the resizing process.

  On the other hand, the processing circuit 200 does not perform the resizing process immediately after the video data decoding process is completed, but performs the resizing process at a timing corresponding to the next display clock.

  As described above, in this embodiment, moving image data having a large number of pixels and a large frame rate can be reproduced using two processing circuits. Furthermore, when reproducing moving image data using two processing circuits, it is possible to efficiently use the memory and compensate for the overhead for data transfer between the processing circuits.

Claims (8)

Imaging means;
A circuit having a first CPU and a first communication unit, which acquires moving image data from the imaging unit, processes the acquired moving image data, and outputs the acquired moving image data by the first communication unit Circuit,
A circuit having a second CPU and a second communication unit, wherein the moving image data of a frame different from the frame acquired by the first processing circuit is acquired from the imaging unit, processed, and processed; A second processing circuit for recording the moving image data and the moving image data output from the first processing circuit by the first communication unit on a recording medium ;
The second CPU uses the first processing circuit so that the first processing circuit acquires and processes moving image data of a frame different from the frame processed by the second processing circuit from the imaging unit. Control information for controlling processing timing is transmitted to the first processing circuit by the second communication means;
The first CPU controls the timing at which the first processing circuit processes the moving image data based on the control information transmitted from the second communication means,
The imaging apparatus , wherein the first processing circuit acquires moving image data of a frame processed by the first processing circuit from the imaging unit according to the controlled timing . The first processing circuit includes a generating unit that generates a timing signal for controlling processing timing in the first processing circuit,
The imaging apparatus according to claim 1, wherein the first CPU changes a processing timing by the generation unit based on the control information transmitted from the second communication unit . The second processing circuit stores the video data in the frame in which the second processing circuit has processed a video data frame output from the first processing circuit by the first communication means to the memory The imaging apparatus according to claim 1, wherein the moving image data stored in the memory is read and recorded on the recording medium.   The imaging apparatus according to claim 3, wherein each of the first processing circuit and the second processing circuit performs a process of encoding moving image data acquired from the imaging unit. Imaging means;
A circuit having a first CPU and a first communication unit, which acquires moving image data from the imaging unit, processes the acquired moving image data, and outputs the acquired moving image data by the first communication unit Circuit,
A circuit having a second CPU and a second communication unit, wherein the moving image data of a frame different from the frame acquired by the first processing circuit is acquired from the imaging unit, processed, and processed; A second processing circuit for recording the moving image data and the moving image data output from the first processing circuit by the first communication unit on a recording medium;
Each of the first processing circuit and the second processing circuit performs a process of encoding the moving image data acquired from the imaging unit,
The second processing circuit includes the moving image data of the frame encoded by the second processing circuit and the moving image of the frame encoded by the first processing circuit and output by the first communication unit. Data is stored in a memory, moving image data stored in the memory is read and recorded on the recording medium,
The first CPU outputs data amount information of one frame of moving image data encoded by the first processing circuit to the second processing circuit by the first communication means, and the second processing circuit the CPU, based on the said amount of data of information outputted by the first communication means, shooting the second processing circuit and determines the storage area in the memory of the moving image data processed coded Image device.   The second CPU determines a storage address in the memory for storing the moving image data encoded by the second processing circuit, and uses the second communication means to store the storage address information. The image pickup apparatus according to claim 5, wherein the image pickup apparatus outputs the image to a processing circuit.   The first CPU outputs the information on the amount of data to the second processing circuit by the first communication unit, and then the encoded video data is output to the second processing unit by the first communication unit. The imaging apparatus according to claim 6, wherein the image data is output to the processing circuit and written to the storage address output from the second communication unit in the memory via the second communication unit.   8. The method according to claim 1, wherein the first processing circuit and the second processing circuit acquire one frame of moving image data from the imaging unit alternately every other frame. Imaging device.

Images