JP2009088878A - Imaging device, recorder, reproducing device, imaging method, recording method, and reproducing method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging device capable of quickly and accurately performing image processing when reproducing RAW image data sequentially outputted from an imaging element in a frame unit, and to provide a recorder, a reproducing device, an imaging method, a recording method, and a reproducing method and program. <P>SOLUTION: The imaging device is provided with a parameter calculating part 120 for calculating a parameter required for image processing in each frame when reproducing RAW image data on the basis of the RAW image data sequentially outputted from an imaging element 102 in a frame unit, and a medium recording part 150 for recording the calculated parameter corresponding to each frame as a parameter file different from the RAW image data as well as each RAW image data in a recording medium 152. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an imaging device, a recording device, a playback device, an imaging method, a recording method, a playback method, and a program.

  Imaging devices such as video cameras and camcorders capable of recording moving images generally perform predetermined image processing on a signal output from an image sensor such as a CCD, and then perform compression processing such as MPEG-2 to provide a flash memory or HDD Recording is performed on a recording medium such as a DVD.

  Image processing includes, for example, white balance control and exposure control. Patent Document 1 discloses a technique for recording information about white balance obtained by image processing in an image file together with image data about an image when a part of a moving image is recorded as a still image.

  Normally, the compression processing causes the image recording / reproduction quality to be lower than the data before the compression processing. For this reason, some image capturing apparatuses such as digital cameras that record still images directly record signals output from the image sensor without performing compression processing in the RAW format. Patent Documents 2 to 4 disclose a technique of selecting and recording a compressed format such as JPEG or a RAW format for directly recording a signal when recording an image signal.

JP 2006-80892 A JP 11-261933 A JP 2001-60876 A JP 2001-61067 A

  By the way, when a signal from the image sensor is directly recorded on a recording medium, an image can be recorded with high image quality. However, since the data capacity is increased, it has not been performed for recording a moving image.

  Further, when displaying a still image, first, image data recorded in the RAW format is read to calculate parameters optimal for image display. Then, image processing is performed using the parameters, and an image is displayed. Here, the parameters are a white balance correction value for controlling white balance, a luminance correction value for correcting luminance, and the like.

  On the other hand, when trying to reproduce a moving image using a signal directly recorded from an image sensor, high-speed and continuous image processing is required unlike display of a still image. For this reason, if each frame image constituting a moving image is subjected to image data reading, parameter calculation, and image processing using parameters, as in the case of RAW still images, a series of processing takes time. Therefore, there is a problem that it is difficult to reproduce the moving image.

  Therefore, the present invention has been made in view of the above problems, and an object of the present invention is to perform image processing quickly and accurately when reproducing RAW image data sequentially output from the image sensor in units of frames. It is an object of the present invention to provide a new and improved imaging device, recording device, playback device, imaging method, recording method, playback method, and program that can be performed.

  In order to solve the above-described problem, according to an aspect of the present invention, parameters necessary for image processing when reproducing RAW image data based on RAW image data sequentially output in units of frames from an image sensor are set as frames. A recording system comprising: a parameter calculating unit that calculates for each frame; and a medium recording unit that records a parameter corresponding to each calculated frame on a recording medium together with each RAW image data as a parameter file different from the RAW image data; A data reading unit for reading each RAW image data and parameter file recorded on the recording medium, a parameter file processing unit for obtaining parameters necessary for sequentially reproducing the RAW image data in units of frames from the read parameter file, Based on each parameter, the corresponding RAW image data is The imaging device is provided characterized by having a reproducing system and an image processing unit for image processing required for reproducing each over arm.

  With this configuration, the parameter calculation unit calculates, for each frame, parameters necessary for image processing when reproducing the RAW image data based on the RAW image data sequentially output from the image sensor in units of frames. Records the parameter corresponding to each calculated frame on the recording medium together with each RAW image data as a parameter file different from the RAW image data. The data reading unit reads each RAW image data and parameter file recorded on the recording medium, and the parameter file processing unit parameters necessary for sequentially reproducing the RAW image data in units of frames from the read parameter file. The image processing unit performs image processing necessary for reproduction of the corresponding RAW image data for each frame based on each parameter. As a result, parameters necessary for image processing when reproducing RAW image data are calculated in advance for each frame, and when RAW image data is reproduced, the RAW image data is reproduced based on the parameters calculated in advance. Therefore, the image processing at the time of reproduction can be performed quickly and accurately.

  In the playback system, the parameter file processing unit uniformly sets the parameter corresponding to each RAW image data to a specified value for RAW image data for a plurality of frames corresponding to a desired series of scenes. May be. With such a configuration, variation in parameters in a series of scenes can be suppressed in advance, and smooth reproduction is possible.

  In order to solve the above-described problem, according to another aspect of the present invention, it is necessary for image processing when reproducing RAW image data based on RAW image data sequentially output from the image sensor in units of frames. A parameter calculation unit that calculates parameters for each frame; and a medium recording unit that records the parameters corresponding to the calculated frames on a recording medium together with each RAW image data as a parameter file different from the RAW image data. Is provided.

  In order to solve the above problems, according to another aspect of the present invention, RAW image data in units of frames and a parameter file including parameters necessary for image processing when reproducing each RAW image data are recorded. A data reading unit for reading each RAW image data and parameter file from the recorded recording medium, a parameter file processing unit for obtaining parameters necessary for sequentially reproducing the RAW image data in units of frames from the read parameter file, An image processing unit is provided that includes an image processing unit that performs image processing necessary for reproduction of corresponding RAW image data for each frame based on parameters.

  The parameter file processing unit may uniformly set a parameter corresponding to each RAW image data to a designated value for a plurality of frames of RAW image data corresponding to a desired series of scenes.

  In order to solve the above-described problem, according to another aspect of the present invention, it is necessary for image processing when reproducing RAW image data based on RAW image data sequentially output from the image sensor in units of frames. After calculating the parameters for each frame, a recording processing step for recording the parameters corresponding to the calculated frames on the recording medium together with each RAW image data as a parameter file different from the RAW image data; After each RAW image data and parameter file are read out, parameters necessary for sequentially reproducing the RAW image data in units of frames are obtained from the read parameter file, and then a frame for the corresponding RAW image data is obtained based on each parameter. Each has a playback processing step that performs image processing necessary for playback. Imaging wherein the Rukoto is provided.

  In the above reproduction processing step, the parameters corresponding to each RAW image data are uniformly set to specified values for the RAW image data for a plurality of frames corresponding to a desired series of scenes before the image processing. It may be.

  In order to solve the above-described problem, according to another aspect of the present invention, it is necessary for image processing when reproducing RAW image data based on RAW image data sequentially output from the image sensor in units of frames. After the parameter is calculated for each frame, a recording method is provided, wherein the parameter corresponding to each calculated frame is recorded on a recording medium together with each RAW image data as a parameter file different from the RAW image data. The

  In order to solve the above problems, according to another aspect of the present invention, RAW image data in units of frames and a parameter file including parameters necessary for image processing when reproducing each RAW image data are recorded. After each RAW image data and parameter file is read from the recorded recording medium, parameters necessary for sequentially reproducing the RAW image data in units of frames are obtained from the read parameter file, and then, based on each parameter, There is provided a reproduction method characterized by performing image processing necessary for reproduction of corresponding RAW image data for each frame.

  Prior to the image processing, for a plurality of frames of RAW image data corresponding to a desired series of scenes, a parameter corresponding to each RAW image data may be uniformly set to a designated value. .

  In order to solve the above-described problem, according to another aspect of the present invention, it is necessary for image processing when reproducing RAW image data based on RAW image data sequentially output from the image sensor in units of frames. A first step of calculating a parameter for each frame; and a second step of recording the parameter corresponding to each calculated frame on a recording medium together with each RAW image data as a parameter file different from the RAW image data. A program for causing a computer to execute is provided.

  According to the present invention, when RAW image data sequentially output from the image sensor in units of frames is reproduced, image processing can be performed quickly and accurately.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

(Configuration of the first embodiment)
A configuration of the imaging apparatus according to the first embodiment of the present invention will be described. The imaging apparatus according to the present embodiment includes a recording device 100 (recording system) that records images and sounds, and a reproducing device 200 (playback system) that reproduces images and sounds. Note that the recording device 100 and the playback device 200 may be configured integrally, or may be configured separately as independent devices.

  Hereinafter, configurations of the recording device 100 and the playback device 200 will be described. First, a recording apparatus 100 according to the first embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing a recording apparatus 100 according to this embodiment.

  The recording apparatus 100 is, for example, a video camera, a camcorder, or the like that has an optical system 102 such as a lens and an image sensor 104 and can capture and record a moving image. The recording apparatus 100 includes, for example, an optical system 102, a driver 110, an image sensor 104, a microphone 112, A / D conversion units 106 and 114, an image signal compression unit 108, an audio signal compression unit 116, and parameter calculation. Unit 120, data file generation unit 122, parameter file generation unit 124, CPU (Central Processing Unit) 130, image signal processing unit 140, D / A conversion unit 142, display control unit 144, display unit 146, a recording medium control unit 150, a recording medium 152, a memory 160, a VRAM 162, an operation unit 170, and the like.

  Although not shown in detail, the optical system 102 includes, for example, a focus lens, a zoom lens, a diaphragm, and the like. The optical system 102 transmits light from the subject and forms an image of the subject on the image sensor 104. The focus lens is adjusted so that the subject is focused on the image sensor 104. The zoom lens is adjusted so that the focal length changes. The diaphragm is adjusted so that the amount of light incident on the image sensor 104 changes.

  The driver 110 drives a focus lens, a zoom lens, a diaphragm, and the like of the optical system 102. The driver 110 drives each component of the optical system 102 based on focus control and exposure control in the CPU 130.

  The image sensor 104 is an image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor), for example, and converts optical information imaged on the image sensor 104 into an electrical signal (image signal) by photoelectric conversion. Then, an electrical signal for each frame is output to the A / D converter 106.

  The A / D conversion unit 106 converts the electrical signal sequentially output from the image sensor 104 in units of frames into digital data (RAW image data) by digital conversion. Note that a circuit such as a CDS or an amplifier is provided between the image sensor 104 and the A / D converter 106 to remove or amplify noise included in the electrical signal output from the image sensor 104. Good. The A / D conversion unit 106 sequentially outputs the image data obtained by digital conversion to the image signal compression unit 108 and the parameter calculation unit 120 in units of frames. The image data output from the A / D converter 106 is handled in the RAW format without performing image processing such as white balance control and exposure control, and irreversible compression processing such as JPEG format.

  The image signal compressor 108 compresses the image data output from the A / D converter 106 by a reversible compression method, and outputs the compressed image data to the data file generator 122. The amount of image data can be reduced by compression processing using a reversible compression method, and image data can be recorded and reproduced without further degrading image quality. A reversible compression method includes, for example, entropy coding using a Huffman code. The reversible compression method is not limited to strict “reversible”.

  The microphone 112 is an example of an audio input unit, collects audio from the outside, converts it into an electric signal, and outputs the electric signal to the A / D conversion unit 114. The A / D conversion unit 114 converts the electrical signal output from the microphone 112 into digital data (audio data) related to audio by digital conversion. The A / D converter 114 outputs the audio data obtained by digital conversion to the audio signal compressor 116.

  The audio signal compressor 116 compresses the audio data output from the A / D converter 114 and outputs the compressed audio data to the data file generator 122. The compression method by the audio signal compression unit 116 is, for example, MP3, AAC or the like.

  The parameter calculation unit 120 calculates, for each frame, parameters necessary for image processing when reproducing RAW format image data, based on the RAW format image data output from the A / D conversion unit 106. Here, the parameters are a white balance correction value (WB value) for controlling white balance, an exposure correction value (EE value) for controlling exposure, and the like. The parameter calculation by the parameter calculation unit 120 may be performed simultaneously with the recording of the image data, or may be performed based on the image data recorded after the series of recording of the image data is completed. The parameters calculated by the parameter calculation unit 120 are output to the parameter file generation unit 124.

  The data file generation unit 122 generates a data file based on the RAW format image data output from the image signal compression unit 108 and the audio data output from the audio signal compression unit 116. The data file generation unit 122 outputs the generated data file to the recording medium control unit 150, and the data file is recorded on the recording medium 152.

  The parameter file generation unit 124 generates a parameter file having a plurality of frame parameters in a series of image data based on the parameters output from the parameter calculation unit 120. The parameter file generation unit 124 outputs the generated parameter file to the recording medium control unit 150, and the parameter file is recorded on the recording medium 152.

  The CPU 130 functions as an arithmetic processing device and a control device according to a program, and can control processing of each component provided in the recording device 100. The CPU 130 drives the optical system 102 by outputting a signal to the driver 110 based on, for example, focus control or exposure control. Further, the CPU 130 controls each component of the recording apparatus 100 based on a signal from the operation unit 170. In the present embodiment, the number of CPUs is one. However, a signal system command and an operation system command may be executed by different CPUs.

  The image signal processing unit 140 performs image processing necessary when the display unit 146 reproduces a live view image for each frame of the image data output from the A / D conversion unit 106. For example, the image data is subjected to white balance control, exposure control, and the like in the image signal processing unit 140.

  The image signal processing unit 140 includes a YC conversion unit 141. The image data directly output from the image sensor 104 via the A / D conversion unit 106 is in the RAW format, and the YC conversion unit 141 performs YC conversion processing on the RAW format image data. The YC conversion unit 141 can convert RAW format image data into a luminance signal and a color difference signal, and acquire the luminance signal and the color difference signal. By obtaining the luminance signal and the color difference signal by YC conversion, an image can be displayed on the display unit 146. The luminance signal and the color difference signal acquired by the image signal processing unit 140 are output to the D / A conversion unit 142.

  The D / A conversion unit 142 converts the luminance signal and the color difference signal, which are digital data output from the image signal processing unit 140, into analog signals by analog conversion. The D / A converter 142 outputs an analog signal to the display controller 144.

  The display control unit 144 drives the display unit 146 and controls the output of the display unit 146. The display unit 146 includes display means such as an LCD (Liquid Crystal Display). The display unit 146 performs live view display before performing the imaging operation read from the VRAM 162, various setting screens of the recording apparatus 100, and display of images that have been captured and recorded.

  The recording medium control unit 150 is an example of a medium recording unit, and controls writing of image data to the recording medium 152 or reading of image data and setting information recorded on the recording medium 152. The recording medium 152 is, for example, an optical recording medium (CD, DVD, etc.), a magneto-optical disk, a magnetic disk, a semiconductor storage medium, etc., and records captured image data. The recording medium control unit 150 and the recording medium 152 may be configured to be detachable from the recording apparatus 100.

  The memory 160 is composed of a semiconductor storage element such as SDRAM (synchronous DRAM), for example, and temporarily stores captured images. The memory 160 has a storage capacity sufficient to store a plurality of frames in the image data. The memory 160 stores an operation program for the CPU 130.

  The VRAM 162 is a memory for image display, and includes a memory having a plurality of channels so that display image writing and display on the display unit 146 can be performed simultaneously. The VRAM 162 temporarily stores an image signal for live view display on the display unit 146 during recording and display on the display unit 146 during playback.

  The operation unit 170 includes a power switch, mode switching means, a recording start button, a recording stop button, and the like, and is used by the user to perform operations of the recording apparatus 100, various settings, and the like. The recording start button and the recording stop button are buttons for performing a shooting operation.

  Next, the configuration of the playback device 200 according to the first embodiment of the present invention will be described with reference to FIG. FIG. 2 is a block diagram showing the playback apparatus 200 according to this embodiment.

  The playback device 200 can play back image data and audio data based on the data file and parameter file recorded by the recording device 100 according to the present embodiment. The playback apparatus 200 includes, for example, a recording medium control unit 202, a recording medium 204, a CPU 210, a driver 212, an operation unit 220, a data file processing unit 230, a parameter file processing unit 232, and an image signal expansion unit 240. An audio signal expansion unit 260, an image signal processing unit 242, D / A conversion units 244 and 262, a display control unit 250, a display unit 252, a speaker 264, and the like.

  The recording medium control unit 202 is an example of a data reading unit, and controls reading of image data and setting information recorded on the recording medium 204. The recording medium 204 is, for example, an optical recording medium (CD, DVD, etc.), a magneto-optical disk, a magnetic disk, a semiconductor storage medium, etc., and recorded image data and recorded audio data are recorded. The recording medium control unit 202 and the recording medium 204 may be configured to be detachable from the playback device 200. When the data file and the parameter file generated and recorded by the recording device 100 are recorded on the recording medium 204, the data file and the parameter file are read by the recording medium control unit 202. The data file is output to the data file processing unit 230, and the parameter file is output to the parameter file processing unit 232.

  The CPU 210 functions as an arithmetic processing device and a control device according to a program, and can control processing of each component provided in the playback device 200. For example, the CPU 210 outputs a signal to the driver 212 based on the operation of the operation unit 220 to drive the recording medium control unit 202. Further, the CPU 210 controls each component of the playback device 200 based on a signal from the operation unit 220. In the present embodiment, the number of CPUs 210 is only one. However, the CPU 210 may be composed of a plurality of CPUs, such as a signal system command and an operation system command executed by separate CPUs.

  The driver 212 drives the recording medium control unit 202. The driver 212 drives the recording medium control unit 202 based on reproduction control of the recording medium 204 in the CPU 210.

  The operation unit 220 includes a power switch, a playback operation button, various setting keys, and the like, and is used by the user to perform operations, various settings, and the like of the playback device 200. For example, the reproduction operation button is a button for starting reproduction of image data.

  The data file processing unit 230 decomposes the data file output from the recording medium control unit 202 into compressed RAW format image data and compressed audio data. The data file processing unit 230 outputs image data to the image signal expansion unit 240 and outputs audio data to the audio signal expansion unit 260.

  The parameter file processing unit 232 acquires parameters from the parameter file output from the recording medium control unit 202 and outputs the parameters to the image signal processing unit 242.

  The image signal decompressing unit 240 acquires the compressed image data output from the data file processing unit 230. When the image data has been subjected to lossless compression processing by the image signal compression unit 108 of the recording apparatus 100, the image signal decompression unit 240 decompresses the compressed image data and acquires the original image data. The original image data is the same as the image data before the lossless compression process output from the A / D conversion unit 106 in the recording apparatus 100. The image signal expansion unit 240 outputs the acquired image data to the image signal processing unit 242.

  The image signal processing unit 242 is an example of an image processing unit, and is necessary when the image data output from the image signal expansion unit 240 is reproduced in units of frames based on the parameters output from the parameter file processing unit 232. Perform image processing. For example, the image data is subjected to white balance control, exposure control, and the like based on parameters.

  The image signal processing unit 242 includes a YC conversion unit 243. For example, when the image data is in the RAW format directly output from the image sensor 104 via the A / D conversion unit 106, the YC conversion unit 243 performs YC conversion processing on the RAW format image data. The YC conversion unit 243 can convert RAW format image data into a luminance signal and a color difference signal, and acquire the luminance signal and the color difference signal. RAW format image data cannot be displayed on the display unit 252 such as a monitor, but an image can be displayed on the display unit 252 by acquiring the luminance signal and the color difference signal by YC conversion. The luminance signal and the color difference signal acquired by the image signal processing unit 242 are output to the D / A conversion unit 244.

  The D / A converter 244 converts the luminance signal and color difference signal, which are digital data output from the image signal processor 242, into analog signals by analog conversion. The D / A converter 244 outputs an analog signal to the display controller 250.

  The display control unit 250 drives the display unit 252 and controls the output of the display unit 252. The display unit 252 includes display means such as an LCD (Liquid Crystal Display). The display unit 252 displays various setting screens of the playback apparatus 200 and images read from the recording medium 204. In the above description, the case where the display control unit 250 and the display unit 252 are provided integrally with the playback device 200 has been described, but the present invention is not limited to such an example. For example, the image signal from the D / A conversion unit 244 may be output to an external display device such as a television monitor.

  The audio signal decompressing unit 260 receives the compressed audio data output from the data file processing unit 230. Then, the audio signal decompressing unit 260 decompresses the compressed audio data, and acquires the decompressed audio signal. The audio signal expansion unit 260 outputs the acquired audio signal to the D / A conversion unit 262.

  The D / A conversion unit 262 converts the audio signal output from the audio signal expansion unit 260 into an analog signal by analog conversion. The D / A conversion unit 262 outputs an analog signal to the speaker 264.

  The speaker 264 outputs sound based on the analog signal output from the D / A conversion unit 262.

(Data structure of data file and parameter file)
Next, the data structure of the data file and the parameter file will be described with reference to FIG. FIG. 3 is an explanatory diagram showing the data structure of the data file and parameter file according to the present embodiment.

  The data file is configured by alternately recording image data related to an image for one frame and audio data for a period corresponding to one frame of the image. In the data file, image data and audio data are associated in units of one frame and arranged in time series. For example, when there are N frames from the first frame to the Nth frame, each data is arranged as shown in FIG.

  Thus, by arranging image data and audio data for each frame, the data file can be cut or connected in units of frames. In the present embodiment, RAW format image data that has not been subjected to compression processing in the time axis direction is handled. Therefore, according to the data file of the present embodiment, video editing in units of frames can be easily performed.

  In the parameter file, various parameters relating to image data and audio data are recorded. For example, basic information such as the pixel structure of image data, pixel size, image data compression format, total number of frames, etc. is recorded in the parameter file, and the basic number of audio data such as the number of bits, sampling frequency, number of samples, data compression format, etc. Information is recorded.

  In the parameter file, recording byte positions in the image data and audio data files of each frame are recorded. Furthermore, the parameters calculated by the parameter calculation unit 120 necessary for image processing of each frame image are recorded. FIG. 3 shows white balance correction values (WB values) and exposure correction values (EE values) as examples of parameters. In the parameter file, data positions, WB values, and EE values are arranged in chronological order for each frame unit. ing.

  Since the parameter file is generated and recorded before the data file is played, it is necessary to calculate various parameters for white balance control and exposure control by reading the parameter file when playing the data file. Absent. Therefore, the data file reproduction process is quick and easy.

  Since the parameter file can be read independently from the data file, the parameter file processing unit 232 supports each RAW image data for RAW image data for a plurality of frames corresponding to a desired series of scenes. The parameter to be set can be uniformly set to a specified value. For example, in order to suppress changes such as white balance and luminance correction at the time of reproduction of a series of scenes, calculations for averaging parameter fluctuations can be performed in advance. As a result, variations in color and luminance within a series of scenes can be suppressed, and smooth reproduction can be achieved.

(Operation of the first embodiment)
Next, parameter file generation processing of the recording apparatus 100 according to the first embodiment of the present invention will be described. FIG. 4 is a flowchart showing parameter file generation processing of the recording apparatus 100 according to the present embodiment.

  First, image data is read into the parameter calculation unit 120 via the A / D conversion unit 106 (step S101). Further, the parameter file generating unit 124 generates an initialized parameter file in which parameters can be written (step S102). At that time, the parameter file includes, for example, the basic information such as the pixel structure of the image data, the pixel size, the image data compression format, the total number of frames, the number of bits of the audio data, the sampling frequency, the number of samples, the data compression format, etc. Basic information is recorded.

  Then, the parameter calculation unit 120 sequentially calculates parameters necessary for image processing of each frame from the first frame that is the first recorded frame (steps S103 to S107).

  For example, first, n = 0 is set (step S103), and then n = n + 1 is calculated (step S104). Since n = 1 at first, the parameter of the first frame is calculated (step S105). Next, the calculated parameters of the first frame are recorded as parameter file data (step S106). Then, it is determined whether or not recording has been stopped (step S107). For example, steps S104 to S107 are repeated until the recording is stopped by pressing the recording stop button by the user.

  When the recording is stopped, the generation of the parameter file including the basic information of the image data and the audio data and the parameters from the first frame to the Nth frame which is the last frame recorded is completed, and the generated parameter file Is recorded on the recording medium 152, for example (step S108). The generation of the above parameter file may be performed simultaneously with the recording of the image data and the sound data, or may be performed after all the recording is completed.

  Next, data file generation processing of the recording apparatus 100 according to the first embodiment of the present invention will be described. FIG. 5 is a flowchart showing data file generation processing of the recording apparatus 100 according to the present embodiment.

  First, in the recording apparatus 100, image data is read via the image sensor 104, and audio data is read via the microphone (step S201). The image data is compressed by the image signal compression unit 108 and the audio data is compressed by the audio signal compression unit 116 (step S202).

  The data file generation unit 122 generates an initialized data file in which image data and audio data can be written (step S203). Then, the data file generation unit 122 sequentially records the image data and audio data of each frame as data file data from the first frame that is the first recorded frame (steps S204 to S208).

  For example, first, n = 0 is set (step S204), and then n = n + 1 is calculated (step S205). Since n = 1 at first, image data of the first frame is recorded (step S206), and audio data of the first frame that is the same frame as the image data is recorded (step S207). Then, it is determined whether or not recording has been stopped (step S208). For example, steps S205 to S208 are repeated until the recording is stopped by pressing the recording stop button by the user.

  When the recording is stopped, the generation of the data file including the image data and the audio data for each frame from the first frame to the Nth frame which is the last frame recorded is completed, and the generated data file is recorded, for example, The information is recorded on the medium 152 (step S209).

  Next, playback processing of the playback device 200 according to the first embodiment of the present invention will be described. FIG. 6 is a flowchart showing the playback process of the playback device 200 according to this embodiment.

  First, the recording medium control unit 202 reads a data file and a parameter file recorded on the recording medium 204 (step S301). Further, the parameter file processing unit 232 receives the parameter file and acquires parameters from the parameter file (step S302). Then, the parameter file processing unit 232 outputs the acquired parameters to the image signal processing unit 242.

  Then, the data file processing unit 230 acquires the data file and decomposes the data file into image data and audio data (step S303). The data file processing unit 230 outputs image data to the image signal expansion unit 240 and outputs audio data to the audio signal expansion unit 260.

  Further, the image signal decompression unit 240 decompresses the compressed image data, and the audio signal decompression unit 260 decompresses the compressed audio data (step S304). The decompressed image data is output to the image signal processing unit 242, and the decompressed audio data is output to the D / A conversion unit 262.

  Then, the image signal processing unit 242 acquires image data from the image signal decompression unit 240, acquires parameters from the parameter file processing unit 232, and then performs image processing necessary for reproduction of the image data for each frame based on the parameters. This is performed (step S305). Further, the YC conversion unit 243 of the image signal processing unit 242 converts the image data into an image signal including a luminance signal and a color difference signal (step S306).

  Then, the image signal is output to the display control unit 250 via the D / A conversion unit 244, and an image is displayed on the display unit 252. The audio signal is output from the D / A conversion unit 262 to the speaker 264, and the audio is output from the speaker 264 in accordance with the image displayed on the display unit 252 (step S307).

  According to the present embodiment, in the recording apparatus 100, parameters necessary for image processing at the time of reproduction corresponding to the image of each frame are calculated in advance. Then, the parameters are recorded as a parameter file different from the data file containing the image data and audio data. When image data and audio data are reproduced by the reproduction apparatus 200, parameters corresponding to the image of each frame are acquired from the parameter file, and the parameters are used for image processing of the image of each frame. As a result, since a procedure necessary for parameter calculation at the time of reproduction can be omitted, time required for moving image processing is shortened, and moving image processing is facilitated.

  In the image sensor 104 such as a CCD or a CMOS, a checkered color filter may be provided for each pixel for color photographing. In a conventional recording apparatus, predetermined image processing is performed from the checkered color filter pattern to generate a luminance signal and a color signal, and video recording is performed by compressing these signals. At the time of reproduction, the recorded compressed image information is expanded, and the luminance signal and the color signal are restored and displayed on a display device such as a monitor.

  On the other hand, in the recording apparatus 100 according to the present embodiment, the color filter pattern is directly converted into data, compressed, and recorded on the recording medium 152 without performing image processing on the color filter pattern (recording of RAW format image data). Then, the playback apparatus 200 reads image data from the recording medium 152 and decompresses the data to obtain a color filter pattern. During reproduction, the color filter pattern is subjected to predetermined image processing to generate a luminance signal and a color difference signal, and an image is displayed on a display unit 252 such as a monitor or a display device.

  That is, in the recording / reproducing method according to the present embodiment, the color filter pattern is directly recorded as data (the signal from the image sensor 104 is recorded as it is), so that the image quality can be recorded with no deterioration. Further, since the image signal processing unit 242 is provided on the reproduction apparatus 200 side, it is not necessary to perform image processing when recording a data file. Therefore, the circuit provided in the recording apparatus 100 is very simple and power consumption can be reduced.

  In the image processing for generating the luminance signal and the color difference signal from the color filter pattern, white balance information and luminance correction information are calculated from each color filter pattern. In a moving image, it is not desirable that these values change suddenly. Therefore, in a conventional recording apparatus, control is performed so that the white balance value and the luminance correction value calculated from each recorded image change gradually. In a conventional recording apparatus, after performing image processing using these values, processed image data is recorded on a recording medium. Therefore, white balance and brightness adjustment can be separately performed on the image data later. could not.

  On the other hand, in the recording / reproducing apparatus according to the present embodiment, the white balance information and the luminance correction information are recorded as data in a parameter file for each image of each frame. During reproduction, image processing is performed using these pieces of information, and white balance and brightness are adjusted. At this time, since the image data is recorded in the data file independently of the white balance information and the luminance correction information, the image quality can be adjusted by changing these values during reproduction. These adjustments can also be made by the user. In addition, even if these parameters are rewritten, the image data is not affected. Therefore, there is no problem even if the parameter file is rewritten with the adjusted values, and the reproduction process can be performed with the image quality set by the user. . In addition to white balance and luminance level, chromatic aberration can be corrected after shooting.

  Also, by playing back using a personal computer or the like, it is possible to use a software decoder according to technological progress rather than a hardware decoder that is difficult to rewrite, and high-quality playback is possible. Furthermore, since image processing is not performed at the time of recording but image processing is performed at the time of reproduction, recorded image data is reproduced with higher-quality image processing in accordance with the progress of image processing technology such as developing software for RAW format data. It becomes possible.

  The recorded data file has RAW format data that is recorded independently of a display device such as a monitor for display, and is converted into an arbitrary video signal (NTSC / HDTV, etc.) during playback. It is possible to output. Furthermore, the color space can be arbitrarily set during reproduction.

  In addition, according to the data structure of the data file of the present embodiment, since video and audio are recorded in units of frames, video and audio can be easily synchronized and edited.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to this example. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Understood.

  For example, in the above embodiment, the image signal compression unit 108 and the audio signal compression unit 116 are provided in the recording apparatus 100. However, the image signal compression unit 108 and the audio signal compression unit 116 may be omitted. At this time, image data and audio data that are not compressed are recorded in the data file. When image data and audio data that are not subjected to compression processing are recorded in the data file, the playback device 200 does not need to perform image data and audio data expansion processing in the image signal expansion unit 240 and the audio signal expansion unit 260. Become.

  In the above embodiment, the case where the playback apparatus 200 reads out the parameters recorded by the recording apparatus 100 and uses the parameters as they are for image processing has been described. However, the present invention is not limited to such an example. For example, the parameters used for the image processing may be arbitrarily set by correcting the parameters on the parameter file acquired by the recording apparatus 100. According to this embodiment, the image data is recorded in the data file in the RAW format, and the parameters are recorded in the parameter file different from the data file. Therefore, it is easy to process the image that the user desires to reproduce. is there.

  In the above embodiment, the YC conversion unit 243 is provided in the image signal processing unit 242 of the playback device 200, but the present invention is not limited to such an example. For example, a YC conversion unit is provided after the A / D conversion unit 106 of the recording apparatus 100, and the YC converted image signal is reversibly compressed, and the compressed YC converted image signal is converted into a data file for each frame. It may be recorded. At this time, parameters necessary for image processing are calculated based on the YC-converted image signal, and a parameter file is generated using the calculated parameters.

  As described above, when the data file having the image signal after YC conversion is generated, the playback apparatus 200 does not need YC conversion processing. Then, the image signal processing unit 242 of the playback device 200 acquires the image signal after YC conversion from the data file, and performs image processing based on the parameters calculated based on the YC converted image signal.

1 is a block diagram showing a recording apparatus according to a first embodiment of the present invention. It is a block diagram which shows the reproducing | regenerating apparatus concerning the embodiment. It is explanatory drawing which shows the structure of the data file and parameter file which concern on the embodiment. It is a flowchart which shows the production | generation process of the parameter file of the recording device which concerns on the same embodiment. It is a flowchart which shows the production | generation process of the data file of the recording device which concerns on the same embodiment. It is a flowchart which shows the reproduction | regeneration processing of the reproducing | regenerating apparatus concerning the embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Recording apparatus 102 Optical system 104 Image pick-up element 106,114 A / D conversion part 108 Image signal compression part 110,212 Driver 112 Microphone 116 Audio | voice signal compression part 120 Parameter calculation part 122 Data file generation part 124 Parameter file generation part 130 CPU
140, 242 Image signal processing unit 141, 243 YC conversion unit 142, 244, 262 D / A conversion unit 144, 250 Display control unit 146, 252 Display unit 150, 202 Recording medium control unit 152, 204 Recording medium 160 Memory 162 VRAM
170, 220 Operation unit 200 Playback device 210 CPU
212 Driver 230 Data File Processing Unit 232 Parameter File Processing Unit 240 Image Signal Expansion Unit 260 Audio Signal Expansion Unit 264 Speaker

Claims (11)

Based on the RAW image data sequentially output from the image sensor in units of frames, a parameter calculation unit that calculates, for each frame, parameters necessary for image processing when reproducing the RAW image data, and for each calculated frame A recording system including a medium recording unit that records corresponding parameters on the recording medium together with the RAW image data as a parameter file different from the RAW image data;
A data reading unit for reading each RAW image data and the parameter file recorded on the recording medium, and a parameter for obtaining parameters necessary for sequentially reproducing the RAW image data in units of frames from the read parameter file An imaging apparatus comprising: a file processing unit; and a reproduction system including an image processing unit that performs image processing necessary for reproduction of the corresponding RAW image data for each frame based on each parameter. In the reproduction system,
The parameter file processing unit uniformly sets a parameter corresponding to each RAW image data to a specified value for the RAW image data for a plurality of frames corresponding to a desired series of scenes. The imaging device according to claim 1. A parameter calculation unit that calculates, for each frame, parameters necessary for image processing when reproducing the RAW image data based on the RAW image data sequentially output from the image sensor in units of frames;
A recording apparatus comprising: a medium recording unit that records a parameter corresponding to each calculated frame on a recording medium together with each RAW image data as a parameter file different from the RAW image data. Data reading for reading out each RAW image data and parameter file from a recording medium in which RAW image data in units of frames and a parameter file including parameters necessary for image processing when reproducing each RAW image data are recorded And
A parameter file processing unit for acquiring parameters necessary for sequentially reproducing the RAW image data in units of frames from the read parameter file;
A playback apparatus comprising: an image processing unit that performs image processing necessary for playback of the corresponding RAW image data for each frame based on each parameter. The parameter file processing unit uniformly sets a parameter corresponding to each RAW image data to a specified value for the RAW image data for a plurality of frames corresponding to a desired series of scenes. The reproducing apparatus according to claim 4. Based on the RAW image data sequentially output from the image sensor in units of frames, parameters necessary for image processing when reproducing the RAW image data are calculated for each frame, and then parameters corresponding to the calculated frames are calculated. Recording processing step for recording the image data together with each RAW image data as a parameter file different from the RAW image data,
After reading each RAW image data and the parameter file recorded on the recording medium, obtaining parameters necessary for sequentially reproducing the RAW image data in units of frames from the read parameter file, And a reproduction processing step for performing image processing necessary for reproduction of the corresponding RAW image data on a frame-by-frame basis. In the reproduction processing step,
A parameter corresponding to each RAW image data is uniformly set to a specified value for the RAW image data for a plurality of frames corresponding to a desired series of scenes before the image processing. The imaging method according to claim 6. Based on the RAW image data sequentially output from the image sensor in units of frames, parameters necessary for image processing when reproducing the RAW image data are calculated for each frame, and then parameters corresponding to the calculated frames are calculated. Is recorded on a recording medium together with each RAW image data as a parameter file different from the RAW image data. After reading out each RAW image data and the parameter file from a recording medium in which RAW image data in units of frames and a parameter file including parameters necessary for image processing when reproducing each RAW image data are recorded Then, parameters necessary for sequentially reproducing the RAW image data in units of frames are obtained from the read parameter file, and then the corresponding RAW image data is necessary for reproduction for each frame based on the parameters. A reproduction method characterized by image processing. A parameter corresponding to each RAW image data is uniformly set to a specified value for the RAW image data for a plurality of frames corresponding to a desired series of scenes before the image processing. The reproduction method according to claim 9. A first step of calculating, for each frame, parameters necessary for image processing when reproducing the RAW image data based on the RAW image data sequentially output from the image sensor in units of frames, and each calculated frame A program for causing a computer to execute a second step of recording, on a recording medium, a parameter corresponding to the RAW image data as a parameter file different from the RAW image data.

Images