JP2016039578A - Image transmission apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate small capacity data for a pre-recording function of a moving image in a configuration for transmitting RAW moving image data.SOLUTION: The image transmission apparatus is so configured that a moving image of a pre-recording period before an imaging instruction is stored in a temporary storage memory in a state of being developed and compressed, and when transmitting RAW moving image data in accordance with the imaging instruction, the compressed moving image data of the pre-recording period of the RAW data as well as the RAW data are read from the temporary storage memory and output.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image transmission apparatus that transmits moving image data.

  In recent years, in an imaging device, moving image data based on a captured moving image may be transmitted to an external storage device such as a cloud server and recorded. In moving image data transmitted in such a use environment, a captured image acquired by an imaging unit is generally developed and processed by MPEG or H.264. The video is compressed and encoded using a moving image compression technique such as H.264 / AVC.

  On the other hand, some image pickup apparatuses record a picked-up image acquired by an image pickup unit in an undeveloped state. A moving image in an undeveloped state is called a raw moving image (RAW moving image), which means the raw state of the image. Patent Document 1 discloses an apparatus for recording RAW moving image data before development processing.

  By recording the RAW moving image data, it is possible to perform a desired development process with a personal computer or the like when the RAW moving image data is reproduced. That is, an image can be generated by a user-specific development process that does not become a development process unique to the imaging apparatus. For example, it is possible to generate a desired image from the recorded RAW moving image data by generating an image having an image quality suitable for the characteristics of the user's own monitor, or by adjusting the tone to a desired tone or hue. .

JP 2009-055335 A

  By the way, the amount of data of RAW moving image data is enormous. In order to transmit the RAW moving image data to an external storage device such as a cloud server, it is desirable to transmit the RAW moving image data immediately after shooting through a high-speed communication path, and to shorten the period for buffering the RAW moving image data as much as possible. However, the imaging apparatus may be required to have a function of recording retroactively from a recording start instruction to a moving image a predetermined time ago (referred to as a pre-record function). With the pre-recording function, even if the instruction to start recording is not in time when a decisive scene occurs, for example, scenes from several seconds ago are stored in memory, so there is an advantage that the decisive scene can be recovered with it. is there.

  When the pre-recording function is executed when shooting a RAW moving image, it is necessary to continuously buffer the RAW moving image data in the memory for a predetermined time. In order to constantly store RAW moving image data for a predetermined time, a huge storage memory such as a DRAM is required. This is directly connected to an increase in hardware scale and cost for a mobile device such as an imaging apparatus, which is not preferable for a user.

  Therefore, an object of the present invention is to generate a small amount of data for a moving image pre-recording function in a configuration for transmitting RAW moving image data.

  In order to achieve the above object, an image transmission apparatus according to the present invention unprocesses the moving image data in accordance with a sensor that performs imaging to generate moving image data and a shooting instruction that indicates the start of recording or transmission of moving images Transmission means for outputting in the state of RAW data representing an image, development processing means for developing the moving image data, compression encoding means for compressing the developed moving image data, and a predetermined time past the shooting instruction And a temporary storage memory for storing the moving image data in a state compressed by the compression encoding unit, and the transmission unit replays the moving image in the pre-recording period of the RAW data to be output. The compressed moving image data stored in the temporary storage memory is output.

  According to the present invention, a moving image pre-recording function can be realized with a small amount of data in a configuration for transmitting RAW moving image data.

1 is a block diagram of an image transmission apparatus according to an embodiment of the present invention. It is a flowchart of the operation | movement which concerns on this embodiment. It is explanatory drawing of the RAW moving image data and development restoration data which are transmitted. It is explanatory drawing of a development process.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram of an image transmission apparatus according to an embodiment of the present invention. Hereinafter, as an example of the image transmission apparatus according to the present embodiment, an example applied to the imaging apparatus 100 that can capture moving images and transmit moving image data will be described.

  In FIG. 1, an image transmission apparatus (imaging apparatus 100) includes a lens 1 serving as a light incident path, and a sensor 2 for imaging such as a CCD or CMOS that converts light into an electrical signal. Here, an A / D conversion function for converting an analog signal into a digital signal is also included. The imaging apparatus 100 further includes a defect correction unit 3 that complements the influence of a sensor defect included in the signal, and a gain adjustment unit 4 that adjusts the gain of the signal according to the brightness of the subject. In this embodiment, the signal after gain adjustment is referred to as RAW data representing an undeveloped (raw) image signal after imaging. Since the imaging apparatus 100 can capture a moving image, the RAW data when the moving image data is captured is RAW moving image data. Note that a processing process until RAW data is generated is referred to as preprocessing.

  The imaging apparatus 100 further adjusts the white balance, γ processing, and converts the RGB signal from luminance (Y) and color difference (Cr, Cb) to the RAW data output from the gain adjustment unit 4. And a YC processing unit 6 that performs aperture processing and the like. The imaging apparatus 100 further includes a panel display unit 7 that displays the image signal after YC processing as an image, MPEG and H.264. It has a compression encoding unit 8 that converts an image by a compression technique such as H.264 / AVC. The image data obtained by compression by the compression encoding unit 8 becomes low-rate compressed image data and is recorded by the temporary storage memory 9.

  The temporary storage memory 9 has a memory capacity capable of storing compressed image data as a moving image for a predetermined time period of T seconds (for example, 3 seconds) or more. Used at runtime. In the pre-recording function according to the present embodiment, the past compressed moving image data is accumulated for T seconds from the “shooting instruction” meaning the start of recording or transmission of moving images, and the timing of the shooting instruction is determined according to the shooting instruction. The moving image for T seconds is read back from the temporary storage memory 9 and transmitted.

  The imaging apparatus 100 further includes a selection unit 10 that selects RAW data and compressed image data, and a transmission processing unit 11 that converts the input-selected RAW data or compressed image data into a format suitable for transmission and recording. The image forming apparatus also includes a development information adding unit 12 that adds related information related to parameters for generating compressed image data to the compressed image data. The output side of the transmission processing unit 11 is connected to a wireless or wired transmission interface, and can transmit and record the compressed video data to a cloud server or an external storage. The transmission processing unit 11 may be directly connected to a local recording medium.

  The imaging apparatus 100 further includes a recording instruction key 13 for receiving an input of an imaging instruction (recording instruction), a mode key 14 for determining an operation mode of the imaging apparatus 100, and a system controller 15 including an arithmetic circuit such as a CPU. Have Here, when there is an instruction from the recording instruction key 13, the operation of each block of the imaging apparatus 100 is controlled by an instruction from the system controller 15. Further, development parameters are set by the system controller 15.

  FIG. 2 is a flowchart of the operation of the image transmission apparatus according to this embodiment. This flowchart is executed by control of an arithmetic circuit in the system controller 15.

  After the start (STEP 1), it is determined whether or not the mode key 14 is set to the RAW mode (RAW moving image shooting mode) (STEP 2). If the mode is not the RAW mode, the operation shifts to a transmission mode for normal compressed data (compressed moving image data). At this time, the selection unit 10 always selects compressed data, and the compressed moving image data is transmitted to the outside or a recording medium. On the other hand, if the RAW mode is determined in STEP 2, it is determined whether the setting of the mode key 14 is a mode in which the pre-recording function is valid (pre-record mode) (STEP 3).

  If it is not the pre-recording mode, normal RAW data is transmitted (pre-recording in response to the recording instruction is not performed). At this time, the selection unit 10 always selects RAW data, and the RAW moving image data is transmitted to the outside or a recording medium. On the other hand, if it is determined in STEP 3 that the pre-recording mode is selected, the recording pause state in which recording can be performed at any time is maintained, the captured RAW moving image data is developed and compressed, and the compressed data is stored in the temporary storage memory 9. (STEP 4). The temporary storage memory 9 is updated so as to always store the latest T seconds (for example, 3 seconds) and waits for reception of a recording instruction (STEP 5).

  When a recording instruction is given in STEP 5, the selection unit 10 first selects the compressed data side, reads the compressed moving image data for T seconds stored in the temporary storage memory 9, and the transmission processing unit 11 performs an external cloud server or recording. It is transmitted to the medium (STEP 6). At this time, the development information adding unit 12 adds the parameter information of the development processing to the compressed data and sends it. The information added here may be parameter information of reverse development processing for returning the development processing to the original.

  When transmission of the compressed moving image data for T seconds from the temporary storage memory 9 is completed (STEP 7), subsequently, the selection unit 10 selects the RAW data side. Then, the RAW moving image data after the timing of the recording instruction is transmitted to an external cloud server or recording medium by the transmission processing unit 11 (STEP 8). Transmission of RAW data continues until a recording end instruction is given (STEP 9).

  A description of transmission data created by this series of processing flows and how to use the information will be described with reference to FIG. FIG. 3 is an explanatory diagram of transmitted RAW moving image data and development restoration data.

  FIG. 3 (a) is an explanation of a RAW data transmission method when the pre-recording function is valid. In response to the recording instruction, first, the compressed data in the temporary storage memory 9 is transmitted. If it is a 60 field image, it will become 180 fields in 3 seconds. N in the figure indicates a screen counter. Assuming that the recording instruction timing is N = 0, compressed data of a screen with N ranging from −180 to −1 is read from the temporary storage memory 9 and transmitted. At this time, as described above, parameter information obtained when the compressed data is developed by the development information adding unit 12 is added to the compressed data and sent. Thereafter, after the timing of the recording instruction (N = 0, 1, 2, 3,...), RAW data is transmitted instead of the compressed data. This state is continued until the recording instruction is canceled (until the recording stop is accepted).

  FIG. 3B shows data processing when the RAW data that has been transmitted to the personal computer or stored in the cloud server or the like via the network is generated as a file. Yes. This process is executed by an external computer or terminal, but the main body of the imaging apparatus 100 may have the same function. As processing of the file, a computer (not shown) first decodes the compressed data in accordance with the compression format, and restores the moving image. Thereafter, based on the parameter information of the development processing added to the compressed data, the reverse processing of the development performed at the time of developing the compressed data is performed (reverse development), and the restored moving image becomes the original RAW data. To pseudo RAW data similar to the above. At this time, it is not reversible and does not partially return to the original, so it is referred to as pseudo RAW data. By combining the pseudo RAW data and the subsequent RAW data, a RAW moving image file in which the pre-recording period is also RAW data is completed. By performing development desired by the user on the completed RAW file, it can be reproduced as a moving image having a desired image quality.

  FIG. 4 is an explanatory diagram of development processing. FIG. 4A is a detailed block diagram illustrating an example of development processing performed by the development processing unit 5. The white balance processing circuit 51, the γ (gamma) processing circuit 52, and the color difference matrix processing circuit 53 generate luminance (Y) and color difference (Cr, Cb) signals and send them to the YC processing unit 6. 4B shows input / output characteristics of the γ (gamma) processing circuit 52, and FIG. 4C shows an example of an input / output conversion formula of the color difference matrix processing circuit 53.

  In the reverse development processing, processing for returning the processing by each parameter is performed. The completed pseudo RAW data file and RAW data file are combined into one file. It is possible to change the data of the file to an image quality suitable for a monitor such as a television, to change the color to an effective color, or to adjust the development process so that a desired image is obtained. Therefore, the moving image before the shooting instruction (pre-recording period) can obtain a moving image of image processing that is continuous with the moving image after the recording instruction, although the image quality is slightly inferior to that after that.

  In the present embodiment, the RAW data is data after gain adjustment, but the same effect can be obtained even in the case of data before gain adjustment.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

DESCRIPTION OF SYMBOLS 1 Lens 2 Sensor 3 Scratch correction part 4 Gain adjustment part 5 Development processing part 6 YC processing part 7 Panel display part 8 Compression encoding part 9 Temporary storage memory 10 Selection part 11 Transmission processing part 12 Development information addition part 13 Recording instruction | indication key 14 Mode key 15 System controller 100 Imaging device

Claims (3)

A sensor that captures and generates video data;
Transmission means for outputting the moving image data in the state of RAW data representing an undeveloped image in response to a shooting instruction indicating the start of recording or transmission of the moving image;
Development processing means for developing the moving image data;
Compression encoding means for compressing the developed moving image data;
A temporary storage memory that stores moving image data for a predetermined time past the shooting instruction in a state compressed by the compression encoding means;
The image transmission apparatus, wherein the transmission means outputs compressed moving image data stored in the temporary storage memory so that a moving image of a pre-recording period of the RAW data to be output can be reproduced.   2. The transmission means, when transmitting the compressed moving image data from the temporary storage memory, outputs information related to a parameter relating to a development process when creating the compressed moving image data together with the compressed moving image data. The image transmission apparatus described.   The transmission means, when transmitting the compressed moving image data from the temporary storage memory, outputs information relating to a reverse development parameter for restoring the development processing by the development processing means together with the compressed moving image data. The image transmission apparatus according to claim 1.

Images