JP2011097366A - Image processor and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To display an image by HDR representation without failure even when suspension is instructed during reproduction of an HDR image where its gradation properties are improved by displaying a plurality of images of different exposure in predetermined order. <P>SOLUTION: An image processor 102 includes an HDR detection unit 104 which determines whether an image to be detected is an HDR image or not, a writing selection unit 106 which selects the data written in a frame memory unit 108, and a reading selection unit 107 which selects the data to be read out from the frame memory unit 108. When suspension is designated by an operation unit 110, a CPU 105 controls the writing selection unit 106 to read out the image data of a plurality of frames constituting one image with the video data detected by the HDR detection unit 104 repeatedly from the frame memory unit 108 and to output the image data to a display unit 109. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and an image processing method using, for example, a wide dynamic range technique, and relates to a technique for processing moving image data in which one image is composed of image data of a plurality of frames.

It is called HDR (High Dynamic Range) that enriches the gradation expression of the combined image by combining a plurality of different images exposed under different shooting conditions in consideration of the exposure state of each image. Technology is drawing attention.
Currently, HDR images are mainly used in movies, special effects, 3D (three-dimensional) video works, and some high-end photos. In general, HDR images are synthesized using an information processing apparatus such as a computer. On the other hand, means for realizing HDR expression by displaying a plurality of images with different exposures in a predetermined order has been proposed. In this case, since image composition can be performed without subjecting the target image to complicated arithmetic processing, it is possible to perform simple and high-real-time HDR display. For example, FIG. 6 shows a moving image (hereinafter referred to as an HDR moving image) in which one image is composed of image data of a plurality of (3 in this example) frames with different exposures. In the figure, VSYNC represents a vertical synchronizing signal, and H, M, and L shown in FRAME represent image data sequences having different exposures. That is, H represents a frame with high exposure, M represents a frame with intermediate exposure, and L represents a frame with low exposure. The numbers following H, M, and L indicate the chronological order, and in this example, three sets of frames are displayed in order from 1 in synchronization with VSYNC.

  In playback of an HDR video by such a display method, if the user instructs the apparatus to pause in a specific scene, one frame with uneven exposure is displayed. That is, depending on the timing of the pause instruction, there is a possibility that an overexposed or blackened image may be displayed, which causes a problem of giving the user unpleasant feeling. Therefore, a method for avoiding the display of a broken and difficult-to-see still image during pause during video playback is disclosed in, for example, Patent Document 1, and pauses during playback of a pull-down converted interlaced video. Is mentioned. In this example, in a configuration in which re-conversion to progressive video is not performed, a pause image in which blurring occurs between fields is prevented from being displayed. Memory control is performed so that the top field and the bottom field belonging to the same frame are read out, and these are alternately read out so that a blurred pose image is not displayed.

JP 2006-319481 A

However, the conventional technique has a problem that it cannot cope with a case where a plurality of images having different exposures are used. The above-described technique only detects a top field and a bottom field belonging to the same frame and outputs them alternately. Therefore, the technique cannot be applied to synthesis of a plurality of images with different exposures such as an HDR video. Also, how many images with different exposures constitute one image differs depending on the video source. For this reason, when a means for detecting the number of constituent images is not provided, it is difficult to display a still image obtained by combining a plurality of images with different exposures without failure.
Therefore, an object of the present invention is to display an image in HDR expression without failure when an instruction to pause image reproduction is issued in an HDR video processing apparatus.

  In order to solve the above problems, an apparatus according to the present invention is an image processing apparatus that processes moving image data in which one image is composed of image data of a plurality of frames, and instructs to pause the reproduction of the moving image data. Operating means, memory means for storing the image data of the plurality of frames, and when the operation means instructs to pause, the image data of the plurality of frames are sequentially read out from the memory means. Control means for controlling to output repeatedly.

  According to the present invention, even when a user instructs to pause image reproduction, an image expressed in HDR can be displayed without failure.

FIG. 4 is a block diagram schematically showing an example of the configuration of an apparatus for explaining the first embodiment of the present invention in conjunction with FIGS. 2 and 3. It is a flowchart explaining the temporary stop process of a HDR moving image. It is a timing chart explaining the temporary stop process of a HDR moving image. FIG. 6 is a block diagram schematically showing a configuration example of an apparatus for explaining a second embodiment of the present invention in conjunction with FIG. 5. 10 is a flowchart for describing HDR moving image printing processing. It is a timing chart explaining the example of input of a HDR animation.

[First embodiment]
Hereinafter, an image processing apparatus according to a first embodiment of the present invention will be described with reference to the accompanying drawings. FIG. 1 is a block diagram schematically showing an image transmission apparatus and an image processing apparatus as an example of a system configuration to which the present embodiment is applied.
In this system example, a video camera is used as the image transmitting apparatus 101, and a television broadcast receiving apparatus is used as the image processing apparatus. The image transmission device 101 transmits HDR moving image data or normal moving image data to the image processing device 102. In this example, the image transmission apparatus 101 transmits the vertical synchronization signal VSYNC, the video data 111, and the HDRSYNC signal 112 to the image processing apparatus 102 at the time of transmission. The HDR SYNC signal 112 is a signal indicating how many HDR moving images form one image (still image) and indicating transmission timing, and is output to the image processing apparatus 102 in synchronization with the vertical synchronization signal VSYNC. Is done. The image processing apparatus 102 receives the HDR SYNC signal 112 and performs processing according to the signal.

The image processing apparatus 102 includes an input unit 103, an HDR detection unit 104, a CPU (central processing unit) 105, a write selection unit 106, a read selection unit 107, a frame memory unit 108, a display unit 109, and an operation unit 110. The input unit 103 is an input interface unit that receives the vertical synchronization signal VSYNC, the video data 111, and the HDR SYNC signal 112. The vertical synchronization signal VSYNC and the video data 111 are sent to the write selection unit 106, and the HDR SYNC signal 112 is sent to the HDR detection unit 104.
The HDR detection unit 104 receives the HDR SYNC signal 112, detects whether or not the target moving image data is HDR moving image data, and how many frames form one image in the case of the HDR moving image. To detect. Then, the HDR detection unit 104 notifies the CPU 105 of the detection result.

The CPU 105 secures the memory area of how many frames in the frame memory unit 108 according to the detection result from the HDR detection unit 104, that is, information indicating how many images constitute one image in the case of an HDR video. Decide whether to accumulate data. For example, in the case of an HDR video that constitutes one image with three frames, if the input / output frame delay is a time corresponding to one frame, it is necessary to secure a frame memory area for a total of four frames. The CPU 105 issues a command to the write selection unit 106 and the read selection unit 107 according to the above determination, and controls selection of the write area and the read area of the frame memory unit 108.
In response to a command from the CPU 105, the write selection unit 106 writes the video data from the input unit 103 into the frame memory unit 108. In response to a command from the CPU 105, the read selection unit 107 reads video data from the frame memory unit 108 and transmits it to the display unit 109 at the subsequent stage.

The frame memory unit 108 secures a memory area for the above-described frame (four frames in this example). The memory area is divided for each frame, and in the example of FIG. 1, memory areas with numbers 0 to 3 are used so that data for four frames can be stored in the frame memory unit 108. For example, at a certain point in time, an area indicated by “0” in the frame memory unit 108 becomes a write area, and an area indicated by “1” becomes a read area. When data of the next frame is input, an area indicated by “1” is a writing area, and an area indicated by “2” is a reading area. As described above, by securing a memory area for four frames and using it while sequentially shifting the writing area and the reading area, it is possible to always accumulate data for three frames. Here, for the sake of simplicity, an example in which the memory area for 4 frames is secured in the frame memory unit 108 has been described, but a memory area for “3 frames + several lines” is secured in consideration of the blanking period. May be.
The display unit 109 processes the video data read from the read selection unit 107 and displays an image on the display screen.

  The operation unit 110 is an interface unit that receives user operation instructions. For example, when an ON / OFF instruction for pause of image reproduction is notified from the operation unit 110 to the CPU 105, the CPU 105 controls writing and reading of data. When the pause of reproduction is instructed, the writing area is fixed, and control is performed so that the data reading operation for the reading area is repeated with 3 frames as one set. Details will be described later. Although not shown, the image processing apparatus 102 is provided with a storage unit for programs and control parameters that are interpreted and executed by the CPU 105.

Next, the pause process during playback of an HDR video will be described using the flowchart of FIG. This process is realized according to a program interpreted and executed by the CPU 105.
In S201, the process starts. In S202, HDR moving image data is reproduced and an image is displayed. In the next S203, it is determined whether or not there has been a pause instruction from the user. As a result, if there is a pause instruction, the process proceeds to S205. If there is no such instruction, the process proceeds to S204, and the process ends.

  In S205, images of a plurality of frames are repeatedly displayed within one cycle of the HDR SYNC signal. FIG. 3 is a timing chart showing the state. In the figure, VSYNC IN indicates the input VSYNC, FRAME IN indicates the data of the input frame, and HDR SYNC IN indicates the input HDR SYNC. . In the HDR moving image of this example, one image is composed of three frames, and the meanings of H, M, and L are as described in FIG. The numbers following H, M, and L are numbers given along the time series with 3 frames as a set. STOP SIGNAL is a pause signal for reproduction by the operation unit 110, and in this example, pause is instructed during the period of the frame M2 (ON instruction), and cancellation of the pause is instructed during the period of the frame L4. (OFF instruction). VSYNC OUT and FRAME OUT indicate output signals to the display unit 109. VSYNC OUT indicates the output VSYNC, FRAME OUT indicates the data of the output frame, and is output with a delay time of one frame with respect to VSYNC IN and FRAME IN.

  As shown in the STOP SIGNAL, when there is a pause instruction during the input of the frame M2, the output frame is output in the order of H2, M2, and L2. Thereafter, the frames H2, M2, and L2 are repeatedly output again without shifting to H3 at the time of switching by HDR SYNC. That is, the frames H2, M2, and L2 are repeatedly output in this order during the period up to the point when the input frame L4 is switched by HDR SYNC.

In 206 of FIG. 2, it is determined whether or not there is an instruction to cancel the pause. As a result, when there is no cancellation instruction, the process returns to S205 and a set of images is repeatedly displayed in three frames as described above. On the other hand, if there is an instruction to cancel the pause, the process proceeds to S207, and the image display is continued sequentially from the next HDR SYNC, and the process returns to S202. As shown in FIG. 3, if there is an instruction to cancel the pause when the frame L4 is input, the frames H5, M5, and L5 are output at the timing of VSYNC OUT following the HDR SYNC after the frame L4.
According to the first embodiment, even if there is a pause instruction by a user operation, an image by HDR expression can be displayed without failure by repeatedly displaying images of a plurality of frames constituting one image.
[Second Embodiment]

Next, a second embodiment of the present invention will be described with reference to FIGS. In the first embodiment, the display form when the HDR video is paused has been described, but in the second embodiment, a case where an image is printed during the pause will be described. In the case of printing, since the HDR effect cannot be obtained by a method of displaying frame images in a predetermined order, it is necessary to generate an HDR image by calculation. At this time, the HDR image generated by the calculation is displayed in the preview display before printing.
FIG. 4 is a schematic block diagram showing a configuration example when the second embodiment is applied. In the following, components having the same functions as those shown in FIG. 1 will be omitted by using the same reference numerals as those in the first embodiment, and differences will be mainly described.

  In response to a command from the CPU 105, the composition selection unit 402 of the image processing apparatus 401 selects three frames of image data that are repeatedly displayed by the pause process described above. The selected three frames of image data are transferred to the HDR synthesizing unit 403. The HDR synthesizing unit 403 performs an operation for synthesizing three frames of image data, and generates data for one HDR image. The generated HDR image data is sent to the display selection unit 404 and the printing apparatus 405. The printing device 405 receives the data and prints the HDR image. Upon receiving an instruction from the CPU 105, the display selection unit 404 selects the image data of the frame read out by the read selection unit 107 and outputs it to the display unit 109 when the image is displayed by the pause processing, and repeatedly displays the image. Further, the display selection unit 404 selects the HDR image data from the HDR synthesizing unit 403 and outputs the data to the display unit 109 when the print preview is displayed.

Next, an example of print processing during HDR image reproduction will be described using the flowchart of FIG. Note that the processing in S201 to S204 is the same as that in the first embodiment, and a description thereof will be omitted.
If a pause instruction is received by a user operation, the process proceeds to S205. Here, an image of a set of three frames is repeatedly displayed in the HDR SYNC. In this state, in next step S501, it is determined whether or not a print request has been made by a user operation. As a result, if there is no print request, the process proceeds to S506, but if there is a print request, the process proceeds to S502.

  In S502, the HDR synthesizing unit 403 receives the image data of the three frames selected by the synthesis selecting unit 402, and generates print preview image data by performing HDR synthesizing processing. When the preview image data is generated, the display unit 109 displays the preview image in S503. The next step S504 is a process for determining whether or not a print execution instruction has been issued by a user operation. If there is a print execution instruction (Yes in S504), the process proceeds to S505, and the HDR synthesizing unit 403 sends the print apparatus 405. Data is sent and printing processing is performed. If there is no print execution instruction (No in S504), the process proceeds to S506.

S506 is a process for determining whether or not an instruction to cancel suspension has been issued by a user operation. If there is an instruction to cancel the pause, the process proceeds to S207, and the image display is continued sequentially from the next HDR SYNC, and the process returns to S202. If there is no instruction to cancel the pause, the process returns to S205, and the images of the frames constituting the HDR image are repeatedly displayed.
According to the second embodiment, by performing a print preview display of an image that has been subjected to the HDR synthesis process, the user can determine whether to actually perform printing while confirming the image.

102 image processing apparatus 104 HDR detection unit 105 CPU
DESCRIPTION OF SYMBOLS 106 Write selection part 107 Read selection part 108 Frame memory part 109 Display part 110 Operation part 401 Image processing apparatus

Claims (5)

An image processing apparatus that processes moving image data in which one image is composed of image data of a plurality of frames,
Operation means for instructing to pause the reproduction of the video data;
Memory means for storing image data of the plurality of frames;
Control means for controlling the image data of the plurality of frames to be sequentially read out from the memory means and repeatedly output when a pause instruction is given by the operation means. Processing equipment. Detecting means for detecting input of moving image data in which one image is constituted by image data of a plurality of frames and detecting the number of frames constituting one image;
Write selection means for selecting image data of a frame to be written to the memory means;
Reading selection means for selecting image data of a frame to be read from the memory means,
When a pause instruction is given by the operation means, the control means controls the write selection means and the read selection means to form one image with the moving image data detected by the detection means. The image processing apparatus according to claim 1, wherein the image data of a plurality of frames is repeatedly output.   When the moving image data is detected by the detecting unit, the control unit controls the writing unit, and the image data of a plurality of frames constituting one image with the moving image data is stored in the memory unit for each frame. The image processing apparatus according to claim 2, wherein the image processing apparatus is stored in a memory area. When printing is instructed after an instruction for pause is given by the operating means, image data of a plurality of frames constituting one image is selected from the moving image data detected by the detecting means. A selection means;
The image processing apparatus according to claim 2, further comprising a combining unit that combines and outputs the image data selected by the selecting unit. An image processing method for processing moving image data in which one image is composed of image data of a plurality of frames,
Receiving an instruction to pause playback of the video data;
Storing image data of the plurality of frames in a memory;
And a step of sequentially reading out and sequentially outputting the image data of the plurality of frames from the memory when the pause instruction is issued.

Images