JP2018074208A - Electronic apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow for slow motion display only of a specific scene, and normal speed display of other scenes, without using high speed decode operation, by using time hierarchical coding.SOLUTION: An electronic apparatus has means for reproducing image data, and means for decoding time hierarchical image data, decodes the image data subjected to time hierarchical coding up to a layer of high frame rate for a specific period, and decodes up to a layer of low frame rate for other than the specific period, and by displaying all decoded images at a constant display frame rate, reproduces the specific period by slow motion effect.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an electronic apparatus capable of reproducing compressed image data.

  There has been proposed a method capable of simultaneously recording video shot at a high frame rate along with video shot at a normal frame rate. For example, Patent Document 1 introduces time hierarchical coding as a method of compressing and decoding video. The time layer coding is one of SVC (Scalable Video Coding). In temporal hierarchical coding using a hierarchical B picture proposed as an H.264 extension method, as shown in FIG. 2, a B picture (B picture: a picture that can be predicted from temporally preceding and succeeding pictures) is used as a reference picture. In addition, the reference structure is configured to be hierarchical.

  In the hierarchical B picture compression coding scheme, for example, in FIG. 2, the B2 picture can refer to the lower B1 picture and at the same time can be the reference picture of the upper B3 picture. With the time hierarchical code configured as described above, it is possible to reproduce videos having a plurality of different time sampling frequencies by using time scalability.

  One of the shooting methods for realizing smooth slow motion images is overcrank shooting used in movies. Overcrank shooting achieves a slow motion effect by shooting and recording at a frame rate faster than the display frame rate and then playing back the recorded video at the display frame rate.

  FIG. 4 illustrates a case where a double slow motion effect is to be realized at a display frame rate of 60 fps (frame per sec). Shooting is performed at 120 fps, and the captured frame is recorded so as to be displayed at 60 fps. When the recorded video is played back, a double slow motion effect is realized.

JP 2009-212759 A

  For example, when it is desired to express a bird's flapping scene with a slow motion effect, it is necessary to record an image shot at a high frame rate from before the flapping, since it is not known when the bird flutters in actual shooting. At this time, the reproduction method for displaying all images including the scene where the bird is not flapping in slow motion is not convenient in some cases. Assume that only a specific scene (swinging bird) is displayed in slow motion, and other scenes are displayed at normal speed. A conventional example at this time will be described with reference to FIG.

  Assume that pictures (swinging birds) to be displayed in slow motion are 4, 5, and 6. The photographing itself is performed at 120 fps. Since the encoded pictures have a reference relationship, the picture (0, 2, 4, 5, 6, 8, 10) to be displayed and the picture (3, 9) being referred to are once decoded, Thereafter, the display pictures are rearranged and displayed at 60 fps. Since it is necessary to decode a picture (3, 9) that is not displayed, the conventional method requires a decoding operation faster than the display frame rate.

  Therefore, the present invention uses temporal hierarchical coding to display only a specific scene in slow motion without using a high-speed decoding operation, and to display other scenes at normal speed. Objective.

  An electronic apparatus according to the present invention has means for reproducing image data and means for decoding time-hierarchized image data. The period is decoded up to a layer with a high frame rate, and is decoded to a layer with a low frame rate except for a specific period, and all the decoded images are displayed at a constant display frame rate, thereby the specific period. Is played with a slow motion effect.

  According to the present invention, by using temporal hierarchical coding, only a specific scene can be displayed in slow motion and other scenes can be displayed at normal speed without using a high-speed decoding operation.

It is a figure for demonstrating an example of the relationship between the encoding picture in the case of carrying out slow motion display only of the specific scene of the image | video by which the time hierarchy encoding was carried out. It is a figure for demonstrating time hierarchy encoding. 4 is a block diagram for explaining a configuration of an electronic device according to Embodiment 1. FIG. It is a figure for demonstrating the slow motion realization method in overcrank imaging | photography. It is a figure for demonstrating the method of displaying only a specific scene in slow motion.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiments.

[Embodiment 1]
FIG. 3 is a block diagram for explaining the configuration of the electronic apparatus according to the first embodiment. The electronic device according to the first embodiment operates as a recording / reproducing device.

  In FIG. 3, 401 is a lens, 402 is a main image sensor, 403 is a sensor correction unit, 404 is a development processing unit, 405 is a compression / decompression unit, 406 is a recording medium, 407 is a display unit, and 408 is a display device such as a television. 409, a sub-sensor, 410 a subject recognition unit, 411 a lens control unit, 412 a CPU, 413 a user operation unit, and 420 an entire image recording / playback control unit controlled by the CPU 412.

  At the time of shooting, an instruction for shooting is transmitted from the user operation unit 413 to the CPU 412. The CPU 412 instructs the image recording / playback control unit 420 to perform a shooting operation. The lens control unit 411 performs lens control such as focus and diaphragm on the lens 401. Image data input from the lens 401 is imaged by the main image sensor 402. The image data formed by the main image sensor 402 is corrected by a sensor correction unit 403 for scratches caused by the sensor, and then processed by the development processing unit 404 into an image to be recorded and displayed.

  The image data processed by the development processing unit 404 is encoded by the compression / decompression unit 405 and recorded on the recording medium 406. The image data processed by the development processing unit 404 is processed by the display unit 407 into image data that can be displayed on the display device 408 and displayed on the display device 408. The subject recognition unit 410 recognizes, for example, a human face, an animal, or an object from the output image data of the development processing unit 404 and outputs it to the lens control unit 411 as lens control information. The sub sensor 409 forms an image of the image data input from the lens 401, generates in-focus information, for example, and outputs it to the lens control unit 411.

  At the time of reproduction, a reproduction instruction is transmitted from the user operation unit 413 to the CPU 412. The CPU 412 instructs the image recording / playback control unit 420 to perform a playback operation. The compression / decompression unit 405 reads the encoded image data from the recording medium 406 and decodes it. The decoded image data is processed by the display unit 407 into image data that can be displayed on the display device 408 and displayed on the display device 408.

  Details of the encoding / decoding method of the compression / decompression unit 405 will be described with reference to FIG. The compression / decompression unit 405 performs encoding / decoding of image data using time hierarchical encoding, which is one of SVCs. In FIG. 1, description will be made by taking an example of time layer coding having two time layers. Assume that the first layer is TID0 and is encoded at an interval of 60 fps. Assume that the second layer is TID1, and pictures taken between pictures in the first layer are encoded.

  Here, B2, P2, and B3 are pictures to be played back in slow motion. In the encoded data recorded by using the time hierarchy coding having two hierarchies, at the time of reproduction, only the pictures (I, P0, P1, P2, B2, P3, B3, P4) to be displayed are decoded. Next, the decoded image data is rearranged in the display order (I, P0, P1, B2, P2, B2, P3, P4) and displayed at 60 fps. Thus, a double slow motion effect can be obtained during the display period of the B2, P2, and B3 pictures.

  That is, only the TID0 layer is used during the normal playback period, and both TID0 and TID1 layers are used during the slow motion playback period. By using temporal hierarchical coding, the picture to be decoded matches the picture to be displayed, so that a high-speed decoding operation is not necessary.

  There is a method for designating a period during which slow motion playback is performed, by designating a period in which the user wants to display the slow motion as an in point / out point. In addition, as a method for specifying the slow motion period, there is a method for specifying the slow motion period according to the information of the subject recognized by the subject recognition unit 410. The compression / decompression unit 405 uses the motion vector of the image as the code of the image. As a method of specifying the slow motion period at the time of recording, there is a method of specifying according to the information of the motion vector calculated by the compression / decompression unit 405.

  When a slow playback period is specified during recording, if the IN point is set to an I picture, it is convenient for cueing playback and editing during the slow motion period. When specifying the period for slow motion playback during playback, the same effect can be obtained by specifying the IN point only for the I picture. Further, when the slow motion playback period is specified at the time of playback, and the IN point is not an I picture, the same effect can be obtained by re-encoding so as to become an I picture.

  In the first embodiment, an electronic device having a camera function is illustrated, but the same effect can be obtained even in an electronic device having no camera function.

401 Lens 402 Main image sensor 403 Sensor correction unit 404 Development processing unit 405 Compression / decompression unit 406 Recording medium 407 Display unit 408 Display device 409 Sub sensor 410 Subject recognition unit 411 Lens control unit 412 CPU
413 User operation unit 420 Image recording / playback control unit

Claims (7)

Means for reproducing image data;
Means for decoding time-layered image data,
For the time-layer encoded image data, during a specific period, decoding is performed up to a layer with a high frame rate,
Decodes to a layer with a lower frame rate except for a specific period,
An electronic apparatus, wherein the specific period is reproduced with a slow motion effect by displaying all decoded images at a constant display frame rate. The electronic device according to claim 1,
An electronic device characterized in that a user directly designates as a designation means for a specific period. The electronic device according to claim 1,
Having means for image recognition;
An electronic apparatus characterized by designating a period based on image-recognized information as means for designating a specific period. The electronic device according to claim 1,
Means for recording images with time hierarchical encoding;
An electronic apparatus characterized by designating a period based on motion vector information used at the time of encoding as means for designating a specific period. The electronic device according to claim 1,
Means for recording images with time hierarchical encoding;
An electronic apparatus characterized by encoding a start picture as an I picture when a specific period is specified at the time of encoding. The electronic device according to claim 1,
An electronic apparatus characterized by limiting a start picture to an I picture when a specific period is specified for encoded image data. The electronic device according to claim 1,
Further comprising means for re-encoding the decoded image data;
An electronic apparatus, wherein when a specific period is specified for encoded image data, if the start picture is other than an I picture, reencoding is performed so that the picture becomes an I picture.