JP5084586B2 - Image recording / reproducing apparatus and image reproducing apparatus - Google Patents

Description

The present invention relates to an image recording / reproducing apparatus and an image reproducing apparatus for capturing an image at a high frame rate and reproducing the captured image.

  2. Description of the Related Art Conventionally, images are generally captured by digital cameras and digital video cameras by specifying various parameters. The image here includes both a moving image and a still image. Examples of various parameters include shutter speed, white balance, and exposure.

  These parameters are set at the time of shooting, and an image is shot with parameters according to the set values. Therefore, basically, one parameter such as shutter speed is determined for each image, and at the time of reproduction, an image based on the shutter speed setting at the time of shooting is reproduced.

  Since the above various parameters determine the characteristics of the image, it is possible to shoot as if the characteristics of the image were in the image by shooting with various parameters.

  For example, in the technique disclosed in Patent Document 1, when a moving image is shot, a frame with a high shutter speed value for a still image is shot one by one within a predetermined time, and inserted. It also realizes image recording suitable for still images.

  In recent years, it has become possible to take an image at an extremely high speed by increasing the speed of a sensor for taking an image. Until now, shooting of moving images at about 60 fps (frame per second) was the limit, but the feasibility of shooting at a frame rate of 600 fps or higher is increasing.

A technique has also been proposed in which a shutter speed is recorded in an image as metadata by using this sensor, and a frame is displayed by combining a plurality of frames from the shutter speed recorded as metadata during playback. .
JP-A-6-205284

  Since various parameters such as the shutter speed are set at the time of shooting, the viewer can only perform the playback depending on the parameters at the time of recording.

  For example, a moving image shot with a slow shutter speed is reproduced only as a moving image with a slow shutter speed. In other words, only reproduction reflecting the photographer's intentions and preferences was possible.

  Some viewers may wish to perform playback with parameters different from the photographer's preferences. It is easily conceivable that there is a case where it is desired to view an image with higher time resolution while viewing the reproduced image. At that time, the conventional technique cannot change the parameters at the time of shooting on the playback side for playback, so that there is a problem that the playback mode desired by the viewer cannot be provided.

  In order to solve such a problem, a technique has been considered in which the shutter speed is recorded as metadata in an image and is changed to an arbitrary value during reproduction. In this technique, an image is recorded at a high frame rate, the image is added based on a shutter speed value recorded as metadata, and a display image is created.

  However, in this technique, since an image is recorded at a constant frame rate, an addition process is indispensable when creating an image with a slow shutter speed. Therefore, depending on the image, the image quality of the displayed image may be deteriorated.

  Further, since the ratio of the gap time between frames increases by increasing the frame rate, the image quality of the recorded image may be deteriorated.

An object of the present invention is to provide an image recording / reproducing apparatus and an image reproducing apparatus that can provide reproduced images according to various viewing environments and improve the reproduced image quality.

In order to achieve the above object, an image recording and reproducing apparatus of the present invention, changes in the image recording and reproducing apparatus composed of an image recording apparatus and an image reproducing apparatus, the image recording apparatus, the shooting shutter speed for each image changing means for, in a certain time, and an imaging means for recording a plurality of images with the changed the imaging shutter speed, the image reproducing apparatus, that determine the playback shutter speed of the image to be reproduced decide means If, selection means for selecting an image to be used in accordance with the reproduction shutter speed determined in the previous Kike' constant means and generating means for generating images to be displayed on the display device from the image selected by said selection means comprising, if the image photographing shutter speed closer to the reproduction shutter speed is recorded, the selection means to the reproduction shutter speed If a shooting shutter speed image close to the playback shutter speed is not recorded, the selection means shot at a shooting shutter speed shorter than the playback shutter speed and different from each other. an image selecting multiple, said creating means is characterized that you create an image for displaying a plurality of images selected by the selecting means synthesizing and display device.
In order to achieve the above object, an image reproduction apparatus according to the present invention includes a reading means for reading a plurality of images shot at a plurality of different shooting shutter speeds recorded on a recording medium, and a reproduction shutter speed for the image to be reproduced. When an image with a shooting shutter speed close to the reproduction shutter speed determined by the determination means is recorded on the recording medium, a determination means for determining, and selecting an image with a shooting shutter speed close to the reproduction shutter speed, When an image based on the selected image is generated and an image with a shooting shutter speed close to the reproduction shutter speed determined by the determination unit is not recorded on the recording medium, the image is shorter than the reproduction shutter speed and is different from each other. Select multiple images shot at the shooting shutter speed, It characterized in that it has a generating means for generating an image based on the selected image.

According to the onset bright, it is possible to provide a reproduced image corresponding to different viewing environments, and it is possible to improve the playback quality.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram of an image recording / reproducing apparatus according to an embodiment of the present invention.

  In FIG. 1, the image recording / reproducing apparatus includes a recording apparatus (image recording apparatus) 100 and a reproducing apparatus (image reproducing apparatus) 110.

  The recording apparatus 100 is an apparatus that records and stores image information input from the outside. Specifically, a digital camera, a camcorder, etc. are mentioned. Each element constituting the recording apparatus 100 will be described below.

  The recording apparatus operation unit 101 is a unit by which a user operates a recording method during image recording or various operations of the recording apparatus 100. Specific examples of operations include operations for changing various settings during image recording, and recording start / stop operations. The recording device operating means 101 is assumed to be a button provided on the recording device 100, a remote controller, or the like.

  The recording device interface unit 102 receives information operated by the recording device operation unit 101 and transmits information to the setting information storage unit 103. Thereafter, the information received by the setting information storage unit 103 is accumulated (stored).

  The shutter speed control unit 104 performs a process of determining the shutter speed of each image based on the setting value at the time of recording stored in the setting information storage unit 103. In addition, the shutter speed control unit 104 notifies the encoder 108 of the determined shutter speed, and assists the process of superimposing the shutter speed information in the image.

  The shutter speed change processing unit 105 performs processing for controlling the shutter 106 according to the shutter speed value determined by the shutter speed control unit 104.

  The shutter 106 is a mechanism for blocking light input from the outside at regular intervals, and the sensor 107 accumulates light at intervals blocked by the shutter 106. Specific examples of the sensor 107 include a charge coupled device (CCD) sensor and a complementary metal oxide semiconductor (CMOS) sensor.

  The encoder 108 performs processing for converting the light accumulated in the sensor 107 into an image. The encoded result is compression encoded. As a compression method, moving images are MPEG-2 (Moving Picture Expert Group-2) or AVC (Advanced Video Coding) / H. H.264. An example of a still image is JPEG (Joint Photographic Expert Group). Further, RAW data stored without being compressed is also conceivable.

  Further, the encoder 108 performs processing for superimposing the shutter speed value determined by the shutter speed control unit 104 as metadata in the image.

  The recording medium 109 records the image data created by the encoder 108. Specific examples include an HDD, an SD card, a DVD, and the like.

  Next, the playback device 110 and each component constituting it will be described. The playback device 110 is a device that plays back the image data created by the recording device 100.

  The playback device operation unit 111 provides a function that allows the user to operate various processes and settings related to playback. Specifically, an infrared remote controller or the like is assumed. As examples of user operations, a playback start / end instruction, a playback method setting, and the like can be considered.

  The playback device interface unit 112 performs processing for receiving various types of operation information instructed by the playback device operating means 111.

  The frame creation control unit 113 performs processing for controlling a creation method of an image to be displayed on the display device 117 based on a user operation acquired through the playback device interface unit 112. The processing performed by the frame creation control unit 113 includes processing for determining which frame to use as an image to be displayed among the frames decoded by the decoder 114, and creating an image to be displayed from the selected frame. There is a process to determine the creation method.

  The determined frame selection information is transmitted to the frame selection unit 115, and the determined creation method is transmitted to the frame creation unit 116. Further, information used for control in the frame creation control unit 113 includes operation information from the user, metadata output from the decoder 114 and metadata superimposed on the image, and a display device information acquisition unit 118 from the display device 117. It is the information acquired via.

  The decoder 114 performs processing for decoding the image data created by the recording apparatus 100 and reproducing it as a frame. In addition, the decoder 114 extracts metadata information such as shutter speed information superimposed on the image data, and transmits the metadata information to the frame creation control unit 113.

  The frame selection unit 115 actually performs a frame selection process based on the image information used for display determined by the frame creation control unit 113.

  The frame creation unit 116 performs processing for creating an image to be displayed based on the frame creation method determined by the frame creation control unit 113.

  The display device 117 is a device that displays the display image created by the playback device 110.

  The display device information acquisition unit 118 performs processing for acquiring information on the display device 117. A specific example of the information is a frame rate value that can be reproduced by the display device 117.

  In FIG. 1, the recording device 100 and the playback device 110 are described as separate configurations, but a configuration in which the recording device 100 and the playback device 110 are integrated is also conceivable.

  FIG. 2 is a flowchart showing a procedure of image recording processing executed by the recording apparatus in FIG.

  First, in step S <b> 201, a user who uses the recording apparatus 100 sets a recording mode using the recording apparatus operation unit 101. Here, the recording mode indicates how the shutter speed is changed for each frame for recording. Hereinafter, the recording mode will be described with reference to FIG.

  FIG. 3 shows examples of two types of recording modes. The mode shown in FIG. 3A is the normal recording mode, and FIG. 3B is the moving image priority recording mode. In the diagram showing each mode, the horizontal axis indicates time, and the shutter speed time of a frame to be recorded with a rectangular horizontal width. The maximum shutter speed time that can be recorded by the recording apparatus 100 shown in the present embodiment is set to 1/480 seconds.

  The normal recording mode shown in FIG. 3A is a mode in which the shutter speed is taken at 1/480 seconds for all frames. The normal recording mode is a mode in which a frame is photographed at the maximum shutter speed value that the recording apparatus is capable of.

  The advantage of the normal recording mode is that an image having various shutter speed values can be created as a reproduced image by selecting and combining a plurality of frames. The disadvantage of the normal recording mode is that an image is divided into a plurality of images in the time direction and recorded, so that the time taken to capture light into the sensor as a whole is reduced and the image quality is deteriorated by a certain amount.

  The moving image priority recording mode shown in FIG. 3B includes a frame in which the shutter speed value is recorded with a shutter speed value suitable for reproducing a moving image, and the shutter speed is changed for each frame for recording.

  In the example of FIG. 3B, a frame having a plurality of types of shutter speeds is recorded within a certain period, and this is repeated to form an entire image. In FIG. 3, 1/60 seconds is described as a fixed period, and 1/480 seconds, 1/240 seconds, and 1/120 seconds are described as examples of a plurality of shutter speed values.

  The advantage of the moving image priority recording mode is that the number of times the shutter is closed is smaller than that in the normal recording mode of FIG.

  For example, when an image recorded in the normal recording mode is played back with a shutter speed of 1/120 seconds, all frames are recorded in 1/480 seconds, so 4 images shot in 1/480 seconds are recorded. It is necessary to use and regenerate by performing a synthesis process.

  However, in the case of an image recorded in the moving image priority recording mode, there is an image taken in 1/120 seconds. By using this as it is, it is possible to provide a reproduced image with better image quality.

  The disadvantage of the moving image priority recording mode is that there are frames shot with a somewhat slow shutter speed value, so that there is a lack of flexibility in selecting the shutter speed value during playback.

  The recording mode shown here is merely an example, and various other recording modes are conceivable.

  Returning again to FIG. 2, after the recording mode is set in step S201, actual image recording is started.

  In step S <b> 202, the user instructs the start of image recording by the recording apparatus operation unit 101 of the recording apparatus 100. After step S202, a process for switching the shutter speed for each image and recording the image is started.

  In step S203, the shutter speed of the image is determined by the shutter speed control unit 104 of the recording apparatus 100 based on the recording mode designated by the user. Thereafter, in step S204, the shutter speed change processing unit 105 changes the shutter speed.

  Step S203 functions as a first determination unit that sets the shooting shutter speed (shutter speed at the time of shooting) according to the recording mode. Step S204 functions as a changing unit that changes the shooting shutter speed determined by the first determining unit for each image.

  In step S205, an image corresponding to the shutter speed changed in step S204 is read from the sensor 107 and stored in the memory. Step S205 functions as a photographing unit that records a plurality of images at a changed photographing shutter speed within a certain time. In step S206, the image is encoded.

  From step S203 to step S206 described above, the process for creating one image is completed.

  In step S207, an end of image recording is determined. If not, the process returns to step S203 to continue image recording. When it is determined in step S207 that the image recording has ended, the process proceeds to step S208, and the recorded image is stored in the recording medium 109 in FIG. Then, this process ends.

  As described above, the image recording processing according to the recording mode shown in FIG.

  FIG. 4 is a flowchart showing the procedure of image reproduction processing executed by the reproduction apparatus in FIG.

  First, in step S401, the user starts image reproduction. The reproduction operation is performed by the reproduction apparatus operation unit 111.

  In step S402, the shutter speed value during reproduction is determined. The determination of the shutter speed value is performed by the frame creation control unit 113. Step S402 functions as a second determination unit that determines a reproduction shutter speed (shutter speed during reproduction) of an image to be reproduced.

  As a method for determining the shutter speed at the time of reproduction, a determination method by a user operation, a method that is automatically determined from the characteristics of the display device 117, a method that automatically determines according to an image to be reproduced, and the like are given.

  In the determination method by the user operation, the user sets the shutter speed value at the time of reproduction from the reproduction apparatus operation unit 111, and is determined according to the setting.

  The method of automatically determining from the characteristics of the display device 117 is determined by the frame creation control unit 113 based on the information of the display device 117 acquired by the display device information acquisition unit 118 from the display device 117. For example, when the frame rate that can be displayed by the display device 117 is 120 fps, the shutter speed value is set to 1/120 seconds, and so on.

  The method of automatically determining according to the image to be reproduced is a method of determining the shutter speed at the time of reproduction based on the image information acquired from the decoder 114. For example, when the image is a photograph of sports such as soccer, the movement in the photographed image is expected to be fast. In addition, although the three types of determination methods were demonstrated here, it is not restrict | limited to this.

  Based on the shutter speed determined in step S402, a method for creating a frame to be displayed is determined in step S403. The frame creation method is a method for synthesizing frame information used for creating a frame to be displayed.

  The frame information to be used is transmitted to the frame selection unit 115, and the synthesis method is transmitted to the frame creation unit 116 of FIG. The process performed in step S403 will be described with reference to FIG.

  FIG. 5 is a diagram illustrating an example when the image captured in the moving image priority recording mode of FIG. 3 is reproduced with a shutter speed value of 1/120 seconds in the recording apparatus of FIG.

  In the example of FIG. 5, since a frame having a shutter speed value of 1/120 seconds is recorded, a frame used for creating a frame to be displayed is one recorded 1/120 second frame. In addition, since the number of frames used is one, there is no need to combine them, and the frames are displayed as they are.

  After step S404, a flow for creating one display image is started. In step S404, an image used to create a frame used for display is selected. The processing in step S404 is performed by the frame selection unit 115.

  Step S404 functions as a selection unit that selects an image to be used in accordance with the reproduction shutter speed determined by the second determination unit. The selection means selects one recorded image having a shutter speed close to the reproduction shutter speed of the image to be reproduced. Further, the selection means performs addition processing on the reproduction shutter speed of the image to be reproduced, and selects a plurality of images having a shutter speed that is close to the reproduction shutter speed.

  In step S405, an image to be displayed is created. The process of step S405 is performed by the frame creation unit 116. Step S405 functions as a creation unit that creates an image to be displayed from the image selected by the selection unit. Thereafter, in step S406, the display frame is transmitted to the display device 117 and presented to the user.

  In step S407, it is determined whether or not reproduction is to be continued. If reproduction is to be continued, the process returns to step S404 and the image presentation process is continued. When the reproduction process is finished, this flow is finished.

  As described above, in the present embodiment, the playback device 110 can generate a playback image at a shutter speed corresponding to the playback situation.

  For example, when playing back a moving image, it is possible to provide a high-quality moving image by selecting a frame having a shutter speed value shot in a format suitable for the moving image.

  Further, when reproducing a still image, it is possible to reproduce a clear image with a high time resolution by selecting a frame having a shutter speed value faster than that of a moving image.

  Next, a playback method in the case where a frame recorded with a designated shutter speed value is not included in the recorded image when the playback device 110 plays back an image will be described.

  FIG. 6 is a diagram showing the process executed in step S404 of FIG.

  Specifically, FIG. 6 is a diagram illustrating an example when the image captured in the moving image priority recording mode of FIG. 3 is reproduced with a shutter speed value of 1/80 seconds in the recording apparatus of FIG.

  In the example of FIG. 6, since a frame having a shutter speed value of 1/80 seconds is not recorded, a process of selecting and combining a plurality of frames is necessary in order to create a frame to be displayed for reproduction.

  Here, two frames are used to create a frame to be displayed, that is, a recorded 1/240 frame (I) and a 1/120 second frame (II). By synthesizing the frames (I) and (II), a shutter speed value of 1/80 second can be created by synthesis. The display frame creation method is determined as addition processing.

  In this description, an example is shown in which there is a combination that matches the designated 1/80 second, but this is not restrictive. If there is no match, a method of selecting a value close to the designated shutter speed value can be considered. Further, regarding the number of frames to be selected, the method of selecting two frames is shown in the present description, but this is not restrictive.

  In addition, although the addition process is shown as an example of the synthesis process, the present invention is not limited to this.

  As described above, in the present embodiment, the playback device 110 can synthesize and create a shutter speed according to the playback situation, and provides a relatively high-quality playback image at an arbitrary shutter speed. It becomes possible.

1 is a block diagram of an image recording / reproducing apparatus according to an embodiment of the present invention. 6 is a flowchart illustrating a procedure of image recording processing executed by the recording apparatus. It is a figure explaining the recording mode of the image data recorded with the recording device in FIG. 3 is a flowchart illustrating a procedure of image reproduction processing executed by the reproduction apparatus in FIG. 1. It is a figure explaining the reproducing method of the image by the reproducing | regenerating apparatus in FIG. (1). It is a figure explaining the reproducing method of the image by the reproducing | regenerating apparatus in FIG. (2).

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Recording apparatus 101 Recording apparatus operation means 102 Recording apparatus interface part 103 Setting information storage part 104 Shutter speed control part 105 Shutter speed change process part 106 Shutter 107 Sensor 108 Encoder 109 Recording medium 110 Playback apparatus 111 Playback apparatus operation means 112 Playback apparatus interface Unit 113 Frame creation unit 114 Decoder 115 Frame selection unit 116 Frame creation unit 117 Display device 118 Display device information acquisition unit

Claims (6)

In an image recording / reproducing apparatus composed of an image recording apparatus and an image reproducing apparatus,
The image recording apparatus includes:
And changing means for changing the shadow shutter speed for each image shooting,
Photographing means for recording a plurality of images at the changed photographing shutter speed within a certain period of time;
The image reproduction device includes:
And decision means that determine the playback shutter speed of the image to be reproduced,
Selection means for selecting an image to be used in accordance with the reproduction shutter speed determined in the previous Kike' constant means,
Creating means for creating an image to be displayed on a display device from the image selected by the selecting means ;
When an image with a shooting shutter speed close to the playback shutter speed is recorded, the selection means selects an image with a shooting shutter speed close to the playback shutter speed,
If an image with a shooting shutter speed close to the playback shutter speed is not recorded, the selection means selects a plurality of images shot at different shooting shutter speeds that are shorter than the playback shutter speed, and the creation means an image reproducing apparatus characterized that you create an image for displaying a plurality of images selected by the selecting means synthesizing and display device.   Reading means for reading a plurality of images shot at a plurality of different shooting shutter speeds recorded on a recording medium;
  Determining means for determining a playback shutter speed of an image to be played back;
  When an image with a shooting shutter speed close to the reproduction shutter speed determined by the determining means is recorded on the recording medium, an image with a shooting shutter speed close to the reproduction shutter speed is selected, and the selected image is selected. An image having a shooting shutter speed close to the reproduction shutter speed determined by the determination unit is not recorded on the recording medium, the images are shot at different shooting shutter speeds shorter than the reproduction shutter speed. An image reproducing apparatus comprising: a generating unit that selects a plurality of images and generates an image based on the selected images.   Output means for outputting the image generated by the generating means to a display device;
  3. The image reproducing apparatus according to claim 2, wherein the determining unit determines the reproduction shutter speed according to a display frame rate of the display apparatus.   2. The image recording / reproducing apparatus according to claim 1, wherein the determining unit determines the reproduction shutter speed according to a display frame rate of the display apparatus.   4. The image reproducing apparatus according to claim 2, wherein the photographing shutter speed is a shutter speed n times a predetermined shutter speed (n is a natural number of 2 or more).   5. The image recording / reproducing apparatus according to claim 1, wherein the photographing shutter speed is a shutter speed n times a predetermined shutter speed (n is a natural number of 2 or more).

Images