JP2008039890A - Image data generating device, image reproducing device, image data generating method, image reproducing method, recording medium, and computer program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To display an image of high quality with less moving picture blurring and screen flickering on a display device. <P>SOLUTION: Image data wherein images of a predetermined time section including a moving picture are generated through processes mentioned below. Namely, the image data including data representing frame images constituting at least part of the images of the predetermined time section including the moving picture and data made to correspond to the data representing the frame images, and also representing frame rates are generated. In this configuration, the image data for the image display of high quality with less moving picture blurring and screen flickering can be generated without increasing the amount of the data by suitably setting the frame rates according to the contents of the moving picture. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to generation and display of image data for displaying an image on a display device.

  Conventionally, when an image is displayed on a display device, the image is displayed at a predetermined frame rate in accordance with a standard. For example, it was 29.97 fps for the NTSC system and 25 fps for the PAL system. “Fps” is an abbreviation of “Frames Per Sec (frame / second)” and represents the number of frames displayed per unit time. When a moving image is displayed, different still images are displayed in order at a predetermined frame rate.

  However, in a hold-type display device that maintains a substantially constant display until the display is updated by the next video signal, the following problem occurs when displaying a moving image. That is, when a moving image is displayed, a still image may be slightly blurred by a caretaker because different still images are displayed by being sequentially switched on the screen. On the other hand, there is a technique for reducing motion blur by inserting a black image at a time between a displayed still image and the next still image. However, in such an aspect, the screen may appear to flicker to the viewer.

  In order to deal with such a problem, in the technique of Patent Document 1, a black image is not inserted when displaying a still image, and a black image is inserted when displaying a moving image, and the image is displayed. . As a result, the problem of flickering on the screen is solved in the display of a still image that originally does not cause moving image blur. Further, in the technique of Patent Document 2, a black image is not inserted into a region for displaying a still image in a display region of a display device, and a black image is inserted into a region for displaying a moving image. I do. As a result, the problem of flickering on the screen is solved in the display of a still image that does not cause motion blur.

JP 2003-262846 A JP 2002-123223 A

  However, the above technique does not solve the problem of moving image blur and screen flicker in an image including a moving image at the same time.

  It is an object of the present invention to display a high-quality image with less moving image blur and screen flickering on an image including a moving image in a display device.

  In order to achieve the above object, the present invention performs the following processing when generating image data in which an image of a predetermined time interval including a moving image is recorded. That is, image data including data representing a frame image constituting at least a part of an image in a predetermined time interval including a moving image and data representing a frame rate associated with the data representing the frame image is generated.

  With such an aspect, by appropriately setting the frame rate according to the content of the video, image data that can display high-quality images with less video blur and screen flickering without increasing the amount of data. Can be generated. The “frame rate” can be information that substantially represents how many frame images should be displayed per unit time.

  Note that the following processing is preferably performed when generating image data. That is, a plurality of images are acquired at a first time interval to generate a plurality of first frame image data. Then, the amount of motion of the first frame image data image from the reference image is calculated. Thereafter, a part of the frame image data is selected from the plurality of first frame image data based on the motion amount. Then, the second frame image data representing the image of the selected first frame image data and the reproduction data as the data representing the frame rate, the image of the second frame image data at the time of reproducing the image data Is generated as image data.

  According to such an aspect, data of frame rates having different values can be stored in the image data in association with the second frame image data. Note that the second frame image data can be the same data as the first frame image data. The second frame image data is data generated based on the corresponding first frame image data, and may be data different from the first frame image data.

  When calculating the motion amount, one image of the first frame image data selected by the frame selection unit so far is used as a reference image for the first frame image data for calculating the motion amount. Preferably used to calculate the amount of motion.

  In such an aspect, the motion amount of the first frame image data to be examined is calculated with reference to the image of the frame image data that is already determined to be used as the frame image data of the image data to be generated. . Then, based on the amount of motion, it is determined whether or not the first frame image data should be selected. For this reason, according to such an aspect, it is possible to generate image data in which the motion of the moving image looks natural for the viewer.

  The reference image is the image of the first frame image data with the latest time when the image was acquired by the first frame data generation unit among the first frame image data selected by the frame selection unit so far. It is preferable that However, the image of the other first frame image data among the first frame image data selected so far can be used as the reference image.

  In addition, when generating image data, it is preferable to perform the following processing. That is, the compression rate for generating the second frame image data is determined based on the display time. Then, the selected first frame image data is compressed in accordance with the compression ratio to generate second frame image data. Then, as the image data, the compression rate of each of the second frame image data is represented, and image data further including compression rate data respectively associated with the second frame image data is generated. With such an aspect, the amount of image data can be reduced.

  Note that the selected first frame image data F1 and the first frame image data F0 which is the selected first frame image data F0 and an image is acquired immediately before the first frame image data F1. Preferably, the display time of the image of the second frame image data corresponding to the first frame image data F0 is determined based on the difference between the image acquisition times.

  With such an aspect, it is possible to appropriately set the display time of the image of the second frame image data. Note that “an image has been acquired immediately before” means that the first frame image data F1 having an image acquisition time in the selected set of first frame image data is earlier than the first frame image data F1. This means that it is closest to the acquisition time of the image of the frame image data F1.

  The first time interval is preferably 1 / (300 × n) seconds (n is a positive integer). With such an aspect, it is possible to easily generate image data based on both NTSC and PAL video signals.

  In addition, when reproducing an image based on image data in which an image of a predetermined time interval including a moving image is recorded, it is preferable to perform the following processing. That is, first, image data including data representing a frame image and data representing a frame rate associated with the data representing the frame image is acquired. Then, based on the data representing the frame image, the frame image is displayed for a time corresponding to the frame rate, and the image data is reproduced. With such an aspect, it is possible to perform high-quality image display with less moving image blur and screen flickering in the display device.

  The data representing the frame image is data generated by compressing other data representing the image, and the image data is further correlated with the data representing the frame image representing the compression rate at the time of the compression. In an aspect including data, it is preferable to perform the following processing. That is, the data representing the frame image is decompressed based on the data representing the compression rate. Then, when displaying the image for a time corresponding to the frame rate, the image is displayed based on the decompressed data. With such a correspondence, it is possible to perform high-quality image display with less moving image blur and screen flicker based on image data with a small data amount.

  The present invention can be realized in various forms, for example, an image data generation method and an image data generation device, an image reproduction method and an image reproduction device, and a function of those methods or devices. The present invention can be realized in the form of a computer program, a recording medium recording the computer program, a data signal including the computer program and embodied in a carrier wave, a recording medium recording image data, and the like.

A. First embodiment:
A1. Device configuration:
FIG. 1 is a block diagram showing the configuration of a digital video camera 10 that is an embodiment of the present invention. The digital video camera 10 includes a video processing engine unit 100 that processes image data, an image sensor 200, a memory unit 300, and a display device 500.

  The image sensor 200 is a CCD (Charge Coupled Device). The imaging device 200 can take in light from the outside, emit an electric signal in accordance with the light, and transmit the electric signal to the video processing engine unit 100. The display device 500 is a liquid crystal display. The display device 500 can be controlled by the video processing engine unit 100 to display an image. The memory unit 300 is a semiconductor memory. The video processing engine unit 100 generates image data based on a signal transmitted from the image sensor 200 in the memory unit 300 and processes the image data.

  The digital video camera 10 also has a slot into which the recording medium 400 can be inserted. The digital video camera 10 can read data stored in the recording medium 400 and can write data to the recording medium 400. The recording medium 400 is an SD card (trademark). The recording medium 400 is inserted into a card slot provided in the digital video camera 10, and image data is written by the video processing engine unit 100. Further, the video processing engine unit 100 can read data from the recording medium 400 inserted in the card slot.

  The digital video camera 10 further includes an input / output terminal 600. Via this input / output terminal 600, the digital video camera 10 can receive video signals from the external video playback devices 20 and 30.

A2. Recording video data:
FIG. 2 is a flowchart showing processing when a moving image is recorded by the digital video camera 10. Specifically, the video processing engine unit 100 controls these units (see FIG. 1) of the digital video camera 10.

  In step S10, the video processing engine unit 100 records a moving image for a predetermined time. The moving image data generated in the memory unit 300 at this time is referred to as “first moving image data D1”. On the other hand, the moving image data generated based on the first moving image data D1 and finally stored in the recording medium 400 is referred to as “second moving image data D2”. The data amount of the second moving image data D2 is not more than the first moving image data D1.

  In step S10, specifically, based on the signal from the image sensor 200, for example, a plurality of frame images are acquired at a frame rate of 300 fps (time interval of 1/300), and the first moving image data D1 It is stored in the memory unit 300 as a part.

  FIG. 3 is a diagram showing the contents of the first moving image data D1 generated in step S10 of FIG. The first moving image data D1 includes a vertical synchronization signal VS, a horizontal synchronization signal HS, and frame image data FD1. The vertical synchronization signal VS is a signal representing the timing at which scanning of one screen is started on the display of the display device. The horizontal synchronization signal HS is a signal indicating the timing at which scanning of one scanning line is started on the display of the display device. The frame image data FD1 is image data representing one still image having a certain number of pixels in the YCrCb color system.

  In the first moving image data D1, one pulse of the vertical synchronization signal VS represents the timing at which display of one frame image data of the frame image data FD1 is started. In step S10 in FIG. 2, images such as the respective frame image data F1-0, F1-1, F1-2, etc. are acquired at a time interval of 1/300 seconds. The frequency of the vertical synchronization signal VS is 300 Hz so that it can be reproduced at a high speed. In FIG. 3, the pulse width of the horizontal synchronizing signal HS does not represent the actual signal time. The same applies to the pulse width of the horizontal synchronizing signal HS shown in other drawings.

  FIG. 4 is a diagram showing an example of the frame image recorded in step S10. In FIG. 4, images of the first 10 pieces of frame image data F1-0 to F1-9 of the first moving image data D1 (hereinafter referred to as “frame images”) are shown. Frame images F1-0 to F1-9 are images recorded in that order with an interval of 1/300 seconds. For easy understanding, an acquisition time interval (1/300 between the frame image numbers F1-0 to F1-9 and the frame image recorded immediately before is displayed at the upper right of each frame image. Seconds). In the frame images F1-0 to F1-9, the person is moving from left to right in the frame (particularly, refer to the frame images F1-0 and F1-9 and the frame images F1-6 to F1-8). .

  Note that the function of step S10 is realized by the first frame data generation unit 111 as a function unit of the video processing engine unit 100. The first frame data generation unit 111 (first FD generation unit 111) is shown in FIG.

  In step S20 of FIG. 2, the first frame image data F1-0 among the generated plurality of frame image data is set as “reference frame data”. The reference frame data is used in the following step S30. Of the frame image data of the first moving image data D1, the frame image data determined as the reference frame data is frame image data that is the basis of the frame image data of the second moving image data. The function of step S20 is realized by the frame selection unit 113 as a function unit of the video processing engine unit 100 (see FIG. 1).

  In step S30, when moving image data is recorded, the motion amount Vm of the frame image data recorded next to the reference frame data is calculated. When the process of step S30 is first performed, it is the frame image data F1-1 that calculates the motion amount Vm (see FIG. 4).

  The motion amount Vm is an amount representing the difference between the images of the frame image data. For example, the motion amount Vm can be a shift amount of the barycentric position of the luminance of the pixels included in each of the reference frame data and the frame image data. It should be noted that the “center position of the luminance of the pixel of the frame data” can be obtained by weighted averaging the horizontal position and the vertical position of each pixel included in the frame data with the luminance. The unit of the motion amount Vm can be the number of pixels [pixel / frame]. The function of step S30 is realized by the motion amount calculation unit 112 as a function unit of the video processing engine unit 100 (see FIG. 1).

  In step S35 of FIG. 2, it is determined whether or not the frame image data for which the motion amount Vm has been calculated is the last frame image data in the first moving image data D1. If the frame image data for which the motion amount Vm has been calculated is not the last frame image data, the process proceeds to step S40. If the frame image data for which the motion amount Vm has been calculated is the last frame image data, the process proceeds to step S50.

  In step S40, it is determined whether or not the motion amount Vm of the frame image data from the reference frame data exceeds a predetermined threshold value Th. If the amount of movement Vm exceeds the threshold value Th, the process proceeds to step S50. If the amount of movement Vm is less than or equal to the threshold value Th, the process returns to step S30.

  In step S30, the motion amount Vm of the frame image data recorded next to the frame image data that was the subject of the previous examination is calculated. Until the amount of movement Vm exceeds a predetermined threshold value, the processes of steps S40 and S30 are performed in order for each frame image data. In the example of FIG. 4, it is assumed that steps S40 and S30 are repeated until the processing reaches the frame image data F1-6.

  That is, in step S40, the frame image data of the first moving image data D1 whose motion amount Vm exceeds the threshold value Th is sequentially selected. In FIG. 4, the image selected in step S40 is shown surrounded by a double frame. Note that the function of steps S35 and S40 is realized by the frame selection unit 113 as a functional unit of the video processing engine unit 100 (see FIG. 1).

  In step S50 of FIG. 2, the display time Td of the image of the reference frame data (hereinafter referred to as “reference frame image”) is determined. The display time Td represents the time during which the image of the reference frame data is displayed when the second moving image data D2 is reproduced. Specifically, the interval between the acquisition times of the frame image data determined that the motion amount Vm exceeds the threshold value in step S40 and the reference frame data is set as the reference frame image display time Td. . The display time Td is obtained by [frame image acquisition interval in step S10] × [number of frames between reference frame data and frame image data acquisition time + 1]. Here, the interval between the acquisition times of the reference frame data F1-0 and the frame image data F1-6 is 1/300 [seconds] × 6 [frames], that is, 6/300 [seconds]. The function of step S50 is realized by the display time determination unit 114 as a function unit of the video processing engine unit 100 (see FIG. 1).

  The second moving image data D2 stored in the recording medium 400 includes frame image data generated by compressing a part of the frame image data of the first moving image data D1. In step S60 of FIG. 2, a compression rate Rcp at the time of data compression is determined. The function of step S60 is realized by the compression rate determination unit 115 as a function unit of the video processing engine unit 100 (see FIG. 1).

  FIG. 5 is a graph showing the relationship between the display time Td and the compression rate Rcp. In the graph of FIG. 5, the horizontal axis represents the display time Td, and the vertical axis represents the compression rate Rcp. In step S60, the compression rate Rcp of the reference frame data is determined with reference to the graph C1 in FIG. Specifically, the video processing engine unit 100 holds the graph of FIG. 5 as the compression rate table 116.

  As can be seen from FIG. 5, in step S60, the compression rate Rcp is determined such that the compression rate Rcp is lower, that is, the image quality is higher as the display time Td of the reference frame data determined in step S50 is longer. The In general, even if the quality of an image that is displayed for only a short time is low, the user is not likely to notice. On the other hand, if the quality of an image that is displayed for a long time is low, the low quality is likely to be noticed by the user. According to the present embodiment, by setting the compression rate as shown in FIG. 5, it is possible to achieve both reduction in the data amount of the second moving image data to be generated and image quality viewed from the user's eyes. .

  In step S70 of FIG. 2, the frame image data determined that the motion amount Vm exceeds the threshold value in step S40 is newly set as reference frame data. In the example of FIG. 4, instead of the frame image data F1-0, frame image data F1-6 is newly set as reference frame data (see FIG. 4). The function of step S70 is realized by the frame selection unit 113 as a function unit of the video processing engine unit 100 (see FIG. 1).

  Of the frame image data of the first moving image data D1, the frame image data determined as the reference frame data in steps S20 and S70 is a frame image from which frame image data of the second moving image data is generated based on the frame image data. It is data. The frame image data newly determined as the reference frame data in step S70 is the latest frame image data that has been acquired in step S10 among the frame image data determined as reference frame data. It is data.

  In step S80, it is determined whether or not the frame image data newly set as the reference frame data is the last frame image data in the first moving image data D1. When “the frame image data newly set as the reference frame data is the last frame image data in the first moving image data D1,” the determination result in step S35 is Yes, and the process in step S40 is skipped. This is the case.

  In step S80, when the frame image data newly set as the reference frame data is not the last frame image data, the process returns to step S30. If the frame image data newly set as the reference frame data is the last frame image data, the process proceeds to step S90. Note that the processing in steps S <b> 10 to S <b> 80 described above is performed by the moving image analysis unit 110 as a functional unit of the video processing engine unit 100. The moving image analysis unit 110 is shown in FIG.

  In step S90, the frame image data set as the reference frame data in the processing of steps S20 to S70 (see steps S20 and S70) is compressed according to the respective compression ratios Rcp (see step S60). The frame image data surrounded by a double frame in FIG. 4 is data that becomes the reference frame data in step S70 while the processing of FIG. 2 is executed. Note that the compression rate Rcp of the last frame image data of the first moving image data D1 is set to the minimum value (2) (see FIG. 5).

  The function of step S90 is realized by the second frame data generation unit 122 as a function unit of the video processing engine unit 100. The second frame data generation unit 122 (second FD generation unit 122) is shown in FIG.

  2 includes the compressed frame image data (see step S90), the respective display times Td (see step S50), and the respective compression rates Rcp (see step S60). Data D2 is generated and stored in the recording medium 400. The display time Td of the last frame image data of the first moving image data D1 is set to the minimum value (1/300 seconds). The function of step S100 is realized by the recording data generation unit 124 as a functional unit of the video processing engine unit 100 (see FIG. 1).

  The processing in steps S90 and S100 is performed by the moving image generation unit 120 as a functional unit of the video processing engine unit 100. The moving image generating unit 120 is shown in FIG. Further, when recording the frame image data in step S100, the video processing engine unit 100 can correct the brightness and contrast of the image. A functional unit of the video processing engine unit 100 having such a function is shown in FIG.

  FIG. 6 is a diagram illustrating an example of the frame image data set as the reference frame data in the processes of steps S20 to S70 and the compressed frame image data stored in the second moving image data D2. In the first embodiment, it is assumed that frame image data F1-0, F1-6, F1-7, F1-8, F1-9, etc. are set as reference frame data as shown in the upper part of FIG. (See FIG. 4). The respective display times Td are 6/300, 1/300, 1/300, 1/300, 1/300, and 5/300 seconds. The upper part of FIG. 6 shows images of these reference frame data, numbers, and display time Td.

  The compression rate Rcp of the frame image data F1-0, F1-6, F1-7, F1-8, F1-9 is 20, 50, 50, 50, 25 based on the respective display times Td (FIG. 5). Therefore, in step S90 of FIG. 2, the frame image data F1-0, F1-6, F1-7, F1-8, and F1-9 are respectively 1/20, 1/50, 1/50, 1/50, The data amount is compressed to 1/25. Note that the compressed frame image data also represents the color of each pixel in the YCrCb color system.

  The frame image data compressed in this way is the frame image data F2-0, F2-1, F2-2, F2-3, and F2-4 of the second moving image data D2. A part is configured (see FIG. 6). In the second moving image data D2, the data representing the display time Td and the compression rate Rcp are respectively associated with the frame image data F2-0, F2-1, F2-2, F2-3, and F2-4. And data to be stored are stored.

  Note that the numerical values shown at the upper right of each frame image in the lower part of FIG. 6 are the frame number in the second moving image data D2, the frame number of the corresponding frame in the first moving image data D1, the display time Td, the compression in order from the upper part. The rate Rcp.

  FIG. 7 is a diagram showing the contents of the second moving image data D2 generated in step S100. The second moving image data D2 includes a vertical synchronization signal VS, a horizontal synchronization signal HS, frame image data FD2, the number of continuation frames Nfc, and a compression rate Rcp.

  Also in the second moving image data D2, one pulse of the vertical synchronization signal VS represents the timing at which display of one frame image data of the frame image data FD2 is started. As described above, each frame image data of the frame image data FD2 is selected from each frame image data F1-0, F1-1, F1-2, etc. of the first moving image data D1, and is further compressed and generated. It is data. In FIG. 7, the frame image data numbers F1-0 and F1-6 of the corresponding first moving image data D1 are respectively below the frame image data numbers F2-0, F2-1, and F2-2. , F1-7, etc. are shown in parentheses.

  The continuation frame number Nfc is data representing how many frames of the first moving image data D1 each frame image of the second moving image data D2 is displayed. Each data of the continuation frame number Nfc is associated with one frame image data F2-0, F2-1, F2-2, etc. of the frame image data FD2.

  Further, data Nfcmax indicating how many frame images of the second moving image data D2 can be displayed per second is stored at the head of the data representing the number of continuation frames Nfc. In the first embodiment, Nfcmax = 300 because each frame image is acquired at 1/300 second intervals when recording a moving image (see step S10 in FIG. 2). In the first embodiment, Nfc is Nfcmax or less.

  Since the image display time Td of the frame image data F2-0 is 6/300 seconds (see the lower left in FIG. 6), the number of continuation frames Nfc corresponding to the frame image data F2-0 is as shown in FIG. 6 frames. Further, since the image display time Td of the frame image data F2-1 is 1/300 second, the number of continuation frames Nfc corresponding to the frame image data F2-1 is one frame. The continuation frame number Nfc substantially represents the display time Td of each frame image data together with the maximum frame number Nfcmax per unit time. Further, the continuation frame number Nfc substantially represents the frame rate associated with each frame image data together with the maximum frame number Nfcmax per unit time.

  The compression rate Rcp included in the second moving image data D2 is the compression rate of each frame image data determined in step S60 of FIG. Each data of the compression rate Rcp is associated with one frame image data F2-0, F2-1, F2-2, etc. of the frame image data FD2.

  As can be seen from FIGS. 3 and 7, the second moving image data D2 has a smaller number of stored frame image data than the first moving image data D1. For this reason, the second moving image data D2 has a smaller data amount than the first moving image data D1.

  The frame image data F2-0, F2-1, F2-2, etc. held by the second moving image data D2 are the frame image data F1-0, F1- of the corresponding first moving image data D1. 6, data generated by compressing F1-7 and the like (see step S90 in FIG. 2). Therefore, also from this point, the second moving image data D2 has a smaller data amount than the first moving image data D1.

A3. Playback of video data:
FIG. 8 is a flowchart showing processing when the digital video camera 10 reproduces the second moving image data D2 in the recording medium 400 on the display device 500. In the first embodiment, a case where image data is reproduced by the digital video camera 10 will be described. However, similar processing is performed when image data is played back on a video playback device such as another digital video player.

  First, in step S210, the second moving image data D2 (see FIG. 7) is read from the recording medium 400. In step S220, the second moving image data D2 is analyzed. Specifically, the compression rate Rcp corresponding to each frame image data F2-0, F2-1, F2-2, etc. held by the second moving image data D2 is specified. Further, the display time Td of each frame image data is based on the continuation frame number Nfc corresponding to each frame image data F2-0, F2-1, F2-2, etc. and the maximum frame number Nfcmax per unit time. Calculated. The processing in steps S210 and S220 is performed by the image data analysis unit 132 as a functional unit of the video processing engine unit 100. The image data analysis unit 132 is shown in FIG.

  In step S230 of FIG. 8, the frame image data F2-0, F2-1, F2-2, etc. are decompressed based on the corresponding compression ratios Rcp, and the frame image data F3-0, F3-1, F3-2 are decompressed. Etc. are generated. These frame image data F3-0, F3-1, F3-2, etc. also represent the color of each pixel in the YCrCb color system. The function of step S230 is realized by the decompression unit 134 as a functional unit of the video processing engine unit 100 (see FIG. 1).

  In step S240 of FIG. 8, a third moving image including a vertical synchronization signal VS, a horizontal synchronization signal HS, and frame image data FD3 including individual frame image data F3-0, F3-1, F3-2, and the like. Data D <b> 3 is generated on the memory unit 300. The function of step S240 is realized by the reproduction data generation unit 136 as a functional unit of the video processing engine unit 100 (see FIG. 1). Further, the processing of steps S230 and S240 is performed by the moving image reproduction unit 130 as a functional unit of the video processing engine unit 100. The moving image playback unit 130 is shown in FIG.

  FIG. 9 is a diagram showing third moving image data D3 generated on the memory unit 300 based on the second moving image data D2. In FIG. 9, the frame image data numbers F2-0, F2-1, and the like corresponding to the second moving image data D2, respectively, under the frame image data numbers F3-0, F3-1, and F3-2. F2-2 etc. and the frame image data numbers F1-0, F1-6, F1-7 etc. of the first moving image data D1 are shown in parentheses.

  Also in the third moving image data D3, one pulse of the vertical synchronization signal VS represents the timing at which display of one frame image data of the frame image data FD3 is started. In the third moving image data D3, the interval of the vertical synchronization signal VS is determined based on the display time Td obtained in step S220. For example, in the example of FIG. 9, the interval between the second pulse and the first pulse of the vertical synchronization signal VS is 6/300 seconds. This is equal to the display time Td of the frame image data F2-0. That is, the interval between pulses of the vertical synchronization signal VS can be calculated by Nfc / Nfcmax based on the maximum number of frames Nfcmax per unit time stored in the second moving image data D2 and the number of continuation frames Nfc. The upper part of FIG. 9 shows the display time Td corresponding to each frame image data.

  The display device 500 is a liquid crystal display, and an image once displayed is continuously displayed on the display unless rewritten by a new video signal. Therefore, in the third moving image data D3, when the image of the frame image data is once displayed on the display device 500 and then continues to be displayed, during the period Ph until the next image is displayed, the scanning line is displayed. The horizontal synchronization signal HS indicating the timing to start scanning is not turned on.

  In step S250 of FIG. 8, an image is displayed on the display device 500 based on the third moving image data D3 generated on the memory unit 300. Specifically, images such as frame image data F3-0, F3-1, F3-2, and the like are drawn on the display device 500 in synchronization with the pulse of the vertical synchronization signal VS. At this time, for example, the frame image data F3-0 is displayed for 6/300 seconds after drawing and is rewritten to the frame image data F3-1. The frame image data F3-1 is displayed for 1/300 second after drawing and is rewritten to the frame image data F3-2.

  When an image is displayed on the display device 500, the frame image data F3-0, F3-1, F3-2, and the like are converted according to the number of pixels of the display device 500. This processing is performed by the resizing unit 140 as a functional unit of the video processing engine unit 100. The video processing engine unit 100 controls the display device 500 by converting the data expressed in the YCrCb color system into an RGB color system corresponding to each display device and sending an electric signal to the display device 500. This is a display driver unit 150 as a functional unit. The resizing unit 140 and the display driver unit 150 are shown in FIG.

  According to the aspect described above, when an image is first recorded, the image is acquired at a sufficiently short time interval, and the frame image data FD1 is generated, so that a viewer feels moving image blur and flicker. It is possible to display difficult videos. For still images and moving images with little motion, the number of different frame images displayed per unit time (see FIG. 7) can be reduced to reduce the amount of moving image data recorded on the recording medium. it can.

B. Second embodiment:
In the first embodiment, the vertical synchronization signal VS of the third moving image data D3 generated at the time of reproduction is ON at various time intervals determined based on the number of continuing frames Nfc and the maximum number of frames per unit time Nfcmax. (See FIG. 9). However, in the second embodiment, the vertical synchronizing signal VS of the third moving image data D4 generated at the time of reproduction is turned on at a constant time interval. Then, frame image data corresponding to each pulse of the vertical synchronization signal VS of the third moving image data D4 is prepared. The other points of the second embodiment are the same as those of the first embodiment.

  FIG. 10 is a diagram showing third moving image data D4 generated on the memory unit 300 based on the second moving image data D2. In FIG. 10, under the frame image data numbers F4-0, F4-1, F4-2, etc., the frame image data numbers F2-0, F2-1, F2-2 etc. and the frame image data numbers F1-0, F1-6, F1-7 etc. of the first moving image data D1 are shown in parentheses. The notation of other elements in FIG. 11 is the same as in FIG.

  In the second embodiment, each frame image data of the second moving image data D2 is duplicated by the corresponding number of continuous frames Nfc (see FIG. 7) after decompression. The data thus generated becomes frame image data FD4 of the third moving image data D4. For example, the frame image data F4-0 to F4-5 of the third moving image data D4 are the same data. The frame image data F4-0 to F4-5 are obtained by decompressing and duplicating the frame image data F2-0 of the second moving image data D2.

  The vertical synchronization signal VS of the third moving image data D4 is turned ON at a constant time interval of 1/300 seconds. Also in the third moving image data D4, one pulse of the vertical synchronization signal VS represents the timing at which one frame image data of the frame image data FD4 starts to be displayed. That is, when the image is reproduced on the display device 500, the frame image data F4-0, F4-1, D4-2, etc. of the third moving image data D4 are sequentially 1/1 / based on the vertical synchronization signal VS. Displayed for 300 seconds.

  Even in such an aspect, it is possible to display a moving image in which the viewer does not feel moving image blur or flicker while reducing the amount of moving image data stored on the recording medium.

C. Third embodiment:
In the third embodiment, a process of recording the second moving image data on a recording medium based on a video signal received from another video playback device, not a signal from the image sensor 200 will be described. The point at which a video signal is received from another video playback device, the point that the moving image data input by the video signal records each frame image at a specific frame rate, and the processes of steps S30 to S40 in FIG. 2 are performed. Except for the absence, the third embodiment is the same as the first embodiment.

  The first external video signal Sv1 output from the first external video playback device 20 and input to the digital video camera 10 via the input / output terminal 600 (see FIG. 1) is an NTSC video signal. In the NTSC system, the frame rate is 29.97 fps. And the display of the image in a display apparatus is performed by the interlace system. For this reason, the frequency of the “field” that is displayed by every other scanning line and has half the amount of information of one “frame” is 59.94 Hz that is twice 29.97 Hz, that is, about 60 Hz. .

  FIG. 11 is a diagram showing the contents of the first moving image data D1n generated based on the first external video signal Sv1 in step S10 of FIG. 2 in the third embodiment. The notation of each element in FIG. 11 is the same as in FIG. The frequency of the vertical synchronization signal VS is 300 Hz as described in the first embodiment. On the other hand, in the NTSC external video signal Sv1, the image data of the field corresponds to the frame image data of the first moving image data D1n. The image data of this field exists only at a frequency of about 60 Hz as described above in the external video signal Sv1. For this reason, in the first moving image data D1n generated based on the video signal from the outside in step S10, each frame image data F1-0, F1-1, etc. includes five of the pulses of the vertical synchronization signal VS. Are recorded in association with pulses having one period.

  When the second moving image data is generated by the process of FIG. 2 based on the video signal from the outside, steps S30 to S40 of the process of FIG. 2 are not performed. All the frame image data F1-0, F1-1 and the like stored in the first moving image data are sequentially selected as the reference frame data in steps S20 and S70. As a result, the display time Td and the compression rate Rcp are determined in steps S50 and S60 for all the frame image data stored in the first moving image data.

  In the first moving image data D1n, the frame image data F1-0, F1-1, and the like are present at equal intervals at a rate of one for every five pulses of the vertical synchronization signal VS. For this reason, the display time Td of each frame image data is a fixed value of 5/300 seconds. In addition, the compression rate Rcp determined based on the display time Td with reference to the graph of FIG.

  FIG. 12 is a diagram showing the contents of the second moving image data D2n generated based on the first external video signal Sv1 in step S100 of FIG. The notation method of each element in FIG. 12 is the same as FIG. Since the frame image data F1-0, F1-1, and the like of the first moving image data D1n are present at equal intervals at a rate of one for every five pulses of the vertical synchronization signal VS (see FIG. 11), the second moving image In the data D2n, the continuation frame number Nfc corresponding to each frame image data is a fixed value 5.

  As shown in the lower part of FIG. 12, the compression rate Rcp corresponding to the frame image data F1-0, F1-1, etc. of the first moving image data D1n is a constant value 25. The frame image data F2-0, F2-1, etc. of the second moving image data D2n shown in the middle part of FIG. 25 is generated by data compression.

  Next, a case where a PAL video signal is input will be described. The second external video signal Sv2 output from the external second video playback device 30 and input to the digital video camera 10 via the input / output terminal 600 (see FIG. 1) is a PAL video signal. In the PAL system, the frame rate is 25 fps. And the display of the image in a display apparatus is performed by the interlace system. For this reason, the frequency of the field is 50 Hz, which is twice 25 Hz.

  FIG. 13 is a diagram showing the contents of the first moving image data D1p generated based on the second external video signal Sv2 in step S10 of FIG. 2 in the third embodiment. The notation method of each element in FIG. 13 is the same as FIG. The frequency of the vertical synchronization signal VS is 300 Hz. On the other hand, frame image data is stored at 25 Hz in the PAL external video signal Sv2. For this reason, the image data of the field corresponding to the frame image data of the first moving image data D1p exists only at a frequency of 50 Hz in the external video signal Sv2. Therefore, in the first moving image data D1p generated in step S10, each frame image data F1-0, F1-1, etc. is a pulse having a cycle of one out of six pulses of the vertical synchronization signal VS. Corresponding to and recorded.

  When generating the second moving image data, the frame image data F1-0, F1-1, etc. stored in the first moving image data D1p are all selected as the reference frame data as described above. As a result, the display time Td and the compression rate Rcp are determined in steps S50 and S60 of FIG. 2 for all the frame image data stored in the first moving image data D1p. Since the frame image data F1-0, F1-1, and the like are present at equal intervals at a rate of one for every six pulses of the vertical synchronization signal VS, the display time Td of each frame image data is a fixed value 6/300 seconds. It is. The compression rate Rcp is also a constant value 20.

  FIG. 14 is a diagram showing the contents of the second moving image data D2p generated based on the second external video signal Sv2. The notation method of each element in FIG. 12 is the same as FIG. Since the frame image data F1-0, F1-1, etc. of the first moving image data D1p are present at equal intervals at a ratio of 1 to 6 pulses of the vertical synchronization signal VS (see FIG. 13), the second moving image data In the data D2p, the continuation frame number Nfc corresponding to each frame image data is a fixed value 6.

  As shown in the lower part of FIG. 14, the compression rate Rcp corresponding to the frame image data F1-0, F1-1, etc. of the first moving image data D1p is a constant value 20. The frame image data F2-0, F2-1, etc. of the second moving image data D2p shown in the middle part of FIG. 14 is 1 / frame frame data F1-0, F1-1, etc. of the first moving image data D1p, respectively. 20 is generated by data compression.

  In the third embodiment, the first moving image data can hold frame image data F1-0, F1-1, etc. at a frequency (frequency) of 300 Hz. Therefore, a new frame image is synthesized both when generating the second moving image data based on the NTSC video signal and when generating the second moving image data based on the PAL video signal. Therefore, the second moving image data can be easily generated.

D. Variation:
The present invention is not limited to the above-described examples and embodiments, and can be implemented in various modes without departing from the gist thereof. For example, the following modifications are possible.

D1. Modification 1:
In the above embodiment, the frame image data is recorded at 300 Hz in the first moving image data. However, in the first moving image data, the frame image data can be recorded at other frequencies (frequency). However, if the frame image data can be recorded at (300 × n) Hz (n is a positive integer), both the NTSC video signal and the PAL video signal can be handled. Further, when n is 2 or more, even when a moving image is reproduced on the hold type display device, the viewer hardly feels moving image blur or flicker.

D2. Modification 2:
In the above-described embodiment, the second moving image data holds information on the display time Td in which each frame image is to be displayed based on the maximum frame number Nfcmax per second and the continuation frame number Nfc. However, the information of the display time Td for displaying each frame image can be held by other methods such as holding the data of the display time Td directly in the generated moving image data. In other words, the second moving image data can be in various forms that substantially hold information on the frame rate. For example, the frame rate can be the number of frames recorded or displayed per unit time (eg, 1 second or 1 minute). Further, as described above, the frame rate can be held as the display time of the frame image. The frame rate can also be held as information on unit time and information on how many times the unit time should display the frame image (for example, the number of continuous frames Nfc).

D3. Modification 3:
In the above embodiment, the compression rate Rcp is determined according to the display time Td with reference to the graph of FIG. However, the compression rate Rcp can be determined by other methods. For example, the relationship between the display time Td and the compression rate Rcp is in the form of a “table” in which the compression rate Rcp is determined according to a plurality of sections (for example, 4 to 8 sections) roughly based on the size of the display time Td. Can be held. Further, the relationship between the display time Td and the compression rate Rcp can be held in the form of a mathematical expression. Furthermore, the relationship between the display time Td and the compression rate Rcp can be held in the form of a graph, a table, or an equation as the relationship between the number of continuing frames Nfc and the compression rate Rcp.

  In the above embodiment, the compression rate Rcp was a monotonously decreasing convex with increasing display time Td. However, the relationship of the compression rate Rcp to the display time Td or the number of continuing frames Nfc can be another relationship. That is, as indicated by a broken line C2 in FIG. 5, the compression rate Rcp can be linearly decreased with respect to the increase in the display time Td or the number of continuous frames Nfc. Further, as indicated by a dashed line C3 in FIG. 5, the compression rate Rcp can be set so as to decrease with an upwardly convex curve as the display time Td or the number of continuous frames Nfc increases.

  However, it is preferable that the compression rate Rcp maintain the same value or decrease as the display time Td or the number of continuing frames Nfc increases. This is because the longer the continuous display time, the more noticeable that the image quality is low.

D4. Modification 4:
In the above embodiment, the value of the continuation frame number Nfc is equal to or less than the maximum frame number Nfcmax per second. That is, one frame image of the second moving image data is displayed for a maximum of 1 second. However, the time for continuously displaying one frame image can be a time interval of less than 1 second, or a time interval of less than 1 second. That is, the maximum number of continuing frames Nfc can be set to an arbitrary value.

D5. Modification 5:
In the third embodiment, the field data included in the first external video signal Sv1 and the second external video signal Sv2 is used as the frame image data of the first moving image data, and 5 of the vertical synchronization signal VS. Recording was performed in association with every other or every sixth pulse (see FIGS. 11 and 13). However, when the first moving image data is generated based on the video signal from the outside, it can be generated in another manner.

  For example, when image data is recorded in Fs [Hz] in the original video signal Sv and the frequency of the vertical synchronization signal VS of the generated image data D1 is Fvs, each of the video signals Sv holds. Duplicate field image data (Fvs / Fs). Then, as shown in FIG. 3, the image data can be associated with each pulse of the vertical synchronization signal VS to be field image data of the first moving image data. In such an aspect, corresponding field image data F1-0, F1-1, etc. exist for each pulse of the vertical synchronization signal VS. Moreover, in such an aspect, among the processes for generating the second moving image data (see FIG. 2), the processes after step S20 can be executed in the same manner as described in the first embodiment. it can.

D6. Modification 6:
In the first embodiment, the processing when the digital video camera 10 reproduces the second moving image data D2 in the recording medium 400 on the display device 500 has been described. However, when video is played back on other video playback devices, it can be played back by the same processing as in the first embodiment. Such a video reproduction device can be configured to include no image sensor 200, and the moving image analysis unit 110 and the moving image generation unit 120 as functional units of the video processing engine unit 100.

D7. Modification 7:
In the above embodiment, the display device 500 is a liquid crystal display. However, the display unit that displays an image based on the image data can have another configuration such as an EL display or a plasma display. However, the present invention is more effective when applied to a hold-type display device that continues to display the previous image until the image is rewritten. This is because motion blur and flicker (flicker) felt by a viewer (user) when displaying a moving image on a hold-type display device can be reduced.

E8. Modification 8:
In the above embodiment, a part of the configuration realized by hardware may be replaced with software, and conversely, a part of the configuration realized by software may be replaced by hardware. For example, some of the functions of the video processing engine unit 100 (FIG. 1) may be executed by a computer connected to a photographing device such as the digital video camera 10.

  A computer program for realizing such a function is provided in a form recorded on a computer-readable recording medium such as a floppy disk or a CD-ROM. The computer reads the computer program from the recording medium and transfers it to an internal storage device or an external storage device. Or you may make it supply a computer program to a computer from a program supply apparatus via a communication path. When realizing the function of the computer program, the computer program stored in the internal storage device is executed by the microprocessor of the computer. The computer program recorded on the recording medium may be directly executed by the computer.

  In this specification, the computer is a concept including a hardware device and an operation system, and means a hardware device that operates under the control of the operation system. The computer program causes such a computer to realize the functions of the above-described units. Note that some of the functions described above may be realized by an operation system instead of an application program.

  In the present invention, the “computer-readable recording medium” is not limited to a portable recording medium such as a flexible disk or a CD-ROM, but an internal storage device in a computer such as various RAMs and ROMs, An external storage device fixed to a computer such as a hard disk is also included.

FIG. 1 is a block diagram showing the configuration of a digital video camera 10 that is an embodiment of the present invention. 4 is a flowchart showing processing when a moving image is recorded by the digital video camera 10. The figure which shows the content of the 1st moving image data D1 produced | generated in step S10 of FIG. The figure which shows the example of the frame image recorded by step S10. The graph which shows the relationship between display time Td and compression rate Rcp. The figure which shows the example of the frame image data set to the reference | standard frame data, and the compressed frame image data stored in the 2nd moving image data D2. The figure which shows the content of the 2nd moving image data D2 produced | generated in step S100. 7 is a flowchart showing processing when the digital video camera 10 plays back the second moving image data D2 in the recording medium 400 on the display device 500. The figure which shows the 3rd moving image data D3 produced | generated on the memory part 300 based on the 2nd moving image data D2. The figure which shows the 3rd moving image data D4 produced | generated on the memory part 300 based on the 2nd moving image data D2. The figure which shows the content of the 1st moving image data D1n produced | generated based on 1st external video signal Sv1 in FIG.2 S10 in 3rd Example. The figure which shows the content of the 2nd moving image data D2n produced | generated based on 1st external video signal Sv1 by step S100 of FIG. The figure which shows the content of the 1st moving image data D1p produced | generated based on 2nd external video signal Sv2 in FIG.2 S10 in 3rd Example. The figure which shows the content of the 2nd moving image data D2p produced | generated based on 2nd external video signal Sv2.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Digital video camera 20 ... 1st video reproduction apparatus 30 ... 2nd video reproduction apparatus 100 ... Video processing engine part 110 ... Movie analysis part 120 ... Movie recording part 130 ... Image quality correction part 140 ... Image decompression part 150 ... Resizing Unit 160 ... frame analysis unit 170 ... display driver unit 200 ... imaging device 300 ... memory unit 400 ... recording medium 500 ... display device 600 ... input / output terminal C1 ... graph showing the relationship of compression rate Rcp to display time Td D1 ... first Video data D1n: first video data generated based on NTSC video data D1p: first video data generated based on PAL video data D2: second video data D2n: NTSC format Second video data generated based on the video data of D2p: PAL video data Second moving image data D3, D4 generated based on the third moving image data F1-0 to F1-9 frame image data of the first moving image data F2-0 to F2-4 second moving image data Frame image data F3-0 to F3-4 ... frame image data of third moving image data F4-0 to F3-9 ... frame image data of fourth moving image data FD1 ... frame image data of first moving image data D1 FD2 ... A set of frame image data of the second moving image data D2 FD3 ... A set of frame image data of the third moving image data D3 FD4 ... A set of frame image data of the third moving image data D4 HS ... Horizontal synchronization signal Nfc ... number of continuous frames Nfcmax ... maximum number of frames per second Ph ... period from display of previous image to display of next image Rcp ... compression Sv1 ... first external video signal Sv2 ... second external video signal Td ... display time VS ... vertical sync signal Vm ... motion amount

Claims (13)

An image data generation device that generates image data in which an image of a predetermined time interval including a moving image is recorded,
Image data generation for generating image data including data representing a frame image constituting at least a part of an image in a predetermined time interval including a moving image and data representing a frame rate associated with the data representing the frame image apparatus. The apparatus of claim 1, comprising:
A first frame data generating unit that acquires a plurality of images at a first time interval and generates a plurality of first frame image data;
A motion amount calculation unit for calculating a motion amount from a reference image of the image of the first frame image data;
A frame selection unit that selects a part of the frame image data from the plurality of first frame image data based on the amount of movement;
Second frame image data representing an image of the selected first frame image data, and reproduction data as data representing the frame rate, wherein the second frame image data is reproduced when the image data is reproduced. An image data generation unit that generates data including reproduction data representing display time for displaying the image as the image data. The apparatus of claim 2, comprising:
The motion amount calculation unit uses one image of the first frame image data selected by the frame selection unit so far as the reference image for the first frame image data for calculating the motion amount. An apparatus for calculating the amount of movement. The apparatus of claim 2, further comprising:
A compression rate determination unit that determines a compression rate when generating the second frame image data based on the display time;
A second frame data generation unit that compresses the selected first frame image data according to the compression ratio and generates the second frame image data;
The image data generation unit generates image data that further includes compression rate data representing the compression rate and associated with the second frame image data as the image data. The apparatus of claim 2, further comprising:
The selected first frame image data F1 and the first frame image data F0 which is the selected first frame image data F0 and an image is acquired immediately before the first frame image data F1. And a display time determining unit that determines the display time of the image of the second frame image data corresponding to the first frame image data F0 based on the difference in acquisition time of the image. An apparatus according to claims 2-5, comprising:
The first time interval is 1 / (300 × n) seconds (n is a positive integer). An image reproduction device for reproducing an image based on image data in which an image of a predetermined time interval including a moving image is recorded,
Image data acquisition for acquiring image data including data representing a frame image constituting at least a part of an image in a predetermined time interval including a moving image, and data representing a frame rate associated with the data representing the frame image And
An image data reproducing unit that displays the frame image for a time corresponding to the frame rate based on data representing the frame image, and reproduces the image data. The apparatus of claim 7, wherein
The data representing the frame image is data generated by compressing other data representing the frame image,
The image data further represents a compression rate at the time of compression, and includes data associated with data representing the frame image,
The image playback device further includes:
A decompression unit for decompressing the data representing the frame image based on the data representing the compression ratio;
The image data reproduction unit displays an image based on the decompressed data when displaying the image for a time corresponding to the frame rate. A method for generating image data in which an image of a predetermined time interval including a moving image is recorded,
A method for generating image data including data representing a frame image constituting at least a part of an image in a predetermined time interval including a moving image, and data representing a frame rate associated with the data representing the frame image. A method of reproducing an image based on image data in which an image of a predetermined time interval including a moving image is recorded,
Obtaining image data including data representing a frame image constituting at least a part of an image of a predetermined time interval including a moving image, and data representing a frame rate associated with the data representing the frame image;
Displaying the frame image for a time corresponding to the frame rate based on the data representing the frame image, and reproducing the image data. A recording medium recording image data in which an image of a predetermined time interval including a moving image is recorded,
A recording medium recording image data including data representing a frame image constituting at least a part of an image in a predetermined time interval including a moving image, and data representing a frame rate associated with the data representing the frame image. A computer program for causing an image data generation device including a computer to generate image data in which an image of a predetermined time interval including a moving image is recorded,
A function of generating image data including data representing a frame image constituting at least a part of an image of a predetermined time interval including a moving image, and data representing a frame rate associated with the data representing the frame image; A computer program to be realized by a computer. A computer program for causing an image data reproduction device provided with a computer to reproduce an image based on image data in which an image of a predetermined time interval including a moving image is recorded,
A function of obtaining image data including data representing a frame image constituting at least a part of an image in a predetermined time interval including a moving image, and data representing a frame rate associated with the data representing the frame image;
A computer program for causing the computer to realize a function of displaying the frame image for a time corresponding to the frame rate and reproducing the image data based on data representing the frame image.

Images