JPWO2008129648A1 - Frame rate conversion apparatus, frame rate conversion method, and moving picture encoding apparatus - Google Patents

Abstract

An encoding target image determination unit that determines the level of the frame rate of the input image with respect to the output image from the time of the input image and the time of the output image, selects an image to be output according to the determination result, and controls the number of output frames 4 is provided.

Description

  The present invention relates to a frame rate conversion device that converts a frame rate of a moving image, a frame rate conversion method, and a moving image encoding device that uses the frame rate conversion device.

  In general, a moving image is represented by continuously displaying a plurality of frame images. The number of images displayed per second is called a frame rate and is expressed in fps (Frame Per Second). This frame rate is determined for each standard. For example, in an analog NTSC (National Television Standards Committee) type color television, it is 29.97 fps.

  The higher the frame rate, the smoother the moving image can be expressed. On the other hand, the recording device, the playback device, the display device, etc. are required to have high performance, and the capacity of the recording device also increases. In recent years, a technique for encoding a moving image using a computer has been used. In these bit streams, a code indicating the frame rate is often included.

  There are various moving image encoding methods, and a typical one is the MPEG-2 (Moving Picture Expert Group) method. This MPEG-2 system is adopted as a moving image encoding system of DVD (Digital Veritable Disk) -VIDEO.

  Note that there is generally no compatibility between different video coding schemes. For this reason, in order to cope with different moving image encoding schemes, it is necessary to prepare a decoding device for each scheme or to prepare a device for converting the encoding scheme. In the former case, for example, a commercially available DVD player can reproduce only the MPEG-2 system. That is, it may be difficult to cope with originally different decoding devices.

  In the latter case, the time required for conversion and the image quality after conversion become a problem. Further, the above-mentioned DVD-VIDEO is set to have a frame rate of 29.97 fps in order to match the reproduction capability of the television. Accordingly, when converting a moving picture encoding method having a different frame rate, it is necessary to convert the frame rate. Such restrictions by the frame rate are the same in analog devices.

  From the above viewpoint, an apparatus for converting the frame rate of a moving image is desired in order to cope with different moving image encoding methods. As a conventional frame rate conversion apparatus, for example, there is one disclosed in Patent Document 1. In the frame rate conversion device according to Patent Document 1, a video signal having a specific frame rate set in advance (24P video signal having 24 frames / second) is input, and a video signal having a specific frame rate set in advance (50 fields / second). Second 50I video signal) and output.

  Japanese Patent Application Laid-Open No. 2003-228561 discloses a technique for estimating the motion of a moving image using frames having different appearance times and converting a low frame rate moving image signal to a high frame rate moving image. For example, a moving image of 60 frames / second is converted into a moving image of 120 frames / second. As described above, in addition to the case of converting the frame rate to 2 times, it is possible to convert to a different magnification such as 1.5 times or 3 times.

JP 2006-196954 A JP 2006-165602 A

  Conventional frame rate conversion devices can only convert between specific frame rates, and can only convert from a low frame rate to a high frame rate and only at a frame rate close to a certain level. It was.

  In recent years, moving image encoding methods have been used in many fields, and when the frame rate changes (variable frame rate), for example, the input moving image is displayed as a still image, and the frame rate is very low. Sometimes it becomes.

  In such a case, since the conventional frame rate conversion apparatus has a restriction on the frame rate of the input / output image as described above and supports only limited conversion, the frame rate conversion can be appropriately performed. There was a problem that it was not possible.

  The present invention has been made to solve the above-described problems. A frame rate conversion apparatus, a frame rate conversion method, and a frame rate conversion method capable of converting various image frame rates regardless of the frame rate of input / output images. It is an object of the present invention to obtain a moving picture coding apparatus using a frame rate conversion apparatus.

  The frame rate conversion device according to the present invention determines the level of the frame rate of the input image relative to the output image from the time of the input image and the time of the output image, and selects an image to be output as an output image according to the determination result A determination unit for controlling the number of output frames is provided. This has the effect that the frame rate of various images can be converted regardless of the frame rate of the input / output image.

It is a figure which shows the structure of the moving image encoder by Embodiment 1 of this invention. It is a flowchart which shows the flow of the process which selects encoding object image data. It is a figure which shows the relationship between output image time and the determination threshold value. It is a flowchart which shows the flow of the other process example which selects encoding object image data. It is a flowchart which shows the flow of the encoding target image data selection process of the frame rate conversion apparatus by Embodiment 2 of this invention. 12 is a flowchart showing a flow of another processing example of encoding target image data selection in the second embodiment. It is a figure which shows the structure of the moving image encoder by Embodiment 3 of this invention. It is a figure which shows the example of an image which displayed time information. It is a figure which shows the structure of the moving image encoder by Embodiment 4 of this invention.

Hereinafter, in order to describe the present invention in more detail, the best mode for carrying out the present invention will be described with reference to the accompanying drawings.
Embodiment 1 FIG.
FIG. 1 is a diagram showing a configuration of a moving picture coding apparatus according to Embodiment 1 of the present invention. As shown in FIG. 1, the moving picture encoding apparatus 1 according to Embodiment 1 includes a frame rate conversion apparatus 2 and an image encoding unit 3. The frame rate conversion apparatus 2 includes an encoding target image determination unit 4 and an image storage unit 5. The frame rate conversion device 2 also includes current image data b captured by an imaging unit (not shown) such as a monitoring camera and time information on a moving image in the current image data b (hereinafter referred to as current time information a). Are entered.

  The encoding target image determination unit 4 determines the level of the input frame rate and the output frame rate based on the current time information a of the current image data b and the number of encoded images, and the image is determined according to the determination result. The encoding target image data c to be output to the encoding unit 3 is determined. The image storage unit 5 stores the input image data to be encoded as past image data d. The past image data d is updated with the current image data b by the encoding target image determination unit 4.

  The image encoding unit 3 encodes the encoding target image data c and outputs it as encoded image data e. In the following, for the sake of simplicity of explanation, the moving picture coding method of coding by the picture coding unit 3 is coding in which the arrangement order of frame images in the coded data is the same as the order of frame images to be actually displayed. It shall be a method.

  Note that in the moving image encoding method, the arrangement order of the frame images in the encoded data may be different from the order of the frame images to be actually displayed using the compression in the time direction. Even in this case, it is possible to perform the frame rate conversion of the present invention described later.

  In the following description, it is assumed that the frame rate of the input image is variable and the frame rate of the output image is fixed. However, this is also for ease of explanation, and since the present invention uses time information of the input image and the output image, whether the frame rate of the input image and the output image is fixed or variable. It doesn't matter.

  The above-described encoding target image determination unit 4 and image storage unit 5 read software and hardware on the computer by causing the computer to read a frame rate conversion program according to the gist of the present invention and controlling its operation. The frame rate conversion apparatus 2 can be realized as a specific means in cooperation with each other.

  Further, the frame rate conversion device 2 and the image encoding unit 3 are also loaded on the computer by reading the frame rate conversion program and the moving image encoding program according to the gist of the present invention into the computer and controlling the operation thereof. Thus, the moving picture coding apparatus 1 can be realized as a specific means in which software and hardware cooperate.

  The configuration of the computer itself and the basic functions thereof can be easily recognized by those skilled in the art based on the common general technical knowledge in the technical field of the present invention, and are not directly related to the essence of the present invention. Is omitted.

Next, the operation will be described.
First, the encoding target image determination unit 4 of the frame rate conversion device 2 inputs the current image data b and the current time information a of the current image data b. Hereinafter, for convenience of explanation, the current time information a is represented by Time _in (i). The parameter i of Time _in (i) is the number of frame images input as the current image data b (hereinafter referred to as the number of input images). The number of encoded frame images output from the image encoding unit 3 (hereinafter referred to as the number of output images) is j. The parameters i and j are all set to 0 as an initial value.

In the initial state (i = j = 0), the encoding target image determination unit 4 unconditionally outputs the current image data b to the image encoding unit 3 as encoding target image data c. The encoding target image determination unit 4 increments the parameter j when the encoding target image data c is output to the image encoding unit 3. The current time information Time _in (0) at this time is set as the input reference time Time _{base_in} as shown in the following formula (1). In addition, 0 is set to the output reference time Time _{base_out} in the initial state.
Time _{base_in} = Time _in (0) (1)

  When the encoding target image determination unit 4 increments the parameter j and inputs the current image data b and the current time information a, i> 0, j> 0, and the encoding target image data after the second frame image. Move to the process of selecting. FIG. 2 is a flowchart showing a flow of processing for selecting second and subsequent encoding target image data, and will be described with reference to this figure.

First, when the current image data b and the current time information a of the current image data b are input (step ST1), the encoding target image determination unit 4 inputs the current time information Time _in (i ) And the input reference time Time _{base_in} , and the difference value Time _{abs_in} (i) is calculated (step ST2). This Time _{abs_in} (i) is a value indicating an absolute time on the input image (current image data b) based on the input reference time Time _{base_in} .
Time _{abs_in} (i)
= Time _in (i) _{-Time base_in} (2)

Next, the encoding target image determination unit 4 substitutes the frame rate Rate _out of the encoded image data e output from the image encoding unit 3, the number j of output images, and the output reference time Time _{base_out} into the following equation (3). Then, an output image display time (hereinafter referred to as output image time) Time _out (j) is obtained (step ST3). The output image time Time _out (j) is a value representing an absolute time on the output image with reference to the output reference time Time _{base_out} .
Time _out (j)
= J / Rate _{out -Time base_out} (3)

The encoding target image determination unit 4 uses the output image time Time _out (j), and uses the following formulas (4), (5), and (6) to determine the level of the frame rate between the input image and the output image. A threshold value is calculated (step ST4). However, Val _high , Val _low , and Val _vlow are predetermined values that define the output image time Time _out (j) when the frame rate Rate _out of the output image is a high frame rate, a low frame rate, or a very low frame rate. Value. Val _high , Val _low , and Val _vlow may be fixed values determined in advance, or values specified by the user may be used.
Time _high (j)
= Time _out (j) -Val _high (4)
Time _low (j)
= Time _out (j) + Val _low (5)
Time _vlow (j)
= Time _out (j) + Val _vlow (6)

The encoding target image determination unit 4 uses the three thresholds Time _high (j), Time _low (j), Time _vlow (j) and the above Time _{abs_in} (i) according to the above equations (4) to (6), The _level of the frame rate of the input image with respect to the output image is determined according to which time among (1) to (4) in FIG. 3 Time _{abs_in} (i) belongs to.

First, the encoding target image determination unit 4 determines whether Time _{abs_in} (i) is less than Time _high (j) (step ST5). At this time, if Time _{abs_in} (i) is less than Time _high (j), the encoding target image determination unit 4 determines that the frame rate of the input image giving this Time _{abs_in} (i) is higher than that of the output image. Then, the process proceeds to step ST11. When Time _{abs_in} (i) is less than Time _high (j), it belongs to the time zone shown in (1) in FIG. 3, and is a predetermined time (Val _high ) before the output image time Time _out (j). This represents a state where the input image is input before the time zone. That is, the j-th output image is any one of the (i + 1) th and subsequent input images, and the i-th input image is not output.

  In step ST11, the encoding target image determination unit 4 updates the past image data d in the image storage unit 5 with the input image (current image data b). This input image is not output to the image encoding unit 3 as the encoding target image data c, but is carried over until the next image is input.

On the other _hand, Time abs_in (i) is not less than Time _high (j), and Time _high (j) greater than the encoding target image determining unit 4, the process proceeds to step ST6. In step ST6, the encoding target image determination unit 4 determines _whether Time _{abs_in} (i) of the input image (current image data b) is greater than or equal to Time _high (j) and less than or equal to Time _low (j).

At this time, if Time _{abs_in} (i) is not less than Time _high (j) and not more than Time _low (j), the encoding target image determination unit 4 determines that the frame rate of the input image giving this Time _{abs_in} (i) is the output image. And the process proceeds to step ST10. When Time _{abs_in} (i) is greater than or equal to Time _high (j) and less than or equal to Time _low (j), it belongs to the time zone indicated by (2) in FIG. 3 and is a predetermined time of output image time Time _out (j). This represents a state in which an input image is input in a time zone from before (Val _high ) to after a predetermined time (Val _low ). That is, the jth output image corresponds to the ith input image.

  In step ST10, the encoding target image determination unit 4 outputs the current image data b as an input image to the image encoding unit 3 as encoding target image data c. In this case, the output image is output at the same frame rate as the input image. Thereafter, the encoding target image determination unit 4 increments the parameter j, outputs the current image data b to the image storage unit 5, and updates the past image data d (step ST11).

On the other hand, in Step ST6, when it is determined that Time _{abs_in} (i) is not Time _high (j) or more and Time _low (j) or less, the encoding target image determination unit 4 proceeds to the process of Step ST7.

In step ST7, the encoding target image determination unit 4 determines _whether Time _{abs_in} (i) of the input image (current image data b) is greater than Time _low (j) and less than or equal to Time _vlow (j). At this time, when Time _{abs_in} (i) is greater than Time _low (j) and less than or equal to Time _vlow (j), the encoding target image determination unit 4 determines that the frame rate of the input image that provides Time _{abs_in} (i) is It determines with it being lower than the frame rate of an output image, and transfers to the process of step ST9.

If Time _{abs_in} (i) is greater than Time _low (j) and less than or equal to Time _vlow (j), it belongs to the time zone indicated by (3) in FIG. 3, and the output image time Time _out (j) This represents a state in which an input image is input in a time zone from a predetermined time (Val _low ) to an output image time Time _out (j) a predetermined time (Val _vlow ). That is, the jth output image corresponds to the (i-1) th input image.

  In step ST9, the encoding target image determination unit 4 refers to the past image data d stored in the image storage unit 5, and outputs the past image data d as the encoding target image data c to the image encoding unit 3. . At this time, the encoding target image determination unit 4 increments the parameter j when the past image data d is output to the image encoding unit 3, returns to the process of step ST3, and executes the processes after step ST3.

Thereafter, the encoding target image determination unit 4 proceeds to the process of step ST6 again to perform time determination, and Time _{abs_in} (i) becomes equal to or higher than Time _high (j) and not higher than Time _low (j), so that the current image data The past image data d is continuously output to the image encoding unit 3 until b is output to the image encoding unit 3 as the encoding target image data c. In this way, the past image data d is repeatedly encoded so that frames are inserted, and the encoded image data e is output at the frame rate set for the output image even if the input image has a lower frame rate than the output image. be able to.

If Time _{abs_in} (i) is larger than Time _vlow (j), that is, belongs to the time zone shown in (4) in FIG. 3, the encoding target image determination unit 4 provides an input for giving Time _{abs_in} (i). It is determined that the frame rate of the image is very low compared to the output image. Note that when Time _{abs_in} (i)> Time _vlow (j), the j-th output image corresponds to the i-th input image. At this time, the encoding target image determination unit 4 once discards the time synchronization between the input image and the output image using the input reference time Time _{base_in} and the output reference time Time _{base_out,} and _{newly performs} time synchronization based on the current time. Do.

More specifically, the encoding target image determination unit 4 outputs the current image data b having a very low frame rate to the image encoding unit 3 according to the following equations (7) and (8). The input reference time Time _{base_in} and the output reference time Time _{base_out} are updated with the current time Time _now (i) at which the data b is input.
Time _{base_in} = Time _now (i) (7)
Time _{base_out} = Time _now (i) (8)

  In this way, the input reference time and the output reference time are updated with the current time, and the input image and the output image are newly synchronized with the current time. Thus, the process of repeatedly encoding the same image (past image data d) is not performed. As a result, the number of encodings can be reduced as compared with the case where frame interpolation is performed as in the prior art, and the intermediate image data for frame interpolation is unnecessary, so the capacity of image data to be stored in the image storage unit 5 The increase can be suppressed.

  After performing the above processing, the encoding target image determination unit 4 outputs the current image data b to the image storage unit 5 and updates the past image data d in step ST11.

  When the encoding target image data c is input from the encoding target image determination unit 4 as described above, the image encoding unit 3 encodes the encoding target image data c and outputs it as encoded image data e. Note that the encoding by the image encoding unit 3 is not limited to a specific encoding method, and an existing moving image encoding method can be appropriately used according to the application field of the present apparatus.

  As described above, according to the first embodiment, the level of the frame rate of the input image relative to the output image is determined from the time of the current image data b that is the input image and the time of the encoding target image data c that is the output image. Since the encoding target image determination unit 4 that determines and controls the number of output frames by selecting an image to be output as the encoding target image data c according to the determination result, the current time information a of the current image data b is provided. Accordingly, it is possible to adaptively execute frame insertion or reduction of the input current image data b. Thereby, frame rate conversion of various images is possible regardless of the frame rate of input / output images.

  The frame rate conversion apparatus 2 according to the first embodiment can convert frame rates of various images, and is particularly effective for encoding surveillance camera images. In general, a surveillance camera continues to capture images of the same point in a surveillance area.

  Here, there are few cases where all of the images taken by the surveillance camera are necessary, and for example, there are many cases where only images taken during a part of time in which a suspicious person is shown are sufficient. In recent years, some surveillance camera systems have a function of recording video data only when there is some movement or change in the video.

  When only the desired video data is selected, the recording of the image is shredded, and transmission of the video signal thus recorded may result in a very low frame rate displayed as a still image.

  In the first embodiment, when the input image has a very low frame rate, frame interpolation is not performed as described above, and the input image is output to the image encoding unit 3 as it is, and is encoded and output. As a result, there is an advantage that it is possible to reduce the encoding and the capacity of an image file to be accumulated when interpolating a moving image with a very low frame rate.

  In the first embodiment, since the past image data d that has already been input is inserted into the output image, there is no need to generate an intermediate image to be inserted between frames as in general frame interpolation. Thereby, processing for generating image data for frame interpolation is unnecessary, and the frame rate conversion apparatus according to the first embodiment can be constructed even by a computer with low processing capability.

In the first embodiment, when the frame rate of the input image is very low compared to the output image, the input / output image reference time Time _{base_in} and the i-th input image are output and encoded. Although an example in which Time _{base_out} is updated has been shown, the reference time is updated after n consecutive i-th input images are output, and the frame rate determination for the next (i + 1) -th input image is performed. May be.

  FIG. 4 is a flowchart showing the flow of another processing example for selecting the encoding target image data. The reference time is updated after n consecutive i-th input images are output, and the next (i + 1) 10 shows processing for determining a frame rate for a second input image. In FIG. 4, the processing from step ST1 to step ST7 and from step ST9 to step ST11 is the same as the processing shown in FIG. 2, but the processing of step ST8-1 and step ST8-2 is performed instead of step ST8. .

In step ST8-1, the encoding target image determination unit 4 outputs the current image data b, which is an input image, to the image encoding unit 3 as n encoding target image data c. Subsequently, the encoding target image determination unit 4 inputs the current image data b according to the following equations (9) and (10) when outputting the current image data b having a very low frame rate to the image encoding unit 3. The input reference time Time _{base_in} and the output reference time Time _{base_out} are updated in consideration of input / output of n images from the current time Time _now (i) (step ST8-2).
Time _{base_in} = Time _now (i) + n × Rate _out (9)
Time _{base_out} = Time _now (i) + n × Rate _out (10)

  Thus, after the reference time is updated after n consecutive i-th input images are output, the frame rate determination process from step ST1 is performed on the next (i + 1) -th input image. When only one image with a very low frame rate is output, if the output image is reproduced at a normal speed, the image may be missed or reproduction may become unnatural. In such a case, it is possible to suppress problems such as missing an image by outputting the same image continuously n times as described above.

Embodiment 2. FIG.
The video encoding apparatus according to the second embodiment has the same basic configuration as that of FIG. 1 described in the first embodiment, but the encoding target image determination unit 4 outputs the frame rate of the input image. Even when the frame rate is higher than the frame rate set for the image, all frame images are encoded as the encoding target image data, and the reference time for time synchronization is updated.

Next, the operation will be described.
FIG. 5 is a flowchart showing a flow of encoding target image data selection processing of the frame rate conversion apparatus according to Embodiment 2 of the present invention. Hereinafter, processing different from that of the first embodiment will be mainly described with reference to FIG. 1 and FIG. The encoding target image determination unit 4 inputs the current image data b and the current time information a of the current image data b (step ST1), and calculates a determination threshold using the above equations (4) to (6) (step The processing up to ST4) is the same as in the first embodiment.

In step ST5a, the encoding target image determination unit 4 determines whether or not Time _{abs_in} (i) of the input image is less than Time _high (j). At this time, if Time _{abs_in} (i) is less than Time _high (j), the encoding target image determination unit 4 determines that the frame rate of the input image (current image data b) is the output image (encoded image data e). It is determined that the frame rate is higher than the set frame rate, and the process proceeds to step ST8a.

In step ST8a, the encoding target image determination unit 4 receives the current time Time _in () in which the current image data b giving Time _{abs_in} (i) less than Time _high (j) is input according to the above formulas (7) and (8). The input reference time Time _{base_in} and the output reference time Time _{base_out} are updated using i).

Thereafter, the encoding target image determination unit 4 _{outputs the} current image data b giving Time _{abs_in} (i) less than Time _high (j) to the image encoding unit 3 as encoding target image data c (step ST10a). . Thereby, even if the frame rate of the input image is higher than that of the output image, all the frame images in the input image are encoded.

  When the encoding target image data c is output to the image encoding unit 3 as described above, the encoding target image determination unit 4 uses the current image data b output as the encoding target image data c to store the image storage unit 5. The past image data d is updated (step ST11a). The other cases are the same as those in the first embodiment.

  As described above, according to the second embodiment, even when the input image has a higher frame rate than the output image, all the images of the input image are to be encoded, and time synchronization is achieved at the current time. Therefore, even when an image for which frame deletion is not permitted is input at a high frame rate, it can be reliably encoded and output.

In the second embodiment, when the frame rate of the input image is very low as compared with the output image, the input / output image reference time Time _{base_in} and the i-th input image are output and encoded. Although an example in which Time _{base_out} is updated has been shown, the reference time is updated after n consecutive i-th input images are output, and the frame rate determination for the next (i + 1) -th input image is performed. May be.

  FIG. 6 is a flowchart showing the flow of another example of processing for selecting image data to be encoded. The reference time is updated after n consecutive i-th input images are output, and the next (i + 1) 10 shows processing for determining a frame rate for a second input image. In FIG. 6, the processing from step ST1 to step ST7, step ST8a, and step ST9 to step ST11a is the same as the processing shown in FIG. 5, but the processing of step ST8a-1 and step ST8a-2 is inserted. .

In step ST8a-1, the encoding target image determination unit 4 outputs the current image data b that is an input image to the image encoding unit 3 as n encoding target image data c. Subsequently, the encoding target image determination unit 4 inputs the current image data b according to the above formulas (9) and (10) when outputting the current image data b having a very low frame rate to the image encoding unit 3. The input reference time Time _{base_in} and the output reference time Time _{base_out} are updated in consideration of input / output of n images from the current time Time _now (i) (step ST8a-2).

  Thus, after the reference time is updated after n consecutive i-th input images are output, the frame rate determination process from step ST1 is performed on the next (i + 1) -th input image. When only one image with a very low frame rate is output, if the output image is reproduced at a normal speed, the image may be missed or reproduction may become unnatural. In such a case, it is possible to suppress problems such as missing an image by outputting the same image continuously n times as described above.

Embodiment 3 FIG.
FIG. 7 is a diagram showing a configuration of a moving picture coding apparatus according to Embodiment 3 of the present invention. In FIG. 7, a moving picture encoding apparatus 1A according to Embodiment 3 includes a frame rate conversion apparatus 2A and an image addition / encoding unit 7. The frame rate conversion device 2A includes an encoding target image determination unit 4a and an image / time information storage unit 6. The frame rate conversion device 2A receives the current image data b captured by an imaging unit (not shown) such as a surveillance camera and the current time information a of the current image data b.

  The encoding target image determination unit 4a and the image / time storage unit 6 read software and hardware on the computer by reading the frame rate conversion program according to the gist of the present invention into the computer and controlling its operation. The frame rate conversion device 2A can be realized as a specific means in which the hardware cooperates.

  In addition, the frame rate conversion device 2A and the image addition / encoding unit 7 also read the frame rate conversion program and the moving image encoding program according to the gist of the present invention into a computer and control the operation thereof. The moving image encoding apparatus 1A can be realized as a specific means in which software and hardware cooperate on a computer.

  The encoding target image determination unit 4a of the frame rate conversion device 2A operates in the same manner as in the first and second embodiments, and determines the image according to the determination result of the frame rate of the input image and the frame rate of the output image. The encoding target image data c to be output to the encoding unit 3 is determined. As operations different from those in the first and second embodiments, in addition to the encoding target image data c, time information of the encoding target image data c (hereinafter referred to as encoding target time information f) is added / encoded. Output to unit 7.

  The encoding target image determination unit 4a may always add the encoding target time information f to the encoding target image data c and output it to the image addition / encoding unit 7, or set a certain period. The encoding target time information f may be added periodically.

  Further, the encoding target time information f may be added according to the determination result of the frame rate of the input image and the frame rate of the output image. For example, the encoding target time information f is added when the frame rate of the input image is higher than that of the output image, or when the frame rate of the input image is very low and the input reference time and the output reference time are updated.

  Similar to the image storage unit 5 in the first and second embodiments, the image / time information storage unit 6 stores the image data to be encoded that is input to the frame rate conversion device 2A. Is stored as past image data d and its time information g. The past image data d and its time information g are updated with the current image data b and its time information a by the encoding target image determination unit 4a.

  The image adding / encoding unit 7 operates in the same manner as the image encoding unit 3 in the first and second embodiments, encodes the encoding target image data c, and outputs it as encoded image data e. As an operation different from that of the image encoding unit 3, encoding is performed after adding the time defined by the encoding target time information f to a part of the image of the encoding target image data c.

  FIG. 8 is a diagram illustrating an example of an image reproduced from the encoded image data e to which the encoding target time information f has been added by the image addition / encoding unit 7. When the encoded image data e is reproduced using the encoding target time information f, the image adding / encoding unit 7 displays the time information f1 on a part of the image as shown in FIG. The encoding target image data c is processed and then encoded. Note that a position other than the example shown in FIG. 8 may be used for the position where the time information f1 is added in the image and the display content thereof.

  As described above, according to the third embodiment, since the time information f1 is displayed on the screen on which the encoded image data e is reproduced, the time on the moving image at which the image is captured is displayed on the screen. The effect that it can confirm is acquired. For example, also in the third embodiment, an input image with a variable frame rate is converted to a fixed frame rate and output, but the output image is reproduced in accordance with the fixed frame rate. At this time, if the input image has a very low frame rate, the time on the moving image may deviate from the actual reproduction time. On the other hand, in the third embodiment, since the time information f1 is displayed on the screen, the correct time information can be confirmed on the screen.

Embodiment 4 FIG.
FIG. 9 is a diagram showing a configuration of a moving picture coding apparatus according to Embodiment 4 of the present invention. In FIG. 9, the moving picture coding apparatus 1B according to the fourth embodiment includes a frame rate conversion apparatus 2A and a DVD-VIDEO creation unit 8. Since the frame rate conversion device 2A is the same as that of the third embodiment, a detailed description of its configuration is omitted.

  The DVD-VIDEO creation unit 8 receives the encoding target image data c and the encoding target time information f, and the encoding target image data c is a moving image encoding method such as the MPEG-2 method adopted for DVD-VIDEO. Encode with The encoded image data is output and recorded on a DVD medium that is an optical disk, thereby creating a DVD-VIDEO.

  Note that the frame rate conversion device 2A and the DVD-VIDEO creation unit 8 read the frame rate conversion program and the moving image encoding program according to the gist of the present invention into a computer and control the operation thereof, thereby Thus, the moving picture coding apparatus 1B can be realized as a specific means in which software and hardware cooperate.

Next, the operation will be described.
When the DVD-VIDEO creation unit 8 inputs the encoding target image data c and the encoding target time information f from the encoding target image determination unit 4a, the encoding target image data c is converted into the MPEG- that has been adopted in the DVD-VIDEO. Encoding is performed by a moving image encoding method such as the two method.

  Here, the DVD-VIDEO has information for setting a playback break called a chapter. When reproducing a DVD-VIDEO, encoded image data can be reproduced from a desired chapter or skipped to the beginning of a chapter.

  The DVD-VIDEO creation unit 8 uses the encoding target time information f at the time on the encoded image data when encoding the encoding target image data c by the moving image encoding method such as the MPEG-2 method. In response, a chapter for specifying the playback position is set. For example, in the encoded image data, the input reference time and the output reference time are updated, and by setting a chapter at the position where the time synchronization is newly started, the reproduction position where the time synchronization is newly started can be easily selected. can do.

  Further, as in the third embodiment, the DVD has a function of processing the encoding target image data c with the encoding target time information f and creating a DVD-VIDEO in which the time information f1 is displayed on a part of the reproduced image. -You may add to the VIDEO creation part 8.

  As described above, the fourth embodiment includes the DVD-VIDEO creation unit 8 that creates the DVD-VIDEO of the encoded image data using the encoding target image data c and the encoding target time information f. It is possible to create a DVD-VIDEO recording encoded image data obtained by adaptively selecting a frame image to be encoded from time information of an input image and time information of an output image and performing frame rate conversion.

  In the fourth embodiment, an example in which a DVD-VIDEO is created has been described. However, the present invention can be applied even when an encoding method or a reproduction medium having the same function is used other than the DVD-VIDEO. Is possible.

  In the first to fourth embodiments, the case where the final output destination is a digital image has been described as an example. However, an analog image may be output as the encoded image data e.

  Furthermore, in the first to fourth embodiments, the case where the frame rate of image data input from an imaging unit (not shown) is converted has been described. However, if the frame rate can be specified, the input image is encoded. It doesn't matter. In this case, for example, the output image can be encoded by a moving image encoding method different from that of the input image, and the moving image encoding device according to the first to fourth embodiments is replaced with a so-called moving image encoding method conversion device. Can be used as

  As described above, since the frame rate conversion device according to the present invention can convert the frame rate of various images regardless of the frame rate of the input / output image, the frame rate conversion device that handles the video signal of the variable frame rate. Suitable for

Claims (8)

  Determining whether the frame rate of the input image relative to the output image is high or low from the time of the input image and the time of the output image, and selecting the image to be output as the output image according to the determination result to control the number of output frames Frame rate conversion device comprising a unit. An image storage unit that stores images input in the past as past images;
The determination unit selects the input image or the past image as an output image according to a determination result of a frame rate of the input image with respect to the output image, and controls the number of output frames. Frame rate conversion device.   The frame rate conversion apparatus according to claim 1, wherein the determination unit adds the time information to the output image and outputs the output image.   A frame for determining the level of the frame rate of the input image with respect to the output image from the time of the input image and the time of the output image, and selecting the image to be output as the output image according to the determination result to control the number of output frames Rate conversion method. Determining whether the frame rate of the input image relative to the output image is high or low from the time of the input image and the time of the output image, and selecting the image to be output as the output image according to the determination result to control the number of output frames A frame rate conversion device having a unit;
A moving image encoding apparatus comprising: an encoding unit that encodes an output image of the frame rate conversion apparatus. Frame rate conversion device
An image storage unit that stores images input in the past as past images;
The determination unit selects the input image or the past image as an output image and controls the number of output frames according to a determination result of the frame rate of the input image with respect to the output image. Video encoding device. The determination unit adds the time information to the output image and outputs it,
6. The moving image encoding apparatus according to claim 5, wherein the encoding unit generates encoded data of the output image in which a time display is added to a reproduced image, using the time information of the output image. The determination unit adds the time information to the output image and outputs it,
The encoding unit uses the time information of the output image to generate encoded data of the output image to which information specifying a reproduction position according to the time on the output image is added. Item 6. A moving image encoding apparatus according to Item 5.

Images