JP2006148409A - Frame rate conversion method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a frame rate conversion method capable of improving deterioration in visibility of an image because of a drop frame in the image when a frame rate is decreased. <P>SOLUTION: In the case of decreasing the frame rate when a rate control section 103 detects that an image coding bit quantity reaches a specified value or over, the method allows a correlation value processing section 304 to output thinned image information on the basis of an image correlation value outputted from an encoder core part 302, a thinning determination section 308 determines a thinned image on the basis of thinned image information, thins frames and thereafter the method equalizes time information attached to data after being multiplexed by a time information adjustment section 309 at an interval equal to or more than the thinned interval. Thus, the method prevents the image from causing a drop frame and realizes that the image is not hard to see. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a frame rate conversion method in high-compression moving image processing typified by MPEG used for DVDs and mobile phones.

  Due to the widespread use of DVDs and mobile phones in recent years, there is a growing demand for higher compression and higher image quality for moving image processing. One technique for achieving high compression while maintaining high image quality is a frame rate conversion process that thins out frames when the bit rate exceeds a certain value.

  In the frame rate conversion of the prior art, a frame that is not referred to by others is thinned out in accordance with the frame rate at that time (see, for example, Patent Document 1).

  In addition, an image is divided into scenes, and a process of determining the frame rate by allocating bits according to the contents of the scene is performed (for example, see Patent Document 2).

  Hereinafter, an example of a frame rate conversion method and a frame rate conversion apparatus that implements the method will be described with reference to the drawings.

  FIG. 11A shows a frame rate conversion apparatus that implements the frame rate conversion method of the prior art. In FIG. 11A, reference numeral 1101 denotes an input control unit for inputting moving image data. Reference numeral 1102 denotes an encode core unit that encodes moving image data. Reference numeral 1109 denotes a buffer that holds output data of the encode core unit 1102. Reference numeral 1103 denotes a rate control unit that controls the frame rate from the bit amount of the encoding result of the encoding core unit 1102. Reference numeral 1104 denotes a thinning unit that thins out the encoded data from the buffer unit 1109 based on the thinning instruction signal from the rate control unit 1103. Reference numeral 1105 denotes an output control unit that outputs the thinned encoded data. Reference numeral 1106 denotes an audio unit that performs audio encoding and outputs audio encoded data. Reference numeral 1107 denotes a memory unit that holds encoded image and audio data. Reference numeral 1108 denotes a multiplexing core unit that acquires encoded image data and audio encoded data from the memory unit 1107, multiplexes the encoded image data and audio encoded data, and outputs the multiplexed data to the memory unit 1107. Indicates. Reference numeral 1111 denotes a video part, and reference numeral 1112 denotes a multiplexing part.

  The prior art frame rate control method configured as described above will be described below.

In the prior art frame rate control method, first, moving image data to be encoded is input to the input control unit 1101 and prefiltering or the like is performed. Next, the encode core unit 1102 receives the output image data of the input control unit 1101 and performs an encoding process. Next, the rate control unit 1103 monitors the bit amount from the encode core unit 1102, and issues a frame rate reduction instruction to the thinning unit 1104 when the bit amount exceeds a certain amount. The thinning unit 1104 thins out the encoded data from the encoding core unit 1102 in accordance with the frame rate reduction instruction from the rate control unit 1103, and outputs it to the output control unit 1105. After the post-filter processing or the like is performed by the output control unit 1105, the encoded data is output to the memory unit 1107. Simultaneously with these image encoding processes, audio is encoded by the audio unit 1106 and output to the memory unit 1107. The multiplexing core unit 1108 acquires the encoded image data and the encoded audio data from the memory unit 1107, multiplexes the encoded image data and the encoded audio data, and receives a frame rate reduction instruction from the rate control unit 1103. After the time information is added according to, the encoding process is completed by storing it in the memory unit 1107. The image of frame decimation when the frame rate is reduced to ½ in the prior art is as shown in FIG.
Japanese Patent Laid-Open No. 2001-258039 JP 2002-058029 A

  However, as shown in FIG. 11-2, since the frame rate conversion method of the prior art performs only the process of thinning out the frame when the frame rate is reduced, the frame is dropped during playback, making it difficult to see. .

  Accordingly, an object of the present invention is to provide a frame rate conversion method capable of improving the difficulty of viewing due to dropped frames during reproduction.

  In order to solve the above-described problems, the first invention provides an input control unit that inputs moving image data, an encoding core unit that encodes moving image data provided from the input control unit, and an encoding result by the encoding core unit A rate control unit that outputs a decimation instruction signal for controlling the frame rate from the bit amount of the image, a buffer unit that holds encoded image data output from the encoding core unit, and a decimation instruction signal from the rate control unit A thinning unit for thinning out the encoded image data held in the buffer unit, an output control unit for outputting the encoded image data and thinning information thinned out by the thinning unit, audio encoding and audio encoded data An audio unit for output, a memory unit for storing image encoded data and audio encoded data, and an output control unit A time information adjustment unit that adjusts time information to be added at the time of multiplexing based on the thinning information; and image encoded data and audio encoded data are acquired from the memory unit; and the time information after adjustment from the time information adjustment unit Is a frame rate conversion method in a frame rate conversion apparatus including a multiplexing core unit that multiplexes image encoded data and audio encoded data based on the data and outputs the multiplexed data to a memory unit.

  In this frame rate conversion method, when the bit amount of the encoding result exceeds the specified value and the frame is thinned, the time information that is thinned by the thinning unit is added to the multiplexed data by the time information adjustment unit. Are equalized at intervals of n times (n is a number greater than 1).

  According to this method, frame reduction is performed by adjusting the time information to be added at the time of multiplexing so that the time information to be added at the time of multiplexing is equal to n times as long as the frame is thinned out when the frame rate is reduced. This makes it possible to reduce the level of visual difficulty and visual comfort.

  In the frame rate conversion method according to the first aspect of the invention, when the decimation instruction signal from the rate control unit is taken into the decimation unit and taken into the time information change instruction unit, when the decimation interval continues for a certain period or more, the time information change instruction unit It is preferable that the time information adjustment unit receives the changed time information signal and gradually approaches the original interval by gradually increasing the time interval between frames.

  According to this method, even when the thinning interval continues for a certain period or more, it is possible to prevent the increase in the buffer capacity by gradually widening the frame interval and reduce the visual difficulty.

  2nd invention controls the frame rate from the input control part which inputs moving image data, the encoding core part which encodes the moving image data given from an input control part, and the bit amount of the encoding result by an encoding core part A rate control unit that outputs a decimation instruction signal, a correlation value processing unit that holds a correlation value between images and outputs decimation image information according to the correlation value, and an output that outputs image encoded data from the encoding core unit A control unit; an audio unit that performs audio encoding and outputs audio encoded data; a memory unit that stores image encoded data, audio encoded data, and thinned image information; and a decimation instruction signal from the rate control unit And a thinning determination unit for outputting thinning information for determining a thinned image from the thinned image information held in the memory unit, and a thinning A time information adjustment unit that adjusts time information to be added at the time of multiplexing based on the thinning information from the determination unit; and necessary image encoded data and audio encoded data from the memory unit based on the thinning information from the thinning determination unit; And a multiplexing core unit that multiplexes the image encoded data and the audio encoded data based on the adjusted time information from the time information adjusting unit and outputs the multiplexed data to the memory unit. This is a frame rate conversion method in a frame rate conversion apparatus.

  In this frame rate conversion method, when the bit amount of the encoding result is equal to or greater than the specified value and the frame is thinned out, the correlation value is based on the correlation value between frames at an interval of n times (n> 0) the original thinning-out interval. The time information added to the multiplexed data is thinned out.

  According to this method, when the frame rate is reduced, the time for thinning out frames with a high correlation value based on the correlation value between frames at an interval n times (n> 0) the original thinning-out interval and adding it to the multiplexed data By equalizing the information, it is possible to reduce the level of frame dropping while suppressing deterioration of the image, and to reduce visual difficulty.

  In the second aspect of the invention, the time information adjustment unit has a time adjustment interval instructing unit for instructing an interval for adjusting the time adjustment, and the interval for thinning out the frames and the interval for adjusting the time information can be set separately. Is preferred.

  According to this method, it is possible to further reduce visual difficulty.

  According to a third aspect of the present invention, an input control unit that inputs moving image data, an encoding core unit that encodes moving image data provided from the input control unit, and a frame rate controlled from a bit amount of an encoding result by the encoding core unit A rate control unit that outputs a decimation instruction signal, a multiple correlation value processing unit that holds correlation values between a plurality of images and outputs decimation image information according to the correlation values, and correlation value data from the multiple correlation value processing units A correlation value buffer, an output control unit that outputs image encoded data from the encode core unit, an audio unit that performs audio encoding and outputs audio encoded data, and image encoded data A memory unit for storing audio encoded data and thinned image information, and a decimation instruction signal from the rate control unit and a memory unit From a thinning determination unit that outputs thinning information for determining a thinned image from the image information, a time information adjustment unit that adjusts time information added at the time of multiplexing based on the thinning information from the thinning determination unit, and a thinning determination unit Necessary image encoded data and audio encoded data are acquired from the memory unit based on the thinning information, and the image encoded data and audio encoded data are obtained based on the time information after adjustment from the time information adjusting unit. A frame rate conversion method in a frame rate conversion apparatus including a multiplexing core unit that performs multiplexing and outputs multiplexed data to a memory unit.

  In this frame rate conversion method, when the bit amount of the encoding result exceeds a specified value and frames are thinned out, frames with high correlation values are thinned out based on the correlation value between frames at an interval n times the original thinning interval. The time information added to the multiplexed data is equalized.

  Here, the original interval will be described. The original interval is an interval between frames when the frame rate is reduced (images are thinned) and the present invention is not applied (conventional example). When the frame rate is 1 / n, the original interval is n times the interval between frames before thinning out images.

  According to this method, when the frame rate is reduced, frames with high correlation values are thinned based on correlation values between frames at an interval n times the original thinning interval, and time information added to multiplexed data is equalized. As a result, it is possible to reduce the level of frame dropping while suppressing deterioration of the image, and to reduce visual difficulty.

  In the third aspect of the invention, it is preferable to have a weighting coefficient buffer for giving a weighting coefficient to the multiple correlation value processing unit, and to change the weighting of the correlation value according to the distance from the reference frame.

  According to this method, it is possible to further reduce visual difficulty.

  4th invention controls the frame rate from the input control part which inputs moving image data, the encoding core part which encodes the moving image data given from an input control part, and the bit amount of the encoding result by an encoding core part A rate control unit that outputs a decimation instruction signal, a correlation value ranking unit that ranks the images for decimation from the correlation values between images, and an output control unit that outputs encoded data from the encoding core unit An audio unit that encodes audio and outputs audio encoded data, a memory unit that holds encoded image and audio encoded data and thinned image rank information, a decimation instruction signal from the rate control unit, and a memory unit A thinning determination unit that outputs thinning information for determining a thinned image from the thinned image rank information held in A time information adjustment unit that adjusts time information to be added at the time of multiplexing based on the thinning information from the thinning determination unit, and necessary image encoded data and audio encoded data from the memory unit based on the thinning information from the thinning determination unit And a multiplexing core unit that multiplexes the image encoded data and the audio encoded data based on the time information after adjustment from the time information adjustment unit and outputs the multiplexed data to the memory unit. This is a frame rate conversion method in the frame rate conversion apparatus.

  In this frame rate conversion method, when the bit amount of the encoding result exceeds a specified value and frames are thinned, information ranked based on correlation values between images at an interval n times the original thinning interval is used. The frames with high correlation values are thinned out, and the time information added to the multiplexed data is equalized.

  According to this method, when the frame rate is reduced, frames with high correlation values are thinned out based on information ranked based on correlation values between images at an interval that is n times the original thinning interval, and the data after multiplexing is thinned out. By equalizing the time information to be added, it is possible to reduce the level of frame dropping while suppressing deterioration of the image and to reduce visual difficulty.

  In the fourth aspect of the invention, it is preferable that the correlation value ranking unit has a rank threshold setting unit that gives a threshold for determining the rank, and the threshold for ranking is made variable.

  According to this method, it is possible to further reduce visual difficulty.

  5th invention is the input control part which inputs moving image data, The intermediate image generation part which produces | generates an intermediate image from the input moving image data given from an input control part, The input image and intermediate image generation from an input control part An encoding core unit that encodes intermediate image data from the image processing unit, a rate control unit that outputs a decimation instruction signal that controls the frame rate from the bit amount of the encoding result by the encoding core unit, and a correlation value between images A correlation value processing unit that outputs thinned-out image information corresponding to the correlation value, an output control unit that outputs encoded data from the encoding core unit and intermediate image encoded data generated from the intermediate image, and audio encoding An audio unit that performs audio output and outputs audio encoded data, image encoded data, intermediate image encoded data, audio encoded data, and A decimation determination unit that outputs a decimation information for determining a decimation image from a decimation instruction signal from the rate control unit and decimation image information retained in the memory unit, and a decimation determination unit from the decimation determination unit A time information adjustment unit that adjusts time information to be added at the time of multiplexing based on the thinning information, and necessary image encoded data and audio encoded data are acquired from the memory unit based on the thinning information from the thinning determination unit, A frame rate conversion device comprising: a multiplexing core unit that multiplexes image encoded data and audio encoded data based on the adjusted time information from the time information adjusting unit and outputs the multiplexed data to the memory unit The frame rate conversion method in FIG.

  In this frame rate conversion method, when the bit amount of the encoding result exceeds a specified value and frames are thinned out, frames with high correlation values are thinned out based on the correlation value between frames at an interval n times the original thinning interval. The time information added to the multiplexed data is equalized.

  According to this method, when the frame rate is reduced, frames with high correlation values are thinned based on correlation values between frames at an interval n times the original thinning interval, and time information added to multiplexed data is equalized. As a result, it is possible to reduce the level of frame dropping while suppressing deterioration of the image, and to reduce visual difficulty. Furthermore, by generating and encoding an intermediate image at the time of image encoding, it is possible to reduce the level of frame dropping while suppressing deterioration of the image, and to further reduce visual difficulty.

  A sixth invention includes an input control unit that inputs moving image data, an encoding core unit that encodes moving image data provided from the input control unit and generates intermediate image encoded data from a motion vector of the intermediate image, From an intermediate image motion vector generation unit that generates a motion vector of an intermediate image from a plurality of motion vectors, which are intermediate data generated by the encoding core unit, and supplies the motion vector to the encoding core unit, and a bit amount of an encoding result by the encoding core unit A rate control unit that outputs a decimation instruction signal for controlling a frame rate, a correlation value processing unit that holds a correlation value between images and outputs decimation image information corresponding to the correlation value, encoded data from the encoding core unit, and Output control unit that outputs intermediate image encoded data and audio encoding and audio encoded data output An audio unit, a memory unit that holds encoded image data, intermediate image encoded data, audio encoded data, and thinned image information, a thinning instruction signal from the rate control unit, and thinned image information held in the memory unit A decimation determination unit that outputs decimation information for determining an image to be decimation from, a time information adjustment unit that adjusts time information to be added at the time of multiplexing based on the decimation information from the decimation determination unit, and decimation information from the decimation determination unit The necessary image encoded data and audio encoded data are obtained from the memory unit, and the image encoded data and the audio encoded data are multiplexed based on the time information after adjustment from the time information adjusting unit. A frame rate conversion method in a frame rate conversion apparatus including a multiplexed core unit that outputs multiplexed data to a memory unit.

  In this frame rate conversion method, when the bit amount of the encoding result exceeds a specified value and frames are thinned out, frames with high correlation values are thinned out based on the correlation value between frames at an interval n times the original thinning interval. The time information added to the multiplexed data is equalized.

  According to this method, when the frame rate is reduced, frames with high correlation values are thinned based on correlation values between frames at an interval n times the original thinning interval, and time information added to multiplexed data is equalized. As a result, it is possible to reduce the level of frame dropping while suppressing deterioration of the image, and to reduce visual difficulty. Furthermore, by generating and encoding the motion vector for the intermediate image during image encoding, it is possible to reduce the level of frame dropping while suppressing image deterioration and further reduce visual difficulty. Become.

  As described above, in the frame rate conversion method of the prior art, since only the process of thinning out the frame is performed when the frame rate is reduced, the section becomes a factor that makes it difficult to see the frame during playback, By using the present invention, after thinning an image, time information can be adjusted and frames can be arranged at equal intervals, the level of dropped frames can be reduced, and visual difficulty can be reduced. .

  Further, by thinning out images with high correlation values based on the correlation values of the images, it is possible to reduce the level of frame dropping while suppressing deterioration of the images, and to reduce visual difficulty.

  Also, by generating and encoding an intermediate image or an intermediate image motion vector at the time of image encoding, it is possible to reduce the level of frame dropping while suppressing image degradation, and to reduce visual difficulty. It becomes possible.

  Embodiment 1 of the present invention will be described below with reference to the drawings.

  FIG. 1-1 illustrates a frame rate conversion apparatus that performs the frame rate conversion method according to Embodiment 1 of the present invention. 1-1, reference numeral 101 denotes an input control unit for inputting moving image data. Reference numeral 102 denotes an encode core unit for encoding moving image data. Reference numeral 103 denotes a rate control unit that controls the frame rate from the bit amount of the encoding result. Reference numeral 104 denotes a buffer unit for storing encoded data. Reference numeral 105 denotes a thinning unit that thins out the encoded data held in the buffer unit 104 based on the thinning instruction signal from the rate control unit 103. Reference numeral 106 denotes an output control unit that outputs the thinned encoded data and thinning information. Reference numeral 107 denotes an audio unit that performs audio encoding and outputs audio encoded data. Reference numeral 108 denotes a memory unit that holds encoded data of images and audio. Reference numeral 109 denotes a time information adjustment unit that adjusts time information added at the time of multiplexing based on the thinning information from the output control unit 106. The code 110 acquires the encoded image data and the encoded audio data from the memory unit 108, and multiplexes the encoded image data and the encoded audio data based on the adjusted time information from the time information adjustment unit 109. The multiplexed core unit that performs multiplexed data output to the memory unit 108 is shown. Reference numeral 111 denotes a video part, and reference numeral 112 denotes a multiplexing part.

  The frame rate conversion method of the present embodiment configured as described above will be described below.

  In the frame rate conversion method according to the first embodiment of the present invention, first, moving image data to be encoded is input to the input control unit 101 and prefiltering or the like is performed. Next, the encode core unit 102 receives the output image data from the input control unit 101, performs an encoding process, and stores the encoded data in the buffer unit 104. Next, the bit amount from the encode core unit 102 is monitored by the rate control unit 103, and if the amount exceeds a certain amount, an instruction to reduce the frame rate is issued to the thinning unit 105. The thinning unit 105 acquires necessary encoded image data from the buffer unit 104 according to the frame rate reduction instruction from the rate control unit 103, and outputs it to the output control unit 108. After the post-filter processing or the like is performed by the output control unit 108, the encoded data is output to the memory unit 108, and the thinning information is output to the time information adjustment unit 109. Simultaneously with these image encoding processes, the audio unit 107 encodes audio and outputs encoded data to the memory unit 108. Based on the thinning information from the output control unit 106 by the time information adjustment unit 109, the time information is adjusted so as to be equally spaced at n times the thinned interval, and the adjusted time information is sent to the multiplexing core unit 110. Output. The multiplexing core unit 110 acquires the encoded image data and the encoded audio data from the memory unit 108, multiplexes the encoded image data and the encoded audio data, and adjusts the adjustment time information from the time information adjustment unit 109. After the time information is added according to, the encoding process is completed by storing it in the memory unit 108. The image of frame decimation when the frame rate is reduced to ½ in this embodiment is as shown in FIG.

  By the above method, it is possible to reduce the frame drop level when the frame rate is reduced, and to reduce visual difficulty.

  In the first embodiment of the present invention as described above, the frame dropping level is reduced when the frame rate is reduced by rearranging the frames at equal intervals at an interval n times the interval at which the frames are thinned. Can be reduced.

  Next, a second embodiment of the present invention will be described with reference to the drawings.

  FIG. 2-1 shows a frame rate conversion apparatus that implements the frame rate conversion method according to Embodiment 2 of the present invention. Since the present embodiment is a development of the above-described first embodiment, the same portions are denoted by the same reference numerals, and redundant description is omitted, and only different structures will be described. In FIG. 2A, when the code 201 continues for a certain period of time based on a decimation instruction from the rate control unit 103, the time information adjustment unit 109 has a frame interval corresponding to the reduced frame rate until it reaches a frame interval. The time information change instruction | indication part which gives the instruction | indication which gradually expands the time interval between frames is shown.

  The frame rate conversion method of the present embodiment configured as described above will be described below.

  In the frame rate conversion method according to the second embodiment of the present invention, the time information change instruction unit 201 determines whether or not the decimation interval continues for a certain amount or more based on the decimation instruction from the rate control unit 103 and continues for a certain amount or more. In this case, the time information adjustment unit 109 is instructed to gradually widen the time interval between frames.

  Since other operations are the same as those of the first embodiment of the present invention, the description thereof is omitted. The image of frame decimation when the frame rate is reduced to ½ in the present embodiment is as shown in FIG.

  In the second embodiment of the present invention as described above, the frame dropping level is reduced when the frame rate is reduced by rearranging the frames at equal intervals at an interval n times the interval at which the frames are thinned. Can be reduced. Further, even when the thinning interval continues for a certain amount or more, it is possible to alleviate visual difficulty by preventing the increase in buffer capacity by gradually widening the frame interval.

  Next, a third embodiment of the present invention will be described with reference to the drawings.

  FIG. 3A illustrates a frame rate conversion apparatus that performs the frame rate conversion method according to Embodiment 1 of the present invention. In FIG. 3A, reference numeral 301 denotes an input control unit for inputting moving image data. Reference numeral 302 denotes an encode core unit that encodes moving image data. Reference numeral 303 denotes a rate control unit that controls the frame rate from the bit amount of the encoding result. Reference numeral 304 denotes a correlation value processing unit that holds correlation values between images and outputs thinned image information. Reference numeral 305 denotes an output control unit that outputs encoded data. Reference numeral 306 denotes an audio unit which performs audio encoding and outputs audio encoded data. Reference numeral 307 denotes a memory unit that holds encoded image and audio data and thinned image information. Reference numeral 308 denotes a thinning determination unit that receives a thinning instruction signal from the rate control unit 303, acquires thinned image information held in the memory unit 307, and determines an image to be thinned based on the thinned image information. Reference numeral 309 denotes a time information adjustment unit that adjusts time information added at the time of multiplexing based on the thinning information from the thinning determination unit 308. The code 310 obtains the image encoded data and the audio encoded data from the memory unit 307, and multiplexes the image encoded data and the audio encoded data based on the adjusted time information from the time information adjusting unit 309. This is a multiplexing core unit that performs multiplexed data output to the memory unit 307. Reference numeral 311 indicates a video unit, and reference numeral 312 indicates a multiplexing unit.

  The frame rate conversion method of the present embodiment configured as described above will be described below.

  In the frame rate conversion method according to the third embodiment of the present invention, first, moving image data to be encoded is input to the input control unit 301 and prefiltering or the like is performed. Next, the encode core unit 302 receives the output image data from the input control unit 301 and performs an encoding process, and outputs the inter-image correlation value to the correlation value processing unit 304 and the encoded data to the output control unit 305. Next, the bit amount from the encode core unit 302 is monitored by the rate control unit 303. When the bit amount exceeds a certain amount, an instruction to reduce the frame rate is issued to the thinning determination unit 308. The correlation value processing unit 304 determines an image having a large correlation value based on the correlation value between images within a range n times the frame rate reduction interval, and stores the thinned image information in the memory unit 307. After the post-filter processing or the like is performed by the output control unit 305, the encoded data is output to the memory unit 307. Simultaneously with these image encoding processes, the audio unit 306 performs audio encoding and outputs encoded data to the memory unit 307. Based on the frame rate reduction instruction from the rate control unit 303 and the thinned-out image information stored in the memory unit 307, the thinning-out determination unit 308 actually thins out a section where the image is thinned out as a section that is n times the frame rate reduction section. Generate image information. Based on the decimation information from the decimation determining unit 308, the time information adjusting unit 309 adjusts the time information so that the time information is equally spaced in n times the frame rate reduction interval. Output to. The multiplexing core unit 310 acquires the encoded image data and the encoded audio data from the memory unit 307, multiplexes the encoded image data and the encoded audio data, and adjusts the adjustment time information from the time information adjustment unit 309. After the time information is added according to the above, the encoding process is completed by storing it in the memory unit 307. In this embodiment, an image of frame decimation when the frame rate is reduced to ½ is as shown in FIG.

  By the above method, it is possible to reduce the frame drop level when the frame rate is reduced, and to reduce visual difficulty.

  With the method as described above, in Embodiment 3 of the present invention, a frame rate is obtained by setting a section in which images are thinned out to be n times a frame rate reduction section, thinning out images in that section, and rearranging frames at equal intervals. At the time of reduction, it is possible to reduce the level of frame dropping and to reduce visual difficulty.

Next, a fourth embodiment of the present invention will be described with reference to the drawings.
FIG. 4A shows a frame rate conversion apparatus that implements the frame rate conversion method according to Embodiment 4 of the present invention. Since this embodiment is an extension of the above-described third embodiment, the same portions are denoted by the same reference numerals, and redundant description is omitted, and only different structures will be described. In FIG. 4A, reference numeral 401 denotes a time adjustment interval instruction unit that instructs the time information adjustment unit 309 to specify an adjustment interval for time information.

  The frame rate conversion method of the present embodiment configured as described above will be described below.

  In the frame rate conversion method according to the fourth embodiment of the present invention, the time information adjustment unit 309 uses the time adjustment interval instruction from the time adjustment interval instruction unit 401 and the thinning information from the thinning unit 308 to determine the frame rate reduction interval. In contrast to the image being thinned out in the n-times section, the time adjustment interval is set in a section different from the n-times section of the frame rate reduction section.

  Since the other operations are the same as those of the third embodiment of the present invention, description thereof is omitted. In this embodiment, an image of frame decimation when the frame rate is reduced to ½ is as shown in FIG.

  By the above method, it is possible to reduce the frame drop level when the frame rate is reduced, and to reduce visual difficulty.

  With the method as described above, in Embodiment 4 of the present invention, the interval in which the image is thinned out is n times the frame rate reduction interval, the image is thinned out in that interval, and the interval in which the frames are rearranged at equal intervals is reduced in frame rate. By setting in a section different from n times the section, it is possible to reduce the level of frame dropping when the frame rate is reduced, and to further reduce visual difficulty.

  Next, a fifth embodiment of the present invention will be described with reference to the drawings.

  FIG. 5A shows a frame rate conversion apparatus that implements the frame rate conversion method according to the fifth embodiment of the present invention. Since this embodiment is an extension of the above-described third embodiment, the same portions are denoted by the same reference numerals, and redundant description is omitted, and only different structures will be described. In FIG. 5A, reference numeral 501 denotes a multiple correlation value processing unit that determines an image to be thinned out based on correlation values of a plurality of images. Reference numeral 502 denotes a correlation value buffer that holds and outputs correlation values from the multiple correlation value processing unit 501.

  The frame rate conversion method of the present embodiment configured as described above will be described below.

  In the frame rate conversion method according to the fifth embodiment of the present invention, moving image data to be encoded is first input to the input control unit 301 and prefiltering or the like is performed. Next, the encode core unit 302 receives the output image data from the input control unit 301 and performs an encoding process, and outputs the correlation value between images to the multiple correlation processing unit 501 and the encoded data to the output control unit 305. Next, the bit amount from the encode core unit 302 is monitored by the rate control unit 303. When the bit amount exceeds a certain amount, an instruction to reduce the frame rate is issued to the thinning determination unit 308. The multiple correlation value processing unit 501 reduces the correlation weight as the distance from the reference image for correlation value determination increases based on a plurality of correlation values between images in a range n times the frame rate reduction interval, and the correlation value is large among them. The image is determined and the thinned image information is stored in the memory unit 307.

  Since the subsequent operation is the same as that of the third embodiment of the present invention, the description thereof is omitted. In this embodiment, an image of frame decimation when the frame rate is reduced to ½ is as shown in FIG.

  By the above method, it is possible to reduce the frame drop level when the frame rate is reduced, and to reduce visual difficulty.

  With the method as described above, in Embodiment 5 of the present invention, it is possible to perform thinning with higher accuracy by extending the target of correlation values for determining the thinned image to a plurality of images, and to reduce visual difficulty in viewing. It becomes possible.

  Next, a sixth embodiment of the present invention will be described with reference to the drawings.

  FIG. 6A shows a frame rate conversion apparatus that implements the frame rate conversion method according to Embodiment 6 of the present invention. Since the present embodiment is a development of the above-described fifth embodiment, the same portions are denoted by the same reference numerals, and redundant description will be omitted, and only different structures will be described. In FIG. 6A, reference numeral 601 denotes a weighting coefficient buffer that holds coefficients for changing the weighting coefficients of correlation values of a plurality of images.

  The frame rate conversion method of the present embodiment configured as described above will be described below.

  In the frame rate conversion method according to the sixth embodiment of the present invention, a coefficient held in the weighting coefficient buffer 601 is used as a weighting coefficient necessary for the multiple correlation value processing unit 501.

  Since other operations are the same as those of the fifth embodiment of the present invention, description thereof will be omitted. In this embodiment, an image of frame decimation when the frame rate is reduced to ½ is as shown in FIG.

  By the above method, it is possible to reduce the frame drop level when the frame rate is reduced, and to reduce visual difficulty.

  With the method as described above, in Embodiment 6 of the present invention, the target of the correlation value for determining the image to be thinned is expanded to a plurality of images, and the weighting value between the images can be made variable, so that the accuracy is higher. Thinning can be performed, and visual difficulty can be further alleviated.

  Next, a seventh embodiment of the present invention will be described with reference to the drawings.

  FIG. 7A shows a frame rate conversion apparatus that implements the frame rate conversion method according to Embodiment 7 of the present invention. Since this embodiment is an extension of the above-described third embodiment, the same portions are denoted by the same reference numerals, and redundant description is omitted, and only different structures will be described. In FIG. 7A, reference numeral 701 denotes a correlation value ranking unit that ranks images to be thinned out based on correlation values between images.

  The frame rate conversion method of the present embodiment configured as described above will be described below.

  In the frame rate conversion method according to the seventh embodiment of the present invention, first, moving image data to be encoded is input to the input control unit 301 and prefiltering or the like is performed. Next, the encode core unit 302 receives the output image data from the input control unit 301 and performs an encoding process, and outputs the correlation value between images to the correlation value ranking unit 701 and the encoded data to the output control unit 305. Next, the bit amount from the encode core unit 302 is monitored by the rate control unit 303. When the bit amount exceeds a certain amount, an instruction to reduce the frame rate is issued to the thinning determination unit 308. The correlation value ranking unit 701 ranks correlation values from the correlation values and the threshold values based on several threshold values in a range of n times the frame rate reduction interval, and stores the thinned image rank information in the memory unit 307. Based on the frame rate reduction instruction from the rate control unit 303 and the decimation image rank information stored in the memory unit 307, the decimation determination unit 308 actually sets the decimation interval as n times the frame rate reduction interval. Generate information about the thinned image.

  Since the subsequent operation is the same as that of the third embodiment of the present invention, the description thereof is omitted. In this embodiment, an image of frame decimation when the frame rate is reduced to ½ is as shown in FIG.

  By the above method, it is possible to reduce the frame drop level when the frame rate is reduced, and to reduce visual difficulty.

  With the method as described above, in Embodiment 7 of the present invention, it is possible to perform thinning with higher accuracy by ranking correlation values that determine thinned images, and to further reduce visual difficulty. It becomes possible.

  Next, an eighth embodiment of the present invention will be described with reference to the drawings.

  FIG. 8A shows a frame rate conversion apparatus that implements the frame rate conversion method according to the eighth embodiment of the present invention. Since the present embodiment is an extension of the above-described seventh embodiment, the same portions are denoted by the same reference numerals, and redundant description is omitted, and only different structures will be described. In FIG. 8A, reference numeral 801 denotes a rank threshold value setting unit that gives a threshold value used in the correlation value ranking unit 701.

  The frame rate conversion method of the present embodiment configured as described above will be described below.

  In the frame rate conversion method according to the eighth embodiment of the present invention, the threshold held by the rank threshold setting unit 801 is used as a threshold required by the correlation value ranking unit 701. The image of frame decimation at this time is as shown in FIG.

  Since other operations are the same as those of the seventh embodiment of the present invention, description thereof will be omitted. In this embodiment, an image of frame decimation when the frame rate is reduced to ½ is as shown in FIG.

  By the above method, it is possible to reduce the frame drop level when the frame rate is reduced, and to reduce visual difficulty.

  With the method as described above, in Embodiment 8 of the present invention, the correlation values for determining the thinned image are ranked, and the threshold value required at that time is made variable, so that more accurate thinning is possible, and visual It becomes possible to further alleviate the general difficulty of viewing.

  Next, a ninth embodiment of the present invention will be described with reference to the drawings.

  FIG. 9A shows a frame rate conversion apparatus for executing the frame rate conversion method according to the ninth embodiment of the present invention. Since this embodiment is an extension of the above-described third embodiment, the same portions are denoted by the same reference numerals, and redundant description is omitted, and only different structures will be described. In FIG. 9A, reference numeral 901 denotes an intermediate image generation unit that generates an intermediate image from a plurality of images.

  The frame rate conversion method of the present embodiment configured as described above will be described below.

  In the frame rate conversion method according to the ninth embodiment of the present invention, first, moving image data to be encoded is input to the input control unit 301 and prefiltering or the like is performed. Next, the encode core unit 302 receives the output image data from the input control unit 301 and performs an encoding process. The inter-image correlation value is transferred to the correlation value processing unit 304, the encoded data is transferred to the output control unit 305, and the picture information is transferred to the intermediate image. The data is output to the generation unit 901. Based on the picture information from the encoder core unit 302, the intermediate image generation unit 901 generates an intermediate image as shown in FIG. 9-2 when it is a non-reference image, and sends it to the encoder core unit 302 to perform encoder processing again. The intermediate image encoded data is output to the output control unit 305. Next, the bit amount from the encode core unit 302 is monitored by the rate control unit 303. When the bit amount exceeds a certain amount, an instruction to reduce the frame rate is issued to the thinning determination unit 308. The correlation value processing unit 304 determines an image having a large correlation value based on the correlation value between images in the range of n times the frame rate reduction interval, and stores the thinned image information in the memory unit 307. After the post-filter processing or the like is performed by the output control unit 305, the encoded data and the intermediate image encoded data are output to the memory unit 307. Simultaneously with these image encoding processes, the audio unit 306 performs audio encoding and outputs encoded data to the memory unit 307. On the basis of the frame rate reduction instruction from the rate control unit 303 and the thinned image information stored in the memory unit 307, the thinning determination unit 308 thins out an image by setting a section for thinning out the image as a section that is n times the frame rate reduction section. In this case, two images that are the target of the correlation value are thinned out, and information for inserting an intermediate image is generated instead. Based on the decimation information from the decimation determining unit 308, the time information adjusting unit 309 adjusts the time information so that the time information is equally spaced in n times the frame rate reduction interval. Output to. The multiplexing core unit 310 obtains image encoded data or intermediate value encoded data and audio encoded data from the memory unit 307, and multiplexes the image encoded data or intermediate value encoded data and the audio encoded data. After adding time information according to the adjustment time information from the time information adjustment unit 309, the encoding process is completed by storing the time information in the memory unit 307. An image of frame decimation when the frame rate is reduced to ½ in this embodiment is as shown in FIG.

  In FIG. 9-2, “motion prediction from an intermediate image” is described. This is because an intermediate image of two images is created in advance at the time of encoding, and encoding based on the intermediate image (motion vector) is performed. It is also said that prediction etc. will be performed at the same time.

  By the above method, it is possible to reduce the frame drop level when the frame rate is reduced, and to reduce visual difficulty.

  With the method as described above, in Embodiment 9 of the present invention, the section where the image is thinned out is set to n times the frame rate reduction section, and when the image is thinned out in that section, the intermediate image is replaced with the encoded image, and the frame is replaced. By rearranging them at equal intervals, it is possible to reduce the level of frame dropping when the frame rate is reduced, and to reduce visual difficulty.

  Next, a tenth embodiment of the present invention will be described with reference to the drawings.

  FIG. 10A illustrates a frame rate conversion apparatus that performs the frame rate conversion method according to the tenth embodiment of the present invention. Since the present embodiment is an extension of the above-described ninth embodiment, the same portions are denoted by the same reference numerals, and redundant description is omitted, and only different structures will be described. In FIG. 10A, reference numeral 1001 denotes an intermediate image motion vector generation unit.

  The frame rate conversion method of the present embodiment configured as described above will be described below.

  In the frame rate conversion method according to the tenth embodiment of the present invention, first, moving image data to be encoded is input to the input control unit 301 and prefiltering or the like is performed. Next, the encode core unit 302 receives the output image data from the input control unit 301 and performs an encoding process, the inter-image correlation value to the correlation value processing unit 304, the encoded data to the output control unit 305, and the motion vector information to the intermediate. This is output to the image motion vector generation unit 1001. Based on the motion information from the encoder core unit 302, the motion vector generation unit 1001 for the intermediate image generates a motion vector corresponding to the intermediate image as shown in FIG. Then, the encoder process is performed again, and the intermediate image encoded data is output to the output control unit 305.

  Since the subsequent operation is the same as that of the ninth embodiment of the present invention, the description thereof is omitted. An image of frame decimation when the frame rate is reduced to ½ in this embodiment is as shown in FIG.

  By the above method, it is possible to reduce the frame drop level when the frame rate is reduced, and to reduce visual difficulty.

  With the method as described above, in the tenth embodiment of the present invention, the section where the image is thinned out is set to n times the frame rate reduction section, and the intermediate image is encoded from the intermediate image motion vector when the image is thinned out in that section. By replacing with images and rearranging the frames at equal intervals, it is possible to reduce the level of frame dropping when the frame rate is reduced, and to reduce visual difficulty.

  Although the above embodiment has been described on the assumption that the processing is performed by software, the same applies to the case where the processing is realized only by hardware using a DSP, a microcomputer, a media processor, or the like. In the embodiment and the drawings, the case where the frame rate is reduced to ½ is described as an example, but this is also an example, and the same applies to the case where the frame rate is reduced to other rates. As described above, the above embodiments are intended to explain the present invention, and the contents of the present invention are not limited thereto.

  The frame rate conversion method according to the present invention makes it possible to reduce the level of frame dropping and to reduce visual difficulty, and can be applied to high-compression moving image processing typified by MPEG and the like used in DVDs and mobile phones. Useful.

It is a block diagram of the frame rate conversion apparatus which implements the frame rate conversion method in Embodiment 1 of this invention. It is an image figure of frame thinning-out in Embodiment 1 of this invention. It is a block diagram of the frame rate conversion apparatus which implements the frame rate conversion method in Embodiment 2 of this invention. It is an image figure of the frame decimation in Embodiment 2 of this invention. It is a block diagram of the frame rate conversion apparatus which implements the frame rate conversion method in Embodiment 3 of this invention. It is an image figure of the frame decimation in Embodiment 3 of this invention. It is a block diagram of the frame rate conversion apparatus which implements the frame rate conversion method in Embodiment 4 of this invention. It is an image figure of the frame decimation in Embodiment 4 of this invention. It is a block diagram of the frame rate conversion apparatus which implements the frame rate conversion method in Embodiment 5 of this invention. It is an image figure of the frame decimation in Embodiment 5 of this invention. It is a block diagram of the frame rate conversion apparatus which implements the frame rate conversion method in Embodiment 6 of this invention. It is an image figure of the frame decimation in Embodiment 6 of this invention. It is a block diagram of the frame rate conversion apparatus which implements the frame rate conversion method in Embodiment 7 of this invention. It is an image figure of the frame thinning-out in 7th and Embodiment 8 of this invention. It is a block diagram of the frame rate conversion apparatus which implements the frame rate conversion method in Embodiment 8 of this invention. It is a block diagram of the frame rate conversion apparatus which implements the frame rate conversion method in Embodiment 9 of the present invention. It is an intermediate image generation figure in Embodiment 9 of this invention. It is an image figure of frame thinning-out in the ninth and tenth embodiments of the present invention. It is a block diagram of the frame rate conversion apparatus which implements the frame rate conversion method in Embodiment 10 of this invention. It is an intermediate image generation figure in Embodiment 10 of this invention. It is a block diagram of the frame rate conversion apparatus which implements the frame rate conversion method of the prior art. It is an image figure of the frame decimation of the frame rate conversion method of the prior art.

Explanation of symbols

101, 301, 1101 Input control unit 102, 302, 1102 Encoder core unit 103, 303, 1103 Rate control unit 104 Buffer unit 105, 1104 Decimation unit 106, 305, 1105 Output control unit 107, 306, 1106 Audio unit 108, 307 DESCRIPTION OF SYMBOLS 1107 Memory part 109,309 Time information adjustment part 110,310,1108 Multiplexing core part 201 Time information change instruction part 308 Thinning determination part 401 Time adjustment interval instruction part 501 Multiple correlation value processing part 502 Correlation value buffer part 601 Weighting Coefficient buffer unit 701 Correlation value ranking unit 801 Rank threshold value setting unit 901 Intermediate image generation unit 1001 Intermediate image motion vector generation unit

Claims (10)

An input control unit for inputting moving image data;
An encoding core unit for encoding moving image data provided from the input control unit;
A rate control unit that outputs a decimation instruction signal for controlling a frame rate from a bit amount of an encoding result by the encoding core unit;
A buffer unit for storing encoded image data output from the encode core unit;
A thinning-out unit that thins out the encoded image data held in the buffer unit based on a thinning-out instruction signal from the rate control unit;
An output control unit for outputting the coded image data and the thinning information thinned out by the thinning unit;
An audio unit that performs audio encoding and outputs audio encoded data; and
A memory unit for holding encoded image data and encoded audio data;
A time information adjustment unit for adjusting time information to be added at the time of multiplexing based on the thinning information from the output control unit;
Image encoded data and audio encoded data are acquired from the memory unit, and the image encoded data and audio encoded data are multiplexed based on the time information after adjustment from the time information adjusting unit, and multiplexed. A frame rate conversion method in a frame rate conversion device comprising a multiplexing core unit for outputting data to the memory unit,
When the bit amount of the encoding result exceeds the specified value and the frame is thinned out, the thinning-out unit thins out the frame, and the time information adjustment unit thins out the time information added to the multiplexed data by n times A frame rate conversion method for equalizing at intervals of (n is a number greater than 1).   When the decimation instruction signal from the rate control unit is taken into the decimation unit and taken into the time information change instruction unit, when the decimation period continues for a certain period or more, the time information adjustment unit changes the time information from the time information change instruction unit 2. The frame rate changing method according to claim 1, wherein a signal is received and the time interval between frames is gradually increased to approach the original interval. An input control unit for inputting moving image data;
An encoding core unit for encoding moving image data provided from the input control unit;
A rate control unit that outputs a decimation instruction signal for controlling a frame rate from a bit amount of an encoding result by the encoding core unit;
A correlation value processing unit that holds correlation values between images and outputs thinned image information according to the correlation values;
An output control unit for outputting image encoded data from the encode core unit;
An audio unit that performs audio encoding and outputs audio encoded data; and
A memory unit for storing image encoded data, audio encoded data, and thinned image information;
A decimation determining unit that outputs decimation information for determining an image to be decimation from the decimation instruction signal from the rate control unit and the decimation image information held in the memory unit;
A time information adjustment unit that adjusts time information added at the time of multiplexing based on the thinning information from the thinning determination unit;
Necessary image encoded data and audio encoded data are acquired from the memory unit based on the decimation information from the decimation determining unit, and the image encoded data is based on the time information after adjustment from the time information adjusting unit. A frame rate conversion method in a frame rate conversion apparatus including a multiplexing core unit that multiplexes audio encoded data and outputs multiplexed data to the memory unit,
When the bit amount of the encoding result exceeds the specified value and frames are thinned out, frames with high correlation values are thinned based on the correlation value between frames at n times (n> 0) the original thinning interval, and after multiplexing A frame rate conversion method for equalizing time information added to the data.   4. The frame rate conversion according to claim 3, wherein the time information adjustment unit has a time adjustment interval instructing unit for instructing an interval for adjusting the time adjustment, and an interval for thinning out frames and an interval for adjusting time information can be set separately. Method. An input control unit for inputting moving image data;
An encoding core unit for encoding moving image data provided from the input control unit;
A rate control unit that outputs a decimation instruction signal for controlling a frame rate from a bit amount of an encoding result by the encoding core unit;
A plurality of correlation value processing units that hold correlation values between a plurality of images and output thinned image information according to the correlation values;
A correlation value buffer for holding correlation value data from the plurality of correlation value processing units;
An output control unit for outputting image encoded data from the encode core unit;
An audio unit that performs audio encoding and outputs audio encoded data; and
A memory unit for storing image encoded data, audio encoded data, and thinned image information;
A decimation determining unit that outputs decimation information for determining an image to be decimation from the decimation instruction signal from the rate control unit and the decimation image information held in the memory unit;
A time information adjustment unit that adjusts time information added at the time of multiplexing based on the thinning information from the thinning determination unit;
Necessary image encoded data and audio encoded data are acquired from the memory unit based on the decimation information from the decimation determining unit, and the image encoding is performed based on the time information after adjustment from the time information adjusting unit. A frame rate conversion method in a frame rate conversion apparatus including a multiplexing core unit that multiplexes data and audio encoded data and outputs multiplexed data to the memory unit,
When the bit amount of the encoding result exceeds the specified value and frames are thinned out, frames with high correlation values are thinned based on the correlation values between frames at an interval n times the original thinning interval and added to the multiplexed data. A frame rate conversion method that equalizes time information.   6. The frame rate conversion method according to claim 5, further comprising a weighting coefficient buffer for giving a weighting coefficient to the plurality of correlation value processing units, and changing the weighting of the correlation value according to the distance from the reference frame. An input control unit for inputting moving image data;
An encoding core unit for encoding moving image data provided from the input control unit;
A rate control unit that outputs a decimation instruction signal for controlling a frame rate from a bit amount of an encoding result by the encoding core unit;
A correlation value ranking unit that ranks images for decimation from correlation values between images;
An output control unit for outputting encoded data from the encode core unit;
An audio unit that performs audio encoding and outputs audio encoded data; and
A memory unit that holds encoded data of image and audio and thinned image rank information;
A decimation determination unit that outputs decimation information for deciphering an image to be decimation from the decimation instruction signal from the rate control unit and the decimation image rank information held in the memory unit;
A time information adjustment unit that adjusts time information added at the time of multiplexing based on the thinning information from the thinning determination unit;
Necessary image encoded data and audio encoded data are acquired from the memory unit based on the decimation information from the decimation determining unit, and the image encoding is performed based on the time information after adjustment from the time information adjusting unit. A frame rate conversion method in a frame rate conversion apparatus including a multiplexing core unit that multiplexes data and audio encoded data and outputs multiplexed data to the memory unit,
When the bit amount of the encoding result is equal to or greater than a specified value and frames are thinned out, frames with high correlation values are thinned based on information ranked based on correlation values between images at n times the original thinning interval, A frame rate conversion method for equalizing time information added to multiplexed data.   8. The frame rate conversion method according to claim 7, further comprising: a rank threshold value setting unit that gives a threshold value for determining a rank to the correlation value ranking unit, and the threshold value for ranking is variable. An input control unit for inputting moving image data;
An intermediate image generating unit that generates an intermediate image from input moving image data provided from the input control unit;
An encoding core unit that encodes an input image from the input control unit and intermediate image data from the intermediate image generation unit;
A rate control unit that outputs a decimation instruction signal for controlling a frame rate from a bit amount of an encoding result by the encoding core unit;
A correlation value processing unit that holds correlation values between images and outputs thinned image information corresponding to the correlation values;
An output control unit for outputting encoded data from the encode core unit and intermediate image encoded data generated from the intermediate image;
An audio unit that performs audio encoding and outputs audio encoded data; and
A memory unit for storing image encoded data, intermediate image encoded data, audio encoded data, and thinned image information;
A decimation determination unit that outputs decimation information for deciphering an image to be decimation from the decimation instruction signal from the rate control unit and the decimation image information held in the memory unit;
A time information adjustment unit that adjusts time information added at the time of multiplexing based on the thinning information from the thinning determination unit;
Necessary image encoded data and audio encoded data are acquired from the memory unit based on the decimation information from the decimation determining unit, and the image encoding is performed based on the time information after adjustment from the time information adjusting unit. A frame rate conversion method in a frame rate conversion apparatus including a multiplexing core unit that multiplexes data and audio encoded data and outputs multiplexed data to the memory unit,
When the bit amount of the encoding result exceeds the specified value and frames are thinned out, frames with high correlation values are thinned based on the correlation values between frames at an interval n times the original thinning interval and added to the multiplexed data. A frame rate conversion method that equalizes time information. An input control unit for inputting moving image data;
An encoding core unit that encodes moving image data provided from the input control unit and generates intermediate image encoded data from a motion vector of the intermediate image;
An intermediate image motion vector generation unit that generates an intermediate image motion vector from a plurality of motion vectors, which are intermediate data generated by the encode core unit, and supplies the generated motion vector to the encode core unit;
A rate control unit that outputs a decimation instruction signal for controlling a frame rate from a bit amount of an encoding result by the encoding core unit;
A correlation value processing unit that holds correlation values between images and outputs thinned image information corresponding to the correlation values;
An output control unit that outputs encoded data and intermediate image encoded data from the encode core unit;
An audio unit that performs audio encoding and outputs audio encoded data; and
A memory unit for storing image encoded data, intermediate image encoded data, audio encoded data, and thinned image information;
A decimation determination unit that outputs decimation information for deciphering an image to be decimation from the decimation instruction signal from the rate control unit and the decimation image information held in the memory unit;
A time information adjustment unit that adjusts time information added at the time of multiplexing based on the thinning information from the thinning determination unit;
Necessary image encoded data and audio encoded data are acquired from the memory unit based on the decimation information from the decimation determining unit, and the image encoding is performed based on the time information after adjustment from the time information adjusting unit. A frame rate conversion method in a frame rate conversion apparatus including a multiplexing core unit that multiplexes data and audio encoded data and outputs multiplexed data to the memory unit,
When the bit amount of the encoding result exceeds the specified value and frames are thinned out, frames with high correlation values are thinned based on the correlation values between frames at an interval n times the original thinning interval and added to the multiplexed data. A frame rate conversion method that equalizes time information.

Images