JP2004048530A - Frame rate converting device and conversion information multiplexer - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem that when the frame rate of a specific moving image is frame rate-converted into a smaller value, a person who views the converted moving image is likely to have a sense of incompatibility against the converted moving image. <P>SOLUTION: This frame rate converting device is provided with a difference arithmetic means 102 for calculating the inter-adjacent frame difference of a specific moving image, a thinned-out frame deciding means 104 for deciding a frame to be thinned out when the frame rate of the specific moving image is converted into a smaller value, and a thinning-out means 105 for thinning out the frame to be thinned out which is decided by the thinned-out frame deciding means 104 from frames configuring the specific moving image. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a frame rate conversion device that converts a frame rate of a specific moving image into a smaller value, a conversion information multiplexing device that multiplexes information necessary for the frame rate conversion into the specific moving image, and the like.
[0002]
[Prior art]
Color television broadcasting systems include the NTSC (National Television Standards Committee) standardized in the United States and the PAL (Phase Alteration by Line) standard developed in West Germany. The NTSC system is a system that outputs 30 frames per second, and is mainly used in Japan, North America, and Latin America. On the other hand, the PAL system outputs 25 frames per second, and is mainly used in Western European countries except France, Asian countries such as China, and Africa.
[0003]
As described above, since the number of output frames per second is different between the NTSC system and the PAL system, a moving image of the NTSC system cannot be directly displayed on a display device such as a PAL system display or monitor.
[0004]
Therefore, in order to display a moving image of the NTSC system on a display device compatible with the PAL system, a frame rate conversion device that converts a frame rate from the NTSC system to the PAL system is used.
[0005]
The frame rate conversion from the NTSC system to the PAL system will be described with reference to FIG. As described above, the NTSC system outputs 30 frames per second, whereas the PAL system outputs 25 frames per second. For this reason, the conventional frame rate conversion device that performs the frame rate conversion from the NTSC system to the PAL system uses a sixth frame and a twelfth frame of the NTSC system as shown in FIGS. 10 (a) and 10 (b). ,... Are output at 25 frames per second. In other words, the conventional frame rate conversion apparatus regularly does not output every sixth frame of the NTSC system, thins out 5 frames per second, and outputs 25 frames per second, thereby converting the NTSC system to the PAL system. Performs frame rate conversion.
[0006]
At this time, the conventional frame rate conversion apparatus outputs the output of each frame so that five frames obtained by thinning one frame from the six frames are output at equal time intervals during a period in which six frames are output in the NTSC system. Controlling time.
[0007]
[Problems to be solved by the invention]
However, as is clear from FIGS. 10A and 10B, the conventional frame rate conversion device does not regularly output every sixth frame of the NTSC system, so that the moving image frame-rate converted to the PAL system is used. In an image, a difference between two adjacent frames may be significantly different from a difference between two adjacent frames immediately before or immediately after a so-called scene change is not performed.
[0008]
For example, in the case of the frame rate conversion in FIG. 10A, the sixth frame of the NTSC system is thinned out, the fifth frame of the NTSC system is followed by the fifth frame, and The seventh frame is output at equal time intervals. In this case, the difference between the fourth frame and the fifth frame of the NTSC system is significantly different from the difference between the fifth frame and the seventh frame. Therefore, a viewer of the PAL-type moving image has a sense of discomfort when viewing the sixth frame of the PAL system (the seventh frame of the NTSC system).
[0009]
Similarly, in the case of FIG. 10A, when the frame rate is converted, the twelfth frame of the NTSC system is thinned out, and the eleventh frame is replaced by the tenth frame of the NTSC system. Then, the thirteenth frame is output at equal time intervals. Also in this case, the difference between the tenth frame and the eleventh frame in the NTSC system is significantly different from the difference between the eleventh frame and the thirteenth frame. The user has an uncomfortable feeling when viewing the eleventh frame of the PAL system (the thirteenth frame of the NTSC system).
[0010]
Also, for the same reason, in FIG. 10B, the viewer of the PAL-type moving image whose frame rate has been converted can see the sixth and eleventh frames of the PAL system (the seventh and 13th frames of the NTSC system). I feel uncomfortable when looking at the second frame.
[0011]
As described above, when the frame rate of a specific moving image is converted to a smaller value by the conventional frame rate conversion device, the converted moving image is not necessarily a scene change. In many cases, a difference between two adjacent frames is greatly different from a difference between two adjacent frames immediately before or immediately after the two adjacent frames. In addition, in the converted moving image, the difference between two adjacent frames is almost the same as the difference between two adjacent frames immediately before or immediately after. Therefore, a viewer of the converted moving image often feels uncomfortable with the moving image.
[0012]
In view of the above, the present invention considers the above-described conventional problems, and when converting the frame rate of a specific moving image to a smaller value, a person who views the converted moving image An object of the present invention is to provide a frame rate conversion device that suppresses a feeling of strangeness.
[0013]
It is another object of the present invention to provide a conversion information multiplexing device that multiplexes information necessary for enabling the frame rate conversion into the specific moving image.
[0014]
[Means for Solving the Problems]
In order to solve the above problems and achieve the above object, the frame rate conversion device of the present invention is based on the premise that the frame rate of a specific moving image is converted to a smaller value, and each adjacent frame of the specific moving image is The image processing apparatus further includes a thinning unit that thins a frame to be thinned determined based on a difference between the frames constituting the specific moving image.
[0015]
According to such a frame rate conversion device of the present invention, the frames to be culled out from a large number of frames constituting the specific moving image are not regularly determined at a predetermined number of frame intervals, It is determined based on the difference between each adjacent frame. Therefore, if the frame rate of the specific moving image is converted to a smaller value using the frame rate converting apparatus of the present invention, the difference between two adjacent frames in the converted moving image is reduced. Phenomenon that the difference between two adjacent frames immediately before or immediately after it is not a so-called scene change, or that it is almost the same as the difference between two adjacent frames immediately before or immediately after that Can be suppressed. As a result, a viewer who views a moving image whose frame rate has been converted by the frame rate conversion device of the present invention is less likely to feel uncomfortable with the converted moving image.
[0016]
Also, the conversion information multiplexing device of the present invention includes a difference calculation unit that calculates a difference between each adjacent frame of a specific moving image, and a multiplexing unit that multiplexes information on the difference into the specific moving image. It is characterized by the following.
[0017]
Further, the conversion information multiplexing apparatus of the present invention is configured to perform frame conversion based on a difference between each adjacent frame of a specific moving image when a frame rate of the specific moving image is converted to a smaller value. , And multiplexing means for multiplexing information on a frame to be thinned out into the specific moving image.
[0018]
By using the information on the difference or the information on the frame to be thinned out from the conversion information multiplexing device of the present invention, the difference between two adjacent frames is not necessarily a scene change. Suppressing the occurrence of the phenomenon that the difference between two adjacent frames immediately before or immediately after that is substantially different from the difference between two immediately adjacent frames immediately before or immediately after the occurrence of the phenomenon described above, The frame rate of a moving image can be converted to a smaller value.
[0019]
BEST MODE FOR CARRYING OUT THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings.
[0020]
(Embodiment 1)
FIG. 1 shows a configuration of frame rate conversion apparatus 100 according to Embodiment 1 of the present invention, and FIG. 2 shows an operation procedure of frame rate conversion apparatus 100 according to Embodiment 1 of the present invention.
[0021]
The frame rate conversion device 100 according to the first embodiment includes a specific moving image 201 (hereinafter, referred to as a “first image”) composed of 30 frames per second in the NTSC system as shown in FIG. 3A and FIG. As shown in FIG. 3B and FIG. 4B, for example, as shown in FIG. 3B and FIG. 4B, 5 frames per second are thinned out from a number of frames constituting the first moving image 201, and PAL is performed. This is a device for converting a moving image 202 composed of 25 frames per second (hereinafter, referred to as a “second moving image 202”). The first moving image 201 shown in FIGS. 3A and 4A is a moving image of the NTSC system shown in FIGS. 10A and 10B used in the description of the above-described conventional technology. This is the same moving image as the image.
[0022]
As shown in FIG. 1, the frame rate conversion apparatus 100 according to the first embodiment includes a storage unit 101, a difference calculation unit 102, a difference information storage unit 103, a thinned frame determination unit 104, and a thinning unit 105. It is configured.
[0023]
The accumulation unit 101 accumulates a first moving image 201 composed of, for example, 30 frames per second as shown in FIG. 3A or FIG. 4A input from the outside (step 1 in FIG. 2). .
[0024]
Then, the difference calculation means 102 performs the first moving image stored in the storage means 101 as preprocessing for determining a frame to be thinned out in the frame rate conversion from among a number of frames forming the first moving image 201. A difference between corresponding pixels between adjacent frames forming the image 201 is calculated, and a difference evaluation value between adjacent frames is determined based on the calculation result (step 2 in FIG. 2). The difference evaluation value is information used when determining a frame to be thinned out in the frame rate conversion, as described later. In the present embodiment, for convenience of description, the difference between corresponding pixels between adjacent frames means a difference between density values of the corresponding pixels. However, the difference between the luminance values of the corresponding pixels may be the difference between the corresponding pixels of the adjacent frames. Hereinafter, the difference evaluation value will be described.
[0025]
For example, the relationship between the third frame and the fourth frame in FIG. 3A or the relationship between the seventh frame and the eighth frame in FIG. In the case where the contents of the two frames to be calculated are significantly different and the sum of the absolute values of the differences between the corresponding pixels between the two frames to be calculated is greater than or equal to a predetermined size , The difference calculation means 102 determines the difference evaluation value between the two frames to be calculated as the maximum value 10.
[0026]
Also, as in the case of the two frames shown in FIG. 5 and the relationship between the second frame and the third frame in FIG. 4A, the contents of two adjacent calculation target frames are completely the same. If all the difference values between the corresponding pixels between the two frames to be calculated are 0, the difference calculation unit 102 determines the difference evaluation value between the two frames to be calculated to be the lowest value 0. .
[0027]
The difference calculation means 102 determines the maximum value 10 and the minimum value 0 of the difference evaluation value in this manner, and then determines the sum of the absolute values of the differences between the corresponding pixels between the two frames to be calculated. A difference evaluation value between two frames of each calculation object when the value exceeds 0 and is smaller than the predetermined size is determined as follows. That is, the difference calculation means 102 sets the difference evaluation value between the two frames to be calculated to the maximum value 10 and the magnitude of the sum of the absolute values of the differences between the corresponding pixels between the two frames to be calculated. It is determined as a value obtained by multiplying the ratio with respect to the predetermined size.
[0028]
For example, in the relationship between the fifth frame and the sixth frame in FIG. 3A, the magnitude of the sum of the absolute values of the differences between the corresponding pixels between the two frames to be calculated is determined in advance by the above-described method. If it is の of the determined predetermined size, the difference calculation means 102 calculates the difference evaluation value between the two frames to be calculated as 5 obtained by multiplying the maximum value 10 by １ ／. decide.
[0029]
Further, in the relationship between the second frame and the third frame in FIG. 3A, the magnitude of the sum of the absolute values of the differences between the corresponding pixels between the two frames to be calculated is determined in advance by the above-described method. If it is 3/10 of the determined predetermined size, the difference calculation means 102 calculates the difference evaluation value between the two frames to be calculated as 3 obtained by multiplying the highest value 10 by 3/10. decide.
[0030]
Further, in the relationship between the sixth frame and the seventh frame in FIG. 3A, the magnitude of the sum of the absolute values of the differences between the corresponding pixels between the two frames to be calculated is determined in advance by the above-described method. If it is 7/10 of the determined predetermined size, the difference calculating means 102 calculates the difference evaluation value between the two frames to be calculated as 7 obtained by multiplying the highest value 10 by 7/10. decide.
[0031]
Note that FIGS. 3A and 4A also show the frames constituting the first moving image 201, as well as the difference evaluation values between adjacent frames determined by the above-described method.
[0032]
As described above, when the difference evaluation value between the two adjacent frames is determined by the difference calculation means 102, the difference information accumulation means 103 determines each of the determined difference evaluation values as the difference evaluation value. Are stored in pairs with information specifying two adjacent frames used for this purpose (step 3 in FIG. 2).
[0033]
Next, based on the difference evaluation values accumulated in the difference information converting means 103, the thinned frame determining means 104 thins out a number of frames constituting the first moving image 201 in frame rate conversion. A predetermined number of frames are determined every second (Step 4 in FIG. 2). The frame rate conversion apparatus 100 according to the present embodiment is an apparatus that converts the NTSC system composed of 30 frames per second into the PAL system composed of 25 frames per second. Five frames per second are determined as frames to be subtracted.
[0034]
Hereinafter, a method of determining the frame to be thinned by the thinned frame determining unit 104 will be specifically described.
[0035]
First, when determining the number of frames to be thinned out at 5 frames per second, the thinned-out frame determining means 104 converts continuous difference evaluation values into a predetermined value in order to distribute the five frames to be thinned out as uniformly as possible in time. Grouping into groups composed of the number of difference evaluation values. In the present embodiment, since the frame rate conversion from the NTSC system to the PAL system is performed, the number of difference evaluation values to be present in each group is reduced to 30 from the number of difference evaluation values existing in one second to one second. It is preferable to set it to 6 divided by the number of frames to be inserted.
[0036]
This is because if the five frames to be thinned are assumed to be consecutive five frames, the frame of the joint portion after the thinned five frames will be displayed at a time significantly different from the time at which the frames should be originally displayed. Because.
[0037]
Hereinafter, a specific example of grouping continuous difference evaluation values performed by the thinned frame determination unit 104 will be described with reference to FIGS. 3, 4, and 6. FIG.
[0038]
In the example of FIG. 3A, the thinned-out frame determination means 104 is configured to convert successive difference evaluation values A, B, C, D, E,... Into six successive difference evaluation values A to F. Are grouped into a first group 501 and a second group 502 composed of six consecutive difference evaluation values G to N.
[0039]
In the example of FIG. 4A, the thinned frame determination unit 104 converts the continuous difference evaluation values A ′, B ′, C ′, D ′, E ′,. A first group 501 ′ composed of '' F 'and a second group 502' composed of six successive difference evaluation values G'〜N 'are grouped thereafter.
[0040]
FIG. 6A is a conceptual diagram of a frame for one second and a difference evaluation value of the first moving image 201 composed of 30 frames per second. In the example shown in FIG. 6A, the thinned-frame determining unit 104 converts successive difference evaluation values for one second into five groups 601 and 610, each of which includes six successive difference evaluation values from the top. 602, 603, 604 and 605 are grouped.
[0041]
As described above, when successive difference evaluation values are grouped, the thinned-frame determining unit 104 next checks whether or not there is a difference evaluation value having a value of 0 or 10 in each group. .
[0042]
This is because, for example, two adjacent frames used for calculating the difference evaluation value C whose value in FIG. 3A is 10 and the difference evaluation value B ′ whose value in FIG. This is because the frame is either a frame at the time of a so-called scene change or a frame having exactly the same contents. Since the relationship between the so-called scene change and the two frames sandwiching the thinned frame is a so-called scene change, one of the adjacent frames at the time of the so-called scene change has a frame rate conversion. The viewer of the second moving image 202 does not have a sense of discomfort. Further, even if one of adjacent frames having exactly the same contents is to be thinned out, the amount of information displayed by the frame rate-converted second moving image 202 does not change. A person who views the image 202 does not feel uncomfortable. Therefore, the thinned frame determination unit 104 checks whether or not there is a difference evaluation value having a value of 0 or 10 for each group.
[0043]
As a result of the investigation, if there is only one difference evaluation value having a value of 0 or 10 in each group, the thinned frame determination unit 104 uses the difference evaluation value when determining a frame to be thinned. It is specified as the difference evaluation value of interest. Further, when there are a plurality of difference evaluation values having a value of 0 or 10 in each group, the thinned-frame determining unit 104 determines the difference evaluation value closest to the head among the plurality of difference evaluation values. , The target difference evaluation value. On the other hand, when there is no difference evaluation value having a value of 0 or 10 in each group, the thinned frame determination unit 104 specifies the last difference evaluation value in the group as the difference evaluation value of interest.
[0044]
For example, in the example of FIG. 3A, since only the difference evaluation value C is 10 and the other difference evaluation values are neither 0 nor 10 for the first group 501, the thinned frame determination unit 104 Specifies the difference evaluation value C as the focused difference evaluation value for the first group 501. Next, in the case of the second group 502, since the difference evaluation value G and the difference evaluation value J are 10, and the other difference evaluation values are neither 0 nor 10, the thinned frame determination unit 104 Regarding the second group 502, the difference evaluation value G closest to the head of the difference evaluation value G and the difference evaluation value J is specified as the difference evaluation value of interest.
[0045]
In the example of FIG. 4A, only the difference evaluation value B ′ and the difference evaluation value J ′ are 0 for the first group 501 ′ and the second group 502 ′, respectively, and the other difference evaluation values are different. The values are neither 0 nor 10. For this reason, the thinned frame determination unit 104 specifies the difference evaluation value B ′ as the focused difference evaluation value for the first group 501 ′, and determines the difference evaluation value J ′ for the second group 502 ′. Specify a value.
[0046]
Next, in FIG. 6A, the background of the difference evaluation value whose value is 0 or 10 is displayed in black. In the example of FIG. 6A, only one difference evaluation value having a value of 0 or 10 exists in each of the groups 601, 604, and 605 (the difference evaluation value 601c, the difference evaluation value 604a, and the difference evaluation value 605d). ), There are two difference evaluation values having a value of 0 or 10 in the group 602 (difference evaluation value 602a, difference evaluation value 602c), and the difference evaluation value having a value of 0 or 10 in the group 603 is Not at all.
[0047]
For this reason, the thinned frame determining unit 104 specifies the difference evaluation values 601c, 604a, and 605d of which the background is displayed in black for the groups 601, 604, and 605 as the difference evaluation values of interest. For the group 602, since the difference evaluation value 602a and the difference evaluation value 602c are 0 or 10, the thinned-frame determining unit 104 calculates the difference evaluation value 602a of the group It is specified as the difference evaluation value of interest. Since there is no difference evaluation value having a value of 0 or 10 for the group 603, the thinned-frame determining unit 104 calculates the difference evaluation value 603f marked with the last star in the group. It is specified as the difference evaluation value of interest.
[0048]
As described above, in Embodiment 1, as one mode of implementation, when there are a plurality of difference evaluation values having a value of 0 or 10 in each group, the plurality of difference evaluation values Of the difference evaluation values that are closest to the head are specified as the difference evaluation values of interest. If there are a plurality of difference evaluation values having a value of 0 or 10 in the group, Either of them may be specified as the focused difference evaluation value. In the first embodiment, as one mode of implementation, when there is no difference evaluation value having a value of 0 or 10 in a group, the last difference evaluation value in the group is regarded as the difference evaluation value of interest. Although it is specified, any difference evaluation value in the group may be specified as the focused difference evaluation value.
[0049]
When the target difference evaluation values are specified one by one for each group in this way, the thinned frame determining unit 104 next determines the frames to be thinned in the frame rate conversion as described later.
[0050]
In other words, for each group, if the difference evaluation value of interest is not the last difference evaluation value of the group, the thinning frame determination unit 104 determines that the rearmost frame of the two adjacent frames that specify the difference evaluation value of interest is Is determined as the frame to be decimated. On the other hand, for each group, if the difference evaluation value of interest is the last difference evaluation value of the group, the thinned-out frame determination unit 104 determines the first frame of the two adjacent frames that specify the difference evaluation value of interest. Is determined as a frame to be thinned.
[0051]
For example, in the example of FIG. 3A, since the difference evaluation value C is the difference evaluation value of interest for the first group 501, the thinned frame determination unit 104 determines the third frame specifying the difference evaluation value C. And the fourth frame on the tail side of the fourth frame is determined as the frame to be thinned. For the second group 502, since the difference evaluation value G is the difference evaluation value of interest, the thinned-out frame determination unit 104 determines whether the difference evaluation value G is specified on the rear side of the seventh and eighth frames. Is determined as the frame to be thinned.
[0052]
In addition, in the example of FIG. 4A, for the first group 501 ′, the difference evaluation value B ′ is the difference evaluation value of interest, so the thinned frame determination unit 104 specifies the difference evaluation value B ′. The third frame on the tail side of the second frame and the third frame is determined as the frame to be thinned. For the second group 502 ′, since the difference evaluation value J ′ is the target difference evaluation value, the thinned-out frame determination unit 104 determines the difference evaluation value J ′ between the ninth frame and the tenth frame. Is determined as the frame to be thinned out.
[0053]
Next, in the example of FIG. 6A, for the group 601, the difference evaluation value 601c is the difference evaluation value of interest, and the difference evaluation value of interest is not the last difference evaluation value of the group 601. The thinned frame determining unit 104 determines the fourth frame on the trailing side of the third frame and the fourth frame that specify the difference evaluation value 601c as the frame to be thinned. Similarly, for the groups 602, 604, and 605, the thinned-out frame determination unit 104 calculates the tail eighth and twentieth frames of the adjacent two frames that respectively specify the difference evaluation values 602 a, 604 a, and 605 d. , The 29th frame is determined as the frame to be thinned. Also, for the group 603, the difference evaluation value of interest is the last difference evaluation value 603f in the group 603, and therefore the thinning frame determination unit 104 determines whether the 18th frame and the 19th frame specify the difference evaluation value 603f. Is determined as the frame to be thinned out.
[0054]
When the frames to be thinned out are determined by such a method, as is apparent from FIGS. 3 (a), 4 (a) and 6 (a), 30 frames existing in one second are evenly continuous. Frames to be decimated can exist one by one in each group when the frame is divided into five groups. As a result, it is possible to temporally disperse the five frames thinned out in one second as temporally as possible.
[0055]
As described above, in the first embodiment, as one mode of implementation, when the difference evaluation value of interest is not the last difference evaluation value of the group for each group, the adjacent difference specifying the difference evaluation value of interest is specified. The frame on the tail side of the two frames to be processed is determined as the frame to be thinned. If the target difference evaluation value is the last difference evaluation value of the group, the leading frame of two adjacent frames that specify the target difference evaluation value is determined as the frame to be thinned out. However, as described above, even if any of the two adjacent frames that specify the target difference evaluation value is determined to be a frame to be thinned out, a person who views the frame rate-converted second moving image 202 does not feel uncomfortable. Do not have. Therefore, the present invention is not limited to the above-described method, and it is only necessary to determine one of two adjacent frames that specify the target difference evaluation value as a frame to be thinned.
[0056]
For example, for each group, the leading frame of two adjacent frames that specify the target difference evaluation value may be determined as the frame to be thinned. In the example of FIG. 3A, since the difference evaluation value C is the difference evaluation value of interest for the first group 501, the thinned frame determination unit 104 determines whether the third frame specifying the difference evaluation value C The leading third frame of the four frames may be determined as the frame to be thinned.
[0057]
As described above, when the frame to be thinned out is determined by the thinned-out frame determining unit 104, the thinning-out unit 105 selects the above-mentioned frame from the many frames constituting the first moving image 201 stored in the storage unit 101. The frames constituting the first moving image 201 are sequentially output to the outside by thinning out the frames to be thinned out (step 5 in FIG. 2). At this time, the thinning means 105 outputs five frames obtained by thinning one frame from the six frames during a period in which six consecutive frames constituting the first moving image 201 are to be originally output in the NTSC system at equal time intervals. So that the output time of each frame is controlled.
[0058]
In the example of FIG. 3A, as described above, the fourth frame and the eighth frame among the 13 frames forming the first moving image 201 are thinned out by the thinned frame determining unit 104. It has been determined that there is. Therefore, as shown in FIG. 3B, the thinning means 105 thins out the fourth frame and the eighth frame from the 13 frames shown in FIG. 3A, and makes the first frame → the second frame → the third frame. Frames are output at equal time intervals in the order of frame → fifth frame → sixth frame → seventh frame → ninth frame → tenth frame → eleventh frame → twelfth frame → thirteenth frame, and the frame rate The converted second moving image 202 is output. At this time, during a period in which a total of six frames from the first frame to the sixth frame constituting the first moving image 201 should be originally output in the NTSC system, a total of five frames from the first frame to the sixth frame excluding the fourth frame are provided. In order that the frames are output at equal time intervals, and during the period in which the total of six frames from the seventh frame to the twelfth frame constituting the first moving image 201 are to be originally output in the NTSC system, the eighth frame is output. The thinning-out means 105 controls the output time of each frame so that a total of five frames from the seventh frame to the twelfth frame are output at equal time intervals.
[0059]
In the example of FIG. 4A, as described above, of the thirteen frames forming the first moving image 201, the third frame and the tenth frame are thinned out by the thinned frame determining unit 104. It has been determined that there is. Therefore, as shown in FIG. 4B, the thinning means 105 thins out the third and tenth frames from the 13 frames shown in FIG. 4A, and outputs each frame at equal time intervals. Then, the second moving image 202 having undergone the frame rate conversion is output.
[0060]
In the example of FIG. 6A, the decimating unit 105 includes the fourth, eighth, eighteenth, and twentieth frames determined by the decimating frame determining unit 104, as shown in FIG. , And the 29th frame are thinned out, and the remaining frames constituting the first moving image 201 are sequentially output at equal time intervals, and the second moving image 202 whose frame rate has been converted is output.
[0061]
In this way, the frame rate conversion apparatus 100 according to the first embodiment uses the first 30 frames / sec NTSC frame shown in FIGS. 3A, 4A, and 6A. The moving image 201 is subjected to frame rate conversion into a second moving image 202 composed of 25 frames per second in the PAL system shown in FIGS. 3B, 4B, and 6B.
[0062]
As a result, as is clear when comparing FIG. 3B and FIG. 10A, and as clear when comparing FIG. 4B and FIG. Regarding the second moving image 202 of FIGS. 3B and 4B that has undergone the frame rate conversion by the frame rate conversion device 100, the difference between two adjacent frames is a so-called scene change. Although this is not necessarily the case, the phenomenon that the difference between two adjacent frames immediately before or immediately after that is greatly different is suppressed. Further, the occurrence of the phenomenon that the difference between two adjacent frames does not differ from the difference between two adjacent frames immediately before or immediately after that is also suppressed. Therefore, a person who views the second moving image 202 is less likely to feel uncomfortable with the second moving image 202.
[0063]
In the first embodiment, for the sake of convenience, the frame rate conversion apparatus 100 according to the first embodiment converts the first moving image 201 composed of 30 frames per second of the NTSC system to 25 frames per second of the PAL system. Although the conversion into the second moving image 202 composed of frames is described, the frame rate conversion is not limited to the conversion from 30 frames per second to 25 frames per second. The frame rate conversion includes, for example, (1) conversion from a first moving image 201 composed of 30 frames per second in the NTSC system to a moving image of a movie film composed of 24 frames per second, and (2) PAL. The conversion from a moving image composed of 25 frames per second to a moving image of a movie film composed of 24 frames per second is also included. Further, for example, (3) conversion from the first moving image 201 composed of 30 frames per second of the NTSC system to a moving image composed of 22 frames per second is also included.
[0064]
In the case of the above (1) conversion from the first moving image 201 composed of 30 frames per second in the NTSC system to the moving image of a movie film composed of 24 frames per second, unlike the first embodiment described above, Since it is necessary to thin out six frames per second, the thinned frame determining unit 104 first converts the 30 difference evaluation values for each second into, for example, six groups each consisting of five successive difference evaluation values. Group. Then, the thinned frame determining unit 104 determines one focused difference evaluation value for each of the six groups as described above.
[0065]
Further, in the case of (2) the conversion from a moving image composed of 25 frames per second in the PAL system to a moving image of a movie film composed of 24 frames per second, it is necessary to thin one frame per second. The deciding means 104 decides one difference evaluation value of interest one by one from the 25 difference evaluation values per second as described above.
[0066]
In addition, in the case of (3) conversion from the first moving image 201 composed of 30 frames per second in the NTSC system to a moving image composed of 22 frames per second, it is necessary to thin out 8 frames per second. In order to distribute the eight frames to be thinned out as evenly as possible in time, the thinned-out frame determination unit 104 firstly constructs 30 difference evaluation values for each one second from four continuous difference evaluation values. The group is grouped into a total of eight, that is, six groups and two groups each including three consecutive difference evaluation values. Then, the thinned-out frame determination unit 104 determines the target difference evaluation values one by one for each of the eight groups as described above.
[0067]
The operations of the thinned-out frame determining unit 104 and the thinning-out unit 105 after the target difference evaluation value is determined are the same as the operations of the thinned-out frame determining unit 104 and the thinning-out unit 105 described above.
[0068]
In the first embodiment, the thinned frame determining unit 104 determines a frame to be thinned using the difference evaluation value between adjacent frames. However, the thinned frame determining unit 104 may determine the frame to be thinned by using the magnitude itself of the sum of the absolute values of the differences between the corresponding pixels between adjacent frames, instead of using the above-described difference evaluation value. . Alternatively, if the first moving image 201 has been compression-encoded by MPEG2, the thinned-out frame determination unit 104 determines the difference between the mutually correlated blocks between adjacent frames in the compression-encoded data. May be used to determine the frames to be thinned.
[0069]
Further, in the first embodiment described above, the difference evaluation value that can be the target difference evaluation value is a difference evaluation value having a value of 0 or 10, but instead of the difference evaluation value having a value of 0, a value May be any of 0 to 2 as a difference evaluation value that can be a difference evaluation value of interest. In this case, in the example of FIG. 3A, for the first group 501, the difference evaluation value A having a value of 2 together with the difference evaluation value C can be a candidate for the noted difference evaluation value. Similarly, instead of the difference evaluation value having a value of 10, a difference evaluation value having a value of any of 8 to 10 may be set as a difference evaluation value that can be a target difference evaluation value. In that case, in the example of FIG. 4A, for the first group 501 ′, the difference evaluation value A ′ having a value of 9 can be a candidate for the difference evaluation value of interest together with the difference evaluation value B ′.
[0070]
As described above, when there are a plurality of candidate difference evaluation value candidates in the same group, the difference evaluation value closest to the maximum value 10 or the minimum value 0 among the plurality of candidate difference evaluation value candidates is determined. The difference evaluation value of interest may be used. This is because the difference evaluation value closest to the maximum value 10 or the minimum value 0 is a difference evaluation value that specifies two frames whose contents are greatly changed, as in a so-called scene change, or This is because the difference evaluation value specifies two adjacent frames having a very small difference. Therefore, if the difference evaluation value closest to the maximum value 10 or the minimum value 0 is taken as the difference evaluation value of interest, one of the adjacent frames at the time of a so-called scene change or one of the adjacent frames having exactly the same contents is specified. Thus, it is possible to specify a frame that provides the same effect.
[0071]
In the case where the same group includes a plurality of difference evaluation values whose values are any one of 0 to 2 or any one of 8 to 10 and are separated from the maximum value 10 or the minimum value 0 by the same number, The evaluation value may be used as the difference evaluation value of interest. For example, when a difference evaluation value having a value of 1 and a difference evaluation value having a value of 9 exist in the same group, the difference evaluation value having a value of 1 may be used as the difference evaluation value of interest. The difference evaluation value of 9 may be used as the difference evaluation value of interest.
[0072]
In the first embodiment, the thinned-out frame determining unit 104 groups the continuous 30 difference evaluation values into five groups each including six continuous difference evaluation values, and The target difference evaluation values are determined one by one. However, for example, as shown in FIG. 6A, two difference evaluation values having a value of 0 or 10 exist in the group 602, and no difference evaluation value having a value of 0 or 10 exists in the group 603. In this case, the thinned-frame determining unit 104 determines both of the two difference evaluation values having a value of 0 or 10 in the group 602 as the noted difference evaluation values, and includes no noted difference evaluation value in the group 603. You may not make it exist. In this way, the thinned-out frame determination unit 104 only needs to determine the focused difference evaluation value such that the same number of focused difference evaluation values as the number of frames to be thinned exist in one second.
[0073]
(Embodiment 2)
FIG. 7 shows the configurations of frame rate conversion apparatus 700 and conversion information multiplexing apparatus 701 according to Embodiment 2 of the present invention.
[0074]
As shown in FIG. 7, the frame rate conversion device 700 according to the second embodiment includes a storage unit 101, a difference information storage unit 103, a thinned frame determination unit 104, a thinning unit 105, and a decoding unit 704. It is configured. The conversion information multiplexing device 701 according to the second embodiment includes an encoding unit 702, a difference calculation unit 102, and a multiplexing unit 703.
[0075]
As described above, the second embodiment is characterized in that the difference calculation unit 102 is not provided in the frame rate conversion device 700 but is provided in the conversion information multiplexing device 701.
[0076]
In the second embodiment, a first moving image 201 is input to a conversion information multiplexing device 701, and an encoding unit 702 encodes the first moving image 201 to generate encoded data, and a difference operation unit 102 However, as in the case of the above-described first embodiment, the difference evaluation value between the adjacent frames forming the first moving image 201 is determined. Then, the multiplexing unit 703 multiplexes the difference evaluation value into the encoded data by, for example, inserting the difference evaluation value into the user data portion of the data structure of the encoded data, and The moving image data 205 is generated and transmitted to the frame rate conversion device 700.
[0077]
In the frame rate conversion device 700, the decoding unit 704 decodes the encoded data in the moving image data 205 and causes the storage unit 101 to store each frame constituting the first moving image 201. At the same time, the difference evaluation value in the moving image data 205 is stored in the difference information storage unit 103.
[0078]
In this manner, after each frame constituting the first moving image 201 is stored in the storage unit 101 and the difference evaluation value is stored in the difference information storage unit 103, the same processing as in the first embodiment described above is performed. In addition, the decimating frame determining means 104 determines a frame to be decimated, and in accordance with the determination, the decimating means 105 controls the output of each frame constituting the first moving image 201 to perform frame rate conversion. The second moving image 202 is output.
[0079]
As described above, also in the second embodiment of the present invention, as in the first embodiment described above, the moving image 202 output from the frame rate conversion device 700 has a difference between two adjacent frames. However, it is possible to suppress the occurrence of a phenomenon in which the difference between two adjacent frames immediately before or immediately after the scene change is not so large. Further, the occurrence of the phenomenon that the difference between two adjacent frames is almost the same as the difference between immediately adjacent or immediately following two frames is also suppressed. Therefore, a person who views the second moving image 202 is less likely to feel uncomfortable with the second moving image 202.
[0080]
(Embodiment 3)
FIG. 8 shows the configurations of frame rate conversion apparatus 800 and conversion information multiplexing apparatus 801 according to Embodiment 3 of the present invention.
[0081]
As shown in FIG. 8, the frame rate conversion device 800 according to the third embodiment includes a storage unit 101, a thinning unit 105, and a decoding unit 804. Further, the conversion information multiplexing device 801 according to the third embodiment includes an encoding unit 702, a difference calculation unit 102, a difference information accumulation unit 103, a thinned frame determination unit 104, and a multiplexing unit 803. .
[0082]
As described above, in the third embodiment, not only the difference calculation means 102 but also the difference information accumulation means 103 and the thinned frame determination means 104 are not provided in the frame rate conversion apparatus 800, and the conversion information multiplexing apparatus 801 It is characterized by being provided in.
[0083]
In the third embodiment, a first moving image 201 is input to a conversion information multiplexing device 801, and an encoding unit 702 encodes the first moving image 201 to generate encoded data, and generates an encoded data. However, as in the case of the first embodiment described above, a difference evaluation value between adjacent frames constituting the first moving image 201 is determined. The difference evaluation value is stored in the difference information storage unit 103, and thereafter, the thinned frame determination unit 104 determines a frame to be thinned, as in the case of the above-described first embodiment.
[0084]
Then, the multiplexing unit 803 inserts the information of the frame to be decimated into the user data portion of the data structure of the coded data, for example, so that The video data 210 is multiplexed with the encoded data to generate the video data 210, and the video data 210 is transmitted to the frame rate conversion device 800.
[0085]
In the frame rate conversion device 800, the decoding unit 804 decodes the encoded data in the moving image data 210 and causes the storage unit 101 to store each frame constituting the first moving image 201. Then, the thinning unit 105 controls the output of the frames stored in the storage unit 101 in the same manner as in the first embodiment described above, using the information of the frames to be thinned out in the moving image data 210. Thus, the frame rate conversion is performed, and the second moving image 202 is output.
[0086]
As described above, in the third embodiment of the present invention, similarly to the first and second embodiments described above, the moving image 202 output from the frame rate conversion device 800 undergoes a so-called scene change. Although not necessarily, the occurrence of a phenomenon that the difference between two adjacent frames is significantly different from the difference between two adjacent frames immediately before or immediately after the certain frame is suppressed. Further, the occurrence of a phenomenon in which the difference between two adjacent frames does not differ from the difference between two adjacent frames immediately before or immediately after that is also suppressed. Therefore, a person who views the second moving image 202 is less likely to feel uncomfortable with the second moving image 202.
[0087]
(Embodiment 4)
FIG. 9 shows the configurations of frame rate conversion apparatus 900, conversion information multiplexing apparatus 901, and difference calculation apparatus 902 according to Embodiment 4 of the present invention.
[0088]
As shown in FIG. 9, the frame rate conversion apparatus 900 according to the fourth embodiment includes a storage unit 101, a thinning unit 105, and a decoding unit 804. Further, the conversion information multiplexing device 901 according to the fourth embodiment includes an encoding unit 702, a difference information accumulation unit 103, a thinned frame determination unit 104, and a multiplexing unit 803. Further, the difference calculation device 902 according to the fourth embodiment includes the difference calculation means 102.
[0089]
Thus, the fourth embodiment differs from the third embodiment in that the conversion information multiplexing device 901 is not provided with the difference calculation means 102, and the difference information accumulation means 103 and the thinned frame determination means 104 Are provided.
[0090]
In the fourth embodiment, the first moving image 201 is input to the conversion information multiplexing device 901 and also to the difference calculating device 902. The difference calculation device 902 determines the difference evaluation value between each adjacent frame constituting the first moving image 201 and converts the difference evaluation value into the conversion information multiplexing device 901 as in the case of the first embodiment. Output to
[0091]
In the conversion information multiplexing device 901, the encoding unit 702 encodes the first moving image 201 to generate encoded data, and the difference information accumulation unit 103 accumulates the difference evaluation value from the difference calculation device 902. Then, the thinned frame determining unit 104 determines the frame to be thinned based on the difference evaluation value, as in the case of the first embodiment. Thereafter, the multiplexing unit 803 multiplexes the information of the frame to be decimated determined by the decimated frame deciding unit 104 with the encoded data to generate the moving image data 210, as in the case of the third embodiment. The moving image data 210 is transmitted to the frame rate conversion device 900.
[0092]
In the frame rate conversion device 900, the decoding unit 804 decodes the encoded data in the moving image data 210 and causes the storage unit 101 to accumulate each frame constituting the first moving image 201. Then, the thinning unit 105 controls the output of the frames stored in the storage unit 101 in the same manner as in the first embodiment described above, using the information of the frames to be thinned out in the moving image data 210. Thus, the frame rate conversion is performed, and the second moving image 202 is output.
[0093]
As described above, in the fourth embodiment of the present invention, similarly to the above-described first to third embodiments, a so-called scene change is performed on the moving image 202 output from the frame rate conversion device 900. Although not necessarily, the occurrence of a phenomenon that the difference between two adjacent frames is significantly different from the difference between two adjacent frames immediately before or immediately after the certain frame is suppressed. Further, the occurrence of a phenomenon in which the difference between two adjacent frames does not differ from the difference between two adjacent frames immediately before or immediately after that is also suppressed. Therefore, a person who views the second moving image 202 is less likely to feel uncomfortable with the second moving image 202.
[0094]
Note that each component of the frame rate conversion devices 100, 700, 800, 900, the conversion information multiplexing devices 701, 801, 901 and the difference calculation device 902 in each of the above-described embodiments may be configured by hardware. It may be configured by software.
[0095]
Furthermore, the computer functions as all or a part of the frame rate conversion devices 100, 700, 800, 900, the conversion information multiplexing devices 701, 801, 901 and the difference calculation device 902 in each of the above-described embodiments. Is applied to a predetermined computer, and the computer executes the program of the frame rate conversion apparatuses 100, 700, 800, 900, the conversion information multiplexing apparatuses 701, 801, 901 and the difference calculation apparatus 902 in each of the above-described embodiments. It is also possible to realize the functions of all or some of the components. Specific examples of embodiments of the use of the program include recording the program on a recording medium such as a CD-ROM, transferring a recording medium on which the program is recorded, and communication means on the Internet or the like. And communicating the above program. It also includes installing the above program on a computer.
[0096]
【The invention's effect】
As is apparent from the above description, the present invention provides a method of converting the frame rate of a specific moving image to a smaller value, in which a viewer of the converted moving image It is possible to provide a frame rate conversion device that suppresses discomfort.
[0097]
Further, the present invention can provide a conversion information multiplexing apparatus that multiplexes information necessary for enabling the frame rate conversion into the specific moving image.
[Brief description of the drawings]
FIG. 1 is a configuration diagram of a frame rate conversion device according to a first embodiment of the present invention.
FIG. 2 is a flowchart showing an operation procedure of the frame rate conversion device according to the first embodiment of the present invention.
FIG. 3A is a first diagram illustrating a first moving image and a difference evaluation value before frame rate conversion.
(B) First diagram for explaining frame rate conversion
FIG. 4A is a second diagram for explaining a first moving image and a difference evaluation value before frame rate conversion;
(B) Second diagram illustrating frame rate conversion
FIG. 5 is a diagram illustrating a difference evaluation value.
FIG. 6A is a third diagram illustrating a first moving image and a difference evaluation value before frame rate conversion.
(B) Third diagram for explaining frame rate conversion
FIG. 7 is a configuration diagram of a frame rate conversion device and a conversion information multiplexing device according to a second embodiment of the present invention.
FIG. 8 is a configuration diagram of a frame rate conversion device and a conversion information multiplexing device according to a third embodiment of the present invention.
FIG. 9 is a configuration diagram of a frame rate conversion device, a conversion information multiplexing device, and a difference calculation device according to a fourth embodiment of the present invention.
FIG. 10A is a first diagram for explaining conventional frame rate conversion;
(B) Second diagram for explaining conventional frame rate conversion
[Explanation of symbols]
100 Frame rate converter
101 storage means
102 Difference calculation means
103 difference information storage means
104 Decimation frame determination means
105 Thinning means

Claims (15)

In a frame rate conversion device that converts the frame rate of a specific moving image to a smaller value,
A frame rate conversion device comprising a thinning means for thinning out a frame to be thinned determined based on a difference between adjacent frames of the specific moving image from a frame constituting the specific moving image. 2. The frame rate conversion device according to claim 1, further comprising a thinned frame determining unit that determines the frame to be thinned based on the difference. 2. The frame rate conversion device according to claim 1, further comprising a difference calculating means for calculating a difference between adjacent frames of the specific moving image. The frame rate conversion device according to claim 1, wherein the frames to be thinned are determined in units of a predetermined number of frames. In the difference corresponding to the predetermined frame number unit, there is one or more differences that are equal to or less than a predetermined first value, or equal to or more than a predetermined second value that is larger than the first value. When
The frame rate according to claim 4, wherein the frame to be decimated is one of adjacent frames corresponding to the one difference or corresponding to any one of the plurality of differences. Conversion device. When the difference corresponding to the predetermined frame number unit does not include a difference equal to or less than a predetermined first value, or a difference equal to or more than a predetermined second value larger than the first value,
The frame rate conversion device according to claim 4, wherein the frame to be decimated is one of adjacent frames corresponding to the last difference among the corresponding differences. Difference calculating means for calculating a difference between each adjacent frame of a specific moving image,
Multiplexing means for multiplexing information on the difference into the specific moving image. Based on the difference, comprising a thinning frame determining means for determining a frame to be thinned when the frame rate of the specific moving image is converted to a smaller value,
8. The conversion information multiplexing apparatus according to claim 7, wherein the information on the difference means information on a frame to be thinned determined by the thinned frame determining means. Thinning-frame determining means for determining a frame to be thinned when the frame rate of the specific moving image is converted to a smaller value, based on a difference between each adjacent frame of the specific moving image,
Multiplexing means for multiplexing information on the frame to be thinned out into the specific moving image. 10. The conversion information multiplexing apparatus according to claim 8, wherein said thinned-out frame determining means determines the frame to be thinned out in units of a predetermined number of frames. In the difference corresponding to the predetermined frame number unit, there is one or more differences that are equal to or less than a predetermined first value, or equal to or more than a predetermined second value that is larger than the first value. When
The decimated frame determining means determines one of adjacent frames corresponding to the one difference or corresponding to any one of the plurality of differences as the frame to be decimated. The conversion information multiplexing device according to claim 10. When the difference corresponding to the predetermined frame number unit does not include a difference equal to or less than a predetermined first value, or a difference equal to or more than a predetermined second value larger than the first value,
11. The conversion information multiplexing apparatus according to claim 10, wherein the thinned-out frame determining unit determines one of adjacent frames corresponding to the last difference among the corresponding differences as the frame to be thinned out. A program for converting a frame rate of a specific moving image to a smaller value,
Thinning-frame determining means for determining a frame to be thinned, based on a difference between each adjacent frame of the specific moving image,
A program for causing a computer to function as thinning-out means for thinning out the frames to be thinned out from the frames constituting the specific moving image. Difference calculating means for calculating a difference between each adjacent frame of a specific moving image,
A program for causing a computer to function as a multiplexing unit that multiplexes the information regarding the difference into the specific moving image. Thinning-frame determining means for determining a frame to be thinned when the frame rate of the specific moving image is converted to a smaller value, based on a difference between each adjacent frame of the specific moving image,
A program for causing a computer to function as multiplexing means for multiplexing information on the frame to be thinned out into the specific moving image.

Images