WO2012107985A1 - Image signal processing device and image signal processing method - Google Patents

Abstract

An image signal processing device comprises an image acquisition unit (100) for sequentially acquiring a plurality of frames included in respective input images at a corresponding input frame rate, an output frame rate determining unit (200) for determining an output frame rate, a frame synchro unit (300) for sequentially outputting the plurality of frames for respective input images obtained by the image acquisition unit at the output frame rate, and an image outputting unit (500) for sequentially outputting a composite frame obtained by compositing each frame synchronously outputted at the output frame rate from the frame synchro unit (300), wherein the frame synchro unit (300) performs reconciliation processing that eliminates deviation between the input frame rate and the output frame rate, so as not be noticeable to a viewer, with respect to asynchronous images wherein the corresponding input frame rate among the plurality of input images differs from the output frame rate.

Description

Video signal processing apparatus and video signal processing method

The present invention relates to a video signal processing apparatus and a video signal processing method for displaying a plurality of videos having different frame rates in synchronization.

In recent years, in DTV (digital television) and the like, it has become common to view a plurality of video contents simultaneously by helping to increase the screen size and resolution. For example, a viewer can view videos of different channels at the same time. It is also possible to simultaneously view video content from a plurality of video sources such as TV broadcast video, video from a recording device, or video obtained from an external network.

When displaying content acquired from a plurality of video sources in synchronization, it is necessary to display each content in synchronization with one of the frame rates of the plurality of video sources. In such a case, other video (hereinafter referred to as sub-video) is displayed in synchronization using the frame rate of the main video, which is the largest display size on the display and is easily perceived by the viewer. The method is common. In some cases, a PC or the like determines a frame rate with reference to a free-run generated vertical synchronization signal, and displays all videos in synchronization with the determined frame rate.

In order to display the sub-video in synchronization with the main video, it is common to perform frame synchronization processing, which is adjustment processing for excess or deficiency in the number of frames. In the frame synchronization process, if the frame rate of the sub-video is larger than the frame rate of the main video, the sub-video frames are thinned out at a constant period (skip process) and displayed. Also, when the frame rate of the sub-video is smaller than the frame rate of the main video, the same frame of the sub-video is displayed repeatedly (repeat processing) at a constant period. For example, when the frame rate of the main video is 59.94 fps and the frame rate of the sub video is 60.00 fps, the sub video frame is displayed every time about 1000 frames (about 16.7 seconds) are displayed. One frame is skipped. (For example, see Patent Document 1 and Patent Document 2).

JP 2005-341132 A JP 2007-271848 A

However, in such a frame synchronization process, the sub-picture frames are thinned out at regular intervals, or the same frame is repeatedly displayed, which may give the viewer a sense of discomfort. For example, if frame synchronization processing is performed during a period in which the video pans or tilts, the judder of the video is likely to be perceived by the viewer. That is, the problem is that the viewer feels uncomfortable by the frame sync processing and the viewing quality is impaired.

Therefore, an object of the present invention is to provide a video signal processing apparatus capable of displaying a smooth and high-definition video with little discomfort for a viewer when a plurality of videos with different frame rates are displayed in synchronization.

In order to solve the above problems, a video signal processing apparatus according to an aspect of the present invention is a video signal processing apparatus that outputs a plurality of input videos having different input frame rates in synchronization with each other. A video acquisition unit that sequentially acquires the frames included in the corresponding input frame rate, an output frame rate determination unit that determines an output frame rate, and a frame of each of the plurality of input videos acquired by the video acquisition unit A frame synchronizer that sequentially outputs in synchronism with the output frame rate, and a video output that sequentially outputs a synthesized frame obtained by synthesizing each frame output from the frame synchronizer in synchronism with the output frame rate. And the frame synchronization unit includes a corresponding input frame of the plurality of input images. Mureto respect asynchronous video different from the output frame rate, the adjustment process to eliminate the deviation between the output frame rate to the input frame rate, executes during which it can be determined as a hard period noticeable to the viewer.

Further, the frame synchronization unit detects the adjustment processing period in which the uncomfortable feeling generated in the composite frame by executing the adjustment process on the asynchronous video is a period in which the viewer who views the composite frame is not easily noticed. It is preferable to include a detection unit and a control unit that executes the adjustment process on the asynchronous video during the adjustment process period detected by the synchronization timing detection unit.

As a result, since the frame adjustment process is performed in a period in which the viewer does not feel uncomfortable, a smooth and high-quality video display is realized.

The synchronization timing detection unit detects a motion amount of the asynchronous video, a period in which the motion amount is smaller than a first threshold value, and a second threshold value in which the motion amount is larger than the first threshold value. Is detected as the adjustment process period, or the synchronization timing detection unit detects a change in the scene of the asynchronous video, and a period corresponding to the change in the scene of the video is set as the adjustment process period. It may be detected.

Thus, the frame adjustment process is performed during a period in which there is little video movement, a period in which there is a lot of video movement, and a period corresponding to the switching of the video scene.

The frame synchronization unit, as the adjustment process, outputs a part of a frame when the input frame rate of the asynchronous video is larger than the output frame rate, and the input frame rate of the asynchronous video is If the output frame rate is smaller than the same, the same frame is repeatedly output.

In addition, the frame synchronization unit uses the M (M is an integer of 2 or more) original frames included in the asynchronous video as a frame between the M original frames as the adjustment process. Corresponding N (M ≠ N) interpolation frames may be generated, and the generated N interpolation frames may be output as the asynchronous video frame in the predetermined period.

Thus, even when frame adjustment processing is performed at a fixed period, an interpolated frame is generated and displayed, so that a video display with a small discomfort for the viewer can be realized.

In addition, the frame synchronization unit detects a motion vector between original frames located before and after the generated interpolation frame, and obtains the detected motion vector in proportion to the original frame with a temporal distance of the interpolation frame. The interpolation frame may be generated assuming that the interpolation motion vector to be obtained is a motion vector for the original frame of the interpolation frame.

Thus, a more accurate interpolation frame is generated by using the motion vector, and a smooth video can be displayed.

The output frame rate determining unit may determine any one of the plurality of input videos as the output frame rate, and the output frame rate determining unit may determine the plurality of input videos. A frame rate different from any of the input frame rates may be determined as the output frame rate.

The present invention is also a video signal processing method for outputting a plurality of input videos having different input frame rates in synchronization with each other, and sequentially including the frames included in each of the plurality of input videos at the corresponding input frame rate. A video acquisition step for acquiring, an output frame rate determination step for determining an output frame rate, and a frame synchronization step for sequentially outputting frames of each of the plurality of input videos acquired by the video acquisition unit at the output frame rate; And a video output step for sequentially outputting a synthesized frame obtained by synthesizing each frame output in synchronization with the output frame rate from the frame synchronization step. In the frame synchronization step, the plurality of input videos Among these, the corresponding input frame rate is the output. An adjustment process for eliminating a difference between the input frame rate and the output frame rate is performed on an asynchronous video different from a frame rate so as not to be noticeable to a viewer, and the frame synchronization step includes a frame of the asynchronous video. The synchronization timing detection step for detecting an adjustment processing period in which the uncomfortable feeling generated in the composite frame due to the adjustment process being applied to the viewer who views the composite frame is difficult to detect is detected in the synchronization timing detection step. And a control step of performing the adjustment process on each frame of the asynchronous video and outputting the frame during the adjustment process period.

Also, the video signal processing method of the present invention is a video signal processing method for outputting a plurality of input videos having different input frame rates in synchronization with each other, and corresponding to the frames included in each of the plurality of input videos. An image acquisition step for sequentially acquiring at an input frame rate, an output frame rate determination step for determining an output frame rate, and a frame of each of the plurality of input images acquired by the image acquisition unit is sequentially output at the output frame rate A frame synchronization step, and a video output step for sequentially outputting a synthesized frame obtained by synthesizing each frame output in synchronization with the output frame rate from the frame synchronization step. In the frame synchronization step, Among a plurality of input images, the corresponding input frame Using M (M is an integer of 2 or more) original frames included in a predetermined period of asynchronous video whose frame rate is different from the output frame rate, N (M ≠) corresponding to a frame between the M original frames N) interpolation frames may be generated, and the generated N interpolation frames may be output as the asynchronous video frames in the predetermined period.

The video signal processing apparatus according to one aspect of the present invention can display a smooth and high-quality video in which a viewer does not feel uncomfortable when a plurality of videos with different frame rates are displayed in synchronization.

FIG. 1 is a block diagram showing a configuration of a video signal processing apparatus according to an embodiment of the present invention. FIG. 2 is a flowchart of the operation of the video signal processing apparatus according to Embodiment 1 of the present invention. FIG. 3 is a block diagram showing the configuration of the frame synchronization unit in the first embodiment of the present invention. FIG. 4 is a diagram for explaining conventional frame skip processing. FIG. 5 is a diagram for explaining a conventional frame repeat process. FIG. 6 is a diagram for explaining frame skip processing according to Embodiment 1 of the present invention. FIG. 7 is a diagram illustrating frame repeat processing according to Embodiment 1 of the present invention. FIG. 8 is a flowchart of the adjustment process of the frame synchronization unit in the first embodiment of the present invention. FIG. 9 is a block diagram showing the configuration of the frame synchronization unit in the second embodiment of the present invention. FIG. 10 is a diagram for explaining the adjustment process of the frame synchronization unit in the second embodiment of the present invention. FIG. 11 is a diagram for explaining the adjustment process of the frame synchronization unit in the second embodiment of the present invention. FIG. 12 is a flowchart of the adjustment process of the frame synchronization unit according to the second embodiment of the present invention. FIG. 13 is a diagram showing an application example of the video signal processing apparatus according to Embodiments 1 and 2 of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The embodiment of the present invention described below shows a preferred specific example of the present invention. The numerical values, shapes, components, arrangement of components, connection forms, and the like shown in this embodiment are merely examples, and are not intended to limit the present invention. The present invention is limited only by the claims. Therefore, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims are not necessarily required to achieve the object of the present invention, but are described as constituting more preferable embodiments. The

(Embodiment 1)
Hereinafter, Embodiment 1 of the present invention will be described with reference to FIGS. 1 and 2.

FIG. 1 is a block diagram showing the configuration of the video signal processing apparatus according to Embodiment 1 of the present invention.

FIG. 2 is a flowchart of the operation of the video signal processing apparatus according to Embodiment 1 of the present invention.

As shown in FIG. 1, the video signal processing device 10 includes a video acquisition unit 100, an output frame rate determination unit 200, a frame synchronization unit 300, and a video output unit 500. In FIG. 1, a frame memory 400 and a video display unit 600 are also shown.

The video acquisition unit 100 sequentially acquires frames included in each of the plurality of input videos at a corresponding input frame rate (S10). In addition, the video acquisition unit 100 outputs each frame of the input video signal acquired according to the input frame rate to the frame synchronization unit 300.

The output frame rate determining unit 200 determines the output frame rate. Specifically, one of the input frame rates corresponding to the plurality of video signals input to the video acquisition unit 100 is determined as the output frame rate (S20).

Note that the output frame rate determination unit 200 may determine the output frame rate from the free-run vertical synchronization signal. The free-run vertical synchronization signal is a signal that realizes a predetermined frequency (for example, 60 Hz) using a clock generated by a clock generator having a variable clock frequency.

The frame synchronization unit 300 sequentially outputs the frames of each of the plurality of input videos acquired by the video acquisition unit 100 at the output frame rate determined by the output frame rate determination unit 200 (S30). Specifically, the frames of the plurality of input videos acquired by the video acquisition unit 100 are temporarily stored in the frame memory 400, read at the output frame rate, and output to the video output unit 500. The frame synchronization unit 300 adjusts the number of frames and outputs each frame of an asynchronous video having a corresponding input frame rate different from the output frame rate among a plurality of input videos. A more specific configuration of the frame synchronization unit 300 and details of the adjustment process will be described later.

The frame memory 400 is a storage unit that temporarily stores frames of a plurality of input videos. The specific configuration of the storage unit is not particularly limited. For example, DRAM (Dynamic Random Access Memory), SDRAM (Synchronous Dynamic Random Access Memory), flash memory, ferroelectric memory, HDD (HardDisc Data), etc. Any means can be used as long as it can memorize.

The video output unit 500 sequentially outputs combined frames obtained by combining the frames output from the frame sync unit 300 in synchronization with the output frame rate to the video display unit 600 (S40).

The video display unit 600 displays a video composed of the composite frame output from the video output unit 500 (S50). Specifically, the video display unit 600 is an LCD (Liquid Crystal Display), for example. Not only this but PDP (Plasma Display Panel) and an organic electroluminescent display (OLED: Organic Light Emitting Display) may be sufficient.

Next, the frame synchronization unit 300 will be described in more detail.

In the following description, one asynchronous video input to the frame synchronization unit 300 will be described. However, the following adjustment processing is actually performed for a plurality of asynchronous videos.

FIG. 3 is a configuration diagram of the frame synchronization unit 300 according to the first embodiment.

The frame synchronization unit 300 includes a synchronization timing detection unit 310 and an address control unit 320 (control unit).

The synchro timing detection unit 310 detects an adjustment processing period in which the uncomfortable feeling generated in the composite frame by performing the adjustment process on the frame of the asynchronous video is less noticeable to the viewer who views the composite frame.

For example, the synchronization timing detection unit 310 adjusts a period in which the video motion is small, the video motion is intense, or the video scene changes and a blackout or whiteout occurs when the scene changes. Detect as period.

By performing the adjustment process in such an adjustment process period, it is possible to realize a video display with a little discomfort for the viewer.

The details of the adjustment processing period will be described below.

In the period when the motion of the image is small, the change in luminance between temporally continuous frames is small, and in the period where the motion of the image is intense, the change in luminance between frames is large. For this reason, a period in which the motion of the video is small or a period in which the motion of the video is intense is detected by, for example, SAD (Sum of Absolute Difference).

SAD is a parameter obtained by calculating an absolute value of a luminance difference between corresponding pixels of a frame that is temporally continuous with the frame for each pixel constituting the frame, and summing the absolute values of the calculated luminance differences of the pixels. is there. Hereinafter, a method for detecting the amount of motion of an image using SAD will be described.

First, the synchronization timing detection unit 310 divides each frame into several small areas for temporally continuous frames constituting the asynchronous video.

Next, the synchronization timing detection unit 310 obtains the SAD of each divided small area of one frame with the corresponding small area of the previous frame or the subsequent frame. That is, the synchronization timing detection unit 310 calculates the sum of absolute values of the luminance differences of the pixels constituting the small area and the corresponding pixels in the previous frame or the subsequent frame. As a result, the same number of SADs as the number of small regions is obtained.

Finally, the synchronization timing detection unit 310 obtains the sum of SADs corresponding to each small area. The synchronization timing detection unit 310 sets a threshold for the sum of the SADs, and detects a period in which the video motion is small and a period in which the video motion is intense.

Specifically, the synchronization timing detection unit 310 detects a period corresponding to the frame as a period in which the motion of the video is small for a frame in which the total SAD is smaller than the first threshold. In addition, the synchronization timing detection unit 310 detects a period corresponding to the frame as a period in which the motion of the video is intense for a frame in which the sum of SADs is larger than the second threshold, which is larger than the first threshold.

Note that the synchronization timing detection unit 310 may detect a period in which the video motion is small and a period in which the video motion is intense depending on the number of small regions where the SAD exceeds a predetermined value.

In this case, the synchronization timing detection unit 310 detects a period corresponding to a frame in which the number of small areas exceeding a predetermined value is smaller than the first threshold as a period in which video motion is small. The synchronization timing detection unit 310 detects a period corresponding to a frame in which the number of small areas exceeding a predetermined value is larger than the second threshold and larger than the second threshold as a period in which video motion is intense.

Also, the period during which the video scene changes is detected by, for example, APL (Average Picture Level) which is the average luminance of the entire frame. Specifically, the synchronization timing detection unit 310 calculates the APL for the frame of the asynchronous video, and for the frame in which the difference from the average of the APLs of a plurality of consecutive frames before the frame is large, the frame The period corresponding to is detected as the period during which the video scene changes.

Next, the adjustment process of the address control unit 320 will be described in detail.

First, a conventional adjustment process of the address control unit 320 will be described.

4 and 5 are diagrams for explaining conventional adjustment processing.

Conventionally, the address control unit 320 sets a write address representing a frame stored in the frame memory 400 when a frame included in the input video (asynchronous video) is stored in the frame memory 400 according to the input frame rate. The write address is counted up according to the frame output timing of the input frame rate.

The write address is reset to “0” when the number of frames that can be stored in the frame memory is exceeded. In the example shown in FIGS. 4 and 5, the number of frames that can be stored in the frame memory 400 is eight frames. Therefore, any of the numbers “0” to “7” is set as the address.

On the other hand, the address control unit 320 reads the frame from the frame memory 400 in synchronization with the output frame rate and outputs it. At this time, the address control unit 320 reads a frame from the frame memory 400 by designating the write address set at the time of storing the frame as a read address. The read address is counted up according to the frame output timing of the output frame rate.

Also, the difference between the write address, the read address and the number of frames at the timing when the read address is counted up is called a phase difference. For example, when the phase difference is “1”, it means that the read address is an address representing a frame one frame before the write address at the same timing.

In the conventional example shown in FIGS. 4 and 5, the address control unit 320 performs frame skip processing based on whether the phase difference at the frame output timing of the output frame rate is a predetermined phase difference. In the example shown in FIGS. 4 and 5, the predetermined phase difference is “1”.

When the input frame rate of asynchronous video is larger than the output frame rate, the address control unit 320 thins out a part of the frame and outputs it. Specifically, the address control unit 320 thins out and reads out the frames of the asynchronous video stored in the frame memory 400 and outputs them to the video output unit 500 in synchronization with the output frame rate. This process is called a frame skip process.

FIG. 4 is a diagram for explaining a conventional frame skip process.

Timing (1) in FIG. 4 is the first timing at which the address control unit 320 starts outputting a frame. At timing (1) in FIG. 4, since the write address is “1”, the address control unit 320 sets the address “0” at which the phase difference is “1” which is a predetermined phase difference as the read address, A frame corresponding to the address “0” is read from the frame memory 400 and output.

At timing (2) in FIG. 4, the read address is set to “2” with respect to the write address “3”. At the subsequent timing (3), the read address counted up from the state at the timing (1) is “2” with respect to the write address “4”. That is, at the timing (3), the phase difference is “2” with respect to the predetermined phase difference “1”, and the phase difference is larger than the predetermined phase difference.

Therefore, the address control unit 320 performs frame skip processing at timing (3). Specifically, the address control unit 320 skips the address “2”, sets “3” as a read address, reads the frame corresponding to the address “3” from the frame memory 400, and outputs the frame. That is, the frame corresponding to the address “2” is skipped.

As described above, as a result of the frame skip processing, the phase difference at the timing (4) in FIG. 5 is “1”, which is equal to the predetermined phase difference “1”.

Thereafter, similarly, the address control unit 320 outputs each frame of the asynchronous video according to the output frame rate by performing a frame skip process at a timing when the phase difference is larger than the predetermined phase difference.

Also, when the input frame rate of asynchronous video is smaller than the output frame rate, the address control unit 320 repeatedly outputs the same frame. Specifically, the address control unit 320 repeatedly reads out the frame of the asynchronous video stored in the frame memory 400 and outputs it to the video output unit 500 in synchronization with the output frame rate. This process is called a frame repeat process.

FIG. 5 is a diagram for explaining a conventional frame repeat process.

Timing (1) in FIG. 5 is the first timing at which the address control unit 320 starts outputting a frame. At timing (1) in FIG. 5, since the write address is “1”, the address control unit 320 sets the address “0” at which the phase difference is “1” which is a predetermined phase difference as the read address, A frame corresponding to the address “0” is read from the frame memory 400 and output.

At timing (2) in FIG. 5, the read address is set to “2” with respect to the write address “3”. At the subsequent timing (3), the read address counted up from the state at the timing (2) is “3” with respect to the write address “3”. That is, at the timing (3), the phase difference is “0” with respect to the predetermined phase difference “1”, and the phase difference is smaller than the predetermined phase difference.

Therefore, the address control unit 320 performs frame repeat processing at timing (3). Specifically, the address control unit 320 sets the address “2” as the read address again at the timing (3), reads the frame corresponding to the address “2” from the frame memory 400, and outputs it. That is, the frame corresponding to the address “2” is repeatedly output.

As described above, as a result of the frame repeat process, the phase difference at the timing (4) in FIG. 6 is “1”, which is equal to the predetermined phase difference “1”.

Thereafter, similarly, the address control unit 320 outputs each frame of the asynchronous video according to the output frame rate by performing a frame repeat process at a timing when the phase difference is smaller than the predetermined phase difference.

As described above, in the conventional adjustment processing, the address control unit 320 performs adjustment processing (frame skip processing and frame repeat) processing at regular intervals based on the phase difference, and synchronizes the input frame with the output frame rate. Output. For this reason, depending on the video displayed at the timing when the adjustment process is performed, the viewer may feel uncomfortable with the video.

Next, adjustment processing of the address control unit 320 according to Embodiment 1 of the present invention will be described.

The address control unit 320 according to the first embodiment of the present invention performs an adjustment process (frame skip process, frame repeat process) on each frame of the asynchronous video during the adjustment process period detected by the synchronization timing detection unit 310 and outputs it. To do.

First, the frame skip processing according to Embodiment 1 of the present invention will be described.

When the asynchronous video input frame rate is higher than the output frame rate, the address control unit 320 performs the frame skip processing as described above.

FIG. 6 is a diagram for explaining frame skip processing of the address control unit 320 according to Embodiment 1 of the present invention.

As shown in FIG. 6, the address control unit 320 according to Embodiment 1 of the present invention adjusts the frame output timing in addition to the conventional control for detecting whether or not the phase difference is a predetermined phase difference. Frame skip processing is performed based on whether or not it is included in the processing period.

In the example shown in FIG. 6, the predetermined phase difference is “4”, and the number of frames that can be stored in the frame memory 400 is eight frames.

The timing (1) of the period (a) in FIG. 6 is the first timing at which the address control unit 320 starts outputting a frame. In the timing (1) of the period (a) in FIG. 6, since the write address is “4”, the address control unit 320 reads the address “0” where the phase difference is “4” which is a predetermined phase difference. The frame is set as an address, and the frame corresponding to the address “0” is read from the frame memory 400 and output.

As shown in the period (a) in FIG. 6, the write address is counted up according to the input frame rate, and the read address is counted up according to the output frame rate. In the period (a) in FIG. 6, the phase difference is “4” at any of the timings (1) to (3), and the address control unit 320 does not need to perform the frame skip process.

In the example shown in FIG. 6, the input frame rate is larger than the output frame rate. Accordingly, when the time elapses without the address controller 320 performing frame skip processing, the phase difference between the write address and the read address increases.

The period (b) in FIG. 6 represents a state in which several frames have elapsed from the state of the period (a) in FIG. 6 and the phase difference between the write address and the read address has expanded. The period (b) in FIG. 6 includes the adjustment process period detected by the synchronization timing detection unit 310, and the address control unit 320 performs the frame skip process in the adjustment process period.

In the timing (1) of the period (b) in FIG. 6, the read address is set to “6” with respect to the write address “4”. That is, the phase difference is “6” with respect to the predetermined phase difference “4”, and the phase difference is larger than the predetermined phase difference.

In the timing (2) of the period (b) in FIG. 6, the write address is “5”, and the read address counted up from the state of the timing (1) in the period (b) in FIG. 6 is “7”. . That is, the phase difference at timing (2) is “6”, which is larger than the predetermined phase difference “4”, and the timing (2) in the period (b) in FIG. It is included in the period detected as a period. For this reason, the address control unit 320 performs frame skip processing at the timing (2) of the period (b) in FIG. Specifically, the address control unit 320 skips the address “7”, sets “0” as a read address, reads the frame corresponding to the address “0” from the frame memory 400, and outputs the frame. That is, the frame corresponding to the address “7” is skipped.

Similarly, at the subsequent timing (3), the write address is “6”, and the read address counted up from the state of the timing (2) in the period (b) in FIG. 6 is “1”. At this time, in the timing (3) of the period (b) in FIG. 6, the phase difference is “5”, which is larger than the predetermined phase difference “4”, and the timing (3) in the period (b) in FIG. Included in the adjustment process period. Therefore, at the timing (3) of the period (b) in FIG. 6, the address control unit 320 performs a frame skip process as the adjustment process. Specifically, the address control unit 320 skips the address “1”, sets the address “2” as a read address, reads out the frame corresponding to the address “2” from the frame memory 400, and outputs it. That is, the frame corresponding to the address “1” is skipped.

As described above, as a result of the frame skip processing performed in the adjustment processing period by the address control unit 320, the phase difference at the timing (4) in the period (b) in FIG. 6 is “4”, and the predetermined phase difference “4”. Is equal to

Thereafter, when the phase difference is larger than a predetermined phase difference in the adjustment processing period similarly detected by the synchronization timing detection unit 310, the address control unit 320 performs frame skip processing to output each frame of the asynchronous video as an output frame. Output according to the rate.

In the period (b) in FIG. 6, the address control unit 320 skips and outputs one frame at a time as frame skip processing, but the address control unit 320 skips a plurality of frames at a time. May be output. That is, at the timing (2) of the period (b) in FIG. 6, the address control unit 320 skips the addresses “7” and “0”, sets “1” as the read address, and corresponds to the address “2”. A frame to be output may be output. In this case, two frames of addresses “7” and “0” are skipped at a time.

Note that the number of frames that can be stored in the frame memory 400 is finite, and in the example of FIG. 6, it is 8 frames as described above. Therefore, when the adjustment process period is not detected for a long time and the frame skip process cannot be performed, the address control unit 320 may inevitably perform the frame skip process during a period other than the adjustment process period.

The period (c) in FIG. 6 is a diagram showing such a case.

In the timing (1) of the period (c) in FIG. 6, since the adjustment process period is not detected, the frame skip process is not performed for a long time, and as a result, the phase difference between the write address and the read address is large. Since the write address is “5”, the read address is “6”, the phase difference is “7”.

In the timing (2) of the period (c) in FIG. 6, the write address is “7”, and the read address is counted up from the state of the timing (1) and should be “7” originally. However, the frame corresponding to the read address “7” does not exist in the frame memory 400 because a new frame is stored at the timing when the write address is counted up to “7”.

Therefore, the address control unit 320 performs the frame skip process at the timing (2) of the period (c) in FIG. 6 that is not included in the adjustment process period. Specifically, the address control unit 320 skips the address “7”, sets the address “0” as the read address, reads out the frame corresponding to the address “0” from the frame memory 400, and outputs it.

Next, the frame repeat processing according to Embodiment 1 of the present invention will be described.

When the input frame rate of asynchronous video is smaller than the output frame rate, the address control unit 320 performs frame repeat processing.

FIG. 7 is a diagram for explaining frame repeat processing of the address control unit 320 according to Embodiment 1 of the present invention.

As shown in FIG. 7, the address control unit 320 according to the first embodiment of the present invention has a frame output timing in the adjustment processing period in addition to the conventional control for detecting whether or not the phase difference is a predetermined phase difference. Frame repeat processing is performed based on whether or not it is included. The predetermined phase difference is “4”, and the number of frames that can be stored in the frame memory 400 is eight frames.

The timing (1) of the period (a) in FIG. 7 is the first timing at which the address control unit 320 starts to output a frame. In the timing (1) of the period (a) in FIG. 7, since the write address is “4”, the address control unit 320 reads the address “0” where the phase difference is “4” which is a predetermined phase difference. The frame is set as an address, and the frame corresponding to the address “0” is read from the frame memory 400 and output.

As shown in the period (a) in FIG. 7, the write address is counted up according to the input frame rate, and the read address is counted up according to the output frame rate. In the period (a) in FIG. 7, the phase difference is “4” at any timing of (1) to (3), and the address control unit 320 does not need to perform frame repeat processing.

In the example shown in FIG. 7, the input frame rate is smaller than the output frame rate. Accordingly, when the time elapses without the address control unit 320 performing the frame repeat process, the phase difference between the write address and the read address becomes small.

The period (b) in FIG. 7 represents a state in which the frame repeat process is not performed from the state in the period (a) in FIG. 7 and a time of several frames elapses and the phase difference between the write address and the read address is reduced.

In the timing (1) of the period (b) in FIG. 7, the read address is set to “1” with respect to the write address “4”. That is, the phase difference is “3” with respect to the predetermined phase difference “4”, and the phase difference is smaller than the predetermined phase difference.

In the timing (2) of the period (b) in FIG. 7, the write address is “5”, and the read address counted up from the timing (1) in the period (b) in FIG. 7 is “2”. That is, the phase difference is “3” at the timing (2), which is smaller than the predetermined phase difference “4”, and the synchronization timing detection unit 310 performs the adjustment process at the timing (2) in FIG. It is included in the period detected as a period. Therefore, the address control unit 320 performs a frame repeat process at the timing (2) of the period (b) in FIG. Specifically, the address control unit 320 sets the address “1” as the read address again, reads out the frame corresponding to the address “1” from the frame memory 400, and outputs the frame. That is, the frame corresponding to the address “1” is output again.

As described above, as a result of performing the repeat process in the adjustment process period, the phase difference at the timing (3) of the period (b) is “4”, which is equal to the predetermined phase difference “4”.

Thereafter, when the phase difference is larger than a predetermined phase difference in the adjustment processing period similarly detected by the synchronization timing detection unit 310, the address control unit 320 performs frame repeat processing to output each frame of the asynchronous video as an output frame. Output according to the rate.

Note that, in the timing (2) of the period (b) in FIG. 7, the address control unit 320 repeatedly outputs a frame once as a frame repeat process, but the address control unit 320 outputs a plurality of the same frames. You may output continuously.

Note that the number of frames that can be stored in the frame memory 400 is finite, and in the example of FIG. 7, it is 8 frames as described above. Therefore, when the adjustment process period is not detected for a long time and the frame repeat process cannot be performed, the address control unit 320 may inevitably perform the frame repeat process during a period other than the adjustment process period.

(C) of FIG. 7 is a figure which shows such a case.

In the timing (1) of the period (c) in FIG. 7, the frame repeat process is not performed, and the phase difference between the write address and the read address is small. Since the write address is “4” while the read address is “4”, the phase difference is “0”.

In the timing (2) of the period (c) in FIG. 7, the write address is “4”, and the read address is counted up from the state of the timing (1) and should be “5” originally. However, at the timing (2), the write address is not counted up to “5”. That is, the frame corresponding to the address “5” at the timing (2) is not yet stored, and the frame corresponding to the read address “5” does not exist in the frame memory 400.

Therefore, the address control unit 320 performs the frame repeat process at the timing (2) of the period (c) in FIG. 7 that is not included in the adjustment process period. Specifically, the address control unit 320 sets the address “4” as the read address again, reads out the frame corresponding to the address “4” from the frame memory 400, and outputs it.

Finally, the adjustment processing sequence of the frame synchronization unit 300 will be described.

FIG. 8 is a flowchart of the adjustment process of the frame synchronization unit 300 according to the first embodiment.

The adjustment process of the frame synchronization unit 300 that eliminates the difference between the input frame rate and the output frame rate in the first embodiment is as follows.

First, the frame synchronization unit 300 acquires the input frame of the input video acquired by the video acquisition unit 100 (S110).

Subsequently, the frame synchronization unit 300 acquires the output frame rate determined by the output frame rate determination unit 200 (S120).

Next, the synchronization timing detection unit 310 detects an adjustment process period, which is a period during which the adjustment process performed by the address control unit 320 is difficult for the viewer to perceive the asynchronous video (S130).

The address control unit 320 outputs the input frame according to the output frame rate based on the phase difference. When the phase difference is shifted during frame output (Yes in S140), and the timing when the shift is included in the adjustment processing period (Yes in S150), the address control unit 320 performs the adjustment process to display the frame. Output (S160).

When the timing at which the deviation occurs is not included in the adjustment processing period (No in S140), the address control unit 320 outputs the frame as it is without performing the adjustment processing (S160).

As described above, in the video signal processing apparatus 10 according to Embodiment 1 of the present invention, the frame adjustment processing for eliminating the deviation between the input frame rate and the output frame rate is performed during the period when the video motion is small. It is performed in a period corresponding to the switching of video scenes. As a result, a smooth and high-definition video display is realized in which the viewer does not feel a sense of discomfort in the video.

(Embodiment 2)
Embodiment 2 of the present invention will be described below with reference to the drawings.

The configuration and operation of the video signal processing apparatus 10 according to the second embodiment of the present invention are different from the first embodiment only in the configuration and operation of the frame synchronization unit 300. Therefore, the configuration and adjustment processing of the frame synchronization unit 300 according to Embodiment 2 will be described in detail below.

In the following description, one asynchronous video input to the frame synchronization unit 300 will be described, but actually, adjustment processing is performed for a plurality of asynchronous videos.

FIG. 9 is a block diagram showing the configuration of the frame synchronization unit 300 according to Embodiment 2 of the present invention.

The frame synchronization unit 300 includes an address control unit 320 and an interpolation frame generation unit 360.

The address control unit 320 performs a frame skip process or a frame repeat process on the input frame, which is each frame of the asynchronous video, for a predetermined period, and outputs it to the interpolation frame generation unit 360 as a read frame. The address control unit 320 also outputs to the interpolation frame generation unit 360 the preceding and following frames that are temporally preceding and following the read frame.

The interpolation frame generation unit 360 generates an interpolation frame using the read frame and the previous and next frames output from the address control unit 320. In addition, the interpolation frame generation unit 360 replaces a frame included in a predetermined period of the read frame with the interpolation frame and outputs it as an output frame. Details of the interpolation frame generation method of the interpolation frame generation unit 360 will be described later.

Next, the adjustment processing of the address control unit 320 and the interpolation frame generation unit 360 will be described in detail with specific examples.

FIG. 10 is a diagram for explaining adjustment processing of the address control unit 320 and the interpolation frame generation unit 360 when the input frame rate is smaller than the output frame rate.

The basic operation of storing a frame in the frame memory 400 of the address control unit 320 is as described in the first embodiment. As shown in FIG. 10, the write address is counted up according to the input frame rate, and the read address is counted up according to the output frame rate.

Also, the address control unit 320 detects the timing for performing the frame repeat process based on whether or not the phase difference at the frame output timing of the output frame rate is a predetermined phase difference. In FIG. 10, the predetermined phase difference is “1”, and the number of frames that can be stored in the frame memory 400 is eight frames.

In the following description, the frame corresponding to the input frame address “0” is expressed as frame [0]. For example, the frame [1] is a frame corresponding to the address “1” of the input frame, and the frame [2] is a frame corresponding to the address “2” of the input frame.

∙ Since the address of the input frame is counted up by 1 with the passage of time, the number in [] represents the time distance.

In the example shown in FIG. 10, when the address control unit 320 controls the read address based on the phase difference as in the first embodiment, frames [0], [1], [2], [2] ], [3], [4], [5], [6]... Are continuously output. In this period, since there is no change in the time distance between frames at the timing when frames [2] are continuously output, there is a high possibility that the viewer will feel uncomfortable.

The interpolation frame generation unit 360 uses the interpolation frame to mitigate changes in the time distance between frames. Specifically, for example, the interpolation frame generation unit 360 outputs timings (4), (5), (() that are output timings of output frames corresponding to the frames [2], [2], [3] of the read frame. In 6), interpolation frames [1.8], [2.5], and [3.2] are output, respectively.

As a result, the output frame uses the interpolation frame, and the frames [0], [1], [1.8], [2.5], [3.2], [4], [5]. It becomes. Thereby, since the change of the time distance between frames is relieved, the video display which a viewer does not feel uncomfortable is realized.

In order to perform the control as described above, first, the address control unit 320 detects a replacement frame to be replaced with an interpolation frame in advance from the read frame from the input frame rate and the output frame rate of the asynchronous video. In the example of FIG. 10, the frame [2] output by the frame repeat process and the immediately preceding and immediately following frames are set as the replacement frame. That is, the replacement frame is the frames [2], [2], and [3] of the read frame.

Next, the address control unit 320 determines an interpolation frame generated by the interpolation frame generation unit 360. As described above, the interpolated frames are set to frames [1.8], [2.5], and [3.2] in order to mitigate changes in the temporal distance between frames.

Hereinafter, the input / output of frames of the address control unit 320 and the interpolation frame generation unit 360 will be described in detail with reference to FIG.

Timing (1) in FIG. 10 is the first timing at which the address control unit 320 starts to output a read frame. At timing (1), the write address is “1” and the predetermined phase difference is “1”. Therefore, the address control unit 320 sets the read address to “0”, reads the frame [0] corresponding to the address “0” from the frame memory 400 as a read frame, and outputs it to the interpolation frame generation unit 360.

At timing (2) in FIG. 10, the read address is counted up to “1”, and the phase difference is equal to the predetermined phase difference. For this reason, the frame repeat process is not performed, and the output read frame is frame [1].

Timing (2) is a timing at which the interpolation frame generation unit 360 starts to output an output frame. At timing (2), the interpolation frame generation unit 360 outputs the frame [0] output from the address control unit 320 at timing (1) to the video output unit 500 as an output frame.

Subsequently, at timing (3), the read address is counted up to “2”, and the phase difference is “1” equal to a predetermined phase difference. Therefore, the read frame output at timing (3) is frame [2].

Here, the frame [2] output from the address control unit 320 at timing (3) is a frame to be replaced. The readout frame at timing (3) is replaced with the interpolation frame [1.8] by the interpolation frame generation unit 360. The interpolation frame [1.8] is generated from the frames [1] and [2] by linear interpolation described later.

Therefore, in timing (3), the address control unit 320 sets the address “1” corresponding to the immediately preceding frame as the front and rear addresses in addition to the address “2”. The front-rear address is an address for the address control unit 320 to read a frame from the frame memory 400 so that the interpolation frame generation unit 360 generates an interpolation frame. As a result, at timing (3), in addition to the read frame, frame [1] is output to the interpolated frame generation unit 360 as the previous and subsequent frames.

Also, at timing (3), the interpolation frame generation unit 360 outputs the frame [1] output from the address control unit 320 at timing (2).

Next, the phase difference expands to “2” at timing (4). Therefore, at the timing (4), the frame synchronization unit 300 performs the frame repeat process, sets the address “2” as the read address again, and outputs the frame [2] as the read frame. The read frame output at timing (4) is a frame to be replaced and is replaced with frame [2.5]. Therefore, the address control unit 320 sets the next address “3” after the address “2” as the previous and next addresses, and outputs the frame [3] to the interpolated frame generation unit 360 as the previous and next frames.

At timing (4), the interpolation frame generation unit 360 linearly interpolates the frames [1] and [2] output from the address control unit 320 at timing (3). 8 "is generated and output as an output frame. The linear interpolation will be described later.

Subsequently, at timing (5), the read address is counted up to “3”, and the phase difference is “1” equal to a predetermined phase difference. Therefore, frame [3] is output as a read frame. The read frame is a frame to be replaced, and is replaced with the frame [3.2]. Therefore, frame [4.5] is output as the previous and subsequent frames.

At timing (5), the interpolation frame generation unit 360 generates the interpolation frame [2.5] from the frames [2] and [3] output from the address control unit 320 at timing (4), and outputs the output frame. Output as.

At timing (6), the read address is counted up to “3”, and the phase difference is “1” equal to a predetermined phase difference. Therefore, frame [3] is output as a read frame. Further, since the read frame is not a frame to be replaced, the preceding and following addresses are not set.

At timing (6), the interpolation frame generation unit 360 generates an interpolation frame [3.2] from the frames [3] and [4] output from the address control unit 320 at timing (5), and outputs an output frame. Output as.

Thereafter, similarly, the address control unit 320 outputs only the read frame at the timing when the normal read frame is output, and the read frame (replaced frame) and the preceding and following frames at the timing when the replaced frame is output. Is output. When only the read frame is acquired, the interpolation frame generation unit 360 outputs the read frame as it is as an output frame at the timing after one frame. When the read frame and the preceding and following frames are acquired, the interpolation frame generation unit 360 generates and outputs an interpolation frame from the read frame and the previous and next frames at a timing one frame later.

Next, the interpolation frame generation method will be described in detail.

The interpolation frame is generated by performing linear interpolation using two frames. For example, the interpolation frame [1.8] is generated by linear interpolation using the motion vector obtained from the frame [1] and the frame [2].

First, the interpolation frame generation unit 360 divides, for example, the frame [1] and the frame [2] of the read frame into several small regions.

Next, the interpolation frame generation unit 360 obtains the SAD between one small area of the frame [2] (hereinafter referred to as a small area A) and each small area of the frame [1]. Subsequently, the interpolation frame generation unit 360 obtains a small area (hereinafter, referred to as a small area A ′) of the frame [1] corresponding to the smallest SAD value among the SADs obtained by the number of small areas. Further, the interpolation frame generation unit 360 uses the small area A and the small area A ′ to obtain a motion vector A that represents a change in position from the small area A to the small area A ′.

Next, the interpolation frame generation unit 360 includes a small area (hereinafter, a small area) at a position obtained by the motion vector A × 0.2 from the position of the small area A among the small areas included in the interpolation frame [1.8]. Region A ″).

The luminance of each pixel in the small area A ″ is obtained by proportionally distributing the luminance of each pixel in the small area A and the luminance of each pixel in the small area A ′. For example, the luminance of an arbitrary pixel a in the small area A ″ is (luminance of the pixel a in the small area A ″) = (luminance of the pixel corresponding to the pixel a in the small area A) × 0.8 + (small area A The luminance of the pixel corresponding to the pixel a of ′) × 0.2.

By obtaining the luminance of each pixel as described above, the luminance of the small area A ″ of the interpolation frame [1.8] is obtained. With respect to the other small areas of the frame [2], the luminance of the other small areas of the interpolation frame [1.8] is obtained by obtaining the motion vectors. As a result, an interpolation frame [1.8] is generated.

Similarly, the interpolation frame [2.5] is generated by linearly interpolating from the frame [2] and the frame [3] using a motion vector. The interpolation frame [3.2] is generated by linearly interpolating from the frame [3] and the frame [4] using a motion vector.

In this way, instead of outputting the frame specified by the read address as it is, by performing adjustment processing to generate an interpolated frame, replace it, and output it, the temporal continuity between frames is improved, and the video Judder is reduced.

Although the adjustment processing of the address control unit 320 and the interpolation frame generation unit 360 when the input frame rate is smaller than the output frame rate has been described so far, the same applies when the input frame rate is larger than the output frame rate.

FIG. 11 is a diagram for explaining the adjustment processing of the address control unit 320 and the interpolation frame generation unit 360 when the input frame rate is larger than the output frame rate.

In FIG. 11, when the read frame is output as an output frame as it is, the frame [2] corresponding to the read address “2” is thinned out and output by the frame skip process, so the timing (4) of the read frame The amount of change in address between frames is large.

In order to reduce this address change amount, in the example of FIG. 11, the interpolation frame generation unit 360 performs interpolation at each timing of outputting frames [3] and [4] of the read frame among the output timings of the output frame. Frames [2.3] and [3.7] are output.

Thus, the output frames are frames [0], [1], [2.3], [3.7], [5], [6],... Using the interpolation frame.

Note that the frame control procedure of the address control unit 320 and the interpolation frame generation method are the same as when the input frame rate is smaller than the output frame rate.

Finally, the processing sequence of the adjustment processing of the frame synchronization unit 300 according to the second embodiment will be described.

FIG. 12 is a flowchart showing the elimination of the synchronization error in the second embodiment.

The adjustment process of the frame synchronization unit 300 in the second embodiment is as follows.

First, the frame synchronization unit 300 acquires the input frame of the input video acquired by the video acquisition unit 100 (S210).

Subsequently, the frame synchronization unit 300 acquires the output frame rate determined by the output frame rate determination unit 200 (S220).

Next, the address control unit 320 detects a frame to be replaced based on the input frame rate and the output frame rate (S230).

Further, the address control unit 320 outputs the input frame as a read frame according to the output frame rate based on the phase difference (S240).

When the read frame is a frame to be replaced (YES in S250), the address control unit 320 outputs the previous and next frames (S260).

The interpolation frame generation unit 360 generates an interpolation frame and outputs it as an output frame when the read frame and the preceding and following frames are acquired (S270).

In addition, when only the read frame is acquired, the interpolation frame generation unit 360 outputs the read frame as an output frame as it is (S280).

As described above, in the second embodiment, the frame synchronization unit 300 does not output a frame specified by a read address as it is, but generates an interpolated frame, performs replacement processing, and outputs the interlaced frames. Improves temporal continuity of Thereby, smooth and high-quality video display is realized. Also, since not all frames are replaced with interpolation frames, but only necessary portions are replaced with interpolation frames, the load on signal processing is small, and synchronization deviation can be corrected effectively.

Furthermore, the present invention can be modified as follows.

(1) Each of the above devices is specifically a computer system including a microprocessor, a ROM, a RAM, a hard disk unit, a display unit, a keyboard, a mouse, and the like. A computer program is stored in the RAM or hard disk unit. Each device achieves its functions by the microprocessor operating according to the computer program. Here, the computer program is configured by combining a plurality of instruction codes indicating instructions for the computer in order to achieve a predetermined function.

(2) A part or all of the components constituting each of the above devices may be configured by one system LSI (Large Scale Integration). The system LSI is a super multifunctional LSI manufactured by integrating a plurality of components on a single chip, and specifically, a computer system including a microprocessor, a ROM, a RAM, and the like. . A computer program is stored in the RAM. The system LSI achieves its functions by the microprocessor operating according to the computer program.

(3) A part or all of the constituent elements constituting each of the above devices may be constituted by an IC card or a single module that can be attached to and detached from each device. The IC card or the module is a computer system including a microprocessor, a ROM, a RAM, and the like. The IC card or the module may include the super multifunctional LSI described above. The IC card or the module achieves its function by the microprocessor operating according to the computer program. This IC card or this module may have tamper resistance.

(4) The present invention may be the method described above. Further, the present invention may be a computer program that realizes these methods by a computer, or may be a digital signal composed of the computer program.

The present invention also provides a computer-readable recording medium such as a flexible disk, hard disk, CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, BD (Blu-ray Disc). ), Recorded in a semiconductor memory or the like. Further, the digital signal may be recorded on these recording media.

In the present invention, the computer program or the digital signal may be transmitted via an electric communication line, a wireless or wired communication line, a network represented by the Internet, a data broadcast, or the like.

Further, the present invention may be a computer system including a microprocessor and a memory, the memory storing the computer program, and the microprocessor operating according to the computer program.

In addition, the program or the digital signal is recorded on the recording medium and transferred, or the program or the digital signal is transferred via the network or the like, and executed by another independent computer system. It is good.

(5) The above embodiment and the above modifications may be combined.

As described above, the video signal processing device according to one aspect of the present invention has been described based on the embodiment and the modifications thereof. According to the video signal processing device of the present invention, the frame adjustment processing for eliminating the difference between the input frame rate and the output frame rate is performed during the period when the video motion is low, the video motion is high, and the video scene is switched. Is performed in a period corresponding to. In addition, according to the video signal processing apparatus of the present invention, interpolated frames are generated, replaced and output, thereby improving temporal continuity between frames.

This enables smooth and high-quality video display that makes it difficult for viewers to feel uncomfortable with the video, such as video judder.

For example, the video signal processing apparatus 10 according to each of the above embodiments is realized as the television 700 shown in FIG. At this time, the specific configuration of the video display unit 600 is not particularly limited, and is, for example, a liquid crystal display, a plasma display, an organic EL (Electro Luminescence) display, or the like. In this case, the video acquisition unit 100 acquires a video from a television broadcast, a DVD (Digital Versatile Disc) player 710 and a set top box 720 shown in FIG.

Further, for example, the video signal processing apparatus 10 may be realized as a DVD player 710. In this case, the video acquisition unit 100 acquires video from the inserted DVD. The acquisition source of the video is not limited to the DVD, and can be acquired from any recording medium such as Blu-ray Disc, HDD (Hard Disk Drive) and the like.

Furthermore, the video signal processing apparatus 10 may be realized as a set top box 720. In this case, the video acquisition unit 100 acquires video from cable television broadcasting or the like.

In addition, this invention is not limited to these embodiment or its modification. Unless it deviates from the gist of the present invention, various modifications conceived by those skilled in the art are applied to the present embodiment or the modification thereof, or a form constructed by combining different embodiments or components in the modification. It is included within the scope of the present invention.

According to the present invention, when a plurality of images with different frame rates are displayed in synchronization, the viewer can feel smooth and high-quality images without feeling uncomfortable, and has a function of displaying a plurality of images. The present invention is useful as a video signal processing device and a video signal processing method used for a display device such as a television or a personal computer.

DESCRIPTION OF SYMBOLS 10 Image | video signal processing apparatus 100 Image | video acquisition part 200 Output frame rate determination part 300 Frame synchronization part 310 Synchronization timing detection part 320 Address control part 360 Interpolation frame production | generation part 400 Frame memory 500 Image | video output part 600 Image | video display part 700 Television 710 DVD player 720 Set top box

Claims (11)

1. A video signal processing apparatus that outputs a plurality of input videos having different input frame rates in synchronization with each other,
   A video acquisition unit that sequentially acquires frames included in each of the plurality of input videos at the corresponding input frame rate;
   An output frame rate determining unit for determining an output frame rate;
   A frame synchronization unit that sequentially outputs the frames of the plurality of input videos acquired by the video acquisition unit in synchronization with the output frame rate;
   A video output unit that sequentially outputs a synthesized frame obtained by synthesizing each frame output in synchronization with the output frame rate from the frame synchronization unit;
   The frame synchronization unit adjusts to eliminate a difference between the input frame rate and the output frame rate for an asynchronous video having a corresponding input frame rate different from the output frame rate among the plurality of input videos. Is executed during a period that can be determined as a period in which the viewer does not easily stand out.
2. The frame sync part is
   A synchronization timing detection unit that detects an adjustment processing period in which the uncomfortable feeling that occurs in the composite frame by executing the adjustment process on the asynchronous video is less noticeable to a viewer who views the composite frame;
   The video signal processing apparatus according to claim 1, further comprising: a control unit that executes the adjustment process on the asynchronous video during the adjustment process period detected by the synchronization timing detection unit.
3. The synchronization timing detection unit
   Detecting the amount of motion of the asynchronous video;
   The video according to claim 2, wherein a period in which the amount of motion is smaller than a first threshold and a period in which the amount of motion is larger than a second threshold larger than the first threshold are detected as the adjustment processing period. Signal processing device.
4. The synchronization timing detection unit
   Detecting the switching of the asynchronous video scene,
   The video signal processing apparatus according to claim 2, wherein a period corresponding to switching of the video scene is detected as the adjustment processing period.
5. The frame synchronization unit performs the adjustment process as follows:
   If the input frame rate of the asynchronous video is larger than the output frame rate, a part of the frame is thinned out and output,
   5. The video signal processing apparatus according to claim 1, wherein the same frame is repeatedly output when the input frame rate of the asynchronous video is smaller than the output frame rate.
6. The frame synchronization unit performs the adjustment process as follows:
   Using M (M is an integer of 2 or more) original frames included in the asynchronous video during a predetermined period, N (M ≠ N) interpolation frames corresponding to frames between the M original frames are used. Generate
   The video signal processing apparatus according to claim 1, wherein, in the predetermined period, the generated N interpolation frames are output as frames of the asynchronous video.
7. The frame sync part is
   Detect motion vectors between original frames located before and after the generated interpolation frame,
   Assuming that an interpolated motion vector obtained by proportionally allocating the detected motion vector with a temporal distance of the interpolated frame with respect to the original frame is a motion vector of the interpolated frame with respect to the original frame, the interpolated frame is The video signal processing device according to claim 6 to be generated.
8. The video signal processing device according to any one of claims 1 to 7, wherein the output frame rate determination unit determines any one of the plurality of input videos as the output frame rate.
9. The video signal processing device according to any one of claims 1 to 7, wherein the output frame rate determination unit determines a frame rate different from any of the input frame rates of the plurality of input videos as the output frame rate.
10. A video signal processing method for synchronizing and outputting a plurality of input videos having different input frame rates,
    A video acquisition step of sequentially acquiring frames included in each of the plurality of input videos at the corresponding input frame rate;
    An output frame rate determining step for determining an output frame rate;
    A frame synchronization step for sequentially outputting frames of each of the plurality of input videos acquired by the video acquisition unit at the output frame rate;
    A video output step of sequentially outputting a synthesized frame obtained by synthesizing each frame output in synchronization with the output frame rate from the frame synchronization step;
    In the frame synchronization step, an adjustment process for eliminating a difference between the input frame rate and the output frame rate for an asynchronous video having a corresponding input frame rate different from the output frame rate among the plurality of input videos. To make it inconspicuous,
    The frame synchronization step is a synchronization timing for detecting an adjustment processing period in which the uncomfortable feeling generated in the composite frame due to the adjustment process being performed on the frame of the asynchronous video is less noticeable to a viewer who views the composite frame. A detection step;
    And a control step of performing the adjustment process on each frame of the asynchronous video and outputting the same during the adjustment process period detected in the synchro timing detection step.
11. A video signal processing method for synchronizing and outputting a plurality of input videos having different input frame rates,
    A video acquisition step of sequentially acquiring frames included in each of the plurality of input videos at the corresponding input frame rate;
    An output frame rate determining step for determining an output frame rate;
    A frame synchronization step for sequentially outputting frames of each of the plurality of input videos acquired by the video acquisition unit at the output frame rate;
    A video output step of sequentially outputting a synthesized frame obtained by synthesizing each frame output in synchronization with the output frame rate from the frame synchronization step;
    In the frame synchronization step, M (M is an integer of 2 or more) original frames included in a predetermined period of the asynchronous video having a corresponding input frame rate different from the output frame rate among the plurality of input videos. To generate N (M ≠ N) interpolation frames corresponding to frames between the M original frames,
    A video signal processing method for outputting the generated N interpolation frames as the asynchronous video frame in the predetermined period.

Images