WO2013153568A1

WO2013153568A1 - Video display device and integrated circuit

Info

Publication number: WO2013153568A1
Application number: PCT/JP2012/002472
Authority: WO
Inventors: 厳太朗竹田; 石川　雄一
Original assignee: パナソニック株式会社
Priority date: 2012-04-09
Filing date: 2012-04-09
Publication date: 2013-10-17

Abstract

This invention provides a video display device capable of performing frame interpolation satisfactorily even for a video for which calculating motion information is difficult. A video display device (30) that performs frame interpolation for increasing the frame rate, the video display device (30) being provided with: a drawing processor (31) for generating, using a first frame rate, a user-operable first image on the basis of a first image signal that has been inputted; a drawing controller (33) for acquiring operation information describing an operation and generating control information that includes the operation information when a user performs an operation with respect to the first image via an operation terminal (50); and an interpolation image generator (32) for generating an interpolation image of the first image on the basis of the first image and the control information, and outputting to a display device a video display signal including the first image and the interpolation image of the first image using a second frame rate that is higher than the first frame rate.

Description

Video display device and integrated circuit

The present invention relates to a video display device and a video display circuit, and more particularly to a video display device and an integrated circuit that perform frame interpolation for increasing a frame rate.

In a display device, when a video signal having a frame rate lower than the frame rate that can be displayed by the display device is input, conventionally, for example, frame rate conversion is performed to increase the frame rate in order to display the video smoothly. Yes.

Specifically, for example, when displaying a moving image captured by a mobile phone on a normal TV, or displaying a normal TV broadcast on a TV having a double-speed panel that displays video at a frame rate higher than that of the normal TV. Cases can be considered.

Here, as a technique related to frame rate conversion for increasing the frame rate, for example, by comparing two consecutive images among a plurality of images constituting a video, motion information including a motion direction and its size is obtained. There is a frame interpolation technique based on motion compensation in which an interpolation image is generated in accordance with the calculated motion direction and size, and an interpolation image is inserted between two consecutive images (for example, see Patent Document 1). .

JP 2008-166966 A

However, when the video is a moving image, it is difficult to calculate motion information, such as an image displaying newspaper text or an image displaying text containing small characters. In this case, a frame interpolation error may occur or the image may be blurred.

It is an object of the present invention to provide an image display device that can perform frame interpolation satisfactorily even for images for which motion information is difficult to calculate.

In order to solve the above-described problem, an aspect of the video display device according to the present invention is a video display device that performs frame interpolation for increasing a frame rate, and is based on an input first image signal. A rendering processing unit that generates a first image that can be operated at a first frame rate, and operation information that indicates the operation when the user performs an operation on the first image via an operation terminal And generating a control information including the operation information, and generating an interpolated image of the first image based on the first image and the control information, and the first frame rate An interpolated image generating unit that generates a video display signal including the first image and an interpolated image of the first image at a higher second frame rate, and outputs the video display signal to a display device; Be equipped That.

According to the video display device of the present invention, it is possible to perform frame interpolation satisfactorily even for video for which motion information is difficult to calculate.

FIG. 1 is a schematic block diagram illustrating a schematic configuration example of a video display device and its peripheral devices. FIG. 2A is a flowchart illustrating an example of an operation of detecting an operation of a user configuring the video display device. FIG. 2B is a flowchart illustrating an example of a processing operation of the video display device. FIG. 3A is a block diagram illustrating an example of a composite image. FIG. 3B is a block diagram illustrating an example of an interpolated image of a composite image. FIG. 3C is a block diagram illustrating an example of a composite image. FIG. 4A is a block diagram illustrating an example of a composite image. FIG. 4B is a block diagram illustrating an example of an interpolated image of a composite image. FIG. 4C is a block diagram illustrating an example of a composite image. FIG. 5 is a block diagram illustrating an example of a composite image when there are a plurality of object images. FIG. 6A is a block diagram illustrating an example of a left-eye image. FIG. 6B is a block diagram illustrating an example of a right-eye image. FIG. 7 is a block diagram illustrating a TV as an example of a display device including a video display device. FIG. 8 is a block diagram showing a mobile phone as an example of the operation terminal.

(Background to obtaining one embodiment of the present invention)
In recent years, in TV, in addition to TV broadcasting, an image other than TV broadcasting, for example, an Internet browser or the like, is displayed on a display panel.

Thus, when displaying an image such as a net browser, newspaper text, or magazine paper on a display device such as a TV or projector, not only the operation terminal provided in the display device such as a TV remote control but also a multi-function mobile phone. (For example, a smart phone) and a tablet are connected as an operation terminal, and it is possible to operate the image displayed on the display device with the operation terminal.

Here, as described above, in a display device that inputs a video signal having a frame rate lower than the frame rate that can be displayed on the display panel, such as a TV having a double-speed panel, a frame rate that increases the frame rate by motion compensation has been conventionally used. Conversion is in progress.

Video display devices that perform frame interpolation by motion compensation include, for example, GPU (Graphics Processing Unit) that operates at a lower frame rate than the display panel, FRC (Frame Rate Converter) that generates interpolated images, and various functions of the video display device And a central processing unit (CPU) that manages the system.

In the case of a TV having the above-described double-speed panel, the GPU generates an object image (an example of a first image) at a low frame rate from an object signal indicating a Web page, and a TV image (at a low frame rate from a broadcast signal). An example of the second image) is generated. The object image is, for example, an image of a net browser on which a web page is displayed. Further, the GPU generates a composite image at a low frame rate by combining the object image with the TV image.

FRC generates an interpolation image by motion compensation from two temporally continuous composite images. Specifically, the FRC acquires a composite image from the GPU at a low frame rate, compares two temporally continuous composite images, and generates an interpolated image of the composite image. Further, the FRC generates a video display image by inserting the interpolated image of the generated composite image between the two composite images used for generating the interpolated image, and outputs it to the display panel. With this configuration, an image can be displayed at a frame rate higher than the frame rate of the input video signal and object signal, that is, the frame rate of the display panel.

However, as described above, it is difficult to calculate motion information, such as text displayed on a net browser, it is difficult to perform frame interpolation by motion compensation, resulting in a frame interpolation error or blurred image. There is a case.

Specifically, when a user scrolls a Web page using an operation terminal (mobile phone), noise due to a frame interpolation error may occur or the display of the Web page may be blurred. In addition, when a Web page is moved on a TV screen, noise and display blur, or movement may be awkward.

If the GPU that performs frame interpolation by motion compensation is a high-speed GPU that operates at the same frame rate as the display device, there is no need to perform frame interpolation by motion compensation, which solves the problem of frame interpolation errors and display blurring. Although it is possible, a high-speed GPU is quite expensive and the manufacturing costs are considerably increased.

(Summary of Invention)
A video display device according to an aspect of the present disclosure is a video display device that performs frame interpolation to increase a frame rate, and a first image that can be operated by a user based on an input first image signal. When the user performs an operation on the first image via the operation terminal by the drawing processing unit generated at the first frame rate, the operation information indicating the operation is acquired and the operation information is included. Based on the drawing control unit for generating control information, the first image and the control information, an interpolated image of the first image is generated, and the second frame rate higher than the first frame rate is used to generate the interpolated image. An interpolation image generation unit that generates a video display signal including a first image and an interpolation image of the first image, and outputs the video display signal to a display device.

If the video display device is configured in this way, when an operation such as moving or scrolling the first image (object image) is performed, the operation is not based on motion compensation but based on operation information indicating an operation by the user. Is possible. Since operation information is used for generating an interpolated image instead of motion compensation, it is possible to prevent a frame interpolation error from occurring. Specifically, smooth movement of the first image, scroll without display blur in the first image, and the like can be realized. In addition, since the video display device having the above configuration does not require a high-speed GPU, an increase in manufacturing cost can be suppressed.

In addition, for example, the drawing processing unit further generates a second image that is not a target of the user operation based on the input second image signal, and the second image and the first image are generated. The drawing control unit outputs the control information and identification information for identifying the first image to the interpolation image generation unit, and the interpolation image generation unit And specifying the first image from the composite image using the specifying information, and generating an interpolated image of the first image based on the specified first image and the control information. Also good.

If the video display device is configured in this way, the first image is displayed when, for example, the first image is superimposed on the second image, or the second image and the first image are displayed side by side. Scrolling and moving can be performed smoothly without blurring the display.

Further, for example, the specific information may be configured to include a position of the first image with respect to the second image and a size of the first image.

Further, for example, the control information may include an operation direction and an operation speed of the first image with respect to the second image as the operation information.

For example, the drawing control unit calculates an operation direction and an operation speed in a three-dimensional space for each of the right-eye image and the left-eye image constituting the first image based on the operation information. The operation information is obtained, and the interpolation image generation unit generates a right-eye interpolation image based on an operation direction and an operation speed in the right-eye image and the right-eye image, and in the left-eye image and the left-eye image The left-eye interpolation image may be generated based on the operation direction and the operation speed.

If the video display device is configured in this way, even when a three-dimensional video (3D video) is displayed, smooth display without display blur due to user operation information becomes possible.

In addition, for example, when the first image signal indicates an object image that is a still image, the drawing processing unit generates the first image by drawing the object image at the first frame rate. You may comprise.

An integrated circuit according to an aspect of the present disclosure is an integrated circuit for a video display device that performs frame interpolation for increasing a frame rate, and is based on an input first image signal and can be operated by a user. When the user performs an operation on the first image through the operation terminal, the operation information indicating the operation is acquired, and the operation information is acquired. Based on the first image and the control information, a drawing control unit that generates control information including the first image and the first image and an interpolation image of the first image are generated, and the first image and the first image are generated. An interpolated image generation unit that outputs a video display signal including the interpolated image of the image to the display device.

Note that the present invention can be realized not only as an apparatus but also as a method using steps as processing units constituting the apparatus, as a program for causing a computer to execute the steps, or as a computer read recording the program. It can be realized as a possible recording medium such as a CD-ROM, or as information, data or a signal indicating the program. These programs, information, data, and signals may be distributed via a communication network such as the Internet.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Each of the embodiments described below shows a desirable specific example of the present invention. Constituent elements, arrangement positions and connection forms of constituent elements, processing, processing order, and the like shown in the following embodiments are merely examples, and are not intended to limit the present invention. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims showing the highest concept of the present invention are described as optional constituent elements that constitute a more desirable form.

(Embodiment 1)
The video display apparatus according to Embodiment 1 will be described with reference to FIGS. 1 to 4C.

The video display device 30 is a device that performs frame interpolation to increase the frame rate. In the present embodiment, an example will be described in which an object image that can be operated by the user and a video image that is not an operation target of the user are simultaneously displayed on the display device 20, and the object image is operated by the operation terminal 50. To do.

[Configuration of display device and peripheral device]
First, the device configuration of the display device 20 including the video display device 30 and its peripheral devices will be described with reference to FIG. FIG. 1 is a block diagram illustrating a configuration example of a display device 20 including a video display device 30, a video input device 10 that is a peripheral device, and an operation terminal 50. In the present embodiment, a case where the display device 20 is a TV shown in FIG. 7 will be described as an example.

The video input device 10 outputs a video signal STV (an example of a second image signal) that is not an operation target of the user to the display device 20. In the present embodiment, a case where the video input device 10 is a TV antenna and the video signal STV is a broadcast wave of TV broadcasting will be described as an example. Note that the video input device 10 is not limited to a TV antenna, and may be a set-top box for cable broadcasting (cable television).

In this embodiment, the operation terminal 50 is a multi-function mobile phone having a touch panel as shown in FIG. 7, and is connected to the display device 20 with an AV output cable, an HDMI cable (wired), or infrared communication (wireless). The case where it is comprised so that it may connect by is demonstrated to an example. The operation terminal 50 is configured to output an object signal SOB (corresponding to a first image signal) that can be operated by the user and an operation detection signal SOD to the display device 20. In the present embodiment, the object signal SOB will be described as an example in which the object signal SOB is output from the mobile phone to the video display device 30. However, a router for relaying data from the Internet line or a set top of cable broadcasting (cable television). You may comprise so that it may output to the video display apparatus 30 from other apparatuses, such as a box.

In this embodiment, an example will be described in which the object signal SOB is a signal indicating data such as HTML indicating a Web page on the Internet and an image of a net browser for displaying the Web page. In the present embodiment, the Web page is configured to include text composed of small characters. In this embodiment, the Web page data is data of the entire Web page, and may include a range not displayed on the network browser.

When the touch panel detects an operation on the object image by the user, the operation terminal 50 immediately outputs an operation detection signal SOD indicating the operation content to the video display device 30.

Further, the operation terminal 50 is not limited to a mobile phone. For example, a remote controller for operating a TV, which is an example of the display device 20, a PC (Personal Computer) mouse connected to the display device 20, and the display device 20. Any terminal that transmits the operation content intended by the user, such as a tablet or camera connected to the terminal, may be used. In the case of a camera, for example, a moving image captured by the camera is output as operation detection information to the video display device 30, and the operation detection unit 34 of the video display device 30 described later analyzes the moving image, and the operation content is determined from the user's movement. You may comprise so that it may identify.

As shown in FIG. 1, the display device 20 will be described as an example in which the display device 20 is a TV configured by including a video display device 30 and a display panel 40. The display device 20 is not limited to the TV, and may be a projector.

In the present embodiment, the display device 20 displays a TV broadcast screen (corresponding to a video image or a second image) and a net browser (object image) on which a Web page is displayed on the display panel 40. It can be displayed at the same time. In the present embodiment, the case where the video display device 30 is mounted on the display device 20 will be described as an example, but the present invention is not limited to this. The display device 20 may be configured to be connectable to the display device 20 or may be mounted on another device such as the user operation terminal 50.

The display panel 40 is configured to be able to display an image at a frame rate higher than a broadcast wave of TV broadcast (for example, 29.97 fps or 59.9 fps in terrestrial digital broadcast), for example, 120 fps.

In this embodiment, the video display device 30 includes a drawing processing unit 31 that operates at a first frame rate lower than that of the display panel 40, a drawing control unit 33 that controls each function of the video display device 30, and an operation terminal 50. An operation detection unit 34 that detects a user operation and an interpolated image generation unit 32 that performs frame interpolation to increase the frame rate are provided.

The drawing processing unit 31 is configured by a GPU, generates an object image that can be operated by the user from the input object signal SOB, generates a video image that is not a user operation target from the input video signal STV, A composite image is generated by combining the image and the object image. Note that, when the video signal STV or the object signal SOB indicates a still image, the rendering processing unit 31 of the present embodiment renders the still image at the first frame rate, and performs the second image or the first image. As a video.

The drawing control unit 33 is composed of a CPU and manages identification information SLO for identifying an object image. In this embodiment, the specific information SLO is the position (for example, coordinates on the video image) and size of the object image on the video image. The drawing control unit 33 outputs specific information SLO to the drawing processing unit 31 based on a request from the drawing processing unit 31.

Further, when the user operates the object image via the operation terminal 50, the drawing control unit 33 acquires operation information SOI indicating the operation from the operation detection unit 34. The operation information SOI here is, for example, quantification data of the operation direction and operation speed. The drawing control unit 33 converts the operation direction and operation speed quantification data included in the operation information SOI into a vector amount usable by the interpolation image generation unit 32. The drawing control unit 33 generates the control information SC including the specific information SLO and the operation information SOI after converting the quantified data into the vector amount, and outputs the control information SC to the interpolation image generation unit 32.

The interpolation image generation unit 32 is configured by FRC, and generates an interpolation image of the object image using the control information SC output from the drawing control unit 33. Furthermore, the interpolation image generation unit 32 generates an interpolation image of the video image by motion compensation. The interpolated image generating unit 32 combines the interpolated image of the video image and the interpolated image of the object image to generate an interpolated image of the combined image. The interpolated image generation unit 32 generates a video display signal including the composite image and the interpolated image of the composite image at the second frame rate, and outputs the generated video display signal to the display device.

When the operation detection unit 34 receives the operation detection signal SOD output from the operation terminal 50, the operation detection unit 34 acquires quantification data of the operation direction and the operation speed from the operation detection signal SOD. The operation detection unit 34 outputs the operation direction and operation speed quantification data to the drawing control unit 33 as operation information SOI.

Note that the drawing processing unit 31, the drawing control unit 33, and the interpolated image generation unit 32 are each configured to execute processing in parallel.

[Operation Example of Video Display Device 30]
Hereinafter, an operation example of the video display device 30 will be described with reference to FIGS. 2A, 2B, FIGS. 3A to 3C, and FIGS. 4A to 4C.

In the present embodiment, for the sake of simplicity, the actual TV broadcast frame rate is not necessarily the same, but the FRC processing speed is 50 Hz and the display panel 40 processing speed is 100 Hz.

In this embodiment, as shown in FIGS. 3A to 3C and FIGS. 4A to 4C, the video image 41 is displayed on the entire screen of the display panel 40, and the object image 42 is displayed on a part of the display panel 40. In addition, a case where the image is displayed in a hierarchy above the video image 41 will be described as an example. The size of the object image 42 is set smaller than the video image 41 so that the visibility of the video image 41 is not significantly impaired. Further, in the present embodiment, a case where there is one object image 42 and the size of the object image 42 is constant will be described as an example.

The video display device 30 executes a basic operation for generating an interpolated image of a composite image based on the control signal and an object operation detection operation for detecting a user operation. The basic operation is an operation that is always executed while the video signal STV and the object signal SOB are input. The object operation detection operation is an operation when a user operation is detected, and is executed asynchronously with the basic operation. 2A is a flowchart showing an object operation detection operation, and FIG. 2B is a flowchart showing a basic operation.

[Object operation detection operation]
In the object operation detection operation, as shown in FIG. 2A, the operation detection unit 34 of the video display device 30 receives an operation detection signal SOD that is immediately output when the user operates the operation terminal 50 (mobile phone) ( Step S11).

Upon receiving the operation detection signal SOD, the operation detection unit 34 generates operation information SOI including the operation type, quantified data obtained by quantifying the operation speed and the operation direction from the operation detection signal SOD, and the drawing control unit. It outputs to 33 (step S12).

Here, the operation type is an operation type for the object image, and a scroll operation for scrolling a Web page displayed on the net browser and a move operation for moving the net browser on the video image 41 will be described as examples.

For example, when the first contact detection position is a part corresponding to the outer frame portion of the net browser on the touch panel of the mobile phone, the operation type is determined as a moving operation, and the first contact detection position is outside the net browser. In the case of the Web page portion excluding the frame portion, it may be configured to determine that the operation is a scroll operation. The quantification data is obtained from, for example, the speed of the user's operation and the direction of the operation.

When the operation information SOI is output from the operation detection unit 34, the drawing control unit 33 converts the quantified data of the operation speed and the operation direction into a vector amount that can be used by the interpolation image generation unit 32. The vector amount here is expressed using, for example, the operation direction and the pixel amount. The drawing control unit 33 generates the control information SC including the specific information SLO and the operation information SOI after converting the quantified data into a vector amount, and outputs the control information SC to the interpolated image generation unit 32 (step S13). When the operation type of the operation information SOI indicates a move operation, the drawing control unit 33 updates the position of the object image 42 on the video image 41 in the specific information SLO to be managed. The drawing control unit 33 interpolates the vector amount in accordance with the frequency of the display panel 40 when the detection frequency for the user operation in the operation terminal 50 or the operation detection unit 34 is lower than the frequency of the display panel 40. Do. Vector quantity interpolation is performed using, for example, linear interpolation or Bezier curve interpolation.

[basic action]
In the basic operation, the video display device 30 receives the video signal STV from the video input device 10 and the object signal SOB from the mobile terminal as shown in FIG. 2B (step S21). Note that the video signal STV is continuously input to the video display device 30 because it is a TV broadcast signal in this embodiment. Since the object signal SOB is data of a web page, new web page data is input to the video display device 30 when the web page is opened.

The drawing processing unit 31 generates an object image 42 that can be operated by the user from the object signal SOB, generates a video image 41 that is not an operation target of the user from the video signal STV, and combines the object image 42 with the video image 41. (Step S22).

3A and 3C show an example of the composite image 43 when the Web page is scrolled. FIGS. 4A and 4C show an example of the composite image 43 when the net browser is moved. Show.

More specifically, the drawing processing unit 31 acquires the specific information SLO including the position and size of the object image 42 from the drawing control unit 33 for each video image 41. Next, the drawing processing unit 31 generates a composite image 43 by superimposing the object image 42 on the position on the video image 41 indicated by the specific information SLO.

In the present embodiment, the position of the object image 42 is a coordinate on the video image 41 where the pixel at the upper left corner of the net browser that is the object image 42 is displayed. The coordinates (x ₁ , y ₁ ) of the pixel in the upper left corner of the object image 42 are those in the right direction of the drawing from the pixel in the upper left corner when the coordinate of the pixel in the upper left corner of the video image 41 is (0, 0). This is expressed as (x ₁ , y ₁ ) using the number of pixels x ₁ and the number of pixels y ₁ in the downward direction of the drawing. The position of the object image 42 is not limited to the coordinates of the pixel at the upper left corner, but may be the coordinates of other pixels of the object image 42 such as the coordinates of the center pixel of the object image 42. Further, the coordinates (0, 0) may be set at another position of the video image 41. Furthermore, the position of the object image 42 may be expressed using a distance r from the coordinates (0, 0) and an angle θ from the x axis.

In this embodiment, the size of the object image 42 is the length (number of pixels) in the x-axis direction (horizontal direction in the drawing) and the length (number of pixels) in the y-axis direction (vertical direction in the drawing) of the net browser. However, the present invention is not limited to this.

The interpolation image generation unit 32 acquires the control information SC from the drawing control unit 33, and generates an interpolation image based on the control information SC (step S23).

More specifically, in the present embodiment, the interpolation image generation unit 32 has a processing speed of the drawing processing unit 31 of 50 Hz and a processing speed of the display panel 40 of 100 Hz. Frame interpolation for generating one interpolated image is performed. Note that the number of interpolated images generated and the insertion location for the entire video signal STV are appropriately set based on the frame rate of the input video signal STV and the frame rate that can be displayed on the display panel 40.

Here, the control information SC includes specific information SLO of the object image 42 in each of the preceding and succeeding composite images 43. In addition, when the operation detection unit 34 detects an operation on the object image 42 by the user, the control information SC includes the operation identification information, the operation amount, and the quantified data of the operation direction in addition to the two pieces of specific information SLO. And the operation information SOI converted into. The control information SC is acquired every time the interpolation image 42i of the object image is generated.

When the operation type information is not included in the control information SC, that is, when the object image 42 is not operated, the interpolated image generation unit 32 first determines, based on the position and size of the object image 42 indicated by the specific information SLO. The object image 42 is specified from each of the two composite images 43 that are temporally continuous. After specifying the object image 42, the interpolated image generation unit 32 generates an interpolated image 41i of the video image by motion compensation.

Further, after specifying the object image 42, the interpolation image generating unit 32 acquires the specified object image 42 as an interpolation image 42i of the object image from any one of the two temporally continuous composite images 43. Here, since the object image 42 is not operated, the object image 42 in the composite image 43 and the interpolation image 42i of the object image are the same. The position of the object image 42 in the interpolated image 41 i of the video image is the same as the position of the object image 42 in the video image 41.

The interpolated image generation unit 32 superimposes the interpolated image 42i of the object image on the interpolated image 41i of the video image to generate an interpolated image 43i of the composite image.

When the operation information SOI is included in the control information SC and the operation type information is information indicating a scroll operation, the interpolated image generation unit 32 first determines based on the position and size of the object image 42 indicated by the specific information SLO. The object image 42 is specified from each of the two composite images 43 that are continuous in time. In the present embodiment, the object image 42 is specified from the composite image 43 shown in FIGS. 3A and 3C. After specifying the object image 42, the interpolated image generation unit 32 generates an interpolated image 41i of the video image by motion compensation.

Further, as shown in FIG. 3B, the interpolated image generation unit 32 determines, based on the vector amount of the operation information SOI, from the object image 42 of the two composite images 43 that are temporally continuous after specifying the object image 42. An interpolation image 42i of the object image obtained by scrolling the web page is generated.

Specifically, for example, the interpolated image generation unit 32 scrolls the object image 42 of the previous composite image 43 shown in FIG. 3A according to the vector amount. In other words, the interpolated image generation unit 32 slides the Web page in the upward or downward direction based on the vector amount by the amount indicated by the vector amount. The interpolated image generating unit 32 obtains an image (image indicated by a broken line in FIG. 3A) that is included in the interpolated image 42 i of the object image from the object image 42 of the previous composite image 43 by scroll operation. To do. Note that the interpolated image generation unit 32 uses the object image 42 of the next composite image 43 for the portion missing due to the slide of the image (the image of the portion indicated by the one-dot broken line in FIG. 3C). Here, since it is a scroll operation, the position of the interpolation image 42i of the object image is the same as the position of the object image 42 in the preceding and succeeding composite images 43. Also, if there is a non-scrolling part in the object image, such as the UI (User Interface) of the net browser, the specific information includes the data of the Web data display part (the part to be scrolled) and the non-scrolling part. In addition, it is also desirable to generate an interpolation image by scrolling only the display portion of the Web page by the above-described method, or to generate the interpolation image by considering only the display portion of the Web page as an object image.

3B, the interpolated image generating unit 32 superimposes the interpolated image 42i of the object image on the interpolated image 41i of the video image, and generates an interpolated image 43i of the composite image.

When the operation information SOI is included in the control information SC and the operation type information is information indicating the movement operation of the network browser, the interpolation image generation unit 32 firstly determines the position and size of the object image 42 indicated by the specific information SLO. Based on this, the object image 42 is specified from each of the synthesized images 43 before and after the interpolation image to be inserted. It should be noted that the position of the object image 42 on the video image 41 is the same as that of the previous composite image 43, as can be seen from FIG. 4A and FIG. This is different from the later composite image 43. After specifying the object image 42, the interpolated image generation unit 32 generates an interpolated image 41i of the video image by motion compensation.

After specifying the object image 42, the interpolated image generation unit 32 acquires the specified object image 42 as an interpolated image 42i of the object image from any one of the preceding and succeeding synthesized images 43. In the movement of the network browser, the Web page displayed on the network browser or the display location of the Web page is not changed, so the object image 42 in the composite image 43 and the interpolation image 42i of the object image are the same.

Further, in this embodiment, the interpolation image generation unit 32 moves the position of the object image 42 in the previous composite image 43 that has been moved by the vector amount of the operation information SOI to the object image in the interpolation image 43i of the composite image. 42 is calculated.

4B, the interpolated image generation unit 32 superimposes the acquired object image 42 on the position of the object image 42 in the calculated interpolated image 43i of the interpolated image to generate an interpolated image 43i of the combined image.

After generating the interpolated image 43i of the composite image, the interpolated image generating unit 32 generates a video display signal in which the interpolated image 43i of the generated composite image is inserted between the composite images 43, and outputs it to the display panel 40 (step S24). ).

As described above, in the present embodiment, the object image 42 of the previous or next synthesized image 43 with no display blur is directly used as the interpolation image 42i of the object image, or before and after the display blur is not generated. Since the object image 42 of the composite image 43 is combined as it is and used as the interpolation image 42 i of the object image, the interpolation image 43 i of the composite image becomes an image with no display blur, and the accuracy of the interpolation image can be improved.

(Embodiment 2)
A video display device 30 according to the second embodiment will be described with reference to FIGS. 1 and 5.

The video display device 30 of the present embodiment is different from the video display device 30 of the first embodiment in that a plurality of object images 42a to 42g can be displayed on the screen of the display panel 40. . Therefore, the specific information SLO for specifying the object images 42a to 42g includes identification information of the object images 42a to 42g in the present embodiment.

As shown in FIG. 1, the display device 20 of the present embodiment is connected to the video input device 10 and the operation terminal 50, and includes the video display device 30 and the display panel 40, as in the first embodiment. A case of a TV is described as an example. The configurations of the video input device 10, the operation terminal 50, and the display panel 40 are the same as those in the first embodiment.

Similar to the first embodiment, the video display device 30 includes a drawing processing unit 31 that operates at a lower frame rate than the display panel 40, a drawing control unit 33 that controls each function of the video display device 30, and an operation terminal 50. An operation detection unit 34 that detects a user operation and an interpolated image generation unit 32 that performs frame interpolation to increase the frame rate are provided.

The drawing processing unit 31 is configured by a GPU as in the first embodiment. As shown in FIG. 5, the rendering processing unit 31 of the present embodiment generates a plurality of object images 42a to 42g that can be operated by the user from a plurality of input object signals SOB, and from the input video signal STV. A video image 41 that is not an operation target of the user is generated. The drawing processing unit 31 further synthesizes the video image 41 and the plurality of object images 42a to 42g to generate a composite image 43. In this embodiment, an example will be described in which the object image 42a is a net browser for displaying a Web page, and the object images 42b to 42g are content images. The content image is, for example, a thumbnail image of a movie. When the content image is selected, a movie guidance screen (such as an advertisement screen or a movie data sales screen) is displayed. In the present embodiment, since there are a plurality of object images 42a to 42g, the object signal SOB and the specific information SLO include identification information for identifying the corresponding object image.

The drawing processing unit 31 acquires a plurality of specific information SLOs from the drawing control unit 33, and specifies the specific information SLO corresponding to each of the plurality of object images 42a to 42g using the identification information. The drawing processing unit 31 identifies the position and size of each of the plurality of object images 42a to 42g based on the identification information SLO, and synthesizes the video image 41 and the plurality of object images 42a to 42g to form a composite image 43. Is generated.

In the display example shown in FIG. 5, a desired background image 44 includes a display area for displaying the video image 41, a display area for displaying a net browser, and a display area for displaying a plurality of content images. Is set. The display area of the video image 41 is set in the upper left part of the screen, the display area of the net browser is set in the upper right part of the screen, and the display areas of a plurality of content images are set in the lower part of the screen.

The drawing control unit 33 is configured by a CPU as in the first embodiment. The drawing control unit 33 of this embodiment manages specific information SLO including identification information, position, and size of the object signal SOB for each object signal SOB.

In the basic operation, the drawing control unit 33 outputs specific information SLO to the drawing processing unit 31 based on a request from the drawing processing unit 31.

In addition, in the object operation detection operation, the drawing control unit 33 according to the present embodiment, from the operation detection unit 34, when the user operates the object image 42a via the operation terminal 50, as in the first embodiment. Operation information SOI including identification information indicating the operation target object image 42a and quantification data indicating the operation content is acquired. The drawing control unit 33 generates control information SC including the specific information SLO and the operation information SOI, and outputs the control information SC to the interpolation image generation unit 32. When a plurality of object images are selected, control information SC is generated and output for all the selected object images.

As in the first embodiment, the interpolation image generation unit 32 is configured by FRC, generates an interpolation image based on the plurality of object images 42a to 42g and the control information SC, and interpolates the composite image 43 and the composite image. A video display signal composed of the image 43i is output to the display device.

When the operation type information is not included in the control information SC, the interpolated image generation unit 32 selects a plurality of object images 42a to 42g from each of the previous and next synthesized images 43 based on the control information SC from the drawing control unit 33. Identify. After specifying the plurality of object images 42a to 42g, the interpolated image generating unit 32 generates an interpolated image 41i of the video image by motion compensation. Further, as shown in FIG. 5, the interpolated image generating unit 32 synthesizes the interpolated image 41i of the video image and the plurality of identified object images 42a to 42g with the background image 44 to generate an interpolated image 43i of the synthesized image. To do.

When the operation information SOI is included in the control information SC, the interpolated image generation unit 32 first specifies a plurality of object images 42a to 42g from each of the composite images 43. After specifying the plurality of object images 42a to 42g, the interpolated image generating unit 32 generates an interpolated image 41i of the video image by motion compensation. Further, after specifying the object images 42a to 42g, the interpolation image generation unit 32 generates an interpolation image 42i of the object image corresponding to the operation type for each of the object images to be operated. The method for generating the interpolated image 42i of the object image corresponding to the operation type is the same as in the first embodiment. In addition, after specifying the object image 42a, the interpolation image generation unit 32 sets the object image of the composite image 43 before and after the object image that is not the operation target as the interpolation image 42i of the object image. The interpolated image generating unit 32 superimposes the interpolated image 41 i of the video image and the interpolated image 42 i of the object image on the background image 44 to generate an interpolated image 43 i of the composite image.

After the interpolated image 43i of the composite image is generated, the interpolated image generating unit 32 generates a video display signal in which the interpolated image 43i of the generated composite image is inserted between the composite images 43 and outputs the video display signal to the display panel 40.

The operation detection unit 34 receives the operation detection signal SOD output from the operation terminal 50, specifies the object image to be operated from the operation detection signal SOD, and detects the operation content. Further, the operation detection unit 34 outputs operation information SOI including identification information for specifying the object image and quantification data indicating the operation content to the drawing control unit 33.

When receiving the operation detection signal SOD output from the operation terminal 50, the operation detection unit 34 first specifies an object image to be operated from the plurality of object images 42a to 42g based on the operation detection signal SOD. Specifically, for example, in the case of a touch panel of a mobile phone, the object image at the first contact detection position is determined as the object image to be operated. Furthermore, the operation detection unit 34 detects the operation type, and data obtained by quantifying the operation speed and the operation direction from the operation detection signal SOD. The operation type and the quantification data detection method are the same as those in the first embodiment. The operation detection unit 34 generates operation information SOI including the identification information of the operation target object image, the operation type, and the quantification data, and outputs the operation information SOI to the drawing control unit 33.

(Embodiment 3)
The video display device 30 according to the third embodiment will be described with reference to FIGS. 1, 6A, and 6B.

The video display device 30 of the present embodiment is different from the video display device 30 of the second embodiment in that the display panel 40 can display a 3D image, and the right eye image and the left eye for displaying the 3D image. It is a point which produces | generates the video display signal comprised by the image for work.

As shown in FIG. 1, the display device 20 of the present embodiment is connected to the video input device 10 and the operation terminal 50 as in the first and second embodiments, and the video display device 30, the display panel 40, and the like. A case where the TV is configured to include the above will be described as an example. The configurations of the video input device 10, the operation terminal 50, and the display panel 40 are the same as those in the first and second embodiments.

The drawing processing unit 31 is configured by a GPU, generates a plurality of object images 42a to 42g that can be operated by the user from a plurality of input object signals SOB, and is an operation target of the user from the input video signal STV. A video image 41 that is not to be generated is generated. In order to display a 3D image, two types of images, a right-eye image and a left-eye image, are generated as composite images. Further, the drawing processing unit 31 uses the specific information SLO to superimpose the video image 41 and the plurality of object images 42a to 42g on the background image 44 to generate a composite image.

6A is a block diagram illustrating an example of an interpolation image 43L for the left eye, and FIG. 6B is a block diagram illustrating an example of an interpolation image 43R for the right eye corresponding to FIG. 6A. 6A and 6B, a display area for displaying the video image 41 is set in the upper left part of the desired background image 44, a display area for displaying a net browser is set in the upper right part, and a plurality of display areas are set in the lower part. A display area for displaying the content image is set.

The drawing control unit 33 is constituted by a CPU and manages specific information SLO including identification information, position and size of the object signal SOB for each object signal SOB. The position here is a three-dimensional position. When the drawing control unit 33 outputs the specific information SLO to the drawing processing unit 31 and the interpolated image generation unit 32, the composite image to be generated and the interpolated image of the composite image are a composite image for the right eye or a composite for the left eye. Depending on whether it is an image, it is converted into coordinates on the corresponding image and output.

Further, in the object operation detection operation, the drawing control unit 33 includes identification information and quantified data indicating the operation content from the operation detection unit 34 when an object image is operated by the user via the operation terminal 50. Operation information SOI is acquired. The quantification data here is composed of the operation direction and operation speed in three dimensions. The drawing control unit 33 generates control information SC including the specific information SLO and the operation information SOI, and outputs the control information SC to the interpolation image generation unit 32. When a plurality of object images are selected, control information SC is generated and output for all the selected object images.

The interpolated image generation unit 32 is configured by FRC, generates an interpolated image based on the plurality of object images 42a to 42g and the control information SC, and outputs a video display signal composed of the composite image and the interpolated image of the composite image to the display device. Output.

The method for generating the interpolated image of the composite image when the operation type information is not included in the control information SC is the same as in the second embodiment.

In the case where the operation information SOI is included in the control information SC, the method for generating the interpolated image of the composite image is the same as that in the second embodiment when moving in the plane direction.

A case where movement in the depth direction is detected when the operation information SOI is included in the control information SC will be described with reference to FIGS. 6A and 6B. Here, a case where the net browser is moved to the near side will be described. 6A is a block diagram illustrating an example of an interpolation image 43L for the left eye, and FIG. 6B is a block diagram illustrating an example of an interpolation image 43R for the right eye. 6A and 6B, the broken line indicates the position of the net browser before movement.

In this case, the interpolated image generation unit 32 generates, as the left-eye interpolated image 43L, an image in which the net browser is moved to the right according to the vector amount as shown in FIG. 6A. Similarly, as the interpolation image 43R for the right eye, as shown in FIG. 6B, an image is generated by moving the net browser to the left according to the vector amount. The composite image generation method in this case is the same as that in the moving operation in the second embodiment.

After generating the left-eye interpolation image 43L and the right-eye interpolation image 43R, the interpolation image generation unit 32 synthesizes the generated synthesized image interpolation image for each of the left-eye interpolation image 43L and the right-eye interpolation image 43R. A video display signal inserted between the images is generated and output to the display panel 40.

The operation detection unit 34 receives the operation detection signal SOD output from the operation terminal 50, specifies the object image to be operated from the operation detection signal SOD, and quantifies the three-dimensional operation direction and operation speed. Is detected. The operation detection unit 34 generates operation information SOI including the identification information of the identified object image and the quantification data, and outputs the operation information SOI to the drawing control unit 33.

By configuring in this way, it is possible to cope with movement in the depth direction in the 3D image.

(Another embodiment)
(1) In Embodiments 1 to 3, the scroll operation and the movement operation have been described as examples of the operation type. However, the present invention is not limited to this. Other types such as an enlargement / reduction operation of an object image or an operation of replacing an object image when there are a plurality of object images may be used. Note that the enlargement / reduction of the object image can be handled by changing the position and size of the specific information SLO and adding change information (change amount) of the size of the object image to the operation information SOI.

(2) In the first to third embodiments, the case where the video image 41 and the object image are simultaneously displayed has been described. However, the video image 41 may be configured not to be displayed. That is, the configuration using the video image 41 is not an essential configuration of the present invention, but has been described as a more preferable embodiment. When the video image 41 is not displayed, the operation type may be a scroll operation or an object image enlargement / reduction.

(3) In the first to third embodiments, the first image has been described as an example of a net browser on which a Web page is displayed or a content image. However, the first image is not limited thereto. Absent. It may be another image that is a user's operation target, such as an icon.

Further, the second image is not limited to the TV image, and may be a still image such as a photograph or another moving image.

(4) In the first to third embodiments, each functional block in the block diagram (FIG. 1) is typically realized as an LSI which is an integrated circuit. These may be individually made into one chip, or may be made into one chip so as to include a part or all of them. The name used here is LSI, but it may also be called IC, system LSI, super LSI, or ultra LSI depending on the degree of integration.

Further, the method of circuit integration is not limited to LSI, and implementation with a dedicated circuit or a general-purpose processor is also possible. An FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used.

Furthermore, if integrated circuit technology that replaces LSI emerges as a result of advances in semiconductor technology or other derived technology, it is naturally also possible to integrate functional blocks using this technology. Biotechnology can be applied.

In the first to third embodiments, each component may be configured by dedicated hardware, or a component that can be realized by software is realized by executing a program. Also good.

As mentioned above, although embodiment of this invention was described with reference to drawings, this invention is not limited to embodiment mentioned above. Various modifications and variations can be made to the above-described embodiment within the same range or equivalent range as the present invention.

The video display device according to the present invention is useful when a tablet or a smartphone is connected as an operation terminal to a display device such as a TV or a projector.

DESCRIPTION OF SYMBOLS 10 Video input device 20 Display device 30 Video display device 31 Drawing process part 32 Interpolation image generation part 33 Drawing control part 34 Operation detection part 40 Display panel 41 Video image

41i Interpolation image

42, 42a, 42b, 42c, 42d of a

video image

42e, 42f, 42g Object image 42i Interpolated image 43 of object image 43 Composite image 43i Interpolated image 43L of composite image Interpolated image 43R for left eye Interpolated image 44 for right eye Background image 50 Operation terminal

Claims

A video display device that performs frame interpolation to increase the frame rate,
A rendering processing unit that generates a first image operable by the user at a first frame rate based on the input first image signal;
A drawing control unit that acquires operation information indicating the operation and generates control information including the operation information when the user performs an operation on the first image via an operation terminal;
An interpolated image of the first image is generated based on the first image and the control information, and the first image and the first image are generated at a second frame rate higher than the first frame rate. An interpolated image generating unit that generates a video display signal including the interpolated image and outputs the video display signal to a display device.
The drawing processing unit further generates a second image that is not a target of the user's operation based on the input second image signal, and combines the second image and the first image. Generate a composite image,
The drawing control unit outputs the control information and identification information for identifying the first image to the interpolation image generation unit,
The interpolated image generation unit specifies the first image from the synthesized image using the specifying information, and generates an interpolated image of the first image based on the specified first image and the control information The video display device according to claim 1.
The video display device according to claim 2, wherein the specific information includes a position of the first image with respect to the second image and a size of the first image.
The video display device according to claim 2, wherein the control information includes, as the operation information, an operation direction and an operation speed of the first image with respect to the second image.
The drawing control unit calculates an operation direction and an operation speed in a three-dimensional space for each of the right-eye image and the left-eye image constituting the first image based on the operation information, and calculates the operation information. Seeking
The interpolation image generation unit generates a right-eye interpolation image based on an operation direction and an operation speed in the right-eye image and the right-eye image, and an operation direction and an operation speed in the left-eye image and the left-eye image The video display device according to any one of claims 1 to 4, wherein an interpolated image for the left eye is generated on the basis of.
The drawing processing unit generates the first image by drawing the object image at the first frame rate when the first image signal indicates an object image that is a still image. 5. The video display device described in 5 above.
An integrated circuit for a video display device that performs frame interpolation to increase a frame rate,
A drawing processing unit that generates a first image operable by a user at the frame rate based on the input first image signal;
A drawing control unit that acquires operation information indicating the operation and generates control information including the operation information when the user performs an operation on the first image via an operation terminal;
An interpolated image of the first image at the frame rate is generated based on the first image and the control information, and a video display signal including the first image and the interpolated image of the first image is generated. An integrated circuit comprising: an interpolated image generation unit that outputs to a display device.