KR20200071521A - Image generation method and display device using the same - Google Patents

Abstract

The present invention relates to an image generating method and a display device using the same. The image generator of the display device shifts a first frame data received from a system controller in the direction of a user movement by the amount of user movement to generate image data reflecting the user movement in real-time as one or more frame data. The user can enjoy a moving image in response to the user movement in real-time on the display device without feeling nausea or tired.

Description

Image generation method and display device using the same{IMAGE GENERATION METHOD AND DISPLAY DEVICE USING THE SAME}

The present invention relates to an image generation method for reflecting a user's movement on a screen in real time and a display device using the image.

Virtual reality technology is the fastest developing in defense, architecture, tourism, film, multimedia, and gaming. Virtual reality refers to a specific environment or situation that feels similar to the real environment using stereoscopic image technology.

The virtual reality (hereinafter referred to as “VR”) system moves a stereoscopic image according to a user's movement and outputs a stereoscopic sound to provide a virtual experience to the user. The VR system may be implemented in the form of a head mounted display (HMD) or a face mounted display (FMD). Augmented Reality (hereinafter referred to as “AR”) systems allow digital content to be superimposed on the real world. The augmented reality system may be implemented in the form of an eye glasses-type display (EGD).

In the VR/AR system, the latency until the input image is displayed on the screen of the display panel has a significant effect on image quality. Image data generated from a graphic processing unit (hereinafter referred to as “GPU”) is written to pixels of a display panel through a display driver. The data of the input image is displayed on the pixels after the total delay time, which is the sum of the image processing delay time of the GPU and the delay time of the display device. The GPU of the VR system reflects the user's movement and renders pixel data to create a new image. Due to the rendering operation time, the delay time before generating the image reflecting the movement cannot keep up with the user's movement. . Due to this, the user becomes greatly sick and tired.

The present invention provides an image generation method capable of updating an image reflecting a user's movement in real time on a display panel and a display device using the image.

The method for generating an image of the present invention includes receiving first frame data of an input image from a system controller; Generating, by the system controller, second frame data reflecting the movement of the user by starting rendering in response to a signal from a motion sensor; Determining a user's movement direction and movement amount from a signal of the motion sensor in an image generation unit of a display device; Generating, by the image generation unit of the display device, the first frame data by shifting the amount of movement in the movement direction to generate image data reflecting the user's movement in real time as one or more frame data; After the image generation unit transmits one or more frame data obtained from the shift of the first frame data to a data driver, transmitting the second frame data received from the system controller to the data driver; And displaying, in the order of the first frame data, the first frame data, one or more frame data, and the second frame data, an image in which the movement of the user is reflected in real time on a screen of the display panel. Includes.

During the rendering operation for generating the second frame data in the system control unit, one or more frame data obtained from the shift of the first frame data is displayed on the screen of the display panel.

The image generation method further includes the step of repeatedly transmitting the first frame data to the image generation unit by the system control unit during the rendering operation for generating the second frame data by the system control unit.

The resolution of each of the first and second frame data is the same as the screen resolution of the display panel.

In the image generating unit of the display device, shifting the first frame data by the amount of movement in the movement direction to generate image data reflecting the user's movement in real time as one or more frame data indicates that the user's movement direction is indicated. Defining a reference area with a width equal to the amount of motion at the edge of the first frame data; Shifting the first frame data by the reference area in a direction opposite to the user's movement direction to generate image data reflecting the user's movement in real time as one or more frame data; And copying neighboring pixel data adjacent to the reference area to the reference area.

The resolution of each of the first and second frame data is greater than the screen resolution of the display panel.

In the image generating unit of the display device, the step of shifting the first frame data by the amount of movement in the movement direction to generate image data reflecting the user's movement in real time as one or more frame data is displayed on the screen of the display panel. The first pixel data to be written in the first pixel located on the leftmost side is shifted by the amount of movement in the movement direction on the first frame data to generate one or more frame data.

The display device of the present invention includes a display panel including a screen in which a plurality of data lines, a plurality of gate lines, and a plurality of pixels are arranged in a matrix form; A system control unit connected to a motion sensor to generate first frame data of an input image, start rendering in response to a signal of the motion sensor, and generate second frame data reflecting the user's movement; It is connected to the motion sensor to determine the user's movement direction and amount of movement from the signal of the motion sensor, and shift the first frame data by the amount of movement in the movement direction to display the image data reflecting the movement of the user in real time An image generator that generates the above frame data; A data driver that converts image data received from the image generator into a data voltage and supplies the data lines to the data lines; And a gate driver supplying a gate signal synchronized with the data voltage to the gate lines. In the order of the first frame data, the one or more frame data obtained from the shift of the first frame data, and the second frame data, an image in which the user's movement is reflected in real time is displayed on the screen of the display panel.

According to the present invention, before the image data reflecting the user's movement is received from the system control unit that scales the input image to a preset resolution and transmits it to the timing controller of the display device, the image data in the display device is moved as much as the user's movement in the user's movement direction. By shifting, image data reflecting the user's movement in real time without delay is generated and reflected on the display panel.

Accordingly, the display device of the present invention enables the user to view a moving image in response to a user's movement on the display device without motion sickness and fatigue in the VR/AR system.

1 is an exploded perspective view schematically showing an external configuration of a VR system according to an embodiment of the present invention.
2 is a block diagram showing a display device according to an exemplary embodiment of the present invention.
3 is a view showing an example in which two screens are implemented with two display panels.
4 is a diagram showing an example in which two screens are implemented by one display panel.
5 is a diagram illustrating an example in which an image update is delayed due to a rendering process of a GPU.
6 is a diagram showing an example of image data generated in the rendering process of the GPU.
7 is a view showing image data generated by a display device according to an embodiment of the present invention.
8A and 8B are diagrams illustrating a method of generating an image of a display device according to a first embodiment of the present invention.
9 is a view showing an input image of the timing controller and the image generated by the image generation unit in FIGS. 8A and 8B.
FIG. 10 is a view showing the screen resolution of the display device and the resolution of the overscan image data transmitted from the system control unit to the display device.
11A to 11F are diagrams showing a method of generating an image of a display device according to a second embodiment of the present invention.
12 and 13 are diagrams showing a method of transmitting resolution information of a display panel to a system control unit.
14 and 15 are diagrams showing the image generator in detail.

Advantages and features of the present invention, and methods for achieving them will be clarified with reference to embodiments described below in detail together with the accompanying drawings. It should be noted that the present invention is not limited to the embodiments disclosed below, but can be implemented in various forms. The following examples are provided to make the disclosure of the present invention complete, and to fully inform the scope of the invention to those of ordinary skill in the art. The invention is defined by the claims.

The shapes, sizes, ratios, angles, numbers, etc. disclosed in the drawings for describing the embodiments of the present invention are exemplary and the present invention is not limited to the illustrated matters. The same reference numerals refer to the same components throughout the specification. In addition, in the description of the present invention, when it is determined that detailed descriptions of related known technologies may unnecessarily obscure the subject matter of the present invention, detailed descriptions thereof will be omitted.

When'equipped','include','have','consist of', etc. mentioned in this specification are used, other parts may be added unless'~ only' is used. When a component is expressed as a singular number, the plural number is included unless otherwise specified.

In interpreting the components, it is interpreted as including the error range even if there is no explicit description.

In the case of the description of the positional relationship, for example, if the positional relationship of the two parts is described as'~ on','~ on top','~ on the bottom','~ next to', etc.,'right' Alternatively, one or more other parts may be located between the two parts unless'direct' is used.

In the description of the embodiment, the first, second, etc. are used to describe various components, but these components are not limited by these terms. These terms are only used to distinguish one component from another component.

Each of the features of the various embodiments of the present invention may be partially or wholly combined or combined with each other, technically various interlocking and driving may be possible, and each of the embodiments may be independently implemented with respect to each other or may be implemented together in an associative relationship. It might be.

The display device of the present invention is applicable to a system that updates a frame of image data according to a user's movement, such as a VR system or an AR system. The display device of the present invention may be implemented as a flat panel display device such as a liquid crystal display (LCD) or an organic light emitting display (OLED).

The VR/AR system can detect a user's movement using a motion sensor and updates a frame of image data reproduced on a display device when the user moves. The motion sensor may include a gyro sensor or an acceleration sensor.

The image generation method of the present invention transmits an output signal of a motion sensor to a display device in a rendering process necessary to update image data by reflecting a user's movement in the image processing unit of the system. The display device of the present invention updates the image data on the screen by reflecting the user's movement without rendering by shifting the pixel data by reflecting the user's movement in response to the output signal of the motion sensor. The display device of the present invention can reproduce one or more image data reflecting the user's movement while rendering is being performed by the image processing unit of the system and update it on the screen, thereby reproducing an image on which the user's movement is reflected in real time on the screen. The image processing unit of the system can be interpreted as a GPU.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Throughout the specification, the same reference numerals refer to substantially the same components. In the following description, when it is determined that a detailed description of a known function or configuration related to the present invention may unnecessarily obscure the subject matter of the present invention, the detailed description is omitted.

1 is an exploded perspective view schematically showing an external configuration of a VR system according to an embodiment of the present invention. 2 is a block diagram showing a display device according to an exemplary embodiment of the present invention. 3 is a view showing an example in which two screens are implemented with two display panels. 4 is a diagram showing an example in which two screens are implemented by one display panel.

1 to 4, the VR system includes a lens module 12, a display module 13, a main board 14, a headgear 11, and a side frame. (15), a front cover (front cover) 16, and the like.

The display module 13 displays the input image received from the main board 14. The display module 13 includes a display panel 100 and a display driver 200 that writes pixel data RGB of an input image to the display panel 100. The display module 13 implements two screens AA with two display panels 100A and 100B as shown in FIG. 3 or two display panels in one display panel 100 as shown in FIG. 4. You can implement the screen. The first screen AA1 displays an image seen by the user's left eye, and the second screen AA2 displays an image seen by the user's right eye.

The image reproduced in the display module 13 is shifted according to the user's movement. Conversely, the image is shifted with respect to the user's movement direction. The image reproduced in the display module 13 may be a 2D/3D image in a VR system.

The display panel 100 may be implemented as a display panel of a flat panel display device such as a liquid crystal display (LCD), an organic light emitting display (OLED), or the like. The display panel 100 includes data lines 102 to which a data voltage is applied, gate lines (or scan lines 104) to which a gate pulse (or scan pulse) is applied, and data lines 102 and a gate line. It includes pixels 101 arranged in a matrix form defined by the crossing structure of the fields 104 and electrically connected to data lines and gate lines. Each of the pixels 101 is divided into a red sub-pixel, a green sub-pixel, and a blue sub-pixel for color realization. Each of the pixels may further include a white sub-pixel. Each of the sub-pixels may include one or more thin film transistors (TFTs).

The display driver 200 writes the pixel data RGB of the input image received from the GPU of the main board 14 to the pixels of the display panel 100. The display driver 200 includes a data driver 110, a gate driver 120, and a timing controller 130 as shown in FIG. 3.

The data driver 110 converts the pixel data RGB of the input image into a gamma compensation voltage to generate a data voltage and supplies the data voltage to the data lines 102. The gate driver 120 sequentially supplies a gate signal synchronized with the data voltage to the gate lines 104.

The timing controller 130 transmits the pixel data RGB of the input image received from the GPU of the main boat 150 to the data driver 110. Information required for driving the display panel 100 and basic information of the display device may be stored in a memory connected to the timing controller 130. The memory may be EEPROM (Electrically Erasable Programmable Read-Only Memory, 134). The timing controller 130 compares the timing signal received from the GPU of the main board 14 with the setting data stored in the EEPROM 134 to synchronize the pixel data of the input image so that the data driver 110 and the gate driver 120 Control the driving timing.

The timing controller 130 may be connected to an external ROM writer, and the EEPROM 134 may write data received through the ROM writer to the EEPROM 134. The timing controller 130 may store or update configuration data required for driving the display panel and EDID (Extended Display Identification Data) data in the EEPROM 134. EDID data includes information about a display device such as manufacturer information of a display device, product model information, screen resolution of the display panel 100, driving timing related information, luminance, and pixel information.

In a mobile system such as a VR system or a wearable device, the timing controller 130 and the data driver 110 may be integrated in one drive IC 113 as shown in FIG. 4.

The lens module 12 includes an ultra-wide-angle lens, that is, a pair of fisheye lenses, for widening the angle of view of the left and right eyes of the user. The pair of fish-eye lenses includes a left-eye lens disposed in front of the first screen AA1 and a right-eye lens disposed in front of the second screen AA2.

The head gear 11 includes a back cover exposing fisheye lenses and a band connected to the back cover. The back cover, side frame 15 and front cover 16 of the head gear 11 are assembled to secure the interior space in which the VR system and components 12, 13, 14 are arranged and protect the components. . The band is connected to the back cover. The user wears the VR system on his head as a band. When the user wears the VR system on his head, he sees different screens AA and AA2 through the fish-eye lenses to the left and right eyes.

The side frame 15 is fixed between the head gear 11 and the front cover 16 to secure a gap in the interior space in which the lens module 12, the display module 13, and the main board 14 are disposed. do. The front cover 16 is disposed on the front of the VR system.

The main board 14 includes a system controller 300, a sensor module 302, and a camera module 304. The main board 14 further includes an interface module connected to an external device, a sensor module 26, and the like. The interface module is connected to an external device through an interface such as Universal serial bus (USB), High definition　multimedia interface (HDMI).

The system controller 300 runs virtual reality software. The system controller 300 includes a GPU that supplies pixel data RGB of the input image and a timing signal synchronized with the pixel data RGB to the display module 13.

The sensor module 302 includes a gyro sensor, a motion sensor, such as a 3-axis acceleration sensor. The sensor module 302 senses user movement and transmits it to the system controller 300 and the display driver 200.

The system controller 300 may receive pixel data (RGB) of the input image from the outside through the interface module. The GPU converts the resolution of the input image according to the display information received from the display device. The display information may be EDIC (Extended Display Identification Data). The GPU moves image displayed on the screens AA2 and AA2 according to the user's movement by generating new frame data by rendering image data reflecting the user's movement in response to the output signal of the sensor module 302. Order.

The GPU analyzes an image obtained from the camera 304 by executing a preset eye tracking algorithm to estimate a focus area facing the user's gaze. The GPU may increase the resolution of the focus area facing the user's gaze and lower the resolution of the input image in a peripheral area outside the focus area by using a foveated rendering algorithm.

When the user's movement is detected by motion tracking using a motion sensor, the GPU renders pixel data reflecting the user's movement, generates image data reflecting the user's movement, and transmits the image data to the timing controller 130.

The timing controller 130 includes an image generating unit 132 that generates one or more image data while rendering the GPU by reflecting the user's movement. The image generating unit 132 receives the output signal MSS of the motion sensor from the sensor module 302 and determines a user's motion direction (Vector) and motion amount (Scalar). The image generating unit 132 shifts the current image data according to a user's movement to generate one or more image data faster than frame data generated after a pixel rendering operation by the GPU.

The image generator 132 receives the first frame data of the input image from the system controller 300. The system controller 300 starts rendering in response to a signal from the motion sensor to generate second frame data reflecting the user's movement. The image generating unit 132 determines the user's movement direction and amount of movement in response to a signal from the motion sensor, and shifts the first frame data by the amount of movement in the direction of the user's movement to retrieve the image data reflecting the user's movement in real time. It is created with the above frame data. The image generator 132 transmits one or more frame data obtained from the shift of the first frame data to the data driver 110, and then transmits the second frame data received from the system controller 300 to the data driver 110. do. As a result, on the screen AA of the display panel 100, first frame data, one or more frame data obtained from the shift of the first frame data, and an image in which the user's movement is reflected in real time in the order of the second frame data are displayed. Is displayed.

During the rendering operation for generating the second frame data in the system control unit 300, one or more frame data obtained from the shift of the input image is displayed on the screen of the display panel 100. During the rendering operation for generating the second frame data in the GPU unit, the GPU repeatedly transmits one or more first frame data to the image generation unit 132.

Since the delay time required for the rendering of pixel data or the operation of the Motion Estimation Motion Compensation (MEMC) algorithm is required, the GPU rendering or MEMC delay time causes a time difference between the user's movement and the actual update time of the image reproduced on the display panel. do.

As illustrated in FIG. 5, when a user's movement is detected in motion tracking, the GPU of the system controller 300 reflects the user's movement and performs rendering operations to generate pixel data. Rendering performed in the GPU is a pre-rendering process that adjusts a lighting effect applied to each of the objects in one scene (frame) data and a depth of each pixel data in response to a user's movement. Because of the large amount of calculation required for such rendering, the rendering time is usually 20 ms or more.

MEMC calculates a motion vector with the sum of bilateral absolute difference (SBAD) as follows to compare pixel data between frames to compare the previous frame with the current frame. Not many Since MEMC compares two frame data, it needs two frame memories.

Until the next frame data is generated on the GPU of the VR system, the time required for rendering is about 32ms, and it cannot keep up with the user's movement speed. In FIG. 5, “RF(N)” and “RF(N+1)” are N and N+1 frame data of an input image generated by rendering by reflecting a user's movement in a GPU.

Therefore, the update of the image reproduced on the display panel 100 is delayed due to the rendering processing time of the GPU. Due to this, when the user's movement and the screen update are inconsistent, and the time difference between the rendering and the user's movement reflected in the video update is 20 ms or more, the user may feel motion sickness.

6 is a diagram showing an example of image data generated in the rendering process of the GPU.

Referring to FIG. 6, GPS starts rendering when a user's motion is detected by motion tracking, transmits previous image data [RF(N)] back to a display device in the rendering process, and then renders the user's motion The reflected next image data [RF(N+1)] may be transmitted to the display device. The display device may repeatedly display the same image data (still image) received from the system controller 300 while the user is moving. In this case, the user may feel motion sickness because it is repeatedly displayed on the screen AA of the display panel 100 of the same N-th image data [RF(N)] in which movement is not reflected in the process of the user moving.

7 is a view showing image data generated by a display device according to an embodiment of the present invention. In FIG. 7, RF(N) and RF(N+1) are image data generated by the GPU of the system controller 300. Image1 and Image2 are image data generated by the display device.

Referring to FIG. 7, when the user motion is detected by the motion sensor, the GPU of the system controller 300 performs rendering to generate image data reflecting the user motion. Rendering requires 20ms or more.

The image generating unit 132 of the display device may receive the N-th image data [RF(N)] and the output signal MSS of the motion sensor from the GPU. The output signal of the motion sensor includes motion tracking data indicating the user's movement in real time. The motion tracking data indicates a user's motion direction (Vector) and motion amount (Scalar).

The image generating unit 132 generates the first shift image data Image1 by shifting the N-th image data [RF(N)] in the user's movement direction in response to the output signal MSS of the motion sensor to generate a data driver ( 110). The first shift image data Image1 is data in which the display position of the Nth image data [RF(N)] is shifted by the amount of the user's movement in the direction of the user's movement. Subsequently, the image generation unit 132 generates the second shift image data Image2 by shifting the first shift image data Image1 in the user's movement direction in response to the output signal MSS of the motion sensor. The second shift image data Image2 is data in which the display position of the first shift image data Image1 is shifted by the amount of the user's movement in the direction of the user's movement.

Since the image generation unit 132 simply shifts the display position of the received image data without rendering operation, there is almost no delay time to generate the shifted image data (Image1, Image2). Accordingly, the image generation unit 132 may generate one or more frame data (Image1, Image2) reflecting the user's movement while the GPU performs rendering operation to generate pixel data.

8A and 8B are diagrams showing a method of generating an image of a display device according to a first embodiment of the present invention.

8A and 8B, when the user's movement is detected according to the output signal of the motion sensor, the image generation unit 132 uses the image data [RF(N)] received from the GPU of the system controller 300. Data of the reference area REF indicated by the direction of movement is defined.

The N and N+1 image data (RF(N), RF(N+1)) in which user motion is reflected by the GPU includes first grayscale regions 71 and second grayscales including pixel data of a first grayscale value. A second grayscale region 72 including pixel data of a value and a third grayscale region 73 including pixel data of a third grayscale value may be included. When the user's head moves to the upper right end as shown in Fig. 8A, the first gradation region 71 is reduced and the second and third gradation regions 72 and 73 are widened.

The reference area REF is data to be displayed on the edge of the screen AA indicated by the user's movement direction. For example, as illustrated in FIG. 8A, when the user's head moves in the upper right direction, the reference area REF is defined as a right edge and a top edge area of the screen. As shown in FIG. 8B, when the user's head moves in the lower left direction, the reference area REF is designated as the left edge and bottom edge regions of the screen. The width of the reference area AA is set by the amount of movement of the user.

The image generator 132 shifts the display position of the image data by the reference region REF in the direction opposite to the user's movement direction by the reference region REF and transmits the image data to the data driver 110. The pixel data of the reference area REF is filled with the same pseudo data as the current pixel data. In FIG. 8A, the pseudo data may be generated by copying pixel data of the second grayscale region 72. In FIG. 8B, the pseudo data may be generated by copying pixel data of the first grayscale region 71.

Since the data displayed on the edge of the screen is background data, there is little change in data between frames in the reference area unless the scene is a switched frame, and it is not the user's area of interest. For this reason, even if there is no inter-frame change in the reference area REF, there is a high probability that the user cannot recognize it.

The image generation method according to the first embodiment of the present invention shifts the display position of the current frame data and fills the reference area REF with pseudo data obtained from neighboring pixel data to generate N+1 image data [RF from the GPU] (N+1)] may be generated one or more shift image data (Image1, Image2) before being received by the timing controller 130. Therefore, in the image generating method according to the first embodiment of the present invention, the image displayed on the screen AA can be quickly updated with image data reflecting the user's movement in real time, thereby improving the user's motion sickness and fatigue problems without deteriorating image quality. .

9 shows an input image [RF(N)] of the timing controller 130 in FIGS. 8A and 8B and an image Image1 generated by the image generation unit 132. As shown in FIG. 9, the timing controller 130 receives the Nth image data [RF(N)] having the same resolution as the resolution (x * y) of the screen AA of the display panel, and as much as the user moves. Only the display position of the data [RF(N)] is shifted to generate image data Image1 reflecting the user's movement in real time.

FIG. 10 is a view showing a screen resolution of a display device and a resolution of an overscan image transmitted from the system control unit to the display device. 11A to 11F are diagrams showing a method of generating an image of a display device according to a second embodiment of the present invention.

10 to 11F, the GPU of the system controller 300 may transmit overscan image (OAA) data having a resolution (X * Y) greater than the resolution (x * y) of the display device to the display device. .

The image generation unit 132 uses the pixel driver existing in the selected area SA defined by the size of the screen resolution (AA = x * y) of the display panel 100 in the overscan image received from the GPU. Transfer to. The image generating unit 132 determines a user's motion direction (Vector) and motion amount (Scalar) according to the output signal (MSS) of the motion sensor, shifts the display position of the selection area SA in the motion direction, and shifts the image. Data (Image1, Image2) is generated. The selection area SA is shifted by the amount of movement of the user.

The image generator 132 shifts on the received overscan image data only the display position of the first pixel data transmitted to the data driver 110 every frame period without delaying the overscan image data received from the GPU. Here, the overscan image data is frame data [RF(N)] received from the GPU. The first pixel data is pixel data to be written in the first pixel [P(1,1)] positioned at the upper leftmost position on the screen AA of the display panel 100 as shown in FIGS. 11A to 11F. When the position of the first pixel data is determined, the pixel data of the selection area SA determined by the resolution of the screen based on the first pixel data is defined. Is defined.

11A is an example in which the selection area SA including the first pixel data is shifted by X = +3 and Y = +3 in the X-axis direction and the Y-axis direction according to the user's movement in the overscan image OAA. 11B is an example in which the selection area SA including the first pixel data is shifted by X = -3 and Y = +3 in the X-axis direction and the Y-axis direction according to the user's movement. 11C is an example in which the selection area SA including the first pixel data is shifted by X = +3 and Y = -3 in the X-axis direction and the Y-axis direction according to the user's movement. 11D is an example in which the selection area SA including the first pixel data is shifted by X = -3 and Y = -3 in the X-axis direction and the Y-axis direction according to the user's movement. 11E is an example in which the selection area SA including the first pixel data is shifted by X = +3 and Y = -4 in the X-axis direction and the Y-axis direction according to the user's movement. 11F is an example in which the selection area SA including the first pixel data is shifted by X = -2 and Y = -4 in the X-axis direction and the Y-axis direction according to the user's movement.

12 and 13 are diagrams showing a method of transmitting resolution information of the display panel 100 to the system control unit 300. 12 and 13, “RGB DATA” is input image data generated after rendering by the GPU of the system controller 300. “EDID DATA” is data transmitted to the system controller 300 including resolution information of the display device.

Referring to FIG. 12, an actual resolution (x * y) of the display panel 100 and a different resolution from the actual resolution (x * y) may be stored in the EEPROM 134. The resolution to be transmitted to the system controller 300 may be selected by the timing controller 130. In detail, the timing controller 130 may enable any one of two or more resolutions stored in the EEPROM 134 according to the EEPROM register setting. The EEPROM 134 transmits the resolution enabled by the timing controller 130 to the system controller 300. The EEPROM 134 transmits EDID data including resolution information to the system controller 300 in synchronization with clock timing through a standard interface such as an I2C interface. The GPU of the system controller 300 scales the input image with the resolution received from the EEPROM 134 and transmits it to the timing controller 130.

Referring to FIG. 13, the EEPROM 134 may transmit the actual resolution (x * y) of the display panel 100 to the system controller 300. The timing controller 130 may request a desired resolution from the system controller 300. In this case, the GPU of the system controller 300 scales the input image data according to the resolution information received from the timing controller 130 to obtain image data [RF(N), RF of the resolution REQ requested by the timing controller 130] (N+1)] to the timing controller 130.

7 to 9, when new image data reflecting a user's movement in real time is generated from image data having an actual resolution (x * y) of the display panel 100, the GPU of the system controller 300 is x * The image data (RF(N), RF(N+1)) scaled with y resolution is transmitted to the timing controller 130. In this case, the resolution of the EDID data transmitted from the EEPROM 134 to the system control unit 300 is the actual resolution of the display panel 100.

10 to 12, a system in which new image data reflecting a user's movement in real time is generated from overscan image data having a resolution (X*Y) greater than the actual resolution (x*y) of the display panel 100 The GPU of the controller 300 transmits overscan image data (RF(N), RF(N+1)) scaled with X*Y resolution to the timing controller 130. In this case, the resolution of the EDID data transmitted from the EEPROM 134 to the system controller 300 is a resolution greater than the actual resolution of the display panel 100.

14 and 15 are diagrams showing the image generator 132 in detail.

Referring to FIG. 14, the image generation unit 132 includes a motion determination unit 31, a shift calculation unit 32, an image reception unit 33, a frame memory 34, and a data output unit 35.

The motion determination unit 31 receives an output signal (MSS) of the motion sensor, that is, motion tracking data, and determines a user's motion direction (Vector) and motion amount (Scalar). The shift calculator 32 defines a shift direction and a shift amount of data [RF(N)] received from the GPU of the system controller 300 as much as the user's movement amount in the user's movement direction. The shift calculator 32 indicates the position of the first pixel data to be written in the first pixel [P(1,1)] of the screen AA in the Nth frame data [RF(N)] for every frame period. Shift information can be output.

The image receiving unit 33 supplies the input image data RGB data received from the GPU of the system controller 300 to the frame memory 34. The input image data (RGB DATA) is image data generated after rendering of the GPU (RF(N), RF(N+1)). The frame memory 34 temporarily stores image data (RF(N), RF(N+1)) from the image receiving unit 33 and transmits them to the data output unit 35.

The data output unit 35 shifts the input image data in response to the shift information from the shift calculation unit 32. The image data RGB' output from the data output unit 35 is transmitted to the data driver 110.

As described above, since the image generation unit 132 generates image data in which the user's movement is reflected in real time by simply shifting the input image, there is almost no delay in the input image data. Therefore, as illustrated in FIG. 15, the frame memory 34 serving as a buffer memory can be omitted.

Through the above description, those skilled in the art will appreciate that various changes and modifications are possible without departing from the technical idea of the present invention. Therefore, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be determined by the scope of the claims.

31: motion determination unit 32: shift calculation unit
33: image receiving unit 34: frame memory
35: data output unit 100: display panel
101: pixel 110: data driver
120: gate driver 130: timing controller
132: image generation unit 134: EEPROM
200: display driving unit 300: system control unit
302: sensor module 304: camera
AA: Display panel screen OAA: Overscan image

Claims (17)

Receiving first frame data of the input image from the system controller;
Generating, by the system controller, second frame data reflecting the movement of the user by starting rendering in response to a signal from a motion sensor;
Determining a user's movement direction and movement amount from a signal of the motion sensor in an image generation unit of a display device;
Generating, by the image generation unit of the display device, the first frame data by shifting the amount of movement in the movement direction to generate image data reflecting the user's movement in real time as one or more frame data; And
After the image generation unit transmits one or more frame data obtained from the shift of the first frame data to a data driver, transmitting the second frame data received from the system controller to the data driver; And
And displaying, in the order of the first frame data, one or more frame data obtained from the shift of the first frame data, and the second frame data, an image in which the movement of the user is reflected in real time on a screen of the display panel. How to create video. According to claim 1,
During the rendering operation for generating the second frame data in the system control unit, one or more frame data obtained from the shift of the first frame data is displayed on the screen of the display panel. According to claim 1,
During the rendering operation for generating the second frame data in the system control unit, the system control unit repeatedly transmitting the first frame data to the image generation unit one or more times. The method of claim 3,
A method of generating an image in which the resolution of each of the first and second frame data is the same as the screen resolution of the display panel. The method of claim 4,
In the image generation unit of the display device, shifting the first frame data by the amount of movement in the movement direction to generate image data reflecting the user's movement in real time as one or more frame data,
Defining a reference area with a width equal to the amount of movement from an edge of the first frame data indicated by the direction of movement of the user;
Shifting the first frame data by the reference area in a direction opposite to the user's movement direction to generate image data reflecting the user's movement in real time as one or more frame data; And
And copying surrounding pixel data adjacent to the reference area to the reference area. The method of claim 3,
A method of generating an image in which the resolution of each of the first and second frame data is greater than the screen resolution of the display panel. The method of claim 6,
In the image generation unit of the display device, shifting the first frame data by the amount of movement in the movement direction to generate image data reflecting the user's movement in real time as one or more frame data,
And generating the first pixel data to be written in the first leftmost pixel on the screen of the display panel by shifting the amount of motion in the direction of movement on the first frame data to generate one or more frame data. How to create an image. A display panel including a screen in which a plurality of data lines, a plurality of gate lines, and a plurality of pixels are arranged in a matrix form;
A system control unit connected to a motion sensor to generate first frame data of an input image, start rendering in response to a signal of the motion sensor, and generate second frame data reflecting the user's movement;
It is connected to the motion sensor to determine the user's movement direction and amount of movement from the signal of the motion sensor, and shift the first frame data by the amount of movement in the movement direction to display the image data reflecting the movement of the user in real time An image generator that generates the above frame data;
A data driver that converts image data received from the image generator into a data voltage and supplies the data lines to the data lines; And
And a gate driver supplying a gate signal synchronized with the data voltage to the gate lines,
A display device displaying an image in which the movement of the user is reflected in real time on a screen of the display panel in the order of the first frame data, one or more frame data obtained from the shift of the first frame data, and the second frame data. The method of claim 8,
During the rendering operation for generating the second frame data in the system control unit, the system control unit repeatedly transmits one or more of the first frame data to the image generation unit. The method of claim 9,
A display device having a resolution of each of the first and second frame data equal to a screen resolution of the display panel. The method of claim 10,
The image generation unit,
A reference area is defined by a width corresponding to the amount of movement from the edge of the first frame data indicated by the direction of movement of the user,
By shifting the first frame data by the reference area in a direction opposite to the user's movement direction, image data reflecting the user's movement in real time is generated as one or more frame data,
A display device for copying surrounding pixel data adjacent to the reference area to the reference area. The method of claim 9,
A display device having a resolution of each of the first and second frame data greater than a screen resolution of the display panel. The method of claim 12,
The image generation unit,
A display device that generates first or more frame data by shifting the first pixel data to be written in the first leftmost pixel on the screen of the display panel by the amount of movement in the movement direction on the first frame data. The method of claim 8,
Further comprising a memory for storing display information including the screen resolution of the display panel and one or more resolutions different from the screen resolution,
A display device that the system control unit receives the resolution information activated from the memory, and scales the input image to the received resolution and transmits it to the image generation unit. The method of claim 8,
A memory that stores the screen resolution of the display panel; And
Further comprising a timing controller connected to the memory to control the operation timing of the data driver and the gate driver,
The system control unit receives the resolution information of the display panel from the memory, and scales the input image to the resolution requested by the timing controller and transmits it to the image generation unit. The method of claim 8,
The image generation unit,
A motion determination unit that determines the direction of movement of the user and the amount of movement of the user from the output signal of the motion sensor;
A shift calculator configured to output shift information defining a shift direction and a shift amount of the first frame data as much as the user's motion amount in the user's motion direction;
An image receiving unit receiving the first and second frame data from the system control unit;
A memory for temporarily storing data from the image receiving unit; And
And a data output unit that shifts the first frame data in the shift direction indicated by the shift information by the shift amount to generate image data reflecting the user's movement in real time and transmits the image data to the data driver. The method of claim 8,
The image generation unit,
A motion determination unit that determines the direction of movement of the user and the amount of movement of the user from the output signal of the motion sensor;
A shift calculator configured to output shift information defining a shift direction and a shift amount of the first frame data as much as the user's motion amount in the user's motion direction;
An image receiving unit receiving the first and second frame data from the system control unit; And
And a data output unit that shifts the first frame data in the shift direction indicated by the shift information by the shift amount to generate image data reflecting the user's movement in real time and transmits the image data to the data driver.

Images