KR102629441B1 - Image generation method and display device using the same - Google Patents

Abstract

The present invention relates to an image generation method and a display device using the same. The image generation unit of the display device shifts the first frame data received from the system control unit in the direction of the user's movement by the amount of the user's movement, so that the user's movement is recorded in real time. The reflected image data is generated as one or more frame data. Users can watch images that move in response to the user's movements on the display device without feeling motion sickness or fatigue.

Description

Image generation method and display device using the same {IMAGE GENERATION METHOD AND DISPLAY DEVICE USING THE SAME}

The present invention relates to an image generation method that reflects the user's movements on a screen in real time and a display device using the same.

Virtual reality technology is developing the fastest in the fields of defense, architecture, tourism, movies, multimedia, and games. Virtual reality refers to a specific environment or situation that feels similar to the real environment using stereoscopic imaging technology.

A virtual reality (hereinafter referred to as “VR”) system provides a virtual experience to the user by moving a three-dimensional image and outputting three-dimensional sound according to the user’s movements. VR systems can be implemented in the form of HMD (Head Mounted Display) and FMD (Face Mounted Display). The Augmented Reality (AR) system superimposes digital content on the real world. The augmented reality system can be implemented in the form of an Eye Glasses-type Display (EGD).

In VR/AR systems, the latency until the input image is displayed on the display panel screen has a significant impact on image quality. Image data generated from a graphics processing unit (hereinafter referred to as “GPU”) is written to the pixels of the display panel through the display driver. The data of the input image is displayed in pixels after a total delay time that is the sum of the image processing delay time of the GPU and the delay time of the display device. The VR system's GPU reflects the user's movement and creates a new image by rendering pixel data, but due to the rendering operation time, the delay time to create an image reflecting the movement does not keep up with the user's movement. . As a result, the user experiences increased motion sickness and fatigue.

The present invention provides an image generation method that can update an image reflecting the user's movements in real time on a display panel and a display device using the same.

The image generation method of the present invention includes receiving first frame data of an input image from a system control unit; generating second frame data reflecting the user's movement by starting rendering in response to a signal from a motion sensor at the system control unit; determining the direction and amount of movement of the user from the signal of the motion sensor at the image generator of the display device; Shifting the first frame data in the movement direction by the movement amount in the image generator of the display device to generate image data reflecting the user's movement in real time as one or more frame data; After the image generating unit transmits one or more frame data obtained from shifting the first frame data to a data driving unit, transmitting second frame data received from the system control unit to a data driving unit; and displaying an image reflecting the user's movement in real time on the screen of the display panel in the order of the first frame data, one or more frame data obtained from shifting the first frame data, and the second frame data. Includes.

While the system control unit performs a rendering operation to generate the second frame data, one or more frame data obtained from shifting the first frame data is displayed on the screen of the display panel.

The image generation method further includes the step of the system control unit repeatedly transmitting the first frame data to the image generator one or more times while the system control unit performs a rendering operation to generate the second frame data.

The resolution of each of the first and second frame data is the same as the screen resolution of the display panel.

The step of generating image data reflecting the user's movement in real time as one or more frame data by shifting the first frame data in the movement direction by the movement amount in the image generating unit of the display device is indicated by the user's movement direction. defining a reference area with a width equal to the amount of movement at an edge of the first frame data; Shifting the first frame data by the reference area in a direction opposite to the user's movement direction to generate image data reflecting the user's movement in real time as one or more frame data; and copying neighboring pixel data neighboring the reference area to the reference area.

The resolution of each of the first and second frame data is greater than the screen resolution of the display panel.

The step of generating image data reflecting the user's movement in real time as one or more frame data by shifting the first frame data in the movement direction by the movement amount in the image generator of the display device is performed at the top of the screen of the display panel. First pixel data to be written in the first pixel located on the leftmost side is shifted by the movement amount in the movement direction on the first frame data to generate one or more frame data.

A display device of the present invention includes a display panel including a screen in which a plurality of data lines, a plurality of gate lines, and a plurality of pixels are arranged in a matrix form; a system control unit connected to a motion sensor, generating first frame data of an input image, starting rendering in response to a signal from the motion sensor, and generating second frame data reflecting the user's movement; It is connected to the motion sensor to determine the user's movement direction and movement amount from the signal of the motion sensor, and shifts the first frame data in the movement direction by the movement amount to produce image data reflecting the user's movement in real time. An image generator generating the above frame data; a data driver that converts the image data received from the image generator into a data voltage and supplies it to the data lines; and a gate driver that supplies a gate signal synchronized with the data voltage to the gate lines. An image reflecting the user's movements in real time is displayed on the screen of the display panel in the order of the first frame data, one or more frame data obtained from shifting the first frame data, and the second frame data.

The present invention scales the input image to a preset resolution and transmits it to the timing controller of the display device. Before the image data reflecting the user's movement is received from the system control unit, the image data is distributed within the display device in the direction of the user's movement as much as the user's movement. By shifting, video data reflecting the user's movements in real time is generated without delay and reflected on the screen of the display panel.

Therefore, the display device of the present invention allows the user in a VR/AR system to view images that move in response to the user's movements on the display device without feeling motion sickness or fatigue.

Figure 1 is an exploded perspective view schematically showing the external configuration of a VR system according to an embodiment of the present invention.
Figure 2 is a block diagram showing a display device according to an embodiment of the present invention.
Figure 3 is a diagram showing an example of two screens implemented with two display panels.
Figure 4 is a diagram showing an example of two screens implemented with one display panel.
Figure 5 is a diagram showing an example of an image update being delayed due to the GPU rendering process.
Figure 6 is a diagram showing an example of image data generated during a GPU rendering process.
Figure 7 is a diagram showing image data generated in a display device according to an embodiment of the present invention.
8A and 8B are diagrams showing a method of generating an image of a display device according to a first embodiment of the present invention.
FIG. 9 is a diagram showing the input image of the timing controller and the image generated by the image generator in FIGS. 8A and 8B.
Figure 10 is a diagram showing the screen resolution of the display device and the resolution of overscan image data transmitted from the system control unit to the display device.
11A to 11F are diagrams showing a method of generating an image of a display device according to a second embodiment of the present invention.
Figures 12 and 13 are diagrams showing a method of transmitting resolution information of a display panel to a system control unit.
Figures 14 and 15 are diagrams showing the image generator in detail.

The advantages and features of the present invention and methods for achieving them will become clear by referring to the embodiments described in detail below along with the accompanying drawings. It should be noted that the present invention is not limited to the embodiments disclosed below and can be implemented in various forms. The following examples are provided to make the disclosure of the present invention complete and to fully inform those skilled in the art of the scope of the invention. The invention is defined by the claims.

The shapes, sizes, proportions, angles, numbers, etc. disclosed in the drawings for explaining embodiments of the present invention are illustrative, and the present invention is not limited to the matters shown. Like reference numerals refer to like elements throughout the specification. Additionally, in describing the present invention, if it is determined that a detailed description of related known technologies may unnecessarily obscure the gist of the present invention, the detailed description will be omitted.

When words such as 'provide', 'include', 'have', 'consist', etc. mentioned in the specification are used, other parts may be added unless 'only' is used. When a component is expressed in the singular, the plural is included unless specifically stated otherwise.

When interpreting a component, it is interpreted to include the margin of error even if there is no separate explicit description.

In the case of a description of a positional relationship, for example, if the positional relationship between two parts is described as 'on top', 'on top', 'at the bottom', 'next to ~', 'right next to' Alternatively, there may be one or more other parts placed between the two parts, unless 'directly' is used.

In the description of the embodiments, first, second, etc. are used to describe various components, but these components are not limited by these terms. These terms are merely used to distinguish one component from another.

Each feature of the various embodiments of the present invention can be partially or fully combined or combined with each other, and various technical interconnections and operations are possible, and each embodiment may be implemented independently of each other or together in a related relationship. It may be possible.

The display device of the present invention is applicable to systems that update frames of image data according to the user's movements, such as VR systems and AR systems. The display device of the present invention can be implemented as a flat panel display device such as a liquid crystal display (LCD) or an organic light emitting diode display (OLED display).

VR/AR systems can detect the user's movement using a motion sensor, and update the frame of image data reproduced on the display device when the user moves. A motion sensor may include a gyro sensor or an acceleration sensor.

The image generation method of the present invention transmits the output signal of the motion sensor to the display device during the rendering process required to update image data by reflecting the user's movement in the image processing unit of the system. The display device of the present invention updates image data on the screen by reflecting the user's movement without a rendering process by shifting pixel data to reflect the user's movement in response to the output signal of the motion sensor. The display device of the present invention can reproduce an image reflecting the user's movement in real time on the screen by generating one or more image data reflecting the user's movement and updating it on the screen while rendering is in progress in the image processing unit of the system. The image processing unit of the system can be interpreted as a GPU.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the attached drawings. Like reference numerals refer to substantially the same elements throughout the specification. In the following description, if it is determined that a detailed description of a known function or configuration related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description will be omitted.

Figure 1 is an exploded perspective view schematically showing the external configuration of a VR system according to an embodiment of the present invention. Figure 2 is a block diagram showing a display device according to an embodiment of the present invention. Figure 3 is a diagram showing an example of two screens implemented with two display panels. Figure 4 is a diagram showing an example of two screens implemented with one display panel.

1 to 4, the VR system includes a lens module (12), a display module (13), a main board (14), a headgear (11), and a side frame. (15), front cover (16), etc.

The display module 13 displays the input image received from the main board 14. The display module 13 includes a display panel 100 and a display driver 200 that writes pixel data (RGB) of an input image to the display panel 100. The display module 13 implements two screens AA with two display panels 100A and 100B as shown in FIG. 3 or two screens AA in one display panel 100 as shown in FIG. 4. A screen can be implemented. The first screen AA1 displays an image viewed from the user's left eye, and the second screen AA2 displays an image viewed from the user's right eye.

The image reproduced in the display module 13 shifts according to the user's movement. The image is shifted in the opposite direction of the user's movement direction. The image reproduced in the display module 13 may be a 2D/3D image in a VR system.

The display panel 100 may be implemented as a display panel of a flat panel display such as a liquid crystal display (LCD) or an organic light emitting diode display (OLED display). The display panel 100 includes data lines 102 to which data voltages are applied, gate lines (or scan lines, 104) to which gate pulses (or scan pulses) are applied, and data lines 102 and gate lines. It includes pixels 101 arranged in a matrix form defined by a cross structure of pixels 104 and electrically connected to data lines and gate lines. Each of the pixels 101 is divided into a red sub-pixel, a green sub-pixel, and a blue sub-pixel to implement color. Each of the pixels may further include a white subpixel. Each subpixel may include one or more thin film transistors (TFTs).

The display driver 200 writes pixel data (RGB) of the input image received from the GPU of the main board 14 into pixels of the display panel 100. As shown in FIG. 3, the display driver 200 includes a data driver 110, a gate driver 120, and a timing controller 130.

The data driver 110 converts pixel data (RGB) of the input image into a gamma compensation voltage, generates a data voltage, and supplies it to the data lines 102. The gate driver 120 sequentially supplies a gate signal synchronized to the data voltage to the gate lines 104.

The timing controller 130 transmits pixel data (RGB) of the input image received from the GPU of the main boat 150 to the data driver 110. Information necessary for driving the display panel 100 and basic information about the display device may be stored in the memory connected to the timing controller 130. The memory may be EEPROM (Electrically Erasable Programmable Read-Only Memory, 134). The timing controller 130 compares the timing signal received from the GPU of the main board 14 with the setting data stored in the EEPROM 134 and operates the data driver 110 and the gate driver 120 to synchronize with the pixel data of the input image. Controls driving timing.

The timing controller 130 can connect the EEPROM 134 to an external ROM writer and write data received through the ROM writer to the EEPROM 134. The timing controller 130 can store or update setting data and extended display identification data (EDID) data necessary for driving the display panel in the EEPROM 134. EDID data includes information about the display device, such as display device manufacturer information, product model information, screen resolution of the display panel 100, driving timing information, luminance, and pixel information.

In a mobile system such as a VR system or a wearable device, the timing controller 130 and the data driver 110 may be integrated into one drive IC 113 as shown in FIG. 4.

The lens module 12 includes an ultra-wide-angle lens, that is, a pair of fisheye lenses, to expand the angle of view of the user's left and right eyes. The pair of fisheye lenses includes a left eye lens disposed in front of the first screen AA1 and a right eye lens disposed in front of the second screen AA2.

The headgear 11 includes a back cover exposing the fisheye lenses and a band connected to the back cover. The back cover, side frame 15, and front cover 16 of the head gear 11 are assembled to secure an internal space where the VR system and components 12, 13, and 14 are placed and to protect the components. . The band connects to the back cover. Users wear the VR system on their head with a band. When a user puts the VR system on his or her head, he or she looks at different screens (AA, AA2) with the left and right eyes through the fisheye lenses.

The side frame (15) is fixed between the head gear (11) and the front cover (16) to secure the gap in the internal space where the lens module (12), display module (13), and main board (14) are placed. do. The front cover 16 is placed on the front of the VR system.

The main board 14 includes a system control unit 300, a sensor module 302, and a camera module 304. The main board 14 further includes an interface module and a sensor module 26 that are connected to external devices. The interface module is connected to external devices through interfaces such as Universal serial bus (USB) and High definition multimedia interface (HDMI).

The system control unit 300 executes virtual reality software. The system control unit 300 includes a GPU that supplies pixel data (RGB) of an input image and a timing signal synchronized with the pixel data (RGB) to the display module 13.

The sensor module 302 includes a motion sensor such as a gyro sensor and a 3-axis acceleration sensor. The sensor module 302 senses the user's movement and transmits it to the system control unit 300 and the display driver 200.

The system control unit 300 may receive pixel data (RGB) of an input image from the outside through an interface module. The GPU converts the resolution of the input image to match the display information received from the display device. The display information may be EDIC (Extended Display Identification Data). The GPU responds to the output signal of the sensor module 302 by rendering image data reflecting the user's movement to generate new frame data, thereby moving the images displayed on the screens AA2 and AA2 according to the user's movement. I order it.

The GPU analyzes the image obtained from the camera 304 by executing a preset eye tracking algorithm to estimate the focus area where the user's gaze is directed. The GPU can use a foveated rendering algorithm to increase the resolution of the focus area where the user's gaze is directed and lower the resolution of the input image in surrounding areas outside the focus area.

When the GPU detects the user's movement through motion tracking using a motion sensor, it renders pixel data to reflect the user's movement, generates image data reflecting the user's movement, and transmits it to the timing controller 130.

The timing controller 130 includes an image generator 132 that generates one or more image data while the GPU reflects the user's movements and renders them. The image generator 132 receives the output signal (MSS) of the motion sensor from the sensor module 302 and determines the user's movement direction (Vector) and movement amount (Scalar). The image generator 132 shifts the current image data according to the user's movement and generates one or more image data faster than frame data generated after a pixel rendering operation by the GPU.

The image generator 132 receives the first frame data of the input image from the system control unit 300. The system control unit 300 starts rendering in response to a signal from the motion sensor and generates second frame data reflecting the user's movement. The image generator 132 determines the user's movement direction and movement amount in response to a signal from the motion sensor, and shifts the first frame data to the user's movement direction by the movement amount to produce image data reflecting the user's movement in real time. Created using the above frame data. The image generator 132 transmits one or more frame data obtained from shifting the first frame data to the data driver 110, and then transmits the second frame data received from the system controller 300 to the data driver 110. do. As a result, an image reflecting the user's movement in real time is displayed on the screen AA of the display panel 100 in the order of the first frame data, one or more frame data obtained from shifting the first frame data, and the second frame data. displayed.

While the system control unit 300 performs a rendering operation to generate second frame data, one or more frame data obtained from shifting the input image are displayed on the screen of the display panel 100. While the GPU unit performs a rendering operation to generate second frame data, the GPU repeatedly transmits one or more first frame data to the image generation unit 132.

Because the delay time required for rendering pixel data or calculating the Motion Estimation Motion Compensation (MEMC) algorithm, GPU rendering or MEMC delay time causes a time difference between the user's movement and the actual update time of the image reproduced on the display panel. do.

As shown in FIG. 5, when a user's movement is detected through motion tracking, the GPU of the system control unit 300 reflects the user's movement and performs a rendering operation to generate pixel data. Rendering performed on the GPU is a pre-rendering process that reflects the user's movements and adjusts the lighting effects applied to each object in a single scene (frame) data and the depth of each pixel data. Because of the large amount of calculations required for such rendering, the rendering time is typically 20ms or more.

MEMC compares pixel data between frames and calculates a motion vector using the sum of bilateral absolute difference (SBAD) as shown below for comparison between the previous frame and the current frame, so the delay time is high due to the large calculation amount. Not a lot. MEMC requires two frame memories because it compares two frame data.

The rendering time required for the next frame data to be generated by the VR system's GPU is approximately 32ms, making it unable to keep up with the user's movement speed. In FIG. 5, “RF(N)” and “RF(N+1)” are the N-th and N+1-th frame data of the input image generated by rendering by reflecting the user's movement on the GPU.

Therefore, the update of the image reproduced on the display panel 100 is delayed due to the rendering processing time of the GPU. Because of this, when the user's movement and the screen update do not match, and the time difference between the user's movement and the image update due to rendering is more than 20 ms, the user feels motion sickness.

Figure 6 is a diagram showing an example of image data generated during a GPU rendering process.

Referring to FIG. 6, when the user's movement is detected by motion tracking, the GPS starts rendering and transmits the previous image data [RF(N)] back to the display device during the rendering process, and then the user's movement is detected as a result of the rendering operation. The next reflected image data [RF(N+1)] can be transmitted to the display device. The display device may repeatedly display the same image data (still image) received from the system control unit 300 on the display panel while the user moves. In this case, the user may feel motion sickness because the same N-th image data [RF(N)] that does not reflect the movement is repeatedly displayed on the screen AA of the display panel 100 while the user moves.

Figure 7 is a diagram showing image data generated in a display device according to an embodiment of the present invention. In FIG. 7 , RF(N) and RF(N+1) are image data generated by the GPU of the system control unit 300. Image1 and Image2 are image data generated by the display device.

Referring to FIG. 7, when user movement is detected by a motion sensor, the GPU of the system control unit 300 performs rendering to generate image data reflecting the user movement. Rendering requires more than 20ms.

The image generator 132 of the display device may receive Nth image data [RF(N)] and the output signal (MSS) of the motion sensor from the GPU. The output signal of the motion sensor includes motion tracking data that indicates the user's movements in real time. Motion tracking data indicates the user's movement direction (Vector) and movement amount (Scalar).

The image generator 132 shifts the Nth image data [RF(N)] in the user's movement direction in response to the output signal (MSS) of the motion sensor to generate first shift image data (Image1), and the data driver ( 110). The first shifted image data (Image1) is data in which the display position of the Nth image data [RF(N)] is shifted in the direction of the user's movement by the amount of the user's movement. Next, the image generator 132 generates second shift image data (Image2) by shifting the first shift image data (Image1) in the user's movement direction in response to the output signal (MSS) of the motion sensor. The second shift image data (Image2) is data in which the display position of the first shift image data (Image1) is shifted in the direction of the user's movement by the amount of the user's movement.

Since the image generator 132 simply shifts the display position of the input image data without performing a rendering operation, there is almost no delay before generating the shifted image data (Image1, Image2). Accordingly, the image generator 132 may generate one or more frame data (Image1, Image2) reflecting the user's movement while the GPU performs a rendering operation to generate pixel data.

8A and 8B are diagrams showing a method of generating an image of a display device according to a first embodiment of the present invention.

Referring to FIGS. 8A and 8B, when the user's movement is detected according to the output signal of the motion sensor, the image generator 132 generates the user's image from the image data [RF(N)] received from the GPU of the system control unit 300. Defines the data of the reference area (REF) indicated by the direction of movement.

The Nth and N+1th image data [RF(N), RF(N+1)] in which the user's movement is reflected by the GPU is a first grayscale area 71 including pixel data of the first grayscale value, and a second grayscale It may include a second grayscale area 72 including pixel data of a value, and a third grayscale area 73 including pixel data of a third grayscale value. When the user's head moves to the upper right corner as shown in FIG. 8A, the first grayscale area 71 decreases and the second and third grayscale areas 72 and 73 widen.

The reference area (REF) is data to be displayed at the edge of the screen (AA) indicated by the user's movement direction. For example, as shown in FIG. 8A, when the user's head moves toward the upper right, the reference area (REF) is defined as the right edge and top edge areas of the screen. As shown in FIG. 8B, when the user's head moves toward the bottom left, the reference area (REF) is designated as the left edge and bottom edge areas of the screen. The width of the reference area (AA) is set to the amount of user movement.

The image generator 132 shifts the display position of the image data by the reference area REF in the direction opposite to the user's movement direction and transmits it to the data driver 110. Pixel data in the reference area (REF) is filled with pseudo data identical to the current pixel data. In FIG. 8A , pseudo data may be generated by copying pixel data of the second grayscale area 72. In FIG. 8B, pseudo data may be generated by copying pixel data of the first grayscale area 71.

Because the data displayed at the edge of the screen is background data, unless it is a frame where the scene changes, there is little data change between frames in the reference area and is not an area of interest to the user. Because of this, even if there is no change between frames in the reference area (REF), there is a high probability that the user will not recognize it.

The image generation method according to the first embodiment of the present invention is to obtain the N+1 image data [RF] from the GPU by shifting the display position of the current frame data and filling the reference area (REF) with pseudo data obtained from neighboring pixel data. (N+1)] may be generated before one or more shift image data (Image1, Image2) is received by the timing controller 130. Therefore, the image generation method according to the first embodiment of the present invention can improve the user's motion sickness and fatigue problems without deteriorating image quality by quickly updating the image displayed on the screen (AA) with image data that reflects the user's movements in real time. .

FIG. 9 shows the input image [RF(N)] of the timing controller 130 and the image (Image1) generated by the image generator 132 in FIGS. 8A and 8B. As shown in FIG. 9, the timing controller 130 receives the Nth image data [RF(N)]] having the same resolution as the resolution (x * y) of the screen AA of the display panel and displays the data as much as the user's movement. Only the display position of the data [RF(N)] is shifted to generate image data (Image1) that reflects the user's movements in real time.

Figure 10 is a diagram showing the screen resolution of the display device and the resolution of the overscan image transmitted from the system control unit to the display device. 11A to 11F are diagrams showing a method of generating an image of a display device according to a second embodiment of the present invention.

Referring to FIGS. 10 to 11F, the GPU of the system control unit 300 may transmit overscan image (OAA) data having a resolution (X * Y) greater than the resolution (x * y) of the display device to the display device. .

The image generator 132 converts the pixel data present in the selection area (SA) defined by the screen resolution (AA = x * y) size of the display panel 100 from the overscan image received from the GPU to the data driver 110. send to The image generator 132 determines the user's movement direction (Vector) and movement amount (Scalar) according to the output signal (MSS) of the motion sensor and shifts the display position of the selection area (SA) to the movement direction to create a shift image. Create data (Image1, Image2). The selection area (SA) is shifted by the amount of user movement.

The image generator 132 does not delay the overscan image data received from the GPU, but shifts only the display position of the first pixel data transmitted to the data driver 110 in each frame period on the received overscan image data. Here, the overscan image data is frame data [RF(N)] received from the GPU. The first pixel data is pixel data to be written in the first pixel [P(1,1)] located at the top left of the screen AA of the display panel 100, as shown in FIGS. 11A to 11F. Once the position of the first pixel data is determined, the pixel data of the selection area (SA) determined by the resolution of the screen based on the first pixel data is defined, so once the display position of the first pixel data is determined, the selection area is automatically selected. is defined.

FIG. 11A is an example in which the selection area (SA) including the first pixel data is shifted by X = +3 and Y = +3 in the FIG. 11B is an example in which the selection area SA including the first pixel data is shifted by X = -3 and Y = +3 in the X and Y axes according to the user's movement. FIG. 11C is an example in which the selection area SA including the first pixel data is shifted by X = +3 and Y = -3 in the X-axis direction and Y-axis direction according to the user's movement. FIG. 11D is an example in which the selection area SA including the first pixel data is shifted by X = -3 and Y = -3 in the X-axis and Y-axis directions according to the user's movement. Figure 11e is an example in which the selection area (SA) including the first pixel data is shifted by X = +3 and Y = -4 in the X-axis direction and Y-axis direction according to the user's movement. Figure 11f is an example in which the selection area (SA) including the first pixel data is shifted by X = -2 and Y = -4 in the X-axis direction and Y-axis direction according to the user's movement.

12 and 13 are diagrams showing a method of transmitting resolution information of the display panel 100 to the system control unit 300. In FIGS. 12 and 13, “RGB DATA” is input image data generated after rendering in the GPU of the system control unit 300. “EDID DATA” is data transmitted to the system control unit 300 including resolution information of the display device.

Referring to FIG. 12, the actual resolution (x * y) of the display panel 100 and a resolution different from the actual resolution (x * y) may be stored in the EEPROM 134. The resolution to be transmitted to the system control unit 300 may be selected by the timing controller 130. In detail, the timing controller 130 can enable one of two or more resolutions stored in the EEPROM 134 according to the EEPROM register setting. The EEPROM 134 transmits the resolution enabled by the timing controller 130 to the system control unit 300. The EEPROM 134 transmits EDID data including resolution information to the system control unit 300 in synchronization with clock timing through a standard interface such as an I2C interface. The GPU of the system control unit 300 scales the input image to the resolution received from the EEPROM 134 and transmits it to the timing controller 130.

Referring to FIG. 13, the EEPROM 134 can transmit the actual resolution (x * y) of the display panel 100 to the system control unit 300. The timing controller 130 may request a desired resolution from the system control unit 300. In this case, the GPU of the system control unit 300 scales the input image data according to the resolution information received from the timing controller 130 to produce image data [RF(N), RF of the resolution (REQ) requested by the timing controller 130. (N+1)] is transmitted to the timing controller 130.

7 to 9, when new image data reflecting the user's movement in real time is generated from image data having the actual resolution (x * y) of the display panel 100, the GPU of the system control unit 300 is x * y. Image data [RF(N), RF(N+1)] scaled to y resolution is transmitted to the timing controller 130. In this case, the resolution of EDID data transmitted from the EEPROM 134 to the system control unit 300 is the actual resolution of the display panel 100.

10 to 12, when new image data reflecting the user's movement in real time is generated from overscan image data with a resolution (X * Y) greater than the actual resolution (x * y) of the display panel 100, the system The GPU of the control unit 300 transmits overscan image data [RF(N), RF(N+1)] scaled to X * Y resolution to the timing controller 130. In this case, the resolution of the EDID data transmitted from the EEPROM 134 to the system control unit 300 is a higher resolution than the actual resolution of the display panel 100.

14 and 15 are diagrams showing the image generator 132 in detail.

Referring to FIG. 14, the image generator 132 includes a motion determination unit 31, a shift calculation unit 32, an image reception unit 33, a frame memory 34, and a data output unit 35.

The motion determination unit 31 receives the output signal (MSS) of the motion sensor, that is, motion tracking data, and determines the user's movement direction (Vector) and movement amount (Scalar). The shift calculation unit 32 defines the shift direction and shift amount of the data [RF(N)] received from the GPU of the system control unit 300 by the amount of the user's movement in the direction of the user's movement. The shift calculation unit 32 indicates the position of the first pixel data to be written in the first pixel [P(1,1)] of the screen AA in the N-th frame data [RF(N)] every frame period. Shift information can be output.

The image receiving unit 33 supplies input image data (RGB DATA) received from the GPU of the system control unit 300 to the frame memory 34. Input image data (RGB DATA) is image data [RF(N), RF(N+1)] generated after rendering by GPU. The frame memory 34 temporarily stores the image data [RF(N), RF(N+1)] from the image receiver 33 and transmits it to the data output unit 35.

The data output unit 35 shifts the input image data in response to shift information from the shift calculation unit 32. Image data (RGB') output from the data output unit 35 is transmitted to the data driver 110.

As described above, the image generator 132 generates image data that reflects the user's movements in real time simply by shifting the input image, so there is almost no delay in the input image data. Accordingly, as shown in FIG. 15, the frame memory 34 serving as a buffer memory can be omitted.

Through the above-described content, those skilled in the art will be able to see that various changes and modifications can be made without departing from the technical idea of the present invention. Therefore, the technical scope of the present invention should not be limited to what is described in the detailed description of the specification, but should be defined by the scope of the patent claims.

31: movement determination unit 32: shift calculation unit
33: video receiver 34: frame memory
35: data output unit 100: display panel
101: Pixel 110: Data driver
120: Gate driver 130: Timing controller
132: Image generator 134: EEPROM
200: display driver 300: system control unit
302: sensor module 304: camera
AA: Display panel screen OAA: Overscan image

Claims (17)

Receiving first frame data of an input image from a system control unit;
generating second frame data reflecting the user's movement by starting rendering in response to a signal from a motion sensor at the system control unit;
determining the direction and amount of movement of the user from the signal of the motion sensor at the image generator of the display device;
Shifting the first frame data in the movement direction by the movement amount in the image generator of the display device to generate image data reflecting the user's movement in real time as one or more frame data; and
After the image generator transmits one or more frame data obtained from shifting the first frame data to a data driver, transmitting second frame data received from the system controller to a data driver; and
A step of displaying an image reflecting the user's movement in real time on the screen of the display device in the order of the first frame data, one or more frame data obtained from shifting the first frame data, and the second frame data. do,
The step of shifting the first frame data in the movement direction by the movement amount in the image generator of the display device to generate image data reflecting the user's movement in real time as one or more frame data,
defining a reference area with a width equal to the amount of movement at an edge of the first frame data indicated by the direction of movement of the user;
Shifting the first frame data by the reference area in a direction opposite to the user's movement direction to generate image data reflecting the user's movement in real time as one or more frame data; and
An image generating method comprising copying data of neighboring pixels neighboring the reference area to the reference area. According to claim 1,
An image generating method in which one or more frame data obtained from shifting the first frame data are displayed on the screen of the display device while the system control unit performs a rendering operation to generate the second frame data. According to claim 1,
While the system control unit performs a rendering operation to generate the second frame data, the system control unit repeatedly transmits the first frame data to the image generator one or more times. According to claim 3,
A method of generating an image in which the resolution of each of the first and second frame data is equal to the screen resolution of the display device. delete According to claim 3,
A method of generating an image in which the resolution of each of the first and second frame data is greater than the screen resolution of the display device. According to claim 6,
The step of shifting the first frame data in the movement direction by the movement amount in the image generator of the display device to generate image data reflecting the user's movement in real time as one or more frame data,
Shifting first pixel data to be written in a first pixel located at the upper leftmost position on the screen of the display device by the movement amount in the movement direction on the first frame data to generate one or more frame data. How to create video. A display panel including a screen in which a plurality of data lines, a plurality of gate lines, and a plurality of pixels are arranged in a matrix form;
a system control unit connected to a motion sensor, generating first frame data of an input image, starting rendering in response to a signal from the motion sensor, and generating second frame data reflecting the user's movement;
It is connected to the motion sensor to determine the user's movement direction and movement amount from the signal of the motion sensor, and shifts the first frame data in the movement direction by the movement amount to produce one or more image data reflecting the user's movement in real time. An image generator that generates frame data;
a data driver that converts the image data received from the image generator into a data voltage and supplies it to the data lines; and
A gate driver that supplies a gate signal synchronized to the data voltage to the gate lines,
An image reflecting the user's movement in real time is displayed on the screen of the display panel in the order of the first frame data, one or more frame data obtained from shifting the first frame data, and the second frame data,
The image generator,
Defining a reference area with a width equal to the amount of movement at the edge of the first frame data indicated by the direction of movement of the user,
Shifting the first frame data by the reference area in a direction opposite to the direction of movement of the user to generate image data reflecting the user's movement in real time as one or more frame data,
A display device that copies data of neighboring pixels adjacent to the reference area to the reference area. According to claim 8,
A display device in which the system control unit repeatedly transmits the first frame data to the image generator one or more times while the system control unit performs a rendering operation to generate the second frame data. According to clause 9,
A display device wherein the resolution of each of the first and second frame data is the same as the screen resolution of the display panel. delete According to clause 9,
A display device in which the resolution of each of the first and second frame data is greater than the screen resolution of the display panel. According to claim 12,
The image generator,
A display device that generates one or more frame data by shifting first pixel data to be written in a first pixel located at the upper leftmost position on the screen of the display panel by the amount of movement in the direction of movement on the first frame data. According to claim 8,
It further includes a memory that stores display information including a screen resolution of the display panel and one or more resolutions different from the screen resolution,
A display device in which the system control unit receives activated resolution information from the memory, scales the input image to the received resolution, and transmits it to the image generator. According to claim 8,
a memory that stores the screen resolution of the display panel; and
Further comprising a timing controller connected to the memory to control the operation timing of the data driver and the gate driver,
A display device in which the system control unit receives resolution information of the display panel from the memory, scales the input image to the resolution requested by the timing controller, and transmits it to the image generator. According to claim 8,
The image generator,
a motion determination unit that determines the direction of movement of the user and the amount of movement of the user from the output signal of the motion sensor;
a shift calculation unit that outputs shift information defining a shift direction and shift amount of the first frame data in the direction of the user's movement by the amount of the user's movement;
an image receiving unit that receives the first and second frame data from the system control unit;
a memory for temporarily storing data from the image receiver; and
A display device comprising a data output unit that shifts the first frame data by the shift amount in the shift direction indicated by the shift information to generate image data reflecting the user's movement in real time and transmits the generated image data to the data driver. According to claim 8,
The image generator,
a motion determination unit that determines the direction of movement of the user and the amount of movement of the user from the output signal of the motion sensor;
a shift calculation unit that outputs shift information defining a shift direction and shift amount of the first frame data in the direction of the user's movement by the amount of the user's movement;
an image receiving unit that receives the first and second frame data from the system control unit; and
A display device comprising a data output unit that shifts the first frame data by the shift amount in the shift direction indicated by the shift information to generate image data reflecting the user's movement in real time and transmits the generated image data to the data driver.