KR101870764B1 - Display apparatus using image conversion mechanism and method of operation thereof - Google Patents

Abstract

Display method. A display method according to an exemplary embodiment of the present invention includes: receiving a current input image having a current input pixel; Identifying an input object having an input object pixel based on a difference in characteristics of a current input pixel; Calculating an object pixel depth for the input object pixel from the image depth map of the current input image based on the weighted average of the depth candidates; And generating a processed image having a perceptual depth from a current input image having an image depth map and object pixel depth, for display on the device.

Description

TECHNICAL FIELD [0001] The present invention relates to a display device and a method of operating the same,

The present invention relates to a display device, and more particularly to an image conversion device.

Contemporary consumer electronics and industrial appliances, especially graphic display devices, TVs, projectors, mobile phones, portable digital products and combination devices are helping to create a richer modern life. R & D on currently commercialized technologies is taking place in various directions.

In particular, as users become more privileged with the development of 3D display devices, past or new paradigms are starting to take advantage of these new 3D display spaces. There are a variety of technical methods to take advantage of these new display device opportunities. One such conventional approach is to implement 3D imaging on consumer electronics, industrial electronics, and portable electronic devices such as video projectors, TVs, monitors, gaming systems and personal digital assistants (PDAs).

Service-based three-dimensional displays enable users to create, transfer, store, and consume information on their own, allowing users to create, transfer, store, and consume information on their own. One such use of such a service-based three-dimensional display is to efficiently implement a three-dimensional image on a display.

The three-dimensional display device is configured in a projector, a TV, a notebook, a portable device, and other portable products. Today, these systems are helping consumers by displaying relevant relevant information such as charts, maps or videos. 3D image display provides very useful related information.

However, displaying information in a three-dimensional format is of great concern to consumers. Implementing a three-dimensional image that is remote from the real world diminishes the advantages of using a three-dimensional display device. For example, an in-video object having a difference in three-dimensional sense may make the user uncomfortable.

Therefore, the three-dimensional display device still needs to include an image conversion mechanism for realizing a three-dimensional image. It is becoming increasingly important to find solutions to these problems as market competition intensifies unprecedentedly, consumer expectations increase, and opportunities for distinctive product differentiation within the market are declining. In addition, the need to reduce costs, increase efficiency and performance, and respond to competitive pressures make the critical need for finding solutions to these problems even more urgent.

The solution to these problems has been studied for a long time, but conventional development does not teach or suggest any solution, and therefore the solution to these problems remains a challenge to those of ordinary skill in the art.

A method of operating a display device according to the present invention includes: receiving a current input image including current input pixels; Identifying an input object comprising input object pixels based on a difference in the characteristics of the current input pixels; Calculating an object pixel depth of the input object pixels in the image depth map of the current input image based on the weighted average of the depth candidates; And generating a processed image having a recognized depth for display on the device from a current input image having an image depth map and an object pixel depth.

A display device according to the present invention includes: a communication unit for receiving a current input image including current input pixels; An object detection module, connected to the communication unit, for identifying an input object including input object pixels, based on differences in characteristics of the current input pixels; An object depth module coupled to the object detection module and calculating an object pixel depth of input object pixels from an image depth map of a current input image based on a weighted average of depth candidates; And an image transformation module, coupled to the object depth module, for generating a processed image having a depth recognized from a current input image having an image depth map and an object pixel depth.

The present invention includes any embodiment having other steps or elements added or substituted for the above description. Such steps or embodiments will become apparent to those of ordinary skill in the art upon reading the following detailed description with reference to the accompanying drawings.

1 is a block diagram of a display device including an image conversion mechanism according to an embodiment of the present invention,
Fig. 2 shows an example of the display interface of the first device described in Fig. 1,
3 shows an example of a block diagram of a display device,
4 is a control flowchart of the display device,
5 is a diagram of an object detection module,
6 is a diagram of an object adjustment module,
7 shows a part of the image depth map of the current input image,
Figure 8 is a drawing of an object depth module,
Figure 9 is a drawing of a local depth module,
10 shows an example of a current input image,
11 shows an example of a row sum module,
12 shows an example of a binary conversion module,
Figure 13 shows an example of a maximum filter module,
14 shows an example of a row average depth module,
15 shows an example of a row depth group module,
Figure 16 is an example of a group maximum module,
Figure 17 shows an example of a uniform depth module, and
18 is a flowchart of a method of operating a display device according to a further example of the present invention,

The following examples are provided to enable those of ordinary skill in the art to make and use the invention. It is to be understood that other embodiments will be apparent on the basis of the present disclosure, and that the system, method and mechanical changes are not to be regarded as a departure from the scope of the present invention.

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It should be apparent, however, that the present invention may be practiced without these details. In order to prevent the present invention from becoming unclear, some well-known circuits, system configurations, and processes are not disclosed in detail.

Half of the drawings illustrating embodiments of the system are organized in charts and are not drawn to scale, in particular some of the dimensions are for clarity of presentation and are exaggerated in the drawings. Similarly, although the drawings for easy understanding are drawn in a generally similar direction, the description of the drawings is mostly arbitrary. Generally, an invention can operate in any direction. Embodiments of the present invention will be described with reference to the first and second embodiments for the sake of convenience, but they are not intended to limit the present invention.

Those of ordinary skill in the art will appreciate that the manner in which image information is displayed is not of significant importance in some embodiments of the present invention. For example, in some embodiments, the image information is in the (X, Y) format, where X and Y serve as two coordinates defining the location of the pixels in the image.

In another alternative embodiment, the three dimensional image information is represented in the (X, Y, Z) format containing relevant information about the color of the pixel. In a further embodiment of the present invention, the three-dimensional image information also includes content regarding brightness or brightness.

Hereinafter, the term " image " may include a two-dimensional image, a three-dimensional image, a video frame, a computer file representation, an image received from a camera, a video frame, or a combination thereof. For example, the image may be a machine-readable digital file, a physical photograph, a digital photograph, an active picture frame, a video frame, an x-ray image, a scan image, or a combination thereof. For example, the hardware may also be implemented as a circuit, a processor, a computer, an integrated circuit, an integrated circuit cores, a pressure sensor, an inertial sensor, a micro-electro mechanical system (MEMS) a passive device, or a combination thereof.

In the following, " module " may include software, hardware, or a combination thereof. For example, the software may include machine code, firmware, embedded code, application software. The hardware may also be a circuit, a processor, a computer, an integrated circuit, an integrated circuit core, a pressure sensor, an inertial sensor, a microelectromechanical system, a passive device, or a combination thereof.

Figure 1 shows a display system 100 with an image conversion mechanism according to an embodiment of the present invention. The display system 100 includes a first device 102 that is a client or server connected to a second device 106 that is a client or server. The first device 102 is connected to the second device 106 via a wireless or wired network path 104.

For example, the first device 102 may be any of a variety of display devices such as a cell phone, a personal digital assistant (PDA), a notebook computer, a liquid crystal display (LCD) device or other multifunctional display or entertainment device. The first device 102 may be directly or indirectly coupled to the communication path 104 to communicate with the second device 106, or may be implemented as a stand-alone device.

For the sake of explanation, it is assumed that the display system 100 includes the first device 102 which is a display device, but the first device 102 may be another kind of device. For example, the first device 102 may be a video output device or a multi-media presentation. The multimedia presentation may comprise a sequence of sounds, a streaming image or a video feed, or a combination thereof. For example, the first device 102 may be a high-definition TV (HDTV), a 3D TV, a computer monitor, a personal digital assistant, a cellular phone, or a multi-media set.

The second device 106 may be any type of centralized or distributed computing device or one of video transmission devices. For example, the second device 106 may be a multimedia computer, a notebook computer, a desktop computer, a video game machine, grid-computing resources, virtualized computer resources, cloud computing resources, routers, A recording device such as a peer-to-peer distributed computing devices, a media player, a digital video disk (DVD) player, a 3D DVD player, a camera or a video camera, or a combination thereof. As another example, the second device 106 may be a TV receiver, a cable box, a satellite broadcast receiver, or a Web-enabled device such as a TiVo (TM) or Slingbox (TM) .

The second devices 106 may be located in the same space, distributed in another space, or located in an electronic communication network and distributed in different geographical areas. The second device 106 may have means for communicating with the communication path 104 to communicate with the first device 102.

For purposes of discussion, it is assumed that the display system 100 includes a second device 106 that is a computing device, but the second device 106 may be another type of device. Also for purposes of explanation, in the display system 100, the second device 106 and the first device 102 are depicted at the end of the communication path 104, but the display system 100 is configured to communicate between the first device 102, the second device 106, Lt; / RTI > For example, the first device 102, the second device 106, or a combination thereof may serve as part of the communication path 104.

The communication path 104 may be a variety of networks. For example, communication path 104 may be wireless, wired, optical, ultrasonic, or a combination thereof. (WiMAX), satellite communication, mobile communication, Bluetooth, the International Infrared Data Association standard (IrDA), wireless fidelity (WiFi) and WiMAX Examples of communication. The Ethernet, digital subscriber line (DSL), fiber to the home (FTTH) and plain old telephone service (POTS) .

Further, the communication path 104 may span multiple network topologies and distances. For example, the communication path 104 may be a direct connection, a personal area network (PAN), a local area network (LAN), a metropolitan area network (MAN), a wide area network : WAN) or a combination thereof.

FIG. 2 illustrates an example of a display interface 210 of the first device 102 of FIG. Display interface 210 is a physical device for video output or multi-media presentation. For example, the display interface may be a screen including a liquid crystal display (LCD) panel, a plasma screen, or a projection screen.

The display interface 210 displays the processed image 212. The processed image 212 is defined as an image that includes a large number of pixels processed to have a perception depth 211 when displayed on the display interface 210.

The perception depth is defined as the object appears relatively close to or farther from the background in another object or image when viewed from a distance from the display interface 210. For example, in the three-dimensional Cartesian coordinate system with the lower left corner of the image as the origin, the X-axis 213 is a reference for measuring the width, the Y-axis 215 is a reference for measuring the depth, and the Z- . An object having a large Y coordinate value may have a larger value of the perception depth 211 than an object having a small Y coordinate value. An object with a larger value of the crust depth 211 will appear closer to an object with a smaller crust depth 211 value.

In another example, the user can identify or sense the perceived depth 211 by noting that the processed image 212 object appears outside or on the plane of the display interface 210. Similarly, the user may perceive the depth by noting that the objects in the processed image appear deeper inside or behind the plane of the display interface 210. For example, the processed image 212 may blur pixels as a way of representing distances. The intensity of the brightness and color is also reduced to provide a perceived depth 211.

However, in yet another example, the perception depth 211 may be expressed as spacing between rows. A part of the narrow-line-processed image 212 has a lower value of the perception depth 211 and a higher perception depth 211 of the object with a large inter-line space. As a specific example, the upper portion of the processed image 212 with a narrow inter-line portion has a low value of the perception depth 211 and the inter-line portion having a high value of the perception depth 211 appears farther away from the lower portion of the processed image 212.

Processing video 212 may include processing object 214. The processing object 214 refers to an object having an equal perception depth 211, and these objects are located at relatively fixed positions. An object with an even crust depth 211 means that all parts of the object have a crust depth 211 of the same value. An object having a relatively fixed position means an object having almost no movement along X-axis 213 or Z-axis 217 from one image to another in a stream or sequence of images.

The processing object 214 may have a perception depth 211 greater than a portion of the processing image 212 adjacent to the processing object 214. [ For example, the processing object 214 may float above or above the processing image 212.

Examples of processing objects 214 include animations or non-animated logos such as subtitles, scoreboards for sporting events, logo emblems for broadcast stations, animations that can appear at relatively fixed locations in processed video 212, a scrolling banner such as a scrolling text or a text field. As another example, pixels within the region 214 of the processing object 214 may be blurred and rendered flexible to provide texture and to indicate that all of the processing objects 214 in the processing image 212 are distinct to feed the processing depth 214 to the processing object 214 .

An example of a block diagram of the display system 100 is shown in FIG. Display system 100 may include a first device 102, a communication path 104, and a second device 106. The first device 102 may send information in the first device transfer 308 to the second device 106 via the communication path 104. The second device 106 may send information in the second device transmission 310 to the first device 102 via the communication path 104.

For purposes of illustration, display system 100 is shown as including a first device 102 that is a client device, but display system 100 may have a different type of first device 102. For example, the first device 102 may be a server including a display interface.

Also for purposes of explanation, the display system 100 may have a second device 106 as a server, but the display system 100 may have a second type of device 106 of another type. For example, the second device may be a client device.

To simplify description of embodiments of the present invention, the first device 102 will be described as a client device and the second device 106 will be described as a server device. The present invention does not limit the type of apparatus. This description is only an embodiment of the present invention.

The first device 102 may include a first control unit 312, a first storage unit 314, a first communication unit 316, and a first user interface 318. The first control unit 312 may include a first control interface 322. The first control unit 312 may execute the first software 326 to provide information of the display system 100.

The first control unit 312 may be implemented in various ways. For example, the first controller 312 may be a processor, an application specific integrated circuit (ASIC), an embedded processor, a microprocessor, hardware control logic, a hardware finite state machine (FSM) a digital signal processor (DSP), or a combination thereof. The first control interface 322 may be used for connection between the first control unit 312 and other functional units in the first device 102. In addition, the first control interface 322 may also be used for connections from the outside to the first device 102.

The first control interface 322 may receive information from other functional units or external sources or may transmit information to other functional units or external destinations. External sources and external destinations are sources and destinations external to the first device 102.

The first control interface 322 may be implemented in other manners and may include other implementations depending on the function or external department associated with the first control interface 322. [ For example, the first control interface 322 may include a pressure sensor, an inertial sensor, a micro-electro mechanical system (MEMS), optical circuitry, waveguides, wireless circuitry, wireline circuitry, or a combination thereof.

The first storage unit 314 may store the first software 326. In addition, the first storage unit 314 may store related information such as data of a received image, data of previously output image, sound file, or a combination thereof.

The first storage unit 314 may be a volatile memory, a non-volatile memory, an internal memory, an external memory, or a combination thereof. For example, the first storage unit 314 may be a non-volatile storage device such as a non-volatile RAM, a flash memory, a disk storage, or a volatile storage device such as a static random access memory (SPAM).

The first storage unit 314 may include a first storage interface 324. The first storage interface 324 may be used for connection between different functionalities of the first device 102. The first storage interface 324 may also be used for external connections to the first device 102.

The first storage interface 324 may receive information from other functionalities or from external sources or may transmit information to other functionalities or external destinations. External sources and external destinations refer to sources and destinations external to the first device 102.

The first storage interface 324 may include other implementations depending on the type of external function or other function that interacts with the first storage 314. The first storage interface 324 may be implemented in a similar manner and in a manner similar to the execution of the first control interface 322.

The first communication unit 316 allows external connection to or from the first device 102. For example, the first communication unit 316 may permit the first device 102 to be connected to the second device 106 of FIG. 1 attached thereto, an attached device such as a peripheral device and a computer desktop, and a communication path 104.

Also, the first communication unit 316 serves as a communication center unit, and the first device 102 functions as a part of the communication path 104 and is not limited to the terminal end or the terminal unit of the communication path 104. The first communication unit 316 may include active and passive components, such as microelectronics or antennas, for interaction with the communication path 104.

The first communication unit 316 may include a first communication interface 328. The first communication interface 328 can be used for communication between the first communication unit 316 and other functional units in the first device 102. The first communication interface 328 may receive information from or send information to other functions.

The first communication interface 328 may include various implementations depending on the functional units connected to the first communication unit. The first communication interface 328 may be implemented in a similar manner and in a manner similar to that of the first control interface 322.

A first user interface 318 allows a user (not shown) to connect and interact with the first device 102. The first user interface 318 may include an input device and an output device. Examples of input devices of the first user interface 318 include a keypad for providing data and communication inputs, a touch pad, a soft-key, a keyboard, a microphone, an infrared sensor for receiving a long distance signal, or a combination thereof .

The first user interface 318 may include a first display interface 330. The first display 330 may include a display, a projector, a video screen, a speaker, or a combination thereof.

The first control unit 312 may execute a first user interface 318 to display information generated by the display system 100. [ In addition, the first control unit 312 may execute the first software 326 for other functions of the display system 100. Further, the first control unit 312 may execute the first software 326 to interact with the communication path 104 via the first communication unit 316. [

The second device 106 may be optimized for running the invention with the first device 102 in various device examples. The second device 106 may provide additional or higher performance processing capability as compared to the first device 102. [ The second device 106 may include a second control unit 334, a second communication unit 336, and a second user interface 338.

A second user interface 338 allows a user (not shown) to connect and interact with the second device 106. The second user interface 338 may include an input device and an output device. Examples of the input device of the second user interface 338 may include a keypad, a touch pad, a soft key, a keyboard, a microphone, or a combination thereof for providing data and communication inputs. An example of an output device of the second user interface 338 may include a second display interface 340. The second display interface 340 may include a display, a projector, a video screen, a speaker, or a combination thereof.

The second control unit 334 may execute the second software 342 to provide information of the second device 106 of the display system 100. [ The second software 342 may operate in conjunction with the first software. The second controller 334 may provide additional performance as compared to the first controller 312. [

The second control unit 334 may operate the second user interface 338 to display information. The second control unit 334 may also execute the second software 342 for other functions of the display system 100, including operating the second communication unit 336 to communicate with the first device 102 via the communication path 104.

The second control unit 334 may be implemented in various ways. For example, the second controller 334 may be a processor, an embedded processor, a microprocessor, a hardware control arithmetic logic, a hardware legacy state machine (FSM), a digital signal processor (DSP), or a combination thereof.

The second control unit 334 may include a second control interface 344. The second control interface 344 may be used for communication between the second controller 334 and other functions in the second device 106. The second control interface 344 may also be used for communication with the outside of the second device 106.

The second control interface 344 may receive information from other functionalities or from external sources or may transmit information to other functionalities or external destinations. External sources and external destinations refer to sources and destinations external to the second device 106.

The second control interface 344 may be implemented in a variety of ways and may include other implementations depending on other functions or external departments connected to the second control interface 344. [ For example, the second control interface 344 may be implemented with a pressure sensor, an inertial sensor, a microelectromechanical system (MEMS), an optical circuit, waveguides, a wireless circuit, a wire circuit, or a combination thereof.

The second storage unit 346 may store the second software 342. In addition, the second storage unit 346 may store related information such as received image data, previously output image data, sound files, or a combination thereof. The second storage unit 346 may be large enough to provide additional storage space for supplementing the first storage unit 314.

Although the second storage 346 is shown as a single element for purposes of illustration, the second storage 346 may be in the form of a distributed storage element. Also, for purposes of discussion, the display system 100 is depicted as including a second storage unit 346, which is a single layer storage system, but the display system 100 may have a second storage unit 346 in a different configuration scheme. For example, the second storage unit 346 may comprise a different storage technology that forms a hierarchical memory system including different levels of caching, main memory, rotating storage, or off-line storage.

The second storage unit 346 may be a volatile memory, a non-volatile memory, an internal memory, an external memory, or a combination thereof. For example, the second storage unit 346 may be a non-volatile storage device such as a non-volatile random access memory (NVRAM), a flash memory, and a disk storage device, or a volatile storage device such as an SRAM.

The second storage unit 346 may include a second storage interface 348. The second storage interface 348 may be used for communication between other functionalities within the second device 106. The second storage interface 348 may also be used for an external connection to the second device 106.

The second storage interface 348 may receive information from other functionalities or external sources or may transmit information to other functionalities or external destinations. External sources and external destinations refer to sources and destinations external to the second device 106.

The second storage interface 348 may include other implementations depending on the other functions or external functions associated with the second receive store 346. The second storage interface 348 may be implemented in a similar manner or in a manner similar to that of the second control interface 344.

And the second communication unit 336 may enable an external connection to or from the second device 106. For example, the second communication unit 336 may allow the second device 106 to communicate with the first device 102 via the communication path 104.

Also, the second communication unit 336 functions as a communication center, and the second device 106 can function as a part of the communication path 104 and can be not limited to the terminal end or the terminal unit of the communication path 104. The second communication portion 336 may include active and passive components such as microelectronics or antennas for interaction with the communication path 104.

The second communication unit 336 may include a second communication interface 350. The second communication interface 350 can be used for communication between the second communication unit 336 and other functional units in the second device 106. The second communication interface 350 may receive information from other functional units or transmit information to other functional units.

The second communication interface 350 may include another execution method depending on the function unit connected to the second communication unit 336. The second communication interface 350 may be implemented in a similar manner and in a manner similar to that of the second communication interface 344.

The first communication unit 316 is connected to the communication path 104 and transmits information to the second device 106 through the first device transmission 308. The second device 106 may receive information from the first device transfer 308 of the communication path 104 at the second communication unit 336. [

The second communication unit 336 is connected to the communication path 104 and transmits information to the first device 102 through the second device transmission 310. The first device 102 may receive information from the second device transmission 310 of the communication path 104 at the first communication unit 316. [ The display system 100 may be implemented by a first control unit 312, a second control unit 334, or a combination thereof.

For purposes of illustration, the second device 106 is shown as a compartment that includes a second user interface 338, a second storage 346, a second control 334, and a second communication unit, but the second device 106 may have other types of compartments . For example, the second software 342 may be differently partitioned so that some or all of its functions may reside in the second control unit 334 and the second communication unit 336. Also, the second device 106 may include other functionalities not described in FIG. 3 for clarity of content.

The functional units in the first device 102 can operate independently of the other functional units. The first device 102 may operate independently of the second device 106 and the communication path 104.

The functional units in the second device 106 can operate independently of the other functional units. The second device 106 may operate independently of the first device 102 and the communication path 104.

For illustrative purposes, the display system 100 has been described in terms of the operation of the first device 102 and the second device 102. It is understood that the first device 102 and the second device 106 can operate all the modules and functions of the display system 100.

Fig. 4 shows a control flow of the display system 100. Fig. Display system 100 may include video sequence 402. The video sequence 402 is an image of a series to be displayed on the device. For example, the image sequence 402 may include a first image to be displayed in a first time period, a second image to be displayed in a second time period, a third image to be displayed in a third time period ..., and the like. The first period, the second period, and the third period are defined as different times or different time slots and are different periods. The video sequence 402 may be processed to be displayed on the first device 102.

The video sequence 402 may include a previous input image 404, a current input image 406, and a next input image 408. The current input image 406 is defined as an image of the processed image sequence 402 to be displayed. The previous input image 404 is an image processed to be displayed immediately before the current input image 406. The next input image 408 is the image to be processed to be displayed immediately after the current input image 406.

For illustrative purposes, the display system 100 includes an image sequence 402 that includes a previous input image 404, a current input image 406, and a next input image 408, although the number of images included in the image sequence 402 may be different. For example, the video sequence 402 may include one or more images prior to the previous input image 404, one or more images after the next input image 408, or a combination thereof. As another example, the video sequence 402 may include only images prior to the previous input image 404 or images subsequent to the next input image 408.

The previous input image 404, the current input image 406, and the next input image 408 may include the previous input pixel 410, the current input pixels 412, and the next input pixel 414, respectively. The previous input pixel 410, the current input pixels 412, and the next input pixel 414 are individual pixels used to generate the previous input image 404, the current input image 406, and the next input image 408, respectively.

The display system 100 may include an image depth map 416. [ The video depth map 416 is a map indicating depth information for each pixel in one image. The image depth map 416 may be a map of image pixel depths 418 for each of the current input pixels 412. The image pixel depth 418 is a measure of the perception distance or perception depth of the image. For example, if the value of the image pixel depth 418 of the first pixel of the current input pixels 412 is greater than that of the second pixel of the current input pixels 412, the first pixel of the current input pixels 412 is the current input pixels 412 Lt; RTI ID = 0.0 > of the second < / RTI >

Display system 100 may include a pixel motion module 420. The pixel motion module 420 tracks the pixels across the image sequences and calculates the instantaneous motion of the pixels. Instantaneous motion refers to a change in the position or position of a specific pixel from one sequence to another in a video sequence or series. The pixel motion module 420 may receive the video sequence 402. The pixel motion module 420 is selectable.

The pixel motion module 420 may calculate a motion vector 422. The motion vector 422 is information indicating an instantaneous operation of the pixel. The motion vector 422 is a measure of the instantaneous motion of the current input pixels 412. For example, the motion vector 422 is shown in FIG. 2 X axis 213 and FIG. 2 along the X axis 217, or as a vector representing the magnitude and direction of the instantaneous motion. The calculation of the motion vector 422 is selectable.

In another example, the pixel motion module 420 identifies a corresponding one of the previous input pixels 410 in the previous input image 404, a corresponding one of the next input pixels in the next input image 408, or a combination thereof to generate the image sequence 402 The current input pixels 412 can be tracked.

The pixel motion module 420 may calculate the motion vector 422 for the current input pixels in various other ways. For example, the pixel motion module 420 may use block matching algorithms, phase correlation methods, and recursive algorithms. In another example, the pixel motion module 420 may be used in various ways such as a mean squared error (MSE), a sum of absolute differences (SAD), or mean absolute differences (MAD) You can see the pixel motion.

Display system 100 may include object detection module 424. The object detection module 424 detects the object in the video and detects the object having the instantaneous motion 426 in the specific area. Instantaneous motion 426 in a particular region refers to motion that occurs within an image or within a specified region or region. Each of these functions will be described in detail in the following description. The object detection module 424 may receive the video sequence 402. Optionally, the object detection module 424 may receive the motion vector 422.

The object detection module 424 may identify the input object pixels 428. The input object pixels 428 indicate a pixel representing an object in a video or an object in a video of a series. The object object pixels 428 may represent an input object 430 within the current input image 406, the previous input image 404, the next input image 408, or a combination thereof.

The input object 430 refers to an object at a relatively fixed position in a video stream or sequence. An object with a relatively fixed position is an object that moves little or no along the X axis 213 or Z axis 217 in the video stream or sequence. For example, the input object 430 may include a series of or a series of subtitle characters, a scoreboard of a sports game, a TV broadcast station logo, or the like, appearing at a relatively fixed position within the current input image 406, the previous input image 404, the next input image 408, A logo created by animation or non-animation, such as animation, or a moving banner or text field, such as a scroll character for an advertisement or the latest news. The input object 430 may be an object represented through a video stream or a sequence of images in a sequence.

Display system 100 may include an object adjustment module 432. The object adjustment module 432 serves to reduce or prevent erroneous detection of all or a part of the input object pixels 428 of the current input image 406. Each of these functions will be described in detail later. The object adjustment module 432 is selectable.

Display system 100 may include an object depth module 434. The object depth module 434 calculates the depth weight and calculates the depth of the object in the image with the depth weight. Each of these functions will be described in detail later.

The object depth module 434 may generate an object pixel depth 436. [ The object pixel depth 436 is a measure of the perception distance or perception depth 211 of the input object pixels 428.

Display system 100 may include an image transformation module 438. The image conversion module 438 serves to generate an image having a perception depth 211. The image conversion module 438 can generate the processed image 212 from the current input image 406 by applying the image depth map 416 to the current input image 406 for display on the first device 102.

The image transformation module 438 may apply the image pixel depth 418 to the current input pixels 412 to generate the current processing pixel 442. The current processing pixel 442 refers to a pixel processed to have a perception depth 211. The processed image 212 may be generated from the current processing pixel 442.

The image transformation module 438 may apply the object pixel depth 436 to the input object pixels 428 to generate the processing object pixel 444. [ The processing object pixel 444 refers to a pixel representing an in-video object having a perception depth 211. Processing object 214 may be generated from processing object pixel 444.

Display system 100 may include an image display module 440. The image display module serves to display an image on the device. The image display module 440 may display the processed image 212 on the first device 102.

The display system 100 may be executed on the first device or on the second device 106 of FIG. 1, or alternatively, between the first device 102 and the second device 106. For example, the second device 106 may transmit the video sequence 402 including the current input video 406, the previous input video 404, the next input video 408, or a combination thereof, over the communication path 104 of FIG. 3 through the second device transmission 310 of FIG. 3 have. The first device 102 may receive the video sequence 402 using the first communication unit of FIG. The first communication interface 328 of FIG. 3 transmits the video stream to the first storage unit of FIG. 3, the first control unit of FIG. 3, or all of them.

3 may execute a pixel motion module 420, an object detection module 424, an object adjustment module 432, an object depth module 434, an image conversion module 438, and an image display module 440.

The first control interface 332 or the first storage interface 324 may transmit the processed image 212 to the first display interface 330 of FIG. The first display interface 330 can receive and display the processed image 212.

Display system 100 illustrates module functions or instructions by way of example. Modules can be partitioned differently. For example, the second software 342 may include a pixel motion module 420 or an object adjustment module 432. Each module can operate independently of the other modules.

Also, data generated in one module can be used by other modules even if they are not directly connected to each other. For example, the object depth module 434 may receive the video sequence 402.

5 is a diagram of the object detection module 424. The object detection module 424 may identify the input object pixels at the current input pixels 412 using the object identification module 502. The object identification module 502 serves to identify pixels representing objects in a video based on differences in pixel characteristics. The object identification module 502 may identify the input object pixels 428 in the current input pixels 412 based on differences in the characteristics of the pixels.

The properties of a pixel refer to the properties or characteristics of a particular pixel. For example, the characteristics of the pixel may include the motion vector 422, the color of the current input pixels 412, the brightness of the current input pixels 412, or other properties or characteristics of the current input pixels 412.

The object identification module 502 can identify input object pixels based on a difference combination on the pixel characteristics of the current input pixels 412. [ For example, the object detection module 424 may determine whether the motion vector 422, the color, the current input pixels 412 based on the difference in luminance between adjacent pixels or between groups of the current input pixels 412, ) Detection to identify the input object pixels 428.

For example, if the motion vector 422 of the current input pixels 412 is lower than the motion vector 422 of one pixel adjacent to the current input pixels 412, the object detection module 424 detects one of the current input pixels 412 as the input object pixels 428 . In another specific example, if a strong or sharp edge is detected in adjacent pixels of the current input pixels 412 or adjacent segments of the current input pixels 412, the object detection module may detect current input pixels 412 may be identified by input object pixels 428. [

The object detection module 424 may include an object motion module 504. The object motion module 504 serves to identify pixels having a localized temporal motion 426 of a particular region. For example, the object motion module 504 may determine the motion vector 422 of the input object pixels 428 identified by input object pixels 428 based on color, brightness, contour detection, and other features. If the motion vector 422 of the input object pixels 428 is small, continuous, non-directional, confined to a limited space of the same size over a plurality of images, or a combination thereof, then the object motion module 504 generates input object pixels 428 < / RTI > includes instantaneous motion 426 in that particular area. Examples of input objects 430 including instantaneous motion 426 in a particular region may include objects such as scrolling text, animated logos, symbols, or animated advertisements.

The display system 100 may be implemented in a manner that is divided into the first device 102 of FIG. 1, the second device 106 of FIG. 1, or the first device 102 and the second device 106. For example, the second storage unit of FIG. 3 may store the object identification module 502 and the first control unit 312 may execute the object identification module 502.

The display system 100 exemplifies the functions and commands of the module. Modules can be partitioned differently. For example, the first control unit 312 may include an object identification module 502, and the first storage unit 314 may include an object motion module 504. Each module can operate independently of the other modules.

In addition, data generated in one module can be used by other modules even if they are not directly connected to each other. For example, the object motion module 504 may receive the video sequence 402.

6 is a diagram showing an object adjustment module 432. Fig. The object adjustment module 432 can prevent false detection of the input object pixels 428 using the image adjustment module 604 and the local adjustment module 606. [ Misdetection refers to inadequate or incorrect detection or identification of pixels within a region. For example, erroneous detection of the input object pixel 428 may occur when there is little or no motion in the scene represented by the current input image 406, such as a still image or a still shot.

The image adjustment module 604 prevents erroneous detection of a pixel representing an object in the entire image based on the entire pixel motion and information of a scene represented by an image. For example, the image adjustment module 604 may compare an image motion average 608 with an image motion threshold 610 to determine if there is sufficient motion within the current input image 406 to correctly identify the input object pixels 428 in the image Can be determined.

The image motion average 608 is a measurement value of the entire pixel motion in one image. For example, the video adjustment module 604 may calculate a video motion average 608 by averaging the magnitude values of the motion vectors 422 of each current input pixels 412 in the current input image 406. In another example, a high value of the image motion average 608 may mean that the current input image 406 represents a scene that includes a high level of motion, such as an action scene of a movie or a sporting event, for example. Conversely, a low value of the video motion average 608 may mean that the current input video 406 includes low-level motion, such as news broadcasts or interviews.

The image motion threshold 610 means a threshold for determining when a single image has sufficient overall pixel motion to accurately identify a pixel representing the object. For example, the image motion threshold 610 may be a predetermined value or a percentage indicating a still scene with little or no motion.

If the image motion average 608 is greater than the image motion threshold 610, then the image adjustment module 604 may determine whether the current input image 406 has enough motion to correctly identify the input object pixels 428. For example, if the image adjustment module 604 determines that the image motion average 608 is greater than the image motion threshold 610, the display system 100 may calculate the object pixel depth 436 of the input object pixels 428 using the object depth module 434. Conversely, if the image motion average 608 is lower than the image motion threshold 610, the display system 100 may skip the calculation of the object pixel depth 436 and use the image pixel depth 418.

The image adjustment module 604 may include a scene adjustment module 612. The scene adjustment module 612 serves to identify a pixel representing an object in the image with little or no motion of the pixel. The scene adjustment module 612 can identify the input object pixels 428 of the current input image 406 based on the scene information 614.

The scene information 614 indicates information describing the type of a scene represented by the current input image 406. The scene information 614 may describe all types of scenes that the current input image 406 represents. For example, the scene information 614 may be a scene of a home shopping network, a direction of a sports broadcast, or a news broadcast.

The scene adjustment module 612 can identify the input object pixels 428 of the current input image 406 including specific scene information 614. [ For example, if the scene information 614 is a scene having an image motion average 608 lower than the image motion threshold value 610 but is a scene connected in common with the input object 430, the scene adjustment module 612 may convert the current input pixels 412 to input object pixels 428 . For example, the scene information 614 commonly associated with the input object 430 may include a scene that commonly includes scenes such as a subtitle, a logo, a boxed phrase or information, for example, a home shopping place or a news broadcast scene.

The local adjustment module 606 prevents erroneous detection of pixels representing an object located in a particular area of the image with little or no pixel motion. For example, the local adjustment module 606 prevents false detection of a pixel representing an object in a zoom scene 616. Zoom scene 616 refers to a scene that is focused on the inside or outside of a part of the image.

The local adjustment module 606 may use the local motion module 618 to identify an area located in a certain area of the current input image 406. The local motion module 618 serves to identify portions of the image using the camera zoom function. For example, a local portion with little or no motion, or a region with little or no motion of the pixel, surrounded by a high-motion region having a motion vector 422 for the current input pixels 412, And can detect a zoom.

When the local motion module 618 detects the zoom scene 616, the local adjustment module 606 prevents the motion from calculating the object pixel depth 436 of the input object pixels 428 located in a portion of the current input image 406 with little or no pixel motion can do.

The display system 100 may be executed on the first device 102 of FIG. 1 or on the second device 106 of FIG. 1, or alternatively, between the first device 102 and the second device 106. For example, the second storage unit 346 of FIG. 3 may store the scene information 614 and the first control unit 312 may execute the scene adjustment module 612.

Display system 100 illustrates module functions and instructions as an example. Modules can be partitioned differently. For example, the first control unit 312 may include an image adjustment module 604, and the first storage unit 314 may include a local adjustment module 606. Each module can operate independently of the other modules.

Also, data generated in one module can be used by other modules even if they are not directly connected. For example, the local adjustment module 606 may receive the video sequence 402.

7 is an example of a part of the video depth map 426 of the current input image 406. FIG. A portion of the image depth map 416 is represented by a 7x9 pixel box. The video depth map 416 may include current input pixels 412, input object pixels 428, and a video pixel depth 418. The current input pixels 412 are drawn with solid line squares. The input object pixels 428 are drawn with a dotted line square. The image pixel depth 418 is represented by a number inside each of the current input pixels 412 and the input object pixels 428, respectively. The underlined numbers represent the image pixel depth 418 of the input object pixels 428. The numbers at the top of the video depth map 416 are column numbers and the left side numbers of the video depth map 416 are row numbers.

For example, the high value of the image pixel depth 418 of the current input pixels 412 and the input object pixels 428 represents a high value of the perception depth 211 of FIG. As a specific example, the current input pixels 412 and input object pixels 428, each having a high value image pixel depth 418, will appear closer to seven rows of pixels having a low image pixel depth 418 value.

8 is a drawing of the object depth module 434. [ The object depth module 434 may include a depth weight module 802. The depth weighting module 802 serves to compute a weighting factor used to calculate the depth of a pixel representing an in-video object. The depth weight module 802 can calculate the depth weight 804.

Depth weight 804 refers to depth candidate 806 and values based on properties and properties of various pixels. The depth weight 804 can be calculated based on the depth candidate 806, which may include a video pixel depth 418, a fixed depth 808, a video maximum depth 810, a temporal maximum depth 812, a row average depth 814, a row maximum depth 816, have. The depth weight 804 may be calculated from each of the input object pixels 428 in the current input image 406.

Fixed Depth 808 refers to the value of the unchanged depth based on the location of the image. For example, the input object 403 having the object pixel depth 436 at the fixed depth 808 has the same perception depth 211 regardless of the position of the current input image 406 or the perception depth 211 of a portion of the current input image 406 adjacent to the input object 430.

The fixed depth 808 may be a predetermined continuous value of the video sequence, and the value may be different between images in the video sequence, or the value of the video sequence may be heuristically determined through an iterative or recursive method Can be judged. As a specific example, the fixed depth 808 may be based on an initial setting such as an 8-bit depth value of 128, for example.

The image maximum depth 810 is a depth measurement based on the maximum depth of the pixels in the image. The image maximum depth 810 may be calculated as a percentage of the current input pixels 412 having the maximum image pixel depth 418 within the current input image 406. [ For example, the value of the perception depth 211 of the input object 430 having the object pixel depth 436 at the image maximum depth 810 will have a lower percentage than the portion of the current input image 406 having the maximum perception depth 211. For example, the image maximum depth 810 may be 95 percent of the image pixel depth 211 of the current input pixels 412 having the largest image pixel depth 418 within the current input image 406.

In one embodiment, the maximum image depth 810 may be illustrated in FIG. In FIG. 7, the image maximum depth 810 may be 9.5, which may be a value corresponding to 95% of the maximum value 10 of the image pixel depth 418 in row 1, line 5-7.

The instantaneous maximum depth 812 refers to the average value of the maximum pixel depth in a plurality of sequence images. For example, the instantaneous maximum depth 812 can be calculated by averaging the maximum image depth 810 of the previous input image 404, the current input image 406, and the next input image 408. The instantaneous maximum depth 812 may be 95 percent of the maximum image depth 810 average of the video sequence 402.

The row average depth 814 refers to the average depth value of one or more rows in the image. The depth weighting module 802 may calculate the row average depth 814 using the row average depth module 820. [ The row average depth module 820 may calculate a row average depth 814 for the current row 822 of the current input pixels 412. The current row 822 refers to the row of current input pixels 412 which averaged by the row average depth module 820. The current row 822 has a row width 824 and the row width refers to the total number of current input pixels 412 that span the width of the current input image 406.

The row average depth 814 may be computed to include an average of the number of rows vertically adjacent to the current input pixels 412 where the rows vertically adjacent to the current input pixels are positioned above or below the current row 811 of the current input pixels 412 Located. The number of rows included in the calculation of the row average depth 814 may be determined by a group factor 826, where the group factor refers to the number of vertically adjacent rows included in the calculation of the row average depth 814. The group factor 826 may be a constant number or may vary in value based on the detection of row input object pixels 428 adjacent to the current row 822.

The row average depth module 820 can calculate the row average depth 814 according to the following equation (1).

And Depth _{row _ average} row average depth 814 In Equation 1, width is the line width (824), j is the current row (822), i is the i th pixel in the current line (822), K is Is the group factor 826 and Depth _background ( ii , jj ) is the image pixel depth corresponding to the current row 822 j and position i along the row width 824. For example, the perceptual depth 211 of the input object 430 having a row average depth 814 as the object pixel depth 436 may be greater than the perceptual depth 211 of the input image 430 at the current input pixels 412, And the crustal depth (211) for the crust.

An example of the row average depth module 820 may be shown in FIG. The depth weighting module 802 may calculate a row average depth 814 for one row that includes input object pixels 428. For example, the row average depth 814 of the second row may be 8.7, the row average depth 814 of the third row may be 7.2, and the row average depth 814 of the fifth row may be 3.6 days , And the row average depth 814 of the sixth row may be 2.5. Similarly, the depth-weighting module 802 may calculate row-mean depth for adjacent rows with input object pixels 428. [ For example, the row average depth 814 of the second and third rows may be 8 and the row average depth 814 of the fifth and sixth rows may be 3.1.

The row maximum depth 816 includes the image pixel depth 418 for a row of the current input pixels 412 or for a set of rows vertically adjacent to the current input pixels 412 with the input object pixels 428, Of the maximum value. The depth weighting module 802 can identify the image pixel depth 418 for each of the current input pixels 412 and the input object pixels 428 and adjusts the row maximum depth 816 to the image pixel depth It can be determined that the maximum value of the reference value 418 is zero. Similarly, for the number of vertically adjacent rows, each with input object pixels 428, the depth weight module 802 determines whether the current input pixels 412 and the input object pixels 428, The depth 418 can be identified and the row maximum depth 816 can be determined as the maximum value of the image pixel depth 418. [

An example of the row maximum depth 816 can be shown in FIG. The depth weight module 802 may calculate a row maximum depth 816 for a single row with input object pixels 428. [ For example, the row maximum depth 816 of the second row may be 9, the row maximum depth 816 of the third row may be 8, the row maximum depth 816 of the fifth row may be 4, The sixth row maximum depth 816 may be three. Similarly, the depth weight module 802 may calculate the row maximum depth for adjacent rows with input object pixels 428. [ For example, the row maximum depth 816 of the second and third rows may be nine and the row maximum depth 816 of the fifth and sixth rows may be four.

The local depth 818 represents the maximum pixel depth between a set of pixel rows. The depth-weighting module 802 may calculate the local depth 818 using the local depth module 828. The function of the local depth module 828 will be described in detail later.

The depth weight module 802 can calculate the depth weight 804 as a weighted average value of the depth candidate 806 including the fixed depth 808, the image maximum depth 810, the instantaneous maximum depth 812, the row average depth 814, and the local depth 818. Preferably, 50% to 75% at the weighted depth 804 may be represented by the image pixel depth 418, 0% to 7% may be represented by the row average depth 814, 9% to 25% Depth 810 or instantaneous maximum depth 812, and 9% to 25% can be expressed as local depth 818

The object depth module 434 may optionally include a weight adjustment module 830. The weight adjustment module 830 adjusts the weight of the factors used to calculate the pixel depth based on the object pixel position 832, the object pixel property 834, or a combination thereof.

The object pixel position 832 indicates the position of the input object pixels in the current input image 406. For example, the object pixel position 832 may be a specific coordinate within the current input image 406 or a general portion of the image, such as the bottom, center, or corner of the current input image 406. As another example, the object pixel position 832 may be associated with a common portion of the input object 430, such as the bottom center or upper left corner of the current input image 406.

The object pixel attribute 834 refers to an attribute generally associated with the input object 430, such as a particular color or level of brightness. For example, an object pixel attribute 834 generally associated with an input object 430 representing a caption phrase is a color such as white or light yellow or a luminance level brighter than the adjacent portion of the current input image 406. In another example, the object pixel attribute 834 may be the same as the boarder of the scoreboard.

The weight adjustment module 830 may adjust the depth weight 804 to raise or lower the perception depth 211 of the input object pixels 428 based on the object pixel position 832 and the object pixel attribute 834. [ For example, if the weight adjustment module 830 perceives that the object pixel position 832 and the object pixel property 834 are generally related to the input object 430, the weight adjustment module 830 adjusts the depth weight 840 to increase the perception depth 211 to obtain the current object image 406 It is possible to distinguish the input object 430 from the input object 430. For example, the weight adjustment module 830 adjusts the depth weight 804 so that the depth weight 804 is 50% the video pixel depth 418, 0% the row average depth 814, 25% the video maximum depth 810, and 25% So that the input object 430 can be distinguished from the current input image 406.

In another example, if the weight adjustment module 830 perceives that the object pixel position 832 and the object pixel property 834 are not generally associated with the input object 430, the weight adjustment module 830 adjusts the depth weight 804 to adjust the perception depth 211 Or may blend the input object 430 into the current input image 406. For example, the object pixel position 832, which is not generally associated with the input object 430, may be at the center of the current input image 406. In another specific example, the weight adjustment module 830 adjusts the depth weight 804 such that the image pixel depth 418 is 75%, the row average depth 814 is 7%, the maximum image depth 810 is 9%, and the local depth 818 is 9% The input object 430 may be mixed with the current input image 406. [

As another example, the weight adjustment module 830 may adjust the depth weight 804 for the input object pixels 428 with the local moment motion 426. For example, the depth weight 804 may be adjusted such that the weight of the image pixel depth 418 is minimized and the weight of the image maximum depth 810 is maximized. In another specific example, depth weight 804 may be 0% video pixel depth 418, 75% video maximum depth 810, and 25% local depth 818.

The object depth module 434 may calculate the object pixel depth 436 by multiplying the image pixel depth 418 corresponding to the input object pixels 428 and the depth weight 804 for the input object pixels 428.

It has been found that the present invention provides a display system 100 with reduced disparity between the processing object 214 and the processing image 212. The depth weight 804, which is a value obtained by calculating the weighted average value of the depth candidates 806, may be used to calculate the object pixel depth 436 and may be applied to the input object 430 to generate a processing object 214 with a reduced depth difference from the processing image 212 .

Processed video 212 The physical transformation of the display can cause motion, such as the motion of people reacting to the processed video 212 when playing a game or viewing a 3D video in the real world. The first display interface 330 can display the processed image 212 by manipulating pixels on the first device 102, thereby causing motion in the real world.

The display system 100 may be implemented on the first device 102 of FIG. 1 or on the second device 106 of FIG. 1 or between the first device 102 and the second device 106. For example, the second storage unit 346 of FIG. 3 may store the object pixel position 832 and the first control unit 312 may execute the weight adjustment module 830.

Display system 100 describes, for example, module functions or commands. Modules can be partitioned differently. For example, the first control unit 312 may include a depth weight module 802, a local depth module 828, and a row average depth module 820, and the first storage unit 314 may include a weight adjustment module 830. Each module can operate independently of the other modules.

In addition, data generated in one module can be used by other modules even if they are not directly connected to each other. For example, the weight adjustment module 830 may receive the image pixel depth 418.

9 is a diagram of the local depth module 828. FIG. The local depth module 828 includes a row sum module 902, a binary conversion module 904, a maximum filter module 906, a row depth group module 908, a group maximum module 910, and a uniform depth module 912 can be used to compute a local depth 818.

An addition module 902 is operative to sum the number of pixels representing an object with respect to each row of image pixels. The conjunction module 902 may calculate an object row sum 914. The object row sum 914 indicates the total number of input object pixels 428 in a particular row of the current input pixels 412. For example, if the row of input object pixels 428 includes five input object pixels 428, the row summing module 902 may calculate the object row sum 914 as five. The combining module 902 may calculate an object row sum 914 for each row of the current input pixels 412 in the current input image 406.

The binary conversion module 904 converts a value representing a pixel count for a specific row of pixels into a binary value. The binary conversion module 904 may convert the object row sum 914 to a binary row value 916 for each row of the current input pixels 406 based on a binary threshold 916. [ Binary row value 916 is a binary measure of whether a row contains pixels representing an object.

The binary threshold 917 is a threshold for determining if there are enough pixels to represent an object in a pixel row. The binary threshold 917 may be determined based on the percentage of the total number of input object pixels 428 located in one row of the current input pixels 412. For example, the binary threshold 917 may be determined to be 4% of the total number of input object pixels 428 in the row of current input pixels 412. As a specific example, if the row of the current input pixels 412 has an object row sum 914 with a value of 480, then the binary threshold 917 may be computed as 19, corresponding to about 4% of 480.

The binary conversion module 904 can compute the binary row value 916 by comparing the object row sum 914 with the binary threshold value 917. For example, if the object row sum 914 for one row of current input pixels 412 is greater than the binary threshold 917, the binary conversion module 904 may set the binary row value 916 for the particular row to one. Conversely, if the object row sum 914 for one row of the current input pixels 412 is less than the binary threshold 917, then the binary conversion module 904 may set the binary row value 916 for that particular row to zero.

The maximum filter module 906 serves to filter and remove pixel rows with a binary value of zero. The maximum filter module 906 may generate a binary row group 918 based on the binary row value 916. [ Binary row group 918 refers to a set of rows vertically adjacent to the current input pixels 412 with a binary row value of one.

The maximum filter module 906 may generate a binary row group 918 by filtering and removing the rows of the current input pixels, where the binary row value 916 is 0, where the rows with a binary row value 916 of 0 are binary row values 916 of 1 Lt; / RTI > is located between narrowly arranged rows of current input pixels.

For example, the maximum filter module 906 can determine if two conditions are met. First, a series of rows vertically adjacent to the current input pixels 412, where the binary row value 916 is 0, is located between the series of rows vertically adjacent to the current input pixels 412, where the binary row value 916 is 1. Second, the number of vertically adjacent rows with binary row value 916 equal to 1 should be greater than the number of vertically adjacent rows with binary row value 916 equal to zero.

The local depth module 828 may utilize the row average depth module 820 to compute a single row depth 920 for each row of the current input pixels 412 in the current input image 406. The single row depth 920 represents the average value of the image pixel depth 418 of all the current input pixels 412 located in the current input image 406 row. The local depth module 828 may compute a single row depth 920 according to Equation (1) with the group factor K being zero.

The local depth module 828 can use the row depth group module 908 to identify a row depth group 922 and a row row value 916 based on a single row depth 920. [ A row depth group 922 refers to a series of successive pixel rows that are contiguous. Successive adjacent pixel rows are those that do not have a gap in the middle. For example, the first, second, and third pixel rows of an image may be consecutively adjacent rows. Conversely, the first, third, and fifth rows can not be consecutive rows because the second and fourth rows form a gap between a series of rows.

The row depth grouping module 908 may identify the row depth group 922 by multiplying the single row depth 920 by the binary row value 916 for each row of the current input pixels 412. [ After the multiplication, successive adjacent rows of the current input pixels 412 having non-zero values can be grouped. For example, if a row of two or more current input pixels 412 has a nonzero value after multiplying a single row depth 920 by a binary row value 916 for each row, the row depth grouping module 908 identifies the row depth group 922 .

The local depth module 828 may use the group max module 910 to determine the group maximum depth 924 of the row depth group 922. [ The group maximum module 910 can determine the group maximum depth 924 for the row depth group 922 by identifying the group maximum depth 914 with the maximum value for the single row depth 920 of the row depth group 922. [ The local depth 818 may be the group maximum depth 924 for the vertically grouped rows of the current input pixels 412.

The local depth module 828 can determine the local depth 818 using the uniform depth module 912. The uniform depth module 912 serves to specify a uniform depth value for all rows of pixels within a group. The uniform depth module 912 may specify a local depth 818 for all current input pixels 412 in each row of the row depth group 922 to a group maximum depth 924. [ For example, the local depth 818 may be the group maximum depth 924 for the vertically grouped rows of the current input pixels 412. The uniform depth module 912 may specify a local depth 818 for each of the row depth groups 922 of the current input image 406.

The display system 100 may be implemented on the first device 102 of FIG. 1 or on the second device 106 of FIG. 1, or between the first device 102 and the second device 106. For example, the second controller 334 of FIG. 3 may execute the summing module 902 and the first controller 312 may execute the binary conversion module 904.

The display system 100 exemplifies the functions or commands of the module. Modules can be partitioned differently. For example, the first controller 312 may include a row summing module 902, a binary transforming module 904, a maximum filter module 906, and the first threading 314 may include a row depth grouping module 908, a grouping maximum module 910, a uniform depth module 912 . Each module can operate independently of the other modules.

Furthermore, data generated in one module can be used by other modules even if they are not directly connected to each other. For example, the group max module 910 may receive the image pixel depth 418.

10 is an illustration of the current input image 406. [ The current input image 406 includes rows with input object pixels 428, which are depicted as dashed lines. The white portions are currently input pixels 412. For example, input object pixels 428 may represent subtitles or other types of characters in the current input image 406.

11 is an example of a row summing module 902. Fig. Arrangement module 902 may calculate an object row sum 914, which represents the total sum of input object pixels 428 in FIG. 10 within each row of current input pixels 412 in FIG. 10, On the chart 1102. FIG. The X-axis of the chart 1402 corresponds to the rows of the current input image 406. The Y axis of the chart 1402 may be a reference to the number of input object pixels 428.

FIG. 12 shows an example of the binary conversion module 904. Binarization module 904 may change the object row sum 914 of FIG. 11 to a binary row value 916 for each row of the current input pixel 412 of FIG. 10, which is shown in the dashed area of the chart 1202. The x-axis of the chart 1202 corresponds to the rows in the current input image 406 of Fig. The y-axis of the chart 1202 is a reference to the binary row value 916.

For example, a chart 1202 depicts a first group 1204 of four rows, one binary row with a value of 1 in the left portion, three binary rows with a value 916 of zero, and four rows. A binary row having a value of 0 in the first group 1204 corresponds to closely located rows of the current input pixel 412 with a binary row having a value of 1. [ Similarly, in a further embodiment chart 1202 depicts two rows with binary rows having a value of one along the right and one row with binary rows having a value of zero.

FIG. 13 illustrates one embodiment of a maximum filter module 906. The maximum filter module 906 may generate a binary row group 918 as depicted in the dotted line portion of chart 1302. The maximum filter module 960 includes a row of current input pixels having a binary row value of zero located between closely located rows of the current input pixel with a binary row having a value of one to produce the binary row group 918 shown in FIG. Can be removed by filtering. The X-axis of the chart 1302 corresponds to the rows in the current input image 406 of FIG. The Y-axis of the chart 1302 becomes a reference to the binary row 916 value.

For example, a chart 1302 depicts a first group 1304 of binary row groups 918 along the left portion. Similarly, in a further embodiment, chart 1302 depicts a second group 1306 of binary row groups 918 on the right hand side.

FIG. 14 shows an example of a row average depth module 820. The row average depth module 820 may calculate a single row depth 920 for each row of the current input pixel in the current input image 406 of FIG. Curve 1404 of chart 1402 represents a single row depth 920 for each row of the current input pixel 412 of the current input image 406. The X-axis of the chart 1402 corresponds to the rows in the current input image 406. [ The Y-axis of the chart 1402 is a reference to a single row depth 920.

FIG. 15 is an illustration of a row depth group module 908. The single row depth 920 of FIG. 14 may be multiplied by the binary row value of FIG. 13 for each row of the current input image 412 of FIG. 10 depicted in chart 1502 to generate a row depth grouping 922. The X-axis of the chart 1502 corresponds to the row of the current input image 406. [ The Y-axis of the chart 1502 is the reference information of the single-row depth 920.

The multiplication after row depth group module 908 groups consecutive adjacent rows of current input pixels 412 having a non-zero value to generate a row depth group 922. For example, a chart 1502 depicts the first group 1504 of row depth groups 922 along the left region. Similarly, in another example, chart 1502 depicts a second group 1506 of row depth groups 922 along the right region of the chart.

FIG. 16 is an illustration of a group max module 910. The group maximum module 910 can determine the group maximum depth 924 of the row depth group 922 by equating the group maximum depth 924 with the maximum value of the single row depth 920 of FIG. 14 in the row depth group 922 depicted in the chart 1602. [ The X-axis of the chart 1602 corresponds to the rows of the current input image 406 of FIG. The Y axis of the chart 1602 is the reference information of the single row depth 920.

For example, a chart 1602 depicts a first group 1604 of row depth groups 922 with a first value 1608 of the group maximum depth 924 along the left portion. Similarly, a further example depicts a second group 1606 of row depth groups 922 with a second value of the group maximum depth 924 along the right part of the chart.

FIG. 17 is an illustration of a uniform depth module 912. The uniform depth module 912 can specify a local depth 818 for all current input pixels in Figure 10 in each row of the row depth group 922 in Figure 16 as a row maximum depth 924 which is depicted on the chart 1702 with portions drawn in dashed lines have. The X-axis of the chart 1702 corresponds to the rows of the current input image 406 of FIG. The Y-axis of the chart 1702 is reference information of the local depth 818.

For example, a chart 1702 places a first group 1704 of rows of the current input pixel 412 with a first value 1708 of local depth 818 to the left of the chart 1702. Similarly, a second group 1706, which is a row of current input pixels 412 with a second value 1710 of local depth 818, is located to the right of the chart 1702.

18 is a flowchart of a method 1800 of operating a display device according to a further embodiment of the present invention. Method 1800 includes receiving a current input image having current input pixels in a block 1802; Identifying an input object having input object pixels based on differences in characteristics of the current input pixels in block 1804; Calculating an object pixel depth of the input object pixels from the image depth map of the current input image based on the weighted average of the depth candidates in block 1806; And generating a processed image having a perceptual depth from a current input image having an image depth map and an object pixel depth, for display on a block 1808 device.

The resulting method, process, apparatus, equipment, article and / or system should be simple, cost effective, uncomplicated, versatile, accurate, sensitive, efficient and prepared, It can be applied to parts that can be applied and utilized. Another important function of the present invention achieves the long-term goal of cost reduction, system simplification, and performance enhancement.

These and other useful aspects of the invention result in advancing the state of the art to the next level.

While the invention has been described in conjunction with the preferred embodiments, it will be understood by those skilled in the art that various modifications and variations are possible in light of the above teachings. Accordingly, it is intended that the present invention cover all modifications and variations that fall within the scope of the appended claims. It is intended that all matter contained in the accompanying drawings shall be interpreted as illustrative and not in a limiting sense the scope of the present invention.

3
102 first device 104 communication path
106 second device 308 first device transfer
310 second device transmission 312 first control unit
314 First storage unit 316 First communication unit
318 First user interface 322 First control interface
324 First storage interface 326 First software
328 First communication interface 330 First display interface
334 Second control interface 336 Second communication section
338 Second user interface 340 Second display interface
342 Second software 348 Second storage interface
350 second communication interface
4
212 processing image 214 processing object
402 Video sequence 404 Previous input video
406 Current input video 408 Next input video
410 previous input pixel 412 current input pixel
414 Next input pixel 416 Image depth map
418 pixel depth 420 pixel motion module
422 motion vector 424 object detection module
426 Local Instant Motion 428 Input Object Pixel
430 input object 432 object adjustment module
434 Object Depth Module 436 Object Pixel Depth
438 image conversion module 440 image display module
442 Current processing pixel 444 Processing object pixel
5
502 object identification module 504 object motion module
6
604 image adjustment module 606 local adjustment module
608 Video Motion Average 610 Video Motion Threshold
612 Scene adjustment module 614 Scene information
616 Zoom Scene 618 Local Motion Module
8
802 Deep Weight Modules 806 Deep Candidates
808 Fixed Depth 810 Image Maximum Depth
812 Moment maximum depth 814 rows Average depth
816 rows maximum depth 818 local depths
820 rows Average depth Module 822 Current row
824 Row Width 826 Group Factor
828 Local depth module 830 Weight adjustment module
832 Object pixel position 834 Object pixel attribute
436 Object pixel depth
8
818 Local Depth 902 Matrix Module
904 Binary conversion module 906 Maximum filter module
908 Row Depth Group Module 910 Group Maximum Module
912 Uniform Depth Module 914 Object Alignment
916 Binary Row Values 917 Binary Thresholds
918 Binary Row Group 920 Single Row Depth
922 Row depth group 924 Group maximum depth
11
902 Input object pixel
12
904 binary value
13
906 binary value
14
820 Depth
16
910 depth
17
912 depth
18
1802 Receive the current input image 1804 Identify the input object
1806 Calculate object depth 1808 Create processed image

Claims (20)

A method of operating a display device,
Receiving a current input image having a current input pixel;
Identifying an input object having an input object pixel based on a difference in characteristics of the current input pixel;
Calculating an object pixel depth for the input object pixel from the image depth map of the current input image using the current row of the current input pixel and the average value of the rows adjacent to the current input pixel; And
Generating a processed image having a perceptual depth from the current input image having the image depth map and the object pixel depth for display on the device. The method according to claim 1,
Wherein identifying the input object comprises identifying an input object having local instantaneous motion. The method according to claim 1,
Wherein the step of calculating the object pixel depth adjusts the average value based on an object pixel position of the input object pixel. The method according to claim 1,
Wherein the step of calculating the object pixel depth adjusts the average value based on an object pixel characteristic of the input object pixel. The method of claim 1, wherein
Further comprising the step of identifying an input object pixel having scene information. A method of operating a display device,
Receiving a current input image having a current input pixel;
Identifying an input object having input object pixels based on differences in characteristics of the current input pixel;
An image depth maximum, an image depth of the current input image based on a weighted average of a row average depth, a local depth, and an image pixel depth calculated using a current row of the current input pixel and an average value of rows adjacent to the current input pixel, Calculating an object pixel depth for an input object pixel from the map; And
And generating an image processed from a current input image having an image depth map and an object pixel depth for display on the device. The method according to claim 6,
Wherein calculating the object pixel depth comprises calculating the object pixel depth based on an instantaneous maximum depth. delete The method according to claim 6,
Wherein calculating the object pixel depth based on the local depth comprises:
And specifying a local depth as a group maximum depth of rows grouped vertically to the current input pixel. The method according to claim 6,
Wherein the step of generating the processed image comprises:
And applying the object pixel depth to the input object pixels to generate a processed object having a perceptual depth. In the display device,
A communication unit for receiving a current input image having a current input pixel;
An object detection module, coupled to the communication unit, for identifying an input object having an input object pixel based on a difference in characteristics of current input pixels;
Based on the image maximum depth, the weighted average of the row average depth, the local depth, and the image pixel depth calculated using the current row of the current input pixel and the average value of the rows adjacent to the current input pixel, An object depth module for calculating an object pixel depth for an input object pixel from an image depth map of the current input image; And
And an image transformation module coupled to the object depth module for generating a processed image having a perception depth to be displayed on the device from a current input image having the image depth map and the object pixel depth, Device. 12. The method of claim 11,
Further comprising an object motion module coupled to the object detection module for identifying an input object having instantaneous motion of a specific region. 12. The method of claim 11,
And a weight adjustment module coupled to the object depth module to adjust weight averages of the depth candidates based on an object pixel position of the input object pixel. 12. The method of claim 11,
And a weight adjustment module coupled to the object depth module and configured to adjust the weighted average based on the object pixel characteristics of the input object pixel. 12. The method of claim 11,
And a scene adjustment module connected to the object detection module and identifying an input object pixel having scene information. delete 12. The method of claim 11,
Wherein the object depth module calculates the object pixel depth based on the instantaneous maximum depth. delete 12. The method of claim 11,
The object depth module
Wherein said local depth is specified as a group maximum depth of vertically grouped rows of current input pixels. 12. The method of claim 11,
Wherein the image conversion module comprises:
And applies the object pixel depth to the input object pixel to generate a processed object having the perceptual depth.

Images