KR102665125B1 - Display apparatus, electronic apparatus, display system comprising display appartus and electronic apparatus, and control methods thereof - Google Patents

Abstract

A display device is disclosed. The present display device includes a display for displaying an image, a communication unit that receives data about a background image behind the display device and location data about a specific area within the background image from an electronic device, and based on the location data about the specific area, Within the received background image, an area having an image pattern that matches the image pattern of a surrounding area adjacent to the specific area is extracted, the corresponding area of the specific area in the background image is determined based on the extracted area, and the specific area corresponds to the determined area. It includes a processor that controls the background image replaced by the area to be displayed on the display.

Description

Display device, electronic device, display system including these, and control method thereof

The present disclosure relates to a display device, an electronic device, a display system including these, and a control method thereof, and more specifically, to a display device capable of displaying a background image behind the display device, an electronic device controlling the display device, and a method for controlling the same. It relates to a display system and a control method thereof.

Recently, functions that provide various user experiences using display devices have been developed. In particular, as shown in FIG. 1, a background image for the background behind the display device 100 is acquired and the display device 100 displays the background image, thereby providing the user with a visual effect such that the display becomes a transparent glass window. can be provided. This background image is obtained by photographing the background behind the display device. Since the location where the display device is mainly installed is the wall, the background image is often an image of the wallpaper behind the display device.

However, if there are unnecessary areas (defect areas) such as outlets, scratches, stains, nail marks, etc. on the wallpaper behind the display device, they are exposed on the display screen and do not look good. Therefore, there is a need to find a way to effectively remove such defect areas from the background image.

The present invention focuses on the fact that most wallpaper behind a display device has a repetitive pattern, and provides a display device capable of removing defect areas from a background image behind a display device, an electronic device, and a display system including the same. and providing a control method thereof.

According to an embodiment of the present invention for achieving the above-described object, a display device is provided, from a display for displaying an image, an electronic device, data about a background image behind the display device and information about a specific area within the background image. Based on the communication unit for receiving location data and the location data for the specific area, extract an area having an image pattern that matches the image pattern of a surrounding area adjacent to the specific area within the received background image, and extract the extracted area. and a processor that determines a corresponding area of the specific area in the background image based on the area and controls the display to display a background image in which the specific area is replaced with the determined corresponding area.

Here, the background image may include a pattern image that is repeated at least twice.

Additionally, in an area that includes the specific area, the remaining area excluding the specific area may be set as the peripheral area.

Additionally, the processor may extract an area having an image pattern most similar to the image pattern of the peripheral area from the received background image using characteristic information about the image pattern of the peripheral area.

Additionally, the processor may calculate brightness data of the peripheral area and correct the brightness of the replaced corresponding area based on the calculated brightness data.

Additionally, the processor may apply a blending effect based on the corrected brightness data of the replaced corresponding area and the brightness data of the peripheral area.

Additionally, the specific area may be at least one area selected by the user of the electronic device.

Meanwhile, a method of controlling a display device according to an embodiment of the present invention includes receiving, from an electronic device, a background image behind the display device and location data for a specific area within the background image, the location of the specific area Based on data, extracting an area within the received background image that has an image pattern that matches an image pattern of a surrounding area adjacent to the specific area, based on the extracted area, the specific area in the background image. It includes determining a corresponding area and displaying a background image in which the specific area is replaced with the determined corresponding area.

Here, the background image may include a pattern image that is repeated at least twice.

Additionally, in the extracting step, in an area including the specific area, the remaining area excluding the specific area may be set as the peripheral area.

Additionally, in the extracting step, an area having an image pattern most similar to the image pattern of the surrounding area may be extracted from the received background image using characteristic information about the image pattern of the surrounding area.

Additionally, the control method may further include calculating brightness data of the peripheral area and correcting brightness of the replaced corresponding area based on the calculated brightness data.

Additionally, the control method may further include applying a blending effect based on the corrected brightness data of the replaced corresponding area and the brightness data of the surrounding area.

Additionally, the specific area may be at least one area selected by the user of the electronic device.

Meanwhile, an electronic device according to an embodiment of the present invention includes a display, a communication unit that performs communication with the display device, and a background image behind the display device on the display, and a specific area in the background image is selected by the user. Then, an area having an image pattern that matches the image pattern of a surrounding area adjacent to the selected specific area is extracted, a corresponding area of the specific area is determined in the background image based on the extracted area, and the specific area is and a processor that controls the communication unit to generate a background image replaced with the determined corresponding area and transmit the generated background image to the display device.

Additionally, the electronic device may further include a camera, and in this case, the processor may control the camera to generate a background image behind the display device.

Additionally, the processor may set the remaining area excluding the specific area as the peripheral area in an area including the specific area.

Additionally, the processor may use characteristic information about the image pattern of the peripheral area to extract an area within the background image that has an image pattern most similar to the image pattern of the peripheral area.

Additionally, the display may include a touch display, and in this case, the processor may select a dragged area within the background image as the specific area through the touch display.

Meanwhile, according to an embodiment of the present invention, a display system including a display device and an electronic device includes an electronic device that transmits data about a background image behind the display device and location data about a specific area within the background image. Based on the location data for a specific area, extract an area with an image pattern that matches the image pattern of a surrounding area adjacent to the specific area within the received background image, and extract the area from the background image based on the extracted area. and a display device that determines a corresponding area of the specific area and displays a background image in which the specific area is replaced with the determined corresponding area on the display.

According to the various embodiments of the present invention described above, it is possible to display an image in which areas that detract from aesthetics, such as nail marks, power outlets, and stains on the wallpaper behind the display device, are removed, thereby creating a more aesthetically pleasing interior effect.

1 and 2 are diagrams illustrating an example of providing an image effect as if the display were a transparent glass window in a display system according to an embodiment of the present disclosure;
3 is a block diagram showing a simplified configuration of a display device according to an embodiment of the present disclosure;
4 is a block diagram showing a specific configuration of a display device according to an embodiment of the present disclosure;
5 and 6 are diagrams showing a method of selecting a specific area and surrounding areas in an electronic device, according to an embodiment of the present disclosure;
7 is a diagram illustrating in detail the surrounding area used to determine the corresponding area of a specific area, according to an embodiment of the present invention;
8 is a diagram showing a determined corresponding area, according to an embodiment of the present invention;
9 is a diagram illustrating a screen in which a specific area is replaced with a determined corresponding area, according to an embodiment of the present invention;
10 is a diagram illustrating a method of applying a blending effect according to an embodiment of the present invention;
11 is a block diagram showing a simplified configuration of an electronic device according to an embodiment of the present invention;
12 is a flowchart illustrating a control method of a display device according to an embodiment of the present invention;
Figure 13 is a flowchart for explaining a control method of an electronic device according to an embodiment of the present invention.

Before explaining the present invention in detail, the description method of the specification and drawings will be explained.

First, the terms used in this specification and claims are general terms selected in consideration of their functions in various embodiments of the present invention. However, these terms may vary depending on the intention of technicians working in the field, legal or technical interpretation, and the emergence of new technologies. Additionally, some terms may be terms arbitrarily selected by the applicant. These terms may be interpreted as defined in this specification, and if there is no specific term definition, they may be interpreted based on the overall content of this specification and common technical knowledge in the relevant technical field.

In addition, the same reference numerals or symbols in each drawing attached to this specification indicate parts or components that perform substantially the same function. For convenience of explanation and understanding, different embodiments will be described using the same reference numbers or symbols. That is, even if all components having the same reference number are shown in multiple drawings, the multiple drawings do not represent one embodiment.

Additionally, in this specification and claims, terms including ordinal numbers such as “first”, “second”, etc. may be used to distinguish between components. These ordinal numbers are used to distinguish identical or similar components from each other, and the meaning of the term should not be interpreted limitedly due to the use of these ordinal numbers. For example, components combined with these ordinal numbers should not be interpreted as having a limited order of use or arrangement based on the number. If necessary, each ordinal number may be used interchangeably.

In this specification, singular expressions include plural expressions, unless the context clearly dictates otherwise. In this application, terms such as “comprise” or “constitute” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are intended to indicate the presence of one or more other It should be understood that this does not exclude in advance the possibility of the presence or addition of features, numbers, steps, operations, components, parts, or combinations thereof.

In embodiments of the present invention, terms such as “module”, “unit”, “part”, etc. are terms to refer to components that perform at least one function or operation, and these components are divided into hardware or software. It may be implemented or may be implemented through a combination of hardware and software. In addition, a plurality of “modules”, “units”, “parts”, etc. are integrated into at least one module or chip and run on at least one processor, except in cases where each needs to be implemented with individual specific hardware. It can be implemented as (not shown).

Additionally, in an embodiment of the present invention, when a part is connected to another part, this includes not only direct connection but also indirect connection through other media. In addition, the fact that a part includes a certain component does not mean that other components are excluded, but that it may further include other components, unless specifically stated to the contrary.

Hereinafter, the present invention will be described in detail using the attached drawings.

1 and 2 are diagrams for explaining an example of providing an image effect such that the display becomes a transparent glass window in a display system according to an embodiment of the present disclosure.

The display device 100 according to an embodiment of the present disclosure has two operation modes. First, the first operation mode (eg, normal mode) is a mode that displays a normal image. Specifically, the first operation mode is a mode in which content pre-stored in the display device 100 or broadcast received from the outside is displayed using the entire screen of the display device 100.

And, in the second operation mode (e.g., background mode), the display device 100 displays a content screen including a background image behind the display device 100, making the display device 100 appear as if it were a glass window. This mode provides the same visual effect. When displayed in the second operation mode, the display device 100 displays the background area behind the display device 100 as a background image, so the user may have the illusion that the display device 100 is a transparent glass window. there is.

Meanwhile, the background image may be transmitted to the display device 100 from the electronic device 200 that communicates with the display device 100, and is an image captured by the user without being obscured by the display device 100. It can be. As shown in FIG. 1, the background image may be an image of the wallpaper behind the display device 100, and thus an interior effect where the display device 100 appears transparent can be obtained.

Meanwhile, as shown in FIG. 2, when there is an area (hereinafter referred to as a defect area 21) that spoils the aesthetics, such as a nail mark, a power outlet, a stain, or an area where the wallpaper has peeled off, on the wallpaper behind the display device 100, There is a problem that the defect area 21 is exposed as is in the background image, and the user does not want the defect area to be displayed in the background image displayed on the display device 100, so the defect area 21 is removed. At this time, the display device 100 or the electronic device 200 of the present invention wants to repeat the captured background image, such as when the background area behind the display device 100 is wallpaper with a repeating pattern of a certain pattern. When a displayed image pattern is included, the defect area can be removed by searching for another area within the background image that can replace the defect area 21.

Hereinafter, the above-described operation will be described in detail using specific configurations of the display device 100.

Figure 3 is a block diagram showing a simplified configuration of a display device according to an embodiment of the present disclosure.

Referring to FIG. 3, the display device 100 includes a display 110, a communication unit 120, and a processor 130.

The display device 100 of the present invention includes various types of devices capable of displaying images, such as TVs (smart TVs, etc.) and laptop PCs (laptop personal computers), but according to the main embodiment of the present invention, fixed devices such as TVs It is desirable to implement it as a device installed in the area. Hereinafter, unless otherwise specified, the description will be made assuming that the display device 100 of the present invention is implemented as a TV.

The display 110 displays an image. In particular, while the display device 100 is operated in the first operation mode, the display 110 may display video content acquired from an external source (eg, a broadcasting station, a server, a DVD, etc.). Additionally, while operating in the second operation mode, the display 110 may display a content screen including a pre-stored background image.

The display 110 may be implemented as various types of displays, such as a Liquid Crystal Display (LCD) or a Plasma Display Panel (PDP). The display 110 may also include a driving circuit and a backlight unit that may be implemented in the form of a-si TFT, low temperature poly silicon (LTPS) TFT, or organic TFT (OTFT). Meanwhile, the display 110 may be implemented as a touch screen by combining it with a touch sensor. The display 110 can display various images under the control of the processor 130 and can display a background image according to the present invention.

The communication unit 120 is a component that communicates with various types of external devices according to various types of communication methods. The communication unit 120 can communicate with external devices through various communication methods using RF (Radio Frequency) and IR (Infrared), such as BT (Bluetooth), Wi-Fi, Zigbee, and NFC (Near Field Communication). Accordingly, it may include communication modules to support various communication methods such as Bluetooth chips and Wi-Fi chips.

In particular, the communication unit 120 may communicate with the electronic device 200 capable of controlling the display device 100. The electronic device 200 capable of controlling the display device 100 may include, for example, a remote control, a smart phone, a tablet PC, a PDA, a PMP, etc. Hereinafter, unless otherwise specified, the description will be made assuming that the electronic device 200 of the present invention is implemented as a smart phone.

The communication unit 120 may receive various control commands for controlling the display device 100 from the electronic device 200, and may also transmit various control commands for controlling the electronic device 200 to the electronic device 200. there is. In particular, the communication unit 120 may receive a background image behind the display device 100 from the electronic device 200, various commands for removing a defect area in the background image, and a background image from which the defect area has been removed.

In addition, the communication unit 120 includes a USB port to which a USB connector can be connected, various external input ports for connection to various external terminals such as headsets, mice, LANs, etc., and a DMB for receiving and processing Digital Multimedia Broadcasting (DMB) signals. Of course, additional chips, etc. may be included.

The processor 130 is a component that controls the overall operation of the display device 100. Specifically, the processor 130 may receive data about the background image behind the display device 100 and location data about a specific area within the background image from the electronic device 200 through the communication unit 120. . At this time, the background image may include a pattern image that is repeatedly displayed at least twice. As a background image including a pattern image that is repeatedly displayed at least twice, a representative example is a captured image of wallpaper with a repeating image pattern (see FIGS. 1 and 2). That is, the background image may be an image of the wallpaper behind the display device 100 that is obscured by the display device 100.

Here, the specific area within the background image is an area selected by the user of the electronic device 200 and may include a defect area. Additionally, a specific area within the background image may be an area containing a defect area automatically detected by the electronic device 200 based on image characteristic information.

When data about the background image and location data about a specific area within the background image are received, the processor 130 creates an image pattern that matches the image pattern of a surrounding area adjacent to the specific area within the background image, based on the received data. The area that has can be extracted. Location data for a specific area may be data in the form of x, y coordinates indicating the location of the specific area based on the entire area of the background image. The processor 130 may detect feature information about the image pattern of the surrounding area, and use the detected feature information to extract an area within the background image that has an image pattern most similar to the image pattern of the surrounding area.

Specifically, the processor 130 may use the detected feature information to search for other areas where the feature information matches. Since it is difficult to completely match the feature information, it can be set to determine that the feature information matches if the feature information matches more than a preset percentage. Accordingly, the processor 130 can extract at least one area in the background image that has an image pattern that matches the image pattern of a surrounding area adjacent to a specific area, and extract only one area that has the most similar image pattern. It may be possible.

Thereafter, the processor 130 may determine the corresponding area of the specific area in the background image based on the extracted area, generate a background image in which the specific area is replaced with the determined corresponding area, and display it on the display 110. Specifically, since the image pattern of the extracted area matches the surrounding area adjacent to the specific area, a corresponding area of the specific area exists at a location corresponding to the location of the specific area, based on the extracted area. Once the corresponding area is determined, the processor 130 replaces the determined corresponding area with the specific area, thereby generating a background image in which the specific area is replaced with the corresponding area. At this time, the existing background image may be changed to a background image in which a specific area is replaced with a corresponding area, or a new background image may be created in which a specific area is replaced with a corresponding area.

Figure 4 is a block diagram showing a specific configuration of a display device according to an embodiment of the present disclosure. Referring to FIG. 4, the display device 100 includes a display 110, a communication unit 120, a processor 130, a broadcast reception unit 140, a signal separation unit 145, an A/V processing unit 150, and an audio It may include an output unit 155, an image signal generator 160, a storage unit 165, and an operation unit 170. Hereinafter, description of parts that overlap with the description in FIG. 3 will be omitted.

The broadcast receiver 140 receives a broadcast by wire or wirelessly from a broadcasting station or satellite and demodulates it. Specifically, the broadcast receiver 140 may receive a transport stream through an antenna or cable, demodulate it, and output a digital transport stream signal.

The signal separation unit 145 separates the transport stream signal provided from the broadcast reception unit 140 into a video signal, an audio signal, and an additional information signal. And the signal separation unit 145 transmits the video signal and audio signal to the A/V processing unit 150.

The A/V processing unit 150 performs signal processing such as video decoding, video scaling, and audio decoding on the video and audio signals input from the broadcast receiver 140 and the storage unit 165. Then, the A/V processing unit 150 outputs a video signal to the video signal provider 160 and outputs an audio signal to the audio output unit 155.

On the other hand, when storing the received video and audio signals in the storage unit 165, the A/V processing unit 150 can output the video and audio to the storage unit 165 in a compressed form.

The audio output unit 155 converts the audio signal output from the A/V processing unit 150 into sound and outputs it through a speaker (not shown) or to an external device connected through an external output terminal (not shown). .

The image signal generator 160 generates a graphic user interface (GUI) to provide to the user. And the image signal generator 160 adds the generated GUI to the image output from the A/V processor 150. And the image signal generator 160 provides an image signal corresponding to the image to which the GUI is added to the display 110. Accordingly, the display 110 displays various information provided by the display device 100 and images transmitted from the image signal generator 160.

Additionally, the image signal generator 160 may process and output the content screen generated by the processor 130. Specifically, the image signal generator 160 may output a plurality of layers as is or synthesize (or merge) the plurality of layers and provide them to the display 110.

The storage unit 165 stores various data and programs for controlling the display device 100. In addition, the storage unit 165 can receive video content with compressed video and audio from the A/V processing unit 150 and store it, and the stored video content is stored in the A/V processing unit 150 under the control of the processor 130. It can be printed to . In particular, the storage unit 165 may store data about the background image.

Meanwhile, the storage unit 165 may be implemented as a hard disk, non-volatile memory, volatile memory, etc.

The manipulation unit 170 is implemented with a touch screen, touch pad, key button, keypad, etc., and provides user manipulation of the display device 100. In this embodiment, an example of receiving a control command through the operation unit 170 provided in the display device 100 has been described, but the operation unit 170 may also receive a user operation from an external control device (for example, a remote control). there is.

The processor 130 controls the overall operation of the display device 100. Specifically, in the first operation mode, the processor 130 may control the image signal generator 160 and the display 110 to display an image according to a control command input through the manipulation unit 170.

The processor 1300 may include a ROM 131, a RAM 132, a Graphics Processing Unit (GPU) 133, a CPU 134, and a bus. ROM 131, RAM 132, GPU 133, The CPU 134 and the like may be connected to each other through a bus.

The CPU 134 accesses the storage unit 165 and performs booting using the operating system (O/S) stored in the storage unit 165. And the CPU 134 can perform various operations using various programs, content, data, etc. stored in the storage unit 165. Since the operation of the CPU 134 is the same as the operation of the processor 130 in FIG. 2, redundant description will be omitted.

The ROM 131 stores a set of instructions for booting the system. When a turn-on command is input and power is supplied, the CPU 134 copies the O/S stored in the storage unit 165 to the RAM 132 according to the command stored in the ROM 131 and executes the O/S to maintain the system. Boot the . When booting is complete, the CPU 134 copies various programs stored in the storage unit 165 to the RAM 132 and executes the programs copied to the RAM 132 to perform various operations.

When the display device 100 completes booting, the GPU 133 may generate a screen including various objects such as icons, images, and text. The GPU 133 may be configured as a separate component such as the image signal generator 160, and may be implemented as a SoC (System on Chip) combined with the CPU 134 within the processor 130. It may be possible.

Hereinafter, the operation of the processor 130 will be described in more detail.

First, the display device 100 may receive data about the background image from the external electronic device 200 and store it in the memory 165.

In particular, the display device 100 may receive data about a background image obtained using a guide member from the electronic device 200 before the display device 100 is installed.

Specifically, the user may fix the guide member to a location (eg, a wall) where the display device 100 is to be installed.

When the guide member is fixed, the electronic device 200 may use a camera to obtain an image including the guide member located in the area where the display device 100 will be installed. And, the electronic device 200 can display the acquired image. In this case, the displayed image may include a plurality of indicators that guide the position of the mark of the guide member to obtain an optimal background image.

Then, the electronic device 200 analyzes the background of the area (e.g., wall area) where the display device 100 is located within the guide member among the captured images and creates a background image of the location where the display device 100 is installed. data can be obtained. At this time, the background image is an image of the area (for example, a wall) where the display device 100 is installed. When the background image is displayed on the display device 100, the user uses the display device 100. Through this, the glass window effect can be provided.

Additionally, while operating in the first operation mode (i.e., normal mode), the processor 130 may control the display 110 to display image content received from the outside or previously stored image content. For example, the processor 130 may control the display 110 to display broadcast content received through a tuner.

While operating in normal mode, a preset user command (for example, a command to select a specific button on the remote control) is input or a preset event (for example, the display device 100 is in a standby state (display 110 is turned off) When an event that detects a user in a state) is detected, the processor 130 may change the operation mode of the display device 100 from the first operation mode to the second operation mode (i.e., background mode).

While operating in the second operation mode, the processor 130 may control the display 110 to display a pre-stored background image.

5 and 6 are diagrams showing a method of selecting a specific area and surrounding areas in an electronic device, according to an embodiment of the present disclosure.

FIG. 5 illustrates a screen on which a background image 10 is displayed behind the display device 100 in the electronic device 200 according to an embodiment of the present disclosure. The electronic device 200 may receive a user manipulation command for selecting a specific area of the background image 10 while an application capable of transmitting a control command to the display device 100 is running. According to FIG. 5, in order to remove the outlet 51 from the background image 10, the user may input a user operation command to select the outlet. Hereinafter, FIGS. 5 to 10 relate to an embodiment in which a specific area including the area where the outlet is displayed is an area selected by the user. Hereinafter, the area where the outlet is displayed means an area with a defect that disturbs the aesthetics. It should be described as a defect area.

FIG. 6 illustrates a method of selecting a specific area and surrounding areas in the electronic device 200 according to an embodiment of the present disclosure.

The user may select a specific area 61 from the background image 10 displayed on the display of the electronic device 200. At this time, the electronic device 200 may receive a selection input for a specific area in the background image 10 and display an application execution screen that can transmit location data for the selected specific area to the display device 100.

Additionally, the display of the electronic device 200 may include a touch display, and the user may select a specific area 61 including the defect area by performing a drag operation within the background image 10. there is. Depending on the drag operation, the selected specific area 61 may be displayed separately from other areas, as shown in FIG. 6 .

Meanwhile, FIG. 6 shows a screen where the user selects one specific area 61, but there may be multiple defect areas in the background image 10, and in this case, the user may select multiple specific areas.

When the specific area 61 is selected, data about the background image 10 and location data about the specific area 61 within the background image 10 may be transmitted to the display device 100 according to a user command. The location data for the specific area 61 may be data in the form of x, y coordinates indicating the location of the specific area 61 based on the entire area of the background image 10. However, the location data for the specific area 61 may also be image data for the specific area.

Hereinafter, FIGS. 7 to 10 will describe the operation of the display device 100 that has received data about the background image 10 and location data about a specific area 61 within the background image 10.

Figure 7 is a diagram illustrating in detail the surrounding area used to determine the corresponding area of a specific area, according to an embodiment of the present invention.

The display device 100 may receive data about the background image 10 and location data about the specific area 61 from the electronic device 200. At this time, the display device 100 receives data about the background image 10 in advance from the electronic device 200 and, while displaying the background image 10, receives location data about the selected specific area 61. You can also receive it.

The processor 130 of the display device 100 may detect a specific area 61 of the background image 10 based on the received location data for the specific area 61. At this time, the processor 130 may set a surrounding area close to the specific area 61 as an area used to search for a corresponding area to the specific area 61 within the background image 10. At this time, the processor 130 may set the remaining area excluding the specific area as a peripheral area in an area including the specific area 61 therein.

As shown in FIG. 7, the processor 130 can set an area 62 that includes the specific area 61, based on the selected specific area 61, and the surrounding area 63 is one area. As an area excluding the specific area 61 from (62), it may be an area of a certain range surrounding the specific area 61 on all sides. The range of the surrounding area 63 can be set in various other ways. However, the range of the surrounding area 63 must be a range that includes the image pattern around the specific area 61, and accordingly, one area 62 that includes the specific area 61 inside is limited to the surrounding area 61. It is desirable to set it to a size that includes the image pattern.

Figure 8 is a diagram showing a determined corresponding area according to an embodiment of the present invention.

The processor 130 may extract an area having an image pattern that matches the image pattern of the surrounding area 63 within the background image 10. Specifically, the processor 130 uses characteristic information about the image pattern of the peripheral area 63 to select an area that has an image pattern most similar to the image pattern of the peripheral area 63 within the received background image. It can be extracted. At this time, masking processing may be performed on the specific area 61.

Meanwhile, since the background image 10 includes a pattern image that is repeated at least twice, there may be one or more extracted areas. When there are multiple extracted areas, the processor 130 may select one of the extracted areas according to a random process, or may select an area that has an image pattern most similar to the image pattern of the surrounding area 63. At this time, the processor 130 calculates the brightness data of the peripheral area 63, and selects the area with the most similar brightness among the plurality of areas extracted based on the calculated brightness data as the image most similar to the image pattern of the peripheral area 63. You can select an area with a pattern.

FIG. 8 shows that one area 81 is extracted as an area having an image pattern most similar to that of the surrounding area 63. At this time, the area 83 that is only partially similar to the image pattern of the surrounding area 63 may not be extracted as an area with a similar image pattern, or may be excluded even if extracted.

The processor 130 may determine the corresponding area 82 of the specific area 61 based on the extracted area 81. The relative position of the corresponding area 82 based on the extracted area 81 is the same as the relative position of the specific area 61 based on the surrounding area 63.

Meanwhile, when there are a plurality of specific areas selected by the user, areas similar to the image patterns of the surrounding areas of each of the plurality of specific areas may be extracted, and based on each extracted area, the corresponding area of each of the plurality of specific areas may be extracted. The area may be determined.

This method of extracting areas with similar image patterns can also be implemented using a template matching technique, and is already implemented in the OpenCV library.

Figure 9 is a diagram illustrating a screen in which a specific area is replaced with a determined corresponding area, according to an embodiment of the present invention.

When the corresponding area 82 is determined, the processor 130 may control the display 110 to display the background image 10 in which the specific area 61 is replaced with the corresponding area 82.

At this time, the existing background image 10 may be changed to a background image 10 in which a specific area is replaced with a corresponding area, or a new background image 10' may be created in which a specific area is replaced with a corresponding area.

However, even if the corresponding area 82 has the same image pattern as the specific area 61, the brightness of each area of the wallpaper may be different depending on the lighting position and angle of the place where the display device 100 is located. . For example, an area of wallpaper that is close to a light will be relatively brighter than an area of wallpaper that is further away from the light. Therefore, as shown in FIG. 9, a difference in brightness occurs between the replaced area and the surrounding area. At this time, the processor 130 may perform image processing to correct the difference in brightness between the corresponding area 82 and the specific area 61 in the background image 10.

Specifically, the processor 130 may calculate brightness data of the peripheral area 63. At this time, the brightness data includes a representative value for brightness, and the representative value may be determined by various definitions such as average, median, or mode. The processor 130 may correct the brightness of the replaced corresponding area 82 to be similar to the brightness of the surrounding area 63, based on the calculated brightness data of the surrounding area 63. For example, the processor 130 may correct the brightness of the corresponding area 82 so that the representative value of the brightness of the corresponding area 82 becomes the same as the representative value of the brightness of the surrounding area 63.

The brightness correction described above can also be implemented through the OpenCV library.

Figure 10 is a diagram illustrating a method of applying a blending effect according to an embodiment of the present invention.

Even if the brightness of the replaced corresponding area 82 is corrected according to the process shown in FIG. 9, the difference between the corresponding area 82 and the surrounding area 63 varies depending on the slight difference in brightness of the pixels at the boundary between the corresponding area 82 and the surrounding area 63. A problem may occur where the areas 63 are displayed separately. Accordingly, the processor 130 performs secondary correction for brightness by applying a blending effect to the surrounding area 63 based on the corrected brightness data of the replaced corresponding area 82 and the brightness data of the surrounding area 63. can be performed.

Specifically, referring to FIG. 10, the blending effect can be applied using the following equation.

Here, x' means the final luminance value for each pixel in the peripheral area 63 to which the blending effect is applied. x means the luminance value for each pixel in the peripheral area 63 before the blending effect is applied. y means the luminance value for each pixel in the corresponding area 82. α is a parameter that changes depending on the location of each pixel. As the pixel location approaches the corresponding area 82, it approaches 0, and as it approaches the surrounding area 63, it approaches 1.

According to Equation 1, when the α value is 0, the pixel is very close to the corresponding area 82, so the luminance value of the pixel can be corrected according to the equation x'=y. When the value of α is 1, the pixel is very close to the surrounding area 63, so the luminance value of the pixel can be corrected according to the equation x'=x. That is, according to the above equation, the closer the pixel is to the corresponding area 82, the closer the luminance value of the corresponding area 82 is, and the closer the pixel is to the peripheral area 63, the closer the luminance value is to the luminance value of the peripheral area 63. As they get closer, a natural blending effect can be applied.

According to the above-described process, the display device 100 can display a natural background image from which defect areas have been removed.

The above-described blending effect can also be implemented through the OpenCV library.

Figure 11 is a block diagram showing a simplified configuration of an electronic device according to an embodiment of the present invention.

Referring to FIG. 11 , the electronic device 200 includes a display 210, a communication unit 220, and a processor 230.

The display 210 is configured to display a screen on which a specific area in the background image 10 can be selected. Since the specific configuration of the display 210 is the same or similar to that of the display device 100, detailed description will be omitted.

The communication unit 220 is a component that performs communication with the display device 100. Specifically, the communication unit 220 may transmit the background image 10 and location data for a specific area selected from the background image 10 to the display device 100 under the control of the processor 230. Since the specific configuration of the communication unit 220 corresponds to that of the display device 100, detailed description will be omitted.

The processor 230 calculates location data for a specific area selected by the user in the background image 10, and operates the communication unit 220 to transmit the background image 10 and the calculated location data to the electronic device 200. You can control it.

Meanwhile, according to another embodiment of the present invention, the process of removing the defect area from the background image 10 may be performed in the electronic device 200.

Specifically, the processor 230 may display the background image 10 behind the display device 100 on the display 210 and receive a selection input for a specific area within the background image 10 by the user. When a specific area is selected, the processor 230 extracts an area with an image pattern that matches the image pattern of a surrounding area adjacent to the selected specific area, and selects a corresponding area of the specific area in the background image 10 based on the extracted area. can be decided. When the corresponding area is determined, the processor 230 generates a background image 10' in which the specific area is replaced with the determined corresponding area, and transmits the generated background image 10' to the display device 100. 220) can be controlled. At this time, the display device 100 displays the received background image 10'.

In particular, the electronic device 200 may further include a camera (not shown), and the processor 230 may control the camera to generate the background image 10 behind the display device 100. At this time, the camera may generate the background image 10 behind the display device 100 by capturing the background image 10 that will be obscured by the display device 100 in the absence of the display device 100.

In addition, the processor 230 may set the remaining area excluding the specific area as the surrounding area in an area including the specific area, and use characteristic information about the image pattern of the surrounding area to set the area inside the background image. The area with the image pattern most similar to that of the surrounding area can be extracted. In addition, the processor 230 may perform all image processing that the processor 130 of the display device 100 performs on the background image 10 in the same manner.

Figure 12 is a flowchart for explaining a method of controlling a display device according to an embodiment of the present invention.

First, a background image behind the display device 100 and location data for a specific area within the background image are received from the electronic device 200 (S1210). At this time, the background image includes a pattern image that is repeated at least twice, and the specific area may be at least one area selected by the user of the electronic device 200.

Thereafter, based on the location data for the specific area, an area having an image pattern that matches the image pattern of a surrounding area adjacent to the specific area is extracted from the received background image (S1220). At this time, in an area that includes a specific area, the remaining area excluding the specific area can be set as a peripheral area. Afterwards, using feature information about the image pattern of the surrounding area, an area having an image pattern most similar to the image pattern of the surrounding area can be extracted from the received background image.

Afterwards, based on the extracted area, the corresponding area of the specific area in the background image is determined (S1230).

Afterwards, a background image in which the specific area is replaced with the determined corresponding area is displayed (S1240).

Afterwards, brightness data of the surrounding area may be calculated, and based on the calculated brightness data, the brightness of the replaced corresponding area may be corrected.

Additionally, a blending effect can be applied based on the corrected brightness data of the replaced corresponding area and the brightness data of the surrounding area.

Figure 13 is a flowchart for explaining a method of controlling an electronic device according to an embodiment of the present invention.

First, a background image behind the display device 100 is displayed (S1310).

Thereafter, when a specific area in the background image is selected by the user, an area having an image pattern that matches the image pattern of a surrounding area adjacent to the selected specific area is extracted (S1320).

Afterwards, the corresponding area of the specific area in the background image is determined based on the extracted area (S1330).

Afterwards, a background image in which the specific area is replaced with the determined corresponding area is generated (S1340).

Afterwards, the generated background image is transmitted to the display device 100 (S1350).

According to the various embodiments of the present invention as described above, it is possible to display an image of wallpaper from which areas that detract from the wallpaper's aesthetics, such as nail marks, power outlets, and stains, can be removed, thereby creating a more aesthetically pleasing interior effect.

The control method of the display device according to the various embodiments described above may be implemented as a program and stored in various recording media. That is, computer programs that are processed by various processors and can execute the various display methods described above may be used while stored in a recording medium.

As an example, receiving, from an electronic device, a background image behind the display device and location data for a specific area within the background image, based on the location data for the specific area, a surrounding area adjacent to the specific area within the received background image. Extracting an area having an image pattern that matches the image pattern, determining a corresponding area of a specific area in the background image based on the extracted area, and displaying a background image in which the specific area is replaced with the determined corresponding area. A non-transitory computer readable medium storing a program that performs the steps may be provided.

A non-transitory readable medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as registers, caches, and memories. Specifically, the various applications or programs described above may be stored and provided on non-transitory readable media such as CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, etc.

In addition, although the preferred embodiments of the present disclosure have been shown and described above, the present disclosure is not limited to the specific embodiments described above, and the technical field to which the disclosure pertains without departing from the gist of the present disclosure as claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be understood individually from the technical idea or perspective of the present disclosure.

100: display device 110: display
120: Communication unit 130: Processor
200: Electronic device 210: Display
220: Communication unit 230: Processor

Claims (20)

In the display device,
display;
Ministry of Communications; and
Receiving, from an electronic device through the communication unit, a background image for a background having a pattern repeated twice or more behind the display device and location data indicating a specific area within the background image that does not have the pattern,
Based on the location data representing the specific area, extract an area with an image pattern matching an image pattern of a surrounding area adjacent to the specific area within the background image,
Determining a corresponding area of the specific area in the background image based on the extracted area,
A processor that controls the display to display a background image in which the specific area is replaced with the determined corresponding area. delete According to paragraph 1,
The processor,
A display device, characterized in that, in an area including the specific area, the remaining area excluding the specific area is set as the peripheral area. According to paragraph 1,
The processor,
A display device, characterized in that an area having an image pattern most similar to the image pattern of the peripheral area is extracted from the received background image using characteristic information about the image pattern of the peripheral area. According to paragraph 1,
The processor,
A display device, characterized in that: calculating brightness data of the peripheral area, and correcting brightness of the replaced corresponding area based on the calculated brightness data. According to clause 5,
The processor,
A display device, characterized in that applying a blending effect based on the corrected brightness data of the replaced corresponding area and the brightness data of the surrounding area. According to paragraph 1,
The specific area is,
A display device, characterized in that at least one area is selected by a user of the electronic device. In a method of controlling a display device,
Receiving, from an electronic device, a background image for a background having a pattern repeated twice or more behind the display device and location data indicating a specific area within the background image that does not have the pattern;
extracting an area having an image pattern matching an image pattern of a surrounding area adjacent to the specific area within the background image, based on the location data representing the specific area;
determining a corresponding area of the specific area in the background image based on the extracted area; and
A control method comprising: displaying a background image in which the specific area is replaced with the determined corresponding area. delete According to clause 8,
The extraction step is,
A control method, characterized in that, in an area including the specific area, the remaining area excluding the specific area is set as the peripheral area. According to clause 8,
The extraction step is,
A control method, characterized in that, using characteristic information about the image pattern of the surrounding area, an area having an image pattern most similar to the image pattern of the surrounding area is extracted from the received background image. According to clause 8,
calculating brightness data of the surrounding area; and
The control method further comprising: correcting the brightness of the replaced corresponding area based on the calculated brightness data. delete delete delete delete delete delete delete delete

Images