WO2021167230A1

WO2021167230A1 - Electronic device and control method therefor

Info

Publication number: WO2021167230A1
Application number: PCT/KR2020/019035
Authority: WO
Inventors: 조영훈; 최윤식; 김신행; 문영수; 민병석; 윤종수; 이상수
Original assignee: 삼성전자(주); 연세대학교 산학협력단
Priority date: 2020-02-19
Filing date: 2020-12-23
Publication date: 2021-08-26
Also published as: KR20210105635A

Abstract

An electronic device according to one embodiment of the present invention can comprise a display and a processor, which divides an inputted image into a plurality of frame sets having a first number of frames, divides the first number of frames included in the frame sets into a plurality of subframe sets having a second number of frames, identifies a predefined key frame from among frames included in each subframe set with respect to each of the plurality of subframe sets, and identifies an area inside the image in which a brightness change of the frame sets is less than a threshold value, on the basis of the brightness change, between the frames, using the identified key frame.

Description

Electronic device and its control method

The present disclosure relates to an electronic device for preventing deterioration of an image displayed on a display, and a method for controlling the same.

Recently, various types of display devices, such as an organic light emitting display device, a liquid crystal display device, and a plasma display device, have been widely used. In such display devices, as a driving time increases, a specific pixel may be deteriorated due to a so-called burn-in phenomenon, and thus the performance thereof may be deteriorated. For example, since a digital information display device used for information transfer in public places and vehicles continuously outputs a specific image or text for a long time, deterioration of a specific pixel may be accelerated and an afterimage may occur. have. In particular, since most broadcasts display a program name or a corresponding broadcaster name at the top of the screen, pixel deterioration may occur in a fixedly displayed area within a channel. In order to solve this problem, a pixel shift technology that moves and displays an image on a display panel at a certain period, a luminance reduction technology that increases the lifespan by reducing the luminance of a specific area, and the like are used. However, in order to use these technologies, it is necessary to accurately determine the area where pixel deterioration occurs, which is directly related to the efficiency of each technology.

SUMMARY OF THE INVENTION An object of the present disclosure is to provide an electronic device and a control method therefor that more effectively prevent deterioration of an image displayed on a display.

An electronic device according to an embodiment of the present invention, comprising: a display; dividing an input image into a plurality of frame sets having a first number of frames, dividing the first number of frames included in the frame set into a plurality of subframe sets having a second number of frames, and dividing the plurality of subframes For each set, a predefined keyframe is identified among frames included in each subframe set, and based on the luminance change between the frames using the identified keyframe, the region in the image in which the luminance change of the frame set is less than a threshold value. It may include a processor that identifies

The processor may identify a luminance change of the frame set based on a luminance change between the keyframe included in the subframe set and the other frame.

The processor may identify a luminance change of the set of frames based on a sequential luminance change between the keyframe and the frame subsequent to the keyframe.

The keyframe may be a first frame of the set of subframes including the keyframe.

The processor may identify a luminance change of the frame set based on a luminance change between the keyframes.

The processor may identify a luminance change of the set of frames based on a calculation between luminance changes in the plurality of subframe sets.

The frame set is a first frame set, and the processor selects a region within the image based on a luminance change of a second frame set in a section shifted by a predefined number of subframe sets in the section of the first frame set. can be identified.

The processor may identify a region in the image based on a luminance change of the first frame set, and identify a change in the region based on a luminance change of the second frame set based on the identified region.

The processor may identify a region in the image based on a calculation between luminance changes of the first frame set and the second frame set.

The subframe set is a first subframe set, the keyframe is a first keyframe, and the processor is configured to convert the second number of frames included in the subframe set to a third number of frames. It is divided into frame sets, and for each of the plurality of second sub-frame sets, a predefined second key frame is identified among frames included in each second sub-frame set, and the luminance change between frames using the identified second key frame is determined. A region in the image may be identified based on the image.

In a control method of an electronic device according to an embodiment of the present invention, the method comprising: dividing an input image into a plurality of frame sets having a first number of frames; dividing a first number of frames included in the frame set into a plurality of subframe sets having a second number of frames; identifying a predefined keyframe among frames included in each subframe set for each of the plurality of subframe sets; and identifying a region in the image in which the luminance change of the set of frames is less than a threshold value based on the luminance change between the frames using the identified keyframe.

The identifying of the region in the image may include identifying a luminance change of the frame set based on a luminance change between the keyframe included in the subframe set and the other frame.

The identifying the luminance change of the frame set may include identifying the luminance change of the frame set based on the sequential luminance change between the keyframe and the frame subsequent to the keyframe.

The identifying of the region in the image may include identifying a luminance change of the frame set based on a luminance change between the keyframes.

The identifying of the region in the image may include identifying a luminance change of the frame set based on a calculation between luminance changes in the plurality of subframe sets.

The frame set is a first frame set, and the step of identifying a region in the image is based on a luminance change of a second frame set in a section moved by a predefined number of subframe sets in the section of the first frame set. to identify a region within the image.

The identifying a region in the image may include: identifying a region in the image based on a change in luminance of the first frame set; and identifying a change in the region based on a change in luminance of the second frame set based on the identified region.

The subframe set is a first subframe set, the keyframe is a first keyframe, and dividing a second number of frames included in the subframe set into a plurality of second subframe sets having a third number of frames step; identifying a predefined second keyframe among frames included in each second subframe set for each of the plurality of second subframe sets; and identifying a region in the image based on a change in luminance between frames using the identified second keyframe.

A computer readable code, wherein the recording medium stores a computer program including a code for performing a control method of an electronic device, the control method of the electronic device includes: dividing into sets of frames; dividing a first number of frames included in the frame set into a plurality of subframe sets having a second number of frames; identifying a predefined keyframe among frames included in each subframe set for each of the plurality of subframe sets; and identifying a region in the image in which the luminance change of the set of frames is less than a threshold value based on the luminance change between the frames using the identified keyframe.

In order to prevent deterioration of the image displayed on the display, it is possible to precisely identify an area where deterioration occurs. In addition, even when region loss due to noise or compression of an image occurs, it is possible to identify a region in which deterioration occurs stably.

1 is a diagram illustrating a configuration of an electronic device according to an embodiment of the present invention.

2 is a block diagram illustrating an operation flowchart of an electronic device according to an embodiment of the present invention.

3 is a diagram illustrating a frame processed by an electronic device according to an embodiment of the present invention.

4 is a diagram illustrating a frame processed by an electronic device according to an embodiment of the present invention.

5 is a diagram illustrating a frame processed by an electronic device according to an embodiment of the present invention.

6 is a diagram illustrating a frame processed by an electronic device according to an embodiment of the present invention.

7 is a diagram illustrating a frame processed by an electronic device according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numbers or symbols refer to components that perform substantially the same function, and the size of each component in the drawings may be exaggerated for clarity and convenience of description. However, the technical spirit of the present invention and its core configuration and operation are not limited to the configuration or operation described in the following embodiments. In describing the present invention, if it is determined that a detailed description of a known technology or configuration related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted.

In an embodiment of the present invention, terms including an ordinal number such as first, second, etc. are used only for the purpose of distinguishing one element from another element, and the expression of the singular is plural unless the context clearly indicates otherwise. includes the expression of In addition, in an embodiment of the present invention, terms such as 'consisting', 'comprising', 'having' and the like are one or more other features or the presence of numbers, steps, operations, components, parts, or combinations thereof. Or it should be understood that the possibility of addition is not excluded in advance. In addition, in an embodiment of the present invention, a 'module' or 'unit' performs at least one function or operation, and may be implemented as hardware or software or a combination of hardware and software, and is integrated into at least one module. and can be implemented. Further, in an embodiment of the present invention, at least one of the plurality of elements refers to all of the plurality of elements as well as each one or a combination thereof excluding the rest of the plurality of elements.

In this case, the electronic device 100 may be implemented as a display device capable of displaying an image. For example, the electronic device 100 may include a TV, a computer, a smart phone, a tablet, a portable media player, a wearable device, a video wall, an electronic picture frame, and the like.

1 , the electronic device 100 may include an interface unit 110 . The interface unit 110 may include a wired interface unit 111 . The wired interface unit 111 includes a connector or port to which an antenna capable of receiving a broadcast signal according to a broadcasting standard such as terrestrial/satellite broadcasting can be connected, or a cable capable of receiving a broadcast signal according to a cable broadcasting standard can be connected. do. As another example, the electronic device 100 may have a built-in antenna capable of receiving a broadcast signal. Wired interface unit 111 is an HDMI port, DisplayPort, DVI port, Thunderbolt, composite video, component (component) video, super video (super video), such as SCART, video and / or audio transmission standards according to It may include a connector or port, and the like. The wired interface unit 111 may include a connector or port according to a universal data transmission standard, such as a USB port. The wired interface unit 111 may include a connector or a port to which an optical cable can be connected according to an optical transmission standard. The wired interface unit 111 is connected to an external microphone or an external audio device having a microphone, and may include a connector or a port capable of receiving or inputting an audio signal from the audio device. The wired interface unit 111 is connected to an audio device such as a headset, earphone, or external speaker, and may include a connector or port capable of transmitting or outputting an audio signal to the audio device. The wired interface unit 111 may include a connector or port according to a network transmission standard such as Ethernet. For example, the wired interface unit 111 may be implemented as a LAN card connected to a router or a gateway by wire.

The wired interface unit 111 is wired through the connector or port in a 1:1 or 1:N (N is a natural number) method such as an external device such as a set-top box, an optical media playback device, or an external display device, speaker, server, etc. By being connected, a video/audio signal is received from the corresponding external device or a video/audio signal is transmitted to the corresponding external device. The wired interface unit 111 may include a connector or port for separately transmitting video/audio signals.

Also, according to the present embodiment, the wired interface unit 111 is embedded in the electronic device 100 , but may be implemented in the form of a dongle or a module to be detachably attached to the connector of the electronic device 100 .

The interface unit 110 may include a wireless interface unit 112 . The wireless interface unit 112 may be implemented in various ways corresponding to the implementation form of the electronic device 100 . For example, the wireless interface unit 112 is a communication method RF (radio frequency), Zigbee (Zigbee), Bluetooth (bluetooth), Wi-Fi (Wi-Fi), UWB (Ultra WideBand) and NFC (Near Field Communication), etc. Wireless communication can be used. The wireless interface unit 112 may be implemented as a wireless communication module that performs wireless communication with an AP according to a Wi-Fi method, or a wireless communication module that performs one-to-one direct wireless communication such as Bluetooth. The wireless interface unit 112 may transmit and receive data packets to and from the server by wirelessly communicating with the server on the network. The wireless interface unit 112 may include an IR transmitter and/or an IR receiver capable of transmitting and/or receiving an IR (Infrared) signal according to an infrared communication standard. The wireless interface unit 112 may receive or input a remote control signal from a remote control or other external device through an IR transmitter and/or an IR receiver, or transmit or output a remote control signal to another external device. As another example, the electronic device 100 may transmit/receive a remote control signal to and from the remote control or other external device through the wireless interface unit 112 of another method such as Wi-Fi or Bluetooth.

When the video/audio signal received through the interface unit 110 is a broadcast signal, the electronic device 100 may further include a tuner for tuning the received broadcast signal for each channel.

The electronic device 100 may include a display unit 120 . The display unit 120 includes a display panel capable of displaying an image on the screen. The display panel is provided with a light-receiving structure such as a liquid crystal type or a self-luminous structure such as an OLED type. The display unit 120 may further include additional components according to the structure of the display panel. For example, if the display panel is a liquid crystal type, the display unit 120 includes a liquid crystal display panel and a backlight unit for supplying light. and a panel driving substrate for driving the liquid crystal of the liquid crystal display panel.

The electronic device 100 may include a user input unit 130 . The user input unit 130 includes various types of input interface related circuits provided to perform user input. The user input unit 130 may be configured in various forms depending on the type of the electronic device 100 , for example, a mechanical or electronic button unit of the electronic device 100 , a remote controller separated from the electronic device 100 , and an electronic device. There is an input unit in an external device connected to the device 100 , a touch pad, and a touch screen installed on the display unit 120 .

The electronic device 100 may include a storage unit 140 . The storage unit 140 stores digitized data. The storage unit 140 has a non-volatile property that can preserve data regardless of whether or not power is provided, and data to be processed by the processor 170 is loaded and data is stored when power is not provided. It includes memory of volatile properties that cannot. Storage includes flash-memory, hard-disc drive (HDD), solid-state drive (SSD), read-only memory (ROM), etc., and memory includes buffer and random access memory (RAM). etc.

The electronic device 100 may include a microphone 150 . The microphone 150 collects sounds of the external environment including the user's voice. The microphone 150 transmits the collected sound signal to the processor 170 . The electronic device 100 may include a microphone 150 for collecting user voices or may receive a voice signal from an external device such as a remote controller having a microphone or a smart phone through the interface unit 110 . A remote controller application may be installed in an external device to control the electronic device 100 or perform functions such as voice recognition. In the case of an external device installed with such an application, a user's voice can be received, and the external device can transmit/receive and control data using the electronic device 100 and Wi-Fi/BT or infrared rays, etc. A plurality of interface units 110 that can be implemented may exist in the electronic device 100 .

The electronic device 100 may include a speaker 160 . The speaker 160 outputs audio data processed by the processor 170 as sound. The speaker 160 may include a unit speaker provided to correspond to audio data of one audio channel, and may include a plurality of unit speakers to respectively correspond to audio data of a plurality of audio channels. As another embodiment, the speaker 160 may be provided separately from the electronic device 100 . In this case, the electronic device 100 may transmit audio data to the speaker 160 through the interface unit 110 .

The electronic device 100 may include a processor 170 . The processor 170 includes one or more hardware processors implemented with a CPU, a chipset, a buffer, a circuit, etc. mounted on a printed circuit board, and may be implemented as a system on chip (SOC) depending on a design method. When the electronic device 100 is implemented as a display device, the processor 170 includes modules corresponding to various processes such as a demultiplexer, a decoder, a scaler, an audio digital signal processor (DSP), and an amplifier. Here, some or all of these modules may be implemented as SOC. For example, a module related to image processing such as a demultiplexer, a decoder, and a scaler may be implemented as an image processing SOC, and an audio DSP may be implemented as a chipset separate from the SOC.

When the processor 170 obtains a voice signal for the user's voice by the microphone 150 or the like, the processor 170 may convert the voice signal into voice data. In this case, the voice data may be text data obtained through a speech-to-text (STT) process for converting a voice signal into text data. The processor 170 identifies a command indicated by the voice data, and performs an operation according to the identified command. The voice data processing process and the command identification and execution process may all be executed in the electronic device 100 . However, in this case, since the system load and required storage capacity required for the electronic device 100 become relatively large, at least a part of the process is performed by at least one server communicatively connected to the electronic device 100 through a network. can be performed.

The processor 170 according to the present invention may call at least one command among commands of software stored in a storage medium readable by a machine such as the electronic device 100 and execute it. This enables a device such as the electronic device 100 to be operated to perform at least one function according to the called at least one command. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not include a signal (eg, electromagnetic wave), and this term refers to a case in which data is semi-permanently stored in a storage medium and a case in which data is temporarily stored. It does not distinguish between cases where

Meanwhile, the processor 170 divides the input image into a plurality of frame sets having a first number of frames, and divides the first number of frames included in the frame set into a plurality of subframe sets having a second number of frames. , for each subframe set, identify a predefined keyframe among frames included in each subframe set, and based on the luminance change between frames using the identified keyframe, within the image in which the luminance change of the frame set is less than a threshold value. At least a part of data analysis, processing, and result information generation for identifying a region is performed using at least one of machine learning, a neural network, or a deep learning algorithm as a rule-based or artificial intelligence algorithm can do.

For example, the processor 170 may perform the functions of the learning unit and the recognition unit together. The learning unit may perform a function of generating a learned neural network, and the recognition unit may perform a function of recognizing (or inferring, predicting, estimating, and judging) data using the learned neural network. The learning unit may generate or update the neural network. The learning unit may acquire learning data to generate a neural network. For example, the learning unit may acquire the learning data from the storage unit 140 or the outside. The learning data may be data used for learning of the neural network, and the neural network may be trained by using the data obtained by performing the above-described operation as learning data.

The learning unit may perform a preprocessing operation on the acquired training data before training the neural network using the training data, or may select data to be used for learning from among a plurality of training data. For example, the learning unit may process the learning data into a preset format, filter it, or add/remove noise to process the learning data into a form suitable for learning. The learner may generate a neural network set to perform the above-described operation by using the preprocessed learning data.

The learned neural network network may be composed of a plurality of neural network networks (or layers). Nodes of the plurality of neural networks have weights, and the plurality of neural networks may be connected to each other so that an output value of one neural network is used as an input value of another neural network. Examples of neural networks include Convolutional Neural Network (CNN), Deep Neural Network (DNN), Recurrent Neural Network (RNN), Restricted Boltzmann Machine (RBM), Deep Belief Network (DBN), Bidirectional Recurrent Deep Neural Network (BRDNN) and It can include models such as Deep Q-Networks.

Meanwhile, the recognizer may acquire target data to perform the above-described operation. The target data may be obtained from the storage 140 or from the outside. The target data may be data to be recognized by the neural network. The recognizer may perform preprocessing on the acquired target data before applying the target data to the learned neural network, or select data to be used for recognition from among a plurality of target data. For example, the recognition unit may process the target data into a preset format, filter, or add/remove noise to process the target data into a form suitable for recognition. The recognizer may obtain an output value output from the neural network by applying the preprocessed target data to the neural network. The recognition unit may obtain a probability value or a reliability value together with the output value.

For example, the control method of the electronic device 100 according to the present invention may be provided by being included in a computer program product. The computer program product may include instructions for software executed by the processor 170 , as described above. Computer program products may be traded between sellers and buyers as commodities. The computer program product is distributed in the form of a machine-readable storage medium (eg, CD-ROM), or via an application store (eg, Play Store™) or between two user devices (eg, smartphones). It can be distributed directly, online (eg, downloaded or uploaded). In the case of online distribution, at least a part of the computer program product may be temporarily stored or temporarily created in a machine-readable storage medium such as a memory of a server of a manufacturer, a server of an application store, or a relay server.

The processor 170 may divide the input image into a plurality of frame sets having the first number of frames (S210).

An image to be displayed on the display may be composed of a plurality of frames, and each frame may be composed of a plurality of pixels. When the electronic device 100 is a TV and an image displayed on the TV is, for example, a broadcast program, a program name or a corresponding broadcaster name may be displayed on the upper portion of the display. Since the broadcast program name or broadcaster name (hereinafter referred to as the logo) is continuously displayed in the fixed area while broadcasting is continued, pixels existing in the fixed area are compared with the area in which the luminance of the pixel continuously changes. Accordingly, functions such as degradation may be reduced. Alternatively, when the electronic device is a smartphone, a telecommunication company logo, a signal strength indication, a remaining battery level, and the like are always displayed at the same position on the top of the smartphone. For this reason, when content that uses the entire screen, such as video, is played back, the upper part of the smartphone may look blotchy due to deterioration of pixels. Hereinafter, a method for solving such a problem is presented.

Accordingly, the processor 170 bundles a plurality of frames existing in a section in which pixels are expected to be fixed to a luminance value within a certain range in a plurality of frames constituting an image, that is, a section in which a logo is expected to be displayed in the image. You can set a frame set. According to an embodiment of the present invention, the processor 170 may divide the input image into a plurality of frame sets having the first number of frames. Since the area corresponding to the degradation target such as the logo (hereinafter referred to as the logo area) is not fixed on the screen, and deterioration prevention is continuously required, the processor 170 may identify the logo area based on a plurality of frame sets. have. The longer the frame set, the more accurate the logo area selection can be in consideration of the frame luminance change in many areas within the frame. However, since there is a trade-off relationship between the frame set length and the logo area selection time, it is possible to determine an appropriate length section by comparing them.

The processor 170 may divide the first number of frames included in the frame set into a plurality of subframe sets including the second number of frames ( S220 ). The subframe set is a measure of how many frames the luminance change of an image will be updated, and is obtained by dividing a plurality of frames included in the frame set into the second number of frames. However, the first number and the second number, which are the number of frames in the frame set and the subframe set, need not be limited to a constant number.

The processor 170 may identify a predefined keyframe among frames included in each subframe set for each of the plurality of subframe sets ( S230 ). A keyframe is used as a reference for finding a logo area in an image, and the keyframe may be identified for each subframe set. Keyframes can contribute to more accurate detection of the logo area. Details will be described later.

The processor 170 may identify a region in the image in which the luminance change of the frame set is less than a threshold value based on the luminance change between frames using the identified keyframe ( S240 ).

The processor 170 may use the keyframe to accumulate the luminance change between each subframe set in the image on a pixel-by-pixel basis, and calculate the luminance change of the frame set by using the luminance change. At this time, since it is the largest value among RGB that most affects the deterioration of the pixel, the largest value of the R, G, and B channels may be selected and used as luminance, but the present invention is not limited thereto. If the calculated luminance change value of the frame set does not exceed a threshold, the processor 170 may determine that there is no movement in the image, and identify an area in which the luminance change is lower than the threshold value as the logo area.

According to an embodiment of the present invention, unnecessary luminance fluctuations can be reduced by using a keyframe, so that a region in which a logo is fixedly displayed on an image can be more accurately identified.

3 is a diagram illustrating a frame processed by an electronic device according to an embodiment of the present invention. As shown in FIG. 3, the present invention proposes an algorithm for selecting a logo area by configuring a plurality of frames in the form of a time pyramid. Referring to FIG. 3 , there is a frame set 310 composed of a first number of frames, and the processor 170 converts the frame set 310 back to a subframe set 1 and a subframe set 2 composed of a second number of frames. , … , can be divided into a plurality of subframe sets 320 like the subframe set N. Each subframe set 320 consists of frames including a keyframe 330 .

In the present invention, the processor 170 may configure the frame set 310 in the form of a time pyramid, and identify the keyframe 330 for each subframe set 320 . The reason for using the keyframe 330 in identifying the logo area is significant in the following. When the image is compressed according to the broadcasting environment, etc., the luminance of the logo area in the image may be fluctuated. If the luminance change of the previous frame and the current frame is simply calculated due to such a change, the luminance change that changes little by little is accumulated despite the logo area, so that it cannot be detected as the logo area. When the keyframe 330 is provided, the processor 170 calculates and accumulates the luminance change between the keyframe 330 and the next frame in each subframe set 320 . By using the keyframe 330 as a criterion for luminance change, it is possible to reduce the luminance difference due to minute luminance fluctuations in the logo region, thereby contributing to more accurate detection of the logo region. For example, the keyframe 330 may be the first frame of the subframe set 320 including the keyframe 330 . However, the present invention is not limited thereto.

In addition, in order to select a stable logo area when the logo in the image is constant, the processor 170 sets a frame set 310 in the form of a time pyramid, and then uses it to select a background other than the logo in which the luminance change occurs. It is possible to reliably guarantee the logo area while excluding it.

Therefore, according to an embodiment of the present invention, even when a logo area loss occurs due to noise or compression of the logo area, it is possible to stably select the logo area.

4 is a diagram illustrating a frame processed by an electronic device according to an embodiment of the present invention. For example, when an image has 30 fps, that is, 30 images (frames) per second, if only the luminance change between two adjacent frames is considered, most have similar luminance values except for the case of a scene change, so the logo area It is difficult to find the corresponding pixel. Therefore, since it is necessary to design an algorithm by accumulating the luminance change of several frames, in this figure, a method of calculating the luminance change using a key frame will be described in detail.

4 shows the frame set 310 in the form of a time pyramid described above and a plurality of subframe sets 320 divided therefrom. The subframe set 320 again consists of frames, and among them, a keyframe 330, which is the first frame, exists. The processor 170 may identify the luminance change of the frame set 310 based on the luminance change between the keyframe 330 included in the subframe set 320 and another frame. For example, the processor 170 calculates a luminance change between the keyframe 1 330 included in the subframe set 1 320 and the

other frames

410 , 420 , 430 , ... It is possible to calculate the luminance change of the entire frame set 310 by calculating for each.

However, as shown in this figure, the key frame 1 330 does not need to calculate the luminance change between all frames included in the subframe set 1 320 , it can be calculated only for some frames, and the frames are selected. There is no limit on how to do it.

According to another embodiment of the present invention, the processor 170 may identify a luminance change of a frame set based on a luminance change between a keyframe and frames subsequent to the keyframe. In this case, the keyframe and the frame following the keyframe do not need to belong to the same subframe set 320 .

In the present invention, a stable logo area can be ensured by configuring a frame in the form of a time pyramid and using a key frame, rather than using a change in luminance between two consecutive frames.

5 is a diagram illustrating a frame processed by an electronic device according to an embodiment of the present invention. This figure relates to another method of calculating a luminance change using a keyframe. In this embodiment, as described above, the frame set 310 is configured in the form of a time pyramid.

The processor 170 may identify the luminance change of the frame set 310 based on the luminance change between the

keyframes

330 , 520 , ... . If only the luminance change between successive frames is used to select the logo area, if the bright background continues in the image, the luminance change of the bright background part in the frame set is constant, so there may be a problem in recognizing it as the logo part. In addition, it is important to select a stable logo area when the logo in the image is constant, and the logo area shows a lot of difference according to the luminance change value of the frame set. Accordingly, it is possible to select a keyframe for each subframe set divided by a predetermined number of frames within the frame set, and calculate the luminance change between the selected keyframes in units of pixels. In this case, the number of frames divided into the subframe set may be related to the frequency at which errors such as luminance fluctuations due to image compression or the like occur. For example, when an error occurs once for every predetermined number of frames in the image, the number of frames included in the subframe set may be appropriately divided so that the key frame, which is a reference for calculating the luminance change, does not correspond to this error period. However, since the error may not occur according to a certain period and may be difficult to predict, at least a part of data analysis, processing, and result information generation to distinguish it is machine learning as a rule-based or artificial intelligence algorithm. , a neural network, or a deep learning algorithm.

According to an embodiment of the present invention, when the luminance change is less than a threshold value even though it is not a logo area, even when there is an error in an image, the logo area can be accurately and stably identified.

6 is a diagram illustrating a frame processed by an electronic device according to an embodiment of the present invention. In this drawing, after selecting a logo area using one frame set, a method of updating a logo area that is changed according to a subsequent frame is presented. The processor 170 may select a logo area using the first frame set 310 . In addition, the processor 170 may identify a region in the image based on the luminance change of the second frame set 610 in the section moved by a predefined number of subframe sets in the section of the first frame set 310 . have. For example, the processor 170 subtracts the luminance change from the first subframe set of the first frame set 310 for each new subframe set displayed after the first frame set 310 , and the difference in luminance of the new subframe set By adding , we can recalculate the total luminance change to find the logo area. Referring to FIG. 6 , the first frame set 310 moves by two subframe sets to form the second frame set 610 . Accordingly, the processor 170 subtracts the luminance changes of the subframe set 1 and the subframe set 2 from the calculated luminance values of the first frame set 310 and adds the luminance changes of the two newly added subframe sets to the second A change in luminance of the frame set 610 may be obtained. Thereafter, the processor 170 may update an area in which the recalculated luminance change does not exceed the threshold value as a new logo area. Through this, when there is a change in the logo area in the image, it is possible to detect the logo area.

The processor 170 identifies a region in the image based on the luminance change of the first frame set 310 and changes the region based on the luminance change of the second frame set 610 based on the identified region rather than the entire region. can be identified. Since the change of the logo area is identified based on the area identified through the first frame set 310, the speed of processing the process can be increased and the efficiency can be increased.

In addition, the processor 170 may identify a region in the image based on the calculation between the luminance changes of the first frame set 310 and the second frame set 610 . After the processor 170 identifies a region in the image through the first frame set 310, the first frame set 310 and the second subframe set 610 moved by a predefined number of subframe sets An area in an image may be identified by comprehensively considering the luminance changes. Through this, the identified region may be more precisely distinguished.

According to an embodiment of the present invention, after the logo area is identified, it is possible to continuously reflect changes such as the movement of the logo area over time, so that deterioration of pixels according to the logo area is continuously prevented. and can increase processing speed and efficiency.

7 is a diagram illustrating a frame processed by an electronic device according to an embodiment of the present invention. In this figure, the luminance change is calculated by dividing the upper subframe set into a plurality of lower subframe sets.

As shown in FIG. 7 , the processor 170 converts the second number of frames included in one first subframe set 710 among the plurality of first subframe sets into a plurality of frames having a third number of frames. It can be divided into a second subframe set. And the processor 170 identifies a predefined second key frame 730 among the frames included in each second sub-frame set 720 for each of the plurality of second sub-frame sets 720, and the identified second key A region in the image may be identified based on a change in luminance between frames using the frame 730 . For example, the processor 170 may calculate the change in luminance of the second subframe set 1 through an operation between frames included in the second subframe set 1 based on the second keyframe 1 , which is each second subframe It is possible to operate on a set-by-set basis.

If the luminance change does not exceed the threshold value through the luminance change using the second key frame 730 in the second sub-frame set 720 set in FIG. 7 , the logo area is identified through this and not the entire frame set 310. A logo area may be selected using a difference in luminance of the predetermined second sub-frame set 720 . Accordingly, a shorter frame than using the luminance change of the entire frame set 310 can be considered, and then, the processor 170 performs the operation of each second sub-frame set 720 , so that the luminance changes like the background part. The part is filtered so that only the desired logo area can be selected. However, as in the previous embodiment, the second number and the third number, which are the number of frames of the first subframe set and the second subframe set, are not necessarily limited to constant numbers.

According to an embodiment of the present invention, by configuring a frame in the form of a time pyramid, noise removal is more effective than when only a subframe set of one stage is used, and it has a robust property in detecting a logo area.

In addition, the logo and logo area mentioned throughout this specification are one example in which pixel deterioration may occur, and the present invention is not limited thereto. It will be applicable to all embodiments that cause .

Claims

In an electronic device,

a display;

Divide the input image into a plurality of frame sets having a first number of frames,

dividing a first number of frames included in the frame set into a plurality of subframe sets having a second number of frames;

Identifying a predefined keyframe among frames included in each subframe set for each of the plurality of subframe sets,

and a processor configured to identify a region in the image in which the luminance change of the set of frames is less than a threshold value, based on the luminance change between the frames using the identified keyframe.
According to claim 1,

The processor is

An electronic device for identifying a luminance change of the frame set based on a luminance change between the keyframe included in the subframe set and the other frame.
3. The method of claim 2,

The processor is

An electronic device for identifying a luminance change of the set of frames based on a sequential luminance change between the keyframe and the frame subsequent to the keyframe.
According to claim 1,

The keyframe is an electronic device that is a first frame of the set of subframes including the keyframe.
According to claim 1,

The processor is

An electronic device for identifying a luminance change of the set of frames based on a luminance change between the keyframes.
According to claim 1,

The processor is configured to identify a luminance change of the frame set based on a calculation between luminance changes in the plurality of subframe sets.
According to claim 1,

The frame set is a first frame set,

The processor is

An electronic device for identifying a region in the image based on a luminance change of a second frame set in a section moved by a predefined number of subframe sets in the section of the first frame set.
8. The method of claim 7,

The processor is

identify a region in the image based on a change in luminance of the first frame set;

An electronic device for identifying a change in the region based on a change in luminance of the second frame set based on the identified region.
8. The method of claim 7,

The processor is configured to identify a region in the image based on a calculation between luminance changes of the first frame set and the second frame set.
According to claim 1,

the subframe set is a first subframe set, the keyframe is a first keyframe,

The processor is

dividing a second number of frames included in the subframe set into a plurality of second subframe sets having a third number of frames;

identifying a predefined second keyframe among frames included in each second subframe set for each of the plurality of second subframe sets;

An electronic device for identifying a region in the image based on a change in luminance between frames using the identified second key frame.
In the control method of an electronic device,

dividing the input image into a plurality of frame sets having a first number of frames;

dividing a first number of frames included in the frame set into a plurality of subframe sets having a second number of frames;

identifying a predefined keyframe among frames included in each subframe set for each of the plurality of subframe sets; and

and identifying a region in the image in which the luminance change of the frame set is less than a threshold value based on the luminance change between the frames using the identified keyframe.
12. The method of claim 11,

The step of identifying a region in the image comprises:

and identifying a luminance change of the frame set based on a luminance change between the keyframe included in the subframe set and the other frame.
13. The method of claim 12,

The step of identifying the luminance change of the frame set comprises:

and identifying a luminance change of the set of frames based on a sequential luminance change between the keyframe and the frames following the keyframe.
12. The method of claim 11,

The keyframe is a first frame of the set of subframes including the keyframe.
A computer-readable code, comprising: a computer program storing a code for performing a control method of an electronic device;

dividing the input image into a plurality of frame sets having a first number of frames;

dividing a first number of frames included in the frame set into a plurality of subframe sets having a second number of frames;

identifying a predefined keyframe among frames included in each subframe set for each of the plurality of subframe sets; and

based on the luminance change between the frames using the identified keyframe, identifying a region in the image in which the luminance change of the set of frames is less than a threshold value. recording medium.