KR20170018562A - Digital device and method of processing data the same - Google Patents
Digital device and method of processing data the same Download PDFInfo
- Publication number
- KR20170018562A KR20170018562A KR1020150112338A KR20150112338A KR20170018562A KR 20170018562 A KR20170018562 A KR 20170018562A KR 1020150112338 A KR1020150112338 A KR 1020150112338A KR 20150112338 A KR20150112338 A KR 20150112338A KR 20170018562 A KR20170018562 A KR 20170018562A
- Authority
- KR
- South Korea
- Prior art keywords
- data
- roi
- text data
- interest
- signal
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/4728—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
- H04N21/42204—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/47202—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for requesting content on demand, e.g. video on demand
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
- H04N21/47205—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/81—Monomedia components thereof
- H04N21/816—Monomedia components thereof involving special video data, e.g 3D video
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Human Computer Interaction (AREA)
- Databases & Information Systems (AREA)
- Controls And Circuits For Display Device (AREA)
Abstract
A digital device and a method for processing data in the digital device are disclosed herein. Here, in the digital device for processing data according to an exemplary embodiment of the present invention, the digital device includes a receiver for receiving the content, a decoder for decoding the received content, a region of interest (ROI) selection signal of the reproduced content Identifying the region of interest from the image frame of the reproduced content according to the region of interest selection signal, extracting first text data from the identified region of interest, extracting the extracted region of interest, And to display the restored first ROI data on an image frame of the content. The control unit and the display unit control the first ROI data and the second ROI data, respectively.
Description
BACKGROUND OF THE
A mobile device such as a smart phone, a tablet PC, a wearable device and the like, in addition to a standing device such as a personal computer (PC) or a television (TV) Let this snow. Conventionally, the fixed device and the mobile device have conventionally developed their own areas and have been developed individually. However, the area has become obscure due to the recent boom of digital convergence, and various services are provided through data communication with each other ought.
When the content is being played back on a digital device, the user may experience distorted or fast scene changes such as occluded or blind or blurred objects, fast scene change, text size, There are many cases in which the text or the like included in the content being played back is not correctly recognized or missed due to various reasons such as perspectivity, resolution, and color. This inconveniences the user who watches the content, and it can be difficult to accurately understand the content or accurately catch the desired portion.
In order to solve the above inconvenience, the present invention discloses a digital device and a method for processing data in the digital device.
An object of the present invention is to process an image, text, and the like included in the content so that the user can identify the content in a content reproduction process.
Another object of the present invention is to provide data corresponding to the image, text, and the like even when identification of all or a part of images, texts, etc. contained in the content is not easy.
The technical problem to be solved by the present invention is not limited to the above-described technical problems and other technical problems which are not mentioned can be clearly understood by those skilled in the art from the following description .
This document discloses various embodiments (s) of digital devices and processing methods in the digital devices.
A method of processing data in a digital device according to an embodiment of the present invention includes the steps of reproducing content, receiving a region of interest (ROI) selection signal of the reproduced content, Extracting first text data from the identified region of interest; reconstructing image data corresponding to first text data of the extracted ROI into first ROI data; 1 ROI data on an image frame of the content.
In a digital device for processing data according to an exemplary embodiment of the present invention, a digital device for processing data includes a receiver for receiving content, a decoder for decoding the received content, a user for receiving a region of interest (ROI) An interface unit for reproducing the content, identifying the region of interest from an image frame of the reproduced content according to the region of interest selection signal, extracting first text data from the identified region of interest, 1 text data to first ROI data, and to display the restored first ROI data on an image frame of the content, and a display unit.
The technical solutions obtained by the present invention are not limited to the above-mentioned solutions, and other solutions not mentioned are clearly described to those skilled in the art from the following description. It can be understood.
The effects of the present invention are as follows.
According to one embodiment of the various embodiments of the present invention, the digital device has the effect of being able to process the image, text, and the like contained in the content during the reproduction of the content so that the user can identify it.
According to another embodiment of the various embodiments of the present invention, the digital device provides data corresponding to the image, text, and the like even when identification of all or a part of images, text, etc. contained in the content is not easy There is an effect that can be.
The effects obtained by the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those skilled in the art from the following description will be.
1 schematically illustrates a service system according to an embodiment of the present invention;
2 is a block diagram illustrating a digital device according to one embodiment of the present invention;
FIG. 3 is a block diagram of another configuration or detailed configuration of FIG. 2; FIG.
4 is a block diagram illustrating an external device according to one embodiment of the present invention;
5 is a block diagram illustrating a digital device or external device according to another embodiment of the present invention;
6 illustrates control means for digital device control according to an embodiment of the present invention;
7 is a flowchart of a method of processing image frame data in a digital device according to the present invention;
Figures 8-18 illustrate a method for processing distorted ROI data in an image frame in accordance with the present invention; And
Figures 19 to 21 are flowcharts of a method of processing data in a digital device according to an embodiment of the present invention.
Hereinafter, various embodiments (s) of a digital device and a data processing method in the digital device according to the present invention will be described with reference to the drawings.
The suffix "module "," part ", etc. relating to components used in the present specification are given only for ease of specification, and both may be used as needed. Also, even when described in ordinal numbers such as " 1st ", "2nd ", and the like, it is not limited to such terms or ordinal numbers. In addition, although the terms used in the present specification have been selected from the general terms that are widely used in the present invention in consideration of the functions according to the technical idea of the present invention, they are not limited to the intentions or customs of the artisan skilled in the art, It can be different. However, in certain cases, some terms are arbitrarily selected by the applicant, which will be described in the related description section. Accordingly, it should be understood that the term is to be interpreted based not only on its name but on its practical meaning as well as on the contents described throughout this specification. It is to be noted that the contents of the present specification and / or drawings are not intended to limit the scope of the present invention.
According to the present invention, in accordance with the present invention, a digital device describes a digital device that processes an image, text, and the like included in the content in the course of content reproduction so that the user can identify the content. The digital device can provide data corresponding to the image, text, and the like even when the digital device is not easy to identify all or a part of images, texts, etc. contained in the content. The content includes all kinds of processing, reproduction, provision, and the like in a digital device such as a broadcast program, a still image such as a photograph or a picture, a moving image, a web page, and the like.
As used herein, a " digital device " includes all devices that perform at least one of, for example, receiving, processing, outputting, The digital device can stream or download information about the content or the content through a server such as a broadcasting station, external input, or the like. The digital device can transmit / receive data including the content and the server through a wire / wireless network. The digital device may be either a standing device or a mobile device (handheld device). The fixed device may include a network TV, an HBBTV, a smart TV, an IPTV, a PC, and the like. The mobile device may be a personal digital assistant (PDA), a smart phone, a tablet PC, a notebook, a digital broadcasting terminal, a portable multimedia player (PMP), a navigation, A slate PC, an Ultrabook, a wearable device (e.g., a smart watch, a glass glass, a head mounted display (HMD), etc.) . 2 and 3 are digital TVs, one of the fixed devices, and FIGS. 4 and 5 show mobile terminals and wearable devices (for example, smart watches) as one example of a digital device. And will be described in detail in the corresponding section. In the case where the digital device is a fixed type device, the digital device may be in the form of a sign including only a display panel, or may be set in another configuration, for example, a set-top box (STB) (SET).
On the other hand, the wired / wireless network described herein includes all the hardware and / or software for connection or pairing, data communication, etc. between the server and the digital device described above, It also includes supported and future networks. Meanwhile, the wired / wireless network may support one or more communication protocols for data communication. Such a wired / wireless network includes, for example, a universal serial bus (USB), a composite video banking sync (CVBS), a component, an S-video (analog), a digital visual interface (DVI) (WLAN), Wi-Fi direct, RFID, and the like for a wire connection such as an RGB, a D-SUB, and a communication standard or protocol therefor, (Radio Frequency Identification), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, DLNA (Digital Living Network Alliance), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access) A high speed downlink packet access (HSDPA), a long term evolution (LTE-A), or the like, and a communication standard or protocol for the wireless connection.
On the other hand, a digital device can be a general-purpose OS (Operating System), a web OS (OS), or the like. The digital device can be added by adding various services or applications to a general-purpose OS kernel, a Linux kernel, ), Deleting, amending, updating, etc., thereby providing a more user-friendly environment.
1 is a schematic diagram illustrating a service system according to an embodiment of the present invention.
Referring to FIG. 1, a service system may be basically implemented including a
In a digital device for processing data according to an exemplary embodiment of the present invention, a digital device for processing data includes a receiver for receiving content, a decoder for decoding the received content, a user for receiving a region of interest (ROI) An interface unit for reproducing the content, identifying the region of interest from an image frame of the reproduced content according to the region of interest selection signal, extracting first text data from the identified region of interest, 1 text data to first ROI data, and to display the restored first ROI data on an image frame of the content, and a display unit.
2 is a block diagram illustrating a digital TV according to an embodiment of the present invention.
The
The
The TCP /
The
The
The
The audio /
The application manager includes, for example, a
The
The
The
The
The
The SI &
Meanwhile, the
FIG. 3 is a block diagram of another configuration or detailed configuration of FIG. 2;
3A, the digital TV includes a
The
The
The external
The A / V input / output section includes a USB terminal, a CVBS (Composite Video Banking Sync) terminal, a component terminal, an S-video terminal (analog), a DVI (digital) Visual Interface) terminal, an HDMI (High Definition Multimedia Interface) terminal, an RGB terminal, a D-SUB terminal, and the like.
The wireless communication unit can perform short-range wireless communication with another digital device. Digital TVs are used for communication such as, for example, Bluetooth, Radio Frequency Identification (RFID), infrared data association (IrDA), Ultra Wideband (UWB), ZigBee, DLNA (Digital Living Network Alliance) Depending on the protocol, it can be networked with other digital devices.
Also, the external
The
In addition, the
The
The user
The
Although not shown in the figure, the digital TV may further include a channel browsing processing unit for generating a channel signal or a thumbnail image corresponding to an external input signal. The channel browsing processing unit receives a stream signal TS output from the
The
In order to detect the user's gesture, a sensing unit (not shown) having at least one of a touch sensor, a voice sensor, a position sensor, and an operation sensor may be further included in the digital TV, as described above. A signal sensed by a sensing unit (not shown) may be transmitted to the
The power supply unit 340 supplies the corresponding power source throughout the digital TV. Particularly, it is possible to supply power to a
The
The digital TV described above may be a digital broadcast receiver capable of processing digital broadcast signals of a fixed or mobile ATSC scheme or DVB scheme. In addition, the digital TV according to the present invention may further include a configuration which is not shown or which is not reversed, as needed, among the configurations shown. On the other hand, the digital TV does not have a tuner and a demodulator but may receive and reproduce the content through the network interface unit or the external device interface unit.
3B, an example of the control unit includes a
The
The image processing unit performs image processing of the demultiplexed video signal. To this end, the image processing unit may include a
The
The
A frame rate converter (FRC) 380 converts a frame rate of an input image. For example, the frame
The
On the other hand, the voice processing unit (not shown) in the control unit can perform the voice processing of the demultiplexed voice signal. Such a voice processing unit (not shown) may support processing various audio formats. For example, even when a voice signal is coded in a format such as MPEG-2, MPEG-4, AAC, HE-AAC, AC-3, or BSAC, a corresponding decoder can be provided. In addition, the voice processing unit (not shown) in the control unit can process the base, the treble, the volume control, and the like. A data processing unit (not shown) in the control unit can perform data processing of the demultiplexed data signal. For example, the data processing unit can decode the demultiplexed data signal even when it is coded. Here, the encoded data signal may be EPG information including broadcast information such as a start time and an end time of a broadcast program broadcasted on each channel.
Meanwhile, the above-described digital TV is an example according to the present invention, and each component can be integrated, added, or omitted according to specifications of a digital TV actually implemented. That is, if necessary, two or more components may be combined into one component, or one component may be divided into two or more components. In addition, the functions performed in each block are intended to illustrate the embodiments of the present invention, and the specific operations and devices thereof do not limit the scope of rights of the present invention. On the other hand, the digital TV may be a video signal processing device that performs signal processing of an image stored in the device or an input image. Other examples of the video signal processing device include a set-top box (STB) excluding the
4 is a block diagram illustrating a mobile terminal according to an embodiment of the present invention.
4, the
The
The
The
The
The short-
The
The A /
The image frame processed by the
The
The user input unit 430 generates input data for the user's operation control of the terminal. The user input unit 430 may include a key pad, a dome switch, a touch pad (static / static), a jog wheel, a jog switch, and the like.
The
The
The
The
Some of these displays may be transparent or light transmissive so that they can be seen through. This can be referred to as a transparent display, and a typical example of the transparent display is TOLED (Transparent OLED) and the like. The rear structure of the
There may be two or
The
The touch sensor may be configured to convert a change in a pressure applied to a specific portion of the
If there is a touch input associated with the touch sensor, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and transmits the corresponding data to the
A
Examples of the proximity sensor include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. And to detect the proximity of the pointer by the change of the electric field along the proximity of the pointer when the touch screen is electrostatic. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.
Hereinafter, for convenience of explanation, the act of recognizing that the pointer is positioned on the touch screen while the pointer is not in contact with the touch screen is referred to as "proximity touch & The act of actually touching the pointer on the screen is called "contact touch. &Quot; The position where the pointer is proximately touched on the touch screen means a position where the pointer is vertically corresponding to the touch screen when the pointer is touched.
The proximity sensor detects a proximity touch and a proximity touch pattern (e.g., a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, a proximity touch movement state, and the like). Information corresponding to the detected proximity touch operation and the proximity touch pattern may be output on the touch screen.
The
The
The
The
The
The
The identification module is a chip for storing various information for authenticating the use right of the
When the
The
The
The various embodiments described herein may be implemented in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.
According to a hardware implementation, the embodiments described herein may be implemented as application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays May be implemented using at least one of a processor, a controller, micro-controllers, a microprocessor, and an electrical unit for performing other functions. In some cases, May be implemented by the
According to a software implementation, embodiments such as the procedures and functions described herein may be implemented with separate software modules. Each of the software modules may perform one or more of the functions and operations described herein. Software code may be implemented in a software application written in a suitable programming language. Here, the software code is stored in the
On the other hand, a wearable device that can be worn on the body beyond the dimension that the user mainly grasps and uses can operate or function as a digital device or an external device in this specification. These wearable devices include smart watch, smart glass, and head mounted display (HMD).
As shown in Fig. 1, the wearable device can mutually exchange (or interlock) data with another device. The short
5 is a block diagram illustrating a digital device or external device in accordance with another embodiment of the present invention.
5, a watch-type mobile terminal, that is, a
The
The
A
The
The
On the other hand, the
The
6 is a diagram illustrating control means for digital device control according to an embodiment of the present invention.
A front panel (not shown) or a control means (input means) provided on the
The control means includes a
The input means may be a communication protocol such as Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (UDA), Ultra Wideband (UWB), ZigBee, Digital Living Network Alliance (DLNA) At least one can be employed as needed to communicate with the digital device.
The
The
Since the
The control means such as the
Hereinafter, various embodiments of a method of processing data included in an image frame constituting content in a digital device according to the present invention will be described in detail with reference to the accompanying drawings. Hereinafter, a data processing method related to a selected or zoomed area (hereinafter referred to as " zoomed ") within the corresponding image frame during the reproduction of the content will be described in order to facilitate understanding of the present invention and for convenience of explanation . However, the present invention is not limited to this, and it is applicable to data included in the image frame regardless of the selection or zoom.
Meanwhile, as described above, the content according to the present invention may include a still image as well as a moving picture. In other words, the content may include all types of data.
Alternatively, the content may consist of one or more scenes and each scene may consist of one or more image frames. Here, the image frame may include at least one of a background, an object, an image, and text.
7 is a flowchart illustrating a method of processing image frame data in a digital device according to the present invention.
Referring to FIG. 7, in operation S702, the digital device determines whether there is a region of interest (ROI) according to a zoom request signal in an image frame for the content (S704).
If it is determined in step S704 that the ROI exists, the digital device extracts data (hereinafter referred to as ROI data) such as an image and text in the ROI (S706). Here, the step S706 may be performed irrespective of the step S704 described above. In other words, during the reproduction of the content, data such as images, text, etc. may be automatically extracted in the image frame for the content according to the setting and the like.
In step S708, the digital device determines whether all or a part of the ROI data extracted in step S706 is blinded or distorted by another object or the like (hereinafter referred to as 'distortion').
If it is determined in step S708 that all or part of the extracted ROI data is distorted, the distorted part is retrieved or decoded (step S714).
As in step S714, the ROI data restoration method will be described herein with reference to the following schemes. One is a method of comparing an image frame containing distorted ROI data with other image frame (s) in the content (S710). For example, in contrast to the ROI data extracted from the previous one or more image frames of the image frame, data for reconstruction of the distorted portion of the ROI data is obtained. At this time, the other image frame (s) may include not only previous image frames but also image frame (s) to be reproduced later, as described above. And the other is a method of receiving external data with respect to the extracted ROI data (S712). Transmits the extracted ROI data to an external server, and obtains data necessary for restoring the distorted ROI data from the external server. In FIG. 7, for example, a method of receiving external data after comparing with other image frames has been exemplified and described. However, the methods may be performed in reverse order or in parallel. Alternatively, only one of the two schemes may be used. In addition, information related to ROI data of the image frame can be used by referring to service information of the content being reproduced or the like in other ways. For example, the digital device extracts and parses information on ROI data of each image frame, determines whether the ROI data is distorted by an object in the image frame, and if it is determined that there is distortion, And to restore the ROI data based on the parsed information. However, this may be used when there is service information regarding content being played back on the server side, and may be performed in preference to or in parallel with the above method (s).
If it is determined in step S708 that there is no distortion in the extracted ROI data, the extracted ROI data is restored in step S714, and the restored ROI data is output in step S716.
The digital device outputs the restored data on the screen (S716).
Hereinafter, embodiments of a specific processing algorithm will be described with reference to a method of processing image frame data of the content of FIG. 7 according to an embodiment of the present invention.
8-18 are diagrams illustrating a method for processing distorted ROI data in an image frame in accordance with the present invention.
Fig. 8 is an embodiment relating to changing the shape of the ROI data.
Assume that an ROI exists in the image frame of the content being reproduced, and the ROI data is output as shown in FIG. 8A. However, although not shown, all or some of the ROI data in FIG. 8A may be distorted and difficult for a user to identify.
In this manner, it is required to process the ROI data so that the ROI data can be discriminated by the user when there is distortion. Hereinafter, Figs. 8B to 8I are processing methods of ROI data to correspond to distortion of the ROI data.
8B to 8I are for determining the cause of the distortion in the ROI data within the ROI, that is, the object, etc., and solving the distortion. For example, if the middle portion of the ROI data is partially distorted by the object, the digital device may be based on the degree of distortion and the object in any of the Figures 8b-8d, 8h, or 8i so that the distortion can be resolved And output the ROI data. Alternatively, if there is distortion on the edge side such as the beginning or the end of the ROI data, the ROI data may be output based on the degree of distortion of the object in any one of FIGS. 8E, 8G, and 8H.
Each of the examples shown in FIG. 8 may be implemented in various forms in consideration of the degree of distortion and the object, in addition to the form shown and described as illustrated to facilitate understanding of the present invention.
Although not shown, in FIG. 8A, it is possible to output all or a part of the ROI data in which the distortion occurs, to an upper layer than the object causing the distortion, or to output the remaining portion except for the object portion in the ROI. In the former case, in order to output all or a part of the ROI data to an upper layer, the ROI data may be implemented in 3D form and provided. In addition, the ROI data of 8b to 8i may be automatically or manually scrolled up and down, right and left depending on the type of data.
Fig. 9 shows an example of the processing when the ROI is a part of the image frame.
Referring to FIG. 9A,
The digital device extracts the
10 and 11 illustrate an example of a processing method in a case where a part of the ROI data is distorted by the object itself or the characteristic although the output is performed in the image frame.
Referring to FIG. 10A, the original data in the
Referring to FIG. 10B, when data in the
On the other hand, FIG. 11 shows an example of a processing method when the user is unable to identify the
In the description of FIGS. 10 and 11, the above description has exemplified a method of referring to other image frame (s) of content for ROI data processing in the corresponding image frame. However, the present invention is not limited thereto as described above. For example, in a case where it is difficult or difficult to obtain information about distorted data in the ROI data by the image frame comparison method, or in place of the image frame comparison method, external data or service information may be used as described above .
In the case of restoring the distorted ROI data using the external data, the digital device may store the ROI data including the entire ROI data, the content title, the attribute, and the genre excluding the distorted part, such as the service server, the web server, Information about the object such as information about the object, the information about the object, the information about the object, the information about the object, the information about the object, the environment, and the surrounding environment, and receives and restores the external data from the server. In this case, when a plurality of external data is received, it is also possible to output on the screen all of the configurable ROI data determined by the digital device or based on the plurality of external data.
Figs. 12 to 14 show embodiments of the present invention in the case where there is no distortion of the ROI data but the identification thereof is not easy.
Referring to FIG. 12A, the
13, if it is determined that the resolution of the ROI is lower than a predetermined standard, it is not easy for the user to identify the ROI. Therefore, ROI data are processed separately. The digital device may distort the
In the present embodiment, whether or not the ROI is easy to identify by the user can be determined by determining whether the ROI is within a predetermined reference range, referring to the color of the ROI, the background screen, In this case, if necessary, distortion of the ROI data may also be determined and applied as in the other embodiments.
12, the digital device may not only correct the resolution, color, etc. of the ROI itself, but also provide ROI data corresponding to the ROI on the screen, as in the above embodiment, as shown in FIG. 12B.
FIG. 14 shows a method of processing the ROI according to the present invention when the ROI is not easily discriminated on the basis of the perspective method or the like, although there is no distortion.
14A, when the
The digital device provides the
The digital device provides the
The digital device executes a web browser to provide related functions when the corresponding
Fig. 15 is an embodiment related to processing when it is difficult to output all the ROI data all at once on the screen.
For example, if the digital device determines that the identification of the
In addition, various embodiments using the same or similar method as the technical idea of the present invention may be included in the scope of the present invention in connection with the present invention not shown or not described herein.
16 and 17 illustrate examples of the method for processing the caption data of the content. Here, the caption data includes caption data of a text form, subtitle of an image form, and the like. On the other hand, such caption data is transmitted together with the content, processed and provided.
Referring to FIG. 16A, the digital device provides caption data 1610 at the bottom of the screen. At this time, it is assumed that the subtitle data 1610 includes two or more languages. If the
Referring to FIG. 17A, the digital device provides
If there is caption data in the received content, the digital device can process and store the caption data. Meanwhile, the digital device can map the image data among the caption data stored in the storage unit by referring to the image data stored in advance in the storage unit. Then, at the time of providing the caption, either automatically or at the request of the user, the digital device can replace or convert some text data of the caption data into an image and provide it as shown in FIG. 17B. On the other hand, when the caption data is processed and stored in advance, the digital device adds a mapping image by referring to external data through data communication with an external server or the like, even if there is no previously mapped image data, The image processing of FIG.
Referring to FIG. 17, when a mapping image related to 'apples' and 'bananas' text data in the caption data is previously stored in the digital device, the digital device provides an apple image and a banana image in place of the text data . This allows digital devices to understand subtitle data that is not easy to read intuitively and quickly through the image.
Fig. 18 shows another embodiment for the case where there is no distortion of the ROI but the identification is not easy.
Referring to FIG. 18A, the
If the ROI data is extracted and the extracted ROI data is text data, the digital device may directly restore the extracted ROI text data and output the
On the other hand, when the ROI data is extracted and the extracted ROI data is text data, the digital device does not directly restore the extracted ROI text data as shown in FIG. 18B, but also stores the image data corresponding to the extracted text data Is present. In this case, the digital device can refer to the mapping image data stored in advance in the storage unit, the external image data through the external server, or the like. The digital device provides the
On the other hand, if the image data corresponding to the extracted text data exists, the digital device does not separately provide the ROI data, and if the user places the pointer again in the ROI after selecting the ROI as described above, , The image may be temporarily replaced with the ROI. And moving the pointer back from the ROI to another location, the provided image may disappear again.
On the other hand, if the ROI image data corresponding to the ROI text data is provided in the present specification, the digital device moves to the homepage according to the selection in the manner shown in FIG. 14 when the ROI image data is selected, Related functions such as search, related content recommendation, discount coupon, order page, etc. may be executed or related data may be provided. Also, if ROI image data is provided, the digital device may provide a list of pre-printed or previously stored ROI image data on the screen.
It has been described herein that the ROI is determined by a selection regarding a partial area within an image frame. Here, the selection is not limited to voice, gesture, hot-key, and the like, without necessarily selecting through a pointer or the like (OK button pressing, hovering, And a function key such as a key. For example, when a content is being played back, a predetermined area is pressed through a pointer provided on the screen through a remote controller, hovering or the like is performed, or in the case of a function key such as voice, gesture or hot- The ROI data may be provided while stopping and continuously reproducing the ROI data.
Figures 19 to 21 are flowcharts of a method of processing data in a digital device according to an embodiment of the present invention.
Referring to FIG. 19, the digital device reproduces the content (S1902) and receives a region of interest (ROI) selection signal of the reproduced content (S1904).
The digital device identifies the region of interest from the image frame of the content (S1906) and extracts the first text data from the identified region of interest (S1908).
The first device stores the first ROI data corresponding to the first text data of the extracted ROI. According to one embodiment of the present invention, the digital device determines that the image stored in the content, Text and the like can be processed so as to be identifiable by the user.
According to another embodiment of the various embodiments of the present invention, the digital device provides data corresponding to the image, text, and the like even when identification of all or a part of images, text, etc. contained in the content is not easy (S1910), and displays the restored first ROI data on the image frame of the content (S1912).
The digital device receives a signal for selecting the displayed first ROI data (S1914), performs an additional function corresponding to the first ROI data or displays additional data according to the first ROI data selection signal (S1916) .
For example, FIG. 20 illustrates a detailed process of extracting the first text data extracted in step S1908 and restoring the first ROI data in step S1910.
First, the digital device determines whether the extracted first text data of the ROI is distorted in step S1908 (S2002). Here, the distortion may include a case where all or a part of the first text data of the extracted region of interest is at least one of occluded, blurred, or unidentifiable by the object.
Hereinafter, when the first text data of the extracted ROI is distorted, the digital device processes it as follows.
The digital device extracts second text data for an area corresponding to the area of interest from one or more previous reproduction image frames of the image frame (S2004), and based on the extracted second text data, The third text data corresponding to the distorted part of the one-text data is obtained (S2006). The digital device reconstructs the second ROI data by combining the obtained third text data with the first text data (S2008), and displays the restored second ROI data together with the first ROI data (S2016 ).
Alternatively, the digital device may transmit the distorted first text data to the external server (S2010), receive the external data corresponding to the distorted first text data from the external server (S2012) Data is combined with the first text data to restore the second ROI data (S2014), and the restored second ROI data is displayed together with the first ROI data (S2016).
The second ROI data may be scrolled vertically and / or horizontally. The second ROI data may be scrolled vertically and / or horizontally, and the resolution or color of the ROI may be within a predetermined range as a result of comparison between the background image of the image frame, , The resolution and color of the ROI can be changed and displayed. Also, the first ROI data may be displayed through a window displayed on the region of interest, the region of interest, or a predetermined region on the screen.
21, the digital device according to the present invention identifies (S2102) full text data including first text data extracted from a region of interest of the image frame in accordance with the ROI selection signal (S2102) Text data is extracted (S2104).
The digital device reconstructs the third ROI data based on the extracted whole text data (S2106), and displays the restored third ROI data together with the first ROI data (S2108).
Meanwhile, the signal for selecting the ROI may include a zoom function request signal.
Accordingly, in accordance with the present invention, a digital device can process an image, text, and the like included in the content during the reproduction of the content so that the user can identify it, and can identify the whole or a part of the image, text, It is possible to provide data corresponding to the image, text, and the like.
The digital device disclosed in this specification and the data processing method in the digital device can be applied to a configuration or a method of the embodiments described above in a limited manner, Some of which may be selectively combined.
Meanwhile, the operation method of the digital device disclosed in this specification can be implemented as a code readable by a processor in a recording medium readable by a processor included in the digital device. The processor-readable recording medium includes all kinds of recording devices in which data that can be read by the processor is stored. Examples of the recording medium readable by the processor include ROM (Read Only Memory), RAM (Random Access Memory), CD-ROM, magnetic tape, floppy disk, optical data storage device, And may be implemented in the form of a carrier-wave. In addition, the processor-readable recording medium may be distributed over network-connected computer systems so that code readable by the processor in a distributed fashion can be stored and executed.
While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Of the right. Further, such modifications are not to be understood individually from the technical idea of the present invention.
201: network interface unit 202: TCP / IP manager
203: service delivery manager 204: SI decoder
205
207: Video decoder 208:
209: Service Control Manager 210: Service Discovery Manager
211: SI & Metadata Database 212: Metadata Manager
213: service manager 214: UI manager
Claims (20)
Reproducing the content;
Receiving a region of interest (ROI) selection signal of the reproduced content;
Identifying the region of interest from an image frame of the content and extracting first text data from the identified region of interest;
Reconstructing the image data corresponding to the first text data of the extracted ROI into first ROI data; And
And displaying the restored first ROI data on an image frame of the content.
Further comprising determining whether the first text data of the extracted ROI is distorted,
Wherein the distortion includes a case where all or a part of the first text data of the extracted region of interest is at least one of occluded, blurred, or unidentifiable by an object.
If the first text data of the extracted ROI is distorted,
Extracting second text data for an area corresponding to the region of interest from one or more previous playback image frames of the image frame;
Obtaining third text data corresponding to a distorted portion of the first text data based on the extracted second text data;
Reconstructing second ROI data by combining the obtained third text data with the first text data; And
And displaying the restored second ROI data together with the first ROI data. ≪ Desc / Clms Page number 19 >
If the first text data of the extracted ROI is distorted,
Transmitting the distorted first text data to an external server;
Receiving external data corresponding to the distorted first text data from the external server;
Reconstructing the second ROI data by combining the received external data and the first text data; And
And displaying the restored second ROI data together with the first ROI data. ≪ Desc / Clms Page number 19 >
Wherein the second ROI data is scrolled vertically and / or horizontally.
And displays the resolution or color of the ROI by changing the resolution or color of the ROI if the resolution or color of the ROI falls within a predetermined range as a result of a comparison between the background screen of the image frame and the resolution or color of the object around the ROI. A method for processing data in a device.
Wherein the first ROI data comprises:
Is displayed through a window displayed on the area of interest, the area of interest, or on a predetermined area on the screen.
Receiving a signal for selecting the displayed first ROI data; And
And performing an additional function corresponding to the first ROI data or displaying additional data according to the first ROI data selection signal.
Identifying and extracting full text data including first text data extracted from a region of interest of the image frame in accordance with the ROI selection signal;
Restoring the third ROI data based on the extracted whole text data; And
And displaying the restored third ROI data together with the first ROI data. ≪ Desc / Clms Page number 19 >
Wherein the signal relating to the ROI includes a zoom function request signal.
A receiving unit for receiving the content;
A decoder for decoding the received content;
A user interface unit for receiving a region of interest (ROI) selection signal of a content to be reproduced;
Identifying the region of interest from the image frame of the reproduced content according to the region of interest selection signal, extracting first text data from the identified region of interest, extracting first text data of the extracted region of interest And to display the restored first ROI data on an image frame of the content; And
A digital device comprising a display portion.
Wherein,
Determining whether the first text data of the extracted ROI is distorted,
Wherein the distortion includes a case where all or a part of the first text data of the extracted region of interest is at least one of occluded, blurred, or unidentifiable by the object.
Wherein,
If the first text data of the extracted ROI is distorted,
Extracting second text data for an area corresponding to the region of interest from one or more previous reproduction image frames of the image frame, and extracting a distorted portion of the first text data based on the extracted second text data And restoring the second ROI data by combining the third text data with the first text data, and restoring the restored second ROI data together with the first ROI data So as to be displayed.
Wherein,
And transmitting the distorted first text data to an external server, receiving external data corresponding to the distorted first text data, combining the received external data with the first text data, and restoring the second ROI data And controls the restored second ROI data to be displayed together with the first ROI data.
Wherein,
And controls the second ROI data to be scrolled up and down and / or left and right.
Wherein,
The resolution or color of the ROI is compared with the resolution or color of the background image of the image frame or the ROI of the ROI, and the resolution or color of the ROI is changed and displayed when the resolution or color is within a predetermined range .
Wherein,
Controls to display a window on the area of interest, the area of interest, or a predetermined area on the screen, and controls the display of the first ROI data through the displayed window.
Wherein,
Wherein the controller controls to perform an additional function corresponding to the first ROI data or to display additional data when a signal for selecting first ROI data displayed through the user interface unit is received.
Wherein,
And when the ROI selection signal is received, controlling to identify and extract the entire text data including the first text data extracted from the ROI of the image frame, restoring the third ROI data based on the extracted whole text data, And controls the restored third ROI data to be displayed together with the first ROI data.
Wherein the signal relating to the selection of the region of interest comprises a zoom function request signal.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR1020150112338A KR20170018562A (en) | 2015-08-10 | 2015-08-10 | Digital device and method of processing data the same |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR1020150112338A KR20170018562A (en) | 2015-08-10 | 2015-08-10 | Digital device and method of processing data the same |
Publications (1)
Publication Number | Publication Date |
---|---|
KR20170018562A true KR20170018562A (en) | 2017-02-20 |
Family
ID=58265173
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
KR1020150112338A KR20170018562A (en) | 2015-08-10 | 2015-08-10 | Digital device and method of processing data the same |
Country Status (1)
Country | Link |
---|---|
KR (1) | KR20170018562A (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2019107981A1 (en) * | 2017-11-29 | 2019-06-06 | Samsung Electronics Co., Ltd. | Electronic device recognizing text in image |
-
2015
- 2015-08-10 KR KR1020150112338A patent/KR20170018562A/en unknown
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2019107981A1 (en) * | 2017-11-29 | 2019-06-06 | Samsung Electronics Co., Ltd. | Electronic device recognizing text in image |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
KR102063075B1 (en) | Service system, digital device and method of processing a service thereof | |
US11949955B2 (en) | Digital device and method of processing data in said digital device | |
CN107018434B (en) | System including digital device and external device and method of processing data thereof | |
KR102273751B1 (en) | Digital device and method of processing data the same | |
US10324595B2 (en) | Digital device configured to connect with plurality of operation devices and method of displaying second user interface associated with second operation device in response to dragging of first user interface associated with first operation device in the same | |
KR20120116613A (en) | Image display device and method of managing contents using the same | |
KR102348957B1 (en) | Digital device and method for controlling the same | |
KR20170042086A (en) | Digital device and method of processing data the same | |
KR101873763B1 (en) | Digital device and method of processing data the same | |
KR20180041961A (en) | Display device and method of processing data the same | |
KR20170017606A (en) | Digital device and method of processing data the same | |
KR20150128010A (en) | Display device and method of processing a content thereof | |
KR20170045829A (en) | Digital device and method of processing data the same | |
KR20160009415A (en) | Video display apparatus capable of sharing ccontents with external input appatatus | |
KR20170018562A (en) | Digital device and method of processing data the same | |
KR102369588B1 (en) | Digital device and method of processing data the same | |
KR20170031898A (en) | Display device and method for controlling the same | |
KR20170022333A (en) | Digital device and method of processing data the same | |
KR20170029832A (en) | Digital device and method of processing data the same | |
KR20170030351A (en) | Mobile terminal and method for controlling the same | |
KR20150128009A (en) | Display device and method of processing a content thereof | |
KR102423492B1 (en) | Digital device and method of processing data the same | |
KR102557573B1 (en) | Digital device and method of processing data the same | |
KR20150024198A (en) | Image controlling apparatus and method thereof | |
KR101688149B1 (en) | Apparatus, system and method for transmitting video |