KR20190031829A - Electric terminal and method for controlling the same - Google Patents

Abstract

The present invention relates to an electronic device for providing a subtitle suitable for a language level of a viewer in viewing a video. To this end, the electronic device comprises: a learning data unit for storing learning information learning a language level of a user; a synonym database unit for classifying and storing synonyms according to a plurality of preset language levels; a display unit for displaying an image with a subtitle; and a control unit for controlling the display unit to extract a synonym of at least one word included in the subtitle from the synonym database unit based on the language level of the user, and to output the extracted synonym rather than the at least one word.

Description

[0001] ELECTRIC TERMINAL AND METHOD FOR CONTROLLING THE SAME [0002]

The present invention relates to an electronic device that utilizes artificial intelligence to provide subtitles at the level of a viewer.

Recently, with the development of hardware, artificial intelligence technology which realizes human thinking process, that is, cognition, reasoning, learning, and the like, with computing technology has been dramatically developed.

Artificial intelligence technology is not only itself a research subject, it can be directly or indirectly combined with other fields of computer science to provide various functions. Especially, it is very active to introduce artificial intelligence elements in various fields of information technology and to utilize them in solving problems in the field.

Artificial intelligence is divided into strong artificial intelligence and weak artificial intelligence. Strong artificial intelligence is a technology that can make human-like thinking and decision-making, and it is a level of self-study and self-judgment. Weak artificial intelligence is a level of technology that provides optimal solutions by performing cognitive processes such as perception, reasoning, etc. through a computational model.

As part of this development, artificial intelligence technology has been applied to the most familiar user terminals, and attempts have been made to provide various functions.

In recent years, with the development of communication technologies, there have been many cases in which video contents are watched in a streaming manner. However, existing video contents are provided by superimposing previously stored subtitles on video contents. Therefore, the viewer has a problem that the terms included in the subtitles are determined according to the language level of the producer who made the subtitles, so that the subtitles are provided with the inappropriate language level.

In order to solve the above-described problems, it is an object of the present invention to provide a caption suitable for a language level of a viewer at the time of watching a video.

In a mobile terminal according to an exemplary embodiment of the present invention, a mobile terminal includes a learning data unit for storing learning information for learning a language level of a user, a synonym database unit for storing synonyms classified according to a predetermined plurality of language levels, And extracting a synonym of at least one word among the words included in the caption from the synonym database part based on the language level of the user, And a control unit for controlling the display unit to output the image data.

In one embodiment of the present invention, the controller extracts at least one word of a level higher than a language level of the user among the words included in the caption.

In one embodiment, the control unit classifies words included in the subtitles into different predetermined language levels, and extracts at least one word corresponding to a level higher than a language level of the user among the classified words .

In one embodiment, the control unit extracts a synonym of the at least one word based on a language level corresponding to a caption mode selected by a user among a plurality of caption modes providing captions according to different language levels .

In one embodiment, the control unit learns words included in at least one of a call voice, a message, contents output on a display unit, and a search history by using a machine learning algorithm, and based on the learned words , And determines the language level of the user.

In one embodiment, the apparatus further includes a learning language database unit that stores the learned words, and when the at least one word is included in the learning language database unit, And controls the display unit to output at least one word.

In one embodiment, the synonym database classifies synonyms into a plurality of predetermined language levels based on the frequency of use of words and the age range of words according to the age.

In one embodiment, the control unit outputs a pop-up window indicating that there is no synonym of the at least one word on the display unit when the synonym of the at least one word does not exist in the synonym database. .

In one embodiment, the control unit displays the at least one word so as to be visually distinguished from the remaining words when the at least one word is output on the display unit.

In one embodiment, the control unit outputs semantic information of the at least one word on the display unit in response to a touch input of the at least one word.

In one embodiment, the image processing apparatus may further include a camera configured to photograph a front of the display unit, wherein when the plurality of users located in front of the display unit are detected using the image information captured by the camera, Extracting synonyms of the at least one word based on a lower level of the user's different language levels.

In one embodiment, when the plurality of users located in front of the display unit are detected using the image information photographed by the camera, the control unit controls the at least one Extracts a plurality of synonyms of the words of the words of the sentence, and outputs the extracted plurality of synonyms together on the display unit.

In one embodiment, the control unit extracts the at least one synonym based on a language level of a user satisfying a predetermined condition among the plurality of users.

In one embodiment, the predetermined condition is one of a condition that the level information of the image corresponds to a level of the user's language, and a condition of the main viewer of the image.

In one embodiment, the predetermined condition is one of a condition for holding a remote controller for controlling an electronic device, and a condition that a genre of an image matches a user's taste.

According to another aspect of the present invention, there is provided a method of controlling an electronic device, the method comprising the steps of: displaying an image including a caption; and displaying at least one of words included in the caption, Extracting one word and extracting a synonym of the extracted at least one word from a synonym database part in which synonyms are classified according to a language level; and outputting the synonym including the extracted synonym to the subtitle And the extracted synonym is a word corresponding to the language level of the user.

The present invention can improve the subtitle understanding degree of the viewer who watches the moving picture by changing the words included in the subtitles provided in the moving picture to other synonyms suitable for the language level of the viewer according to the language level of the viewer.

In addition, the present invention can provide a more user-friendly form of the subtitles by learning the words that the viewer learns or frequently using, and by changing the words included in the subtitles by utilizing the learned words.

Referring to Figure 1, Figure 1 is a block diagram illustrating an electronic device associated with the present invention.
2 is a flowchart illustrating a control method of providing a caption in a form suitable for a language level of a viewer at the time of moving picture reproduction in an electronic apparatus according to the present invention.
3A to 3C are conceptual diagrams showing a synonym database part of an electronic device according to the present invention.
4 is a conceptual diagram showing the control method of FIG.
FIG. 5 is a flowchart illustrating a method of providing subtitles using a plurality of subtitle modes providing subtitles at different language levels.
6 is a conceptual diagram showing the control method of FIG.
7 is a flowchart illustrating a method of providing a caption using words used by a viewer.
8 and 9 are conceptual diagrams illustrating a method of providing information of changed words according to a language level of a user.
FIG. 10 is a flowchart illustrating a method of providing a plurality of viewers with language-level subtitles when a plurality of viewers exist for a moving image.
Figs. 11 and 12 are conceptual diagrams showing the control method of Fig.
13 is a conceptual diagram showing a method of searching for a synonym if a synonym does not exist on the synonym database.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like reference numerals are used to designate identical or similar elements, and redundant description thereof will be omitted. The suffix " module " and " part " for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role. In the following description of the embodiments of the present invention, a detailed description of related arts will be omitted when it is determined that the gist of the embodiments disclosed herein may be blurred. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed. , ≪ / RTI > equivalents, and alternatives.

Terms including ordinals, such as first, second, etc., may be used to describe various elements, but the elements are not limited to these terms. The terms are used only for the purpose of distinguishing one component from another.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The singular expressions include plural expressions unless the context clearly dictates otherwise.

In the present application, the terms "comprises", "having", and the like are used to specify that a feature, a number, a step, an operation, an element, a component, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

The mobile terminal described in this specification includes a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, a slate PC A tablet PC, an ultrabook, a wearable device such as a smartwatch, a smart glass, and a head mounted display (HMD). have.

However, it will be appreciated by those skilled in the art that the configuration according to the embodiments described herein may be applied to fixed terminals such as a digital TV, a desktop computer, a digital signage, and the like, will be.

Referring to Figure 1, Figure 1 is a block diagram illustrating an electronic device associated with the present invention.

The electronic device 100 includes a wireless communication unit 110, an input unit 120, a sensing unit 140, an output unit 150, an interface unit 160, a memory 170, a control unit 180, ), And the like. The components shown in FIG. 1 are not essential for implementing an electronic device, so that the mobile terminal described herein may have more or fewer components than the components listed above.

More specifically, the wireless communication unit 110 among the above-described components can communicate with the mobile terminal 100 and the wireless communication system, between the electronic device 100 and another electronic device 100, or between the electronic device 100 and the external server 100. [ Lt; RTI ID = 0.0 > wireless < / RTI > In addition, the wireless communication unit 110 may include one or more modules that connect the electronic device 100 to one or more networks.

The wireless communication unit 110 may include at least one of a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short distance communication module 114, and a location information module 115 .

The input unit 120 includes a camera 121 or an image input unit for inputting a video signal, a microphone 122 for inputting an audio signal, an audio input unit, a user input unit 123 for receiving information from a user A touch key, a mechanical key, and the like). The voice data or image data collected by the input unit 120 may be analyzed and processed by a user's control command.

The sensing unit 140 may include one or more sensors for sensing at least one of the information in the electronic device, the surrounding information surrounding the electronic device, and the user information. For example, the sensing unit 140 may include a proximity sensor 141, an illumination sensor 142, a touch sensor, an acceleration sensor, a magnetic sensor, A G-sensor, a gyroscope sensor, a motion sensor, an RGB sensor, an infrared sensor, a finger scan sensor, an ultrasonic sensor, A microphone 226, a battery gauge, an environmental sensor (for example, a barometer, a hygrometer, a thermometer, a radiation detection sensor, A thermal sensor, a gas sensor, etc.), a chemical sensor (e.g., an electronic nose, a healthcare sensor, a biometric sensor, etc.). On the other hand, the electronic device disclosed in this specification can combine and use information sensed by at least two of the sensors.

The output unit 150 includes at least one of a display unit 151, an acoustic output unit 152, a haptic tip module 153, and a light output unit 154 to generate an output related to visual, auditory, can do. The display unit 151 may have a mutual layer structure with the touch sensor or may be integrally formed to realize a touch screen. Such a touch screen may function as a user input 123 that provides an input interface between the electronic device 100 and a user and may provide an output interface between the electronic device 100 and a user.

The interface unit 160 serves as a path to various kinds of external devices connected to the electronic device 100. The interface unit 160 is connected to a device having a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, And may include at least one of a port, an audio I / O port, a video I / O port, and an earphone port. In the electronic device 100, corresponding to the connection of the external device to the interface unit 160, it is possible to perform appropriate control with respect to the connected external device.

The memory 170 also stores data that supports various functions of the electronic device 100. The memory 170 may store a number of application programs (application programs or applications) running on the electronic device 100, data for operation of the electronic device 100, and instructions. At least some of these applications may be downloaded from an external server via wireless communication. At least some of these applications may also reside on the electronic device 100 since the time of shipment for the basic functions of the electronic device 100 (e.g., phone call incoming, outgoing, message receiving, origination functions). On the other hand, an application program may be stored in the memory 170, installed on the electronic device 100, and operated by the control unit 180 to perform the operation (or function) of the electronic device.

In addition to the operations associated with the application program, the control unit 180 typically controls the overall operation of the electronic device 100. The control unit 180 may process or process signals, data, information, and the like input or output through the above-mentioned components, or may drive an application program stored in the memory 170 to provide or process appropriate information or functions to the user.

In addition, the controller 180 may control at least some of the components illustrated in FIG. 1A to drive an application program stored in the memory 170. FIG. Further, the controller 180 may operate at least two of the components included in the electronic device 100 in combination with each other for driving the application program.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power to the components included in the electronic device 100. The power supply unit 190 includes a battery, which may be an internal battery or a replaceable battery.

At least some of the components may operate in cooperation with each other to implement an operation, control, or control method of an electronic device according to various embodiments described below. Also, the method of operation, control, or control of the electronic device may be implemented on an electronic device by driving at least one application program stored in the memory 170. [

Hereinafter, the components listed above will be described in more detail with reference to FIG. 1 before explaining various embodiments implemented through the electronic device 100 as described above.

First, referring to the wireless communication unit 110, the broadcast receiving module 111 of the wireless communication unit 110 receives broadcast signals and / or broadcast-related information from an external broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. Two or more broadcast receiving modules may be provided to the mobile terminal 100 for simultaneous broadcast reception or broadcast channel switching for at least two broadcast channels.

The mobile communication module 112 may be a mobile communication module or a mobile communication module such as a mobile communication module or a mobile communication module that uses technology standards or a communication method (e.g., Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), Code Division Multi Access 2000 (Enhanced Voice-Data Optimized or Enhanced Voice-Data Only), Wideband CDMA (WCDMA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution And an external terminal, or a server on a mobile communication network established according to a long term evolution (AR), a long term evolution (AR), or the like.

The wireless signal may include various types of data depending on a voice call signal, a video call signal or a text / multimedia message transmission / reception.

The wireless Internet module 113 is a module for wireless Internet access, and may be embedded in the electronic device 100 or externally. The wireless Internet module 113 is configured to transmit and receive a wireless signal in a communication network according to wireless Internet technologies.

Wireless Internet technologies include, for example, wireless LAN (WLAN), wireless fidelity (Wi-Fi), wireless fidelity (Wi-Fi) Direct, DLNA (Digital Living Network Alliance), WiBro Interoperability for Microwave Access, High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE) and Long Term Evolution-Advanced (LTE-A) 113 transmit and receive data according to at least one wireless Internet technology, including Internet technologies not listed above.

The wireless Internet module 113 for performing a wireless Internet connection through the mobile communication network can be used for wireless Internet access by WiBro, HSDPA, HSUPA, GSM, CDMA, WCDMA, LTE or LTE- May be understood as a kind of the mobile communication module 112.

The short-range communication module 114 is for short-range communication, and includes Bluetooth ™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB) (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technology. The short range communication module 114 may be used to communicate between the electronic device 100 and the wireless communication system via the wireless area networks or between the electronic device 100 and another electronic device 100 or between the electronic device 100 ) And another electronic device 100 (or an external server) are located. The short-range wireless communication network may be a short-range wireless personal area network.

Herein, the other electronic device 100 is a wearable device (e.g., a smartwatch, a smart glass, etc.) capable of interchanging data with the electronic device 100 according to the present invention (smart glass), HMD (head mounted display)). The short range communication module 114 can detect (or recognize) a wearable device capable of communicating with the electronic device 100 around the electronic device 100. [ Further, the control unit 180 may control at least a part of the data processed in the electronic device 100 to be transmitted to the short distance communication module (or the short distance communication module), if the detected wearable device is an authorized device to communicate with the electronic device 100 according to the present invention 114 to the wearable device. Therefore, the user of the wearable device can use the data processed in the electronic device 100 through the wearable device. For example, according to this, the user can make a phone call through the wearable device when a telephone call is received in the electronic device 100, or when the message is received in the electronic device 100, It is possible to check the message.

The position information module 115 is a module for obtaining the position (or current position) of the electronic device, and representative examples thereof include a Global Positioning System (GPS) module or a Wireless Fidelity (WiFi) module. For example, an electronic device can utilize a GPS module to obtain the position of an electronic device using a signal sent from a GPS satellite. As another example, an electronic device can utilize a Wi-Fi module to obtain the location of an electronic device based on information of a wireless access point (wireless AP) that transmits or receives a wireless signal with a Wi-Fi module. Optionally, the location information module 115 may perform any of the other modules of the wireless communication unit 110 to obtain data regarding the location of the electronic device, in addition or alternatively. The location information module 115 is a module used to obtain the location (or current location) of the electronic device, and is not limited to a module that directly calculates or obtains the location of the electronic device.

Next, the input unit 120 is for inputting image information (or signal), audio information (or signal), data, or information input from a user. For inputting image information, Or a plurality of cameras 121 may be provided. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in the video communication mode or the photographing mode. The processed image frame may be displayed on the display unit 151 or stored in the memory 170. [ The plurality of cameras 121 provided in the electronic device 100 may be arranged to form a matrix structure and the electronic device 100 may be provided with various angles or foci through the camera 121 having the matrix structure A plurality of pieces of image information can be input. In addition, the plurality of cameras 121 may be arranged in a stereo structure to acquire a left image and a right image for realizing a stereoscopic image.

The microphone 122 processes the external acoustic signal into electrical voice data. The processed voice data can be utilized variously according to functions (or application programs being executed) being performed in the electronic device 100. Meanwhile, the microphone 122 may be implemented with various noise reduction algorithms for eliminating noise generated in receiving an external sound signal.

The user input unit 123 is for receiving information from a user and when the information is inputted through the user input unit 123, the control unit 180 can control the operation of the electronic device 100 to correspond to the input information . The user input unit 123 may include a mechanical input means (or a mechanical key such as a button located on the front, rear or side of the electronic device 100, a dome switch, a jog wheel, Jog switches, etc.) and touch-type input means. For example, the touch-type input means may comprise a virtual key, a soft key or a visual key displayed on the touch screen through software processing, The virtual key or the visual key can be displayed on the touch screen with various forms. For example, the virtual key or the visual key can be displayed on the touch screen, ), An icon, a video, or a combination thereof.

The running data portion 130 may be configured to receive, classify, store and output information to be used for data mining, data analysis, intelligent decision making, and machine learning algorithms and techniques. The running data unit 130 stores information received, detected, detected, generated, predefined information or other information output through the terminal, received by the terminal, received, detected, detected , And one or more memory units configured to store generated, predefined or output data.

The running data portion 130 may be integrated into an electronic device or may include a memory. In one embodiment, the running data portion 130 may be implemented via the memory 170. The running data portion 130 may be implemented in memory associated with the electronic device 100 (e.g., an external memory connected to the electronic device 100 (either wired or electrically) May be implemented through a memory included in a server that can communicate with the electronic device 100. In another embodiment, the running data portion 130 may be implemented through a memory maintained in a cloud computing environment, or another remote memory accessible by a terminal via a communication method such as a network.

The running data part 130 is used to identify, index, classify, manipulate, store, retrieve and output data for use in supervised or non-supervised learning, data mining, predictive analysis or other machine learning techniques, To one or more databases. The information stored in the running data section 130 may be used by the control section 180 using a different type of data analysis, machine learning algorithm and machine learning techniques, or by a plurality of controls included in the electronic device . Examples of such algorithms and techniques are K nearest neighbor systems, fuzzy logic (for example, possibility theory), neural networks, Boltzmann (Boltzmann such as vector quantization, pulsed neural nets, support vector machines, maximum margin classifiers, hill-climbing, inductive logic systems, But are not limited to, baysian networks, petri nets (e.g., finite state machines, mealy machines, moore finite state machines), classifiers trees (e.g., perceptron trees, support vector trees, markov trees, decision tree forests, random forests), horses All models and systems (pandemon ium models and systems, clustering, artificially intelligent planning, artificially intelligent forecasting, data fusion, sensor fusion, image fusion, Reinforcement learning, augmented reality, pattern recognition, and automated planning.

The control unit 180 may determine or predict at least one executable operation of the electronic device based on information determined or generated using data analysis, machine learning algorithms, and machine learning techniques. To this end, the control unit 180 can request, search, receive, or utilize the data of the running data unit 130. The controller 180 may perform various functions for implementing a knowledge-based system, an inference system, a knowledge acquisition system, and the like, and may include a system (e.g., a fuzzy logic system) for uncertain reasoning, an adaptation system, Neural networks, and the like.

The control unit 180 may be a sub-module that enables voice and natural language processing such as an I / O processing module, an environmental condition module, a voice-text (STT) processing module, a natural language processing module, a workflow processing module, Lt; / RTI > Each of the sub-modules may have access to one or more systems or data and models in the electronic device, or a subset or superset thereof. Here, the object to which each of the submodules has access rights may include a scheduling, a vocabulary index, a user data, a task flow model, a service model, and an automatic speech recognition (ASR) system. In another embodiment, the controller 180 or the electronic device may be implemented with the submodule, system, or data and model.

In some embodiments, based on the data in the running data portion 130, the controller 180 may be configured to detect and detect what the user requires based on contextual conditions or intentions of the user represented by user input or natural language input Lt; / RTI > In addition, the control unit 180 can actively derive or acquire the information necessary to completely determine what the user requests in accordance with the context condition or the user's intention. For example, the control unit 180 can detect and sense what the user requests by analyzing historical data including past input and output, pattern matching, unambiguous words, input intentions, and the like. In addition, the control unit 180 can determine a work flow for executing a function requested by a user according to a context condition or a user's intention. In addition, the control unit 180 can execute a workflow for satisfying a user's request based on a context condition or a user's intention.

In some embodiments, the controller 180 may be implemented as a microprocessor, such as a memory, a memristors, a transconductance amplifier, a pulsed neural circuit, an artificial intelligent nanotechnology system (e.g., an autonomous nano machine), or an artificial intelligent quantum mechanical system A neural network), and the like. In some embodiments, the controller 180 may include a pattern recognition system, such as a machine vision system, an acoustic recognition system, a handwriting recognition system, a data fusion system, a sensor fusion system, and a soft sensor. The machine vision system may include content based image retrieval, optical character recognition, augmented reality, egomotion, tracking and optical flow, and the like.

The control unit 180 can sense or receive information in the electronic device, surrounding information surrounding the electronic device, or other information through the sensing unit 140. [ In addition, the controller 180 can receive broadcast signals and / or broadcast-related information, wireless signals, wireless data, and the like through the wireless communication unit 110. Also, the control unit 180 may receive image information (or a corresponding signal), audio information (or a corresponding signal), data, or information input from a user, from an input unit.

The controller 180 collects information, processes or classifies the information (e.g., a knowledge graph, a command policy, a personalization database, a dialog engine, etc.), and transmits the processed or classified information to the memory 170 or the running data (130).

Based on the data analysis, the machine learning algorithm, and the machine learning technique, when the operation of the electronic device is determined, the controller 180 controls the components of the electronic device to perform the determined operation. Can be controlled. The control unit 180 can execute the determined operation by controlling the electronic device based on the control command.

In one embodiment, when a specific operation is performed, the controller 180 analyzes history information indicating performance of a specific operation through a data analysis, a machine learning algorithm, and a machine learning technique. Based on the analyzed information, An update to the learned information can be performed. Accordingly, the control unit 180 can improve the accuracy of the future performance of the data analysis based on the updated information, the machine learning algorithm, and the machine learning technology together with the running data unit 130.

Meanwhile, the sensing unit 140 senses at least one of information in the electronic device, surrounding environment information surrounding the electronic device, and user information, and generates a corresponding sensing signal. The control unit 180 may control the driving or operation of the electronic device 100 or perform data processing, function or operation related to the application program installed in the electronic device 100 based on the sensing signal. Representative sensors among various sensors that may be included in the sensing unit 140 will be described in more detail.

First, the proximity sensor 141 refers to a sensor that detects the presence of an object approaching a predetermined detection surface, or the presence of an object in the vicinity of the detection surface, without mechanical contact by using electromagnetic force or infrared rays. The proximity sensor 141 may be disposed in the inner area of the electronic device or the proximity sensor 141 near the touch screen.

Examples of the proximity sensor 141 include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. In the case where the touch screen is electrostatic, the proximity sensor 141 can be configured to detect the proximity of the object with a change of the electric field along the proximity of the object having conductivity. In this case, the touch screen (or touch sensor) itself may be classified as a proximity sensor.

On the other hand, for convenience of explanation, the act of recognizing that the object is located on the touch screen in proximity with no object touching the touch screen is referred to as " proximity touch & The act of actually touching an object on the screen is called a " contact touch. &Quot; The position at which the object is closely touched on the touch screen means a position where the object corresponds to the touch screen vertically when the object is touched. The proximity sensor 141 can detect a proximity touch and a proximity touch pattern (e.g., a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, have. Meanwhile, the control unit 180 processes data (or information) corresponding to the proximity touch operation and the proximity touch pattern sensed through the proximity sensor 141 as described above, and further provides visual information corresponding to the processed data It can be output on the touch screen. Furthermore, the control unit 180 can control the electronic device 100 such that different operations or data (or information) are processed depending on whether the touch to the same point on the touch screen is a proximity touch or a touch contact .

The touch sensor uses a touch (or touch input) applied to the touch screen (or the display unit 151) by using at least one of various touch methods such as a resistance film type, a capacitive type, an infrared type, an ultrasonic type, Detection.

For example, the touch sensor may be configured to convert a change in a pressure applied to a specific portion of the touch screen or a capacitance generated in a specific portion to an electrical input signal. The touch sensor may be configured to detect a position, an area, a pressure at the time of touch, a capacitance at the time of touch, and the like where a touch object touching the touch screen is touched on the touch sensor. Here, the touch object may be a finger, a touch pen, a stylus pen, a pointer, or the like as an object to which a touch is applied to the touch sensor.

Thus, when there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and transmits the corresponding data to the controller 180. Thus, the control unit 180 can know which area of the display unit 151 is touched or the like. Here, the touch controller may be a separate component from the control unit 180, and may be the control unit 180 itself.

On the other hand, the control unit 180 may perform different controls or perform the same control according to the type of the touch object touching the touch screen (or a touch key provided on the touch screen). Whether to perform different controls or perform the same control depending on the type of the touch object can be determined according to the operating state of the current electronic device 100 or an application program being executed.

On the other hand, the touch sensors and the proximity sensors discussed above can be used independently or in combination to provide a short touch (touch), a long touch, a multi touch, a drag touch ), Flick touch, pinch-in touch, pinch-out touch, swipe touch, hovering touch, and the like. Touch can be sensed.

The ultrasonic sensor can recognize the position information of the object to be sensed by using ultrasonic waves. Meanwhile, the controller 180 can calculate the position of the wave generating source through the information sensed by the optical sensor and the plurality of ultrasonic sensors. The position of the wave source can be calculated using the fact that the light is much faster than the ultrasonic wave, that is, the time when the light reaches the optical sensor is much faster than the time the ultrasonic wave reaches the ultrasonic sensor. More specifically, the position of the wave generating source can be calculated using the time difference with the time when the ultrasonic wave reaches the reference signal.

The camera 121 includes at least one of a camera sensor (for example, a CCD, a CMOS, etc.), a photo sensor (or an image sensor), and a laser sensor.

The camera 121 and the laser sensor may be combined with each other to sense a touch of the sensing object with respect to the three-dimensional stereoscopic image. The photosensor can be laminated to the display element, which is adapted to scan the movement of the object to be detected proximate to the touch screen. More specifically, the photosensor mounts photo diodes and TRs (Transistors) in a row / column and scans the contents loaded on the photosensor using an electrical signal that varies according to the amount of light applied to the photo diode. That is, the photo sensor performs coordinate calculation of the object to be sensed according to the amount of change of light, and position information of the object to be sensed can be obtained through the calculation.

The display unit 151 displays (outputs) information to be processed in the electronic device 100. For example, the display unit 151 may display execution screen information of an application program driven by the electronic device 100, or UI (User Interface) and GUI (Graphic User Interface) information according to the execution screen information .

In addition, the display unit 151 may be configured as a stereoscopic display unit for displaying a stereoscopic image.

In the stereoscopic display unit, a three-dimensional display system such as a stereoscopic system (glasses system), an autostereoscopic system (no-glasses system), and a projection system (holographic system) can be applied.

The sound output unit 152 may output audio data received from the wireless communication unit 110 or stored in the memory 170 in a call signal reception mode, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, The sound output unit 152 also outputs sound signals related to functions (e.g., call signal reception tones, message reception tones, etc.) performed in the electronic device 100. [ The audio output unit 152 may include a receiver, a speaker, a buzzer, and the like.

The haptic module 153 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 153 may be vibration. The intensity and pattern of the vibration generated in the haptic module 153 can be controlled by the user's selection or the setting of the control unit. For example, the haptic module 153 may synthesize and output different vibrations or sequentially output the vibrations.

In addition to vibration, the haptic module 153 may be configured to perform various functions such as a pin arrangement vertically moving with respect to the contact skin surface, a spraying force or suction force of the air through the injection port or the suction port, a touch on the skin surface, And various tactile effects such as an effect of reproducing a cold sensation using an endothermic or exothermic element can be generated.

The haptic module 153 can transmit the tactile effect through the direct contact, and the tactile effect can be felt by the user through the muscles of the finger or arm. At least two haptic modules 153 may be provided according to the configuration of the electronic device 100.

The light output unit 154 outputs a signal for notifying the occurrence of an event using the light of the light source of the electronic device 100. Examples of events that occur in the electronic device 100 may include message reception, call signal reception, missed call, alarm, schedule notification, email reception, receiving information via an application, and the like.

The signal output by the light output unit 154 is implemented as the electronic device emits light of a single color or a plurality of colors to the front or rear surface. The signal output may be terminated by the electronic device sensing an event confirmation of the user.

The interface unit 160 serves as a path for communication with all the external devices connected to the electronic device 100. The interface unit 160 receives data from an external device or supplies power to each component in the electronic device 100 or allows data in the electronic device 100 to be transmitted to an external device. For example, a port for connecting a device equipped with a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, an audio I / O port, a video I / O port, an earphone port, and the like may be included in the interface unit 160.

The identification module is a chip for storing various kinds of information for authenticating the usage right of the electronic device 100. The identification module includes a user identification module (UIM), a subscriber identity module (SIM) A universal subscriber identity module (USIM), and the like. Devices with identification modules (hereinafter referred to as "identification devices") can be manufactured in a smart card format. Accordingly, the identification device can be connected to the terminal 100 through the interface unit 160. [

The interface unit 160 may be a path through which power from the cradle is supplied to the electronic device 100 when the electronic device 100 is connected to an external cradle, And various command signals may be transmitted to the electronic device 100. [ Various command signals or power signals input from the cradle may be operated with a signal for recognizing that the electronic device 100 is correctly mounted on the cradle.

The memory 170 may store a program for the operation of the controller 180 and temporarily store input / output data (e.g., a phone book, a message, a still image, a moving picture, etc.). The memory 170 may store data related to vibration and sound of various patterns outputted when a touch is input on the touch screen.

The memory 170 may be a flash memory type, a hard disk type, a solid state disk type, an SDD type (Silicon Disk Drive type), a multimedia card micro type ), Card type memory (e.g., SD or XD memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read memory, a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and / or an optical disk. The electronic device 100 may operate in association with a web storage that performs the storage function of the memory 170 on the Internet.

Meanwhile, as described above, the control unit 180 controls the operation related to the application program and the general operation of the electronic device 100. [ For example, when the state of the electronic device meets a set condition, the controller 180 can execute or release a lock state for restricting input of a user's control command to applications.

In addition, the control unit 180 performs control and processing related to voice communication, data communication, video call, or the like, or performs pattern recognition processing to recognize handwriting input or drawing input performed on the touch screen as characters and images, respectively . Further, the control unit 180 may control any one or a plurality of the above-described components in order to implement various embodiments described below on the electronic device 100 according to the present invention.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power necessary for operation of the respective components. The power supply unit 190 includes a battery, the battery may be an internal battery configured to be chargeable, and may be detachably coupled to the terminal body for charging or the like.

In addition, the power supply unit 190 may include a connection port, and the connection port may be configured as an example of an interface 160 through which an external charger for supplying power for charging the battery is electrically connected.

As another example, the power supply unit 190 may be configured to charge the battery in a wireless manner without using the connection port. In this case, the power supply unit 190 may use at least one of an inductive coupling method based on a magnetic induction phenomenon from an external wireless power transmission apparatus and a magnetic resonance coupling method based on an electromagnetic resonance phenomenon Power can be delivered.

In the following, various embodiments may be embodied in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

The electronic device according to the present invention may include a display unit capable of outputting moving images and subtitles together. Hereinafter, a method of converting the subtitles provided with moving pictures into a form suitable for the level of the user and providing the subtitles will be described.

2 is a flowchart illustrating a control method of providing a caption in a form suitable for a language level of a viewer at the time of moving picture reproduction in an electronic apparatus according to the present invention. 3A to 3C are conceptual diagrams showing a synonym database part of an electronic device according to the present invention. 4 is a conceptual diagram showing the control method of Fig.

Referring to FIG. 2, the electronic device according to the present invention may display a moving picture playback screen on the display unit 151 (S210).

The electronic device 100 according to the present invention may be provided with a moving image application that provides a moving image playback function. The moving picture application may be installed at the factory of the electronic device, installed or shipped, or downloaded from an application providing server (e.g., Google Play) by a user's control command and installed on the electronic device.

The control unit 180 can reproduce the moving picture through the moving picture application in response to the input of the user's control command for reproducing the moving picture.

The moving picture may be stored in the memory 170 of the electronic device 100 or an external server capable of communicating with the electronic device 100. [ In the case of a moving picture stored in an external server, the electronic device 100 can receive the moving picture data corresponding to the moving picture in real time in a streaming manner and reproduce the moving picture.

The moving picture may be composed of video data and audio data. The moving picture may further include caption data in which audio data is converted into characters. The caption data may be generated as a separate file from the moving image or may be generated in a form including a caption file in the moving image.

The caption data may be synchronized with the audio data, and the caption data may be output in accordance with the output time of the audio data when the moving image is reproduced.

When the moving image is reproduced, the control unit 180 may output a moving image playback screen on the display unit 151. FIG. The moving picture playback screen may include video information according to video data and subtitle information according to subtitle data.

When the moving picture playback screen is displayed, the control unit 180 may extract at least one word from the words included in the caption based on the language level of the viewer (S220).

First, the controller 180 can detect a viewer when the moving picture playback screen is displayed. Here, the viewer may be a user who uses the electronic device, a user located on the front side of the display unit 151, or a user whose sight line is pointed toward the display unit 151, as a user watching the moving image.

When the controller 180 determines that the viewer is located at the front of the display unit 151, the controller 180 controls the camera 121 to photograph the front of the display unit 151 to detect the viewer It is possible to take a picture of the front surface of the display unit 151. The control unit 180 analyzes the photographed image based on the image analysis algorithm, and extracts a graphic object corresponding to the viewer from the analyzed image.

Thereafter, the control unit 180 can extract the viewer information from the memory 170 or generate it based on the image analysis result.

More specifically, the control unit 180 compares the image information of the viewer stored in the memory 170 with the graphic object corresponding to the viewer extracted through the image, and the viewer extracted through the comparison result image In the case of the same viewer, the viewer can be determined through the viewer information of the previously stored viewer. Alternatively, when the viewer extracted through the comparison result image does not match the pre-stored viewer, the controller 180 may extract the appearance information such as the age of the viewer and the gender of the viewer from the analyzed image to generate viewer information have.

In a similar manner, when the controller 180 determines that the viewer is a viewer whose sight line is on the display unit 151, the viewer 180 can extract the user's gaze information from the above-mentioned analyzed image. Then, the control unit 180 can determine that the viewer is a viewer. In this case, the controller 180 can extract the viewer information from the memory 170 or directly generate the viewer information based on the image analysis result. The method of detecting the viewer may be preset on the electronic device.

The controller 180 may determine the language level of the viewer when the viewer is sensed. The language level of the viewer represents the viewer's ability to understand the language. The control unit 180 can determine the language level of the viewer based on at least one of the age group of the viewer, the gender of the viewer, the audio information that the viewer normally utteres, and the content information viewed by the viewer.

First, the control unit 180 can learn the audio information that the viewer normally utteres and the content information viewed by the viewer to determine the language level of the viewer. The control unit 180 can store voice information that the viewer uttered on one area of the memory unit 170, character information extracted from the content watched by the viewer, and the like. One area of the memory unit 170 in which the learning information is stored may be referred to as a learning-word database unit 172 (FIG. 3A).

The voice information uttered by the user may be voice information received through the microphone 122 of the electronic device 100. [

The content information viewed by the viewer is character information reproduced through the electronic device or extracted from the content having the history displayed on the display unit. For example, the content information viewed by the viewer may include character information included on the web page, character information included in the electronic book, character information included in the message, character information uploaded on the SNS, and the like.

The control unit 180 can learn the character information stored in the learning word database unit 172 based on the machine learning algorithm. Then, the control unit 180 can determine the language level of the user based on the learning result.

More specifically, the control unit 180 compares the predetermined classification criterion and the learning result, and can determine the language level of the viewer based on the comparison result.

The classification criterion is a criterion for judging the language level, and can be divided into age groups. For example, the classification criteria are 0 to 10 years old for Level 1, 10 to 50 years for Level 2, 50 to 80 years for Level 3, ). The judgment criteria of the language level for each age group may be learned in advance based on the machine learning algorithm.

When the language level of the viewer is determined, the controller 180 can extract at least one word higher than the language level of the viewer among the words included in the caption, based on the language level of the viewer.

Referring to FIG. 3A, the memory unit 170 of the present invention may further include a synonym database 171 for storing words having the same or similar meaning. The synonym database 171 may classify synonyms according to the language level in the same manner as the language level of the viewer, and set synonyms classified into the same language level into one group. For example, the synonym database 171 includes a first group (Level 1), a second group (Level 2), and a third group (Level 3) having different language levels, It may contain words corresponding to a group. In FIG. 3B, the group showing the lowest language level is shown as the first group, the group showing the middle language level is shown as the second group, and the group showing the highest language level is shown as the third group.

On the other hand, the language level of words can be determined by frequency of use of words according to time, frequency of use of terminology, and frequency of word usage by age group.

For example, as shown in FIG. 3C, when the usage frequency increases or maintains a certain level according to the flow of time, the control unit 180 determines that the language is easy for people to understand, It can be judged as a word corresponding to a lower language level. On the other hand, when the frequency of use decreases with the passage of time such as "serene", the control unit 180 determines that the word is difficult to understand by people and judges it as a word corresponding to a middle language level have.

The controller 180 compares the language level of the words included in the caption with the language level of the viewer and may detect at least one word higher than the language level of the viewer based on the comparison result.

The control unit 180 may extract the synonym of the extracted at least one word from the synonym database unit 171 at step S230.

The control unit 180 may extract synonyms belonging to the language level of the viewer among the synonyms of the at least one word stored in the synonym database unit 171. [ For example, when the language level of the viewer is Level 1 and the language level of the words extracted from the subtitles is Level 2, the controller 180 obtains a synonym of the words extracted from the subtitle belonging to Level 1 from the synonym database 171, Can be extracted.

Then, the control unit 180 may display the extracted synonyms instead of the at least one word on the moving picture playback screen (S240).

If the synonym of at least one word included in the caption is extracted, the controller 180 may display the extracted synonym in place of the caption on the reproduction screen of the moving picture.

For example, as shown in FIG. 4A, the control unit 180 can detect a word "SERENA (310)" which is higher than the user's language level among the words included in the subtitles. 4 (b), the control unit 180 extracts synonyms of "SERENA 310" or "CALM 320" having a lower language level than "SERENA 310" (320) " can be displayed as a subtitle. Therefore, the viewer can be provided with a subtitle suitable for the language level of the viewer, instead of subtitles that do not match the language level of the viewer.

On the other hand, when a plurality of synonyms of the at least one word are extracted, the control unit 180 may display any one of a plurality of synonyms based on the priority.

The priority may be determined according to the frequency with which the viewer uses the word. For example, the priority may be such that the higher the frequency of word usage of the viewer, the higher the priority of the word. Therefore, the present invention can provide subtitles using words more familiar to the viewer.

In the above, a method of providing a caption suitable for a user's language level has been described. Accordingly, the present invention can improve the subtitle understanding of a viewer who watches a moving picture by changing a word included in the subtitle provided in the moving picture to another synonym suitable for the language level of the viewer, according to the language level of the viewer.

Hereinafter, a method of providing subtitles using a plurality of subtitle modes providing subtitles at different language levels will be described. FIG. 5 is a flowchart illustrating a method of providing subtitles using a plurality of subtitle modes providing subtitles at different language levels. 6 is a conceptual diagram showing the control method of FIG.

Referring to FIG. 5, the controller 180 may output a pop-up window for setting the language level of the subtitles before the moving picture is reproduced (S510).

The controller 180 may output a pop-up window for setting the language level of the subtitles so that the viewer can select the language level of the subtitles, instead of immediately starting the reproduction of the moving picture. For example, as shown in (a) of FIG. 6, the controller 180 may detect that a moving image playback command of AAA.avi is applied. In this case, a pop-up window for setting the language level of the subtitles may be output on the display unit 151 as shown in FIG. 6 (b). The pop-up window may include a plurality of predetermined caption modes. The plurality of subtitle modes may be modes for generating subtitles according to different language levels.

The control unit 180 can select one of the plurality of subtitle modes based on the user control command for selecting the subtitle mode (S520).

The plurality of subtitle modes may provide subtitles corresponding to different language levels. For example, the plurality of subtitle modes include Level 1, Level 2, and Level 3, and each subtitle mode may correspond to the lowest, middle, and highest language levels.

The control unit 180 may change the words included in the caption based on the language level corresponding to the selected caption mode (S530).

For example, as shown in (c) of FIG. 6, when the caption mode corresponding to Level 1 is selected, the controller 180 can provide captions at a language level corresponding to Level 1. [ Accordingly, the controller 180 can extract at least one word corresponding to a language level higher than the language level corresponding to Level 1, among the words included in the caption. Then, the control unit 180 may convert the extracted at least one word into a synonym corresponding to Level 1, and output it to the display unit 151.

Accordingly, the present invention can provide subtitles corresponding to a desired language level to viewers of a moving image, and a viewer can understand the subtitles more easily.

In the foregoing description, the method of setting the caption mode before the reproduction of the moving image has been described. However, the present invention can set the caption mode based on the user's control command in a similar manner during reproduction of the moving image. In this case, the control unit 180 can stop the reproduction of the moving picture for setting the caption mode. When the setting of the caption mode is completed, the moving picture can be reproduced again.

Hereinafter, a method of providing captions using words used by viewers will be described. 7 is a flowchart illustrating a method of providing a caption using words used by a viewer.

Referring to FIG. 7, the control unit 180 may extract at least one word included in the subtitle based on the language level of the viewer (S710).

As described above, when the reproduction screen of the moving picture is displayed on the display unit 151, the control unit 180 displays at least one word from the words included in the subtitles displayed together with the moving picture on the basis of the language level of the viewer Can be extracted.

The control unit 180 may determine whether the at least one word is included in the learning database unit (S720). If at least one word is included in the learning database unit 172, the control unit 180 may output the subtitle including at least one word as it is (S730). More specifically, if the at least one word is determined to be a word learned by the user, even if at least one word higher than the user's language level is extracted, It may not be changed. Through this, the present invention can provide subtitles containing more familiar and familiar words to the user.

If at least one word is not included in the learning database unit 172, the control unit 180 may output a caption including a synonym of at least one word (S740). In this case, the control unit 180 can improve the user's understanding of the subtitles by providing the subtitles in a form suitable for the language level of the user, in the same manner as described above.

Hereinafter, a method of providing information of changed words according to a user's language level will be described. 8 and 9 are conceptual diagrams illustrating a method of providing information of changed words according to a language level of a user.

When the subtitles are displayed on the moving picture playback screen, the control unit 180 can visually distinguish words at a level higher than the user's language level from the remaining words. 8A, when the term " Emissary (810) " among the subtitles displayed on the moving picture playback screen is a word having a level higher than the user's language level, the control unit 180 highlights Can be displayed. Highlighting is a method of changing the color, size, contrast, and the like.

In response to the touch input being applied to the term " Emissary (810) ", the control unit 180 can execute a search function for searching the term " Emissary (810) " In this case, as shown in FIG. 8B, the control unit 180 may output a web search page of the term " Emissary (810) " on the display unit 151. [ Through this, the present invention makes it possible to more easily confirm the meaning of a word that the viewer is unlikely to know.

On the other hand, when the subtitles are displayed on the moving picture playback screen, the control unit 180 may display the changed words visually different from the remaining words according to the language level of the user. For example, as shown in FIG. 9A, when "AMBASSADOR (910)" is a changed word, highlighting can be performed so that "AMBASSADOR" is distinguished from remaining words.

The controller 180 may also output the meaning of the AMBASSADOR 910 on the display unit 151 when a touch input to the AMBASSADOR 910 is applied. For example, as shown in FIG. 9B, the control unit 180 can display the Korean meaning "Embassy" of "AMBASSADOR 910" on an area adjacent to the area where "AMBASSADOR 910" is displayed . Therefore, the present invention can quickly check the meaning of a word without disturbing the viewing of a moving picture.

In the above, a method of providing semantic information of subtitles in a case where a word higher than the user's language level is included in the subtitles has been described.

Hereinafter, a method of providing a caption suitable for a language level to a plurality of viewers when a plurality of viewers exist for a moving image will be described. FIG. 10 is a flowchart illustrating a method of providing a plurality of viewers with a caption suitable for a language level when a plurality of viewers exist for a moving image, and FIGS. 11 and 12 are conceptual diagrams showing the control method of FIG.

Referring to FIG. 10, the controller 180 may detect a plurality of viewers having different language levels (S1010).

As shown in FIG. 11 (a), the control unit 180 can detect two viewers who are watching a current moving image. The controller 180 may determine the language level of each of the plurality of viewers. Here, the determination method of the language level is replaced with the description of FIG.

The controller 180 may provide the subtitles in a form suitable for the same language level when a plurality of viewers have the same language level. However, if the language levels of a plurality of viewers are different, the control unit 180 may provide subtitles based on a predetermined reference condition.

More specifically, the control unit 180 may provide the subtitles based on the language level that satisfies predetermined conditions among different language levels (S1020).

The predetermined condition may be set to various conditions.

For example, the preset condition may be a condition of a language level corresponding to a viewing rating of a video. More specifically, when there is a language level corresponding to the viewing rating of the image among the plurality of language levels of the viewers, the controller 180 may provide the caption based on the language level corresponding to the viewing rating of the image . The language level corresponding to the viewing rating of the image may be preset. For example, Level 1 corresponds to the 6th grade, Level 2 corresponds to the 15th grade, and Level 3 corresponds to 19th grade.

As another example, the predetermined condition may be a condition of a language level of the main viewer. The control unit 180 can be a viewer having a remote control among a plurality of viewers, a viewer having an age range corresponding to a viewing rating of a video, or a viewer having a high preference for a video as a main viewer. A viewer with a high preference for the image may be determined based on the previously learned image preference, which may be determined by a generally known method of determining image preference. In this case, the control unit 180 can provide the subtitles in a form suitable for the language level of the main viewer.

As another example, the predetermined condition may be a condition of a lower language level among different language levels. The control unit 180 may provide the subtitles in a form suitable for the lowest language level among the different language levels of a plurality of viewers. Accordingly, the control unit 180 can provide captions in a form that can be understood by all viewers.

As shown in FIG. 11 (b), the control unit 180 can provide captions in a form suitable for a specific language level.

On the other hand, when there are a plurality of viewers, the control unit 180 can simultaneously display the forms of the captions suitable for each of the plurality of viewers. For example, referring to FIGS. 12A and 12B, when there are a plurality of viewers having different language levels, the control unit 180 determines whether the word " EMISSARY " But if the word is higher than the language level of the second viewer, " EMISSARY " and its synonym " AMBASSADOR " may be displayed together. Thus, viewers can identify both terms at the same time.

Hereinafter, a synonym search method will be described when a synonym does not exist on the synonym database. 13 is a conceptual diagram showing a method of searching for a synonym if a synonym does not exist on the synonym database.

13A, when there is no synonym of the word " EMISSARY " on the synonym database, the control unit 180 outputs a pop-up window indicating that no synonym exists on the display unit 151 .

The control unit 180 may ask the viewer to search for a synonym through a pop-up window. In this case, when the viewer inputs a synonym search request, the control unit 180 searches for synonyms through the website as shown in FIG. 13B, or inputs a synonym search request through the audio input of the viewer as shown in FIG. Synonyms can be entered.

Then, as shown in FIG. 13C, the control unit 180 can output the synonym "AMBASSARY" instead of the "EMISSARY" when a synonym is input through a web site or a user's voice input.

Through this, the present invention can retrieve synonyms through various methods even if the words are not in the synonym database, and include them in the subtitles.

The present invention can improve the subtitle understanding degree of the viewer who watches the moving picture by changing the words included in the subtitles provided in the moving picture to other synonyms suitable for the language level of the viewer according to the language level of the viewer.

In addition, the present invention can provide a more user-friendly form of the subtitles by learning the words that the viewer learns or frequently using, and by changing the words included in the subtitles by utilizing the learned words.

The present invention described above can be embodied as computer-readable codes on a medium on which a program is recorded. The computer readable medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of the computer readable medium include a hard disk drive (HDD), a solid state disk (SSD), a silicon disk drive (SDD), a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, , And may also be implemented in the form of a carrier wave (e.g., transmission over the Internet). Also, the computer may include a control unit 180 of the terminal. Accordingly, the above description should not be construed in a limiting sense in all respects and should be considered illustrative. The scope of the present invention should be determined by rational interpretation of the appended claims, and all changes within the scope of equivalents of the present invention are included in the scope of the present invention.

Claims (16)

A learning data part for storing learning information that has learned a user's language level;
A synonym database part in which synonyms are classified and stored according to a predetermined plurality of language levels;
A display unit for displaying subtitles and images together; And
Extracting synonyms of at least one word included in the subtitles from the synonym database based on the language level of the user,
And control the display unit to output the extracted synonyms instead of the at least one word. The method according to claim 1,
Wherein,
Extracts at least one word of a level higher than a language level of the user among the words included in the caption. 3. The method of claim 2,
The control unit
Classifying the words included in the caption into the predetermined language levels,
And extracts at least one word corresponding to a level higher than a language level of the user among the classified words. The method according to claim 1,
The control unit
Extracts synonyms of the at least one word based on a language level corresponding to a caption mode selected by a user among a plurality of caption modes providing captions according to different language levels. The method according to claim 1,
The control unit
Learning is performed by using a machine learning algorithm, words included in at least one of a call voice, a message, contents output on a display unit, and a search history,
And determines a language level of the user based on the learned words. The method of claim 3,
And a learning language database unit for storing the learned words,
The control unit
And controls the display unit to output the at least one word instead of the synonym of the at least one word when the at least one word is included in the learning word database unit. The method according to claim 1,
The synonym database
Classifies the synonyms into a plurality of predetermined language levels based on the frequency of use of the word and the age range of use of the word according to the age. The method according to claim 1,
The control unit
And outputs a pop-up window indicating that there is no synonym of the at least one word on the display unit if the synonym of the at least one word does not exist in the synonym database. The method according to claim 1,
The control unit
When the at least one word is output on the display unit, displays the at least one word so as to be visually distinguishable from the remaining words. The method according to claim 1,
The control unit
And outputs semantic information of the at least one word on the display unit in response to a touch input for the at least one word. The method according to claim 1,
Further comprising a camera configured to photograph a front of the display unit,
The control unit
When a plurality of users located in front of the display unit are detected using the image information photographed by the camera, a synonym of the at least one word based on a lower language level among the different language levels of the plurality of users And extracting the electronic device. 12. The method of claim 11,
Wherein,
Extracting a plurality of synonyms of the at least one word based on different language levels of the plurality of users when a plurality of users located in front of the display unit are detected using image information photographed by the camera, And outputs the extracted plurality of synonyms together on the display unit. 12. The method of claim 11,
The control unit
And extracts the at least one synonym based on a language level of a user satisfying a predetermined condition among the plurality of users. 14. The method of claim 13,
The predetermined condition is
A condition that the level information of the image corresponds to a level of the user's language level, and a condition of a main viewer of the image. 14. The method of claim 13,
The predetermined condition is
A condition for holding a remote controller for controlling the electronic device, and a condition for matching the genre of the image with the taste of the user. Displaying an image including a caption;
Extracting at least one word among the words included in the caption based on a language level of the user stored in the learning data unit;
Extracting a synonym of the extracted at least one word from a synonym database part in which synonyms are classified according to a language level;
And outputting the extracted synonyms to the subtitles,
Wherein the extracted synonym is a word corresponding to a language level of the user.

Images