KR102535705B1 - Mobile terminal and method for controlling the same - Google Patents

Abstract

The present invention relates to a mobile terminal visually providing non-visual environment information, which includes a display unit, a sensing unit for detecting environment information, and based on the sensed environment information, detecting an expected scenario and displaying the detected expected scenario. and a control unit controlling the display unit to output an output to the display unit, and the expected scenario is prediction information representing a possible environment state based on the sensed environment information.

Description

Mobile terminal and its control method {MOBILE TERMINAL AND METHOD FOR CONTROLLING THE SAME}

The present invention relates to a mobile terminal visually providing non-visual environment information and a control method thereof.

Terminals can be divided into mobile/portable terminals and stationary terminals according to whether or not they can be moved. Again, the mobile terminal may be divided into a handheld terminal and a vehicle mounted terminal depending on whether the user can directly carry the mobile terminal.

Functions of mobile terminals are diversifying. For example, there are functions of data and voice communication, photographing and video recording through a camera, voice recording, music file reproduction through a speaker system, and outputting an image or video to a display unit. Some terminals have an electronic game play function added or a multimedia player function. In particular, recent mobile terminals can receive multicast signals providing visual content such as broadcasting and video or television programs.

As the functions of such terminals diversify, for example, it is a multimedia player equipped with complex functions such as taking pictures or videos, playing music or video files, playing games, and receiving broadcasts. is being implemented

In order to support and increase the functions of such a terminal, it may be considered to improve the structural part and/or the software part of the terminal.

Recently, Internet of Things technology, which is a technology for remotely controlling objects such as home appliances through a mobile terminal through the Internet, has been provided.

In addition, the mobile terminal is attached with various sensors that sense the surrounding environment, and the mobile terminal senses the surrounding environment through these sensors and simply provides numerical information about the environment sensed by the various sensors.

However, in the prior art, the sensed information was simply provided to the mobile terminal, and a method of specifically utilizing it was not provided. Therefore, in the prior art, there was a problem in that the user simply checked the sensed information and could not utilize it. .

An object of the present invention is to visually provide non-visual information.

Another object of the present invention is to predict and provide future environmental information in advance.

Another object of the present invention is to appropriately control objects according to surrounding environments.

A mobile terminal according to an embodiment of the present invention detects an expected scenario based on a display unit, a sensing unit for detecting environment information, and the sensed environment information, and displays the detected expected scenario to the display unit. and a control unit for controlling a control unit, and the expected scenario is characterized in that it is prediction information for predicting a possible environment state based on the sensed environment information.

In an embodiment, the control unit may detect different expected scenarios based on at least one of a location and a time at which the environment information is sensed.

In one embodiment, the control unit detects a user gesture with respect to the display unit on which the predicted scenario is output, and controls operations of external devices related to the expected scenario in response to the detected user gesture. do.

In one embodiment, the controller may operate different external devices in response to detection of different user gestures.

In one embodiment, the controller may stop the operation of the different external devices when a predetermined time elapses after operating the different external devices.

In an embodiment, the expected scenario includes information on at least one recommended device for which execution is recommended, and the controller, in response to input of a user gesture, creates a recommendation list including the at least one recommended device. Characterized in that the output is output to the display unit.

In an embodiment, the controller may execute a specific recommended device among at least one recommended device included in the recommendation list based on a touch input.

In one embodiment, the display unit includes a plurality of displays disposed at different locations, and the controller outputs the predicted scenario to a specific display disposed at a location related to the expected scenario among the plurality of displays. characterized by

In one embodiment, a location detection unit for detecting a user's location is further included, and the control unit outputs the expected scenario to a display disposed closest to the location of the user detected by the location detection unit among the plurality of displays. characterized by

In one embodiment, the control unit may make the expected scenario disappear from the display unit when new environmental information different from the sensed environment information is detected through the sensing unit.

As described above, according to the mobile terminal according to the present invention, the user can more intuitively recognize the state related to the surrounding environment by visually providing environment information related to the surrounding environment that cannot be visually seen.

In addition, according to the mobile terminal according to the present invention, based on environment information related to the surrounding environment, an expected scenario indicating a predictable state of the surrounding environment is provided, so that the user can predict the future situation in advance.

In addition, according to the mobile terminal according to the present invention, it is possible to control the operation of the surrounding objects appropriately to the current environment by controlling the surrounding objects through the Internet based on environment information related to the surrounding environment.

1 is a block diagram for explaining a mobile terminal related to the present invention.
2 is a flowchart illustrating a control method of a mobile terminal related to the present invention.
FIG. 3 is a conceptual diagram illustrating the control method of FIG. 2 .
4A and 4B are conceptual diagrams illustrating that different prediction scenarios are provided according to locations for the same environment information.
5A and 5B are conceptual diagrams illustrating that different controls are performed according to user gestures in a mobile terminal related to the present invention.
6A and 6B are conceptual diagrams illustrating that different controls are performed based on surrounding environment information in a mobile terminal related to the present invention.
7 is a conceptual diagram illustrating a method of providing a recommended device according to a user gesture in a mobile terminal related to the present invention.
8 is a flowchart illustrating a control method related to an operation of a mobile terminal when an external environment is changed in a mobile terminal related to the present invention.
9 is a conceptual diagram illustrating an Internet of Things system that performs control according to environmental information related to the present invention.
10 is a block diagram of an IoT system that performs control according to environment information related to the present invention.
11 is an example of an Internet of Things system that performs control according to environment information related to the present invention.

Hereinafter, the embodiments disclosed in this specification will be described in detail with reference to the accompanying drawings, but the same reference numerals will be assigned to the same or similar components regardless of reference numerals, and overlapping descriptions thereof will be omitted. The suffixes "module" and "unit" for components used in the following description are given or used together in consideration of ease of writing the specification, and do not have meanings or roles that are distinct from each other by themselves. In addition, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of a related known technology may obscure the gist of the embodiment disclosed in this specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in this specification, the technical idea disclosed in this specification is not limited by the accompanying drawings, and all changes included in the spirit and technical scope of the present invention , it should be understood to include equivalents or substitutes.

Terms including ordinal numbers, such as first and second, may be used to describe various components, but the components are not limited by the terms. These terms are only used for the purpose of distinguishing one component from another.

It is understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle.

Singular expressions include plural expressions unless the context clearly dictates otherwise.

In this application, terms such as "comprise" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

Mobile terminals described in this specification include mobile phones, smart phones, laptop computers, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, and slate PCs. , tablet PC, ultrabook, wearable device (eg, watch type terminal (smartwatch), glass type terminal (smart glass), HMD (head mounted display)), etc. may be included there is.

Mobile terminals described in this specification include mobile phones, smart phones, laptop computers, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, and slate PCs. , tablet PC, ultrabook, wearable device (eg, watch type terminal (smartwatch), glass type terminal (smart glass), HMD (head mounted display)), etc. may be included there is.

However, those skilled in the art can easily recognize that the configuration according to the embodiments described in this specification may be applied to fixed terminals such as digital TVs, desktop computers, digital signage, etc., except when applicable only to mobile terminals. will be.

Referring to Figure 1, 1 is For explaining a mobile terminal related to the present invention It is a block diagram .

The mobile terminal 100 includes a wireless communication unit 110, an input unit 120, a sensing unit 140, an output unit 150, an interface unit 160, a memory 170, a control unit 180, and a power supply unit 190. ) and the like. The components shown in FIG. 1A are not essential to implement a mobile terminal, so a mobile terminal described herein may have more or fewer components than those listed above.

More specifically, among the components, the wireless communication unit 110 is between the mobile terminal 100 and the wireless communication system, between the mobile terminal 100 and other mobile terminals 100, or between the mobile terminal 100 and an external server. It may include one or more modules enabling wireless communication between Also, the wireless communication unit 110 may include one or more modules that connect the mobile terminal 100 to one or more networks.

The wireless communication unit 110 may include at least one of a broadcast reception module 111, a mobile communication module 112, a wireless Internet module 113, a short-distance communication module 114, and a location information module 115. .

The input unit 120 includes a camera 121 or video input unit for inputting a video signal, a microphone 122 for inputting an audio signal, or a user input unit 123 for receiving information from a user, for example , a touch key, a push key (mechanical key, etc.). Voice data or image data collected by the input unit 120 may be analyzed and processed as a user's control command.

The sensing unit 140 may include one or more sensors for sensing at least one of information within the mobile terminal, surrounding environment information surrounding the mobile terminal, and user information. For example, the sensing unit 140 may include a proximity sensor 141, an illumination sensor 142, a touch sensor, an acceleration sensor, a magnetic sensor, and gravity. Sensor (G-sensor), gyroscope sensor (gyroscope sensor), motion sensor (motion sensor), RGB sensor, infrared sensor (IR sensor), finger scan sensor, ultrasonic sensor , an optical sensor (eg, a camera (see 121)), a microphone (see 122), a battery gauge, an environmental sensor (eg, a barometer, a hygrometer, a thermometer, a radiation detection sensor, It may include at least one of a heat detection sensor, a gas detection sensor, etc.), a chemical sensor (eg, an electronic nose, a healthcare sensor, a biometric sensor, etc.). Meanwhile, the mobile terminal disclosed in this specification may combine and utilize information sensed by at least two or more of these sensors.

The output unit 150 is for generating an output related to sight, hearing, or touch, and includes at least one of a display unit 151, a sound output unit 152, a haptic module 153, and an optical output unit 154. can do. The display unit 151 may implement a touch screen by forming a mutual layer structure or integrally with the touch sensor. Such a touch screen may function as a user input unit 123 providing an input interface between the mobile terminal 100 and the user, and provide an output interface between the mobile terminal 100 and the user.

The interface unit 160 serves as a passage for various types of external devices connected to the mobile terminal 100 . The interface unit 160 connects a device equipped with a wired/wireless headset port, an external charger port, a wired/wireless data port, a memory card port, and an identification module. It may include at least one of a port, an audio input/output (I/O) port, a video input/output (I/O) port, and an earphone port. In response to an external device being connected to the interface unit 160, the mobile terminal 100 may perform appropriate control related to the connected external device.

In addition, the memory 170 stores data supporting various functions of the mobile terminal 100 . The memory 170 may store a plurality of application programs (application programs or applications) running in the mobile terminal 100 , data for operation of the mobile terminal 100 , and commands. At least some of these application programs may be downloaded from an external server through wireless communication. In addition, at least some of these application programs may exist on the mobile terminal 100 from the time of shipment for the basic functions of the mobile terminal 100 (eg, incoming and outgoing calls, outgoing functions, message receiving, and outgoing functions). Meanwhile, the application program may be stored in the memory 170, installed on the mobile terminal 100, and driven by the controller 180 to perform an operation (or function) of the mobile terminal.

The controller 180 controls general operations of the mobile terminal 100 in addition to operations related to the application program. The control unit 180 may provide or process appropriate information or functions to the user by processing signals, data, information, etc. input or output through the components described above or by running an application program stored in the memory 170.

In addition, the controller 180 may control at least some of the components reviewed in conjunction with FIG. 1A in order to drive an application program stored in the memory 170 . Furthermore, the controller 180 may combine and operate at least two or more of the components included in the mobile terminal 100 to drive the application program.

Under the control of the controller 180, the power supply unit 190 receives external power and internal power and supplies power to each component included in the mobile terminal 100. The power supply unit 190 includes a battery, and the battery may be a built-in battery or a replaceable battery.

At least some of the above components may operate in cooperation with each other to implement an operation, control, or control method of a mobile terminal according to various embodiments described below. In addition, the operation, control, or control method of the mobile terminal may be implemented on the mobile terminal by driving at least one application program stored in the memory 170 .

Hereinafter, prior to examining various embodiments implemented through the mobile terminal 100 described above, the above-listed components will be described in more detail with reference to FIG . 1 .

First, looking at the wireless communication unit 110, the broadcast reception module 111 of the wireless communication unit 110 receives a broadcast signal and/or broadcast-related information from an external broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. Two or more broadcast receiving modules may be provided in the mobile terminal 100 to receive simultaneous broadcasting of at least two broadcast channels or to switch broadcast channels.

The mobile communication module 112 complies with technical standards or communication methods for mobile communication (eg, Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), Code Division Multi Access 2000 (CDMA2000), EV) -DO(Enhanced Voice-Data Optimized or Enhanced Voice-Data Only), WCDMA(Wideband CDMA), HSDPA(High Speed Downlink Packet Access), HSUPA(High Speed Uplink Packet Access), LTE(Long Term Evolution), LTE-A (Long Term Evolution-Advanced), etc.) to transmit and receive radio signals with at least one of a base station, an external terminal, and a server on a mobile communication network.

The wireless signal may include a voice call signal, a video call signal, or various types of data according to text/multimedia message transmission/reception.

The wireless Internet module 113 refers to a module for wireless Internet access, and may be built into or external to the mobile terminal 100 . The wireless Internet module 113 is configured to transmit and receive radio signals in a communication network according to wireless Internet technologies.

Wireless Internet technologies include, for example, WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), WiMAX (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), etc., and the wireless Internet module ( 113) transmits and receives data according to at least one wireless Internet technology within a range including Internet technologies not listed above.

From the viewpoint that wireless Internet access by WiBro, HSDPA, HSUPA, GSM, CDMA, WCDMA, LTE, LTE-A, etc. is made through a mobile communication network, the wireless Internet module (113) performing wireless Internet access through the mobile communication network ) may be understood as a kind of the mobile communication module 112.

The short-range communication module 114 is for short-range communication, and includes Bluetooth™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and NFC. Near Field Communication (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and wireless USB (Wireless Universal Serial Bus) technology may be used to support short-distance communication. The short-range communication module 114 may be used between the mobile terminal 100 and a wireless communication system, between the mobile terminal 100 and other mobile terminals 100, or between the mobile terminal 100 through wireless area networks. ) and a network where another mobile terminal 100 (or an external server) is located. The local area wireless communication network may be a local area wireless personal area network (Wireless Personal Area Networks).

Here, the other mobile terminal 100 is a wearable device capable of exchanging (or interlocking) data with the mobile terminal 100 according to the present invention (for example, a smart watch), smart glasses (smart glass) or HMD (head mounted display). The short-distance communication module 114 may detect (or recognize) a wearable device capable of communicating with the mobile terminal 100 around the mobile terminal 100 . Furthermore, if the detected wearable device is a device authorized to communicate with the mobile terminal 100 according to the present invention, the controller 180 transmits at least a portion of data processed by the mobile terminal 100 to the short-range communication module ( 114) can be transmitted to the wearable device. Accordingly, the user of the wearable device may use data processed by the mobile terminal 100 through the wearable device. For example, according to this, when a phone call is received in the mobile terminal 100, the user makes a phone call through the wearable device, or when a message is received in the mobile terminal 100, the user receives the received message through the wearable device. It is possible to check the message.

The location information module 115 is a module for acquiring the location (or current location) of the mobile terminal, and representative examples thereof include a Global Positioning System (GPS) module or a Wireless Fidelity (WiFi) module. For example, when a mobile terminal utilizes a GPS module, the location of the mobile terminal can be obtained using signals transmitted from GPS satellites. As another example, when the mobile terminal utilizes the Wi-Fi module, the location of the mobile terminal may be obtained based on information of the Wi-Fi module and a wireless access point (AP) that transmits or receives a wireless signal. If necessary, the location information module 115 may perform any function among other modules of the wireless communication unit 110 to obtain data on the location of the mobile terminal in substitution or addition. The location information module 115 is a module used to obtain the location (or current location) of the mobile terminal, and is not limited to a module that directly calculates or acquires the location of the mobile terminal.

Next, the input unit 120 is for inputting video information (or signals), audio information (or signals), data, or information input from a user. For inputting video information, the mobile terminal 100 has one Alternatively, a plurality of cameras 121 may be provided. The camera 121 processes an image frame such as a still image or a moving image obtained by an image sensor in a video call mode or a photographing mode. The processed image frame may be displayed on the display unit 151 or stored in the memory 170 . Meanwhile, the plurality of cameras 121 provided in the mobile terminal 100 may be arranged to form a matrix structure, and through the cameras 121 forming the matrix structure, various angles or focal points may be provided to the mobile terminal 100. A plurality of image information having may be input. In addition, the plurality of cameras 121 may be arranged in a stereo structure to obtain left and right images for realizing a stereoscopic image.

The microphone 122 processes external sound signals into electrical voice data. The processed voice data may be utilized in various ways according to the function (or application program being executed) being performed in the mobile terminal 100 . Meanwhile, various noise cancellation algorithms for removing noise generated in the process of receiving an external sound signal may be implemented in the microphone 122 .

The user input unit 123 is for receiving information from a user. When information is input through the user input unit 123, the control unit 180 can control the operation of the mobile terminal 100 to correspond to the input information. . The user input unit 123 is a mechanical input means (or a mechanical key, for example, a button located on the front, rear or side of the mobile terminal 100, a dome switch, a jog wheel, jog switch, etc.) and a touch input means. As an example, the touch input means consists of a virtual key, soft key, or visual key displayed on a touch screen through software processing, or a part other than the touch screen. It can be made of a touch key (touch key) disposed on. On the other hand, the virtual key or visual key can be displayed on the touch screen while having various forms, for example, graphic (graphic), text (text), icon (icon), video (video) or these can be made of a combination of

Meanwhile, the sensing unit 140 senses at least one of information within the mobile terminal, surrounding environment information surrounding the mobile terminal, and user information, and generates a sensing signal corresponding thereto. Based on these sensing signals, the controller 180 may control driving or operation of the mobile terminal 100 or may process data related to applications installed in the mobile terminal 100, perform functions or operations. Representative sensors among various sensors that may be included in the sensing unit 140 will be described in more detail.

First, the proximity sensor 141 refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object existing nearby without mechanical contact by using the force of an electromagnetic field or infrared rays. The proximity sensor 141 may be disposed in an inner area of the mobile terminal covered by the touch screen described above or in the vicinity of the touch screen.

Examples of the proximity sensor 141 include a transmissive photoelectric sensor, a direct reflection photoelectric sensor, a mirror reflection photoelectric sensor, a high-frequency oscillation type proximity sensor, a capacitance type proximity sensor, a magnetic proximity sensor, an infrared proximity sensor, and the like. If the touch screen is a capacitive type, the proximity sensor 141 may be configured to detect the proximity of a conductive object as a change in electric field according to the proximity of the object. In this case, the touch screen (or touch sensor) itself may be classified as a proximity sensor.

On the other hand, for convenience of description, an action of approaching a touch screen without contacting an object to recognize that the object is located on the touch screen is referred to as a "proximity touch", and the touch An act of actually touching an object on a screen is called a "contact touch". The location at which an object is proximity-touched on the touch screen means a location at which the object corresponds vertically to the touch screen when the object is proximity-touched. The proximity sensor 141 may detect a proximity touch and a proximity touch pattern (eg, proximity touch distance, proximity touch direction, proximity touch speed, proximity touch time, proximity touch position, proximity touch movement state, etc.) there is. Meanwhile, as described above, the controller 180 processes data (or information) corresponding to the proximity touch operation and proximity touch pattern detected through the proximity sensor 141, and furthermore, visual information corresponding to the processed data. It can be output on the touch screen. Furthermore, the controller 180 may control the mobile terminal 100 to process different operations or data (or information) depending on whether a touch to the same point on the touch screen is a proximity touch or a contact touch. .

The touch sensor detects a touch (or touch input) applied to the touch screen (or the display unit 151) using at least one of various touch methods such as a resistive film method, a capacitive method, an infrared method, an ultrasonic method, and a magnetic field method. do.

As an example, the touch sensor may be configured to convert a change in pressure applied to a specific part of the touch screen or capacitance generated in a specific part into an electrical input signal. The touch sensor may be configured to detect a location, area, pressure upon touch, capacitance upon touch, and the like, at which a touch object that applies a touch to the touch screen is touched on the touch sensor. Here, the touch object is an object that applies a touch to the touch sensor, and may be, for example, a finger, a touch pen, a stylus pen, or a pointer.

As such, when there is a touch input to the touch sensor, the corresponding signal(s) is sent to the touch controller. The touch controller processes the signal(s) and then transmits corresponding data to the controller 180. Thus, the controller 180 can know which area of the display unit 151 has been touched. Here, the touch controller may be a component separate from the controller 180 or may be the controller 180 itself.

Meanwhile, the controller 180 may perform different controls or the same control according to the type of the touch object that touches the touch screen (or a touch key provided other than the touch screen). Whether to perform different controls or the same control according to the type of the touch object may be determined according to an operating state of the mobile terminal 100 or an application program currently being executed.

On the other hand, the touch sensor and proximity sensor described above are independently or combined, short (or tap) touch, long touch, multi-touch, drag touch on the touch screen ), flick touch, pinch-in touch, pinch-out touch, swipe touch, hovering touch, etc. Touch can be sensed.

The ultrasonic sensor may recognize location information of a sensing object by using ultrasonic waves. Meanwhile, the controller 180 may calculate the location of the wave generating source through information detected by the optical sensor and the plurality of ultrasonic sensors. The position of the wave generating source can be calculated using the property that light is much faster than ultrasonic waves, that is, the time for light to reach the optical sensor is much faster than the time for ultrasonic waves to reach the ultrasonic sensor. More specifically, the location of the wave generating source may be calculated by using light as a reference signal and a time difference between the arrival time of ultrasonic waves.

Meanwhile, looking at the configuration of the input unit 120, the camera 121 includes at least one of a camera sensor (eg, CCD, CMOS, etc.), a photo sensor (or image sensor), and a laser sensor.

The camera 121 and the laser sensor may be combined with each other to detect a touch of a sensing target for a 3D stereoscopic image. A photo sensor may be stacked on a display device, and the photo sensor is configured to scan a motion of a sensing target close to the touch screen. More specifically, the photo sensor scans the content placed on the photo sensor by mounting photo diodes and transistors (TRs) in rows/columns and using electrical signals that change according to the amount of light applied to the photo diodes. That is, the photo sensor calculates the coordinates of the sensing object according to the change in light, and through this, the location information of the sensing object can be obtained.

The display unit 151 displays (outputs) information processed by the mobile terminal 100 . For example, the display unit 151 may display execution screen information of an application program driven in the mobile terminal 100 or UI (User Interface) and GUI (Graphic User Interface) information according to such execution screen information. .

Also, the display unit 151 may be configured as a 3D display unit that displays a 3D image.

A 3D display method such as a stereoscopic method (glasses method), an auto stereoscopic method (non-glasses method), or a projection method (holographic method) may be applied to the 3D display unit.

The audio output unit 152 may output audio data received from the wireless communication unit 110 or stored in the memory 170 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, and the like. The sound output unit 152 also outputs sound signals related to functions performed by the mobile terminal 100 (eg, call signal reception sound, message reception sound, etc.). The sound output unit 152 may include a receiver, a speaker, a buzzer, and the like.

The haptic module 153 generates various tactile effects that a user can feel. A representative example of the tactile effect generated by the haptic module 153 may be vibration. The strength and pattern of the vibration generated by the haptic module 153 may be controlled by a user's selection or a setting of a controller. For example, the haptic module 153 may synthesize and output different vibrations or sequentially output them.

In addition to vibration, the haptic module 153 responds to stimuli such as an array of pins that move vertically with respect to the contact skin surface, air blowing force or suction force through a nozzle or suction port, rubbing against the skin surface, electrode contact, and electrostatic force. It is possible to generate various tactile effects, such as the effect of heat absorption and the effect of reproducing the feeling of coolness and warmth using an element capable of absorbing heat or generating heat.

The haptic module 153 not only transmits a tactile effect through direct contact, but also can be implemented so that a user can feel a tactile effect through a muscle sense such as a finger or an arm. Two or more haptic modules 153 may be provided according to the configuration of the mobile terminal 100 .

The light output unit 154 outputs a signal for notifying occurrence of an event using light from a light source of the mobile terminal 100 . Examples of events occurring in the mobile terminal 100 may include message reception, call signal reception, missed calls, alarms, schedule notifications, e-mail reception, and information reception through applications.

A signal output from the light output unit 154 is implemented when the mobile terminal emits light of a single color or multiple colors to the front or back side. The signal output may be terminated when the mobile terminal detects the user's confirmation of the event.

The interface unit 160 serves as a passage for all external devices connected to the mobile terminal 100 . The interface unit 160 receives data from an external device or receives power and transmits it to each component inside the mobile terminal 100, or transmits data inside the mobile terminal 100 to an external device. For example, a wired/wireless headset port, an external charger port, a wired/wireless data port, a memory card port, and a port for connecting a device having an identification module. The interface unit 160 may include an audio I/O (Input/Output) port, a video I/O (Input/Output) port, an earphone port, and the like.

On the other hand, the identification module is a chip that stores various types of information for authenticating the use authority of the mobile terminal 100, and includes a user identification module (UIM), a subscriber identity module (SIM), and universal user authentication. module (universal subscriber identity module; USIM) and the like. A device equipped with an identification module (hereinafter referred to as 'identification device') may be manufactured in the form of a smart card. Accordingly, the identification device may be connected to the terminal 100 through the interface unit 160 .

In addition, the interface unit 160 becomes a passage through which power from the cradle is supplied to the mobile terminal 100 when the mobile terminal 100 is connected to an external cradle, or a user inputs power from the cradle. It can be a passage through which various command signals are transmitted to the mobile terminal 100 . Various command signals or the power input from the cradle may operate as a signal for recognizing that the mobile terminal 100 is correctly mounted on the cradle.

The memory 170 may store programs for operation of the controller 180 and may temporarily store input/output data (eg, phonebook, message, still image, video, etc.). The memory 170 may store data related to vibration and sound of various patterns output when a touch is input on the touch screen.

The memory 170 may be a flash memory type, a hard disk type, a solid state disk type, a silicon disk drive type, or a multimedia card micro type. ), card-type memory (eg SD or XD memory, etc.), RAM (random access memory; RAM), SRAM (static random access memory), ROM (read-only memory; ROM), EEPROM (electrically erasable programmable read -only memory), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. The mobile terminal 100 may be operated in relation to a web storage that performs a storage function of the memory 170 on the Internet.

Meanwhile, as described above, the control unit 180 controls operations related to applications and general operations of the mobile terminal 100 in general. For example, if the state of the mobile terminal satisfies a set condition, the controller 180 may execute or release a locked state that restricts a user's control command input to applications.

In addition, the controller 180 may perform control and processing related to voice calls, data communications, video calls, etc., or perform pattern recognition processing capable of recognizing handwriting input or drawing input performed on the touch screen as characters and images, respectively. can Furthermore, the control unit 180 may control any one or a combination of a plurality of components described above in order to implement various embodiments described below on the mobile terminal 100 according to the present invention.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power necessary for the operation of each component. The power supply unit 190 includes a battery, and the battery may be a built-in battery capable of being charged, or may be detachably coupled to the terminal body for charging.

In addition, the power supply unit 190 may have a connection port, and the connection port may be configured as an example of an interface 160 electrically connected to an external charger that supplies power to charge the battery.

As another example, the power supply unit 190 may charge the battery in a wireless manner without using the connection port. In this case, the power supply unit 190 uses at least one of an inductive coupling method based on magnetic induction from an external wireless power transmitter or a magnetic resonance coupling method based on electromagnetic resonance. power can be delivered.

Meanwhile, various embodiments hereinafter may be embodied in a recording medium readable by a computer or a similar device using, for example, software, hardware, or a combination thereof.

Hereinafter, a mobile terminal having at least one of the components described above may include a sensing unit 140 composed of at least one sensor capable of sensing various environmental information related to the surrounding environment.

The sensing unit 140 may detect various environmental information related to the surrounding environment, particularly environments that humans cannot visually perceive, for example, air and sound having a frequency band other than the audible frequency. there is.

To this end, the sensing unit 140 may include various sensors. For example, the sensing unit 140 may include an ultrasonic sensor, a temperature sensor, a humidity sensor, a carbon dioxide sensor, a gas sensor, a fine dust detection sensor, a volatile organic compound (VOC) sensor, and the like.

The sensing unit 140 may detect surrounding environments, particularly environmental information related to air, and transmit the same to the controller 180 .

Here, the environment information may be information capable of measuring the state or quality of the surrounding environment, for example, temperature information, humidity information, carbon dioxide concentration information, fine dust concentration information, VOC information, and the like.

The sensing unit 140 may be activated based on a user request or when a preset condition is satisfied. Activation of the sensing unit 140 means a state in which sensors of the sensing unit 140 can detect environmental information related to the surrounding environment. Conversely, inactivation of the sensing unit 140 means a state in which sensors cannot detect environmental information related to the surrounding environment.

The sensing unit 140 may sense environmental information related to the surrounding environment in real time or at a predetermined time period. Here, the preset time period may be preset at the time of shipment from the factory or set by the user.

In addition, the sensor 140 may detect environmental information related to the surrounding environment based on a control command of the controller 180 or a user request.

Meanwhile, unlike the foregoing description, the mobile terminal according to the present invention does not include the sensing unit 140 and may receive environment information from external sensors connected by wire or wirelessly.

In the following description, it is assumed that the sensing unit 140 is provided in the mobile terminal, but the present invention is not limited thereto, and the sensing unit 140 can be equally applied even when connected to the mobile terminal by wire or wirelessly. there is.

Hereinafter, in a mobile terminal equipped with such a sensor 140, an operation method using environment information sensed by the sensor 140 will be described in detail.

2 is a flowchart illustrating a control method of a mobile terminal related to the present invention. FIG. 3 is a conceptual diagram illustrating the control method of FIG. 2 .

First, the control unit 180 of the mobile terminal according to the present invention may detect environment information related to the surrounding environment (S210).

A mobile terminal according to the present invention can provide functions related to Internet of Things (IoT). Functions related to the Internet of Things (IoT) technology may be provided through applications installed in mobile terminals.

These applications are installed and shipped when the mobile terminal is shipped from the factory by the manufacturer or carrier of the mobile terminal, or the user can communicate in an open market, for example, in an application market such as Google Play. It can be downloaded through and installed in a mobile terminal.

When the application is executed, the controller 180 may start sensing environment information.

The controller 180 may execute the application when a predetermined condition is satisfied or based on a user request. The preset condition may be a specific time condition or a condition related to a schedule stored in schedule information. For example, the preset condition may be a time to go to work or a time to leave work. As another example, the preset condition may be a condition in which an outing schedule is stored on schedule information.

When the application is executed, the control unit 180 may activate sensors constituting the sensing unit 140 . In addition, the controller 180 may deactivate sensors constituting the detector 140 when the application is terminated.

When the application is executed, the controller 180 may execute functions related to the Internet of Things (IoT) technology provided by the application.

Functions related to the Internet of Things technology are functions that remotely control objects connectable to the Internet through communication. Here, things may be electronic devices capable of communication, such as air conditioners, air purifiers, dehumidifiers, refrigerators, air conditioners, window control devices, door control devices, and the like.

More specifically, the controller 180 may control the operation of things through communication. For example, the controller 180 may operate the air conditioner by transmitting a control command for operating the air conditioner to the air conditioner through communication.

Meanwhile, when the application is executed, the controller 180 may sense information related to the surrounding environment through the sensor 140 .

The control unit 180 may detect the surrounding environment of a space within a preset range based on the position of the mobile terminal through the sensing unit 140, or may detect the surrounding environment of the space selected by the user's request. There is also

If the space selected by the user's request is a location far from the location of the mobile terminal, the controller 180 detects information related to the surrounding environment using an external sensor disposed in the selected space, It can be received through communication with an external sensing unit.

When sensing the surrounding environment, the controller 180 may detect at least one piece of environment information among a plurality of pieces of environment information. For example, the controller 180 may detect only temperature information among a plurality of pieces of environmental information based on a control command.

As another example, the controller 180 may detect at least one piece of environment information among a plurality of pieces of environment information based on a preset condition. The preset condition may be environment information preset in various setting modes provided by the application. For example, in the case of a ventilation mode related to ventilation, the controller 180 may detect temperature information, humidity information, and fine dust information previously set to be detected in the ventilation mode. As another example, in the case of a heating mode related to heating, the controller 180 may detect temperature information preset to be sensed in the heating mode.

When the environment information is sensed, the controller 180 may output the sensed environment information on the display unit 151 . For example, as shown in (a) of FIG. 3 , the controller 180 provides humidity information 310a, temperature information 310b, fine dust concentration information 310c, and carbon dioxide concentration information 310d according to the indoor space. each can be displayed. Interior space refers to the space that makes up the interior, such as the kitchen, master bedroom, and bathroom. Through this, the user can numerically check information related to the environment of each space.

The controller 180 may detect an expected scenario based on the sensed environment information (S220).

Here, the predicted scenario refers to information that predicts a future environmental state based on environmental information currently sensed through the sensing unit 140 . More specifically, the predicted scenario may be virtual state information related to an environment that may occur in a specific space based on currently sensed environment information, or health information related to an effect that may have on a user's body in a specific space. .

For example, if the humidity in a specific space is 70% or higher, an expected scenario may be a scenario in which mold develops on a wall. As another example, when the concentration of fine dust in a specific space is higher than the preset reference concentration, the expected scenario may be a respiratory disease scenario.

That is, the present invention does not simply provide environmental information numerically, but provides information that can be more helpful to the user's real life by providing predictive information such as an expected scenario that predicts a future state based on the environmental information. can provide

The predicted scenario may be pre-stored in the mobile terminal or may be generated by the controller 180 based on environment information. Alternatively, the predicted scenario may be stored in an external server. Here, the external server may transmit/receive data through communication with the mobile terminal, and may be a server storing expected scenario information according to environment information.

More specifically, environment information and predicted scenarios may be linked and stored in the memory 170 of the mobile terminal. For example, the memory 170 may store different expected scenarios corresponding to different environment information.

Also, the controller 180 may generate an expected scenario based on the environment information sensed by the sensor 140 . For example, the controller 180 may generate a scenario in which mold occurs when it is detected that the humidity of the humidity information is 70% or more.

Also, the controller 180 may transmit environment information sensed by the sensor 140 to an external server and receive an expected scenario from the external server. The external server may detect an expected scenario based on environment information received from the mobile terminal and transmit the detected expected scenario to the mobile terminal.

Meanwhile, even if the same environment information is sensed, the controller 180 may detect different expected scenarios based on at least one of a location and a time when the environment information is sensed. That is, the predicted scenario may be detected based on at least one of environment information and a location and time at which the environment information is sensed.

Here, the location where the environmental information is sensed may include an indoor location such as a bathroom, living room, main room, or kitchen, and time may include time by season or time such as day, night, morning, afternoon, winter, or summer.

For example, if indoor temperature information in winter is detected as 18 degrees, the controller 180 detects an expected scenario in which the window is frosted, and if indoor temperature information in summer is detected as 18 degrees, the controller 180 detects an expected scenario in which a user indoors gets air-conditioning sickness. can be detected.

Meanwhile, the predicted scenario may further include information on external devices related to the predicted scenario along with information predicting a future state. Here, the external devices may be IoT devices, that is, external devices controllable through communication with the mobile terminal. An external device included in the predicted scenario may be a recommended device for which execution is recommended based on current environment information.

For example, in the case of an expected scenario in which mold develops on a wall, dehumidifier and air conditioner information may be further included. As another example, in the case of an expected scenario in which a respiratory disease is caused, window control device and air purifier information may be further included.

That is, when there are a plurality of expected scenarios, at least one piece of recommended device information may be included for each expected scenario. Through this, the user may know recommended device information for which an operation is recommended on current environment information. Accordingly, the user can utilize the Internet of Things technology more efficiently.

The controller 180 may output the predicted scenario to a display unit (S230).

When an expected scenario is detected based on environment information, the controller 180 may output the detected expected scenario on the display unit 151 . For example, as shown in (b) of FIG. 3 , the controller 180 may output on the display unit 151 an expected scenario 320 in which mold occurs on a wall. Accordingly, the user can visually check the predicted scenario output on the display unit 151 .

In addition to the visual method, the controller 180 may output the predicted scenario in at least one of an auditory method and a tactile method.

Meanwhile, the controller 180 may output an expected scenario on the display unit 151 provided in the mobile terminal, but may also output visual information to an external display device through communication. That is, the controller 180 may transmit predicted scenarios to an external display device through communication, and the external display device may output predicted scenario information.

When there are a plurality of display units, a location for outputting an expected scenario may be determined by a user's selection or a preset condition. Here, the preset condition may be a display unit preset condition for the user's current location or expected scenario.

For example, in the case of an expected scenario in which mold develops on a wall, the controller 180 may control the display device to output expected scenario information to a display device (eg, a TV installed on the wall) installed near the wall.

As another example, in the case of an expected scenario causing a respiratory disease, the controller 180 may control the display unit to output predicted scenario information to the display unit of a mobile terminal installed closest to the location of the current user.

Through this, the mobile terminal according to the present invention can provide an expected scenario at the most appropriate location for the user to perceive.

Meanwhile, although not shown, the controller 180 may make the expected scenario disappear from the display unit 151 based on a preset condition being satisfied. For example, the controller 180 may make the predicted scenario disappear from the display unit 151 when newly sensed environment information after detection of the predicted scenario is different from environmental information sensed before detection of the predicted scenario.

Hereinafter, unless otherwise noted, description will be given based on displaying an expected scenario on the display unit of the mobile terminal. However, the following description can be equally applied even when an expected scenario is displayed on an external display unit.

The controller 180 may detect a user gesture in a state where the expected scenario is output to the display unit (S240).

The controller 180 may detect a user gesture through the user input unit 123 . A user gesture is a user's motion related to the input of a control command, such as a touch gesture touching or approaching a display unit, a user's movement sensed at a close distance to a mobile terminal, and a gesture generating a sound.

For example, as shown in (c) of FIG. 3 , the controller 180 may detect a touch input 330 in the form of rubbing the display unit 151 through a touch sensor. As another example, as shown in (a) of FIG. 5A , the controller 180 may detect a user gesture in the form of a “hu” sound through a microphone.

The controller 180 may control the operation of an external device in response to the user's gesture (S250).

When a user gesture is input while an expected scenario is displayed on the display unit 151, the controller 180 may control an operation of an external device related to the expected scenario in response to the user gesture.

For example, as shown in (d) of FIG. 3 , the controller 180 may operate the air conditioner in response to a user gesture. More specifically, the controller 180 may transmit an operation command to the air conditioner through communication in order to operate the air conditioner. Also, the controller 180 may output notification information 340 indicating that the air conditioner has been operated to the display unit 151 . Through this, the present invention can more intuitively provide the current operating state to the user.

Meanwhile, although not shown, the control unit 180 may perform operation-related control, such as operation time and operation intensity of external devices, based on the sensed environment information.

For example, when the humidity is 70%, the controller 180 transmits an operation command for setting the operation time of the air conditioner to 1 hour to the air conditioner, and when the humidity is 90%, the operation time of the air conditioner is set to 2 hours. One operation command can be transmitted to the air conditioner. In this case, the air conditioner can automatically stop its operation when the operating time elapses.

That is, the controller 180 may control power on/off, operating time, operating intensity, operating state, and the like of the external device as a control related to the operation of the external device. Through this, the present invention can automatically control an external device appropriate for the current environment without a user's separate manipulation.

In the above, a method for providing an expected scenario and performing control related thereto by sensing environmental information related to the surrounding environment in a mobile terminal according to the present invention has been described. Through this, the present invention not only visually provides environmental information that is difficult to visually confirm to the user, but also provides expected information related thereto, so that the user can help the user more intuitively recognize the influence of this environmental information. there is. In addition, the present invention can implement a more convenient Internet of Things technology by directly performing an operation related to the expected scenario when a user gesture input is applied when the expected scenario is provided.

Hereinafter, providing different expected scenarios based on at least one of environment information, location, and time will be described together with FIGS. 4A and 4B.

Even when the same environment information is sensed, the controller 180 may provide different expected scenarios based on at least one of location and time. That is, since the environmental information may be set differently depending on the location at which the environmental information is detected or the time at which the environmental information is detected, even if the temperature is the same, the control unit 180 determines whether the environment information is detected even if the environment information is the same. Depending on the location or time, it can provide different expected scenarios.

For example, as shown in (a) of FIG. 4A, the controller 180 can detect that the current humidity is 50%, the temperature is 25 degrees, the fine dust concentration is good, and the carbon dioxide concentration is 30% as environmental information of the main room in the house. there is.

The controller 180 may detect an expected scenario in which mold is generated on the wall based on the location information indicating the main room and the sensed environment information. The controller 180 may output an expected scenario 320 in which mold occurs on a wall on the display unit 151 .

On the other hand, as shown in (a) of FIG. 4B, the controller 180 can detect that the current humidity is 50%, the temperature is 25 degrees, the fine dust concentration is good, and the carbon dioxide concentration is 30% as environmental information of the bathroom in the house.

The controller 180 may generate an expected scenario in which frost occurs on a window based on the location information indicating the toilet and the sensed environment information. In addition, the controller 180 may output to the display unit 151 an expected scenario 420 in which frost occurs on the window.

That is, the controller 180 may output different expected scenarios according to the location where the environment information is sensed, even if the environment information is the same. Accordingly, the present invention may provide a more realistic expected scenario to the user.

Hereinafter, a control method related to the expected scenario described in FIG. 2 will be described in detail. 5A and 5B are conceptual diagrams illustrating that different controls are performed according to user gestures in a mobile terminal related to the present invention. Also, FIGS. 6A and 6B are conceptual views illustrating that different controls are performed based on environment information of a surrounding environment in a mobile terminal related to the present invention.

When a user gesture is detected while the expected scenario is output on the display unit 151, the controller 180 may control operations of external devices related to the expected scenario in response to the user gesture.

Meanwhile, the controller 180 may control operations of different external devices in response to different user gestures in the same expected scenario.

More specifically, the controller 180 may execute a specific external device corresponding to a user gesture among a plurality of external device information included in the expected scenario.

For example, as shown in (a) of FIG. 5A, the controller 180 detects an expected scenario in which the visibility distance due to fine dust is shortened based on the environmental information measured as the fine dust concentration is bad, and the detected An expected scenario may be output to the display unit 151 . The controller 180 may detect input of a first user gesture in the form of a “hu” sound while the expected scenario is displayed on the display unit 151 .

In this case, as shown in (b) of FIG. 5A, the controller 180 may operate the air purifier in response to the first user's gesture in the form of a “hu” sound. Here, the air purifier is one of a plurality of external devices included in an expected scenario in which the visible distance due to fine dust is shortened, and may be an external device corresponding to a first user gesture in the form of a “hu” sound.

As another example, as shown in (a) of FIG. 5B, the controller 180 detects an expected scenario in which the visibility distance due to fine dust is shortened based on the environmental information measured as the fine dust concentration is bad, and the detected An expected scenario may be output to the display unit 151 . That is, the same expected scenario as in (a) of FIG. 5A can be detected.

At this time, as shown in (a) of FIG. 5B, the controller 180 inputs a second user gesture, which is a touch input in the form of rubbing the display unit 151, while the expected scenario is displayed on the display unit 151. can sense what is going on.

In this case, as shown in (b) of FIG. 5B, the controller 180 controls the window control device in response to the second user gesture, which is a touch input in the form of rubbing the display unit 151, to close the window. can be done

That is, even in the same expected scenario, the control unit 180 can determine the user's intention through the user gesture by performing an operation of controlling different external devices according to the user gesture, and appropriately control the IoT devices accordingly. can

As shown in FIGS. 5A(B) and 5B(B), when an external device is operated, the controller 180 visually displays notification information (eg, a pop-up window) on the display unit 151. can be output as Accordingly, the user can immediately recognize the operating state of the external device.

Meanwhile, the controller 180 may control the operations of different external devices even when the same user gesture is applied in the same expected scenario.

More specifically, the controller 180 may control the operation of different external devices according to environment information and product performance even when the same user gesture is applied to the same expected scenario.

For example, as shown in (a) of FIG. 6A , the controller 180 may detect application of a touch input to an expected scenario in which mold on the wall occurs. At this time, as shown in (b) of FIG. 6A, when the environment information is the first humidity information, the controller 180 may operate a dehumidifier that is an external device corresponding to the first humidity information.

In contrast, as shown in (a) of FIG. 6B, the controller 180 may detect application of a touch input for the same expected scenario as in (a) of FIG. 6A. At this time, as shown in (b) of FIG. 6B, when the environment information is second humidity information having a higher humidity than the first humidity information, the controller 180 may operate an air conditioner that is an external device corresponding to the second humidity information. .

That is, when humidity is high, the controller 180 uses devices having good humidity removal performance, that is, an air conditioner.

As described above, the controller 180 may operate different external devices according to environmental information and product performance even when the same touch input is applied in the same expected scenario.

Meanwhile, although not shown, the controller 180 may display the operating state of the external device on the display unit 151 when the external device is operating.

For example, when the air conditioner is operating, the controller 180 may output a graphic object representing the strength and direction of wind output from the air conditioner on the display unit 151 .

At this time, the controller 180 may control the operating state of the air conditioner based on a touch input applied to the graphic object. For example, the controller 180 may change the wind strength of the air conditioner based on a touch input for inducing movement of the graphic object. That is, the present invention can control not only the operation and termination of external devices, but also the execution state.

Hereinafter, a method of providing a recommended device for an expected scenario will be described. 7 is a conceptual diagram illustrating a method of providing a recommended device according to a user gesture in a mobile terminal related to the present invention.

The controller 180 may detect at least one of a plurality of external devices related to the expected scenario as a recommended device in response to a user gesture being detected while the expected scenario is being displayed.

The recommended device is an external device related to an expected scenario and refers to a device for which execution is recommended.

Also, the recommended device may be at least one external device with a high use frequency among a plurality of external devices related to the expected scenario. For example, if there are five external devices related to the expected scenario, the recommended devices may be two frequently used external devices. Alternatively, the recommended device may be at least one external device having good device performance among a plurality of external devices related to the expected scenario. For example, if there are 5 external devices related to the expected scenario, the recommended devices may be 2 external devices in order of highest device performance.

After that, the controller 180 may output a recommendation list including the at least one recommended device on the display unit 151 . For example, as shown in (a) and (b) of FIG. 7 , the controller 180 selects recommended devices based on a user's touch input applied to the display unit 151 on which an expected scenario is displayed. A recommendation list 700 including the list may be output.

The controller 180 may execute a specific recommended device in response to a touch input being applied to a specific recommended device among the recommended devices included in the recommendation list 700 . For example, as shown in (c) of FIG. 7 , the controller 180 may operate the air conditioner in response to a touch input applied to an item representing the air conditioner.

Through this, the present invention can more conveniently perform an operation related to an expected scenario.

Hereinafter, when environment information of the surrounding environment is changed, operations related thereto will be described. 8 is a flowchart illustrating a control method related to an operation of a mobile terminal when an external environment is changed in a mobile terminal related to the present invention.

As described in FIG. 3 , the controller 180 may operate an external device related to an expected scenario when a user gesture is input.

The controller 180 may detect new environment information while an external device is operating according to an expected scenario (S810).

The controller 180 may continue to sense surrounding environment information in real time or at a preset period while the external device is in operation.

The controller 180 may terminate the operation of the external device when the environment information sensed through the sensor 140 is different from the environment information sensed before the operation of the external device (S820). Alternatively, after the operation of the external device, the controller 180 may terminate the operation of the external device when the environment information sensed through the sensor 140 corresponds to pre-stored environment information.

That is, the controller 180 may end the operation of the external device when it is determined that the currently sensed environment information no longer requires the operation of the external device.

The above description has been based on the case where the operation according to the present invention is implemented in a mobile terminal. However, the control described above may equally operate in a server constituting the IoT system. Hereinafter, the Internet of Things system will be reviewed together with FIGS. 9 to 11 .

In the IoT system, one server can control things through communication with things existing in a house.

Referring to FIG. 10 , the IoT system may include a communication unit 1100, a location detection unit 1200, a display unit 1500, a detection unit 1400, and a control unit 1800.

The communication unit 1100 is a component that performs communication between things and the Internet of Things system. The location detection unit 1200 is a component that detects location information of a user's location or objects in the house. The display unit 1500 is a component that outputs visual information, and the sensor 1400 is a component that detects environmental information related to the surrounding environment of the house.

The control unit 1800 is a component that controls the components described above, and is used as the same term as a server of the Internet of Things system. That is, the controller 1800 may communicate with each component through wired or wireless communication to control objects in the house.

The Internet of Things system can be installed in each space in the house. For example, as shown in FIG. 9 , when a space in a house is divided into a bathroom 930, a main room 920, and a living room 910, display units 910a, 910c, 920a, and 930a for each space can be installed. In addition, detection units 910b, 920b, and 930b may be installed in each space. That is, the Internet of Things system includes a separate display unit and a sensing unit for each space, and accordingly, the controller 1800 can easily obtain and provide information of each space.

The control unit 1800 of the IoT system according to the present invention may perform the control operation of FIG. 3 .

For example, the controller 1800 may detect environmental information through the sensor 1400 and output a predicted scenario based on the detection on the display unit 1500 .

Meanwhile, since the controller 1800 includes separate display units 910a, 910c, 920a, and 930a and detectors 910b, 920b, and 930b for each space, the predicted scenario is appropriately displayed on the display units 910a, 910c, and 910c. 920a, 930a).

More specifically, the controller 1800 may output an expected scenario to a specific display unit among the plurality of display units 910a, 910c, 920a, and 930a based on a location related to the expected scenario or a user's location.

For example, in the case of an expected scenario detected based on environment information sensed through the sensor 930b installed in the bathroom 930, the controller 1800 displays the expected scenario on the display unit 930a installed in the bathroom 930. can output

As another example, when the user's location information is the living room 920, the controller 1800 may output an expected scenario to the display unit 920a installed in the living room 920.

Also, when the location information of the user is changed, the controller 1800 may output the expected scenario to a display unit disposed closest to the changed location information. For example, as shown in FIG. 11 , when the user's location information is moved from the bedroom 920 to the living room 910, the controller 180 displays a plurality of display units 910a, 910c, 920a, and 930a. An expected scenario may be output to the display unit 910c closest to the user's location.

As described above, according to the mobile terminal according to the present invention, the user can more intuitively recognize the state related to the surrounding environment by visually providing environment information related to the surrounding environment that cannot be visually seen.

In addition, according to the mobile terminal according to the present invention, based on environment information related to the surrounding environment, an expected scenario indicating a predictable state of the surrounding environment is provided, so that the user can predict the future situation in advance.

In addition, according to the mobile terminal according to the present invention, it is possible to control the operation of the surrounding objects appropriately to the current environment by controlling the surrounding objects through the Internet based on environment information related to the surrounding environment.

The above-described present invention can be implemented as computer readable code on a medium on which a program is recorded. The computer-readable medium includes all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. , and also includes those implemented in the form of a carrier wave (eg, transmission over the Internet). Also, the computer may include the control unit 180 of the terminal. Accordingly, the above detailed description should not be construed as limiting in all respects and should be considered illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

Claims (10)

display unit;
a sensing unit for detecting environmental information; and
a controller configured to detect an expected scenario based on the detected environment information and control the display unit to output the detected expected scenario to a display unit;
The sensed environmental information,
Current environment information of a specific space within a preset range based on the location of the mobile terminal or a specific space selected by a user's request,
The expected scenario is
Based on the sensed current environment information, prediction information representing an environment state that can occur in the specific space and information on at least one recommended device for which execution is recommended are included;
The control unit,
Detect a user gesture with respect to the display unit on which the expected scenario is output;
In response to the detected user gesture, controlling the operation of one or more recommended devices related to the expected scenario;
In a state in which the one or more recommended devices are in operation, environmental information is sensed at a predetermined cycle,
The mobile terminal of claim 1 , wherein the mobile terminal stops the operation of the one or more recommended devices when the detected environment information is different from the environment information sensed prior to the operation of the one or more recommended devices or corresponds to preset environment information.
According to claim 1,
The control unit,
A mobile terminal characterized in that different expected scenarios are detected based on at least one of a location and a time when the environment information is sensed.
delete According to claim 1,
The control unit
A mobile terminal characterized by operating different external devices in response to detection of different user gestures.
delete According to claim 1,
The control unit,
The mobile terminal characterized in that in response to a user gesture being input, a recommendation list including the at least one recommended device is output to the display unit.
According to claim 6,
The control unit,
A mobile terminal characterized in that executing a specific recommended device among at least one recommended device included in the recommendation list based on a touch input.
According to claim 1,
The display unit includes a plurality of displays disposed at different positions,
The control unit
The mobile terminal characterized in that the expected scenario is output to a specific display disposed at a location related to the predicted scenario among the plurality of displays.
According to claim 8,
Further comprising a location detector for detecting a user's location,
The control unit
The mobile terminal characterized in that the expected scenario is output to a display disposed closest to the location of the user detected by the location detector among the plurality of displays.
According to claim 1,
The control unit
The mobile terminal of claim 1 , wherein the expected scenario disappears from the display unit when new environment information different from the sensed environment information is detected through the sensor unit.

Images