KR102140740B1 - A mobile device, a cradle for mobile device, and a method of managing them - Google Patents

Abstract

The present invention relates to a mobile terminal and a cradle for a mobile terminal driven according to a user's operation or voice, and a driving method thereof. The mobile terminal of the present invention includes an output unit for outputting signals and information; An input unit for sensing a user's motion and voice; And a control unit generating the behavior pattern information of the user based on the detected user's motion and voice, and predicting the user's behavior according to the behavior pattern information.

Description

Mobile terminal, cradle for mobile terminal and their driving method{A MOBILE DEVICE, A CRADLE FOR MOBILE DEVICE, AND A METHOD OF MANAGING THEM}

The present invention relates to a mobile terminal and a cradle for a mobile terminal, and more particularly, to a mobile terminal driven by a user's motion and voice, a cradle for a mobile terminal, and a driving method thereof.

2. Description of the Related Art Recently, as the use of portable terminals such as smartphones and tablet PCs has become common, development of devices for charging and mounting of portable terminals has also been actively conducted. In general, the mobile terminal's cradle cannot be moved by itself, and is used only for charging and mounting the mobile terminal because it has a structure for fixing the mobile terminal.

In contrast, users have a desire to use the cradle for more various uses. For example, it is desired to be able to use the functions of the portable terminal even when the portable terminal mounted on the cradle cannot be directly operated by hand. Therefore, there is a need for a new portable terminal and a portable terminal holder that provide more various functions.

The present invention is proposed to meet the above-mentioned needs, and it is an object of the present invention to provide a mobile terminal, a mobile terminal cradle driven by a user's motion or voice, and a driving method for the mobile terminal.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by a person having ordinary knowledge in the technical field to which the present invention belongs from the following description. Will be able to...

In order to achieve the above object, the present invention, an output unit for outputting signals and information; An input unit for sensing a user's motion and voice; It provides a mobile terminal including a control unit for generating the behavior pattern information of the user based on the detected user's motion and voice, and predicts the user's behavior according to the behavior pattern information.

Here, the behavior pattern information includes information on the user's home appliance use history, and the controller predicts the user's home appliance use schedule based on the user's home appliance use history, and the The output unit is controlled to output at least one of signals and information for guiding the user to use the home appliance.

The control unit controls the mobile terminal cradle so that the front of the mobile terminal faces toward the position where the user's motion or voice is sensed, and controls the operation of the mobile terminal cradle according to the state change of the mobile terminal.

In addition, the control unit controls the output unit to predict the user's behavior using the environment pattern information together with the behavior pattern information, and output at least one of signals and information for guiding the predicted behavior.

In addition, to achieve the above object, the present invention comprises the steps of sensing a user's motion and voice; Generating behavior pattern information of the user based on the detected user's motion and voice; It provides a method of driving a mobile terminal comprising the step of predicting the user's behavior according to the behavior pattern information.

When explaining the effect of the mobile terminal and the cradle for a mobile terminal according to the present invention.

According to at least one of the embodiments of the present invention, since the mobile terminal and the mobile terminal cradle recognize and react to the user's motion or voice, the user can control the mobile terminal and the mobile terminal cradle even without direct contact. have.

In addition, since the mobile terminal and the cradle for the mobile terminal predict the user's behavior based on the user's motion or voice and perform the corresponding motion, various conveniences can be provided to the user.

The effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those skilled in the art from the following description. will be.

1 is a block diagram showing an example of a home robot according to the present invention.
2 is a view showing an example of a mobile terminal according to the present invention.
3 is a view showing an example of a cradle for a mobile terminal according to the present invention.
4A and 4B are views showing a home robot in which a mobile terminal and a cradle are combined.
5A to 5C are diagrams showing examples of emotional expression using a mobile terminal and a cradle.
6 is a view showing an example of the function of the home robot according to the present invention.
7 is a flow chart showing a method of driving a home robot according to the present invention.
8 shows an example of behavior pattern information and surrounding environment information.
9A to 9D are diagrams illustrating examples of the operation of a mobile terminal according to a user's operation and voice.

Hereinafter, exemplary embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but the same or similar elements are assigned the same reference numbers regardless of the reference numerals, and overlapping descriptions thereof will be omitted. The suffixes "modules" and "parts" for components used in the following description are given or mixed only considering the ease of writing the specification, and do not have meanings or roles distinguished from each other in themselves. In addition, in the description of the embodiments disclosed herein, when it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed herein, detailed descriptions thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in the present specification, and the technical spirit disclosed in the specification is not limited by the accompanying drawings, and all modifications included in the spirit and technical scope of the present invention , It should be understood to include equivalents or substitutes.

Terms including ordinal numbers such as first and second may be used to describe various components, but the components are not limited by the terms. The terms are used only for the purpose of distinguishing one component from other components.

When an element is said to be "connected" or "connected" to another component, it is understood that other components may be directly connected to or connected to the other component, but there may be other components in between. It should be. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that no other component exists in the middle.

Singular expressions include plural expressions unless the context clearly indicates otherwise.

In this application, terms such as “comprises” or “have” are intended to indicate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, and that one or more other features are present. It should be understood that the existence or addition possibilities of fields or numbers, steps, operations, components, parts or combinations thereof are not excluded in advance.

1 is a view for explaining an example of a home robot according to the present invention. As shown in Figure 1, the home robot of the present invention includes a mobile terminal 100, a mobile terminal 200. When the mobile terminal 100 is mounted on the cradle 200, it can be used as a fixed robot, and when the traveling platform 270 is coupled to the fixed robot, it can be used as a mobile robot. The functions of the home robot largely include information transmission, emotional transmission, pet function, user's health management, etc., and through these functions, it can contribute to providing convenience to users, especially single and elderly people, and resolving loneliness. have. In addition, it is possible to provide a necessary function to a user by combining a cleaning module or an air cleaning module with such a home robot.

The mobile terminal 100 described herein includes a mobile phone, a smart phone, a laptop computer, a terminal for digital broadcasting, a personal digital assistants (PDA), a portable multimedia player (PMP), navigation, a slate PC ( slate PC), a tablet PC (tablet PC), or an ultrabook (ultrabook). However, it will be readily appreciated by those skilled in the art that the configuration according to the embodiment described herein may be applied to a fixed terminal including a camera, except when applicable only to a mobile terminal.

2 is a block diagram illustrating an example of a mobile terminal 100 related to the present invention.

The mobile terminal 100 includes a wireless communication unit 110, an input unit 120, a sensing unit 140, an output unit 150, an interface unit 160, a memory 170, a control unit 180, and a power supply unit 190 And the like. The components shown in FIG. 2 are not essential for implementing a mobile terminal, so the mobile terminal described herein may have more or fewer components than those listed above.

More specifically, among the above components, the wireless communication unit 110 moves between the mobile terminal 100 and the wireless communication system, between the mobile terminal 100 and another mobile terminal 100, or different from the mobile terminal 100. The terminal 100, or an external server, may include one or more modules that enable wireless communication between networks.

The wireless communication unit 110 may include at least one of a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short-range communication module 114, and a location information module 115. .

The input unit 120 may include a camera 121 for inputting a video signal or a video input unit, a microphone for inputting an audio signal (microphone 122), or an audio input unit, a user input unit 123 for receiving information from a user, for example , A touch key, a mechanical key, and the like. The voice data or image data collected by the input unit 120 may be analyzed and processed by a user's control command.

The sensing unit 140 may include one or more sensors for sensing at least one of information in the mobile terminal, surrounding environment information surrounding the mobile terminal, and user information. For example, the sensing unit 140 includes a proximity sensor 141, an illumination sensor 142, a touch sensor, an acceleration sensor, a magnetic sensor, and gravity G-sensor, gyroscope sensor, motion sensor, RGB sensor, infrared sensor (IR sensor), fingerprint scan sensor, ultrasonic sensor , Optical sensor (e.g. camera (see 121)), microphone (see 122), battery gage, environmental sensor (e.g. barometer, hygrometer, thermometer, radioactivity sensor, heat Detection sensors, gas detection sensors, etc.), chemical sensors (eg, electronic nose, health care sensors, biometric sensors, etc.), on the other hand, the mobile terminal disclosed herein, such sensors, these sensors Among them, information sensed by at least two or more sensors may be used in combination.

The output unit 150 is for generating output related to vision, hearing, or tactile sense, and includes at least one of a display unit 151, an audio output unit 152, a hap tip module 153, and an optical output unit 154 can do. The display unit 151 may form a mutual layer structure with the touch sensor or may be integrally formed, thereby realizing a touch screen. The touch screen may function as a user input unit 123 that provides an input interface between the mobile terminal 100 and a user, and at the same time, provide an output interface between the mobile terminal 100 and the user.

The interface unit 160 serves as a passage with various types of external devices connected to the mobile terminal 100. The interface unit 160 connects a device equipped with a wired/wireless headset port, an external charger port, a wired/wireless data port, a memory card port, and an identification module. It may include at least one of a port, an audio input/output (I/O) port, a video input/output (I/O) port, and an earphone port. In the mobile terminal 100, in response to an external device being connected to the interface unit 160, appropriate control related to the connected external device may be performed.

The memory 170 may store a number of application programs (application programs) driven in the mobile terminal 100, data for operating the mobile terminal 100, and instructions. At least some of these applications can be downloaded from external servers via wireless communication. In addition, at least some other of these application programs may exist on the mobile terminal 100 from the time of shipment for basic functions of the mobile terminal 100 (for example, an incoming call, a calling function, a message reception, and a calling function). . Meanwhile, the application program may be stored in the memory 170 and installed on the mobile terminal 100 to be driven by the controller 180 to perform an operation (or function) of the mobile terminal.

The controller 180 controls the overall operation of the mobile terminal 100 in addition to the operations related to the application program. The controller 180 may provide or process appropriate information or functions to the user by processing signals, data, information, etc. input or output through the above-described components or by driving an application program stored in the memory 170.

In addition, the controller 180 may control at least some of the components of FIG. 2A to drive the application program stored in the memory 170. Furthermore, the controller 180 may operate by combining at least two or more of the components included in the mobile terminal 100 to operate the application program.

Under the control of the controller 180, the power supply unit 190 receives external power and internal power to supply power to each component included in the mobile terminal 100. The power supply unit 190 includes a battery, and the battery may be a built-in battery or a replaceable battery.

At least some of the components may operate in cooperation with each other to implement an operation, control, or control method of a mobile terminal according to various embodiments described below. Also, the operation, control, or control method of the mobile terminal may be implemented on the mobile terminal by driving at least one application program stored in the memory 170.

Hereinafter, the components listed above will be described in more detail with reference to FIG. 2.

First, referring to the wireless communication unit 110, the broadcast reception module 111 of the wireless communication unit 110 receives a broadcast signal and/or broadcast-related information from an external broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. Two or more broadcast reception modules may be provided to the mobile terminal 100 for simultaneous broadcast reception or switching of broadcast channels for at least two broadcast channels.

The mobile communication module 112 includes technical standards or communication methods for mobile communication (eg, Global System for Mobile Communication (GSM), Code Division Multi Access (CDMA), Wideband CDMA (WCDMA), High Speed HSDPA) Downlink Packet Access (LTE, Long Term Evolution (LTE), etc.) transmits and receives wireless signals to and from at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data according to transmission and reception of a voice call signal, a video call signal, or a text/multimedia message.

The wireless Internet module 113 refers to a module for wireless Internet access, and may be built in or external to the mobile terminal 100. The wireless Internet module 113 is configured to transmit and receive wireless signals in a communication network according to wireless Internet technologies. Wireless Internet technologies include, for example, WLAN (Wireless LAN), WiFi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink) Packet Access), LTE (Long Term Evolution), and the like, and the wireless Internet module 113 transmits and receives data according to at least one wireless Internet technology in a range including Internet technologies not listed above.

The short-range communication module 114 is for short-range communication, Bluetooth™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, NFC (Near Field Communication), by using at least one of Wi-Fi (Wireless-Fidelity), Wi-Fi Direct technology, it can support short-range communication. The short-range communication module 114, between the mobile terminal 100 and the cradle 200, between the mobile terminal 100 and another mobile terminal, or the mobile terminal 100 through a wireless local area network (Wireless Personal Area Networks) And other mobile terminals (or external servers).

Here, another mobile terminal is a wearable device (for example, a smartwatch, smart glass, HMD) capable of exchanging data (or interoperable) with the mobile terminal 100 according to the present invention. (head mounted display)). The short-range communication module 114 may detect (or recognize) a wearable device capable of communicating with the mobile terminal 100 around the mobile terminal 100. Furthermore, when the detected wearable device is a device authenticated to communicate with the mobile terminal 100 according to the present invention, the controller 180 may include at least a part of data processed by the mobile terminal 100, and the short-range communication module ( 114) to the wearable device. Therefore, the user of the wearable device can use data processed by the mobile terminal 100 through the wearable device. For example, according to this, the user performs a phone call through the wearable device when a call is received by the mobile terminal 100 or, when a message is received by the mobile terminal 100, the received message through the wearable device It is possible to check the message.

The location information module 115 is a module for acquiring a location (or current location) of a mobile terminal, and typical examples thereof include a Global Positioning System (GPS) module or a Wireless Fidelity (WiFi) module. For example, if the mobile terminal utilizes a GPS module, the location of the mobile terminal can be obtained using a signal sent from a GPS satellite. As another example, when the mobile terminal utilizes the Wi-Fi module, the location of the mobile terminal may be obtained based on information of a Wi-Fi module and a wireless access point (AP) that transmits or receives a wireless signal.

Next, looking at the input unit 120 in more detail, the input unit 120 is for input of image information (or signals), audio information (or signals), or information input from a user, and inputs image information. To this end, the mobile terminal 100 may include one or a plurality of cameras 121. The camera 121 processes image frames such as still images or moving pictures obtained by an image sensor in a video call mode or a shooting mode. The processed image frame may be displayed on the display unit 151. Meanwhile, the plurality of cameras 121 provided in the mobile terminal 100 may be arranged to form a matrix structure, and through the camera 121 forming the matrix structure, various angles or focuses may be applied to the mobile terminal 100. A plurality of image information may be input. Also, the plurality of cameras 121 may be arranged in a stereo structure to acquire left and right images for realizing a stereoscopic image.

The microphone 122 processes external sound signals as electrical voice data. The processed voice data may be used in various ways according to a function (or a running application program) being performed by the mobile terminal 100. Meanwhile, various noise cancellation algorithms for removing noise generated in the process of receiving an external sound signal may be implemented in the microphone 122.

The user input unit 123 is for receiving information from a user. When information is input through the user input unit 123, the controller 180 may control the operation of the mobile terminal 100 to correspond to the input information. . The user input unit 123 is a mechanical input means (or a mechanical key, for example, a button located on the front, rear or side of the mobile terminal 100, a dome switch, a jog wheel, Jog switch, etc.) and a touch input means. As an example, the touch-type input means is made of a virtual key, a soft key or a visual key displayed on the touch screen through software processing, or a part other than the touch screen It may be made of a touch key (touch key) disposed on. Meanwhile, the virtual key or the visual key may be displayed on the touch screen while having various forms, for example, a graphic, text, icon, video, or the like. It can be made of a combination of.

Meanwhile, the sensing unit 140 senses at least one of information in the mobile terminal, surrounding environment information surrounding the mobile terminal, and user information, and generates a sensing signal corresponding thereto. The controller 180 may control driving or operation of the mobile terminal 100 based on the sensing signal, or perform data processing, function, or operation related to an application program installed in the mobile terminal 100. The representative sensors among various sensors that may be included in the sensing unit 140 will be described in more detail.

First, the proximity sensor 141 refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object in the vicinity using mechanical force or infrared light without mechanical contact. The proximity sensor 141 may be provided with a proximity sensor 141 in an inner region of the mobile terminal wrapped by the touch screen described above or near the touch screen. The proximity sensor 141 has a longer lifespan and a higher utilization than the contact sensor.

The display unit 151 displays (outputs) information processed by the mobile terminal 100. For example, the display unit 151 may display execution screen information of an application program driven by the mobile terminal 100, or user interface (UI) or graphical user interface (GUI) information according to the execution screen information. . In addition, the display unit 151 may be configured as a stereoscopic display unit for displaying a stereoscopic image. A three-dimensional display method such as a stereoscopic method (glasses method), an auto stereoscopic method (glasses-free method), or a projection method (holographic method) may be applied to the stereoscopic display unit.

The audio output unit 152 may output audio data received from the wireless communication unit 110 or stored in the memory 170 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, or the like. The sound output unit 152 may also output sound signals related to functions (for example, call signal reception sound, message reception sound, etc.) performed by the mobile terminal 100. The sound output unit 152 may include a receiver, a speaker, a buzzer, and the like.

The haptic module 153 generates various tactile effects that the user can feel. A typical example of the tactile effect generated by the haptic module 153 may be vibration. The intensity and pattern of vibration generated in the haptic module 153 may be controlled by a user's selection or setting of a control unit. For example, the haptic module 153 may synthesize different vibrations and output them sequentially or sequentially. In addition to vibration, the haptic module 153 is used for stimulation such as pin arrangement that vertically moves with respect to the contact skin surface, jet or suction power of air through a jet or intake, grazing on the skin surface, contact with an electrode, and electrostatic force. Various tactile effects can be generated, such as the effect caused by and the effect of reproducing the feeling of cold and warm using an element capable of absorbing heat or generating heat.

The light output unit 154 outputs a signal for notifying the occurrence of an event using the light of the light source of the mobile terminal 100. Examples of events occurring in the mobile terminal 100 may include receiving messages, receiving call signals, missed calls, alarms, schedule notifications, receiving emails, and receiving information through applications. The signal output by the light output unit 154 is implemented as the mobile terminal emits light of a single color or a plurality of colors to the front or rear. The signal output may be terminated by the mobile terminal detecting the user's event confirmation.

The interface unit 160 serves as a passage with all external devices connected to the mobile terminal 100. The interface unit 160 receives data from an external device, receives power, transfers it to each component inside the mobile terminal 100, or allows data inside the mobile terminal 100 to be transmitted to the external device. For example, a wired/wireless headset port, an external charger port, a wired/wireless data port, a memory card port, a port for connecting a device equipped with an identification module An interface unit 160 may include a (port), an audio input/output (I/O) port, a video input/output (I/O) port, and an earphone port.

The memory 170 may store a program for the operation of the controller 180 and may temporarily store input/output data (eg, a phone book, a message, a still image, a video, etc.). In particular, the memory 170 stores data such as video, voice, and sound transmitted from the wearable terminal 300/400. Further, the memory 170 may store data related to various patterns of vibration and sound output when a touch is input on the touch screen.

The memory 170 is a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory, etc.), RAM. (random access memory; RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic It may include a storage medium of at least one type of a disc and an optical disc. The mobile terminal 100 may be operated in connection with a web storage performing a storage function of the memory 170 on the Internet.

Meanwhile, as described above, the controller 180 controls the operation related to the application program and the overall operation of the mobile terminal 100 in general. For example, if the state of the mobile terminal satisfies a set condition, the controller 180 may execute or release a lock state that limits the input of a user's control command to applications.

In addition, the controller 180 performs control and processing related to voice calls, data communication, video calls, etc., or performs pattern recognition processing capable of recognizing handwriting input or picture drawing input performed on a touch screen as text and images, respectively. Can be. Furthermore, in order to implement various embodiments described below on the mobile terminal 100 according to the present invention, the controller 180 may control any one or a combination of the above-described components.

Meanwhile, various embodiments below may be implemented in a recording medium readable by a computer or a similar device using, for example, software, hardware, or a combination thereof.

3 is a block diagram illustrating an example of a cradle 200 for a mobile terminal related to the present invention.

The cradle 200 for a mobile terminal of the present invention includes a cradle 210, a driver 220, a controller 230, a display 240, and a sensor 250. The components shown in FIG. 3 are not essential to the implementation of the cradle, so the cradle described herein may have more or fewer components than those listed above.

The mounting unit 210 may include a mounting arm for mounting the mobile terminal 10 and a charging terminal (not shown) for charging the mobile terminal 100. Since the charging terminal of the mounting portion 210 must be compatible with the charging port of the mobile terminal 100, the position of the charging terminal of the mounting portion 210 depends on the location of the charging port of the mobile terminal 100 to be used. It may vary. For example, if the charging port of the mobile terminal 100 is located on the lower surface, the charging terminal of the mounting portion 210 may be configured to be located on the lower surface of the mounting portion 210. In addition, when the charging terminal is connected to the charging port of the mobile terminal 100 using a cable, the charging terminal may be configured to be located on the side, rear or top surface of the mounting portion 210. In addition, the charging terminal of the mounting unit 210 is used for communication between the mobile terminal 100 and the mobile terminal 200.

The driving unit 220 includes one or more motors that provide power to move the mounting unit 210 in six directions. The driving unit 220 is configured to provide tilt and tilt, a 360-degree left-right rotation (pan), and a vertical/portrait (portrait/landscape) to the mounting portion 210.

The display unit 240 is configured to include LED bulbs for providing a notification function and a lighting function, and is located in front of the mounting unit 210 so that the user can see the mobile terminal 100 even when mounted. It is preferred. As an example, the display unit 240 may be configured to be positioned on the holder arm.

The sensor unit 250 includes an infrared proximity sensor (not shown) for detecting a user's motion, and a sound source recognition sensor (not shown) capable of detecting a user's voice and surrounding sounds. As an example, the sensor unit 250 may be configured to be positioned at the bottom of the mounting unit 210. In addition, the sensor unit 250 may detect the user's location based on the user's motion or voice, and may also detect the location of ambient sounds such as music.

The control unit 230 controls the operation of the mounting unit 210 according to at least one of the detected user's operation and voice. For example, the control unit 230 rotates or moves the mounting unit 210 so that the front of the mobile terminal 100 faces toward a position where a user's motion or voice is sensed. As described above, the control unit 230 can directly control the operation of the mounting unit 210 according to the user's motion or voice sensed by the sensor unit 250, but the control signal of the mobile terminal 100 is Accordingly, the operation of the mounting unit 210 is controlled. In addition, the control unit 230 may control the operation of the mounting unit 210 according to a control signal provided according to a change in state of the mobile terminal 100.

As shown in Figures 4a and 4b, the mobile terminal 100 is detachable to the cradle 200, the cradle 200 may be combined with the driving platform (270). The mobile terminal 100 may control the cradle 200 and the driving platform 270, respectively. In addition, the driving platform 270 is configured such that a cleaning module or an air cleaning module can be combined.

5A to 5C are diagrams illustrating examples of emotional expression of the mobile terminal 100 and the cradle 200. As shown in FIG. 5A, when the user calls, the controller 180 controls the cradle 200 such that the front faces the user. At this time, the display unit of the mobile terminal 100 is recognized by recognizing the user's face. 151), the movement of the cradle 200 is controlled such that the character/avatar's gaze is displayed toward the user's face. In addition, when the mobile terminal 100 is switched from the stand by state to the wake state, the controller 180 changes the displayed character/avatar's pupil, and the like. In addition, the controller 180 may control the cradle 200 to shake the mobile terminal 100 back and forth or left and right in response to the user's voice.

As shown in FIG. 5B, the controller 180 displays a charge amount of the battery as a background screen to indicate the battery state of the mobile terminal 100, or displays the mobile device 100 to express the swelling like water whenever the mobile terminal 100 moves. 151) can be controlled, and with this, the cradle 200 can be controlled to shake the mobile terminal 100 back and forth or from side to side. In addition, the controller 180 controls the operation of the mobile terminal 100 and the cradle 200 to express changes in the state of the mobile terminal 100, for example, message/email reception, communication status, charging status, and the like. . In addition, the controller 180 may control the operation of the mobile terminal 100 and the cradle 200 to inform the indoor temperature or warn of an obstacle.

As shown in FIG. 5C, the controller 180 controls the mobile terminal 100 and the cradle 200 in response to a user's motion or voice. For example, if the user puts a finger on the lips and says'쉿', the controller 180 controls the mobile terminal 100 and the cradle 200 not to move without making a sound.

As described above, the mobile terminal 100 provides a chat application capable of chatting with a user as shown in FIG. 6 in addition to a function of expressing emotion. For example, the controller 180 receives and executes a command through another user's mobile terminal 300. In particular, the chat application (Robot talk) detects the user's lifestyle and patterns using the sensor unit 140 and the input unit 120 built in the mobile terminal 100, and periodically performs a server or mobile terminal 100. To save. In addition, the user's behavior is predicted according to the user's lifestyle and patterns.

Hereinafter, with reference to the accompanying drawings for embodiments related to the control method that can be implemented in the mobile terminal 100 and the cradle 200 for the mobile terminal of the present invention. 7 is a flow chart showing an example of a control method according to the present invention.

When the mobile terminal 100 is mounted on the cradle 200, the application in the mobile terminal 100 is executed and the mobile terminal 100 is controlled by the application. The operations executed and controlled by the application are implemented by the controller 180. First, the controller 180 controls the mobile terminal 100 to display characters of various expressions on the screen of the display unit 151.

Subsequently, the controller 180 controls the camera 121 and the microphone 122 to detect at least one of the user's motion and voice (S71), identifies the user using the user's motion or voice, and locates the user. Detects. At this time, the controller 180 compares the image captured through the camera 121 or the voice detected through the microphone 122 with the user information stored in the memory 170 to identify the user, and compares this Users are identified based on the results.

In addition, the controller 180 generates information on the user's behavior pattern based on the user's continuously sensed motion and voice and stores it in the memory 170 (S72). Here, the user's behavior pattern information includes information on the household appliance use history, the user's schedule, and bed/wake time. In addition, the controller 180 stores information about the user's surrounding environment, such as weather, traffic conditions, indoor/outdoor temperatures, in the memory 170. 8 shows an example of behavior pattern information and surrounding environment information stored in the memory 170.

Then, the controller 180 predicts the user's behavior using the environment information along with the behavior pattern information (S73). For example, the controller 180 predicts a user's home appliance use schedule based on information on the user's home appliance use history, wherein the probability of using each home appliance by day and time based on the frequency of use is determined. Calculate and predict that the user will use the highest probability of home appliances. In the case of a TV, a channel/program to be watched by the user may be predicted based on information on the channel/program that the user has regularly watched.

In addition, the controller 180 controls the mobile terminal 100 and the cradle 200 to provide a service corresponding to the predicted user behavior (S74). For example, the controller 180 controls the output unit 150 to provide a signal and information for guiding the user to use the home appliance. In the case of a TV, details such as channels/programs can also be suggested to the user.

In addition, the controller 180 controls the operation of the mobile terminal 100 and the cradle 200 according to changes in the communication status, charging status, message/email reception status, etc. of the mobile terminal 100. For example, when a phone call is received, the control unit 180 displays the picture or a designated character on the screen of the mobile terminal 100 and shows the screen of the mobile terminal 100 to the user. The mounting portion 210 is controlled so that the front surface of the user faces the user. Due to this, the user can confirm the call reception with eyes, not sound or vibration.

9A to 9D show examples of services of the mobile terminal 100 provided in response to a user's operation and voice. To this end, the controller 180 controls the operation of the mobile terminal 100 and the cradle 200 according to at least one of the detected user's motion and voice. 9A, when the user leaves the office, the controller 180 recognizes the user and controls the mounting unit 210 such that the front of the mobile terminal 100 faces toward the user. And, it controls the output unit 150 to output a voice message such as "You are here today! Bored alone". 9B, the controller 180 controls the cradle 200 to follow the user and proposes a TV program. 9C, the controller 180 may charge the mobile terminal 100 and the driving platform 270 through a charging deck or a wireless charger when the user falls asleep. In addition, as shown in FIG. 9D, the controller 180 can recognize the user's belongings or necessary items and inform the location thereof. In particular, if an object that is frequently lost is input, the controller 180 can always check the location of the object and notify the user when necessary.

In this way, the home robot using the mobile terminal 100 and the cradle 200 can provide information while following the user, and can guide/recommend exercise time, meal time, and medicine taking time based on the behavior pattern information. have. In addition, it can be configured to detect the user's health through the camera 121 or the sensing unit 140, and to call an emergency center or medical staff in an emergency.

In addition, if the user does not want to answer the phone or is in trouble, when the user makes a hand gesture moving in the left and right directions, the controller 180 controls the mounting unit 210 so that the screen of the mobile terminal 100 returns. At this time, the control unit 180 controls the mounting unit 210 to interpret a user's hand gesture based on information stored in the memory 170 and perform a corresponding command.

In addition, when a text or email is received, a chat, a social network service, or the like is activated, the controller 180 displays a specified character expression on the screen of the mobile terminal 100 or through the movement of the mobile terminal 100 to the user. Inform and control the mounting unit 210 so that the display unit 151 faces the user's face. At this time, the user can selectively notify only the message transmitted through the designated phone number or email address, and it is possible to set an icon or sticker according to the type of the transmitted message. In addition, the controller 180 recognizes a specific phrase or word in the message content and displays the corresponding character on the screen of the mobile terminal 100 or controls the movement of the mounting unit 210 according to the specified movement.

The home robot of the present invention, that is, the mobile terminal 100 includes an emotional engine, which is composed of desire, emotion, and growth/personality formation. Needs vary according to the external stimulus (touch, conversation, shock, sound) and surrounding environment, which are the criteria for changing the robot's immediate emotion and long-term personality. For example, if the robot is left alone for a certain period of time, the “affection” level is lowered to become an emotional state of “sadness”, and when the user strokes the robot, the “affection” value is increased to become an emotional state of “joy”. In addition to affection, the robot's desires include search, exercise, relaxation, and self-protection. The emotions of the robot change according to the satisfaction of the needs. For example, if a user handles a robot roughly, the robot " It is an emotional state of "smell." In addition, the growth/personality of the robot depends on changes in the external stimulus and the surrounding environment. That is, the robot of the present invention has a different satisfaction from the needs depending on the external stimulus and the surrounding environment. According to the satisfaction level, an immediate emotional change is seen, and the behavioral pattern changes due to the emotional change, and the personality changes and grows due to the change in the behavioral pattern. You can see that there is.

In addition, the home robot of the present invention, that is, the mobile terminal 100 includes a behavioral style engine, which consists of expression, training, and adaptation. Expression means the expression and movement corresponding to emotion and situation. 5A-5C show examples of these expressions. Training refers to a process in which a robot maps a user's voice command to a specific action. This ability can be increased according to the user's command and reward. For example, if the user compensates (for example, stroking) when the robot executes a user's command, the robot's performance is improved. Adaptation means that the robot recognizes the user's behavior pattern and performs an appropriate action. For example, if the user greets each morning (for example, "good morning"), the robot recognizes the time as a wake-up time and first outputs a morning greeting voice message to the user. As such, it can be seen that the robot of the present invention can be changed through user training.

It will be apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit and essential features of the present invention.

The above-described present invention can be embodied as computer readable codes on a medium on which a program is recorded. The computer-readable medium includes all kinds of recording devices in which data readable by a computer system is stored. Examples of computer-readable media include ROM, RAM, CD-ROM, magnetic tapes, floppy disks, optical data storage devices, etc., which are also implemented in the form of carrier waves (eg, transmission over the Internet). Also includes. In addition, the computer may include a control unit 180 of the terminal.

Accordingly, the above detailed description should not be construed as limiting in all respects, but should be considered illustrative. The scope of the invention should be determined by rational interpretation of the appended claims, and all changes within the equivalent scope of the invention are included in the scope of the invention.

100: mobile terminal 110: wireless communication unit
120: input unit 140: sensing unit
150: output unit 160: interface unit
170: memory 180: control unit
190: power supply 200: mobile terminal cradle
210: mounting unit 220: driving unit
230: control unit 240: display unit
250: sensor unit 270: driving platform

Claims (10)

An output unit for outputting signals and information;
An input unit for sensing a user's motion and voice;
And a control unit generating the behavior pattern information of the user based on the detected user's motion and voice, and predicting the user's behavior according to the behavior pattern information,
The behavior pattern information,
It includes information about the user's history of using external home appliances,
The control unit,
The output to predict the user's schedule of using the external home appliance based on information on the user's history of using the external home appliance, and output at least one of signals and information to guide the user to use the external home appliance. Mobile terminal characterized in that to control the wealth. delete delete According to claim 1,
The control unit,
The mobile terminal, characterized in that for controlling the mobile terminal cradle so that the front of the mobile terminal toward the position where the user's motion or voice is detected. According to claim 1,
The control unit,
A mobile terminal, characterized in that for controlling the operation of the mobile terminal cradle according to the change in the state of the mobile terminal. The method of claim 5,
The control unit,
And controlling the operation of the cradle for the mobile terminal according to at least one of a communication state, a charging state, and a message receiving state. According to claim 1,
The control unit,
A mobile terminal characterized by predicting the user's behavior using the environment information along with the behavior pattern information, and controlling the output unit to output at least one of signals and information for guiding the predicted behavior. When the mobile terminal is mounted on the cradle for the mobile terminal, the step of executing the mobile terminal control application;
Detecting a user's motion and voice;
Generating behavior pattern information of the user based on the detected user's motion and voice;
Collecting the environment information of the user;
Predicting the user's behavior according to the behavior pattern information and surrounding environment information; And
Providing a service corresponding to the predicted user behavior,
The surrounding environment information,
A driving method of a mobile terminal, characterized in that it includes at least one of weather, traffic conditions, and indoor temperature of the user's surrounding environment. The method of claim 8,
Predicting a schedule of the user's home appliance use based on information on the home appliance use history included in the behavior pattern information;
And outputting at least one of signals and information for guiding the user to use the home appliance. delete

Images