KR20180083185A - Apparatus and method for providing adaptive user interface - Google Patents

Abstract

Various embodiments of the present invention relate to an electronic device and a method for providing a user interface (UI). The electronic device providing a UI includes: a display displaying a UI; and a processor combined with the display. The processor is configured to display a UI including an object having a first size for executing at least one function of an application executed in the electronic device, to detect the movement of the electronic device based on data obtained through a sensor of the electronic device, and to display a UI including the object having a second size changed from the first size over the UI based on information for indicating the detected movement and an input frequency of user input sensed from the object.

Description

[0001] APPARATUS AND METHOD FOR PROVIDING ADAPTIVE USER INTERFACE [0002]

TECHNICAL FIELD This disclosure relates generally to electronic devices, and more particularly to an apparatus and method for adaptively providing a user interface (UI) according to a user's condition.

Electronic devices such as laptop computers, tablet PCs, ultra-mobile PCs (UMPCs), and smartphones provide users with greater mobility and convenience. The electronic device is required to provide a more advanced user interface (UI) depending on the user's situation. For example, when the user is moving, when the sensitivity of the touch input is decreased due to the winter weather, when the user wears gloves, when the user's sight is bad, or when the user has difficulty in correct touch input , The user may not be able to correctly select a button that the user intends to input among the displayed input buttons. Therefore, the electronic device is required to adaptively provide the UI according to the state of the user.

Based on the above discussion, the present disclosure provides an apparatus and method for adaptively providing a user interface (UI) according to a user's condition.

According to various embodiments of the present disclosure, an operating method of an electronic device that provides a user interface (UI) includes providing an object having a first size for executing at least one function of an application executing in the electronic device object, the method comprising the steps of: displaying a UI; detecting movement of the electronic device based on data obtained through a sensor of the electronic device; And displaying the UI including the object converted from the first size to the second size in the UI based on the information for indicating the input frequency of the UI.

According to various embodiments of the present disclosure, an electronic device that provides a UI includes a display that displays a UI and a processor coupled with the display, wherein the processor is operable to perform at least one function Comprising: displaying a UI including an object having a first magnitude for performing a motion of the electronic device; detecting movement of the electronic device based on data acquired via a sensor of the electronic device; And display the UI including the object whose first size is converted into the second size in the UI, based on information for indicating the input frequency of the user input sensed by the object.

Electronic devices and methods in accordance with various embodiments of the present invention may improve the accuracy of a user's touch input by displaying objects that are converted based on pre-stored information and motion of the sensed electronic device.

The effects obtainable in the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below will be.

Figure 1 illustrates an operational flow diagram of an electronic device for providing a user interface (UI) in accordance with various embodiments.
Figure 2 illustrates the operation of displaying a UI including objects that have been resized in accordance with various embodiments.
Figures 3A-3C illustrate operation of an electronic device that stores heat map information for user input in accordance with various embodiments.
Figures 4A-4C illustrate operation of an electronic device for adaptively providing a UI according to various embodiments.
Figures 5A and 5B illustrate the operation of an electronic device for providing a UI based on the input frequency level of each object in accordance with various embodiments.
6A and 6B illustrate operation of an electronic device for providing a UI based on a user's facial image obtained according to various embodiments.
Figures 7A and 7B illustrate the operation of an electronic device for providing a UI based on information about the motion of the sensed electronic device in accordance with various embodiments.
FIG. 8 illustrates an operation of an electronic device for providing a UI in response to detection of entry into a vehicle according to an embodiment.
FIG. 9 illustrates an operation of an electronic device for providing a UI in response to detection of entry into a vehicle according to another embodiment.
10 illustrates an operation for displaying a UI including an enlarged object according to various embodiments.
11 illustrates a network environment including an electronic device according to various embodiments.
12 shows a block diagram of an electronic device according to various embodiments.
13 shows a block diagram of a program module according to various embodiments.

Hereinafter, various embodiments of the present document will be described with reference to the accompanying drawings. It is to be understood that the embodiments and terminologies used herein are not intended to limit the invention to the particular embodiments described, but to include various modifications, equivalents, and / or alternatives of the embodiments. In connection with the description of the drawings, like reference numerals may be used for similar components. The singular expressions may include plural expressions unless the context clearly dictates otherwise.

In this document, the expressions "A or B" or "at least one of A and / or B" and the like may include all possible combinations of the items listed together. Expressions such as " first, "" second," " first, "or" second, " But is not limited to those components. When it is mentioned that some (e.g., first) component is "(functionally or communicatively) connected" or "connected" to another (second) component, May be connected directly to the component, or may be connected through another component (e.g., a third component).

In this document, the term " configured to (or configured) to "as used herein is intended to encompass all types of hardware, software, , "" Made to "," can do ", or" designed to ". In some situations, the expression "a device configured to" may mean that the device can "do " with other devices or components. For example, a processor configured (or configured) to perform the phrases "A, B, and C" may be implemented by executing one or more software programs stored in a memory device or a dedicated processor (e.g., an embedded processor) , And a general purpose processor (e.g., a CPU or an application processor) capable of performing the corresponding operations.

Electronic devices in accordance with various embodiments of the present document may be used in various applications such as, for example, smart phones, tablet PCs, mobile phones, videophones, electronic book readers, desktop PCs, laptop PCs, netbook computers, workstations, a portable multimedia player, an MP3 player, a medical device, a camera, or a wearable device. Wearable devices may be of the type of accessories (eg, watches, rings, bracelets, braces, necklaces, glasses, contact lenses or head-mounted-devices (HMD) (E.g., a skin-pads or tattoo), or a bio-implantable circuit. In some embodiments, the electronic device may be, for example, a television, a digital video disk (Such as Samsung HomeSync ^™ , Apple TV ^™ , or Google TV ^™ ), a refrigerator, air conditioner, vacuum cleaner, oven, microwave oven, washing machine, air purifier, set top box, home automation control panel, A game console (e.g., Xbox ^TM , PlayStation ^TM ), an electronic dictionary, an electronic key, a camcorder, or an electronic frame.

In an alternative embodiment, the electronic device may be any of a variety of medical devices (e.g., various portable medical measurement devices such as a blood glucose meter, a heart rate meter, a blood pressure meter, or a body temperature meter), magnetic resonance angiography (MRA) A navigation system, a global navigation satellite system (GNSS), an event data recorder (EDR), a flight data recorder (FDR), an automobile infotainment device, a marine electronic equipment (For example, marine navigation systems, gyro compasses, etc.), avionics, security devices, head units for vehicles, industrial or domestic robots, drones, ATMs at financial institutions, of at least one of the following types of devices: a light bulb, a fire detector, a fire alarm, a thermostat, a streetlight, a toaster, a fitness device, a hot water tank, a heater, a boiler, . According to some embodiments, the electronic device may be a piece of furniture, a building / structure or part of an automobile, an electronic board, an electronic signature receiving device, a projector, or various measuring devices (e.g., Gas, or radio wave measuring instruments, etc.). In various embodiments, the electronic device is flexible or may be a combination of two or more of the various devices described above. The electronic device according to the embodiment of the present document is not limited to the above-described devices. In this document, the term user may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

In the present disclosure, an object is represented by a display and includes an image, text, etc. that can sense user input. For example, the object may be an icon displayed on a background screen, a play button displayed by a music application, a pause button, a stop button, and / or a keyboard input button.

A user using an electronic device may not normally be able to select an object displayed on the display of the electronic device according to a particular user state. For example, if the user is moving while using the electronic device, the user may not be able to accurately select the displayed object on the display. For example, if the user wears gloves, if the user's eyesight is bad, if the cold weather reduces the sensitivity of the input sensor included in the display of the electronic device, or because of the particular disease the user is experiencing, May not accurately select the displayed object.

 In the following description, an electronic device and an operation method for providing a more advanced user interface (UI) according to the user state are described.

Figure 1 illustrates an operational flow diagram of an electronic device for providing a UI in accordance with various embodiments. For convenience of illustration, the operations shown in Figure 1 are shown as being implemented by an electronic device, but each of these operations may be implemented by a component included in the electronic device. For example, each of the above operations may be executed by at least one processor included in the electronic device. Further, each of the operations may be an application programming interface (API) or a set of instructions stored in a memory of the electronic device. Further, each of the operations shown in FIG. 1 may be omitted or added to some operations according to the implementation method, and the same operations may be repeatedly performed.

Referring to FIG. 1, in step 110, an electronic device displays a UI including an object having a first size for executing at least one function of an application executed in the electronic device. In the present disclosure, the first size refers to a default size preset by an application. Depending on the implementation method, there may be a plurality of such objects. For example, when the music application is executed in the electronic device, the object may include an album image of a specific artist, a play button, a pause button, a stop button, a rewind button, a fast forward button, And so on.

In operation 120, the electronic device detects movement of the electronic device based on data obtained through the sensor of the electronic device. The method of detecting motion of the electronic device may vary depending on the implementation method. For example, the electronic device may utilize one or more of a global positioning system (GPS) or a geo-magnetic sensor, a gravity sensor, an acceleration sensor, or a gyro sensor included in the electronic device, Can be detected. In another example, when the electronic device enters a vehicle, the electronic device can detect that the electronic device has entered the vehicle by communicating with a beacon device or a vehicle system installed in the vehicle.

In step 130, the electronic device displays the UI including the object whose first size is converted to the second size, based on the information for indicating the movement and input frequency of the detected electronic device. In the present disclosure, the second size means a size different from the first size, and the second size can be determined according to various methods. In one embodiment, the second size may be a size according to a default ratio in the electronic device. In another embodiment, the second size may be determined differently according to a level that classifies the input frequency value of the object according to a certain criterion. For example, if the level corresponding to the input frequency value of the object is one level, the electronic device may enlarge the size of the object by a ratio corresponding to one level. In another example, if the input frequency value of the object is greater than the input frequency value corresponding to one level (i.e., if the input frequency value is two levels), then the electronic device is set to a ratio corresponding to two levels The size of the object may be enlarged by a ratio greater than the corresponding ratio.

In yet another embodiment, the second size may be determined based on information about erroneous inputs. For example, when it is detected by the electronic device that the user of the electronic device frequently inputs the upper region of the object without inputting the object normally, the electronic device enlarges the size of the object vertically .

The information for indicating the input frequency may vary according to the implementation method. In one embodiment, the information for indicating the frequency of the input may be heat map information indicating an input frequency of a user input sensed within a region of the object. The heat map is information for displaying the occurrence frequency of an event (e.g., detected user input) generated in a specific area in different colors. The electronic device can recognize the object frequently input by the user of the electronic device by using the generated heat map information according to the frequency value of the user input sensed for a predetermined time.

In another embodiment, the information for indicating the frequency of the input may be a frequency value of the user input sensed in an area other than the object. For example, when the music application is executed in the electronic device, the user can not normally enter the play button displayed by the electronic device in a walking state, and can input an area other than the play button. The electronic device may store the heat map information generated according to the frequency value of the user input sensed outside the area of the object for a certain period of time (in this disclosure, the user input sensed outside the area of the object may be referred to as an erroneous input) Can be used to grasp the pattern that the user of the electronic device erroneously inputs.

The electronic device may update previously stored information in addition to information indicating the frequency of input to the object. In one embodiment, the pre-stored information may be magnification information preset in the electronic device. In other words, when the movement of the electronic device is detected, the electronic device can display the UI enlarged (or reduced) in the first size to the second size according to a preset ratio to the electronic device. In another embodiment, the pre-stored information may include one or more of age, gender, sight, and disease history of the user of the electronic device. For example, the user's visual acuity may be below a certain reference value (e.g., an acuity of 0.1), so that the user may not be able to identify the object displayed on the electronic device, If the displayed object can not be normally input, the electronic device may display a UI including the object whose first size is converted into the second size. The pre-stored information may be updated for a certain period of time (e.g., a day, a week, or a month).

Figure 2 illustrates the operation of displaying a UI including objects that have been resized in accordance with various embodiments. 2 assumes a case where a music application is executed in an electronic device, and the electronic device is also the same when another application (for example, a media playback application, a game application, a schedule application, a character or currency application, etc.) UI can be provided according to the principle.

Referring to FIG. 2, the electronic device displays a UI 200a indicating a state in which music is being played back. The UI 200a includes at least one object. For example, the UI 200a includes an image 210 related to the currently playing music, a rewind button 212a, a pause button 212b, a stop button 212c, a fast forward button 212d, a random button 214a, and a repeated playback button 214b. Although not shown in FIG. 2, other objects may be included in the UI 200a according to an embodiment. For example, a play button, a music list button, and / or a menu button may be further included.

The user of the electronic device can normally select an object included in the UI 200a in a general situation. However, if the electronic device detects that the user can not normally select the object (e.g., when the user is moving), the electronic device displays the enlarged UI 200b of some objects . In addition, the remaining objects, which are judged by the user to not frequently input, may be displayed at the same size or displayed at a size smaller than the normal state. For example, the rewind button 212a, the pause button 212b, the stop button 212c, and the fast forward button 212d can be displayed in an enlarged size. Further, if it is determined that the user does not frequently select the image 210, the electronic device can display the reduced-size image 210. Each of the objects included in the UI 200b can be rearranged according to the changed size. For example, the rewind button 212a and the fast forward button 212d may be located in the center of the UI 200b, and the pause button 212b and the stop button 212c may be located at the bottom of the UI 200b. Also, the image 210, the random button 214a, and the repeat play button 214b may be arranged at regular intervals.

Figures 3A-3C illustrate operation of an electronic device that stores heat map information for user input in accordance with various embodiments. Each of the operations shown in FIG. 3A may be omitted or added in accordance with an implementation method, and the same operation may be repeatedly performed.

Referring to FIG. 3A, after the electronic device displays the UI including the object having the first size in step 110, the electronic device determines in step 310 whether or not the user input is detected within the area of the displayed object . The area of the object means an input range in which the user can recognize that the user has selected the object. For example, referring to FIG. 3B, when a user input is detected on the rewind button 212a, the pause button 212b, the stop button 212c, or the fast forward button 212d, It can be determined that the input is detected. In this case, the electronic device performs step 320.

The electronic device performs step 330 if the user input is sensed outside the area of the displayed object. The method for determining whether the user input is detected outside the area of the object varies according to the implementation method. In one embodiment, if a user input is detected outside the area of the displayed plurality of objects (i.e., the area where the objects are not displayed), the electronic device determines that the user input is not detected within the area of the object . For example, referring to FIG. 3C, the user of the electronic device can not normally input the rewind button 212a according to a specific user situation (for example, when the user is moving, when the visual acuity is poor) You can type it repeatedly. In this case, the electronic device may determine that the user input sensed at the upper portion of the rewind button 212a is an erroneous input. Also, the user of the electronic device can not normally input the pause button 212b and the stop button 212c according to a specific user situation, and can repeatedly input the area between the pause button 212b and the stop button 212c. In this case, the electronic device can judge the user input sensed between the pause button 212b and the stop button 212c as an erroneous input.

In another embodiment, the electronic device may determine that the user has selected an object that is not intended. For example, when an electronic device receives a user input for one of a plurality of displayed objects and receives an input for canceling the user input within a specified time after the user input is received, The device may determine that the user has entered an object that is not intended (i.e., a false input is detected).

In step 320, the electronic device stores heat map information about the object generated based on the input frequency value of the user input sensed during a predetermined period. The heat map is information indicating the occurrence frequency of an event (for example, detected user input) generated within a specific area in different colors. The heat map information may include input frequency values (e.g., 10 times, 50 times, or 100 times) of user inputs accumulated over a period of time (e.g., 1 year, 6 months, 1 month or a week) Lt; / RTI > information). The heat map may be displayed in various colors according to input frequency values. As the input frequency is higher, the heat map can be displayed in a variety of colors. For example, referring to FIG. 3B, when a user's input is detected at each of the rewind button 212a, the pause button 212b, the stop button 212c, and the fast forward button 212d, Create a map. (For example, 130 times a week) detected by the rewind button 212a and the fast forward button 212d is the frequency value of the user input sensed by the pause button 212b and the stop button 212c Gt; 75 times < / RTI > per week), the electronic device may generate a heat map that exhibits more colors in the rewind button 212a and the fast forward button 212d. For example, the heat map for each of the pause button 212b and the stop button 212cd may be displayed using one or more of green and yellow, and the heat map for each of the rewind button 212a and the fast forward button 212d may be green, yellow, Red. ≪ / RTI > The electronic device may store information on the generated heat map in a storage unit of the electronic device. Also, the electronic device may update the heat map information based on the newly sensed user input for a predetermined period of time.

In step 330, the electronic device stores the generated heat map information based on the input frequency value of the erroneous input. The electronic device may generate heat map information for the erroneous input using a principle similar to the operation implemented in operation 340. [ For example, referring to FIG. 3C, the electronic device may generate heat map information for the repeatedly sensed user input on top of the rewind button 212a (or fast-forward button 212d). In addition, the electronic device may generate heat map information for the user input repeatedly sensed between the pause button 212b and the stop button 212c.

Figures 4A-4C illustrate operation of an electronic device for adaptively providing a UI according to various embodiments. The operations shown in FIG. 4A are more specifically implemented in the operations of step 130 shown in FIG. Each of the operations shown in FIG. 4A may be omitted or added in accordance with an implementation method, and the same operation may be repeatedly performed.

Referring to FIG. 4A, after the movement of the electronic device is detected in step 120, the electronic device determines whether the input frequency value of the detected user input is loaded within the area of the object or the input frequency value of the incorrect input Or not. In one embodiment, the electronic device may load the input frequency value of the detected user input within the area of the object automatically, when motion of the electronic device is detected, according to a predetermined setting. In this case, the electronic device can change the setting according to the update history of the information indicating the input frequency of the sensed user input. For example, although the electronic device is set to load the input frequency value of the user input sensed within the area of the object, the frequency of the erroneous input sensed during a certain period of time is less than the frequency of the user input sensed within the area of the object If the certain percentage is higher, the electronic device may change the setting by loading the input frequency value of the erroneous input. In another embodiment, a user of the electronic device may specify information for loading. For example, a user of the electronic device may be configured to load an input frequency value of a user input sensed within an area of an object when movement of the electronic device is detected. If it is determined that the input frequency value of the user input sensed within the area of the object is to be loaded, the electronic device loads the heat map information for the sensed user input within the area of the object in step 420. If it is determined to load the input frequency value of the erroneous input, the electronic device loads the heat map information for the erroneous input in step 450. [

In step 430, the electronic device determines whether the input frequency value of the object is greater than or equal to a predetermined reference frequency value. The reference frequency value may be variously set according to the implementation method. In one embodiment, the reference frequency value may be a default value for the electronic device. In another embodiment, the reference frequency value may be a value specified by a user of the electronic device. For example, a user of the electronic device may set a reference frequency value that one hundred user input is detected in one object during one week. In this case, if 120 user input is detected for a week in a specific object, the electronic device can determine that the input frequency value for the object is equal to or greater than a predetermined reference frequency value. If the input frequency value of the object is less than a predetermined reference frequency value, the electronic device terminates the algorithm for the object. In other words, the electronic device can display an object of the same size or smaller size as the object already displayed. If the input frequency value of the object is equal to or greater than a predetermined reference frequency value, the electronic device performs step 440.

In step 440, the electronic device displays a UI including the object having the first size converted into the second size according to a predetermined ratio. The above ratios can be variously determined according to the implementation method. For example, the ratio may be a default value for the electronic device. In another example, the ratio may be a value specified by a user of the electronic device. The ratio may be set as a percentage (%) or a multiple (e.g., 1.5 times, 2 times, 3 times).

For example, referring to FIG. 4B, before the movement of the electronic device is detected, the electronic device includes an image, a rewind button 210, a random button 214a, a repeat play button 214b, a rewind button 212a, a pause button 212b, 212c, and fast forward button 212d. When the motion of the electronic device is detected, the input frequency value of the user input sensed on the rewind button 212a, the pause button 212b, the stop button 212c, and the fast forward button 212d is equal to or greater than a predetermined reference frequency value, The UI 400b including the enlarged rewind button 212a, the pause button 212b, the stop button 212c, and the fast forward button 212d can be displayed according to the predetermined ratio. The image 210 can be displayed at a smaller size than the previous one in the UI 400b since the user input sensed during a certain period is less than the reference frequency value. The random button 214a and the repeat play button 214b may be displayed on the UI 400b in the same size as before.

In step 460, the electronic device determines whether the input frequency value of the erroneous input is greater than or equal to a predetermined reference frequency value. The reference frequency value for the erroneous input may be variously set according to the implementation method. For example, the reference frequency value for the erroneous input may be the same as the reference frequency value described in step 430, or may be an independently determined value. If the frequency value of the erroneous input to the object is less than the predetermined reference frequency value, the electronic device terminates the algorithm for the object. In other words, the electronic device can display an object of the same size or smaller size as the object already displayed. If the frequency value of the erroneous input to the object is equal to or greater than a predetermined reference frequency value, the electronic device performs step 470. [

In step 470, the electronic device displays a UI including a first-size-converted object based on the erroneous input. For example, referring to FIG. 4C, before the motion of the electronic device is detected, the electronic device includes an image, a rewind button 210, a random button 214a, a repeat play button 214b, a rewind button 212a, a pause button 212b, 212c, and fast forward button 212d. When the movement of the electronic device is detected, the electronic device can display the UI 400c whose shape of each object has been changed. In the case of the rewind button 212a and the fast forward button 212d, since the user input is mainly detected at the upper portion of the area of each object, the electronic device has a rewind button 212a that is enlarged in the vertical direction to correct a user's erroneous input, Button 212d can be displayed. In the case of the pause button 212b and the stop button 212c, since the user input is mainly detected on the left or right side of the area of each object, the electronic device has a pause button 212b enlarged in the horizontal direction And the stop button 212c.

Figures 5A and 5B illustrate the operation of an electronic device for providing a UI based on the input frequency level of each object in accordance with various embodiments. Each of the operations shown in FIG. 5A may be omitted or added, depending on the implementation method, and the same operation may be repeatedly performed.

Referring to FIG. 5A, after the electronic device loads the heat map information for the sensed user input within the area of the object in step 420, the electronic device determines a level corresponding to the input frequency value of the object in step 510 . The level can be variously set according to the implementation method. In one embodiment, the level may be a default value for the electronic device. In another embodiment, the level may be a value specified by a user of the electronic device. For example, if the frequency of the user input detected in one object is less than 50 for a certain period of time (for example, one year, six months, one month, or one week) Can be set to be one level. In another example, the user of the electronic device can set the level of the object to be two levels when the frequency value of the user input sensed in one object during the same period is 50 times or more and less than 100 times. In another example, the user of the electronic device may set the level of the corresponding object to be three levels when the frequency value of the user input sensed in one object during the same period is 100 or more.

For example, referring to FIG. 5B, the electronic device may include an image 210 in which user input is not detected (i.e., an input frequency value is less than 50), a random button 214a, and a repeat play button 214b Can be determined to be one level. In addition, the electronic device can determine the levels of the pause button 212b and the stop button 212c, which have a user input input frequency value of 75, to be two levels. In addition, the electronic device can determine the level of the rewind button 212a and the fast-forward button 212d, which are the input frequency values of the user input 130, to be three levels. The period and the number of times for classifying the level are not limited to the specific period and the number of times only for the purpose of illustration. In addition, although FIG. 5A shows only three levels, three or more levels can be set according to an implementation method.

In step 520, the electronic device determines whether the level corresponding to the object is one level. If the level of the object is one level, the electronic device displays an object of the same size as the previously displayed object or a size smaller than the previously displayed object, and ends the algorithm. If the level of the object is not one level, the electronic device performs step 530.

In step 530, the electronic device determines whether the level corresponding to the object is two levels. If the level of the object is two levels, the electronic device displays the object in which the first size is converted into the second size in step 540. If the level of the object is not two levels, the electronic device displays the object having the first size converted into the third size in step 550. [ And the third size means an enlarged size than the second size. For example, referring to FIG. 5B, the pause button 212b and the stop button 212c corresponding to the second level are displayed in an enlarged size larger than the image 210 corresponding to one level, the random button 214a, and the repeated playback button 214b. In addition, the rewind button 212a and the fast forward button 212d corresponding to the three levels are displayed in an enlarged size larger than the pause button 212b and the stop button 212c corresponding to two levels. The rate at which the size of the object is enlarged according to each level can be variously determined according to the implementation method. For example, the ratio may be a default value for the electronic device. In another example, the ratio may be a value specified by a user of the electronic device. The ratio may be set as a percentage (%) or a multiple (e.g., 1.5 times, 2 times, 3 times).

The electronic device may display the UI including the object having the first size converted into the second size using the information stored in advance in the electronic device in addition to the information indicating the frequency of the input to the object. For example, the pre-stored information may include information on the user's vision and whether the user wears glasses normally. If the user's visual acuity is below a certain level and the user does not wear glasses, the electronic device will need to provide an enlarged UI of the size of the displayed object.

6A and 6B illustrate operation of an electronic device for providing a UI based on a user's facial image obtained according to various embodiments. Each of the operations shown in FIG. 6A may be omitted or added, depending on the implementation method, and the same operation may be repeatedly performed.

Referring to FIG. 6A, when the motion of the electronic device is detected in step 120, the electronic device obtains a user's face image using the image sensor included in the electronic device in step 610. FIG. The operation of the electronic device to obtain the face image may be triggered according to various methods. In one embodiment, the electronic device is capable of obtaining the facial image in response to detecting a change in the slope of the electronic device. In another embodiment, the electronic device is capable of acquiring the facial image in response to an initiation of an application in the electronic device. For example, referring to reference numeral 600a in FIG. 6B, when the user of the electronic device 610a brings the electronic device 610a to the user's eyes to confirm the UI displayed on the electronic device 610a, the electronic device 610a The image sensor 610b included in the user can obtain the face image of the user.

In step 620, the electronic device determines whether the user of the electronic device wore glasses based on the obtained face image. If it is determined that the user wears glasses as shown at 600b in Fig. 6b, the electronic device terminates the algorithm and displays UI 600c where the size of previously displayed objects remains unchanged. If it is determined that the user has not worn glasses as shown at 600d in Figure 6b, the electronic device performs step 630. [

In step 630, the electronic device displays a UI including the object having the first size converted into the second size. For example, referring to FIG. 6B, the electronic device may display a UI 600e including a rewind button 212a, a pause button 212b, a stop button 212c, and a fast forward button 212d in a predetermined ratio. The ratio may be a predetermined value for the electronic device, and may be a value specified by a user of the electronic device. The ratio may be set as a percentage (%) or a multiple (e.g., 1.5 times, 2 times, 3 times).

Figures 7A and 7B illustrate the operation of an electronic device for providing a UI based on information about the motion of the sensed electronic device in accordance with various embodiments. Each of the operations shown in FIG. 7A may be omitted or added, depending on the implementation method, and the same operation may be repeatedly performed. In addition, the modes (for example, the general mode, the walking mode, and the running mode) of the electronic device described below are merely referred to for convenience of explanation only and are not limited to the corresponding names.

Referring to FIG. 7A, in step 720, the electronic device measures the moving speed of the electronic device using at least one sensor included in the electronic device. The moving speed of the electronic device can be measured variously according to the implementation method. In one embodiment, the electronic device measures the position of the electronic device using a GPS contained in the electronic device, and measures the moving speed of the electronic device based on the distance of the electronic device that has been moving for a period of time . In another embodiment, the electronic device can measure the tilt, rotation, and / or speed of movement of the electronic device using at least one of a gravity sensor, a gyro sensor, or an acceleration sensor.

The electronic device can measure the moving degree of the electronic device in addition to the moving speed of the electronic device. For example, the electronic device can detect the number of times the motion of the electronic device has been sensed (e.g., 5, 5, or 10 seconds) by the at least one sensor for a period of time (e.g., 10 times or 10 times) can be measured. The electronic device measures the moving speed (or degree of movement) of the electronic device to determine whether the user is moving (general mode), walking (walking mode), and running (running mode) You can decide.

In step 730, the electronic device determines whether the measured speed is greater than or equal to a first speed threshold. The first rate threshold may be set variously according to the implementation method. In one embodiment, the first rate threshold may be a default value for the electronic device. In another embodiment, the first rate threshold may be arbitrarily set by a user of the electronic device. In yet another embodiment, the first rate threshold may be a value updated in consideration of sex, age, and disease history of a previously stored user. For example, the first rate threshold may be set to a typical adult male walking speed (e.g., 6 to 8 km / h per hour, or 100 times per minute). In another example, when information about a specific disease (e.g., arthritis) of a user of the electronic device is input to the electronic device, the first rate threshold may be set to a lower value For example, 2 to 3 km / h per hour, or 50 movements per minute). If it is determined that the measured speed is less than the first speed threshold, the electronic device performs step 740. [ If it is determined that the measured speed is greater than or equal to the first speed threshold, the electronic device performs step 750.

In step 740, the electronic device determines that the electronic device is in the normal mode. For example, referring to FIG. 7B, as in reference numeral 700a, the electronic device may determine that the user is not in motion. In this case, the electronic device terminates the algorithm and displays the UI 700b where the size of previously displayed objects remains unchanged.

In step 750, the electronic device determines whether the measured velocity is greater than or equal to a second velocity threshold. The second rate threshold may be set differently on a principle similar to the first rate threshold. In other words, the second rate threshold value may be a predetermined value, a value arbitrarily designated by the user, or an updated value based on previously stored user information. For example, the second rate threshold may be set to a typical adult male running speed (e.g., 8 km / h or more per hour, or 150 times per minute). If it is determined that the measured speed is less than the second speed threshold, the electronic device performs step 760. [ If it is determined that the measured speed is equal to or greater than the second speed threshold, the electronic device performs step 780. [

In step 760, the electronic device determines that the electronic device is in the walking mode. For example, referring to FIG. 7B, as in reference numeral 700c, the electronic device can determine that the user is walking at a constant speed. In this case, the electronic device may display the UI including the object having the first size converted into the second size according to a predetermined ratio in step 770. [ For example, the electronic device may display a UI 700d that includes a rewind button 212a, a pause button 212b, a stop button 212c, and a fast forward button 212d that are wider than the size displayed in the normal mode. The above ratios can be variously determined according to the implementation method. The ratio may be a predetermined value for the electronic device, and may be a value specified by a user of the electronic device. The ratio may be set as a percentage (%) or a multiple (e.g., 1.5 times, 2 times, 3 times).

In step 780, the electronic device determines that the electronic device is in the running mode. For example, referring to FIG. 7B, like the reference numeral 700e, the electronic device can determine that the user is jumping at a constant speed. In this case, in step 790, the electronic device may display a UI including an object whose first size is converted into a third size that is larger in a certain ratio than the second size. For example, as shown in FIG. 7B, the electronic device includes a UI 700f including a rewind button 212a, a pause button 212b, a stop button 212c, and a fast forward button 212d having an enlarged size larger than the size displayed in the walking mode Can be displayed. The above ratios can be variously determined according to the implementation method. The ratio may be a predetermined value for the electronic device, and may be a value specified by a user of the electronic device. The ratio may be set to a percentage or a multiple.

FIG. 8 illustrates an operation of an electronic device for providing a UI in response to detection of entry into a vehicle according to an embodiment. Each of the operations shown in Fig. 8 may be omitted or added to some operations according to the implementation method, and the same operations may be repeatedly performed.

8, in step 840, the electronic device 810 displays a UI including an object having a first size. For example, when the music application is executed in the electronic device, the object may be an album image, a play button, a pause button, a stop button, a rewind button, a fast forward button, a repeat play button, a random button, .

In step 845, the electronic device 810 receives a beacon signal from the beacon device 820. The electronic device 810 may receive a beacon signal from the beacon device 820 via various communication methods. For example, the beacon device 820 may broadcast the fixed information contained in the packet to the electronic device 810. The fixed information may include various information. For example, the beacon signal may include an identifier of the beacon device 820.

In step 850, the electronic device 810 may determine that the electronic device 810 has entered the vehicle based on the received beacon signal. For example, the electronic device 810 may identify the vehicle system 830 of the vehicle where the electronic device 810 is located based on the identification information contained in the beacon signal. The vehicle system 830 may be an embedded device inside the vehicle or an external device connected to the vehicle by wire or wireless connection.

In step 855, the electronic device 810 displays the UI including the object having the first size converted into the second size according to a predetermined ratio based on the previously stored information. The pre-stored information includes heat map information indicating an input frequency value of a user input to the object, heat map information indicating an input frequency value of a user input sensed in addition to the area of the object, For example, the user's vision information). The ratio may be a preset value in the electronic device 810, a value designated by the user of the electronic device 810, or updated information based on information about the user. 8 illustrates an operation in which the electronic device 810 automatically displays a UI including an object whose first size is converted to the second size in response to receiving a beacon signal, A new UI for confirming whether to display the UI including the object converted to the second size can be displayed. In this case, the electronic device 810 may display a UI including an object converted to the second size in response to a user's input that the UI including the object converted to the second size will be displayed.

In step 860, the electronic device 810 establishes a communication link with the vehicle system 830 to share information about the running application in the electronic device 810 with the vehicle system 830. For example, the electronic device 810 may be a wireless device such as WiFi direct, Infrared Ray, Bluetooth, Zigbee, Z-Wave, Visible Light Communication (VLC), 3G, LTE, A communication connection can be established with the vehicle system 830 through the communication standard.

In step 865, the electronic device 810 may send information about the application running on the electronic device 810 to the vehicle system 830 via the connected wireless communication connection. The information on the application may include information on the UI of the application displayed on the electronic device 810, and information on enlargement of the sizes of the objects included in the UI.

In step 870, the vehicle system 830 displays the UI displayed on the electronic device 810 based on the information received from the electronic device 810. For example, when the music application is running in the electronic device 810, the vehicle system 830 may include a UI including objects such as a play button, a pause button, a stop button, a rewind button, a fast forward button, Can be displayed. For convenience of the user in operation, the objects may be displayed in an enlarged size.

FIG. 9 illustrates an operation of an electronic device for providing a UI in response to detection of entry into a vehicle according to another embodiment. Each of the operations shown in FIG. 9 may be omitted or added to some operations according to the implementation method, and the same operations may be repeatedly performed.

Referring to FIG. 9, in step 930, the electronic device 910 displays a UI including an object having a first size. For example, when the music application is executed in the electronic device, the object may be an album image, a play button, a pause button, a stop button, a rewind button, a fast forward button, a repeat play button, a random button, .

In step 940, the electronic device 910 senses a communication event with the vehicle system 920. For example, the electronic device 910 may communicate with the vehicle system 920 via communication standards such as WiFi direct, IR, near field communication (NFC), Bluetooth, Zigbee, Z-Wave, VLC, 3G, LTE, Can be performed. The vehicle system 920 may be a device embedded in the vehicle or an external device connected to the vehicle by wire or wirelessly.

In step 950, based on the sensed communication event, the electronic device 910 may determine that the electronic device 910 has entered the interior of the vehicle including the vehicle system 920. In step 960, the electronic device 910 displays a UI including the object having the first size converted into the second size according to a predetermined ratio based on the previously stored information. 9 illustrates the operation of displaying the UI including the object automatically converted to the second size in response to the electronic device 910 receiving the beacon signal, but according to an implementation method, And display a new UI for confirming whether to display the UI including the converted object. In this case, the electronic device 910 may display a UI including an object converted to the second size in response to a user's input that the UI including the object converted to the second size will be displayed.

In step 970, the electronic device 910 transmits information about the application to share information about the running application in the electronic device 910 with the vehicle system 920. The information on the application may include an identifier of the application, information for expanding the first size to the second size (for example, heat map information, enlargement ratio, etc.).

In step 980, the vehicle system 920 displays the UI displayed on the electronic device 910 based on information about the application received from the electronic device 910. For example, when a music application is running in the electronic device 910, the vehicle system 920 may include a UI including objects such as a play button, a pause button, a stop button, a rewind button, a fast forward button, Can be displayed. The objects may be displayed in an enlarged size.

10 illustrates an operation for displaying a UI including an enlarged object according to various embodiments.

Referring to Fig. 10, the electronic device shows a UI 1000a for an application running in an electronic device when in a normal situation (i.e., normal mode). For example, when the application executed in the electronic device is a music application, the electronic device can display a UI including unfolded images or input buttons.

In response to the motion of the electronic device being detected, the electronic device superimposes a UI 1010 on the displayed UI 1000a to confirm whether to switch to the enlarged mode. The enlarged mode may include a mode in which the size of some objects is enlarged based on the heat map information, a mode in which the size of some objects is enlarged based on the level corresponding to each object, And the size of the objects is enlarged. The sensed movement may include one of a speed of movement of the electronic device, a face image of the user, whether the user wears glasses, and whether the electronic device has entered the vehicle. UI 1010 may include select buttons 1012, 1014. If a user input is detected on the selection button 1012 indicating "Yes (YES) ", the electronic device displays the enlarged UI 1000c of each object. If a user input is detected on the select button 1014 indicating "NO ", the electronic device retains the previously displayed UI 1000a.

As described above, an operating method of an electronic device providing a user interface includes displaying an UI including an object having a first size for executing at least one function of an application executed in the electronic device Storing information for indicating a frequency of user input sensed by the object; detecting movement of the electronic device based on data acquired through a sensor of the electronic device; And displaying the UI including the object converted from the first size to the second size in the UI, based on the information indicating the movement and the input frequency. Wherein the step of storing information for indicating the input frequency comprises the steps of: determining an input frequency value of a user input sensed within an area of the object; and calculating a heat map of the input frequency of the object based on the determined input frequency value And storing the information in the electronic device.

In one embodiment, the step of displaying the UI including the object having the first size converted into the second size includes the steps of: determining whether the input frequency value of the object is equal to or greater than a predetermined reference frequency value; And displaying the UI including the object whose first size is converted into the second size when the input frequency value of the first size is greater than or equal to the predetermined reference frequency value.

In another embodiment, the step of displaying the UI including the object whose first size is converted into the second size may include the steps of: determining a level corresponding to an input frequency value of the object; And displaying the UI including the object having the first size converted into the second size or the third size according to the level, wherein the third size is larger than the second size, and the second size is larger than the second size, And may be larger than the first size.

In one embodiment, the step of storing information for indicating the input frequency includes the steps of: determining a user input sensed in an area other than the object as an erroneous input; and transmitting heat map information on the erroneous input to the electronic device Wherein the step of displaying the UI including the object having the first size converted into the second size comprises the steps of: determining, based on the detected motion and the frequency value of the erroneous input, And displaying the UI including the object whose shape is transformed based on the erroneous input.

The method may further comprise displaying a UI including the object whose first size is converted into the second size in the UI based on the detected motion and information previously stored in the electronic device, The stored information may include at least one of size enlargement information preset in the electronic device, age, sex, sight, and disease history of the user of the electronic device.

Acquiring a face image of the user using an image sensor included in the electronic device when the visual acuity of the user of the electronic device is less than a predetermined visual acuity reference value; Wherein the step of displaying the UI including the object having the first size converted into the second size comprises the steps of determining whether the user wears glasses And displaying the UI including the object having the first size converted into the second size when the first size is converted into the second size.

In one embodiment, the step of detecting movement of the electronic device includes the steps of measuring a speed at which the electronic device moves through at least one sensor included in the electronic device, Determining that the electronic device is in a normal mode if the measured speed is less than a threshold and determining that the electronic device is in a walking mode if the measured speed is greater than or equal to the first speed threshold and less than a predetermined second speed threshold Determining that the electronic device is in a running mode if the measured speed is greater than the second rate threshold, and determining that the electronic device is in a running mode when the measured speed is greater than the second speed threshold, Wherein the step of displaying the UI includes the steps of: when the electronic device is in the walking mode, Displaying a UI including the object whose first size is converted into a third size in the UI when the electronic device is in a running mode, 3 size may be larger than the second size, and the second size may be larger than the first size.

In another embodiment, the step of detecting the movement of the electronic device may include a step of confirming that the electronic device has entered the inside of the vehicle, and the process of confirming that the electronic device has entered the inside of the vehicle may include: Confirming that the electronic device has entered the inside by receiving a beacon signal installed in the vehicle; confirming that the electronic device has entered the inside by performing proximity communication with a vehicle system installed inside the vehicle; Or the like.

11 illustrates a network environment including an electronic device according to various embodiments.

11, the electronic device 1101 may include a bus (BUS) 1110, a processor 1120, a memory 1130, an input / output interface 1150, a display 1160, and a communication interface 1170. In some embodiments, the electronic device 1101 may omit at least one of the components or additionally include other components.

Bus 1110 may include circuitry that interconnects components 1110-1170 and communicates communication (e.g., control messages or data) between the components. Processor 1120 may include one or more of a central processing unit (CPU), an application processor (AP), or a communications processor (CP). For example, the processor 1120 may perform computations or data processing related to control and / or communication of at least one other component of the electronic device 1101.

Memory 1130 may include volatile and / or non-volatile memory. For example, the memory 1130 may store instructions or data related to at least one other component of the electronic device 1101. According to one embodiment, memory 1130 may store software and / or program 1140. The program 1140 may include, for example, a kernel 1141, a middleware 1143, an application programming interface (API) 1145, and / or an application program (or "application") 1147. At least some of the kernel 1141, middleware 1143, or API 1145 may be referred to as an operating system. The kernel 1141 may include, for example, system resources (e.g., bus 1110, processor 1120, or other hardware) used to execute an operation or function implemented in other programs (e.g., middleware 1143, API 1145, or application program 1147) Memory 1130, etc.). In addition, the kernel 1141 may provide an interface that can control or manage system resources by accessing individual components of the electronic device 1101 in the middleware 1143, API 1145, or application program 1147.

The middleware 1143, for example, can perform an intermediary role so that the API 1145 or the application program 1147 can communicate with the kernel 1141 to exchange data. In addition, the middleware 1143 may process one or more task requests received from the application program 1147 according to the priority order. For example, middleware 1143 may prioritize the use of system resources (e.g., bus 1110, processor 1120, or memory 1130, etc.) of electronic device 1101 in at least one of application programs 1147, Can be processed. The API 1145 is an interface for the application 1147 to control the functions provided by the kernel 1141 or the middleware 1143 and includes at least one interface or function (for example, file control, window control, image processing, Command). The input / output interface 1150 may communicate commands or data entered from, for example, a user or other external device to another component (s) of the electronic device 1101, or to transmit instructions or data received from another component (s) Alternatively, the data can be output to the user or another external device.

The display 1160 may include, for example, a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) Mechanical systems, MEMS) displays, or electronic paper displays. Display 1160 may display various content (or objects, e.g., text, images, video, icons, and / or symbols, etc.) to a user, for example. Display 1160 may include a touch screen and may receive touch, gesture, proximity, or hovering input, for example, using an electronic pen or a portion of the user's body. The communication interface 1170 may establish communication between the electronic device 1101 and an external device (e.g., the first external electronic device 1102, the second external electronic device 1104, or the server 1106). For example, the communication interface 1170 may be connected to the network 1162 via wireless or wired communication to communicate with an external device (e.g., the second external electronic device 1104 or the server 1106).

The wireless communication may include, for example, LTE, LTE-A (LTE Advance), code division multiple access (CDMA), wideband CDMA (WCDMA), universal mobile telecommunications system (UMTS), wireless broadband (WiBro) System for Mobile Communications), and the like. According to one embodiment, the wireless communication may include, for example, wireless fidelity (WiFi), Bluetooth, Bluetooth low power (BLE), Zigbee, near field communication, Magnetic Secure Transmission, A radio frequency (RF), or a body area network (BAN). According to one embodiment, the wireless communication may comprise a GNSS. GNSS may be, for example, Global Positioning System (GPS), Global Navigation Satellite System (Glonass), Beidou Navigation Satellite System (Beidou) or Galileo, the European global satellite-based navigation system. In the present disclosure, "GPS" can be used interchangeably with "GNSS ". The wired communication may include, for example, at least one of a universal serial bus (USB), a high definition multimedia interface (HDMI), a recommended standard 232 (RS-232), a power line communication or a plain old telephone service have. Network 1162 may include at least one of a telecommunications network, e.g., a computer network (e.g., LAN or WAN), the Internet, or a telephone network.

Each of the first external electronic device 1102 and the second external electronic device 1104 may be the same or a different kind of device as the electronic device 1101. [ According to various embodiments, all or a portion of the operations performed in the electronic device 1101 may be performed in one or more other electronic devices (e.g., electronic device 1102, electronic device 1104, or server 1106). According to one embodiment, in the event that the electronic device 1101 has to perform certain functions or services automatically or on demand, the electronic device 1101 may perform at least some functions associated therewith To another device (e.g., electronic device 1102, electronic device 1104, or server 1106). Other electronic devices (e.g., electronic device 1102, electronic device 1104, or server 1106) may execute the requested function or additional function and communicate the result to electronic device 1101. The electronic device 1101 can directly or additionally process the received result to provide the requested function or service. For this purpose, for example, cloud computing, distributed computing, or client-server computing technology may be used.

12 shows a block diagram of an electronic device according to various embodiments. The electronic device 1201 may include all or part of the electronic device 1101 shown in Fig. 11, for example.

12, the electronic device 1201 includes one or more processors (e.g., AP) 1210, a communication module 1220, a subscriber identification module 1224, a memory 1230, a sensor module 1240, an input device 1250, a display 1260, an interface 1270, an audio module 1280, A camera module 1291, a power management module 1295, a battery 1296, an indicator 1297, and a motor 1298.

The processor 1210 may, for example, drive an operating system or application program to control a plurality of hardware or software components coupled to the processor 1210, and may perform various data processing and operations. Processor 1210 may be implemented, for example, as a system on chip (SoC). According to one embodiment, the processor 1210 may further include a graphics processing unit (GPU) and / or an image signal processor. Processor 1210 may include at least some of the components shown in FIG. 12 (e.g., cellular module 1221). Processor 1210 may load and process instructions or data received from at least one of the other components (e.g., non-volatile memory) into volatile memory and store the resulting data in non-volatile memory.

The communication module 1220 may have the same or similar configuration as, for example, the communication interface 1170 shown in FIG. The communication module 1220 may include, for example, a cellular module 1221, a WiFi module 1223, a Bluetooth module 1225, a GNSS module 1227, an NFC module 1228, and an RF module 1229. The cellular module 1221 can provide voice calls, video calls, text services, or Internet services, for example, over a communication network. According to one embodiment, the cellular module 1221 may utilize a subscriber identity module (e.g., a SIM card) 1224 to perform the identification and authentication of the electronic device 1201 within the communication network. According to one embodiment, the cellular module 1221 may perform at least some of the functions that the processor 1210 may provide. According to one embodiment, the cellular module 1221 may include a communications processor (CP). According to another embodiment, at least some (e.g., two or more) of the cellular module 1221, the WiFi module 1223, the Bluetooth module 1225, the GNSS module 1227 or the NFC module 1228 may be included in one integrated chip (IC) . The RF module 1229 can transmit and receive a communication signal (e.g., an RF signal), for example. The RF module 1229 may include, for example, a transceiver, a power amplifier module (PAM), a frequency filter, a low noise amplifier (LNA), or an antenna. According to another embodiment, at least one of the cellular module 1221, the WiFi module 1223, the Bluetooth module 1225, the GNSS module 1227, or the NFC module 1228 can transmit and receive RF signals through separate RF modules. The subscriber identity module 1224 may include, for example, a card or an embedded SIM containing a subscriber identity module and may include unique identification information (e.g., ICCID) or subscriber information (e.g., IMSI mobile subscriber identity).

Memory 1230 (e.g., memory 1130) may include, for example, internal memory 1232 or external memory 1234. The built-in memory 1232 may include, for example, a volatile memory (e.g., a DRAM, an SRAM, or an SDRAM), a non-volatile memory (e.g., an OTPROM, a PROM, an EPROM, an EEPROM, Memory, a hard drive, or a solid state drive (SSD). The external memory 1234 may be a flash drive, for example, a compact flash (CF), a secure digital (SD) SD, a mini-SD, an extreme digital (xD), a multi-media card (MMC) or a memory stick, etc. The external memory 1234 can be functionally or physically connected to the electronic device 1201 through various interfaces.

The sensor module 1240 may, for example, measure a physical quantity or sense an operating state of the electronic device 1201 and convert the measured or sensed information into an electrical signal. The sensor module 1240 includes a gesture sensor 1240A, a gyro sensor 1240B, an air pressure sensor 1240C, a magnetic sensor 1240D, an acceleration sensor 1240E, a grip sensor 1240F, a proximity sensor 1240G, a color sensor 1240H green, and blue sensors), a biological sensor 1240I, an on / humidity sensor 1240J, a light intensity sensor 1240K, or an ultraviolet (UV) sensor 1240M. Additionally or alternatively, the sensor module 1240 may include, for example, an e-nose sensor, an electromyography (EMG) sensor, an electroencephalogram (EEG) sensor, an electrocardiogram An ECG sensor, an infrared (IR) sensor, an iris sensor, and / or a fingerprint sensor. The sensor module 240 may further include a control circuit for controlling at least one or more sensors belonging to the sensor module 240. In one embodiment, the electronic device 1201 may further include a processor configured to control the sensor module 1240, either as part of the processor 1210 or separately, to control the sensor module 1240 while the processor 1210 is in a sleep state .

The input device 1250 may include, for example, a touch panel 1252, a (digital) pen sensor 1254, a key 1256, or an ultrasonic input device 1258. As the touch panel 1252, for example, at least one of an electrostatic type, a pressure sensitive type, an infrared type, and an ultrasonic type can be used. Further, the touch panel 1252 may further include a control circuit. The touch panel 1252 may further include a tactile layer to provide a tactile response to the user. (Digital) pen sensor 1254 may be part of, for example, a touch panel or may include a separate recognition sheet. Key 1256 may include, for example, a physical button, an optical key, or a keypad. The ultrasonic input device 1258 can sense the ultrasonic wave generated by the input tool through the microphone (e.g., the microphone 1288) and confirm the data corresponding to the ultrasonic wave detected.

Display 1260 (e.g., display 1160) may include panel 1262, hologram device 1264, projector 1266, and / or control circuitry for controlling them. The panel 1262 can be implemented, for example, flexibly, transparently, or wearably. The panel 1262 may comprise a touch panel 1252 and one or more modules. According to one embodiment, the panel 1262 can include a pressure sensor (or force sensor) that can measure the intensity of the input to the user's touch. The pressure sensor may be implemented integrally with the touch panel 1252, or may be implemented by one or more sensors separate from the touch panel 1252. The hologram device 1264 can display a stereoscopic image in the air using interference of light. The projector 1266 can display an image by projecting light onto a screen. The screen may, for example, be located inside or outside the electronic device 1201. The interface 1270 may include, for example, an HDMI 1272, a USB 1274, an optical interface 1276, or a D-sub (D-subminiature) 1278. The interface 1270 may be included in the communication interface 1170 shown in Fig. 11, for example. Additionally or alternatively, interface 1270 may include, for example, a mobile high-definition link (MHL) interface, an SD card / MMC (multi-media card) interface, or an IrDA (infrared data association) interface.

The audio module 1280 can, for example, bidirectionally convert sound and electrical signals. At least some of the components of the audio module 1280 may be included, for example, in the input / output interface 1145 shown in FIG. The audio module 1280 can process sound information input or output through, for example, a speaker 1282, a receiver 1284, an earphone 1286, a microphone 1288, or the like. The camera module 1291 is, for example, a device capable of capturing still images and moving images, and may include, according to one embodiment, one or more image sensors (e.g., front or rear sensors), a lens, an image signal processor Flash (e.g., LED or xenon lamp, etc.). The power management module 1295 can manage the power of the electronic device 1201, for example. According to one embodiment, the power management module 1295 may include a power management integrated circuit (PMIC), a charging IC, or a battery or fuel gauge. The PMIC may have a wired and / or wireless charging scheme. The wireless charging scheme may include, for example, a magnetic resonance scheme, a magnetic induction scheme, or an electromagnetic wave scheme, and may further include an additional circuit for wireless charging, for example, a coil loop, a resonant circuit, have. The battery gauge can measure, for example, the remaining amount of the battery 1296, the voltage during charging, the current, or the temperature. The battery 1296 may include, for example, a rechargeable battery and / or a solar cell.

Indicator 1297 may indicate a particular state of the electronic device 1201 or a portion thereof (e.g., processor 1210), such as a boot state, a message state, or a state of charge. The motor 1298 can convert electrical signals into mechanical vibrations, and can generate vibrations, haptic effects, and the like. Electronic device 1201 is, for example, DMB (digital multimedia broadcasting), DVB (digital video broadcasting), or MediaFLO (mediaFlo ^TM) Mobile TV-enabled devices capable of handling media data in accordance with standards such as (for example: GPU ). Each of the components described in this document may be composed of one or more components, and the name of the component may be changed according to the type of the electronic device. In various embodiments, the electronic device (e. G., Electronic device 201) may be configured such that some components are omitted, further components are included, or some of the components are combined into one entity, The functions of the corresponding components can be performed in the same manner.

13 shows a block diagram of a program module according to various embodiments. According to one embodiment, the program module 1310 (e.g., program 1140) includes an operating system that controls resources associated with an electronic device (e.g., electronic device 1101) and / . ≪ / RTI > The operating system may include, for example, Android ^TM , iOS ^TM , Windows ^TM , Symbian ^TM , Tizen ^TM , or Bada ^TM .

13, the program module 1310 includes a kernel 1320 (e.g., a kernel 1141), a middleware 1330 (e.g., middleware 1143), an API 1360 (e.g., API 1145), and / or an application 1370 . At least a portion of the program module 1310 may be preloaded on the electronic device or downloaded from an external electronic device (e.g., electronic device 1102, electronic device 1104, server 1106, etc.).

The kernel 1320 may include, for example, a system resource manager 1321 and / or a device driver 1323. The system resource manager 1321 can perform control, allocation, or recovery of system resources. According to one embodiment, the system resource manager 1321 may include a process manager, a memory manager, or a file system manager. The device driver 1323 may include, for example, a display driver, a camera driver, a Bluetooth driver, a shared memory driver, a USB driver, a keypad driver, a WiFi driver, an audio driver, or an inter-process communication (IPC) driver. The middleware 1330 may provide various functions to the application 1370 via the API 1360, for example, to provide functions that are commonly needed by the application 1370 or allow the application 1370 to use limited system resources within the electronic device. According to one embodiment, the middleware 1330 includes a runtime library 1335, an application manager 1341, a window manager 1342, a multimedia manager 1343, a resource manager 1344, a power manager 1345, a database manager 1346, a package manager 1347, a connectivity manager 1348, A location manager 1350, a graphic manager 1351, or a security manager 1352. [

The runtime library 1335 may include, for example, a library module used by the compiler to add new functionality via a programming language while the application 1370 is running. The runtime library 1335 can perform input / output management, memory management, or arithmetic function processing. The application manager 1341 can manage the life cycle of the application 1370, for example. The window manager 1342 can manage GUI resources used on the screen. The multimedia manager 1343 can recognize a format required for reproducing media files and can perform encoding or decoding of a media file using a codec suitable for the format. The resource manager 1344 can manage the source code of the application 1370 or the space of the memory. The power manager 1345 may, for example, manage the capacity or power of the battery and provide the power information necessary for operation of the electronic device. According to one embodiment, the power manager 345 may interoperate with a basic input / output system (BIOS). The database manager 1346 may create, retrieve, or modify the database to be used, for example, in the application 1370. The package manager 1347 can manage installation or update of an application distributed in the form of a package file.

The connectivity manager 1348 can, for example, manage the wireless connection. The notification manager 1349 can provide an event to the user, for example, an arrival message, an appointment, a proximity notification, and the like. The location manager 1350 can manage the location information of the electronic device, for example. The graphic manager 1351 may manage, for example, a graphical effect to be presented to the user or a user interface associated therewith. Security manager 1352 may provide, for example, system security or user authentication. According to one embodiment, the middleware 1330 may include a telephony manager for managing the voice or video call capability of the electronic device, or a middleware module capable of forming a combination of the functions of the above-described components. According to one embodiment, the middleware 1330 can provide a module specialized for each type of operating system. Middleware 1330 may dynamically delete some existing components or add new ones. The API 1360 may be provided in a different configuration depending on the operating system, for example, as a set of API programming functions. For example, for Android or iOS, you can provide a single API set for each platform, and for Tizen, you can provide two or more API sets for each platform.

The application 1370 may include, for example, a home 1371, a dialer 1372, an SMS / MMS 1373, an instant message 1374, a browser 1375, a camera 1376, an alarm 1377, a contact 1378, a voice dial 1379, 1382, an album 1383, a watch 1384, an application that provides healthcare (e.g., to measure momentum or blood glucose), or environmental information (e.g., pressure, humidity, or temperature information). According to one embodiment, the application 1370 may include an information exchange application that can support the exchange of information between the electronic device and the external electronic device. The information exchange application may include, for example, a notification relay application for communicating specific information to an external electronic device, or a device management application for managing an external electronic device. For example, the notification delivery application can transmit notification information generated in another application of the electronic device to the external electronic device, or receive notification information from the external electronic device and provide the notification information to the user. The device management application may, for example, control the turn-on / turn-off or brightness (or resolution) of an external electronic device in communication with the electronic device (e.g., the external electronic device itself Control), or install, delete, or update an application running on an external electronic device. According to one embodiment, the application 1370 may include an application (e.g., a healthcare application of a mobile medical device) designated according to the attributes of the external electronic device. According to one embodiment, application 1370 may include an application received from an external electronic device. At least some of the program modules 1310 may be implemented (e.g., executed) in software, firmware, hardware (e.g., processor 1210), or a combination of at least two of the same, and may include modules, An instruction set, or a process.

The term "module" as used herein includes units made up of hardware, software, or firmware and may be used interchangeably with terms such as, for example, logic, logic blocks, components, or circuits. A "module" may be an integrally constructed component or a minimum unit or part thereof that performs one or more functions. "Module" may be implemented either mechanically or electronically, for example, by application-specific integrated circuit (ASIC) chips, field-programmable gate arrays (FPGAs) And may include programmable logic devices. At least some of the devices (e.g., modules or functions thereof) or methods (e.g., operations) according to various embodiments may be implemented with instructions stored in a computer readable storage medium (e.g., memory 1130) . When the instruction is executed by a processor (e.g., processor 1120), the processor may perform a function corresponding to the instruction. The computer-readable recording medium may be a hard disk, a floppy disk, a magnetic medium such as a magnetic tape, an optical recording medium such as a CD-ROM or DVD, a magneto-optical medium such as a floppy disk, And the like. The instructions may include code generated by the compiler or code that may be executed by the interpreter. Modules or program modules according to various embodiments may include at least one or more of the elements described above, some of which may be omitted, or other elements. Operations performed by modules, program modules, or other components, in accordance with various embodiments, may be performed sequentially, in parallel, repetitively, or heuristically, or at least some operations may be performed in a different order, .

According to the above, an electronic device providing a user interface (UI) comprises a display for displaying a UI and a processor coupled with the display, the processor comprising: at least one Comprising: displaying a UI including an object having a first size for performing a function of the electronic device; detecting movement of the electronic device based on data obtained via a sensor of the electronic device; And display the UI including the object converted from the first size to the second size in the UI, based on information for indicating movement and frequency of input to the object stored in the electronic device.

In one embodiment, the processor is further configured to determine an input frequency value of the sensed user input in an area of the object and to store heat map information for the object generated based on the determined input frequency value in the electronic device Lt; / RTI >

In another embodiment, the processor checks whether the input frequency value of the object is equal to or greater than a predetermined reference frequency value, and when the input frequency value of the object is equal to or greater than the predetermined reference frequency value, And to display a UI including the object converted to a second size.

In another embodiment, the processor is further configured to determine a level corresponding to an input frequency value of the first object, and to convert the first size into the second size or the third size according to the level of the object Wherein the third size is larger than the second size and the second size is larger than the first size.

In another embodiment, the processor is further configured to determine a user input sensed in an area other than the object as an erroneous input, determine a frequency value for the erroneous input, determine a frequency value for the detected motion and the erroneous input Based on the input, the shape of the object in the UI is displayed based on the erroneous input.

In yet another embodiment, the processor is further configured to display a UI in the UI based on the detected motion and information pre-stored in the electronic device, the UI including the object whose first size has been converted to the second size And the previously stored information includes at least one of magnification information preset in the electronic device, information on the age, sex, and visual acuity of the user of the electronic device.

In another embodiment, the processor is configured to acquire the user's facial image using an image sensor included in the electronic device if the user's visual acuity of the electronic device is less than a predetermined visual acuity reference value, And displaying the UI including the object whose first size has been converted to the second size when it is determined that the user does not wear glasses based on the image, do.

In another embodiment, the processor measures the speed at which the electronic device is moved by a sensor included in the electronic device, and when the measured speed is less than a predetermined first speed threshold, And if the measured speed is greater than or equal to the first speed threshold and less than a predetermined second speed threshold, the electronic device is determined to be in a walking mode, and if the measured speed is less than the second speed Determining that the electronic device is in a running mode when the electronic device is in a walking mode and displaying a UI including the object whose first size is converted to a second size in the UI when the electronic device is in the walking mode And when the electronic device is in the running mode, displaying the UI including the object whose first size is converted into the third size in the UI, Is further configured to, wherein the third size is greater than the second size, the second size is larger than the first size.

In another embodiment, the processor is further configured to verify that the electronic device has entered the interior of the vehicle, wherein the processor receives a beacon signal installed in the interior of the vehicle to confirm that the electronic device has entered the interior of the vehicle Or performing an operation of confirming that the electronic device has entered the inside by performing a close communication with a vehicle system installed inside the vehicle.

While the invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments. Therefore, the scope of the present disclosure should not be limited to the embodiments described, but should be determined by the scope of the appended claims, as well as the appended claims.

It is to be understood that both the foregoing general description and the following detailed description of the present invention are exemplary and explanatory and are intended to provide further explanation of the invention as claimed. Accordingly, the scope of the present invention should be construed as being included in the scope of the present invention, all changes or modifications derived from the technical idea of the present invention.

Claims (20)

1. A method of operating an electronic device that provides a user interface (UI)
Displaying an UI including an object having a first size for executing at least one function of an application executed in the electronic device;
Detecting movement of the electronic device based on data obtained through a sensor of the electronic device;
And displaying the UI including the object having the first size converted into the second size in the UI based on the detected motion and the information for indicating the input frequency of the user input sensed by the object How to.
The method according to claim 1,
Determining an input frequency value of a user input sensed within an area of the object;
And storing the heat map information on the input frequency of the object on the electronic device based on the determined input frequency value.
The method of claim 2, wherein displaying the UI including the object having the first size converted into the second size comprises:
Determining whether an input frequency value of the object is equal to or greater than a predetermined reference frequency value,
And displaying the UI including the object whose first size is converted into the second size when the input frequency value of the object is equal to or greater than the predetermined reference frequency value.
The method of claim 2, wherein displaying the UI including the object having the first size converted into the second size comprises:
Determining a level corresponding to an input frequency value of the object;
And displaying the UI including the object having the first size converted into the second size or the third size according to the level of the object,
Wherein the third size is greater than the second size and the second size is greater than the first size.
The method according to claim 1,
Determining a user input sensed in an area other than the object as an erroneous input;
Storing the heat map information on the erroneous input in the electronic device,
And displaying the UI including the object having the first size converted into the second size,
And displaying a UI including the object whose shape is transformed based on the erroneous input in the UI, based on the detected motion and the frequency value for the erroneous input. .
The method according to claim 1,
Further comprising the step of displaying a UI including the object whose first size is converted into the second size in the UI based on the detected motion and information previously stored in the electronic device,
Wherein the pre-stored information includes at least one of size enlargement information preset in the electronic device, information on the age, sex, sight, and disease history of the user of the electronic device.
The method according to claim 6, further comprising the steps of: acquiring a face image of the user using an image sensor included in the electronic device when the visual acuity of the user of the electronic device is less than a predetermined visual acuity reference value;
And checking whether the wearer wears glasses based on the obtained face image,
Wherein the step of displaying the UI including the object having the first size converted into the second size comprises the steps of: if the first size is determined not to wear glasses, And displaying a UI including the object.
The method of claim 1, wherein the step of detecting movement of the electronic device comprises:
Measuring a speed at which the electronic device moves through at least one sensor included in the electronic device;
Determining that the electronic device is in a normal mode if the measured speed is less than a predetermined first speed threshold and if the measured speed is greater than or equal to the first speed threshold and less than a predetermined second speed threshold, Determining that the electronic device is in a walking mode and determining that the electronic device is in a running mode if the measured speed is greater than the second speed threshold,
And displaying the UI including the object having the first size converted into the second size in the UI,
Displaying a UI including the object whose first size is converted into a second size in the UI when the electronic device is in the walking mode;
When the electronic device is in a running mode, displaying a UI including the object whose first size is converted into a third size in the UI,
Wherein the third size is greater than the second size and the second size is greater than the first size.
The method of claim 1, wherein the step of detecting movement of the electronic device comprises:
And confirming that the electronic device has entered the inside of the vehicle.
The method of claim 9, wherein the step of verifying that the electronic device has entered the vehicle comprises:
Confirming that the electronic device has entered the interior by receiving a beacon signal installed inside the vehicle;
And confirming that the electronic device has entered the inside by performing proximity communication with a vehicle system installed inside the vehicle.
1. An electronic device for providing a user interface (UI)
A display for displaying a UI,
A processor coupled to the display,
The processor comprising:
Displaying an UI including an object having a first size for executing at least one function of an application executed in the electronic device,
Based on data obtained via a sensor of the electronic device, detecting movement of the electronic device,
Configured to display a UI including the object whose first size is converted into a second size in the UI, based on the information indicating the detected motion and the input frequency of the user input detected in the object, .
12. The system of claim 11,
Determining an input frequency value of a user input sensed within an area of the object,
And store heat map information for an input frequency of the object in the electronic device based on the determined input frequency value.
The system of claim 12,
Checking whether an input frequency value of the object is equal to or greater than a predetermined reference frequency value,
And if the input frequency value of the object is greater than or equal to the predetermined reference frequency value, display the UI including the object whose first size has been converted to the second size.
The system of claim 12,
Determining a level corresponding to an input frequency value of the first object,
And display the UI including the object whose first size has been converted to the second size or third size according to the level of the object,
Wherein the third size is larger than the second size and the second size is larger than the first size.
12. The system of claim 11,
Determines a user input sensed in an area other than the object as an erroneous input,
Determining a frequency value for the erroneous input,
And wherein the UI is further configured to display the UI including the object whose shape is transformed based on the erroneous input in the UI, based on the detected motion and the frequency value for the erroneous input.
12. The system of claim 11,
And display the UI including the object whose first size is converted to the second size in the UI based on the detected motion and information previously stored in the electronic device,
Wherein the pre-stored information includes at least one of magnification information preset in the electronic device, information on the age, sex, and visual acuity of the user of the electronic device.
17. The system of claim 16,
Acquiring a face image of the user using an image sensor included in the electronic device when the visual acuity of the user of the electronic device is less than a predetermined visual acuity reference value,
Checking whether the user wears glasses based on the obtained face image,
And wherein if the user is determined not to wear glasses, the UI is further configured to display the UI including the object whose first size has been converted to the second size.
12. The system of claim 11,
A speed of the electronic device is measured by a sensor included in the electronic device,
Determining that the electronic device is in a normal mode if the measured speed is less than a predetermined first speed threshold and if the measured speed is greater than or equal to the first speed threshold and less than a predetermined second speed threshold, Determines that the electronic device is in a walking mode, and determines that the electronic device is in a running mode if the measured speed is above the second speed threshold,
When the electronic device is in the walking mode, displaying the UI including the object whose first size is converted into the second size in the UI,
If the electronic device is in a running mode, display the UI including the object converted from the first size to a third size in the UI,
Wherein the third size is larger than the second size and the second size is larger than the first size.
12. The system of claim 11,
And to confirm that the electronic device has entered the interior of the vehicle.
20. The system of claim 19,
By confirming that the electronic device has entered the inside by receiving a beacon signal installed inside the vehicle,
And performing an operation of confirming that the electronic device has entered the inside by performing a close communication with a vehicle system installed inside the vehicle.

Images