KR20160124643A - Method and Apparatus for Providing a Controller - Google Patents

Abstract

An electronic apparatus is disclosed. The electronic apparatus comprises: at least one display; a memory for storing a database including information on an application and a plurality of controllers; and a processor. The processor displays, in a first area of the display, content including at least one object related to execution of the application, determines a controller corresponding to the application based on the database, and outputs the controller in a second area of the display. Besides, various embodiments realized through the specification are possible.

Description

METHOD AND APPARATUS FOR PROVIDING A CONTROLLER

The embodiments disclosed herein relate to a technique for providing a controller independent of an application in an electronic device.

Various embodiments of the present invention are directed to a method, apparatus and computer readable medium for providing a controller for control of an application.

Generally, a controller for controlling an application, such as a conventional gaming device, is provided as a hardware having a physical form (e.g., a keyboard, a keypad, a mouse, a joystick, and a game pad). With the success of the game market using a mobile terminal such as a smart phone or a tablet, the electronic device provides a software-type game pad operated by a touch input in addition to a physical game pad.

It is difficult or impossible for the physical controller to customize (change the button configuration / position / color, etc.) in the form desired by the user. In addition, there are cases where the position / configuration of the main buttons to be utilized differs depending on the characteristics of the application, making it difficult for the user to adapt or operate.

Further, even in the case of using a software-type controller, only a controller that is predefined and provided by an application developer (or a game maker) can use the controller.

The embodiments disclosed in this document can provide an electronic device and a method thereof that provide a controller suitable for the type of application and provide a controller that can be changed / defined according to the user's intention.

An electronic device according to an embodiment disclosed herein may include a memory for storing a database containing information about at least one display, an application, and a plurality of controllers, and a processor. Wherein the processor displays content including at least one object in association with execution of the application in a first area of the display, determines a controller corresponding to the application based on the database, Can be output to the second area of the display.

Also, a method according to an embodiment disclosed herein may include the steps of: executing an application; placing content containing at least one object in association with execution of the application in a first area of a display functionally associated with the electronic device; Determining a controller to move the at least one object based on a database, and outputting the determined controller to a second area of the display.

In addition, the storage medium according to one embodiment disclosed herein may store computer-readable instructions for performing the above-described method.

According to the embodiments disclosed in this document, it is possible to provide a controller suitable for the characteristics of the application and the user's preference in cooperation with the electronic device itself or with the server, thereby enhancing the usability of the user and the application.

In addition, various effects can be provided that are directly or indirectly understood through this document.

1 illustrates an exemplary form of an electronic device according to one embodiment.
2 shows a configuration of an electronic device and a server according to an embodiment.
3 illustrates a process for providing a controller suitable for an application according to an embodiment.
Figure 4 shows a reference relationship between databases in accordance with one embodiment.
5 illustrates a process for determining a controller suitable for an application according to an embodiment.
6A, 6B and 6C show application execution screens and controllers of different categories according to an embodiment.
7 shows a process of determining a controller based on communication with a server according to an embodiment.
8 shows a process for setting a button controller according to an embodiment.
9 shows a button controller according to an embodiment.
10 shows a process for changing a controller according to an embodiment.
FIG. 11 shows a mapping between a button and a controller constituting the button controller according to an embodiment.
12 shows a user adjustable controller according to one embodiment.
13 shows an example of controlling a cursor with a controller according to an embodiment.
14 illustrates an electronic device in a network environment in accordance with various embodiments.
15 shows a block diagram 1500 of an electronic device 1501 according to various embodiments.
16 shows a block diagram of a program module according to various embodiments.

Various embodiments of the invention will now be described with reference to the accompanying drawings. It should be understood, however, that the invention is not intended to be limited to the particular embodiments, but includes various modifications, equivalents, and / or alternatives of the embodiments of the invention. In connection with the description of the drawings, like reference numerals may be used for similar components.

In this document, the expressions "have," "may," "include," or "include" may be used to denote the presence of a feature (eg, a numerical value, a function, Quot ;, and does not exclude the presence of additional features.

In this document, the expressions "A or B," "at least one of A and / or B," or "one or more of A and / or B," etc. may include all possible combinations of the listed items . For example, "A or B," "at least one of A and B," or "at least one of A or B" includes (1) at least one A, (2) Or (3) at least one A and at least one B all together.

The expressions "first," " second, "" first, " or "second ", etc. used in this document may describe various components, It is used to distinguish the components and does not limit the components. For example, the first user equipment and the second user equipment may represent different user equipment, regardless of order or importance. For example, without departing from the scope of the rights described in this document, the first component can be named as the second component, and similarly the second component can also be named as the first component.

(Or functionally or communicatively) coupled with / to "another component (eg, a second component), or a component (eg, a second component) Quot; connected to ", it is to be understood that any such element may be directly connected to the other element or may be connected through another element (e.g., a third element). On the other hand, when it is mentioned that a component (e.g., a first component) is "directly connected" or "directly connected" to another component (e.g., a second component) It can be understood that there is no other component (e.g., a third component) between other components.

As used herein, the phrase " configured to " (or set) to be "adapted to, " To be designed to, "" adapted to, "" made to, "or" capable of ". The term " configured (or set) to "may not necessarily mean " specifically designed to" Instead, in some situations, the expression "configured to" may mean that the device can "do " with other devices or components. For example, a processor configured (or configured) to perform the phrases "A, B, and C" may be a processor dedicated to performing the operation (e.g., an embedded processor), or one or more software programs To a generic-purpose processor (e.g., a CPU or an application processor) that can perform the corresponding operations.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of the other embodiments. The singular expressions may include plural expressions unless the context clearly dictates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by one of ordinary skill in the art. The general predefined terms used in this document may be interpreted in the same or similar sense as the contextual meanings of the related art and are intended to mean either ideally or in an excessively formal sense It is not interpreted. In some cases, even the terms defined in this document can not be construed as excluding the embodiments of this document.

An electronic device in accordance with various embodiments of the present document may be, for example, a smartphone, a tablet personal computer, a mobile phone, a video phone, an e-book reader, Such as a desktop PC, a laptop PC, a netbook computer, a workstation, a server, a personal digital assistant (PDA), a portable multimedia player (PMP) A camera, or a wearable device. According to various embodiments, the wearable device may be of the type of accessory (e.g., a watch, a ring, a bracelet, a bracelet, a necklace, a pair of glasses, a contact lens or a head-mounted-device (HMD) (E. G., Electronic apparel), a body attachment type (e. G., A skin pad or tattoo), or a bioimplantable type (e.g., implantable circuit).

In some embodiments, the electronic device may be a home appliance. Home appliances include, for example, televisions, DVD players, audio, refrigerators, air conditioners, vacuum cleaners, ovens, microwaves, washing machines, air cleaners, set- (Such as a home automation control panel, a security control panel, a TV box such as Samsung HomeSync ™, Apple TV ™ or Google TV ™), a game console (eg Xbox ™, PlayStation ™) A dictionary, an electronic key, a camcorder, or an electronic frame.

In an alternative embodiment, the electronic device may be any of a variety of medical devices (e.g., various portable medical measurement devices such as a blood glucose meter, a heart rate meter, a blood pressure meter, or a body temperature meter), magnetic resonance angiography (MRA) Navigation system, global navigation satellite system (GNSS), event data recorder (EDR), flight data recorder (FDR), infotainment (infotainment) ) Automotive electronic equipment (eg marine navigation systems, gyro compass, etc.), avionics, security devices, head units for vehicles, industrial or home robots, automatic teller's machines (ATMs) Point of sale, or internet of things (eg, light bulbs, various sensors, electrical or gas meters, sprinkler devices, fire alarms, thermostats, street lights, A toaster, a fitness equipment, a hot water tank, a heater, a boiler, and the like).

According to some embodiments, the electronic device is a piece of furniture or a part of a building / structure, an electronic board, an electronic signature receiving device, a projector, Water, electricity, gas, or radio wave measuring instruments, etc.). In various embodiments, the electronic device may be a combination of one or more of the various devices described above. An electronic device according to some embodiments may be a flexible electronic device. Further, the electronic device according to the embodiment of the present document is not limited to the above-described devices, and may include a new electronic device according to technological advancement.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An electronic apparatus according to various embodiments will now be described with reference to the accompanying drawings. In this document, the term user may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

1 illustrates an exemplary form of an electronic device according to one embodiment.

Referring to FIG. 1, an electronic device 100 according to one embodiment may include a plurality of displays. For example, the electronic device 100 may include a first display 110 and a second display 120. The first display 110 and the second display 120 may be interconnected via a connection 115. The connection portion 115 may be formed of various materials. For example, plastic, metal, leather, synthetic resin, or the like. In another embodiment, the first display 110 and the second display 120 may be in contact with each other without a connection 115. [ In addition, the electronic device 100 may include a physical button 101, a receiver 103 for other functions (e.g., calls), and the like. Various components (not shown in FIG. 1, such as touch buttons, volume keys, cameras, sensors, etc. disposed outside the display area) may be included in the electronic device 100, but in order to prevent blurring of the present invention, It is omitted.

According to one embodiment, an execution view of an application may be displayed on the first display 110. A control UI (user interface) for controlling the execution screen of the application may be output to the second display 120. For example, a racing game screen may be displayed on the first display 110, and a UI for controlling the racing game may be displayed on the second display 120. The UI may be output, for example, a direction control menu (arrow keys or handles), a speed control menu (accelerator pedal, brake), game control menus (pause, save, end, etc.). In this document, such a control UI is referred to as a controller, and unless otherwise noted, the controller may refer to a software-style control UI output to the display. Although various embodiments are described centering on a controller for controlling a game application for convenience, various embodiments may be applied to an application that can use a control UI such as an application, for example, a composition or an image editing .

Although FIG. 1 shows an electronic device 100 with two displays, in a modified embodiment, even when the electronic device 100 has three or more displays, various embodiments disclosed in this document can be applied. For example, when the electronic device has a plurality of displays, a controller may be output to one of the displays, and an execution screen of the application may be displayed on at least one of the remaining displays.

In one embodiment, the embodiment of the present invention can be applied even when the electronic device has one display. For example, the application execution screen and the controller can be displayed on the same display. In this case, the execution screen of the application can be output to the first layer, and the controller can be output to the second layer having a higher z-order than the first layer. In another example, the electronic device 100 divides one display into two or more areas, and can display an execution screen of the application in the first area and a control UI in the second area.

According to one embodiment, when connected to another electronic device using wired or wireless communication, the execution screen of the application of the electronic device 100 is output to a screen (e.g., a monitor, TV, etc.) of the external device, A controller can be output. In other words, the electronic device outputs a controller to its own display, and outputs an execution screen to a display of an external device functionally connected to the electronic device, thereby providing an environment in which a function of a larger screen or a more free controller is available . For example, when connected to a wearable device such as a head-mounted-device (HMD), the execution screen of the application of the electronic device 100 is output to the wearable device, can do.

Hereinafter, various embodiments of the present invention will be described with reference to FIGS. 2 to 16. FIG. For convenience, the electronic device 100 is described as having two displays, but as described above, the present invention can be applied to various types of electronic devices.

2 shows a configuration of an electronic device and a server according to an embodiment.

Referring to FIG. 2, the electronic device 100 may include a first display 110, a second display 120, a processor 130, a memory 140, and a communication module 150. The electronic device 100 may include additional components, or may omit some components. A description of various hardware / software components that may be applied to the electronic device 100 is provided with reference to Figs. 14, 15, and 16.

The display of at least one of the first display 110 and the second display 120 may receive a user's touch input or hovering input. For example, the processor 130 may output an execution screen of the application to the first display 110 and output a controller corresponding to the application to the second display 120. In this case, at least the second display 120 can recognize the user's touch input and hovering input. For example, the second display 120 may include a cover glass, a touch panel, and a display panel. When the touch input or the hovering input to the second display 120 sensed by the touch panel is received, the processor 130 may perform an operation based on the input and output the result to the first display 110. Also, in one embodiment, the first display 110 may also recognize touch and hovering inputs.

The processor 130 may correspond to hardware having computing power such as an application processor (AP) and a central processing unit (CPU). The processor 130 may be comprised of one or more processors or one or more cores. Also, in one embodiment, the processor 130 may be implemented in the form of an integrated chip, such as a system on chip (SoC).

In one embodiment, the processor 130 may manage the provision of a controller suitable for the application via a controller manager. The controller manager may be understood as a software module implementable by the processor 130. In this specification, it is understood that, among the operations performed by the processor 130, the operations of determining and managing the controller are performed by the controller manager.

In the memory 140, an application 141, an application meta DB 143, and a controller meta DB 145 may be stored. The application 141 may be, for example, a game application. As described above, the application 141 may correspond to any application capable of providing a controller for controlling an execution screen and an execution screen (execution result). For example, in addition to the game, an application that can be utilized by a user who can provide a content service such as music or moving picture may correspond to the application 141.

The application meta DB 143 includes a list of applications available to the user, and may include additional information fields for each application. In one embodiment, the application meta DB 143 may be referred to as a first database. The controller meta DB 145 includes a list of pre-defined controllers that may be provided to the user, i. E., Output to the second display 120, and may also include appropriate additional information fields. In one embodiment, the controller meta DB 145 may be referred to as a second database. The processor 130 may refer to the application meta DB 143 and the controller meta DB 145 to determine a controller suitable for the application 141. [ The application meta DB 143 and the controller meta DB 145 will be described in detail with reference to FIG.

 The electronic device 100 can communicate with the server 200 via a network. The server 200 may include applications 210, an application meta DB 220, and a controller meta DB 230. Further, in one embodiment, the server 200 may correspond to a plurality of servers. For example, the server 200 may include a second server (e.g., an Android Market such as Google Play Store, or Apple's AppStore) that includes an application 210, an application meta DB 220, and / Server (or second and third servers). The electronic device 100 can be connected to the server 200, download the application, and install it in the electronic device 100. The application meta DB 220 and the controller meta DB 230 included in the server 200 may correspond to the application meta DB 143 and the controller meta DB 145 included in the electronic device 100, respectively. The database stored in the server 200 may include relatively more records than the database stored in the memory 140 of the electronic device 100. [ For example, when the electronic device 100 can not confirm the information corresponding to the application 141 from the application meta DB 143, the electronic device 100 can access the server 200 and check or download the corresponding information from the application meta DB 220 of the server 200. An embodiment related to this will be described with reference to Fig.

3 illustrates a process for providing a controller suitable for an application according to an embodiment.

Referring to FIG. 3, at operation 310, the electronic device 100 (e.g., processor 130) may execute an application (e.g., application 141). The application may be an application provided with the controller, or an application in which the controller must be provided during execution of the application. Alternatively, the application may be an application that does not provide a controller. For example, when executing the game application, the electronic device 100 may output an execution screen for selecting a menu, a character, a car type, or the like to the first display 110 at the beginning of execution. When a controller is to be provided during execution of the game application, the electronic device 100 can output the appropriate controller to the second display 120. [ According to an embodiment, even if both the first display 110 and the second display 120 support touch and hovering input, the electronic device 100 can not recognize the touch input from the first display 110 until the controller is output to the second display 120 , Or ignore the touch input input via the first display 110 This can prevent unintended touch input from occurring.

At operation 320, the electronic device 100 (e.g., processor 130) may determine the controller corresponding to the application. For example, the electronic device 100 may store a unique identification of an application, for example, a package name or a predetermined serial number of an application, a hash value, and the like in an application package or an application And the like. According to one embodiment, when a controller corresponding to unique identification information of an application is defined, the electronic device 100 can output the predefined controller to the second display 120. Alternatively, when the controller corresponding to the unique identification information of the application is not defined, the electronic device 100 can determine the type of the application, for example, the category of the application classified based on the controller suitable for the application. For example, if the application is a racing game category, the electronic device 100 may output to the second display 120 a controller suitable for a racing game. For example, the electronic device 100 may output to the second display 120 a controller including a handle (or directional key), a brake, an accelerator, and the like.

The electronic device 100 may utilize the first database and / or the second database described above to determine a controller suitable for the application. With reference to a concrete database utilization method, the description will be made with reference to Fig.

At operation 330, the electronic device 100 (e.g., processor 130) may output an execution screen of the application to the first display 110. The electronic device 100 may also output the determined controller to the second display 120 at operation 340, if a controller suitable for the application is determined.

In Fig. 3 or the process described below, the sequence of operations shown in the form of a flow chart is not limiting. Unless logically contradicted, the posterior relationship may be changed, or two or more operations may be executed simultaneously. For example, in the process of FIG. 3, operation 330 is performed first, and operation 320 may be performed prior to operation 340. That is, the process may proceed in the order of operation 310, operation 330, operation 320, operation 340.

Figure 4 shows a reference relationship between databases in accordance with one embodiment.

The application 141 includes an application ID 411 which is unique identification information of an application, a resource 413 containing data related to execution of an application executable file or image, and other applications, and setting information 415 including signature information, can do. For example, in the case of the Android operating system, the application ID 411 may correspond to the application package name, the configuration information 415 may correspond to the AndriodManifest file, and the resource 413 may correspond to other remaining files or information. In one embodiment, the application is installed in the electronic device 100 from a source file, such as an application package. According to one embodiment, the application 141 in this document can be understood as a concept including a source file such as an application package and a program (application) installed and executable from a source file.

The processor 130 may check the unique identification information from the application 141 and check whether there is data corresponding to the unique identification information in the first database (application meta DB) 143. [ The application meta DB 143 includes fields of an ID 431, an application title 433, an application ID 435, a category 437, a pad-controller support flag 439, and a user-selected controller ID 441 You can have a list of records to do. In other words, the application meta DB 143 includes a plurality of records, and each record may include the fields 431, 433, 435, 437, 439, and 441 described above. A similar description can be applied to the controller meta DB 145 as well.

ID 431 may correspond to information (e.g., a serial number) for identifying each record. The application title 433 is the name of the application. If the application is installed in the electronic device 100, the application title 433 may correspond to an identifiable system-registered name. The application ID 435 may correspond to unique identification information that enables identification of each application when the application is installed in the electronic device 100. [ For example, the package name of the Android application, or other unique value, such as a hash value, may correspond to application ID 435. In the first database 143, the ID 431 and the application ID 435 have unique values. ID 431 may be omitted.

The category 437 is a criterion for judging a controller suitable for an application, and may correspond to a classification of an application in which a specific controller can be utilized. For example, in the case of a game application, category 437 may be action, racing, roll playing game, simulation, puzzle, touch, and the like. According to one embodiment, the category is for an application to which a predefined controller is not applied, and may correspond to a controller that is directly generated by the user.

The pad controller support information 439 may indicate whether the application can be controlled by linking a physical external controller. Alternatively, in one embodiment, the pad controller support information 439 may indicate whether the application is appropriate to drive using a padded controller, such as an X-box or Nintendo. The field 439 may have a value of yes / no or 0/1.

The user-defined controller ID 441 indicates the ID of the controller when the controller to be used when driving the application is already defined. The field 441 may have a value or may not have a value.

The processor 130 can identify a record having the same (or corresponding) application ID 435 as the application ID 411 of the application 141. [ When the record is confirmed, the processor 130 can determine whether the value specified in the user-defined controller ID 441 is present in the record. If there is a value assigned to the user-defined controller ID 441, the processor 130 may output the controller having the specified value (controller ID 451) to the second display 120 in the second database 145. If there is no value defined in the field 441, the processor 130 identifies the category 437, determines one of the controllers having the category 453 corresponding to the category 437 in the second database 145, and outputs the determined controller to the second display 120 . A specific process related to this will be described with reference to Figs. 5, 7, and 8. Fig.

The second database (controller meta DB) 145 includes a controller ID 451, a category 453, a configuration file destination 455, a user-defined flag 457, and a reference controller ID 459 . The category 453 may correspond to the category 437 of the first database 143. For example, if the category 437 is an RPG, the processor 130 may cause the second database 145 to output a suitable controller (e.g., controller with the highest user rating) on the second display 120 among the controllers whose category 453 is RPG.

The configuration file location 455 may indicate the location of the file defining the UI of the controller. The file may use a tag of a markup language such as XML, for example. The UI of the controller can be stored in a structured document format using various languages (eg XML, JavaScript Object Notation (JSON), PHP, HTML, XHTML, DHTML). For example, a file, a location, a size, and the like of a background (<background>), a stick (eg, joystick), and a button (<button> The processor 130 may output the controller to the second display 120 based on the file defining the UI.

<? xml version = "1.0" encoding = "UTF-8"?>

<background src = "image / background.png" width = 800 height = 600 />

<stick>

<stick id = "A" src = "image / Button_A.png" x = 80 y = 300 width = 100 height = 100>

</ stick>

<button>

Button id = "A" src = "image / Button_A.png" x = 320 y = 480 width = 30 height =

<button id = "B" src = "image / Button_B.png" x = 360 y = 460 width = 30 height = 30>

<button id = "X" src = "image / Button_X.png" x = 320 y = 440 width = 30 height = 30>

<button id = "Y" src = "image / Button_A.png" x = 360 y = 420 width = 30 height = 30>

<button id = "select" src = "image / Button_Select.png" x = 200 y = 440 width = 60 height = 20>

<button id = "start" src = "image / Button_Start.png" x = 200 y = 420 width = 60 height = 20>

</ button>

<feature>

<feature id = "chat" src = "image / Button_Chat.png" x = 120 y = 200 width = 50 height = 50>

<feature id = "screenshot" src = "image / Button_Screenshot.png" x = 180 y = 200 width = 50 height = 50>

<feature id = "record" src = "image / Button_Record.png" x = 240 y = 200 width = 50 height = 50>

<feature id = "edit" src = "image / Button_Edit.png" x = 300 y = 200 width = 50 height = 50>

</ feature>

The user-defined information 457 may indicate whether the controller is a controller created or modified by the user. Also, reference controller ID 459 can have the controller ID (controller ID 451) value of the referenced controller if the controller is modified or built based on another controller.

The database and its fields described with reference to Fig. 4 are illustrative and may be replaced with other fields or fields may be added. For example, since there may be a plurality of controller IDs corresponding to one category, the second database 145 may store additional fields such as priority, rating, preference, number of use, recommendation count, etc. for determining the priority of the controller ID . &Lt; / RTI &gt; Also, the first database 143 and the second database 145 can be understood as one database having at least two tables.

5 illustrates a process for determining a controller suitable for an application according to an embodiment.

Operation 310 is the operation described with reference to Figure 3. The electronic device 100, e.g., the processor 130, may execute the application. When the application is executed, the processor 130 determines in operation 520 that the application ID is a database The processor 130 may determine whether a record exists in the first database 143 with a field containing the application ID value of the executed application.

Once the application ID is identified in the database, the electronic device 100 (e.g., the processor 130) can determine whether a predefined controller exists for the application ID identified in operation 530. [ For example, the processor 130 may determine whether the user-defined controller ID 441 field in the record specifies a specific controller ID.

If there is a predefined controller for the identified application ID, then the electronic device 100 (e.g., processor 130) may determine the predefined controller at act 540 as the controller for the executed application.

The electronic device 100 (e.g., processor 130) may output an execution screen of the application to the first display at operation 550. As noted in FIG. 3, operation 550 may be understood to be performed immediately after operation 310.

At operation 560, the electronic device 100 (processor 130) may output the determined controller to the second display. For example, the processor 130 can determine the configuration file location 455 corresponding to the controller ID value of the controller determined in the second database 145. The processor 130 may output the controller to the second display 120 based on the UI defined in the file corresponding to the setting file location 455. [

At operation 520, the application's ID may not be present in the database, for example because the application's release date is relatively recent. In this case, the processor 130 may communicate with the server 200 to identify the category of the application ID or the designated controller. Operation 700 may be performed in this regard. Operation 700 will be described below with reference to FIG.

If there is no controller defined for the application at operation 530, the electronic device 100 (e.g., processor 130) may perform operation 532. [ At operation 532, the processor 130 may determine whether the application supports a pad type of controller. For example, the processor 130 may check the pad controller support information 439 field in the database.

As a result, if the application supports the pad controller, the operation 534 for setting the pad controller is performed, and if not, the operation 800 for setting the touch controller can be performed. Operation 800 will be described below with reference to FIG.

At operation 534, the electronic device 100 (e.g., processor 130) may determine a pad controller corresponding to the category of the application. For example, the processor 130 may determine a category (e.g., category 437) corresponding to the application identification information and determine a suitable pad controller based on the determined category. Examples of the pad controller will be described with reference to Figs. 6A, 6B, and 6C.

6A, 6B and 6C show application execution screens and controllers of different categories according to an embodiment.

6A shows a pad controller corresponding to the execution screen 610. Fig. The execution screen 610 is a screen for selecting a game, and since the game has not yet started, the general-purpose controller can be output. For example, a controller including a direction button 601, a left key 602, a right key 603, a game selection button 604, an end button 605, and a control button 606 may be output.

6B shows a pad controller when a game in the RPG category is selected. When the execution screen 620 of the RPG is output to the first display 110, the processor 130 may output the controller designated for the executed game or the controller corresponding to the category to which the executed game belongs to the second display 120. [ For example, the controller may include a direction key 611, a left key 612, a right key 613, a start button 614, a selection button 615, function keys (jump, acceleration, launch etc.) 616 and a game setting related key 617.

6C shows a pad controller corresponding to the game execution screen 630 in the racing category. The controller may include a handle 621, a brake pedal 622, an accelerator pedal 623, an ignition plug button 624, a game setting related button 625, and the like.

7 shows a process of determining a controller based on communication with a server according to an embodiment.

7, if the application ID is not identified in the database, the electronic device 100 (e.g., the processor 130) transmits unique identification information (e.g., application ID 411) of the application to the server 200 via the communication module 150 at operation 710 .

Upon receiving the identification information of the application from the electronic device 100, the server 200 can refer to the application meta DB 220 of the server 200 and determine the predefined controller ID or category information corresponding to the identification information of the received application. Alternatively, in another embodiment, the server 200 may determine the controller directly based on the identification of the received application, and may send information about the determined controller to the electronic device 100. When the server 200 transmits a determination result to the electronic device 100, the server 200 may transmit a record of the application meta DB 220 corresponding to the application identification information and / or a record of the controller meta DB 230 corresponding to the ID to the electronic device 100 as a control.

In operation 720, the electronic device 100 (e.g., the processor 130) may receive controller ID or category information corresponding to the application ID from the server 200 in response to the transfer. The processor 130 may determine the controller in the second database 145 using, for example, the received controller ID or category based on the received information. The subsequent operation corresponds to the process after the operation 530 in FIG. 5, and a description thereof will be omitted.

As another example, the electronic device 100 (e.g., processor 130) may receive controller information from the server 200 in response to the transmission at operation 730. Here, the controller information may include a file defining the UI of the controller. The processor 130 may output the controller to the second display 120 based on the received information. The subsequent operation corresponds to the process after operation 550 of FIG. 5, and a description thereof will be omitted.

In the embodiment of FIG. 7, the electronic device 100 (e.g., processor 130) may update the database stored in the memory 140 using information (e.g., a record) provided from the server 200.

8 shows a process for setting a button controller according to an embodiment. FIG. 8 may be performed at the time when it is determined that the application does not support the pad controller in the process of FIG. 5 (after the operation of operation 532). According to an exemplary embodiment, It may be performed at the time of requesting the generation of the button controller.

At operation 810, the electronic device 100 (e.g., processor 130) may output an execution screen of the application to the first display 110. Operation 810 may correspond to operation 550 described above.

At operation 820, the electronic device 100 (e.g., the processor 130) may recognize the button control area included in the execution screen of the application. Based on the recognition performance, the processor 130 may copy the execution screen to another display. For example, a specific frame of the execution screen output to the first display 110 may be copied to the second display 220. The processor 130 can recognize the button on the basis of the copied screen or directly recognize the button on the execution screen. The processor 130 may recognize an object corresponding to a size of the button region that is generally used in the application as a button among the objects recognized as buttons.

At operation 830, the electronic device (e.g., processor 130) may display the recognized button on the second display 220. The displayed button may have the nature of a basic guide controller. If the button is misrecognized or requires editing, the button controller can be finally edited through the operation 840.

Editing of the button, such as adding or deleting buttons, may be performed at operation 840. For example, when a touch and hovering input is generated for the displayed button, the corresponding button can be deleted from the recognition target button. If a touch and hovering input for an unrecognized area of the button occurs, the processor 130 may add the area to the recognized touch (button) and hovering area. The position of the button may also be adjusted via a touch-and-drag input.

At operation 850, the electronic device 100 (e.g., processor 130) may store the settings of the button controller. The process of button editing of the button controller, such as operation 840 and operation 850, may be applied to the pad controller in a similar manner. For example, addition, deletion, position, size change, etc. of control areas for the control items included in the pad controller can be performed.

According to one embodiment, the processor 130 may store information about the recognized buttons in a database for a controller ID. This will be described later with reference to FIG.

9 shows a button controller according to an embodiment.

According to one embodiment, when an application such as an execution screen 910 is executed or a specific input of a user occurs, the processor 130 can recognize the button included in the execution screen 910. [ For example, a button included in the execution screen 910 may be recognized as a button controller 920. [ The buttons indicated by 1 to 7 on the execution screen 910 may correspond to the buttons corresponding to 1 to 7 respectively in the button controller 920.

When the user selects button 2, button 2 may be deleted from the button controller 920. Also, if the user selects an arbitrary position not recognized on the execution screen 910, a button corresponding to the position can be added to the button controller 920.

In addition, since the buttons included in the button controller 920 can be efficiently distributed in a specific area for fast inputting, the full screen 900 includes the execution screen 910 and the button controller 920, as well as the first spare area 921 and the second spare area 922 &lt; / RTI &gt; Appropriate buttons or menus may be added to the first spare area 921 or the second spare area 922. Also, in one embodiment, the full screen 900 may be composed of an execution screen 910 and a button controller 920, without a spare area.

10 shows a process for changing a controller according to an embodiment.

10 can be applied to a button constituting the pad controller or a button constituting the button controller described above.

When the editing process of the controller is started, at operation 1010, the electronic device 100 (e.g., processor 130) may change the controller type or type. For example, if a controller such as the one shown in FIG. 6A is provided, the user can change the controller to the controller shown in FIG. 6C or change the controller to the button controller.

In operation 1020, the electronic device 100 (e.g., processor 130) may change the position or size of a button or control object included in the controller. Further, at operation 1030, the electronic device 100 (e.g., processor 130) may change the color or theme of the controller. For example, the processor 130 may change the color of some buttons or objects in the controller, or change the controller's theme (e.g., change from a PlayStation theme to a Nintendo theme).

If it is determined at operation 1010, 1020, and 1030 that there is a change, at operation 1001, the electronic device 100 (e.g., processor 130) may temporarily store the change. When an input for storing the change occurs at operation 1050, the processor 130 may terminate the change process of the controller. The processor 130 may store information about the changed controller, for example, by assigning a new controller ID 451 to the second database 145. If you have made some changes to the source controller, the controller ID information of the source controller may be stored in reference controller ID 459. The processor 130 may also generate a markup document file corresponding to the changed UI and store the path of the generated file in the configuration file location 455. [

FIG. 11 shows a mapping between a button and a controller constituting the button controller according to an embodiment.

Referring to FIG. 11, the second database 145 may include a controller list. Each record included in the list can be identified by a controller ID. When the controller ID corresponds to the button controller, a record corresponding to each of at least one button constituting the button controller may correspond to the controller ID.

For example, when the controller ID # 2 1110 is a button controller including five buttons, the controller ID # 2 may correspond to the information 1120 corresponding to each button in a one-to-many relationship. Each of the information 1120 may include an identification field 1121 of a button, a controller ID field 1122, a coordinate (center coordinate) field 1123 of a button, a path field 1124 of a button image if the button is an image, and other flag fields 1125. Here, the controller ID field 1122 may all have a value corresponding to the controller ID # 2.

Figures 12 and 13 illustrate examples of available controllers.

12 shows a user adjustable controller according to one embodiment. A screen 1210 shown on the left side of FIG. 12 may correspond to a screen on which a DEFAULT controller is displayed. The user can pause the execution of the application and edit the controller if the position of the object constituting the controller is inconvenient or there is an unused object during the game using the basic controller. For example, the process of FIG. 10 may be performed.

For example, in the controller of the screen 1210, when the position of the buttons A, B, C, D, and CD is inconvenient to use without using the button AB, the user removes the button AB, D and the position of the CD is adjusted as shown on the screen 1220, the game can be used again.

13 shows an example of controlling a cursor with a controller according to an embodiment. 13, the first spare area 1321 and the second spare area 1322 correspond to the contents described in FIG. 9, and a description thereof will be omitted.

When a cursor is displayed on the screen 1310, a controller 1320 including a touch pad 1332 may be provided. (E.g., selection, deletion, exit, etc.) may be assigned to the buttons 1331 designated as 1, 2, 3, and 4, respectively. The user may also provide a gesture input, such as a touch or drag, to the touch pad associated with the cursor 1301 to control the cursor 1301. For example, when a user input is generated through the touch pad 1332, the processor 130 maps the user input and the cursor 1301, moves the cursor 1301 in a manner corresponding to the user input, or selects an object on which the cursor 1301 is located Can be performed. Or a gesture input input through the touch pad 1332, for example, a swipe or a double tap.

According to various embodiments, an electronic device for performing a method of providing or providing a controller as described above includes a memory that stores a database containing information about at least one display, an application, and a plurality of controllers, and a processor Wherein the processor is further configured to display content including at least one object in association with execution of the application in a first area of the display, determine a controller corresponding to the application based on the database, To the second area of the display.

The display may also include a first display and a second display, wherein the first area corresponds to the first display and the second area corresponds to the second display.

According to one embodiment, the processor is further configured to determine an attribute corresponding to the application and, if the attribute is a first attribute, to cause the first controller to move the at least one object through the second area of the display And if the attribute is a second attribute, a second controller for moving the at least one object can be displayed through the second area.

According to an embodiment, the processor may output the controller to the second area based on information on the controller, when information on the controller corresponding to the unique identification information of the application exists in the database . Here, the information on the controller may include information on a file defining a user interface (UI) of the controller, and the processor may output the UI of the controller defined in the file to the second area.

Further, the processor may determine the controller based on the attribute information of the application when the database does not have a controller specified for unique identification information of the application.

If the controller does not have a controller assigned to the unique identification information of the application, the processor determines whether or not the application supports a pad type controller, and if the controller supports a pad type controller or a button type controller And outputting it to the second area.

According to one embodiment, the electronic device further comprises a communication module, wherein the processor transmits the unique identification information of the application to the server via the communication module when the application is not identified in the database, And output the controller determined based on the controller ID received from the server or category information of the application to the second area in response to the request.

The processor may output the controller to the second area based on controller information received from the server in response to the transmission.

The processor may also update the database based on information received from the server.

According to an embodiment, the processor may determine, by the controller, a controller having the highest priority among a plurality of controllers corresponding to the category information.

Also, the processor may change at least one of the number, size, position, and color of the control objects included in the controller based on the user input when the user input for changing the UI of the controller output to the second region is received .

According to one embodiment, the processor can change the type or theme of the controller based on the user input when a user input for changing the UI of the controller output to the second area is received.

The processor may recognize the button included in the execution screen of the application and output the controller of the button type including at least one button in the second area based on the recognized button. Here, when the recognized button is selected, the processor may delete a button corresponding to the selected button among the at least one buttons included in the button-type controller. In addition, the processor may add a button corresponding to the selected area to the button-type controller when an area in which the button is not recognized is selected.

The method for providing a controller using various electronic apparatuses described above includes the steps of displaying an object including at least one object in a first area of a display functionally connected to the electronic device in association with an operation of executing an application, Determining a controller to move the at least one object based on the database, and outputting the determined controller to a second area of the display.

Wherein the act of outputting the determined controller to the second region comprises: displaying, through the second region of the display, a first controller for moving at least one object when the attribute is a first attribute; And if the attribute is a second attribute, displaying a second controller for moving the at least one object through the second area.

An electronic device (e.g., electronic device 100) according to one embodiment may include a memory that stores a database containing information about at least one display, an application, and a plurality of controller sets, have. The processor (e.g., the processor 130) displays the plurality of controller sets with respect to the contents displayed on the display through the application, selects one of the plurality of controller sets, The selected controller set can be displayed in association with the content.

Also, a method of displaying a set of controllers according to one embodiment in association with content comprises: displaying a plurality of controller sets for content displayed on a display operatively connected to the electronic device through an application; Selecting one of the plurality of controller sets; And displaying the selected controller set in association with the content. The at least one display includes a first display and a second display, wherein the content is displayed on the first display, and the selected controller set can be displayed on the second display.

Wherein the determining of the controller corresponding to the application comprises an operation of providing identification information of the application to the server, an operation of receiving controller information or category information from the server in response to the transmission, And determining the controller.

A computer-readable medium having computer-executable instructions stored thereon, the instructions executable by at least one processor to cause an electronic device to perform the steps of: executing an application; The method comprising: displaying content including at least one object in a first area of a display functionally associated with the electronic device; determining a controller for moving the at least one object based on a database; To the second area of the display.

14 illustrates an electronic device in a network environment in accordance with various embodiments.

14, electronic devices 1401, 1402, 1404, or servers 1406 in various embodiments may be interconnected via network 1462 or near-field communication 1464. [ The electronic device 1401 may include a bus 1410, a processor 1420, a memory 1430, an input / output interface 1450, a display 1460, and a communication interface 1470. In some embodiments, the electronic device 1401 may omit at least one of the components, or may additionally include other components.

The bus 1410 may include circuitry, for example, for connecting the components 1410-1470 to each other and for communicating (e.g., control messages and / or data) between the components.

Processor 1420 may include one or more of a central processing unit (CPU), an application processor (AP), or a communications processor (CP). Processor 1420 may perform computations or data processing related to, for example, control and / or communication of at least one other component of electronic device 1401. [

Memory 1430 may include volatile and / or non-volatile memory. Memory 1430 may, for example, store instructions or data related to at least one other component of electronic device 1401. According to one embodiment, the memory 1430 may store software and / or programs 1440. The program 1440 may include, for example, a kernel 1441, a middleware 1443, an application programming interface (API) 1445, and / or an application program (or "application" At least a portion of the kernel 1441, middleware 1443, or API 1445 may be referred to as an Operating System (OS).

The kernel 1441 may include, for example, system resources (e.g., bus 1410, processor 1420, etc.) used to execute an operation or function implemented in other programs (e.g., middleware 1443, API 1445, or application program 1447) Memory 1430, etc.). In addition, the kernel 1441 may provide an interface that can control or manage system resources by accessing individual components of the electronic device 1401 in the middleware 1443, API 1445, or application program 1447.

The middleware 1443 can perform an intermediary function, for example, such that the API 1445 or the application program 1447 communicates with the kernel 1441 to exchange data.

In addition, the middleware 1443 may process one or more task requests received from the application program 1447 according to the priority order. For example, middleware 1443 may prioritize the use of system resources (e.g., bus 1410, processor 1420, or memory 1430, etc.) of electronic device 1401 in at least one of application programs 1447. For example, the middleware 1443 can perform the scheduling or load balancing of the one or more task requests by processing the one or more task requests according to the priority assigned to the at least one task.

The API 1445 is an interface for the application 1447 to control the functions provided by the kernel 1441 or the middleware 1443. The API 1445 includes at least one interface for file control, window control, image processing, Or functions (e.g., commands).

The input / output interface 1450 may serve as an interface through which commands or data input from, for example, a user or other external device can be communicated to another component (s) of the electronic device 1401. The input / output interface 1450 may also output commands or data received from other component (s) of the electronic device 1401 to a user or other external device.

Display 1460 may include, for example, a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light emitting diode (OLED) microelectromechanical systems (MEMS) displays, or electronic paper displays. Display 1460 may display various content (e.g., text, image, video, icon, or symbol, etc.) to a user, for example. Display 1460 may include a touch screen and may receive touch, gesture, proximity, or hovering input, for example, using an electronic pen or a portion of the user's body.

The communication interface 1470 may establish communication between the electronic device 1401 and an external device (e.g., the first external electronic device 1402, the second external electronic device 1404, or the server 1406). For example, the communication interface 1470 may be connected to the network 1462 via wireless or wired communication to communicate with the external device (e.g., the second external electronic device 1404 or the server 1406).

Wireless communication is, for example, a cellular communication protocol such as Long-Term Evolution (LTE), LTE-Advanced (LTE-A), Code Division Multiple Access (CDMA), Wideband CDMA (WCDMA) Telecommunications System), WiBro (Wireless Broadband), or Global System for Mobile Communications (GSM). The wireless communication may also include, for example, local communication 1464. The local area communication 1464 may include at least one of, for example, Wireless Fidelity (Wi-Fi), Bluetooth, Near Field Communication (NFC), magnetic stripe transmission (MST)

The MST generates a pulse according to the transmission data using an electromagnetic signal, and the pulse can generate a magnetic field signal. The electronic device 1401 transmits the magnetic field signal to a point of sale (POS), the POS uses the MST reader to detect the magnetic field signal, and converts the detected magnetic field signal into an electric signal to restore can do.

The GNSS may be implemented by a GPS (Global Positioning System), Glonass (Global Navigation Satellite System), Beidou Navigation Satellite System (Beidou), or Galileo (European Global Satellite-based navigation system) Or the like. Hereinafter, in this document, "GPS" can be interchangeably used with "GNSS ". The wired communication may include at least one of, for example, a universal serial bus (USB), a high definition multimedia interface (HDMI), a recommended standard 1532 (RS-232), or plain old telephone service (POTS). The network 1462 may include at least one of a telecommunications network, e.g., a computer network (e.g., a LAN or WAN), the Internet, or a telephone network.

Each of the first and second external electronic devices 1402, 1404 may be the same or different kind of device as the electronic device 1401. According to one embodiment, the server 1406 may include one or more groups of servers. According to various embodiments, all or a portion of the operations performed on the electronic device 1401 may be performed on one or more other electronic devices (e.g., electronic device 1402, 1404, or server 1406). According to one embodiment, in the event that the electronic device 1401 has to perform some function or service automatically or upon request, the electronic device 1401 may, instead of or in addition to executing the function or service itself, (E.g., electronic device 1402, 1404, or server 1406). Other electronic devices (e.g., electronic device 1402, 1404, or server 1406) may execute the requested function or additional function and forward the results to electronic device 1401. The electronic device 1401 can directly or additionally process the received result to provide the requested function or service. For this purpose, for example, cloud computing, distributed computing, or client-server computing technology may be used.

15 shows a block diagram 1500 of an electronic device 1501 according to various embodiments.

Referring to Fig. 15, the electronic device 1501 may include all or part of the electronic device 1401 shown in Fig. 14, for example. The electronic device 1501 includes one or more processors (e.g., AP) 1510, a communication module 1520, a subscriber identification module 1524, a memory 1530, a sensor module 1540, an input device 1550, a display 1560, an interface 1570, an audio module 1580, A module 1595, a battery 1596, an indicator 1597, and a motor 1598.

The processor 1510 may, for example, operate an operating system or an application program to control a number of hardware or software components coupled to the processor 1510, and may perform various data processing and operations. The processor 1510 may be implemented with, for example, a system on chip (SoC). According to one embodiment, the processor 1510 may further include a graphics processing unit (GPU) and / or an image signal processor. Processor 1510 may include at least a portion (e.g., cellular module 1521) of the components shown in FIG. Processor 1510 may load and process instructions or data received from at least one of the other components (e.g., non-volatile memory) into volatile memory and store the various data in non-volatile memory.

The communication module 1520 may have the same or similar configuration as the communication interface 1470 of Fig. The communication module 1520 may include, for example, a cellular module 1521, a Wi-Fi module 1522, a Bluetooth module 1523, a GNSS module 1524 (e.g., a GPS module, a Glonass module, a Beidou module, or a Galileo module), an NFC module 1525, an MST module 1526 , And a radio frequency (RF) module 1527.

The cellular module 1521 can provide voice calls, video calls, text services, or Internet services, for example, over a communication network. According to one embodiment, cellular module 1521 may utilize a subscriber identity module (e.g., SIM card) 1529 to perform the identification and authentication of electronic device 1501 within the communication network. According to one embodiment, the cellular module 1521 may perform at least some of the functions that the processor 1510 may provide. According to one embodiment, the cellular module 1521 may include a communications processor (CP).

Each of the Wi-Fi module 1522, the Bluetooth module 1523, the GNSS module 1524, the NFC module 1525, or the MST module 1526 may include a processor for processing data transmitted and received through the corresponding module, for example. At least some (e.g., two or more) of the cellular module 1521, the Wi-Fi module 1522, the Bluetooth module 1523, the GNSS module 1524, the NFC module 1525, or the MST module 1526, Or an IC package.

The RF module 1527 can, for example, send and receive communication signals (e.g., RF signals). RF module 1527 may include, for example, a transceiver, a power amplifier module (PAM), a frequency filter, a low noise amplifier (LNA), or an antenna. According to another embodiment, at least one of the cellular module 1521, the Wi-Fi module 1522, the Bluetooth module 1523, the GNSS module 1524, the NFC module 1525, and the MST module 1526 can transmit and receive RF signals through separate RF modules.

Subscriber identity module 1529 may include, for example, a card containing a subscriber identity module and / or an embedded SIM and may include unique identification information (e.g., ICCID) or subscriber information (E.g., international mobile subscriber identity (IMSI)).

Memory 1530 (e.g., memory 1430) may include, for example, internal memory 1532 or external memory 1534. The internal memory 1532 may include, for example, volatile memory (e.g., dynamic RAM, SRAM, or synchronous dynamic RAM (SDRAM)), non-volatile memory (e.g., OTPROM one time programmable ROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash, or NOR flash), a hard drive, or a solid state drive (SSD).

The external memory 1534 may be a flash drive, for example, a compact flash (CF), a secure digital (SD), a micro-SD, a mini-SD, an extreme digital (xD), a multi- a memory stick, and the like. The external memory 1534 may be functionally and / or physically connected to the electronic device 1501 through various interfaces.

The security module 1536 may be a module including a storage space having a relatively higher security level than the memory 1530, and may be a circuit that ensures secure data storage and a protected execution environment. The security module 1536 may be implemented as a separate circuit, and may include a separate processor. The security module 1536 may include an embedded secure element (eSE), for example, in a removable smart chip, a secure digital (SD) card, or embedded within a fixed chip of the electronic device 1501 . In addition, the security module 1536 may be run on an operating system different from the operating system (OS) of the electronic device 201. For example, the security module 1536 may operate based on a Java card open platform (JCOP) operating system.

The sensor module 1540 may, for example, measure a physical quantity or sense an operating state of the electronic device 1501 and convert the measured or sensed information into an electrical signal. The sensor module 1540 includes a gesture sensor 1540A, a gyro sensor 1540B, an air pressure sensor 1540C, a magnetic sensor 1540D, an acceleration sensor 1540E, a grip sensor 1540F, a proximity sensor 1540G, a color sensor 1540H An on / humidity sensor 1540J, an illuminance sensor 1540K, or an ultraviolet (UV) sensor 1540M. Additionally or alternatively, the sensor module 1540 may include, for example, an E-nose sensor, an EMG (electromyography) sensor, an EEG (electroencephalogram) sensor, an ECG (electrocardiogram) sensor, an IR And / or a fingerprint sensor. The sensor module 1540 may further include a control circuit for controlling at least one sensor belonging to the sensor module 1540. In some embodiments, the electronic device 1501 may further include a processor configured to control the sensor module 1540, either as part of the processor 1510 or separately, to control the sensor module 1540 while the processor 1510 is in a sleep state .

The input device 1550 may include, for example, a touch panel 1552, a (digital) pen sensor 1554, a key 1556, or an ultrasonic input device 1558. The touch panel 1552 can employ, for example, at least one of an electrostatic type, a pressure sensitive type, an infrared type, and an ultrasonic type. Further, the touch panel 1552 may further include a control circuit. The touch panel 1552 may further include a tactile layer to provide a tactile response to the user.

 (Digital) pen sensor 1554 may be part of, for example, a touch panel or may include a separate sheet of identification. Key 1556 may include, for example, a physical button, an optical key, or a keypad. The ultrasonic input device 1558 can detect the ultrasonic wave generated by the input tool through the microphone (e.g., the microphone 1588) and confirm the data corresponding to the ultrasonic wave detected.

Display 1560 (e.g., display 1460) may include a panel 1562, a hologram device 1564, or a projector 1566. Panel 1562 may include the same or similar configuration as display 1460 of FIG. The panel 1562 can be embodied, for example, flexible, transparent, or wearable. The panel 1562 may be composed of a touch panel 1552 and one module. The hologram device 1564 can display stereoscopic images in the air using the interference of light. The projector 1566 can display images by projecting light onto the screen. The screen may, for example, be located inside or outside the electronic device 1501. According to one embodiment, the display 1560 may further include control circuitry for controlling the panel 1562, the hologram device 1564, or the projector 1566.

The interface 1570 may include, for example, an HDMI 1572, a USB 1574, an optical interface 1576, or a D-sub (D-subminiature) The interface 1570 may be included in the communication interface 1470 shown in Fig. 14, for example. Additionally or alternatively, interface 1570 may include, for example, a mobile high-definition link (MHL) interface, an SD card / MMC interface, or an infrared data association (IrDA) interface.

Audio module 1580 can, for example, convert sound and electrical signals in both directions. At least some of the components of the audio module 1580 may be included in, for example, the input / output interface 1450 shown in FIG. The audio module 1580 can process sound information input or output through, for example, a speaker 1582, a receiver 1584, an earphone 1586, a microphone 1588, or the like.

The camera module 1591 is, for example, a device capable of capturing still images and moving images, and according to one embodiment, one or more image sensors (e.g., a front sensor or a rear sensor), a lens, an image signal processor And may include a flash (e.g., LED or xenon lamp).

The power management module 1595 can manage the power of the electronic device 1501, for example. According to one embodiment, the power management module 1595 may include a power management integrated circuit (PMIC), a charger integrated circuit ("IC"), or a battery or fuel gauge. The PMIC may have a wired and / or wireless charging scheme. The wireless charging scheme may include, for example, a magnetic resonance scheme, a magnetic induction scheme, or an electromagnetic wave scheme, and may further include an additional circuit for wireless charging, for example, a coil loop, a resonant circuit, have. The battery gauge can measure, for example, the remaining amount of the battery 1596, the voltage during charging, the current, or the temperature. The battery 1596 may include, for example, a rechargeable battery and / or a solar battery.

The indicator 1597 may indicate a particular state of the electronic device 1501 or a portion thereof (e.g., processor 1510), such as a boot state, a message state, or a state of charge. The motor 1598 can convert an electrical signal into mechanical vibration and can generate vibration, haptic effects, and the like. Although not shown, the electronic device 1501 may include a processing unit (e.g., a GPU) for mobile TV support. The processing device for supporting mobile TV can process media data conforming to standards such as DMB (Digital Multimedia Broadcasting), DVB (Digital Video Broadcasting), or MediaFLOTM.

Each of the components described in this document may be composed of one or more components, and the name of the component may be changed according to the type of the electronic device. In various embodiments, the electronic device may comprise at least one of the components described herein, some components may be omitted, or may further include additional other components. In addition, some of the components of the electronic device according to various embodiments may be combined into one entity, so that the functions of the components before being combined can be performed in the same manner.

16 shows a block diagram of a program module according to various embodiments.

According to one embodiment, the program module 1610 (e.g., program 1440) includes an operating system (OS) that controls resources associated with an electronic device (e.g., electronic device 1401) Program 1447). The operating system may be, for example, android, iOS, windows, symbian, tizen, or bada.

The program module 1610 may include a kernel 1620, a middleware 1630, an API 1660, and / or an application 1670. At least a portion of the program module 1610 may be preloaded on an electronic device or downloaded from an external electronic device (e.g., electronic device 1402, 1404, server 1406, etc.).

The kernel 1620 (e.g., kernel 1441) may include, for example, a system resource manager 1621 or a device driver 1623. The system resource manager 1621 can perform control, allocation, or recovery of system resources. According to one embodiment, the system resource manager 1621 may include a process manager, a memory manager, or a file system manager. The device driver 1623 may include, for example, a display driver, a camera driver, a Bluetooth driver, a shared memory driver, a USB driver, a keypad driver, a Wi-Fi driver, an audio driver, or an inter-process communication .

The middleware 1630 may provide various functions to the application 1670 through the API 1660, for example, to provide functions that are commonly needed by the application 1670, or to allow the application 1670 to efficiently use limited system resources within the electronic device have. According to one embodiment, the middleware 1630 (e.g., middleware 1443) includes a runtime library 1635, an application manager 1641, a window manager 1642, a multimedia manager 1643, a resource manager 1644 A power manager 1645, a database manager 1646, a package manager 1647, a connectivity manager 1648, a notification manager 1649, a location manager 1650, A graphic manager 1651, a security manager 1652, or a payment manager 1654. [

The runtime library 1635 may include, for example, a library module used by the compiler to add new functionality via a programming language while the application 1670 is running. The runtime library 1635 may perform input / output management, memory management, or functions for arithmetic functions.

The application manager 1641 may, for example, manage the life cycle of at least one of the applications 1670. The window manager 1642 can manage the GUI resources used on the screen. The multimedia manager 1643 can identify the format required for playback of various media files and can encode or decode a media file using a codec suitable for the format. The resource manager 1644 can manage resources such as source code, memory or storage space of at least one of the applications 1670.

The power manager 1645 operates in conjunction with a basic input / output system (BIOS), for example, to manage a battery or a power source, and to provide power information necessary for the operation of the electronic device. The database manager 1646 may create, retrieve, or modify a database to be used in at least one of the applications 1670. The package manager 1647 can manage installation or update of an application distributed in the form of a package file.

The connection manager 1648 may manage wireless connections, such as, for example, Wi-Fi or Bluetooth. The notification manager 1649 may display or notify events such as arrival messages, appointments, proximity notifications, etc. in a manner that is not disturbed to the user. The location manager 1650 can manage the location information of the electronic device. The graphic manager 1651 can manage the graphical effect to be provided to the user or a related user interface. The security manager 1652 can provide all the security functions necessary for system security or user authentication. According to one embodiment, when an electronic device (e.g., electronic device 1401) includes a telephone function, middleware 1630 may further include a telephony manager for managing the voice or video call capabilities of the electronic device.

Middleware 1630 may include a middleware module that forms a combination of various functions of the above-described components. The middleware 1630 can provide a module specialized for each type of operating system to provide differentiated functions. In addition, the middleware 1630 may dynamically delete some existing components or add new ones.

API 1660 (e.g., API 1445) is a collection of API programming functions, for example, and can be provided in different configurations depending on the operating system. For example, for Android or iOS, you can provide one API set per platform, and for tizen, you can provide more than two API sets per platform.

An application 1670 (e.g., an application program 1447) may include, for example, a home 1671, a dialer 1672, an SMS / MMS 1673, an instant message 1674, a browser 1675, a camera 1676, an alarm 1677, 1680, a calendar 1681, a media player 1682, an album 1683, or a clock 1684, providing health care (e.g., measuring exercise or blood glucose), or providing environmental information (e.g., pressure, humidity, And the like) capable of performing the functions of the &lt; / RTI &gt;

According to one embodiment, an application 1670 is an application that supports the exchange of information between an electronic device (e.g., electronic device 1401) and an external electronic device (e.g., electronic device 1402, 1404) "). The information exchange application may include, for example, a notification relay application for communicating specific information to an external electronic device, or a device management application for managing an external electronic device.

For example, the notification delivery application may send notification information generated in other applications (e.g., SMS / MMS applications, email applications, healthcare applications, or environmental information applications) of the electronic device to external electronic devices 1404 via a network. Further, the notification delivery application can receive notification information from, for example, an external electronic device and provide it to the user.

The device management application may include, for example, at least one function of an external electronic device (e.g., electronic device 1402, 1404) in communication with the electronic device (e.g., a turn-on / turn (Eg, install, delete, or update) services provided by an external electronic device or external electronic device (eg, call service or message service) can do.

According to one embodiment, the application 1670 may include an application (e.g., a healthcare application of a mobile medical device) designated according to attributes of an external electronic device (e.g., electronic device 1402, 1404). According to one embodiment, the application 1670 may include an application received from an external electronic device (e.g., server 1406 or electronic devices 1402, 1404). According to one embodiment, application 1670 may include a preloaded application or a third party application downloadable from a server. The names of the components of the program module 1610 according to the illustrated embodiment may vary depending on the type of the operating system.

According to various embodiments, at least some of the program modules 1610 may be implemented in software, firmware, hardware, or a combination of at least two of them. At least some of the program modules 1610 may be implemented (e.g., executed) by, for example, a processor (e.g., processor 1510). At least some of the program modules 1610 may include, for example, modules, programs, routines, sets of instructions or processes, etc., to perform one or more functions.

As used in this document, the term "module" may refer to a unit comprising, for example, one or a combination of two or more of hardware, software or firmware. A "module" may be interchangeably used with terms such as, for example, unit, logic, logical block, component, or circuit. A "module" may be a minimum unit or a portion of an integrally constructed component. A "module" may be a minimum unit or a portion thereof that performs one or more functions. "Modules" may be implemented either mechanically or electronically. For example, a "module" may be an application-specific integrated circuit (ASIC) chip, field-programmable gate arrays (FPGAs) or programmable-logic devices And may include at least one.

At least a portion of a device (e.g., modules or functions thereof) or a method (e.g., operations) according to various embodiments may include, for example, computer-readable storage media in the form of program modules, As shown in FIG. When the instruction is executed by a processor (e.g., processor 1420), the one or more processors may perform a function corresponding to the instruction. The computer-readable storage medium may be, for example, a memory 1430.

The computer-readable recording medium may be a hard disk, a floppy disk, a magnetic media such as a magnetic tape, an optical media such as a CD-ROM, a DVD (Digital Versatile Disc) May include magneto-optical media (e.g., a floppy disk), a hardware device (e.g., ROM, RAM, or flash memory, etc.) Etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the various embodiments. And vice versa.

Modules or program modules according to various embodiments may include at least one or more of the elements described above, some of which may be omitted, or may further include additional other elements. Operations performed by modules, program modules, or other components in accordance with various embodiments may be performed in a sequential, parallel, iterative, or heuristic manner. Also, some operations may be performed in a different order, omitted, or other operations may be added.

And the embodiments disclosed in this document are provided for the explanation and understanding of the disclosed technical contents, and do not limit the scope of the present invention. Accordingly, the scope of this document should be interpreted to include all modifications based on the technical idea of the present invention or various other embodiments.

Claims (20)

In an electronic device,
At least one display,
An application, and a memory that stores a database containing information about the plurality of controllers, and
A processor,
Wherein the processor displays content including at least one object in association with execution of the application in a first area of the display, determines a controller corresponding to the application based on the database, And to output to a second area of the display. The method according to claim 1,
Wherein the display comprises a first display and a second display,
Wherein the first area corresponds to the first display and the second area corresponds to the second display. The system of claim 1,
Determines an attribute corresponding to the application,
If the attribute is a first attribute, displaying a first controller for moving the at least one object through a second area of the display,
And if the attribute is a second attribute, display a second controller for moving the at least one object through the second area. The method according to claim 1,
Wherein the processor is configured to output the controller to the second area based on information about the controller if information on the controller corresponding to the unique identification information of the application exists in the database. The method of claim 4,
Wherein the information on the controller includes information on a file defining a user interface (UI) of the controller,
And the processor is configured to output the UI of the controller defined in the file to the second area. The method of claim 3,
Wherein the processor is configured to determine the controller based on attribute information of the application when the database does not have a controller specified for unique identification information of the application. The method of claim 3,
The processor may determine whether the application supports a pad type controller if the database does not have a controller specified for the unique identification information of the application, and determine whether the pad type controller or the button type controller And to output to the second area. The method according to claim 1,
Further comprising a communication module,
Wherein the processor transmits unique identification information of the application to the server through the communication module when the application is not identified in the database, and transmits the controller ID received from the server or the category information of the application in response to the transmission And to output the controller determined based on the first area to the second area. The method of claim 8,
Wherein the processor is configured to output the controller to the second area based on controller information received from the server in response to the transmission. The method of claim 8,
Wherein the processor is configured to update the database based on information received from the server. The method of claim 6,
Wherein the processor is configured to determine, by the controller, a controller having the highest priority among a plurality of controllers corresponding to the attribute information. The method according to claim 1,
The processor changes at least one of the number, size, position, and color of the control objects included in the controller based on the user input when the user input for changing the UI of the controller outputted to the second region is received Is set. The method according to claim 1,
Wherein the processor is configured to change the type or theme of the controller based on the user input when a user input to change the UI of the controller output to the second region is received. The method of claim 7,
Wherein the processor is configured to recognize a button included in the execution screen of the application and to output the button type controller including at least one button in the second area based on the recognized button, . 15. The method of claim 14,
Wherein the processor is configured to delete a button corresponding to the selected button among the at least one button included in the controller of the button type when the recognized button is selected. 15. The method of claim 14,
Wherein the processor is configured to add a button corresponding to the selected area to the button type controller when an area in which the button is not recognized is selected. A method for providing a controller of an electronic device,
An operation for executing an application,
Displaying content in the first area of the display functionally associated with the electronic device, the content including at least one object in connection with the execution of the application;
Determining a controller for moving the at least one object based on the database, and
And outputting the determined controller to a second region of the display. 18. The method of claim 17, wherein the act of outputting the determined controller to the second region comprises:
Displaying, through the second area of the display, a first controller for moving at least one object if the attribute of the application is a first attribute; And
And if the attribute is a second attribute, displaying a second controller for moving the at least one object through the second area. 18. The method of claim 17,
Wherein the determining of the controller corresponding to the application comprises:
Transmitting the identification information of the application to the server,
Receiving controller information or category information from the server in response to the transmission; and
And determining the controller based on the received information. A computer-readable medium having computer-executable instructions stored thereon, the instructions executable by at least one processor to cause an electronic device to:
An operation for executing an application,
Displaying content in the first area of the display functionally associated with the electronic device, the content including at least one object in connection with the execution of the application;
Determining a controller for moving the at least one object based on the database, and
And outputting the determined controller to the second area of the display.

Images