KR102415984B1 - Method and Apparatus for Providing a Controller - Google Patents

Abstract

An electronic device is disclosed. The electronic device may include a memory that stores a database including information on at least one display, an application, and a plurality of controllers, and a processor. The processor displays content including at least one object in connection with the execution of the application on a first area of the display, determines a controller corresponding to the application based on the database, and sets the determined controller to the controller. It can output to the second area of the display. In addition to this, various embodiments identified through the specification are possible.

Description

Method and Apparatus for Providing a Controller

Embodiments disclosed in this document relate to a technology of providing a controller independently of an application in an electronic device.

Various embodiments of the present invention relate to a method, an apparatus, and a computer-readable recording medium for providing a controller for controlling an application.

In general, a controller for controlling an application like a conventional gaming device is provided as hardware having a physical form (eg, a keyboard, a keypad, a mouse, a joystick, and a game pad). As the game market using a mobile terminal such as a smart phone or a tablet succeeds, the electronic device provides a software game pad operated by a touch input in addition to a physical game pad.

It is difficult or impossible to customize the physical controller (change button configuration/position/color, etc.) into the desired shape. In addition, there are cases in which it is difficult for the user to adapt or operate because the location/configuration of the main buttons to be utilized is different depending on the characteristics of the application.

Also, even when a software-type controller is used, only a controller defined and provided by the application developer (or game producer) in advance may be used.

The embodiments disclosed in this document may provide an electronic device and a method thereof that provide a controller suitable for a type of application, and provide a controller that can be changed/definable according to a user's intention.

An electronic device according to an embodiment disclosed in this document may include a memory for storing at least one display, an application, and a database including information on a plurality of controllers, and a processor. The processor displays content including at least one object in connection with the execution of the application on a first area of the display, determines a controller corresponding to the application based on the database, and sets the determined controller to the controller. It can output to the second area of the display.

In addition, in the method according to an embodiment disclosed in this document, content including at least one object related to an operation of executing an application and execution of the application is placed on a first area of a display functionally connected to the electronic device. It may include an operation of displaying, an operation of determining a controller for moving the at least one object based on a database, and an operation of outputting the determined controller to a second area of the display.

In addition, the storage medium according to an embodiment disclosed in this document may store computer-readable instructions for performing the above-described method.

According to the embodiments disclosed in this document, by providing a controller suitable for application characteristics and user preference, the electronic device itself or in connection with the server, user convenience and application utility can be increased.

In addition, various effects directly or indirectly identified through this document may be provided.

1 illustrates an exemplary form of an electronic device according to an embodiment.
2 illustrates the configuration of an electronic device and a server according to an embodiment.
3 illustrates a process for providing a controller suitable for an application according to an exemplary embodiment.
4 illustrates a reference relationship between databases according to an embodiment.
5 illustrates a process for determining a controller suitable for an application according to an exemplary embodiment.
6A, 6B, and 6C show application execution screens and controllers of different categories according to an embodiment.
7 illustrates a process for determining a controller based on communication with a server according to an embodiment.
8 illustrates a process for setting a button controller according to an embodiment.
9 illustrates a button controller according to an embodiment.
10 illustrates a process for changing a controller according to an embodiment.
11 illustrates mapping of buttons and controllers constituting a button controller according to an embodiment.
12 illustrates a controller adjustable by a user according to an exemplary embodiment.
13 illustrates an example of controlling a cursor with a controller according to an exemplary embodiment.
14 illustrates an electronic device in a network environment according to various embodiments of the present disclosure.
15 is a block diagram 1500 of an electronic device 1501 according to various embodiments of the present disclosure.
16 is a block diagram of a program module according to various embodiments of the present disclosure.

Hereinafter, various embodiments of the present invention will be described with reference to the accompanying drawings. However, this is not intended to limit the present invention to specific embodiments, and it should be understood that various modifications, equivalents, and/or alternatives of the embodiments of the present invention are included. In connection with the description of the drawings, like reference numerals may be used for like components.

In this document, expressions such as "have", "may have", "includes", or "may include" indicate the presence of a corresponding characteristic (eg, a numerical value, function, operation, or component such as a part). and does not exclude the presence of additional features.

In this document, expressions such as “A or B”, “at least one of A or/and B”, or “one or more of A or/and B” may include all possible combinations of the items listed together. . For example, "A or B", "at least one of A and B", or "at least one of A or B" means (1) includes at least one A, (2) includes at least one B; Or (3) it may refer to all cases including both at least one A and at least one B.

Expressions such as "first", "second", "first", or "second" used in this document may modify various elements, regardless of order and/or importance, and may modify one element to another. It is used only to distinguish it from the components, and does not limit the components. For example, the first user equipment and the second user equipment may represent different user equipment regardless of order or importance. For example, without departing from the scope of the rights described in this document, a first component may be referred to as a second component, and similarly, the second component may also be renamed as a first component.

A component (eg, a first component) is "coupled with/to (operatively or communicatively)" to another component (eg, a second component); When referring to "connected to", it will be understood that the certain element may be directly connected to the other element or may be connected through another element (eg, a third element). On the other hand, when it is said that a component (eg, a first component) is "directly connected" or "directly connected" to another component (eg, a second component), the component and the It may be understood that other components (eg, a third component) do not exist between other components.

The expression "configured to (or configured to)" as used in this document, depending on the context, for example, "suitable for", "having the capacity to" It can be used interchangeably with "," "designed to", "adapted to", "made to", or "capable of". The term "configured (or set up to)" may not necessarily mean only "specifically designed to" in hardware. Instead, in some circumstances, the expression “a device configured to” may mean that the device is “capable of” with other devices or parts. For example, the phrase "a processor configured (or configured to perform) A, B, and C" executes a dedicated processor (eg, an embedded processor), or one or more software programs stored in a memory device, to perform the corresponding operations. By doing so, it may refer to a generic-purpose processor (eg, a CPU or an application processor) capable of performing corresponding operations.

Terms used in this document are only used to describe specific embodiments, and may not be intended to limit the scope of other embodiments. The singular expression may include the plural expression unless the context clearly dictates otherwise. Terms used herein, including technical or scientific terms, may have the same meanings as commonly understood by one of ordinary skill in the art described in this document. Among the terms used in this document, terms defined in a general dictionary may be interpreted with the same or similar meaning as the meaning in the context of the related art, and unless explicitly defined in this document, have an ideal or excessively formal meaning. not interpreted In some cases, even terms defined in this document cannot be construed to exclude embodiments of this document.

An electronic device according to various embodiments of the present document may include, for example, a smartphone, a tablet personal computer, a mobile phone, a video phone, an e-book reader, Desktop PC (desktop PC), laptop PC (laptop PC), netbook computer (netbook computer), workstation, server, PDA (personal digital assistant), PMP (portable multimedia player), MP3 player, mobile medical device, It may include at least one of a camera and a wearable device. According to various embodiments, the wearable device may be an accessory type (eg, a watch, ring, bracelet, anklet, necklace, glasses, contact lens, or head-mounted-device (HMD)), a fabric or an integral type of clothing ( For example, it may include at least one of an electronic garment), a body attachable type (eg a skin pad or a tattoo), or a bioimplantable type (eg, an implantable circuit).

In some embodiments, the electronic device may be a home appliance. Home appliances are, for example, televisions, DVD players (Digital Video Disk players), audio systems, refrigerators, air conditioners, vacuum cleaners, ovens, microwaves, washing machines, air purifiers, set-top boxes, and home automation. Control panel (home automation control panel), security control panel (security control panel), TV box (eg Samsung HomeSync™, Apple TV™, or Google TV™), game console (eg Xbox™, PlayStation™), electronics It may include at least one of a dictionary, an electronic key, a camcorder, and an electronic picture frame.

In another embodiment, the electronic device may include various medical devices (eg, various portable medical measuring devices (eg, a blood glucose monitor, a heart rate monitor, a blood pressure monitor, or a body temperature monitor), magnetic resonance angiography (MRA), magnetic resonance imaging (MRI), Computed tomography (CT), imager, or ultrasound machine, etc.), navigation device, global navigation satellite system (GNSS), event data recorder (EDR), flight data recorder (FDR), car infotainment ) devices, ship electronic equipment (e.g. ship navigation systems, gyro compasses, etc.), avionics, security devices, vehicle head units, industrial or domestic robots, automatic teller's machines (ATMs) in financial institutions. , point of sales (POS) in stores, or internet of things (e.g. light bulbs, sensors, electricity or gas meters, sprinkler devices, smoke alarms, thermostats, street lights, toasters) , exercise equipment, hot water tank, heater, boiler, etc.) may include at least one.

According to some embodiments, the electronic device is a piece of furniture or a building/structure, an electronic board, an electronic signature receiving device, a projector, or various measuring devices (eg, water, electricity, gas, or a radio wave measuring device). In various embodiments, the electronic device may be a combination of one or more of the various devices described above. The electronic device according to an embodiment may be a flexible electronic device. In addition, the electronic device according to the embodiment of the present document is not limited to the above-described devices, and may include a new electronic device according to technological development.

Hereinafter, an electronic device according to various embodiments will be described with reference to the accompanying drawings. In this document, the term user may refer to a person who uses an electronic device or a device (eg, an artificial intelligence electronic device) using the electronic device.

1 illustrates an exemplary form of an electronic device according to an embodiment.

Referring to FIG. 1 , an electronic device 100 according to an embodiment may include a plurality of displays. For example, the electronic device 100 may include a first display 110 and a second display 120 . The first display 110 and the second display 120 may be interconnected through the connection unit 115 . The connection part 115 may be formed of various materials. For example, it may be formed of plastic, metal, leather, synthetic resin, or the like. In another embodiment, the first display 110 and the second display 120 may be in contact with each other without the connection part 115 . In addition to this, the electronic device 100 may include a physical button 101 and a receiver 103 for another function (eg, call). Various components not shown in FIG. 1 (eg, a touch button, a volume key, a camera, a sensor, etc. disposed outside the display area) may be included in the electronic device 100, but in order to prevent the subject matter of the invention from being obscured, a detailed description will be given. omit

According to an embodiment, an execution screen of an application may be displayed on the first display 110 . A control UI (user interface) for controlling an execution screen of an application may be output on the second display 120 . For example, a racing game screen may be output on the first display 110, and a UI for controlling a racing game may be output on the second display 120. FIG. The UI may output, for example, a direction control menu (arrow key or handle), a speed control menu (accelerator pedal, brake), and a game control menu (pause, save, end, etc.). In this document, such a control UI is referred to as a controller, and unless otherwise specified, the controller may mean a software-type control UI output on a display. For convenience, various embodiments will be described focusing on a controller that controls a game application, but various embodiments may also be applied to non-game applications, for example, applications that can use the control UI, such as composition or image editing. .

Although the electronic device 100 having two displays is illustrated in FIG. 1 , in a modified embodiment, even when the electronic device 100 includes three or more displays, various embodiments disclosed in this document may be applied. For example, when the electronic device includes a plurality of displays, a controller may be output to one of the displays, and an execution screen of an application may be output to at least one display among the other displays.

In an embodiment, an embodiment of the present invention may be applied even when the electronic device has one display. For example, the execution screen of the application and the controller may be displayed on the same display. In this case, the execution screen of the application may be output to the first layer, and the controller may be output to the second layer having a higher z-order than the first layer. In another example, the electronic device 100 may divide one display into two or more regions, output an application execution screen to the first region, and output a control UI to the second region.

According to an embodiment, when connected to another electronic device using wired or wireless communication, the electronic device 100 outputs an application execution screen to a screen (eg, a monitor, TV, etc.) of the external device, and displays the electronic device 100 . can output the controller. In other words, the electronic device may provide an environment in which a larger screen or more free controller functions can be used by outputting a controller to its display and outputting an execution screen to a display of an external device functionally connected to the electronic device. . For example, when connected to a wearable device such as a head-mounted-device (HMD), the electronic device 100 outputs an application execution screen to the wearable device, and outputs a controller to the display of the electronic device 100 can do.

Hereinafter, various embodiments of the present invention will be described with reference to FIGS. 2 to 16 . For convenience, the electronic device 100 is described as having two displays, but as described above, the present invention can be applied to various types of electronic devices.

2 illustrates the configuration of an electronic device and a server according to an embodiment.

Referring to FIG. 2 , the electronic device 100 may include a first display 110 , a second display 120 , a processor 130 , a memory 140 , and a communication module 150 . The electronic device 100 may include additional components or omit some components. Descriptions of various hardware/software components applicable to the electronic device 100 are provided with reference to FIGS. 14, 15, and 16 .

At least one of the first display 110 and the second display 120 may receive a user's touch input or a hovering input. For example, the processor 130 may output an execution screen of the application on the first display 110 and output a controller corresponding to the application on the second display 120 . In this case, at least the second display 120 may recognize the user's touch input and hovering input. For example, the second display 120 may include a cover glass, a touch panel, and a display panel. When a touch input or a hovering input to the second display 120 sensed by the touch panel is received, the processor 130 may perform an operation based on the input and output the result to the first display 110 . Also, according to an embodiment, the first display 110 may also recognize a touch and hovering input.

The processor 130 may correspond to hardware having computing power, such as an application processor (AP) or a central processing unit (CPU). The processor 130 may include one or more processors or one or more cores. Also, according to an embodiment, the processor 130 may be implemented in the form of an integrated chip such as a system on chip (SoC).

In an embodiment, the processor 130 may manage provision of a controller suitable for an application through a controller manager. The controller manager may be understood as a software module implementable by the processor 130 . In the present specification, an operation of determining and managing a controller among operations performed by the processor 130 may be understood to be performed by a controller manager.

The memory 140 may store an application 141 , an application meta DB (database) 143 , and a controller meta DB 145 . The application 141 may be, for example, a game application. As described above, the application 141 may correspond to any application capable of providing an execution screen and a controller for controlling the execution screen (execution result). For example, in addition to a game, an application usable by a user capable of providing a content service such as music or a video may correspond to the application 141 .

The application meta DB 143 may include a list of applications that a user can utilize, and may include an additional information field for each application. In an embodiment, the application meta DB 143 may be referred to as the first database. The controller meta DB 145 includes a list of pre-defined controllers that can be provided to the user, that is, can be output to the second display 120, and may also include an appropriate additional information field. In an embodiment, the controller meta DB 145 may be referred to as a second database. The processor 130 may refer to the application meta DB 143 and the controller meta DB 145 to determine a controller suitable for the application 141 . The application meta DB 143 and the controller meta DB 145 will be described in detail with reference to FIG. 4 .

The electronic device 100 may communicate with the server 200 through a network. The server 200 may include applications 210 , an application meta DB 220 , and a controller meta DB 230 . Also, according to an embodiment, the server 200 may correspond to a plurality of servers. For example, the server 200 is a first server including the application 210 (eg, Android Market such as Google Play Store, Apple's AppStore, etc.), the application meta DB 220, and/or the second server including the controller meta DB 230 It may correspond to a server (or second and third servers). The electronic device 100 may access the server 200, download an application, and install the application on the electronic device 100 . Also, the application meta DB 220 and the controller meta DB 230 included in the server 200 may correspond to the application meta DB 143 and the controller meta DB 145 included in the electronic device 100, respectively. The database stored in the server 200 may include relatively more records than the database stored in the memory 140 of the electronic device 100 . For example, when the electronic device 100 cannot check information corresponding to the application 141 in the application meta DB 143, it may access the server 200 and check or download corresponding information from the application meta DB 220 of the server 200. An embodiment related thereto will be described with reference to FIG. 7 .

3 illustrates a process for providing a controller suitable for an application according to an exemplary embodiment.

Referring to FIG. 3 , in operation 310 , the electronic device 100 (eg, the processor 130 ) may execute an application (eg, the application 141 ). The application may be an application provided together with the controller or an application for which the controller should be provided while the application is running. Alternatively, the application may be an application that does not provide a controller. For example, when executing a game application, the electronic device 100 may output an execution screen for selecting a menu selection, a character or a car type, etc. on the first display 110 at the initial stage of execution. When the controller needs to be provided while the game application is running, the electronic device 100 may output a suitable controller to the second display 120 . According to an embodiment, even if both the first display 110 and the second display 120 support a touch and hovering input, the electronic device 100 does not recognize the touch input on the first display 110 from the point in time when the controller is output to the second display 120 It may be set not to, or a touch input input through the first display 110 may be ignored. In this way, it is possible to prevent an unintentional touch input from occurring.

In operation 320, the electronic device 100 (eg, the processor 130) may determine a controller corresponding to the application. For example, the electronic device 100 provides unique identification information of an application, for example, a package name, a predetermined serial number, and a hash value of the application, to an application package or application provided for installation of the application. It can be obtained from installation information of According to an embodiment, when a controller corresponding to unique identification information of an application is defined, the electronic device 100 may output the predefined controller to the second display 120 . Alternatively, when the controller corresponding to the unique identification information of the application is not defined, the electronic device 100 may determine the category of the classified application based on the type of the application, for example, a controller suitable for the application. For example, when the application is a racing game category, the electronic device 100 may output a controller suitable for the racing game to the second display 120 . For example, the electronic device 100 may output a controller including a steering wheel (or a direction key), a brake, an accelerator, and the like to the second display 120 .

The electronic device 100 may use the above-described first database and/or second database to determine a controller suitable for an application. A detailed database utilization method will be described with reference to FIG. 4 .

In operation 330 , the electronic device 100 (eg, the processor 130 ) may output an execution screen of the application on the first display 110 . Also, when a controller suitable for an application is determined, the electronic device 100 may output the determined controller to the second display 120 in operation 340 .

In FIG. 3 or a process to be described later, the precedence relationship of the operations shown in the form of a flow chart is not limited. As long as there is no logical contradiction, the precedence relationship may be changed, or two or more operations may be executed simultaneously. For example, in the process of FIG. 3 , operation 330 may be performed first, and operation 320 may be performed before operation 340 . That is, the process may proceed in the order of operation 310, operation 330, operation 320, and operation 340.

4 illustrates a reference relationship between databases according to an embodiment.

The application 141 includes an application ID 411 that is unique identification information of the application, a resource 413 including an executable file or image of the application, and other data related to the execution of the application, and setting information 415 including signature information and permission information of the application can do. For example, in the case of the Android operating system, application ID 411 may correspond to an application package name, setting information 415 may correspond to an AndriodManifest file, and resource 413 may correspond to other files or information. In an embodiment, the application is installed in the electronic device 100 from a source file such as an application package. According to an embodiment, in this document, the application 141 may be understood as a concept including a source file such as an application package and a program (application) that is installed and executable from the source file.

The processor 130 may check the unique identification information from the application 141 and check whether there is data corresponding to the unique identification information in the first database (application meta DB) 143 . Application meta DB 143 includes fields of ID 431, application title 433, application ID 435, category 437, pad-controller support flag 439, and user-selected controller ID 441 You can have a list of records that In other words, the application meta DB 143 may include a plurality of records, and each record may include the aforementioned fields 431, 433, 435, 437, 439, and 441. A similar description can also be applied to the controller meta DB 145.

ID 431 may correspond to information (eg, serial number) for identifying each record. The application title 433 is the name of the application, and when the application is installed in the electronic device 100, it may correspond to a name registered in a checkable system. The application ID 435 may correspond to unique identification information that enables identification for each application when the application is installed in the electronic device 100 . For example, the package name of the Android application or other unique value such as a hash value may correspond to the application ID 435 . In the first database 143, ID 431 and application ID 435 have unique values. ID 431 may be omitted.

Category 437 is a criterion for determining a controller suitable for an application, and may correspond to a classification of an application type in which a specific controller can be utilized. For example, in the case of a game application, category 437 may correspond to action, racing, roll playing game (RPG), simulation, puzzle, touch, and the like. According to an embodiment, the category is for an application to which a predefined controller is not applied, and may correspond to a controller directly created by a user.

Pad controller support information 439 may indicate whether a corresponding application can be controlled by interworking with a physical external controller. Alternatively, according to an embodiment, the pad controller support information 439 may indicate whether it is suitable for the corresponding application to be driven using a pad-type controller such as X-box or Nintendo. Field 439 may have a value of yes/no or 0/1.

The user-defined controller ID 441 indicates the ID of the controller when the controller to be used for running the corresponding application has already been defined. Field 441 may or may not have a value.

The processor 130 may check a record having the same (or corresponding) application ID 435 as the application ID 411 of the application 141 . When the record is checked, the processor 130 may determine whether there is a value specified in the user-defined controller ID 441 in the record. When there is a value specified in the user-defined controller ID 441 , the processor 130 may output a controller having a value (controller ID 451 ) specified in the second database 145 to the second display 120 . If there is no value defined in the field 441, the processor 130 checks the category 437, determines one of the controllers having the category 453 corresponding to the category 437 in the second database 145, and outputs the determined controller to the second display 120. can A specific process related thereto will be described with reference to FIGS. 5, 7, and 8 .

A second database (controller meta DB) 145 includes a controller ID 451, a category 453, a configuration file destination 455, a user-defined flag 457, and a reference controller ID 459 may include Category 453 may correspond to category 437 of the first database 143 . For example, when category 437 is RPG, the processor 130 may output a suitable controller (eg, a controller having the highest user rating) among controllers whose category 453 is RPG from the second database 145 to the second display 120 .

The configuration file location 455 may indicate the location of a file defining the UI of the controller. The file may use, for example, a tag of a markup language such as XML. The UI of the controller can be saved in the form of a structured document using various languages (eg, XML, JavaScript Object Notation (JSON), PHP, HTML, XHTML, DHTML). For example, files such as a background (<background>), a stick (<stick>) (eg, a joystick), a button (<button>), a location, a size, etc. may be defined in the file as shown in the example below. The processor 130 may output the controller to the second display 120 based on the file defining the UI.

<?xml version="1.0" encoding="UTF-8"?>

<background src=”image/background.png” width=800 height=600 />

<stick>

<stick id=”A” src=”image/Button_A.png” x=80 y=300 width=100 height=100>

</stick>

<button>

<button id=”A” src=”image/Button_A.png” x=320 y=480 width=30 height=30>

<button id=”B” src=”image/Button_B.png” x=360 y=460 width=30 height=30>

<button id=”X” src=”image/Button_X.png” x=320 y=440 width=30 height=30>

<button id=”Y” src=”image/Button_A.png” x=360 y=420 width=30 height=30>

<button id=”select” src=”image/Button_Select.png” x=200 y=440 width=60 height=20>

<button id=”start” src=”image/Button_Start.png” x=200 y=420 width=60 height=20>

</button>

<feature>

<feature id=”chat” src=”image/Button_Chat.png” x=120 y=200 width=50 height=50>

<feature id=”screenshot” src=”image/Button_Screenshot.png” x=180 y=200 width=50 height=50>

<feature id=”record” src=”image/Button_Record.png” x=240 y=200 width=50 height=50>

<feature id=”edit” src=”image/Button_Edit.png” x=300 y=200 width=50 height=50>

</feature>

The user-defined information 457 may indicate whether the corresponding controller is a controller manufactured or modified by the user. Also, the reference controller ID 459 may have a controller ID (controller ID 451) value of the referenced controller when the corresponding controller is modified or manufactured based on another controller.

The database and its fields described with reference to FIG. 4 are exemplary, and other fields may be substituted or fields may be added. For example, since there may be a plurality of controller IDs corresponding to one category, the second database 145 includes additional fields such as priority, rating, preference, usage count, recommendation count, etc. for determining the priority of the controller ID. may include. Also, the first database 143 and the second database 145 may be understood as one database having at least two tables.

5 illustrates a process for determining a controller suitable for an application according to an exemplary embodiment.

Operation 310 is an operation described with reference to FIG. 3 , and the electronic device 100 (eg, the processor 130 may execute an application. When the application is executed, in operation 520 , the processor 130 sets the application ID to the database (eg, the first database 143). ), for example, the processor 130 may determine whether a record having a field including an application ID value of an application executed in the first database 143 exists.

When the application ID is checked in the database, the electronic device 100 (eg, the processor 130) may check whether a controller predefined for the application ID checked in operation 530 exists. For example, the processor 130 may determine whether the user-defined controller ID 441 field in the record specifies a specific controller ID.

If a controller predefined for the identified application ID exists, the electronic device 100 (eg, the processor 130) may determine the predefined controller as a controller for the executed application in operation 540 .

The electronic device 100 (eg, the processor 130 ) may output an execution screen of the application to the first display in operation 550 . 3 , operation 550 may be understood to be performed immediately after operation 310 .

In operation 560, the electronic device 100 (processor 130) may output the determined controller to the second display. For example, the processor 130 may check the configuration file location 455 corresponding to the controller ID value of the controller determined in the second database 145 . The processor 130 may output the controller to the second display 120 based on the UI defined in the file corresponding to the setting file location 455 .

In operation 520, the ID of the application may not exist in the database for reasons such as a relatively recent release date of the application. In this case, the processor 130 may communicate with the server 200 to confirm the category of the application ID or the designated controller. In this regard, operation 700 may be performed. Operation 700 will be described later with reference to FIG. 7 .

If there is no controller defined for the application in operation 530 , the electronic device 100 (eg, the processor 130 ) may perform operation 532 . In operation 532, the processor 130 may determine whether the application supports a pad-type controller. For example, the processor 130 may check the pad controller support information 439 field in the database.

As a result of the check, if the application supports the pad controller, operation 534 for setting the pad controller may be performed, and if not, operation 800 for setting the touch controller may be performed. Operation 800 will be described later with reference to FIG. 8 .

In operation 534, the electronic device 100 (eg, the processor 130) may determine a pad controller corresponding to the category of the application. For example, the processor 130 may determine a category (eg, a category 437 ) corresponding to the application identification information, and determine an appropriate pad controller based on the determined category. An example of a pad controller will be described with reference to FIGS. 6A, 6B, and 6C.

6A, 6B, and 6C show application execution screens and controllers of different categories according to an embodiment.

6A illustrates a pad controller corresponding to an execution screen 610 . The execution screen 610 is a screen for selecting a game, and since the game has not yet started, a general-purpose controller may be output. For example, a controller including a direction button 601, a left key 602, a right key 603, a game selection button 604, an end button 605, and other control buttons 606 may be output.

6B shows a pad controller when a game of the RPG category is selected. When the execution screen 620 of the RPG is output on the first display 110 , the processor 130 may output a controller designated for the executed game or a controller corresponding to a category to which the executed game belongs on the second display 120 . For example, the controller may include a direction key 611, a left key 612, a right key 613, a start button 614, a selection button 615, a function key (jump, acceleration, fire, etc.) 616, and a game setting related key 617.

6C illustrates a pad controller corresponding to an execution screen 630 of a game in the racing category. The controller may include a steering wheel 621, a brake pedal 622, an accelerator pedal 623, a spark plug button 624, a game setting related button 625, and the like.

7 illustrates a process for determining a controller based on communication with a server according to an embodiment.

Referring to FIG. 7 , if the application ID is not identified in the database, the electronic device 100 (eg, the processor 130) transmits unique identification information (eg, application ID 411) of the application to the server 200 through the communication module 150 in operation 710 can

When receiving application identification information from the electronic device 100, the server 200 may determine a predefined controller ID or category information corresponding to the received application identification information with reference to the application meta DB 220 of the server 200 . Alternatively, in another embodiment, the server 200 may directly determine a controller based on the received application identification information, and transmit information on the determined controller to the electronic device 100 . When transmitting the determination result to the electronic device 100 , the server 200 may transmit a record of the application meta DB 220 corresponding to the application identification information and/or a record of the controller meta DB 230 corresponding to the controller ID to the electronic device 100 .

In operation 720, the electronic device 100 (eg, the processor 130) may receive a controller ID or category information corresponding to the application ID from the server 200 in response to the transmission. The processor 130 may determine a controller from the second database 145 based on the received information, for example, using the received controller ID or category. Since the subsequent operations correspond to the processes after operation 530 of FIG. 5 , a description thereof will be omitted.

As another example, the electronic device 100 (eg, the processor 130 ) may receive controller information from the server 200 in response to the transmission in operation 730 . Here, the controller information may include a file defining the UI of the controller. The processor 130 may output the controller to the second display 120 based on the received information. Since the subsequent operations correspond to the processes after operation 550 of FIG. 5 , a description thereof will be omitted.

In the embodiment of FIG. 7 , the electronic device 100 (eg, the processor 130 ) may update the database stored in the memory 140 by using information (eg, a record) provided from the server 200 .

8 illustrates a process for setting a button controller according to an embodiment. FIG. 8 may be performed at a point in time when it is determined that the application does not support the pad controller in the process of FIG. 5 (after performing operation 532), and according to an embodiment, while a game is being played using a touch function, the user It may be performed at the time of requesting creation of a button controller.

In operation 810 , the electronic device 100 (eg, the processor 130 ) may output an execution screen of an application to the first display 110 . Operation 810 may correspond to operation 550 described above.

In operation 820, the electronic device 100 (eg, the processor 130) may recognize the button control area included in the execution screen of the application. Based on the recognition performance, the processor 130 may copy the execution screen to another display. For example, a specific frame among execution screens output on the first display 110 may be copied to the second display 220 . The processor 130 may recognize the button based on the copied screen or directly recognize the button on the execution screen. The processor 130 may recognize an object whose size of the button area corresponds to a size generally used in an application among the objects recognized as a button as a button.

In operation 830 , the electronic device (eg, the processor 130 ) may display the recognized button on the second display 220 . The displayed button may have the characteristics of a basic guide controller. If the button is incorrectly recognized or editing is required, the button controller may be finally edited through operation 840 .

In operation 840, editing of a button, such as adding or deleting a button, may be performed. For example, when a touch and hovering input occurs with respect to a displayed button, the corresponding button may be deleted from the recognition target button. If a touch and hovering input to an area not recognized as a button occurs, the processor 130 may add the corresponding area as a recognized touch (button) and hovering area. In addition, the button position may be adjusted through touch and drag input.

In operation 850, the electronic device 100 (eg, the processor 130) may store a button controller setting. The process of performing button editing of the button controller in operations 840 and 850 may be applied to the pad controller in a similar manner. For example, addition, deletion, location, and size change of a control area for a control item included in the pad controller may be performed.

According to an embodiment, the processor 130 may store information on the recognized buttons in a database for one controller ID. In relation to this, it will be described later with reference to FIG. 11 .

9 illustrates a button controller according to an embodiment.

According to an embodiment, when an application such as the execution screen 910 is executed or a specific user input occurs, the processor 130 may recognize a button included in the execution screen 910 . For example, a button included in the execution screen 910 may be recognized as the button controller 920 . Buttons 1 to 7 on the execution screen 910 may correspond to buttons 1 to 7 on the button controller 920, respectively.

When the user selects the button 2, the button 2 may be deleted from the button controller 920 . Also, when the user selects an unrecognized location on the execution screen 910 , a button corresponding to the location may be added to the button controller 920 .

In addition, since buttons included in the button controller 920 may be densely distributed in a specific area for quick input, the full screen 900 includes the first spare area 921 and the second spare area in addition to the execution screen 910 and the button controller 920. 922 may be included. An appropriate button or menu may be added to the first spare area 921 or the second spare area 922 . Also, according to an embodiment, the full screen 900 may include an execution screen 910 and a button controller 920 without a spare area.

10 illustrates a process for changing a controller according to an embodiment.

10 may be applied to a button constituting the aforementioned pad controller or a button constituting a button controller.

When the editing process of the controller starts, in operation 1010 , the electronic device 100 (eg, the processor 130 ) may change the controller type or type. For example, when a racing game is executed and a controller as shown in FIG. 6A is provided, the user may change to the controller shown in FIG. 6C or change to a button controller.

In operation 1020, the electronic device 100 (eg, the processor 130) may change the position or size of a button or control object included in the controller. Also, in operation 1030 , the electronic device 100 (eg, the processor 130 ) may change the color or theme of the controller. For example, the processor 130 may change the color of some buttons or objects of the controller, or change the theme of the controller (eg, change from a PlayStation theme to a Nintendo theme, etc.).

If it is determined that there is a change in operations 1010 , 1020 , and 1030 , the electronic device 100 (eg, the processor 130 ) may temporarily store the change in operation 1001 . When an input for storing a change is generated in operation 1050, the processor 130 may end the change process of the controller. The processor 130 may store information on the changed controller by, for example, assigning a new controller ID 451 to the second database 145 . When the original controller is partially changed, the controller ID information of the original controller may be stored in the reference controller ID 459. In addition, the processor 130 may generate a markup document file corresponding to the changed UI, and store the path of the generated file in the setting file location 455 .

11 illustrates mapping of buttons and controllers constituting a button controller according to an embodiment.

Referring to FIG. 11 , the second database 145 may include a list of controllers. Each record included in the list may be identified by a controller ID. When the controller ID corresponds to the button controller, a record corresponding to each of at least one button constituting the button controller may correspond to the controller ID.

For example, when controller ID #2 1110 is a button controller including five buttons, controller ID #2 may correspond to information 1120 corresponding to each button in a one-to-many relationship. Each of the information 1120 may include an identification field 1121 of the button, a controller ID field 1122, a coordinate (center coordinate) field 1123 of the button, a path field 1124 of the button image when the button is an image, and other flag fields 1125. Here, all controller ID fields 1122 may have values corresponding to controller ID #2.

12 and 13 show examples of usable controllers.

12 illustrates a controller adjustable by a user according to an exemplary embodiment. A screen 1210 shown on the left side of FIG. 12 may correspond to a screen on which a DEFAULT controller is displayed. The user can pause the execution of the application and edit the controller when the location of the objects constituting the controller is inconvenient or there is an unused object while using the game using the default controller. For example, the process of FIG. 10 may be performed.

For example, in the controller of screen 1210, if button AB is not used and the positions of buttons A, B, C, D, and CD are inconvenient to use, the user removes button AB, and buttons A, B, C, After adjusting the position of D and CD as shown in screen 1220, you can use the game again.

13 illustrates an example of controlling a cursor with a controller according to an exemplary embodiment. In FIG. 13 , the first spare area 1321 and the second spare area 1322 correspond to the contents described in FIG. 9 , and thus descriptions thereof will be omitted.

When a cursor is displayed on the screen 1310, a controller 1320 including a touchpad 1332 may be provided. A predefined function (eg, select, delete, exit, etc.) may be assigned to the buttons 1331 designated as 1, 2, 3, and 4, respectively. In addition, the user may control the cursor 1301 by providing a gesture input such as a touch or drag to the touch pad associated with the cursor 1301 . For example, when a user input is generated through the touchpad 1332, the processor 130 maps the user input and the cursor 1301 to move the cursor 1301 in a manner corresponding to the user input or select an object on which the cursor 1301 is located. can be done Alternatively, a specified operation/function may be performed in response to a gesture input input through the touch pad 1332, for example, a touch input such as a swipe or a double tap.

According to various embodiments, an electronic device for providing or performing the method of providing the above-described controller includes at least one display, an application, and a memory for storing a database including information on a plurality of controllers, and a processor and, the processor displays content including at least one object in connection with the execution of the application on a first area of the display, determines a controller corresponding to the application based on the database, and determines the controller may be output to the second area of the display.

In addition, the display may include a first display and a second display, wherein the first area corresponds to the first display, and the second area corresponds to the second display.

According to an embodiment, the processor determines a property corresponding to the application, and when the property is a first property, a first controller for moving the at least one object through the second area of the display and, if the attribute is the second attribute, a second controller for moving the at least one object may be displayed through the second area.

Also, according to an embodiment, when information on a controller corresponding to the unique identification information of the application exists in the database, the processor may output the controller to the second area based on the information on the controller. . Here, the information on the controller may include information on a file defining a user interface (UI) of the controller, and the processor may output the UI of the controller defined in the file to the second area.

Also, when there is no controller designated for the unique identification information of the application in the database, the processor may determine the controller based on attribute information of the application.

In addition, the processor determines whether the application supports a pad-type controller when there is no controller designated for the unique identification information of the application in the database, and selects a pad-type controller or a button-type controller according to the determination result may be output to the second area.

According to an embodiment, the electronic device further includes a communication module, and when the application is not identified in the database, the electronic device transmits unique identification information of the application to the server through the communication module, and the transmission In response, a controller determined based on the controller ID received from the server or category information of the application may be output to the second area.

Also, the processor may output the controller to the second area based on controller information received from the server in response to the transmission.

Also, the processor may update the database based on information received from the server.

According to an embodiment, the processor may determine a controller having the highest priority among a plurality of controllers corresponding to the category information as the controller.

In addition, when a user input for changing the UI of the controller output to the second area is received, the processor changes at least one of the number, size, location, and color of control objects included in the controller based on the user input. can

According to an embodiment, when a user input for changing the UI of the controller output to the second area is received, the processor may change the type or theme of the controller based on the user input.

Also, the processor may recognize a button included in the execution screen of the application, and output the button-type controller including at least one button in the second area based on the recognized button. Here, when the recognized button is selected, the processor may delete a button corresponding to the selected button from among the at least one button included in the button-type controller. Also, when an area in which a button is not recognized is selected, the processor may add a button corresponding to the selected area to the button-type controller.

The above-described method for providing a controller using the various electronic devices includes an operation of executing an application, and an operation of displaying content including at least one object in relation to the execution of the application on a first area of a display functionally connected to the electronic device , determining a controller for moving the at least one object based on a database, and outputting the determined controller to a second area of the display.

Here, outputting the determined controller to the second area may include: when the property is the first property, displaying a first controller for moving at least one object through the second area of the display; and if the property is the second property, displaying a second controller for moving the at least one object through the second area.

The electronic device (eg, the electronic device 100 ) according to an embodiment may include a memory for storing a database including information on at least one display, an application, and a plurality of controller sets, and a processor, and a processor. have. The processor (eg, processor 130) displays the plurality of controller sets with respect to content displayed on the display through the application, selects any one controller set among the plurality of controller sets, and The selected controller set may be displayed in association with the content.

Also, according to an embodiment, a method of displaying a set of controllers in association with content includes: displaying a plurality of sets of controllers for content displayed on a display functionally connected to an electronic device through an application; selecting one of the plurality of controller sets; and displaying the selected controller set in association with the content. The at least one display may include a first display and a second display, the content may be displayed on the first display, and the selected controller set may be displayed on the second display.

The determining of the controller corresponding to the application includes: providing identification information of the application to a server; receiving controller information or category information from the server in response to the transmission; and based on the received information and determining the controller.

In a recording medium storing computer readable instructions according to an embodiment, the instructions are executable by at least one processor, and when executed, cause an electronic device to execute an application, in relation to the execution of the application Displaying content including at least one object on a first area of a display functionally connected to the electronic device, determining a controller for moving the at least one object based on a database, and using the determined controller An operation of outputting to the second area of the display may be performed.

14 illustrates an electronic device in a network environment according to various embodiments of the present disclosure.

Referring to FIG. 14 , electronic devices 1401 , 1402 , 1404 or a server 1406 according to various embodiments may be connected to each other through a network 1462 or short-range communication 1464 . The electronic device 1401 may include a bus 1410, a processor 1420, a memory 1430, an input/output interface 1450, a display 1460, and a communication interface 1470. In some embodiments, the electronic device 1401 may omit at least one of the components or may additionally include other components.

The bus 1410 may include, for example, a circuit that connects the components 1410-1470 to each other and transmits communication (eg, a control message and/or data) between the components.

The processor 1420 may include one or more of a central processing unit (CPU), an application processor (AP), and a communication processor (CP). The processor 1420 may, for example, execute an operation or data processing related to control and/or communication of at least one other component of the electronic device 1401 .

The memory 1430 may include volatile and/or non-volatile memory. The memory 1430 may store, for example, commands or data related to at least one other component of the electronic device 1401 . According to an embodiment, the memory 1430 may store software and/or a program 1440 . The program 1440 may include, for example, a kernel 1441, middleware 1443, an application programming interface (API) 1445, and/or an application program (or “application”) 1447, and the like. At least a portion of the kernel 1441, middleware 1443, or API 1445 may be referred to as an operating system (OS).

Kernel 1441 is, for example, system resources (eg, bus 1410, processor 1420, or memory 1430, etc.) can be controlled or managed. Also, the kernel 1441 may provide an interface capable of controlling or managing system resources by accessing individual components of the electronic device 1401 in the middleware 1443, API 1445, or application program 1447.

Middleware 1443 may, for example, perform an intermediary role so that API 1445 or application program 1447 communicates with kernel 1441 to exchange data.

Also, the middleware 1443 may process one or more work requests received from the application program 1447 according to priority. For example, the middleware 1443 may give priority to at least one of the application programs 1447 to use a system resource (eg, bus 1410, processor 1420, memory 1430, etc.) of the electronic device 1401 . For example, the middleware 1443 may process the one or more work requests according to the priority given to the at least one, thereby performing scheduling or load balancing for the one or more work requests.

API 1445 is, for example, an interface for the application 1447 to control functions provided by the kernel 1441 or middleware 1443, for example, at least one interface for file control, window control, image processing, or character control. Alternatively, it can contain functions (eg, commands).

The input/output interface 1450 may serve as an interface through which, for example, a command or data input from a user or other external device may be transmitted to other component(s) of the electronic device 1401 . Also, the input/output interface 1450 may output a command or data received from other component(s) of the electronic device 1401 to a user or other external device.

The display 1460 may be, for example, a Liquid Crystal Display (LCD), a Light-Emitting Diode (LED) display, an Organic LED (OLED) display, or a microelectromechanical system ( microelectromechanical systems, MEMS) displays, or electronic paper displays. The display 1460 may display, for example, various contents (eg, text, image, video, icon, or symbol) to the user. The display 1460 may include a touch screen, and may receive, for example, a touch input using an electronic pen or a part of the user's body, a gesture, a proximity, or a hovering input.

The communication interface 1470 may establish, for example, communication between the electronic device 1401 and an external device (eg, the first external electronic device 1402 , the second external electronic device 1404 , or the server 1406 ). For example, the communication interface 1470 may be connected to the network 1462 through wireless communication or wired communication to communicate with the external device (eg, the second external electronic device 1404 or the server 1406 ).

Wireless communication is, for example, a cellular communication protocol, for example, LTE (Long-Term Evolution), LTE-A (LTE-Advanced), CDMA (Code Division Multiple Access), WCDMA (Wideband CDMA), UMTS (Universal Mobile) Telecommunications System), at least one of Wireless Broadband (WiBro), or Global System for Mobile Communications (GSM) may be used. Also, the wireless communication may include, for example, short-range communication 1464. The short-range communication 1464 may include, for example, at least one of Wireless Fidelity (Wi-Fi), Bluetooth, Near Field Communication (NFC), magnetic stripe transmission (MST), and GNSS.

The MST may use an electromagnetic signal to generate a pulse according to transmitted data, and the pulse may generate a magnetic field signal. The electronic device 1401 transmits the magnetic field signal to a point of sales (POS), the POS detects the magnetic field signal using an MST reader, and restores the data by converting the detected magnetic field signal into an electrical signal can do.

GNSS is, depending on the region or bandwidth used, for example, GPS (Global Positioning System), Glonass (Global Navigation Satellite System), Beidou Navigation Satellite System (hereinafter "Beidou") or Galileo (the European global satellite-based navigation system) may include at least one of Hereinafter, in this document, "GPS" may be used interchangeably with "GNSS". Wired communication may include, for example, at least one of universal serial bus (USB), high definition multimedia interface (HDMI), recommended standard 1532 (RS-232), or plain old telephone service (POTS). The network 1462 may include at least one of a telecommunications network, for example, a computer network (eg, LAN or WAN), the Internet, or a telephone network.

Each of the first and second external electronic devices 1402 and 1404 may be the same as or different from the electronic device 1401 . According to one embodiment, the server 1406 may include a group of one or more servers. According to various embodiments, all or a part of operations executed by the electronic device 1401 may be executed by one or a plurality of other electronic devices (eg, the electronic device 1402 , 1404 , or the server 1406 ). According to an embodiment, when the electronic device 1401 needs to automatically or request a function or service, the electronic device 1401 performs at least some functions related thereto instead of or in addition to executing the function or service itself. may be requested from another device (eg, the electronic device 1402, 1404, or the server 1406). Another electronic device (eg, the electronic device 1402 , 1404 , or the server 1406 ) may execute the requested function or additional function, and may transmit the result to the electronic device 1401 . The electronic device 1401 may provide a requested function or service by processing the received result as it is or additionally. For this purpose, for example, cloud computing, distributed computing, or client-server computing technology may be used.

15 is a block diagram 1500 of an electronic device 1501 according to various embodiments of the present disclosure.

Referring to FIG. 15 , an electronic device 1501 may include, for example, all or a part of the electronic device 1401 illustrated in FIG. 14 . Electronic device 1501 includes one or more processors (eg AP) 1510, communication module 1520, subscriber identity module 1524, memory 1530, sensor module 1540, input device 1550, display 1560, interface 1570, audio module 1580, camera module 1591, power management a module 1595 , a battery 1596 , an indicator 1597 , and a motor 1598 .

The processor 1510 may control a plurality of hardware or software components connected to the processor 1510 by, for example, driving an operating system or an application program, and may perform various data processing and operations. The processor 1510 may be implemented as, for example, a system on chip (SoC). According to an embodiment, the processor 1510 may further include a graphic processing unit (GPU) and/or an image signal processor. The processor 1510 may include at least some of the components shown in FIG. 15 (eg, a cellular module 1521). The processor 1510 may load and process commands or data received from at least one of other components (eg, non-volatile memory) into the volatile memory, and store various data in the non-volatile memory.

The communication module 1520 may have the same or similar configuration to the communication interface 1470 of FIG. 14 . The communication module 1520 is, for example, a cellular module 1521, a Wi-Fi module 1522, a Bluetooth module 1523, a GNSS module 1524 (eg, a GPS module, a Glonass module, a Beidou module, or a Galileo module), an NFC module 1525, an MST module 1526 , and a radio frequency (RF) module 1527 .

The cellular module 1521 may provide, for example, a voice call, a video call, a text service, or an Internet service through a communication network. According to an embodiment, the cellular module 1521 may use a subscriber identification module (eg, a SIM card) 1529 to identify and authenticate the electronic device 1501 within a communication network. According to an embodiment, the cellular module 1521 may perform at least some of the functions that the processor 1510 may provide. According to an embodiment, the cellular module 1521 may include a communication processor (CP).

Each of the Wi-Fi module 1522, the Bluetooth module 1523, the GNSS module 1524, the NFC module 1525, or the MST module 1526 may include, for example, a processor for processing data transmitted and received through a corresponding module. According to some embodiments, at least some (eg, two or more) of the cellular module 1521, the Wi-Fi module 1522, the Bluetooth module 1523, the GNSS module 1524, the NFC module 1525, or the MST module 1526 is one integrated chip (IC) Alternatively, it may be included in an IC package.

The RF module 1527 may transmit and receive, for example, a communication signal (eg, an RF signal). The RF module 1527 may include, for example, a transceiver, a power amp module (PAM), a frequency filter, a low noise amplifier (LNA), or an antenna. According to another embodiment, at least one of the cellular module 1521, the Wi-Fi module 1522, the Bluetooth module 1523, the GNSS module 1524, the NFC module 1525, and the MST module 1526 may transmit/receive an RF signal through a separate RF module.

The subscriber identification module 1529 may include, for example, a card including a subscriber identification module and/or an embedded SIM (SIM), and may include unique identification information (eg, integrated circuit card identifier (ICCID)) or subscriber information. (eg, international mobile subscriber identity (IMSI)).

The memory 1530 (eg, the memory 1430) may include, for example, an internal memory 1532 or an external memory 1534. The internal memory 1532 may include, for example, a volatile memory (eg, dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM), etc.), non-volatile memory (eg, OTPROM ( one time programmable ROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g. NAND flash) flash) or NOR flash (NOR flash, etc.), a hard drive, or a solid state drive (SSD).

External memory 1534 is a flash drive, for example, CF (compact flash), SD (secure digital), Micro-SD, Mini-SD, xD (extreme digital), MMC (MultiMediaCard), or a memory stick ( memory stick) and the like may be further included. The external memory 1534 may be functionally and/or physically connected to the electronic device 1501 through various interfaces.

The security module 1536 is a module including a storage space having a relatively higher security level than that of the memory 1530, and may be a circuit that ensures safe data storage and a protected execution environment. The security module 1536 may be implemented as a separate circuit and may include a separate processor. The security module 1536 may include, for example, an embedded secure element (eSE) present in a removable smart chip, a secure digital (SD) card, or embedded in a fixed chip of the electronic device 1501 . . Also, the security module 1536 may be driven by an operating system different from the operating system (OS) of the electronic device 201 . For example, the security module 1536 may operate based on a java card open platform (JCOP) operating system.

The sensor module 1540, for example, may measure a physical quantity or sense an operating state of the electronic device 1501, and may convert the measured or sensed information into an electrical signal. The sensor module 1540 may include, for example, a gesture sensor 1540A, a gyro sensor 1540B, a barometric pressure sensor 1540C, a magnetic sensor 1540D, an acceleration sensor 1540E, a grip sensor 1540F, a proximity sensor 1540G, a color sensor 1540H (eg, an RGB sensor), a biometric sensor 1540I. , a temperature/humidity sensor 1540J, an illuminance sensor 1540K, or an ultra violet (UV) sensor 1540M. Additionally or alternatively, the sensor module 1540 may include, for example, an olfactory sensor (E-nose sensor), an electromyography (EMG) sensor, an electroencephalogram (EEG) sensor, an electrocardiogram (ECG) sensor, an infrared (IR) sensor, an iris sensor, and / or may include a fingerprint sensor. The sensor module 1540 may further include a control circuit for controlling at least one or more sensors included therein. In some embodiments, the electronic device 1501 may further include a processor configured to control the sensor module 1540 as part of the processor 1510 or separately to control the sensor module 1540 while the processor 1510 is in a sleep state. .

The input device 1550 may include, for example, a touch panel 1552, a (digital) pen sensor 1554, a key 1556, or an ultrasonic input device 1558. The touch panel 1552 may use, for example, at least one of a capacitive type, a pressure sensitive type, an infrared type, and an ultrasonic type. Also, the touch panel 1552 may further include a control circuit. The touch panel 1552 may further include a tactile layer to provide a tactile response to the user.

The (digital) pen sensor 1554 may be, for example, a part of a touch panel or may include a separate recognition sheet. Key 1556 may include, for example, a physical button, an optical key, or a keypad. The ultrasound input device 1558 may detect an ultrasound generated by an input tool through a microphone (eg, the microphone 1588 ), and may check data corresponding to the detected ultrasound.

The display 1560 (eg, the display 1460 ) may include a panel 1562 , a hologram device 1564 , or a projector 1566 . The panel 1562 may have the same or similar configuration to the display 1460 of FIG. 14 . The panel 1562 may be implemented to be flexible, transparent, or wearable, for example. The panel 1562 may be configured as a single module with the touch panel 1552. The hologram device 1564 may display a stereoscopic image in the air by using light interference. The projector 1566 may display an image by projecting light onto the screen. The screen may be located inside or outside the electronic device 1501, for example. According to an embodiment, the display 1560 may further include a control circuit for controlling the panel 1562, the hologram device 1564, or the projector 1566.

The interface 1570 may include, for example, HDMI 1572, USB 1574, optical interface 1576, or D-sub (D-subminiature) 1578. The interface 1570 may be included in, for example, the communication interface 1470 illustrated in FIG. 14 . Additionally or alternatively, the interface 1570 may include, for example, a mobile high-definition link (MHL) interface, an SD card/MMC interface, or an infrared data association (IrDA) standard interface.

The audio module 1580 may interactively convert a sound and an electrical signal, for example. At least some components of the audio module 1580 may be included in, for example, the input/output interface 1450 illustrated in FIG. 14 . The audio module 1580 may process sound information input or output through, for example, a speaker 1582, a receiver 1584, an earphone 1586, a microphone 1588, or the like.

The camera module 1591 is, for example, a device capable of capturing still images and moving images, and according to an embodiment, one or more image sensors (eg, a front sensor or a rear sensor), a lens, an image signal processor (ISP), or It may include a flash (eg, an LED or a xenon lamp).

The power management module 1595 may manage power of the electronic device 1501, for example. According to an embodiment, the power management module 1595 may include a power management integrated circuit (PMIC), a charger integrated circuit (IC), or a battery or fuel gauge. The PMIC may have a wired and/or wireless charging method. The wireless charging method includes, for example, a magnetic resonance method, a magnetic induction method or an electromagnetic wave method, and may further include an additional circuit for wireless charging, for example, a coil loop, a resonance circuit, or a rectifier. have. The battery gauge may measure, for example, the remaining amount of the battery 1596, voltage, current, or temperature during charging. The battery 1596 may include, for example, a rechargeable battery and/or a solar battery.

The indicator 1597 may display a specific state of the electronic device 1501 or a part thereof (eg, the processor 1510 ), for example, a booting state, a message state, or a charging state. The motor 1598 may convert an electrical signal into mechanical vibration, and may generate vibration or a haptic effect. Although not shown, the electronic device 1501 may include a processing unit (eg, GPU) for supporting mobile TV. The processing device for supporting mobile TV may process media data according to a standard such as Digital Multimedia Broadcasting (DMB), Digital Video Broadcasting (DVB), or MediaFLOTM.

Each of the components described in this document may be composed of one or more components, and the name of the component may vary depending on the type of the electronic device. In various embodiments, the electronic device may be configured to include at least one of the components described in this document, and some components may be omitted or may further include additional other components. In addition, since some of the components of the electronic device according to various embodiments are combined to form a single entity, the functions of the components prior to being combined may be identically performed.

16 is a block diagram of a program module according to various embodiments of the present disclosure.

According to an embodiment, the program module 1610 (eg, program 1440) may include an operating system (OS) that controls resources related to an electronic device (eg, electronic device 1401) and/or various applications (eg, applications) running on the operating system. program 1447). The operating system may be, for example, Android, iOS, Windows, Symbian, Tizen, or bada.

The program module 1610 may include a kernel 1620, middleware 1630, API 1660, and/or an application 1670. At least a portion of the program module 1610 may be preloaded on the electronic device or downloaded from an external electronic device (eg, electronic devices 1402, 1404, server 1406, etc.).

The kernel 1620 (eg, kernel 1441) may include, for example, a system resource manager 1621 or a device driver 1623. The system resource manager 1621 may control, allocate, or recover system resources. According to an embodiment, the system resource manager 1621 may include a process manager, a memory manager, or a file system manager. The device driver 1623 may include, for example, a display driver, a camera driver, a Bluetooth driver, a shared memory driver, a USB driver, a keypad driver, a Wi-Fi driver, an audio driver, or an inter-process communication (IPC) driver. .

For example, the middleware 1630 may provide functions commonly required by the application 1670 or provide various functions to the application 1670 through API 1660 so that the application 1670 can efficiently use limited system resources inside the electronic device. have. According to an embodiment, the middleware 1630 (eg, middleware 1443) includes a runtime library 1635, an application manager 1641, a window manager 1642, a multimedia manager 1643, and a resource manager 1644. , power manager 1645, database manager 1646, package manager 1647, connectivity manager 1648, notification manager 1649, location manager 1650, graphic It may include at least one of a graphic manager 1651, a security manager 1652, and a payment manager 1654.

The runtime library 1635 may include, for example, a library module used by the compiler to add a new function through a programming language while the application 1670 is being executed. The runtime library 1635 may perform I/O management, memory management, or functions for arithmetic functions.

The application manager 1641 may manage a life cycle of at least one application among the applications 1670, for example. The window manager 1642 can manage GUI resources used in the screen. The multimedia manager 1643 may identify a format required for reproduction of various media files, and may encode or decode the media file by using a codec suitable for the format. The resource manager 1644 may manage resources such as a source code, a memory, or a storage space of at least one of the applications 1670 .

The power manager 1645 may, for example, operate together with a basic input/output system (BIOS) to manage a battery or power, and provide power information required for an operation of an electronic device. The database manager 1646 may create, search, or change a database to be used by at least one of the applications 1670 . The package manager 1647 may manage installation or update of applications distributed in the form of package files.

The connection manager 1648 may manage a wireless connection such as Wi-Fi or Bluetooth, for example. The notification manager 1649 may display or notify an event such as an arrival message, an appointment, or a proximity notification in a manner that does not disturb the user. The location manager 1650 may manage location information of the electronic device. The graphic manager 1651 may manage a graphic effect to be provided to a user or a user interface related thereto. The security manager 1652 may provide various security functions necessary for system security or user authentication. According to an embodiment, when the electronic device (eg, the electronic device 1401 ) includes a phone function, the middleware 1630 may further include a telephony manager for managing the voice or video call function of the electronic device.

The middleware 1630 may include a middleware module that forms a combination of various functions of the aforementioned components. The middleware 1630 may provide a module specialized for each type of operating system in order to provide differentiated functions. Also, the middleware 1630 may dynamically delete some existing components or add new components.

API 1660 (eg, API 1445) is, for example, a set of API programming functions, and may be provided in a different configuration according to an operating system. For example, in the case of Android or iOS, one API set may be provided for each platform, and in the case of Tizen, two or more API sets may be provided for each platform.

Application 1670 (eg application program 1447) can be, for example, Home 1671, Dialer 1672, SMS/MMS 1673, Instant Message (IM) 1674, Browser 1675, Camera 1676, Alarm 1677, Contact 1678, Voice Dialer 1679, Email 1680, calendar 1681, media player 1682, album 1683, or clock 1684, providing health care (eg, measuring exercise or blood sugar), or providing environmental information (eg, barometric pressure, humidity, or temperature information, etc.) ) may include one or more applications that can perform functions such as.

According to an embodiment, the application 1670 is an application that supports information exchange between an electronic device (eg, electronic device 1401 ) and external electronic devices (eg, electronic devices 1402 and 1404 ) (hereinafter, for convenience of description, “information exchange application”) ") may be included. The information exchange application may include, for example, a notification relay application for transmitting specific information to an external electronic device or a device management application for managing the external electronic device.

For example, the notification delivery application transmits notification information generated from another application (eg, SMS/MMS application, email application, health management application, or environment information application, etc.) of the electronic device to an external electronic device (eg, electronic device 1402, 1404). Also, the notification delivery application may receive notification information from, for example, an external electronic device and provide it to the user.

The device management application is, for example, turn-on/turn on at least one function (eg, the external electronic device itself (or some component parts) of the external electronic device (eg, electronic devices 1402 and 1404) that communicates with the electronic device) -Off or adjusting the brightness (or resolution) of the display), applications running on the external electronic device, or services provided by the external electronic device (eg, call service or message service, etc.) management (eg installation, deletion, or update) can do.

According to an embodiment, the application 1670 may include an application (eg, a health management application of a mobile medical device) designated according to a property of an external electronic device (eg, the electronic devices 1402 and 1404 ). According to an embodiment, the application 1670 may include an application received from an external electronic device (eg, the server 1406 or the electronic devices 1402 or 1404). According to an embodiment, the application 1670 may include a preloaded application or a third party application downloadable from a server. Names of components of the program module 1610 according to the illustrated embodiment may vary depending on the type of the operating system.

According to various embodiments, at least a part of the program module 1610 may be implemented in software, firmware, hardware, or a combination of at least two or more thereof. At least a portion of the program module 1610 may be implemented (eg, executed) by, for example, a processor (eg, the processor 1510 ). At least a portion of the program module 1610 may include, for example, a module, a program, a routine, sets of instructions, or a process for performing one or more functions.

As used herein, the term “module” may refer to, for example, a unit including one or a combination of two or more of hardware, software, and firmware. The term “module” may be used interchangeably with terms such as, for example, unit, logic, logical block, component, or circuit. A “module” may be a minimum unit or a part of an integrally configured component. A “module” may be a minimum unit or a part of performing one or more functions. A “module” may be implemented mechanically or electronically. For example, a “module” may be one of an application-specific integrated circuit (ASIC) chip, field-programmable gate arrays (FPGAs) or programmable-logic device, known or to be developed, that performs certain operations. It may include at least one.

At least a portion of an apparatus (eg, modules or functions thereof) or a method (eg, operations) according to various embodiments is, for example, a computer-readable storage medium in the form of a program module It can be implemented as a command stored in . When the instruction is executed by a processor (eg, processor 1420 ), the one or more processors may perform a function corresponding to the instruction. The computer-readable storage medium may be, for example, the memory 1430.

Computer-readable recording media include hard disks, floppy disks, magnetic media (eg, magnetic tape), optical media (eg, CD-ROM, DVD (Digital Versatile Disc), magnetic- It may include a magneto-optical media (eg, a floptical disk), a hardware device (eg, ROM, RAM, or flash memory, etc.), etc. In addition, the program instructions are created by a compiler It may include high-level language code that can be executed by a computer using an interpreter, etc., as well as machine language code such as an interpreter, etc. The above-described hardware device may be configured to operate as one or more software modules to perform the operations of various embodiments. and vice versa.

A module or a program module according to various embodiments may include at least one or more of the above-described components, some may be omitted, or may further include additional other components. Operations performed by a module, a program module, or other components according to various embodiments may be executed sequentially, in parallel, iteratively, or in a heuristic method. Also, some operations may be executed in a different order, omitted, or other operations may be added.

And, the embodiments disclosed in this document are provided for description and understanding of the disclosed and technical content, and do not limit the scope of the present invention. Accordingly, the scope of this document should be construed to include all modifications or various other embodiments based on the technical spirit of the present invention.

Claims (20)

In an electronic device,
at least one display;
a memory for storing a database including information about an application, and a plurality of controllers; and
including a processor;
The processor is
Displaying content including at least one object in relation to the execution of the application on the first area of the display;
Determining a property corresponding to the application,
determining a controller corresponding to the application based on the database and the attribute;
If the attribute is the first attribute, determining a first controller for moving the at least one object as the controller;
when the property is the second property, determining a second controller for moving the at least one object as the controller;
The electronic device is configured to output the determined controller to a second area of the display. The method according to claim 1,
The display includes a first display and a second display,
The first area corresponds to the first display, and the second area is set to correspond to the second display. delete ◈Claim 4 was abandoned when paying the registration fee.◈ The method according to claim 1,
The processor is configured to output the controller to the second area based on the information on the controller when information on the controller corresponding to the unique identification information of the application exists in the database. ◈Claim 5 was abandoned when paying the registration fee.◈ 5. The method according to claim 4,
The information about the controller includes information about a file defining a UI (User Interface) of the controller,
and the processor is configured to output the UI of the controller defined in the file to the second area. The method according to claim 1,
The processor is configured to determine the controller based on attribute information of the application when there is no controller designated for the unique identification information of the application in the database. The method according to claim 1,
The processor determines whether the application supports a pad-type controller when there is no controller designated for the unique identification information of the application in the database, and selects a pad-type controller or a button-type controller according to the determination result An electronic device configured to output to the second area. The method according to claim 1,
Further comprising a communication module,
When the application is not identified in the database, the processor transmits unique identification information of the application to the server through the communication module, and receives a controller ID or category information of the application received from the server in response to the transmission The electronic device is configured to output a controller determined based on . ◈Claim 9 was abandoned at the time of payment of the registration fee.◈ 9. The method of claim 8,
The processor is configured to output the controller to a second area based on controller information received from the server in response to the transmission. ◈Claim 10 was abandoned when paying the registration fee.◈ 9. The method of claim 8,
and the processor is configured to update the database based on information received from the server. ◈Claim 11 was abandoned when paying the registration fee.◈ 7. The method of claim 6,
The processor is configured to determine, as the controller, a controller having a highest priority among a plurality of controllers corresponding to the attribute information. The method according to claim 1,
When a user input for changing the UI of the controller output to the second area is received, the processor changes at least one of the number, size, location, and color of control objects included in the controller based on the user input to be established, the electronic device. The method according to claim 1,
The processor is configured to change the type or theme of the controller based on the user input when a user input for changing the UI of the controller output to the second area is received. 8. The method of claim 7,
The processor is configured to recognize a button included in the execution screen of the application, and output the button-type controller including at least one button to the second area based on the recognized button. ◈Claim 15 was abandoned when paying the registration fee.◈ 15. The method of claim 14,
The processor is configured to delete a button corresponding to the selected button from among the at least one button included in the button-type controller when the recognized button is selected. ◈Claim 16 was abandoned when paying the registration fee.◈ 15. The method of claim 14,
and the processor is configured to add a button corresponding to the selected area to the button-type controller when an area in which a button is not recognized is selected. A method for providing a controller for an electronic device, the method comprising:
the action of running the application;
displaying content including at least one object in relation to the execution of the application on a first area of a display functionally connected to the electronic device;
determining a property corresponding to the application;
determining a controller corresponding to the application based on a database and the attribute;
determining, as the controller, a first controller for moving the at least one object when the attribute is the first attribute;
determining, as the controller, a second controller for moving the at least one object when the attribute is the second attribute; and
and outputting the determined controller to a second area of the display. delete ◈Claim 19 was abandoned at the time of payment of the registration fee.◈ 18. The method of claim 17,
The operation of determining the controller corresponding to the application is,
The operation of transmitting the identification information of the application to the server,
receiving controller information or category information from the server in response to the transmission; and
determining the controller based on the received information. In a recording medium storing computer readable instructions, the instructions are executable by at least one processor, and when executed, cause an electronic device,
the action of running the application;
displaying content including at least one object in relation to the execution of the application on a first area of a display functionally connected to the electronic device;
determining a property corresponding to the application;
determining a controller corresponding to the application based on a database and the attribute;
determining, as the controller, a first controller for moving the at least one object when the attribute is the first attribute;
determining, as the controller, a second controller for moving the at least one object when the attribute is the second attribute; and
and outputting the determined controller to a second area of the display.

Images