KR20200080818A - Method for outputting screen and display device for executing the same - Google Patents

Abstract

Provided are a screen output method and a display device for performing the same. According to one embodiment of the present invention, the display device comprises: a sensing part which detects driving of an application or a connection with a mobile device on which the application is driven; and an input/output part which separates a component area disposed inside an application screen related to a currently driven application from an application screen to dispose the same outside the application screen, and outputs an expanded screen including the application screen from which the component area is removed and the component area disposed outside the application screen.

Description

Screen output method and display device for performing the same{METHOD FOR OUTPUTTING SCREEN AND DISPLAY DEVICE FOR EXECUTING THE SAME}

The disclosed embodiments relate to technology for outputting an application screen.

Recently, as the spread of mobile devices such as smartphones and tablet PCs has accelerated, the use of online game services using the mobile devices has also increased. However, in the case of a mobile device such as a smartphone, there is a inconvenience in that a user can play an online game due to the limitation of the screen size. In particular, in the case of online games that require a lot of input, this inconvenience becomes even greater. In addition, the more component areas such as a menu window, a chat window, and a skill window in the game screen, the more difficult it is for the user to focus on various situations and functions of the online game.

Korean Patent Publication No. 10-2017-0103378 (2017.09.13)

According to one embodiment, the sensing unit for sensing the driving of the application or a connection with the mobile device on which the application is driven; And a component area disposed inside the application screen related to the currently running application is separated from the application screen and placed outside the application screen. The application screen from which the component area is removed and the component area disposed outside are A display device including an input/output unit for outputting an included extended screen is provided.

The input/output unit may configure a second screen layout corresponding to the extended screen from a first screen layout corresponding to an application screen in which the component area is not removed.

The first screen layout may be provided on a set first layer panel, and the second screen layout may be provided on a second layer panel positioned on top of the first layer panel.

The input/output unit receives one or more of the plurality of component areas disposed inside the application screen from the user, receives one or more of the placement positions and sizes of each of the selected component areas from the user, and the second screen layout Can be configured.

The arrangement position of each component area is preset for each component area, and the input/output unit receives an arrangement position of the component area by a user's touch command, and an input placement position of the component area is preset. When deviating from the arrangement position, the arrangement position of the inputted component area may be corrected to the preset placement position.

The input/output unit may detect the input of the user in a component area disposed outside the application screen on the extended screen.

The display device may further include an interface unit that delivers location information of a point in the component area where the user's input is sensed to the mobile device.

The display device may be a touch beam projector that forms a screen on the outside by projecting a laser beam and senses a touch input on the screen through a sensor provided therein.

The display device may be a display panel equipped with a touch sensor.

According to another embodiment, a screen output method performed on a display device, comprising: detecting an operation of an application or a connection with a mobile device on which the application is driven; Separating a component area disposed inside an application screen related to the currently running application from the application screen and placing the component area outside the application screen; And outputting an extended screen including the application screen in which the component area is removed and the component area disposed outside the screen.

The process of separating the component area from the application screen and arranging it outside the application screen comprises configuring a second screen layout corresponding to the extended screen from a first screen layout corresponding to an application screen in which the component area is not removed. can do.

The first screen layout may be provided on a set first layer panel, and the second screen layout may be provided on a second layer panel positioned on top of the first layer panel.

In the process of separating the component area from the application screen and placing it outside the application screen, at least one of a plurality of component areas disposed inside the application screen is selected from the user, and each of the selected component areas is disposed. The second screen layout may be configured by receiving at least one of a location and a size from the user.

The arrangement position of each component area is preset for each component area, and the process of separating the component area from the application screen and placing the component area outside the application screen is performed by the user's touch command. When an arrangement position is input and the input arrangement position of the component area deviates from the preset placement position, the placement position of the input component area may be corrected to the preset placement position.

The screen output method may further include sensing the user's input in the component area disposed outside the application screen on the extended screen.

The screen output method may further include transmitting location information of a point in the component area where the user's input is sensed to the mobile device.

According to another embodiment, one or more processors; And one or more input/output devices, wherein the one or more processors detect driving of an application or a connection with a mobile device on which the application is driven, and the one or more input/output devices are applications related to the currently running application. A display for separating a component area disposed inside the screen from the application screen and placing the component area outside the application screen, and outputting an application screen from which the component area is removed and an extended screen including the component area disposed outside the screen. Device is provided.

1 is an example of an application screen and a component area output from a mobile device according to an embodiment.
2 is an example of a screen output system according to an embodiment.
3 is an example of an extended screen according to an embodiment.
4 is an example of a screen output system according to another embodiment.
5 is a block diagram showing a detailed configuration of a display device according to an embodiment.
6 is a flowchart of a screen output method according to an embodiment.
7 is a block diagram illustrating and illustrating a computing environment including a computing device suitable for use in example embodiments.

Hereinafter, specific embodiments will be described with reference to the drawings. The following detailed description is provided to aid in a comprehensive understanding of the methods, devices and/or systems described herein. However, this is only an example and is not limited thereto.

In describing the embodiments, when it is determined that the detailed description of the related known technology may unnecessarily obscure the subject matter of the invention, the detailed description will be omitted. In addition, terms to be described later are terms defined in consideration of functions, which may vary according to a user's or operator's intention or practice. Therefore, the definition should be made based on the contents throughout this specification. The terms used in the detailed description are only for describing the embodiments and should not be limiting. Unless expressly used otherwise, a singular form includes a plural form. In this description, expressions such as “including” or “equipment” are intended to indicate certain characteristics, numbers, steps, actions, elements, parts or combinations thereof, and one or more other than described. It should not be interpreted to exclude the presence or possibility of other characteristics, numbers, steps, actions, elements, or parts or combinations thereof.

Hereinafter, "game" means software that one or more gamers can enjoy using an input device mounted or connected to a gaming device. Usually, it is performed in a manner such as directly controlling the player character or manipulating the movement of an object in the game, and genres are sometimes classified into role-playing, adventure, and arcade.

Hereinafter, "gaming device" refers to hardware that executes game software, performs calculation according to a user's input through an input device, and outputs a result through an output device. For example, personal computers, laptop computers, game consoles, smart phones, tablet PCs, smart bands, smart watches, etc. It may be in the form of a wearable device. In addition, any hardware that satisfies the above definition is interpreted as belonging to a gaming device.

In the following, "online game" means a game that needs to go through a network in order to play the game. It usually means a game where two or more gaming devices at a remote location are connected to each other through a network or synchronized through a server. If it satisfies the above definition, there is no restriction on the type of gaming device that runs the game. In addition to games that run on personal computers or game consoles, games that run on mobiles, such as smartphones, or games where multiple different gaming devices (including different types of gaming devices) are interlocked or run in the cloud also include Anything that satisfies the definition is interpreted as belonging to an online game.

Hereinafter, the term “Input Device” means an apparatus for inputting data to a gaming device. For example, a joystick, a mouse, a keyboard, a touch screen, a hardware button or a steering wheel, a musical instrument, a gun, gloves, and a scaffolding dedicated input device. In addition, various sensors (temperature sensors, infrared sensors, motion sensors, gyro sensors, acceleration sensors, gravity sensors, etc.) mounted on or interlocked with a gaming device, global positioning systems (GPS), indoor positioning systems (Indoor Positioning System) ) May also be an example of an input device.

Hereinafter, "output device" means an apparatus for displaying a calculation result of a gaming device. The calculation result is displayed in a form that can be recognized using the human five senses. It is not necessarily limited to the case of visual or auditory display, but may also be displayed in olfactory or tactile form. For example, a typical hardware such as a speaker, a monitor, or a head mounted display satisfies the above definition, but in addition, a device that outputs in a tactile form, such as a 3D printer, or generates vibration, fragrance, or wind, may be an example of an output device. .

The input device and the output device may be included in a gaming device as an element of the gaming device, or may be connected to a gaming device through a separate wired or wireless interface.

1 is an example of an application screen 100 and a component area 110 output from a mobile device (not shown) according to an embodiment.

In the present embodiments, the mobile device is a device on which an application is driven, and may be, for example, a smart phone, a tablet PC, or a laptop. In addition, the application is a computer program that helps a user provide a specific service or perform a function related to the service. Here, the service may be, for example, an online game service, an instant message service, etc., but the type of the service is not particularly limited. Hereinafter, for convenience of description, it is assumed that the service is an online game service.

The mobile device may output an application screen 100 related to the application by driving an application provided therein according to a user's command. Referring to FIG. 1, the application screen 100 may be, for example, a game screen. At this time, a plurality of component areas 110 such as a menu window, a chat window, a skill window, a status window, and a quest window may be disposed inside the application screen 100. The number and type of the component areas 110 may vary depending on the type of service provided by the application, the application screen 100 currently being output, and the like.

As an example, the user can play an online game while staring at the application screen 100, and input a component area 110 as necessary to execute various additional functions related to the online game. At this time, the user may execute a function corresponding to the component area 110 by touching the component area 110.

However, in the case of such a mobile device, the screen size is small and there is a problem in that user input is inconvenient, and as a plurality of component areas 110 exist in the above-described application screen 100, the user completely concentrates on the application screen 100 There is a difficult problem to do.

Hereinafter, a screen output method capable of improving a user's concentration level related to a service provided by the application will be described in detail with reference to FIGS. 2 to 6.

2 is an example of a screen output system according to an embodiment. As shown in FIG. 2, the screen output system according to an embodiment includes a display device 200 and a mobile device 300. As described above, the mobile device 300 may output the application screen 100.

The display device 200 transmits and receives information related to an application in cooperation with the mobile device 300 and outputs a screen related to a service provided by the application.

In one embodiment, the display device 200 may be a touch beam projector that forms a screen on the outside by projecting a laser beam and senses a touch input on the screen through a sensor provided therein. The display device 200 may detect a touch input on the screen in a manner of recognizing movement on the screen while scanning at 60 frames per second through an infrared sensor embedded in a portion that transmits a laser beam, for example. . Accordingly, the user can directly touch the screen while gazing at the screen formed by the laser beam to execute additional functions related to the service.

At this time, the display device 200 may output an extended screen instead of the application screen 100 on the screen to improve the user's concentration.

Specifically, the display device 200 separates the component area 110 disposed inside the application screen 100 related to the currently running application from the application screen 100 and places the component area 110 outside the application screen 100, The application screen 100 from which the component region 110 is removed and the extended screen including the component region 110 disposed outside may be output.

To this end, the display device 200 may configure a second screen layout corresponding to the extended screen from a first screen layout corresponding to the application screen 100 in which the component area 110 is not removed, and when the screen is output, The extension screen corresponding to the second screen layout can be output without directly calling up the application screen 100 corresponding to the first screen layout.

As an example, the display device 200 may output the extended screen as it detects a connection with the mobile device 300 on which the application is driven. As another example, the display device 200 outputs the application screen 100 corresponding to the first screen layout as it detects a connection with the mobile device 300 on which the application is driven, and according to a user's screen switching command. The application screen 100 corresponding to the screen layout may be converted into an extended screen corresponding to the second screen layout.

In addition, the user may directly set the second screen layout by customizing the placement position or size of the component area 110 on the screen output from the display device 200.

3 is an example of an extended screen 120 according to an embodiment. Here, it is assumed that eight A component regions 110 to H component regions 110 are disposed inside the initial application screen 100 (ie, the application screen corresponding to the first screen layout).

The display device 200 receives one or more of the plurality of component areas 110 disposed inside the application screen 100 from the user, and receives one or more of the location and size of each of the selected component areas 110 from the user. A second screen layout can be configured by receiving input. As an example, a user touches and selects a specific component area 110 on a screen output from the display device 200, and a desired location of the specific component area 110 while touching the specific component area 110 The second screen layout can be configured by dragging with.

Referring to FIG. 3, the user configures the second screen layout by arranging the B component area, the C component area, and the H component area outside the application screen 100 and adjusting the size of each of these component areas. In this case, the B component area, the C component area, and the H component area are removed inside the application screen 100 according to the newly configured second screen layout.

The display device 200 may output an extended screen corresponding to the second screen layout set by the user, and in this case, the user gazes at the application screen 100 with a larger screen and at the same time external to the application screen 100. By operating the component area 110 in, it is possible to focus more on various situations and functions in the application screen 100.

In addition, an arrangement position of each component region 110 may be preset for each component region 110. In this case, the display device 200 may receive an input position of the component area 110 by a user's touch command, but is input when the input position of the input component area 110 deviates from the preset position. The arrangement position of the component area 110 may be corrected to the previously set arrangement position.

Meanwhile, the first screen layout and the second screen layouts customized by the user or preset by the administrator may be stored in an internal database (not shown) of the display device 200. In this case, the first screen layout may be provided on the set virtual first layer panel, and the second screen layout may be provided on the second layer panel located on the top of the first layer panel. That is, the first screen layout and the second screen layout may be sequentially accumulated in a layer form. The user may newly construct a new second screen layout from the first layer panel, and may change or newly configure the second screen layout from the already configured second layer panel.

The display apparatus 200 may output an extended screen corresponding to the second screen layout configured as described above, and detect a user input in the component area 110 disposed outside the application screen 100 on the extended screen. . Here, the user's input may be, for example, a user's touch input, but is not limited thereto. For example, the display device 200 may detect a user input in the mobile controller by interlocking the extended screen with a separate mobile controller (not shown).

Thereafter, the display device 200 may transmit location information of a point in the component area 110 in which the user's input is sensed, to the mobile device 300. In this case, a function corresponding to the component area 110 may be executed based on the location information on an application installed on the mobile device 300.

4 is an example of a screen output system according to another embodiment. here,

Referring to FIG. 4, the display apparatus 200 has an application therein, and after detecting the driving of the application, outputs a screen related to a service provided by the application. Unlike the previous embodiment, the display device 200 has an application and an operating system (OS) for driving the application itself, and thus, interworking with the mobile device 300 is unnecessary.

As an example, the display device 200 may be a display panel equipped with a touch sensor. The display device 200 may output the above-described extended screen and sense a touch input on the extended screen through a touch sensor provided therein. The method for constructing the extended screen and the method for outputting it are the same as described above, and thus detailed description thereof will be omitted.

5 is a block diagram showing a detailed configuration of the display device 200 according to an embodiment. As shown in FIG. 5, the display device 200 according to an embodiment includes a sensing unit 202, an input/output unit 204, and an interface unit 206.

In one embodiment, the sensing unit 202, the input/output unit 204, and the interface unit 206 are implemented by using one or more physically separated devices, or by combining one or more processors or one or more processors and software. It may be, unlike the example shown may not be clearly distinguished in the specific operation.

The sensing unit 202 detects the driving of the application or the connection with the mobile device 300 on which the application is driven. As an example, when the display device 200 has an application and an operating system (OS) for driving the application itself, the sensing unit 202 may detect driving of an application installed in the display device 200. have. As another example, when the display device 200 operates in conjunction with the mobile device 300, the sensing unit 202 may detect a connection with the mobile device 300 on which the application is driven.

The input/output unit 204 separates the component area 110 disposed inside the application screen 100 related to the currently running application from the application screen 100 and places the component area 110 outside the application screen 100, and the component area ( 110) outputs the extended screen including the removed application screen 100 and the externally disposed component area 110. Specifically, the input/output unit 204 may configure a second screen layout corresponding to the extended screen from a first screen layout corresponding to the application screen 100 in which the component area 110 is not removed, and when the screen is output, The extension screen corresponding to the second screen layout can be output without directly calling up the application screen 100 corresponding to the first screen layout. At this time, the input/output unit 204 outputs the application screen 100 corresponding to the first screen layout, and then applies the application screen 100 corresponding to the first screen layout to the second screen layout according to a user's screen switching command. By switching to a corresponding extended screen or detecting an operation of an application or a connection with the mobile device 300 on which the application is driven, an extended screen corresponding to the second screen layout may be output immediately.

In addition, the user can directly set the second screen layout by customizing the placement position or size of the component area 110 on the screen output from the display device 200. Specifically, the input/output unit 204 receives one or more of the plurality of component areas 110 disposed inside the application screen 100 from the user, and at least one of the placement position and size of each of the selected component areas 110 The second screen layout may be configured by receiving the input from the user.

At this time, the arrangement position of each component area may be preset for each component area. The input/output unit 204 receives the arrangement position of the component area 110 by a user's touch command, and when the arrangement position of the input component area 110 deviates from the preset placement position, the input component area 110 The arrangement position of can be corrected to the previously set arrangement position.

In addition, the input/output unit 204 may detect a user input in the component area 110 disposed outside the application screen 100 on the extended screen.

The interface unit 206 transmits and receives information related to an application in conjunction with the mobile device 300. The interface unit 206 may include a wireless communication module such as a Bluetooth module, a Wi-Fi module, etc., and is connected to the mobile device 300 through the wireless communication module to transmit and receive information related to an application.

As an example, when the display device 200 is connected to the mobile device 300, the interface unit 206 outputs driving information of the application from the mobile device 300 (eg, output from the current mobile device 300). Screen information, location information of each component area in the screen, and the like).

In addition, when the input/output unit 204 detects a user input in the component area 110 disposed outside the application screen 100 on the extended screen, the interface unit 206 detects the user input component. Location information of a point in the region 110 may be transmitted to the mobile device 300.

6 is a flowchart of a screen output method according to an embodiment. The method illustrated in FIG. 6 may be performed, for example, by the above-described display device 200. Meanwhile, in the flowchart illustrated in FIG. 6, the method is described as being divided into a plurality of steps, but at least some of the steps are performed by changing the order, combined with other steps, omitted, or divided into detailed steps. Alternatively, one or more steps (not shown) may be added and performed.

In step 602, the display device 200 detects a connection with the mobile device 300. If the display device 200 has an application and an operating system (OS) for driving the application itself, the display device 200 may detect the operation of an application installed therein.

In operation 604, the display device 200 separates the component area 110 disposed inside the application screen 100 related to the currently running application from the application screen 100 and places the component area 110 outside the application screen 100. , The application screen 100 from which the component area 110 is removed and the extended screen including the component area 110 disposed outside are output.

In operation 606, the display device 200 detects a user input in the component area 110 disposed outside the application screen 100 on the extended screen.

In operation 608, the display device 200 transmits location information of a point in the component area 110 where a user's input is detected to the mobile device 300.

7 is a block diagram illustrating and illustrating a computing environment including a computing device suitable for use in example embodiments. In the illustrated embodiment, each component may have different functions and capabilities in addition to those described below, and may include additional components in addition to those described below.

The illustrated computing environment 10 includes a computing device 12. In one embodiment, computing device 12 may be one or more components included in display device 200 or mobile device 300.

The computing device 12 includes at least one processor 14, a computer readable storage medium 16 and a communication bus 18. The processor 14 can cause the computing device 12 to operate in accordance with the exemplary embodiment mentioned above. For example, processor 14 may execute one or more programs stored on computer readable storage medium 16. The one or more programs may include one or more computer-executable instructions, which, when executed by the processor 14, configure the computing device 12 to perform operations according to an exemplary embodiment. Can be.

Computer readable storage medium 16 is configured to store computer executable instructions or program code, program data and/or other suitable types of information. The program 20 stored on the computer readable storage medium 16 includes a set of instructions executable by the processor 14. In one embodiment, the computer readable storage medium 16 is a memory (volatile memory such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash Memory devices, other types of storage media that can be accessed by the computing device 12 and store desired information, or suitable combinations thereof.

The communication bus 18 interconnects various other components of the computing device 12, including a processor 14 and a computer readable storage medium 16.

Computing device 12 may also include one or more I/O interfaces 22 and one or more network communication interfaces 26 that provide an interface for one or more I/O devices 24. The input/output interface 22 and the network communication interface 26 are connected to the communication bus 18. The input/output device 24 may be connected to other components of the computing device 12 through the input/output interface 22. Exemplary input/output devices 24 include pointing devices (such as a mouse or trackpad), keyboards, touch input devices (such as touch pads or touch screens), voice or sound input devices, various types of sensor devices, and/or imaging devices. Input devices and/or output devices such as display devices, printers, speakers and/or network cards. The exemplary input/output device 24 may be included inside the computing device 12 as a component constituting the computing device 12, and may be connected to the computing device 12 as a separate device distinct from the computing device 12. It might be.

According to this embodiment, the component area 110 existing inside the application screen 100 is disposed outside the application screen 100, and the application screen 100 and the component area 110 are removed. By outputting the extended screen 120 including the component area 110 disposed outside, the user can focus on various situations and functions in the application screen 100.

Although the present invention has been described in detail through exemplary embodiments above, those skilled in the art to which the present invention pertains are capable of various modifications to the above-described embodiments without departing from the scope of the present invention. Will understand. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined not only by the claims to be described later, but also by the claims and equivalents.

10: computing environment
12: computing device
14: processor
16: computer readable storage media
18: Communication bus
20: Program
22: I/O interface
24: I/O device
26: network communication interface
100: application screen
110: component area
120: extended screen
200: display device
202: detection unit
204: input and output unit
206: interface unit
300: mobile device

Claims (17)

A sensing unit that detects the driving of an application or a connection with a mobile device on which the application is driven; And
The component area disposed inside the application screen related to the currently running application is separated from the application screen and placed outside the application screen, and the application screen in which the component area is removed and the component area disposed outside are included. And an input/output unit for outputting the expanded screen.
The method according to claim 1,
The input/output unit configures a second screen layout corresponding to the extended screen from a first screen layout corresponding to an application screen in which the component area is not removed.
The method according to claim 2,
The first screen layout is provided on the set first layer panel,
The second screen layout is provided on a second layer panel positioned on top of the first layer panel.
The method according to claim 2,
The input/output unit receives one or more of a plurality of component areas disposed inside the application screen from the user, receives at least one of a placement position and a size of each of the selected component areas from the user, and layouts the second screen. Constituting, display device.
The method according to claim 4,
The arrangement position of each component area is preset for each component area,
The input/output unit receives the arrangement position of the component area according to the user's touch command, and when the arrangement position of the input component area deviates from the preset placement position, the placement position of the input component area is preset. A display device that compensates for the placement position.
The method according to claim 1,
The input/output unit detects the input of the user in a component area disposed outside the application screen on the extended screen.
The method according to claim 6,
And an interface unit for transmitting location information of a point in the component area where the user's input is detected to the mobile device.
The method according to claim 1,
The display device is a touch beam projector that forms a screen on the outside by projecting a laser beam and senses a touch input on the screen through a sensor provided therein.
The method according to claim 1,
The display device is a display panel with a touch sensor.
A screen output method performed on a display device,
Detecting an operation of an application or a connection with a mobile device on which the application is driven;
Separating a component area disposed inside an application screen related to the currently running application from the application screen and placing the component area outside the application screen; And
And outputting an application screen from which the component area is removed and an extended screen including the externally disposed component area.
The method according to claim 10,
The process of separating the component area from the application screen and arranging it outside the application screen comprises configuring a second screen layout corresponding to the extended screen from a first screen layout corresponding to an application screen in which the component area is not removed. To do, the screen output method.
The method according to claim 11,
The first screen layout is provided on the set first layer panel,
The second screen layout is provided on a second layer panel located on top of the first layer panel, the screen output method.
The method according to claim 11,
The process of separating the component area from the application screen and placing it outside the application screen includes selecting one or more of a plurality of component areas disposed inside the application screen from the user, and arranging each of the selected component areas A screen output method comprising configuring at least one of position and size from the user to configure the second screen layout.
The method according to claim 13,
The arrangement position of each component area is preset for each component area,
In the process of separating the component area from the application screen and placing the component area outside the application screen, the arrangement position of the component area is input by the user's touch command, and the arrangement position of the input component area is preset. When it deviates from the arrangement position, a screen output method of correcting the input arrangement position of the component area to the preset placement position.
The method according to claim 10,
And detecting the user's input in a component area disposed outside the application screen on the extended screen.
The method according to claim 15,
And transmitting location information of a point in the component area where the user's input is detected to the mobile device.
One or more processors; And
One or more input/output devices,
The one or more processors detect the driving of the application or the connection with the mobile device on which the application is driven,
The one or more input and output devices,
The component area disposed inside the application screen related to the currently running application is separated from the application screen and placed outside the application screen.
A display device for outputting an application screen from which the component area is removed and an extended screen including the component area disposed outside.

Images