WO2018205745A1

WO2018205745A1 - Input interface display system and method for portable device

Info

Publication number: WO2018205745A1
Application number: PCT/CN2018/079170
Authority: WO
Inventors: 占涛
Original assignee: 北京点石经纬科技有限公司
Priority date: 2017-05-12
Filing date: 2018-03-15
Publication date: 2018-11-15
Also published as: CN108874333A; TW201901485A

Abstract

An input interface display system for a portable device. The portable device has a plurality of display interfaces. The display system comprises a target screen acquisition module (210), a main control module (220), a window adjustment module (230) and a window display module (240), wherein the target screen acquisition module (210) sends, to the main control module (220), an acquired target screen of an input interface needing to be displayed; the main control module (220) acquires a screen serial number corresponding to the target screen so as to generate a first display command; the window adjustment module (230) receives the first display command and adjusts, according to screen attribute information, the size of the input interface so as to generate a second display command; and the window display module (240) displays the input interface on the target screen according to the second display command. By means of the input interface display system, an input interface is displayed on different display screens, and the display efficiency of the input interface is improved.

Description

Input interface display system and method for portable device

Technical field

The present invention relates generally to the field of portable device system design, and more particularly to a system for displaying an input interface in a portable device.

Background technique

Previous portable devices often have display screens and input devices such as physical keyboards. However, with the rapid development of large-screen smart devices, physical input devices are often no longer set on most of today's smart portable devices. For example, in physical phones, physical buttons are not set to obtain a larger display screen, for example, in smart phones. The physical keyboard is no longer set in the laptop and replaced by another display screen. The input devices of these entities are replaced by input interfaces (such as virtual keyboards) displayed on the screen, and the operator completes specific text input by clicking on a specific position of the input interface on the display screen with touch function.

Currently, most of the smart portable devices display the operation of the input interface when the control system integrated in the portable device is completed. The control system controls the input interface to be displayed on the screen in a specific form, and responds to the operator's click operation to complete the text. Input. Since most of the smart portable devices in the past have only one display screen, the operating system of the smart portable device is designed such that the input interface can only be displayed on one screen.

However, many smart portable devices now have more than one display screen, such as a display screen on the front and back of the smartphone, or a display screen in the smart laptop, each having a display screen for these A portable device with multiple screens, if using an existing control system, can only display the input interface in one screen (also called the main screen), but cannot display it in other screens (also called a secondary screen). The input interface, which greatly affects the use of portable devices. Of course, people have made some improvements to the control system, projecting or displaying a background image similar to the input interface on the secondary screen, and using an additional interface to the background image and the control module for displaying the input interface on the main screen in the operating system. The connection is performed in response to an operator's click operation on the sub-screen to complete the input operation of the sub-screen. However, because it does not directly call the relevant functional modules in the control system, the display efficiency and response speed of the input interface will inevitably decrease, which also affects the use of portable devices to a certain extent.

Therefore, there is a need for a display system that can display an input interface in different display screens without affecting display efficiency and response speed.

Summary of the invention

The object of the present invention is to solve the above problem that display efficiency and response speed cannot be displayed on different screen display input interfaces and input interfaces.

In order to solve the above problems, the present invention provides an input interface display system for a portable device, the portable device having a plurality of display screens, wherein the input interface display system has a target screen acquisition module, a main control module, a window adjustment module and a window display module;

The target screen acquisition module is configured to acquire a target screen that needs to display an input interface, and send the target screen to the main control module;

The main control module acquires a screen number corresponding to the target screen from the screen number list to generate a first display command, and sends the first display command including the screen number to the window adjustment module;

The window adjustment module acquires screen attribute information corresponding to the target screen according to the first display command to generate input interface size information, and combines the input interface size information with the first display command to generate a first Second display command;

The window display module receives the second display command to control the target screen to display the input interface.

The interface display system, wherein the target screen acquisition module further comprises a monitoring unit, the monitoring unit determines whether the target screen changes, and if the target screen changes, the monitoring unit sends the current target screen to the main control module.

The interface display system, wherein the input interface display system further has an imaging device, a pressure sensing device, and/or a gravity sensing device;

The camera device acquires a display screen currently viewed by the user, and sends the display screen to the monitoring unit;

The pressure sensing device acquires a display screen currently touched by a user, and sends the display screen to the monitoring unit;

The gravity sensing device acquires a display screen placed upward and transmits the display screen to the monitoring unit.

The interface display system, further comprising a layout adjustment module, wherein the layout adjustment module controls the target screen according to the second display command and the display layout information according to display layout information for acquiring the target screen Changing the display layout; the display layout information is used to describe the size and location of the window displayed by the program running within the portable device in the target screen.

The above interface display system, wherein the layout adjustment module controls the target screen to reduce the size of a program window displayed in the target screen, or add a scroll bar in the program window and scroll up the program window.

The above interface display system, wherein the screen attribute information includes a resolution of the screen and dots per inch.

The above interface display system, wherein the input interface size information includes a length, a width, and a position of the input interface.

The present invention also provides an input interface display method for a portable device having a plurality of display screens, characterized in that:

Obtaining a target screen, obtaining a target screen that needs to display an input interface, and obtaining a screen number corresponding to the target screen;

Adjusting the size of the input interface, adjusting the size of the input interface according to the screen attribute information corresponding to the target screen;

A step of displaying an input interface on which the size-adjusted input interface is displayed.

The above method, further comprising:

a step of monitoring a target screen, monitoring whether the target screen changes in real time, and outputting the current target screen if the target screen changes; and/or

The step of adjusting the display layout adjusts the display layout of the program window displayed in the target screen.

The above method, wherein the step of monitoring the target screen further comprises:

Obtaining a display screen that the user is currently watching, and if the display screen currently viewed by the user is different from the target screen, determining that the target screen changes;

Obtaining a display screen currently touched by the user, if the display screen currently touched by the user is different from the target screen, determining that the target screen changes; and/or

The display screen currently placed upward is obtained, and if the display screen currently placed upward is different from the target screen, it is determined that the target screen changes.

The above method, wherein the step of adjusting the display layout further comprises:

Shrinking the size of the program window; and/or

Add a scroll bar to the program window and scroll up the program window.

DRAWINGS

1 is a schematic diagram of a portable device having multiple display screens in accordance with an embodiment of the present invention;

2 is a modular schematic diagram of an input interface display system in accordance with an embodiment of the present invention;

3 is a schematic structural diagram of a target screen display module of an input interface display system according to another embodiment of the present invention;

4 is a modular schematic diagram of an input interface display system in accordance with another embodiment of the present invention;

5A-5C are schematic diagrams showing a layout of a target screen according to an embodiment of the present invention;

6 is a flow chart of an input interface display method in accordance with an embodiment of the present invention.

detailed description

The technical solutions of the present invention will be described below in conjunction with the specific embodiments of the present invention and the accompanying drawings. It should be noted that the same reference numbers will be used herein to refer to the elements in the various figures. The accompanying drawings are merely illustrative of the preferred embodiments of the invention,

Please refer to FIG. 1. FIG. 1 is a schematic diagram of a portable device 100 according to an embodiment of the present invention. As can be seen from FIG. 1, the portable device 100 is similar to a commonly used notebook computer having an upper housing 110 and a lower housing. 120, and a first screen 130 (also referred to as a main screen) is disposed in the upper housing 110. In a typical notebook computer, a physical keyboard is usually disposed in the lower housing 120 for text input, and in the portable device 100. In the middle and lower housing 120, the second screen 140 (also referred to as a secondary screen) is provided instead of the physical keyboard, so that the portable device has two screens, and the input function of the portable device is realized by using the screen with the touch function. Therefore, this increases the display content of the portable device and improves the display effect. However, corresponding to the current portable devices, the operating system (for example, the Android operating system) can only utilize the function module for displaying the virtual keyboard integrated in the operating system on a fixed screen (for example, the home screen in the portable device 100). 130) displaying an input interface, which may be, for example, a traditional qwerty standard keyboard used on a laptop or a smart tablet, or a numeric keypad in the form of a nine-square grid used on a smart phone, but not in a secondary screen 140 The other input screens are displayed on the screen, which reduces the usability of such portable devices.

Please continue to refer to FIG. 2. FIG. 2 is a modular schematic diagram of an input interface display system 200. In the display system 200, the target screen acquisition module 210 is used to acquire a target screen. The target screen described herein is For example, the portable device 100 shown in FIG. 1 is used as an example. When an operator needs to input text on the main screen 130, an input interface (such as a virtual keyboard) needs to be displayed on the main screen 130. When the target screen is the main screen 130, similarly, when the operator needs to input on the sub-screen 140, the input screen needs to be displayed in the sub-screen 140, and then the target screen is the sub-screen 140. The target screen acquisition module 210 can acquire the target screen by using various methods. For example, in some embodiments, the information of the target screen may be saved in the target screen acquisition module 210 by a preset manner or saved in the target screen acquisition module 210. Sexually connected peripherals (not shown) to serve as the initial target screen. When the user needs to perform text input, the display system is started, and the target screen acquisition module 210 can directly obtain the initial target screen from within itself or from the peripheral device. In other embodiments, the target screen acquisition module 210 may also acquire target screens sensed by the sensing devices from a plurality of sensing devices (eg, camera devices, touch sensors, and gravity sensors, etc.) connected thereto. In still other embodiments, the target screen acquisition module 210 may also have a human-computer interaction interface (not shown) through which the user can specify a target screen on which the input interface needs to be displayed.

After the target screen acquisition module 210 acquires the target screen, the target screen acquisition module 210 sends the target screen to the main control module 220. When the main control module 210 searches the screen number list 251 in the external database 250 according to the information of the target screen, the screen number corresponding to the target screen is retrieved. The screen number list 251 stores the correspondence between all display screens and screen numbers in the portable device. Taking the portable device 100 shown in FIG. 1 as an example, the screen number list 251 can store two display screens, that is, the main screen 130 and the sub-screen 140, and also stores two screen numbers, such as ID_0 and ID_1, wherein the main screen 130 Corresponding to the screen number ID_0, the sub-screen 140 corresponds to the screen number ID_1. In still other embodiments, when the portable device has more than two display screens, the content stored in the screen number list 251 also increases. After obtaining the screen number of the target screen, the main control module 210 generates a first display command by using the screen number. The first display command may include other instruction data, such as the type of the input interface, in addition to the screen number of the target screen (for example, The qwerty standard keyboard or the numeric keypad in the form of a nine-square grid, the display level of the input interface (for example, always displayed on the top of the target screen), the initial size of the input interface, and a series of basic instructions for displaying the data interface. In a specific implementation, the main control module 210 may be a control circuit with a separate processing chip, or may be an arithmetic module integrated in the operating system. For example, in a specific implementation, the main control module 210 may be an Android operating system. The input method service (InputMethodService), through which the Android operating system can initialize and display the input interface in the portable device.

Then, the main control module sends the generated first display command to the window adjustment module 230. After receiving the first display command, the window adjustment module 230 extracts the screen number of the target screen from the first display command again, and then according to the The screen number looks up the screen attribute information of the target screen corresponding to the screen number in the screen attribute list 252 in the database 250. In particular embodiments, the screen attribute list 252 stores attributes of all display screens of the portable device, which may include display resolution, physical size, and/or dots per inch (dpi) of the display screen, and the like. The window adjustment module 230 adjusts the size of the input interface according to the obtained screen attribute information, so that the input interface is displayed on the target screen in an appropriate manner. In some embodiments, the size of the input interface may be automatically adjusted using a window management policy service (PhoneWindowManager) in the Android system. For example, after the window adjustment module 230 sends the attribute information of the target screen to the window management policy service, the window After managing the policy service, the size of the input interface can be automatically adjusted according to the screen attribute information, for example, the length, width and position of the input interface are adjusted, so that the input interface is appropriately sized and at a suitable position in the target screen. Display, the position of the input interface here may be the coordinates of the upper left point of the input interface in the target screen. Of course, other key points of the input interface may also be used to indicate the position of the input interface. After the window adjustment module 230 calculates the size of the input interface, the size information of the input interface is incorporated into the first display command to generate a second display command. Therefore, by setting the window adjustment module 230, the display system 200 can adjust the size of the input interface in real time according to the unique attribute of the target screen, so that the input interface is always presented on the target screen in an optimal manner, which is due to the portable device. Different display screens often have hardware differences. If you always enter the interface in a fixed size, the mismatch between the input screen and the target screen may occur.

Next, the window display module 240 is configured to execute a second display command to display the input interface on the target screen with a suitable input interface size.

Please refer to FIG. 3 in conjunction with FIG. 2. FIG. 3 is a schematic structural diagram of a target screen acquiring module 210 according to another embodiment of the present invention. In some embodiments, the target screen acquisition module 210 may further have a monitoring unit 211, in the process of displaying the input interface of the target screen, the monitoring unit 211 monitors whether the target screen changes in real time, and when the monitoring unit 211 finds that the target screen changes, The monitoring unit 211 transmits the current latest target screen to the main control module 220, so that the control input interface is always displayed on the display screen on which the user needs to perform text input. For example, when the user uses the portable device as shown in FIG. 1 , when the user first performs text editing using the text editing software on the main screen 130 , the target screen acquiring module 210 acquires the current target screen as the main screen 130 and sends it to the main screen. The control module 220 thus controls the home screen 130 to display the input interface while the monitoring unit 211 always monitors whether the target screen has changed. After a period of time, for example, an email program running in the background of the portable device receives a new email, at which point the user may wish to open the email program on the secondary screen 140 and reply to the received email. The monitoring unit 211 can detect that the user is about to use the secondary screen 140 to reply to the mail. Thus, the monitoring unit 211 determines that the target screen has changed from the home screen 130 to the secondary screen 140, and then the current target screen (ie, the secondary screen 140). It is sent to the main control module 220 to control the sub-screen 140 to display the input interface. After the user completes the operation of replying to the mail, the monitoring unit 211 detects that the user will return to the home screen 130 before the text editing is continued, and the monitoring unit 211 determines that the target screen has changed from the secondary screen 140 to the home screen 130, and then the current target screen ( That is, the main screen 130) is sent to the main control module 220, thereby controlling the main screen 130 to display the input interface. As a result, the monitoring unit 211 can detect the latest target screen in real time, so that the input interface is always displayed on the display screen where the user needs to input text, thereby improving the use experience of the portable device.

In a specific embodiment, the monitoring unit 211 can monitor whether the target screen has changed by various sensing devices electrically coupled thereto. In a specific embodiment, the portable device may be provided with a camera 212, and the camera 212 may detect the angle of the user's face orientation or the angle of the user's eye gaze in real time through various existing face detection algorithms or human eye tracking algorithms. The obtained angle information is analyzed to obtain a display screen that the user is currently looking at, and the display screen is fed back to the monitoring unit 211 to assist the monitoring unit 211 in monitoring whether the target screen is changed. In another embodiment, the portable device can set a pressure sensor 213 on each display screen, and when the user applies the appropriate force to the display screen, the pressure sensor 213 corresponding to the display screen senses the power, and The display screen is fed back to the monitoring unit 211 to assist the monitoring unit 211 in monitoring whether the target screen is changed. In still another embodiment, the portable device can set the gravity sensor 214 in the vicinity of each display screen. When the user wants to use one of the display screens for text input, the display screen needs to be placed upward, and the display screen is displayed at this time. The corresponding gravity sensor 214 feeds the display screen placed upward to the monitoring unit 211 to assist the monitoring unit 211 in monitoring whether the target screen is changed. Of course, in other modes, the portable device can simultaneously use the camera 212, the pressure sensor 213, and the gravity sensor 214 to cooperate with the monitoring unit 211 to monitor whether the target screen changes, for example, when the portable device is simultaneously set with the above three transmissions. When the device is sensed, if more than two of the three sensing devices feed back the same display screen to the monitoring unit 211, the monitoring unit 211 compares the display screen with the target screen currently displaying the input interface. To determine if the target screen has changed.

Please refer to FIG. 4. FIG. 4 is a modular schematic diagram of an input interface display system 300 according to another embodiment of the invention. The display system 300 is different from the display system 200 in FIG. 2 in that the main control module 320 and the window adjustment module 330 respectively have a screen number list 321 and a screen attribute list 331 so that the module 320 and the window adjustment module 330 can be Directly finding the screen number and screen attribute information of the desired target screen without accessing an external database further increases the speed at which display system 300 displays the input interface. At the same time, the display system 300 further has a layout adjustment module 350. After receiving the second display command from the window adjustment module 330, the layout adjustment module 350 extracts the screen number of the target screen and the size information of the input interface from the second display command. At the same time, the layout adjustment module 350 further has an interface for accessing the display screen corresponding to the screen number according to the screen number, thereby obtaining display layout information currently displayed on the display screen from the display screen, the display layout information indicating portable The display window of the program running in the device is laid out in the display screen. Since the layout adjustment module 350 extracts the screen number of the target screen from the second display command, the layout information acquired by the layout adjustment module 350 is the layout information of the target screen. At the same time, the layout adjustment module adjusts the layout of the target screen according to the size information of the input interface so that the display of the input screen is not affected after the display screen of the target screen is displayed. In a specific implementation, the layout adjustment module 350 may reduce the size of other program windows in the target screen according to the size information of the input interface, so that the input interface and other program windows are simultaneously displayed on the target screen without affecting each other, for example, in FIG. 5A. In the target screen 400, a program window 410 is displayed. For example, the program window 410 may be a window of a text editing program running in the portable device. In FIG. 5B, when the input interface 420 is displayed in the target screen 400, the program window 410 is Appropriately zooming out and moving up so that the target screen 400 can simultaneously display the program window 410 and the input interface 420 without affecting each other. In a specific implementation manner, the layout adjustment module 350 may add a scroll bar in the program window according to the size of the input interface without changing the display ratio of other program windows, and then scroll up and down in the program window to display the program window. All content inside. For example, in FIG. 5C, after the input interface 420 is displayed in the target screen 400, the program window 410 is appropriately moved upward without changing the display scale, and at the same time, a scroll bar 411 is added on the right side of the program window 410 so that the user can drag Scroll bar 411 to view all of the content within program window 410 such that program window 410 and input interface 420 do not interact with each other. In a specific implementation manner, when the operating system of the portable device is an Android operating system, the window management service (WindowManagerService) in the Android system may also be used to automatically adjust the display layout of the target screen.

Please refer to FIG. 6. FIG. 6 is a flowchart of a method for displaying an input interface according to an embodiment of the present invention, wherein the steps illustrated by the dotted boxes indicate that the steps may be included in some embodiments, or may not be included in others. In a specific embodiment. In step S10, a target screen that needs to display an input interface is acquired, and a screen number corresponding to the target screen is obtained. For example, in some embodiments, the screen number of the target screen can be obtained from a list in which the display screen and screen number correspondence are stored. In step S20, the attributes of the target screen, such as the physical size, resolution, and dpi of the target screen, are acquired according to the screen number of the target screen, and the size of the input interface is adjusted according to the screen attribute information, such as the length, width, and width of the input interface. position. In some embodiments, the attributes of the target screen can be looked up from a list of all display screen attribute information stored. Next, in step S30, the size-adjusted input interface is displayed in the target screen.

In addition, in some embodiments, step S40 and step S50 may be further included. In step S40, whether the target screen changes or not is detected in real time. If the target screen changes, steps S20 and S30 are performed on the changed target screen. In a specific implementation, the angle of the user's face orientation or the angle of the user's eye gaze can be detected by the camera, and the obtained angle information is obtained by analyzing the obtained angle information, and the display screen is performed with the target screen. Compare to determine if the target screen has changed. If the display screen is different from the target screen, it is determined that the target screen has changed. In a specific embodiment, a display screen that the user is touching or is applying force is acquired by the pressure sensor, and the display screen is compared with the target screen to determine whether the target screen changes. If the display screen is different from the target screen, it is determined that the target screen has changed. In a specific embodiment, the display screen currently placed upward is detected by a gravity sensor, and the display screen is compared with the target screen to determine whether the target screen changes. If the display screen is different from the target screen, it is determined that the target screen has changed. In step S50, the display layout of other program windows in the target screen is acquired, and the display layout in the target screen is adjusted according to the size of the input interface, so as to not affect other program windows after the input screen is displayed on the target screen. In a specific implementation, the display layout of the target screen can be adjusted by reducing the size of other program windows. In a specific implementation, the display layout of the target screen can also be adjusted by adding scroll bars to other program windows.

The above is only the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Those skilled in the art can make various corresponding according to the present invention without departing from the spirit and scope of the present invention. Changes and modifications are intended to be included within the scope of the appended claims.

Industrial applicability

In summary, the present invention has the beneficial effects that the target screen that needs to display the input interface can be acquired and specified by the target screen acquisition module, instead of always displaying the input interface on the pre-designated screen, so that the input interface of the present invention The display system can display the input interface on different screens; in addition, since the target main control module directly displays the input interface according to the target screen, without calling an additional interface module to realize the display of the input interface in different screens, The use of additional interface modules results in a drop in the input interface display interface and response speed.

Claims

An input interface display system for a portable device, the portable device having a plurality of display screens, wherein the input interface display system has a target screen acquisition module, a main control module, a window adjustment module, and a window display module;

The target screen acquisition module is configured to acquire a target screen that needs to display an input interface, and send the target screen to the main control module;

The main control module acquires a screen number corresponding to the target screen from the screen number list to generate a first display command, and sends the first display command including the screen number to the window adjustment module;

The window adjustment module acquires screen attribute information corresponding to the target screen according to the first display command to generate input interface size information, and combines the input interface size information with the first display command to generate a first Second display command;

The window display module receives the second display command to control the target screen to display the input interface.
The input interface display system according to claim 1, wherein the target screen acquisition module further comprises a monitoring unit, the monitoring unit determines whether the target screen changes, and if the target screen changes, the monitoring unit sends the current Target screen to the main control module.
The input interface display system according to claim 2, wherein the input interface display system further has an imaging device, a pressure sensing device, and/or a gravity sensing device;

The camera device acquires a display screen currently viewed by the user, and sends the display screen to the monitoring unit;

The pressure sensing device acquires a display screen currently touched by a user, and sends the display screen to the monitoring unit;

The gravity sensing device acquires a display screen placed upward and transmits the display screen to the monitoring unit.
The input interface display system according to claim 1, further comprising a layout adjustment module, wherein the layout adjustment module is configured to obtain display layout information of the target screen, and according to the second display command and The display layout information controls the target screen to change a display layout; the display layout information is used to describe a size and a position of a window displayed by the program running in the portable device in the target screen.
The input interface display system according to claim 4, wherein the layout adjustment module controls the target screen to reduce the size of a program window displayed in the target screen, or add a scroll bar in the program window and Scroll up the program window.
The input interface display system according to any one of claims 1 to 5, wherein the screen attribute information comprises a resolution of the screen and dots per inch.
The input interface display system according to any one of claims 1 to 5, wherein the input interface size information comprises a length, a width and a position of the input interface.
An input interface display method for a portable device having a plurality of display screens, characterized in that:

Obtaining a target screen, obtaining a target screen that needs to display an input interface, and obtaining a screen number corresponding to the target screen;

Adjusting the size of the input interface, adjusting the size of the input interface according to the screen attribute information corresponding to the target screen;

A step of displaying an input interface on which the size-adjusted input interface is displayed.
The method of claim 8 further comprising:

a step of monitoring a target screen, monitoring whether the target screen changes in real time, and outputting the current target screen if the target screen changes; and/or

The step of adjusting the display layout adjusts the display layout of the program window displayed in the target screen.
The method of claim 9, wherein the step of monitoring the target screen further comprises:

Obtaining a display screen that the user is currently watching, and if the display screen currently viewed by the user is different from the target screen, determining that the target screen changes;

Obtaining a display screen currently touched by the user, if the display screen currently touched by the user is different from the target screen, determining that the target screen changes; and/or

The display screen currently placed upward is obtained, and if the display screen currently placed upward is different from the target screen, it is determined that the target screen changes.
The method of claim 9, wherein the step of adjusting the display layout further comprises:

Shrinking the size of the program window; and/or

Add a scroll bar to the program window and scroll up the program window.