TWI844216B - Windows arrangement system, windows arrangement method and windows arrangement program - Google Patents

Abstract

A windows arrangement system, comprising: a memory device for storing an application program; and a central processing device executing the application to implement the windows detection module and the windows arrangement inference module; the windows detection module detects whether the desktop/window status in the operating system meets the window arrangement activation condition, wherein the desktop/window status includes the number of desktops in the operating system and the windows in each of the desktops; the windows arrangement inference module inputs the information about the desktop/window state into the windows arrangement inference model and outputs the windows arrangement inference result when the desktop/window state meets the windows arrangement activation condition, and returns the windows arrangement inference result to the windows detection module to arrange the desktop/window state.

Description

Window layout system, window layout method and window layout program product

The present invention relates to a window layout system, a window layout method and a window layout program product.

As processors become more powerful, it has become common for users to configure multiple desktops or virtual desktops in the computer operating system and open multiple windows in each desktop. For example, when developing software, software developers usually need to open multiple web pages, manual documents, development platforms and other windows, and arrange them in multiple desktops or virtual desktops to facilitate quick switching and browsing. In addition, hardware or firmware developers, audio and video workers, editors, creative workers or word processing personnel may also encounter the need to arrange multiple desktops and windows.

Since the number of windows or virtual desktops opened at work may be very large, the time spent on arranging multiple desktops and windows is also considerable. For example, every time a software developer sets up a work environment, it may take tens of minutes to open each required window, arrange each window on each desktop, and adjust the window size to make it easy to read, which seriously affects work efficiency. Currently, no previous technology can improve this problem.

In view of this, the present invention proposes a window arrangement system, a window arrangement method and a window arrangement program product, which can detect and infer the user's desktop and window arrangement, and help the user quickly establish a working environment.

An embodiment of the present invention provides a window arrangement system, comprising: a memory device for storing an application program; and a processing device for executing the application program to implement a window detection module and a window arrangement inference module; wherein: the window detection module detects whether the desktop/window state in an operating system meets a window arrangement activation condition, and the desktop/window state includes The number of desktops in the operating system and the windows in each desktop; the window arrangement inference module inputs information about the desktop/window state into the window arrangement inference model and outputs a window arrangement inference result when the desktop/window state meets the window arrangement activation condition, and returns the window arrangement inference result to the window detection module to arrange the desktop/window state.

In this embodiment, the window detection module can detect desktop/window change information after arranging the desktop/window state, and transmit the desktop/window change information to an optimization module; the optimization module retrains the window arrangement inference model according to the desktop/window change information.

Another embodiment of the present invention provides a window arrangement method, including: detecting whether a desktop/window state in an operating system meets a window arrangement activation condition, the desktop/window state including the number of desktops in the operating system and the windows in each of the desktops; when the desktop/window state meets the window arrangement activation condition, inputting information about the desktop/window state into a window arrangement inference model and outputting a window arrangement inference result, and arranging the desktop/window state according to the window arrangement inference result.

In this embodiment, after arranging the desktop/window state, a desktop/window change information can be detected, and the window arrangement inference model can be retrained according to the desktop/window change information.

Another embodiment of the present invention provides for executing the following processing in a computer: a window detection process, detecting whether a desktop/window state in an operating system meets a window arrangement activation condition, the desktop/window state including the number of desktops in the operating system and the windows in each of the desktops, and receiving a window arrangement inference result; a window arrangement inference process, when the desktop/window state meets the window arrangement activation condition, inputting information about the desktop/window state into the window arrangement inference model and outputting the window arrangement inference result, and arranging the desktop/window state according to the window arrangement inference result.

The embodiment may further include an optimization process, wherein the window detection process detects desktop/window change information after arranging the desktop/window state; the optimization process retrains the window arrangement inference model according to the desktop/window change information.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. The disclosed embodiments are merely illustrative, and the scope of the present invention is not limited thereto.

Embodiment 1 of the present invention is a window arrangement system 1. The window arrangement system 1 is implemented by a computer device including at least one processing device 11. The processing device 11 in the computer device is connected to a memory device 12, and the window arrangement system 1 is operated by executing a program stored in the memory device 12. The memory device 12 can be a memory or a storage device. When the window arrangement system 1 is executed, the program stored in the storage device can be loaded into the memory and then executed by the processing device 11.

For example, the processing device is a hardware logic operation device such as a central processing unit (CPU), a digital signal processor (DSP), a graphics processing unit (GPU), a vision processing unit (VPU), an application specific integrated circuit (ASIC) and/or a field programmable gate array (FPGA), or a combination of the above logic operation devices.

Memory is a storage device that temporarily stores data. Specific examples are static random access memory (SRAM) or dynamic random access memory (DRAM).

A storage device is a memory device that stores data, and a specific example is a solid state drive (SSD). A storage device can also be a hard disk drive (HDD), a flash memory, etc.

The transmission interface is a hardware interface for transmitting data between hardware such as processing devices, memory and storage devices. Specific examples include buses, signal cables, wired networks and/or wireless networks.

Referring to FIG. 1, the structure of the embodiment 1 of the present invention is described. The window arrangement system 1 includes a window detection module 111 and a window arrangement inference module 112. The window detection module 111 and the window arrangement inference module 112 are implemented by a processing device (e.g., a central processing unit) of a computer executing software (e.g., one or more applications in a memory device). In addition, the window detection module 111 can obtain information about the desktop and windows in the operating system run by the processing device of the computer. The window detection module 111 and the window arrangement inference module 112 are described in detail below.

The window detection module 111 detects the desktop/window status in the operating system. The desktop/window status is the number of current desktops (including virtual desktops) in the operating system and the windows displayed in each desktop. Among them, the screen content of the desktop and virtual desktop in the operating system can be stored in the memory device 12. In addition, when the processing device 11 is connected to multiple monitors, the screen content of the desktops of multiple monitors can be stored in the operating system. Users can switch different desktops or virtual desktops as needed through the operating system and display them on the monitor. In addition, the desktop/window status can also include other information related to the desktop and window. For example, the desktop and window in the operating system can be identified through an AI model of text and object recognition, thereby obtaining the desktop/window status. Alternatively, for example, the window detection module 111 requests the operating system to return information about the desktop/window status, such as the windows currently opened in each desktop and each virtual desktop, and the position and size of each window, through the operating system's application programming interface (such as winAPI, Linux Kernal API, etc.), as the desktop/window status.

FIG. 2 is a schematic diagram of the desktop/window status. In this embodiment, it is assumed that the size of the desktop and the virtual desktop is 1920x1080 pixels. In FIG. 2, the display operating system includes two desktops, desktop T1 and virtual desktop VT1. Desktop T1 displays windows T1W1, window T1W2, and window T1W3, wherein window T1W3 overlaps window T1W1 and window T1W2. Desktop VT1 only displays window VT1W1. In this embodiment, the desktop/window status only includes windows displayed on the desktop and the virtual desktop. In a variation of Embodiment 1, the desktop/window state may also include reduced windows (windows hidden in the toolbar), such as the reduced windows T1HW1, T1HW2, and T1HW3 in the desktop T1 in FIG. 2, and the reduced window VT1HW1 in the desktop VT1.

When the window detection module 111 detects that the desktop/window state meets the window arrangement activation condition, the window detection module 111 transmits the desktop/window state to the window arrangement inference module 112. The window arrangement activation condition is designed to be the desktop/window state that usually appears when the user arranges the desktop and windows of the working environment, so as to help the user open the virtual desktop and window when the user is ready to arrange the working environment, thereby improving the user's work efficiency. In this embodiment, the window arrangement activation condition is that there is only one virtual desktop and there is only one window program in the virtual desktop, or there are multiple windows overlapping in one desktop or virtual desktop. That is, as long as the desktop/window state meets one of the above two conditions, the desktop/window state is sent to the window layout inference module 112 as input.

The window arrangement inference module 112 includes a window arrangement inference model M. The window arrangement inference model M is an artificial intelligence (AI) model that receives the desktop/window state as input and outputs a window arrangement inference result. The detailed description of training the window arrangement inference model M will be described in detail in the following paragraphs.

When the window arrangement inference module 112 receives information about the desktop/window status, it uses the information about the desktop/window status as the input of the window arrangement inference model M for inference. After the inference is completed, the window arrangement inference model M outputs the window arrangement inference result. The window arrangement inference module 112 returns the window arrangement inference result to the window detection module 111. The window arrangement inference result includes at least one of the following: a list of newly added virtual desktops, a list of newly added window paths, a list of newly added web page URLs, and virtual desktop numbers. Among them, the list of newly added virtual desktops is a list of virtual desktops that are inferred to be added. In addition, each desktop and virtual desktop may also have a desktop number to distinguish different desktops. The list of paths to add windows is the path to the windows that need to be added in each desktop and/or virtual desktop. The list of URLs to add web pages is the URLs of the web pages that need to be added in each desktop and/or virtual desktop.

After receiving the window arrangement inference result, the window detection module 111 rearranges the desktop/window status according to the window arrangement inference result. That is, the window detection module 111 adjusts the number of virtual desktops stored in the memory device 12 and the windows displayed in each desktop (including the virtual desktop) according to the window arrangement inference result. For example, a virtual desktop can be added, a window can be added to each desktop, a window can be moved in each desktop, or a window can be moved between desktops according to the list of newly added virtual desktops, the list of newly added window paths, the list of newly added web page URLs, and the virtual desktop number in the window arrangement inference result. In addition, when a desktop or a virtual desktop includes multiple windows, the wallpaper space can be evenly distributed to each window. Alternatively, if the window arrangement inference model M is trained on the window arrangement method preferred by the user, the window size and position in each desktop can be adjusted according to the window arrangement inference result.

The above describes the first embodiment of the present invention. Next, an example of a method for training the window arrangement inference model M will be described.

The training of the window arrangement inference model M can be performed when the computer device running the window arrangement system 1 is connected to the network. When training the window arrangement inference model M, the processing device 11 takes the names of the currently opened windows and the desktop numbers where the windows are located as inputs, and transmits them to the server (e.g., the original equipment manufacturer server (OEM Server)) through the network for training in the server. Next, in the server, the window arrangement inference model M learns at least one of the following: the association between the opened windows; the source path of the window program and the URL; and the desktop number where the window program and the URL are located.

In detail, the association between open windows refers to whether the opened windows in each desktop and virtual desktop often appear at the same time. The window arrangement inference model M can also learn (or record) the window program and the URL source path so that when the inference result is generated, the window program or web page is opened with the correct path. The window arrangement inference model M can also learn (or record) the desktop number where each window program and URL is located to learn the user's preferred desktop layout, window arrangement, and window size. In addition, the training parameters are not limited to this and can be changed as needed. For example, when training the window arrangement inference model M, each window can also be classified. For example, they can be divided into application windows, web page windows, program file windows, etc.

For example, the window arrangement inference model M can be a convolutional neural network (CNN), a recurrent neural network (RNN), a deep neural network (DNN), a support vector machine (SVM) or a generative adversarial network (GANs), etc., but is not limited to these and can be any type of AI model.

FIG. 3 is a block diagram of an example of the structure of the embodiment 1A of the present invention. Compared with the embodiment 1, the embodiment 1A has an optimization module 113. In this embodiment, the optimization module 113 can also be implemented by a processing device (such as a central processing unit) of a computer executing software (such as one or more applications in the memory device 12). The following describes the differences between the embodiment 1 and the embodiment 1A.

After the window detection module 111 rearranges the desktop/window status according to the window arrangement inference result, the window detection module 111 can continue to detect whether the desktop/window status has been changed. When one or more of the following three situations occur, the window detection module 111 generates desktop/window change information of the changed desktops and windows and transmits it to the optimization module 113 to retrain the window arrangement inference model M: the user closes the window, the user moves the window between the desktops, and the window path opening fails.

When the user closes the window, the optimization module 113 can reduce the weight of the parameters of the closed window (such as program name, web page URL, etc.) in the window arrangement inference model M to reduce the probability of being inferred next time. When the user moves the window, or the window path fails to open, the optimization module 113 can update the data set in the window arrangement inference model M. For example, the new virtual desktop number, file or web page source path and other information are updated to the window arrangement inference model M.

Next, Embodiment 2 of the present invention is described with reference to FIGS. 4-5. Embodiment 2 is a window arrangement method for arranging a desktop and windows in a computer, including a window detection step and a window arrangement inference step. The steps in this embodiment are not intended to limit the window arrangement method of the present invention. As long as it does not deviate from the spirit and scope of the present invention, a person skilled in the art can make changes to the above steps. For example, some steps can be omitted, the order of the steps can be rearranged, or other steps can be added.

Each step in the window layout method of Embodiment 2 can be implemented by a processing device. The processing performed by the processing device to implement each step in the window layout method is a window layout program product.

Figure 4 is a flow chart of Embodiment 2 of the present invention.

First, the processing device starts to detect the desktop/window status in the operating system (step S1). The desktop/window status is the number of desktops currently in the operating system and the windows displayed in each desktop. For example, the processing device can identify the desktops and windows in the operating system through an AI model of text and object recognition, thereby obtaining the desktop/window status. Alternatively, the processing device can request the operating system to return information related to the desktop/window status through the operating system's application programming interface. For example, request the operating system to return the windows currently opened in each desktop and each virtual desktop, as well as the position and size of each window, as the desktop/window status.

After the processing device obtains the desktop/window status, it determines whether the desktop/window status meets the window arrangement activation condition (step S2). When the desktop/window status does not meet the window arrangement activation condition, return to step S1 to continue detecting the desktop/window status. When the desktop/window status meets the window arrangement activation condition, go to step S3. In one embodiment, the window arrangement activation condition is designed to be the desktop/window status that usually appears when the user arranges the desktop and windows of the working environment, for example, there is only one virtual desktop and there is only one window program in the virtual desktop, or there are multiple windows overlapping in one of the desktops or virtual desktops.

When the desktop/window status meets the window arrangement activation condition, the processing device inputs the information about the desktop/window status into the window arrangement inference model M (step S3). For example, the window name or URL identified by the object recognition AI module is input into the window arrangement inference model M. Alternatively, the information about the desktop/window status obtained through the application program interface of the operating system is input into the window arrangement inference model M.

Next, the processing device obtains the window arrangement inference result output by the window arrangement inference model M (step S4).

After the processing device obtains the window arrangement inference result output by the window arrangement inference model M, it rearranges the desktop/window status according to the window arrangement inference result (step S5).

Figure 5 is a flow chart of an example of rearranging the desktop/window state of embodiment 2 of the present invention. In this embodiment, the window arrangement inference result includes the desktop number, the program path in each desktop, and the focus window in each desktop.

First, the processing device obtains the window arrangement inference result output by the window arrangement inference model M (step S4), and then obtains the height and width of the screen resolution (step S51). In this embodiment, the screen resolution is, for example, 1920×1080 pixels.

Next, the processing device obtains the number of windows in each desktop (step S52), and divides the height and width of the screen resolution by the number of windows to calculate the size of each window (step S53).

Next, the processing device opens the corresponding windows in each desktop, arranges the windows side by side, and places the focus window in the center (step S54).

The method of rearranging the desktop/window state described in FIG. 5 is only an example. The method of rearranging the desktop/window state can be changed as needed. For example, the window size arranged by the user can be recorded in the window arrangement inference model M, and the window size previously arranged by the user can be used when the window is opened. Alternatively, the window arrangement inference model M can be used to calculate the weight of each window, and the position of the window can be arranged according to the window weight.

FIG. 6 is a flow chart of the optimization step of Example 2A of the present invention. Compared with Example 2, Example 2A has an additional optimization step. The differences between Example 2 and Example 2A are described below.

After rearranging the desktop/window status in step S5, the processing device may continue to detect whether the desktop/window status has been changed (step S6).

When one of the following three situations occurs, the processing device generates desktop/window change information for the changed desktops and windows, and retrains the window arrangement inference model M according to the desktop/window change information (step S7): the user closes the window, the user moves the window between desktops, and the window path opening fails.

When the user closes the window, the processing device reduces the weight of the parameters of the closed window (such as program name, web page URL, etc.) in the window arrangement inference model M to reduce the probability of being inferred next time. When the user moves the window, or the window path fails to open, the data set in the window arrangement inference model M can be updated. For example, the new virtual desktop number, file or web page source path and other information are updated to the window arrangement inference model M.

If the desktop/window status has not been changed, repeat step S6 to continue the detection.

Although a plurality of embodiments are described separately, the above embodiments may be implemented in combination. Alternatively, one of the plurality of embodiments may be partially implemented. Alternatively, the plurality of embodiments may be partially combined. In addition, the configurations and steps recorded in the above plurality of embodiments may be partially changed as needed.

The method of the present invention, or a specific form or part thereof, may be included in a physical medium in the form of program code, such as a floppy disk, an optical disk, a hard disk, or any other machine-readable (such as computer-readable) storage medium, wherein when the program code is loaded and executed by a machine, such as a computer, the machine becomes a device or system for participating in the present invention. The method, system and device of the present invention may also be transmitted in the form of program code through some transmission media, such as wires or cables, optical fibers, or any transmission form, wherein when the program code is received, loaded and executed by a machine, such as a computer, the machine becomes a device or system for participating in the present invention. When implemented on a general-purpose processor, the program code combines with the processor to provide a unique device that operates similarly to application-specific logic circuits.

The above-described embodiments are recorded to make the present invention easy to understand, and are not intended to limit the present invention. Therefore, the components disclosed in the above-described embodiments are intended to include all design changes or equivalents within the technical scope of the present invention.

1: Window layout system

11: Processing device

12: Memory device

111:Window Detection Module

112:Window layout inference module

113: Optimization module

T1: Desktop

VT1: Virtual Desktop

T1W1, T1W2, T1W3, VT1W1: Window

T1HW1, T1HW2, T1HW3, VT1HW1: Reduce the window

S1~S7: Flowchart steps

S51~S54: Flowchart steps

Figure 1 is a block diagram of a configuration example of embodiment 1 of the present invention. Figure 2 is a schematic diagram of a desktop/window state. Figure 3 is a block diagram of a configuration example of embodiment 1A of the present invention. Figure 4 is a flow chart of embodiment 2 of the present invention. Figure 5 is a flow chart of an example of rearranging the desktop/window state of embodiment 2 of the present invention. Figure 6 is a flow chart of the optimization step of embodiment 2A of the present invention.

S1~S5: Flowchart steps

Claims (10)

A window arrangement system includes: a memory device for storing an application program; and a processing device for executing the application program to implement a window detection module and a window arrangement inference module; wherein: the window detection module detects whether the desktop/window state in an operating system meets a window arrangement activation condition, and the desktop/window state includes the number of desktops in the operating system and the position or size of the windows in each of the desktops; the window arrangement inference module inputs information about the desktop/window state into the window arrangement inference model and outputs a window arrangement inference result when the desktop/window state meets the window arrangement activation condition, and returns the window arrangement inference result to the window detection module to arrange the desktop/window state. A window arrangement system as claimed in claim 1, wherein the window detection module detects desktop/window change information after arranging the desktop/window state, and transmits the desktop/window change information to an optimization module; the optimization module retrains the window arrangement inference model according to the desktop/window change information. A window arrangement system as claimed in claim 2, wherein the window arrangement inference result includes at least one of the following: a list of newly added virtual desktops, a list of newly added window paths, a list of newly added web page URLs, and a virtual desktop number, and the desktop/window change information includes at least one of the following information: information on opening failed windows, information on closing windows, information on newly added windows, information on closing virtual desktops, and information on newly added virtual desktops. A window arrangement system as claimed in claim 3, wherein the window arrangement activation condition is one of the following two: each of the desktops in the operating system contains only one virtual desktop and there is only one window program in the virtual desktop, or there are multiple windows overlapping in any of the desktops in the operating system. A window arrangement method includes: detecting whether a desktop/window state in an operating system meets a window arrangement activation condition, wherein the desktop/window state includes the number of desktops in the operating system and the position or size of the windows in each desktop; when the desktop/window state meets the window arrangement activation condition, inputting information about the desktop/window state into a window arrangement inference model and outputting a window arrangement inference result, and arranging the desktop/window state according to the window arrangement inference result. As in claim 5, the window arrangement method, wherein after the desktop/window state is arranged, a desktop/window change information is detected, and the window arrangement inference model is retrained according to the desktop/window change information. The window arrangement method of claim 6, wherein the window arrangement inference result includes at least one of the following: a list of newly added virtual desktops, a list of newly added window paths, a list of newly added web page URLs, and a virtual desktop number, and the desktop/window change information includes at least one of the following information: information on opening a failed window, information on closing a window, information on adding a new window, information on closing a virtual desktop, and information on adding a new virtual desktop. As in claim 7, the window arrangement method, wherein the window arrangement activation condition is one of the following two: each of the desktops in the operating system contains only one virtual desktop and there is only one window program in the virtual desktop, or there are multiple windows overlapping in any of the desktops in the operating system. A window arrangement program product is used to perform the following processing in a computer: a window detection process, detecting whether a desktop/window state in an operating system meets a window arrangement activation condition, the desktop/window state includes the number of desktops in the operating system and the position or size of the windows in each desktop, and receiving a window arrangement inference result; a window arrangement inference process, when the desktop/window state meets the window arrangement activation condition, inputting information about the desktop/window state into the window arrangement inference model and outputting the window arrangement inference result, and arranging the desktop/window state according to the window arrangement inference result. The window arrangement program product of claim 9 further includes: an optimization process; wherein the window detection process detects a desktop/window change information after arranging the desktop/window state; the optimization process retrains the window arrangement inference model according to the desktop/window change information.

Images