CN113093963B - KM self-adaptive control multi-window display driving control method and system - Google Patents

KM self-adaptive control multi-window display driving control method and system Download PDF

Info

Publication number
CN113093963B
CN113093963B CN202110634273.1A CN202110634273A CN113093963B CN 113093963 B CN113093963 B CN 113093963B CN 202110634273 A CN202110634273 A CN 202110634273A CN 113093963 B CN113093963 B CN 113093963B
Authority
CN
China
Prior art keywords
window
display
control
video signal
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110634273.1A
Other languages
Chinese (zh)
Other versions
CN113093963A (en
Inventor
陆长虹
张亮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Jld Display Expert Co ltd
Original Assignee
Shenzhen Jld Display Expert Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Jld Display Expert Co ltd filed Critical Shenzhen Jld Display Expert Co ltd
Priority to CN202110634273.1A priority Critical patent/CN113093963B/en
Publication of CN113093963A publication Critical patent/CN113093963A/en
Application granted granted Critical
Publication of CN113093963B publication Critical patent/CN113093963B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/38—Information transfer, e.g. on bus
    • G06F13/40—Bus structure
    • G06F13/4063—Device-to-bus coupling
    • G06F13/4068—Electrical coupling
    • G06F13/4072—Drivers or receivers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/147—Digital output to display device ; Cooperation and interconnection of the display device with other functional units using display panels

Abstract

The invention discloses a KM self-adaptive control multi-window display driving control method, which comprises the following steps: processing a KM input control signal to obtain first KM window data, and calculating relative displacement information of an effective window to obtain relative coordinate displacement data of the effective window to send to effective terminal equipment; the effective terminal equipment receives the relative coordinate displacement data of the effective window and synchronously outputs a source video signal in due time; and processing each source video signal to obtain a terminal display video signal, and outputting the terminal display video signal to a display signal output module to be output through a display terminal. In addition, the invention also provides a KM self-adaptive control multi-window display driving control system. The technical scheme provided by the invention realizes the multi-layer combination modes of separation and overlapping of interactive interfaces of a plurality of source video signals and the roaming operation of KM control equipment on one or a plurality of displays.

Description

KM self-adaptive control multi-window display driving control method and system
Technical Field
The invention relates to the technical field of multi-window same-screen display, in particular to a multi-window display driving control method and system based on KM self-adaptive control.
Background
Currently, the mode application of simultaneous operation and control of multiple terminal devices or multiple servers is more and more important, and the switching control and display technology of the multi-terminal device KM control device is further popularized. However, the conventional switching and displaying technology of the multi-terminal KM control device is limited to the control and application in the two-dimensional layer, and when there are three or more windows stacked in the same area, the problem cannot be solved by the existing method of determining the cursor scope according to the window boundary and the cursor moving direction, and the KM control device cannot perform adaptive roaming operation in the multi-window stacked or multi-window composite combined mode on the display, and accordingly roaming operation on the displayed interactive interface of the computer device, or the display mode of the display cannot be controlled by the KM control device through fast operation, which forms an obstacle for the application that needs to perform roaming operation and fast interaction between KM and the display in the multi-window stacked or multi-window composite split screen mode application.
Disclosure of Invention
The invention provides a multi-window display driving control method and system for KM self-adaptive control, and aims to solve the problems that the roaming operation of KM control equipment cannot be realized under the condition that three or more windows have complex modes of same-region lamination and the like in the conventional multi-window same-screen display, and the KM and a display are quickly controlled in an interactive mode and self-adaptive mode.
In order to achieve the above object, the present invention further provides a method for controlling adaptive KM control for multi-window display driving, comprising:
step S10: receiving a KM input control signal through a KM input interface;
step S20: processing the KM input control signal by a first HID codec processor to obtain first KM window data, and sending the first KM window data to a second microcontroller processor; the first KM window data comprises an HID-Scaler control command and an HID-Common control command;
step S30: the second micro-control processor acquires the HID-Common control command in the first KM window data and calculates the relative displacement information of the effective window; specifically, the calculating of the relative displacement information of the effective window includes:
step S310: the second micro control processor acquires the display window multi-dimensional structure state data sent by the first micro control processor;
step S320: the second micro-control processor analyzes the display window multi-dimensional structure state data to obtain the number of display windows, the size of the windows, the positions of the windows, the window hierarchy right and a video signal source corresponding to the windows in a terminal display plane;
step S330: the second micro-control processor analyzes the HID-Common control command to obtain the position information of a current KM cursor relative to a terminal display plane;
step S340: the second micro-control processor combines the display window multi-dimensional structure state data and the position information of the current KM cursor relative to the terminal display plane to perform effective window selection and cursor relative displacement calculation so as to obtain a window table to which the KM cursor belongs;
step S350: calculating to obtain a window with the highest cursor level weight according to the window table to which the KM cursor belongs and the window level relation; the window with the highest cursor level weight is a current effective window;
step S360: calculating the relative displacement information of the cursor relative to the current effective window according to the position information of the current KM cursor relative to the terminal display plane and the position and size relation of the current effective window relative to the terminal display plane;
step S370: determining effective terminal equipment acted by a current cursor according to the mapping relation between the window video signal source and the KM output interface and the current effective window;
step S380: generating second KM window data and effective window control data, and sending the second KM window data and the effective window control data to a second HID coding and decoding processor;
step S390: reading the HID-Scaler control command in the first KM window data to generate HID-Scaler control information, and sending the HID-Scaler control information to a first micro control processor;
step S40: the first microcontroller receives the HID-Scaler control information and updates the multidimensional structural state data of the display window according to the HID-Scaler control information;
step S50: the second HID codec processor encodes and decodes the second KM window data and the effective window control data to obtain effective window relative coordinate displacement data, and sends the effective window relative coordinate displacement data to effective terminal equipment through a KM output interface; the effective terminal equipment is determined according to the mapping relation between the window video signal source and the KM output interface;
step S60: according to the mapping relation between the window video signal source and the KM output interface, the effective terminal equipment receives the relative coordinate displacement data of the effective window and timely and synchronously outputs a source video signal;
step S70: the window management and signal processing module processes each source video signal according to the display window multi-dimensional structure state data acquired from the first micro control processor to obtain a terminal display video signal, and outputs the terminal display video signal to the display signal output module; the terminal display video signals comprise integral window video display signals formed by combining source video signals corresponding to each opened display window on a two-dimensional screen according to display window multi-dimensional structure state data;
step S80: the display signal output module determines the composition form of the terminal display signal and the composition form of the ring-out display signal according to the display control signal and outputs the display signal through the first display terminal and/or the second display terminal; the terminal display signal comprises a terminal display video signal and an OSD interactive interface signal; the loop-out display signal comprises a terminal display signal or any source video signal.
Further, the KM adaptively controlled multi-window display driving control method further includes:
modifying the display window multidimensional structure state data through an OSD interaction control module so as to realize data adjustment of KM control equipment and a display window;
and the first micro control processor returns the modification result of the display window multi-dimensional structure state data.
Furthermore, the display window multidimensional structure state data comprises the number of display windows, display window parameters, the hierarchical right of the window and a window video signal source; the display window multi-dimensional structure state data is updated and stored by the first micro-control processor.
Further, the hierarchical weight of the window includes a window effective value, and the window with the largest window effective value in all windows is the current effective window; the display window parameters include any one or more of window resolution, window position coordinates, and window shape.
Meanwhile, the invention provides a KM self-adaptive control multi-window display driving control system, which comprises:
the Scaler display driving module and the KM control processing module;
the Scaler display driving module comprises a first micro control processor, a first communication interface, a window management and signal processing module, a video signal input module and a display signal output module; the first communication interface, the window management and signal processing module, the video signal input module and the display signal output module are respectively connected with the first micro control processor; the video signal input module and the display signal output module are respectively connected with the window management and signal processing module;
the KM control processing module comprises a second micro-control processor, a second communication interface, a KM input module and a KM switching output module; the second communication interface, the KM input module and the KM switching output module are respectively connected with the second micro-control processor;
the Scaler display driving module and the KM control processing module are connected through the first communication interface and the second communication interface.
Preferably, the video signal input module includes a plurality of video signal input interfaces, and the video signal input interfaces are used for detecting and receiving source video signals; the window management and signal processing module is connected with the first micro control processor to acquire display window multi-dimensional structure state data, and processes each source video signal according to the display window multi-dimensional structure state data to obtain a terminal display video signal.
Preferably, the KM switching output module includes a second HID codec processor and a plurality of KM output interfaces; the second HID codec processor is used for generating relative coordinate displacement data according to second KM window data and effective window control data sent by the second microcontroller processor; each KM output interface corresponds to each video signal input interface to be connected with the same terminal equipment, and the KM output interface is used for being connected with the terminal equipment and sending the relative coordinate displacement data to the terminal equipment.
Preferably, the KM input module comprises a first HID codec processor and at least one KM input interface for connecting to a KM control device for receiving KM input control signals; the first HID codec processor is configured to process the KM input control signal to obtain first KM window data; the first KM window data includes an HID-Scaler control command and an HID-Common control command.
Preferably, the Scaler display driving module further comprises an OSD interaction control module, and the OSD interaction control module is respectively connected with the first micro-control processor and the display signal output module; the OSD interaction control module is used for modifying the display window multi-dimensional structure state data in the first micro-control processor so as to realize data adjustment of KM control equipment and the display window.
Preferably, the display signal output module comprises at least one display window signal output interface; the display window signal output interface is used for being connected with a first display terminal to output any one or more of display video signals output by the window management and signal processing module, the OSD interaction control module and the first micro control processor; the display signal output module also comprises at least one video signal loop-out interface; the video signal loop-out interface is used for being connected with a second display terminal to output the display video signal of the window management and signal processing module or the loop-out source video signal in the video signal input module.
Compared with the prior art, the invention has the following beneficial effects:
the invention provides a KM self-adaptive control multi-window display drive control method and system, which controls multi-window co-screen display through a Scaler display drive module, controls operation signal management of KM control equipment through a KM control processing module, realizes display and KM control equipment operation synchronization through a first communication interface and a second communication interface, realizes multi-layer combination modes of separation and overlapping of interaction interfaces of a plurality of source video signals on one or more displays, realizes roaming operation of the KM control equipment in a plurality of windows of the multi-display, realizes multi-window co-screen display modes, one-to-one correspondence of display windows and source video signals and modification of display window parameters, and when the window combination mode or the exchange window is changed, the source video signal interface corresponding to each window display and the KM control equipment operation are self-adaptively updated according to the position spatial relationship of the display windows, the method and the device have the advantages that the source video signal interaction interface corresponding to the display window is replaced or adjusted without manual adjustment on hardware, meanwhile, after splicing or stacking relation among the display windows is adjusted, the display windows and operation of the KM control equipment are not affected, and roaming operation of the KM control equipment under multi-window stacking or multi-window composite split screen mode application is achieved.
Drawings
Fig. 1 is a schematic structural diagram of a KM adaptive control multi-window display driving control system according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a three-terminal device interactive interface on-screen display and KM control device roaming control of a KM adaptive-control multi-window display-driven control system according to a second embodiment of the present invention;
FIG. 3 is a schematic view of the display window of FIG. 2;
fig. 4 is a flowchart illustrating a multi-window display driving control method of KM adaptive control according to a third embodiment of the present invention;
FIG. 5 is a schematic flowchart of step S30 in FIG. 4;
fig. 6 is a schematic flowchart illustrating OSD interaction control in a KM adaptive-controlled multi-window display driving control method according to a third embodiment of the present invention;
fig. 7 is a schematic diagram of a same-screen display window of an interactive interface of four terminal devices of a KM adaptive-control multi-window display driving control system according to a fourth embodiment of the present invention;
fig. 8 is a schematic diagram of a four-terminal device interactive interface split-screen POP display window of the KM adaptive-control multi-window display drive control system according to the fifth embodiment of the present invention.
In the figure, 100, system; 10. a Scaler display driving module; 11. a first micro-control processor; 12. a first communication interface; 13. a window management and signal processing module; 14. a video signal input module; 141. a first video signal input interface; 142. a second video signal input interface; 143. a third video signal input interface; 144. a fourth video signal input interface; 15. a display signal output module; 151. a display window signal output interface; 152. a video signal loop-out interface; 16. an OSD interaction control module; 20. a KM control processing module; 21. a second micro-controller processor; 22. a second communication interface; 23. a KM input module; 231. a first USB input interface; 232. a second USB input interface; 233. a first HID codec processor; 24. a KM switching output module; 241. a first USB output interface; 242. a second USB output interface; 243. a third USB output interface; 244. a fourth USB output interface; 245. a second HID codec processor; 301. a mouse; 302. a keyboard; 41. a first terminal device; 411. a first video signal output port; 412. a first KM control equipment connector; 42. a second terminal device; 421. a second video signal output port; 422. a second KM control equipment connection port; 43. a third terminal device; 431. a third video signal output port; 432. a third KM control equipment connection port; 50. and displaying the terminal.
In the figure, CKM01, KM input control signals; CKM02, first KM window data; CKM03, HID-Scaler control information; CKM04, second KM window data; CKM05, relative coordinate displacement data; CWIN01, valid window control data; PS01, displaying window multidimensional structure state data; SC01, source video signal; SC02, terminal display video signal; SC03, video signal source auxiliary control signal; SC04, video signal input data; SC05, display control signal; SC 06: the terminal displays the signal; SC 07: looping out a display signal; OC01, modification and adjustment data; OC02, OSD interface signal; OC03, returns the modified result.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example one
Referring to fig. 1, a KM adaptive-controlled multi-window display driving control system 100 (hereinafter, referred to as system 100) according to an embodiment of the present invention includes:
the Scaler display driving module 10 and the KM control processing module 20;
the Scaler display driving module 10 comprises a first micro-control processor 11, a first communication interface 12, a window management and signal processing module 13, a video signal input module 14, a display signal output module 15 and an OSD interaction control module 16; the first communication interface 12, the window management and signal processing module 13, the video signal input module 14, the display signal output module 15 and the OSD interaction control module 16 are respectively connected to the first micro-control processor 11; the video signal input module 14 and the display signal output module 15 are respectively connected with the window management and signal processing module 13; the OSD interaction control module 16 is respectively connected with the first micro-control processor 11 and the display signal output module 15; specifically, the video signal input module 14 includes a plurality of video signal input interfaces, and the video signal input interfaces are used for detecting and receiving a source video signal, for example, in the second embodiment, the video signal input module 14 includes a first video signal input interface 141, a second video signal input interface 142, a third video signal input interface 143, and a fourth video signal input interface 144. The window management and signal processing module 13 is configured to process the source video signal and generate a terminal display video signal. The OSD interaction control module 16 is configured to modify the Display window multidimensional structure state data in the first micro-control processor 11 to implement data adjustment of the KM control device and the Display window, where OSD is On-screen Display (OSD), and may be implemented by a remote controller, a key or IR/KEYPAD input control, where IR/KEYPAD is an infrared (Infra-red, IR) remote controller or a key board (KEYPAD). The KM control equipment comprises any one or more of a key board, a remote controller, an operating rod, a mouse and a keyboard. The display signal output module 15 includes at least one display window signal output interface 151; the display window signal output interface 151 is configured to be connected to a first display terminal to output any one or more of the display video signals output by the window management and signal processing module 13, the OSD interaction control module 16, and the first micro control processor 11. The display signal output module 15 further comprises at least one video signal loop-out interface 152; the video signal loop-out interface 152 is configured to connect to a second display terminal to output the display video signal of the window management and signal processing module 13 or the loop-out source video signal in the video signal input module 14.
The KM control processing module 20 includes a second micro-controller 21, a second communication interface 22, a KM input module 23, and a KM switching output module 24; the second communication interface 22, the KM input module 23 and the KM switching output module 24 are respectively connected to the second micro-control processor 21; the KM input module 23 comprises a first HID codec processor 233 and at least one KM input interface for connecting to a KM control device for receiving KM input control signals; specifically, in the first embodiment, the KM input interface includes a first USB input interface 231 and a second USB input interface 232; the first HID codec processor 233 is configured to process the KM input control signal to obtain first KM window data; the first KM window data includes an HID-Scaler control command and an HID-Common control command. The KM switch output module 24 comprises a second HID codec processor 245 and a plurality of KM output interfaces; the second HID codec processor 245 is configured to generate relative coordinate displacement data according to the second KM window data and valid window control data sent from the second micro controller 21; the KM output interface is used for connecting terminal equipment and sending the relative coordinate displacement data to the terminal equipment; specifically, in the first embodiment, the KM output interfaces include a first USB output interface 241, a second USB output interface 242, a third USB output interface 243, and a fourth USB output interface 244. Specifically, the terminal device includes a computer, a server, a terminal, and any other terminal device. Specifically, when the terminal device is connected, the KM switching output module 24 and the video signal input module 14 are respectively and correspondingly connected with the KM control device and the video signal output of the terminal device, and a mapping relationship M01 between the window video signal source and the KM output interface is established through the KM output interface of the KM switching output module 24 and the video signal input interface of the video signal input module 14; the mapping relation M01 between the window video signal source and the KM output interface includes: a first USB output interface 241 of the KM switching output module 24 in the KM control processing module 20 needs to be connected to the terminal device connected to the first video signal input interface 141 of the Scaler display driving module 10, a second USB output interface 242 of the KM switching output module 24 in the KM control processing module 20 needs to be connected to the terminal device connected to the second video signal input interface 142 of the Scaler display driving module 10, and so on; for example, the first USB output interface 241 is connected to a KM control device of one terminal device, i.e. a keyboard and a mouse, the first video signal input interface 141 is connected to a video signal output port of the same terminal device, similarly, the second USB output interface 242 and the second video signal input interface 142 are connected to another terminal device, and so on.
The Scaler display driving module 10 and the KM control processing module 20 are connected through the first communication interface 12 and the second communication interface 22, specifically in this embodiment, the first communication interface 12 and the second communication interface 22 are Universal Asynchronous Receiver/Transmitter (UART) interfaces.
Example two
Referring to fig. 2 and fig. 3, a connection schematic diagram of the system 100 according to the second embodiment of the present invention for connecting three terminal devices is shown, and is used for displaying the interactive interfaces of the three terminal devices on the same screen and performing the roaming operation under the same screen display by using the same keyboard and mouse (i.e., KM control device). Specifically, the three terminal devices are a first terminal device 41, a second terminal device 42, and a third terminal device 43, respectively, and the terminal devices are computer devices. The first terminal device 41 comprises a first video signal outlet 411 and a first KM control device connector 412, the second terminal device 42 comprises a second video signal outlet 421 and a second KM control device connector 422, and the third terminal device 43 comprises a third video signal outlet 431 and a third KM control device connector 432. Correspondingly, the first video signal input interface 141 in the video signal input module 14 of the system 100 is connected to the first video signal output port 411, the second video signal input interface 142 is connected to the second video signal output port 421, and the third video signal input interface 143 is connected to the third video signal output port 431; the first USB output interface 241 is connected to the first KM control device connection port 412, the second USB output interface 242 is connected to the second KM control device connection port 422, and the third USB output interface 243 is connected to the third KM control device connection port 432. The first USB input interface 231 is connected to the mouse 301 and the keyboard 302, and in this embodiment, the first USB input interface 231 includes two USB ports for connecting the mouse 301 and the keyboard 302, respectively. Further, the display window signal output interface 151 is also externally connected to the display terminal 50, so as to output the video signal to the display terminal 50 for displaying.
Fig. 3 shows multiple windows displayed on the display terminal 50, where the display window 1 correspondingly displays an interactive interface of the first terminal device 41, the display window 2 correspondingly displays an interactive interface of the second terminal device 42, and the display window 3 correspondingly displays an interactive interface of the third terminal device 43, and when a cursor of the mouse 301 switches among the display window 1, the display window 2, and the display window 3, the switching among the interactive interface of the first terminal device 41, the interactive interface of the second terminal device 42, and the interactive interface of the third terminal device 43 is automatically achieved.
EXAMPLE III
Referring to fig. 1, fig. 4, and fig. 5, a third embodiment of the present invention provides a method for controlling a multi-window display driver with KM adaptive control, where a system 100 implements KM adaptive multi-window display control by applying the method, specifically, when the system 100 is started, system initialization is performed, and a first micro-control processor 11 reads a first system setting in a memory; the first system settings comprise display window multidimensional structure state data PS 01; after the first micro control processor 11 is started, the display window multi-dimensional structure state data PS01 is sent to the second micro control processor 21 and the window management and signal processing module 13; the second micro controller processor 21 reads the second system setting in the memory and receives the display window multi-dimensional structure state data PS 01.
Specifically, the KM adaptive control multi-window display drive control method includes:
step S10: receiving a KM input control signal CKM01 via a KM input interface; the KM input module 23 of the system 100 is connected to the operator's KM control equipment for receiving KM input control signals CKM01 input by the KM control equipment. Specifically, in this embodiment, the KM input control signal CKM01 is an operation signal of a keyboard and a mouse of the KM control device.
Step S20: processing the KM input control signal CKM01 by a first HID codec processor 233 to obtain first KM window data CKM02, and sending the first KM window data CKM02 to a second micro controller processor 21; the first KM window data CKM02 comprises an HID-Scaler control command and an HID-Common control command; wherein the HID is a Human Interface Device (HID); the HID-Scaler control command is combined to generate the HID-Scaler control information CKM03, and the HID-Scaler control information CKM03 is used for controlling the Scaler display driving module 10 to switch the display multi-window mode and adjust or exchange the corresponding display window video signal source. The HID-Common control command is a general control command for the KM control device, which is used in combination with the display window multi-dimensional structure status data PS01 to generate the second KM window data CKM04 and the valid window control data CWIN 01.
Step S30: the second micro-controller 21 obtains the HID-Common control command in the first KM window data CKM02 and calculates the relative displacement information of the valid window to generate second KM window data CKM04 and valid window control data CWIN01, and sends the second KM window data CKM04 and the valid window control data CWIN01 to the second HID codec processor 245; meanwhile, reading the HID-Scaler control command in the first KM window data CKM02 to generate HID-Scaler control information CKM03, and sending the HID-Scaler control information CKM03 to the first micro-control processor 11; specifically, the calculating of the relative displacement information of the effective window includes:
step S310: the second micro-control processor 21 acquires the display window multi-dimensional structure state data PS01 sent by the first micro-control processor 11;
step S320: the second micro-control processor 21 analyzes the display window multi-dimensional structure state data PS01 to obtain the number of display windows, the size of the windows, the positions of the windows, the window hierarchy weight and the video signal sources corresponding to the windows in the terminal display plane;
step S330: the second micro controller 21 parses the HID-Common control command to obtain a current position information Pi01 of the KM cursor with respect to the terminal display plane;
step S340: the second micro-control processor 21 combines the display window multi-dimensional structure state data PS01 and the position information Pi01 of the current KM cursor relative to the terminal display plane to perform effective window selection and cursor relative displacement calculation, so as to obtain a window table LS01 to which the KM cursor belongs;
step S350: calculating to obtain a window with the highest cursor level weight according to the window table LS01 to which the KM cursor belongs and the window level relation; the window with the highest cursor level weight is a current effective window EW 01;
step S360: calculating the relative displacement information of the cursor relative to the current effective window EW01 according to the position information Pi01 of the current KM cursor relative to the terminal display plane and the position and size relation of the current effective window EW01 relative to the terminal display plane;
step S370: determining effective terminal equipment acted by a current cursor according to a mapping relation M01 between a window video signal source and a KM output interface and the current effective window EW 01;
step S380: generating second KM window data CKM04 and valid window control data CWIN01 and sending said second KM window data CKM04 and said valid window control data CWIN01 to a second HID codec processor 245;
step S390: reading the HID-Scaler control command in the first KM window data CKM02 to generate HID-Scaler control information CKM03, and sending the HID-Scaler control information CKM03 to the first micro-control processor 11;
step S40: the first micro-control processor 11 receives the HID-Scaler control information CKM03 and updates the display window multi-dimensional structure status data PS01 according to the HID-Scaler control information CKM 03;
step S50: the second HID codec processor 245 performs codec processing on the second KM window data CKM04 and the valid window control data CWIN01 to obtain valid window relative coordinate displacement data CKM05, and sends the valid window relative coordinate displacement data CKM05 to the valid terminal device through a KM output interface; the effective terminal equipment is determined according to the mapping relation between the window video signal source and the KM output interface;
step S60: according to the mapping relation between the window video signal source and the KM output interface, the effective terminal equipment receives the relative coordinate displacement data CKM05 of the effective window and timely and synchronously outputs a source video signal SC 01; the source video signal SC01 is input via the video signal input interface of the system 100;
step S70: the window management and signal processing module 13 processes each source video signal SC01 according to the display window multi-dimensional structure state data PS01 acquired from the first micro controller 11 to obtain a terminal display video signal SC02, and outputs the terminal display video signal SC02 to the display signal output module 15; the terminal display video signals SC02 comprise an integral window video display signal formed by combining a source video signal SC01 corresponding to each opened display window on a two-dimensional screen according to display window multi-dimensional structure state data PS 01;
step S80: the display signal output module 15 determines the composition form of the terminal display signal SC06 and the composition form of the loop-out display signal SC07 according to the display control signal SC05, and outputs the terminal display signal and the loop-out display signal through the first display terminal and/or the second display terminal; the terminal display signal SC06 comprises a terminal display video signal SC02 and an OSD interactive interface signal OC 02; the loop-out display signal SC07 includes the terminal display signal SC06 or any source video signal SC 01. Specifically, when the display signal output module 15 outputs a display signal to a first display terminal and/or a second display terminal connected thereto, the control of the display output mode, the display parameter and the loop-out mode is realized by a display control signal SC05 sent by the first micro-control processor 11, specifically, the first display terminal is connected to the display window signal output interface 151, the second display terminal is connected to the video signal loop-out interface 152, the display control signal SC05 determines the composition form of the terminal display signal SC06 and the composition form of the loop-out display signal SC07, and the terminal display signal SC06 includes a terminal display video signal SC02 and an OSD interactive interface signal OC 02; the loop-out display signal SC07 includes a terminal display signal SC06 and any source video signal SC 01.
At any moment, the Scaler display driving module 10 may select to display the content of the source video signal SC01 displayed in the effective window corresponding to each unit pixel, and meanwhile, the KM control processing module 20 determines, according to the display window multidimensional structural state data PS01, whether the highest window level weight of the pixel where the current KM cursor is located changes in the moving process of the KM cursor position, that is, whether the effective window at the right time of the KM cursor changes; if the effective window changes along with the movement of the KM cursor, the KM control processing module 20 may recalculate the relative displacement of the KM cursor with respect to the new effective window, and control the KM cursor to implement the relative displacement control in the currently corresponding effective window, so that the KM cursor controls the new effective terminal device working desktop, and the seamless roaming switching of the KM cursor is implemented in the multi-window display scene in the display plane space;
specifically, the display window multidimensional structure state data PS01 includes the number of display windows, display window parameters, the hierarchical right of the window, and a window video signal source; the hierarchical weight of the window comprises a window effective value, and the window with the largest window effective value in all windows is a current effective window; the display window parameters include any one or more of window resolution, window position coordinates, and window shape. The display window multi-dimensional structure state data PS01 is updated and stored by the first micro controller processor 11; specifically, when the first micro-controller 11 is initialized and started, the system setting is read from the memory, the system setting includes the display window multidimensional structure status data PS01, and when the first micro-controller 11 receives the HID-Scaler control information CKM03, the modification and adjustment data OC01 sent by the OSD interaction control module 16, and the source video signal status SC04 sent by the window management and signal processing module 13, the first micro-controller 11 updates the display window multidimensional structure status data PS01 according to the above information; when the first micro controller 11 starts loading the display window multi-dimensional structure status data PS01 or updating the display window multi-dimensional structure status data PS01, the first micro controller 11 sends the display window multi-dimensional structure status data PS01 to the second micro controller 21 and the window management and signal processing module 13.
Specifically, in the third embodiment of the present invention, a display window cursor positioning method with a multidimensional structure is used to calculate an effective working window, and the display window cursor positioning method is implemented through window high-level priority selection and a cursor relative position according to the display window multidimensional structure state data PS01, specifically, in any unit pixel on a display window, the Scaler display driving module 10 selects a display window with the highest hierarchical weight of the window at any time, and the KM control processing module 20 controls a cursor of the KM control device to correspondingly act on a terminal device working desktop corresponding to the display window with the highest hierarchical weight of the window as the effective working window, the effective working window of the KM control device changes with the hierarchical weight of the window at the cursor position, when the cursor moves and the hierarchical weight of the window at the position corresponding to the cursor changes, the KM control processing module 20 may recalculate the relative position of the cursor with respect to the new valid working window, so as to implement seamless roaming of the cursor of the KM control device in a multi-window display scene in the display plane space; if there is only one display window in the display plane space, the window with the highest hierarchical weight of all the pixels in the display window range is the display window. If two display windows WinA and WinB exist on the display plane space and the WinA and WinB do not have an overlapping interval, the highest level weight windows of the pixels in the two display windows WinA and WinB are respectively WinA and WinB; if WinA and WinB have an overlap interval Sab, and WinA hierarchical weight is 1, WinB hierarchical weight is 2, the pixel display window WinB in the overlap interval Sab is the biggest, overlap interval Sab displays the video signal content of WinB, KM cursor correspondingly acts on the working terminal equipment corresponding to WinB display window, and the pixel on the non-overlap area of WinA and WinB display windows respectively takes the window as the highest hierarchical weight, and likewise, KM cursor correspondingly acts on the working terminal equipment corresponding to the display window, thus realizing that KM input equipment can roam and control the working desktop of single or multiple terminal equipment in single display window or various multiple window display modes, and KM roaming behaviors of all display windows in the display plane can be self-adapted without being influenced no matter how the multiple window mode of the display terminal is set or changed.
Specifically, the display terminal multi-window display mode is conveniently set or adjusted by the KM control device, when a new multi-window display mode is set or adjusted, the Scaler display drive module 10 updates the new window display mode in due time and updates the display window multi-dimensional structure state data PS01 back to the KM control processing module 20, the KM control processing module 20 obtains information such as the number of new Scaler display windows, the resolution and the position of each window, the hierarchical right of the window, the video signal source displayed by the window according to the new display window multi-dimensional structure state data PS01, and the KM roaming behavior can be self-adaptive by combining the display window cursor positioning method with the multi-dimensional structure. The method comprises the steps that a KM control device is used for conveniently and rapidly setting or exchanging video signal sources of each display window, after the video signal sources of the display windows are set or exchanged, a Scaler display driving module 10 timely updates the corresponding window video signal sources and updates display window multi-dimensional structure state data PS01 back to the KM control processing module 20, the KM control processing module 20 can acquire information such as video signal sources displayed by new windows according to the new display window multi-dimensional structure state data PS01, and KM roaming behaviors can be self-adapted by combining the display window cursor positioning method with the multi-dimensional structure. The method comprises the steps that a KM control device is used for conveniently and rapidly setting or exchanging video signal sources of each display window, after the video signal sources of the display windows are set or exchanged, a Scaler display driving module 10 timely updates the corresponding window video signal sources and updates display window multi-dimensional structure state data PS01 back to the KM control processing module 20, the KM control processing module 20 can acquire information such as video signal sources displayed by new windows according to the new display window multi-dimensional structure state data PS01, and KM roaming behaviors can be self-adapted by combining the display window cursor positioning method with the multi-dimensional structure.
Referring to fig. 1 and fig. 6, a third embodiment of the present invention further provides an OSD interaction control method in the KM adaptive control multi-window display driving control method, and the OSD interaction control module 16 modifies the display window multi-dimensional structure state data PS 01. Specifically, the OSD interaction control method includes:
step S210: the OSD interaction control module 16 modifies the display window multidimensional structure state data PS01 to realize data adjustment between KM control equipment and a display window; specifically, the OSD interaction control module 16 modifies the display window multidimensional structure state data PS01 in the first micro-controller processor 11 by modifying and adjusting the data OC 01.
Step S220: the first micro controller processor 11 returns the modified result of the display window multi-dimensional structure state data PS 01; specifically, the display window multidimensional structure state data PS01 includes the number of display windows, display window parameters, the hierarchical right of the window and the window video signal source, so that the data is modified by the OSD interaction control module 16, thereby implementing setting or exchange of the display window combination mode and the display window signal source, setting a new display window combination mode or modifying the parameters of each display window, the hierarchical right of the window, and the like; the first micro controller processor 11 sends a return modification result OC03 to the OSD interaction control module 16.
Further, the first micro-control processor 11 sends the auxiliary control signal SC03 of the video signal source to the video signal input module 14, and the auxiliary control signal SC03 notifies the corresponding video signal input interface to re-read the signal to adjust the input parameters of the source video signal SC01 when the multi-window mode of the display terminal changes; for example, when the multi-window display mode of the display terminal is changed by the operation of the KM control device or the OSD interaction control module 16, and the resolution of the relevant window may change, the terminal device connected to the video signal input interface is notified of synchronous adaptive output by the video signal source auxiliary control signal SC03, so as to ensure that the scale of the displayed picture in each display window is optimal.
From the above description of the embodiments, it is clear to those skilled in the art that the method of the above embodiments can be implemented by software plus a necessary hardware platform. With this understanding in mind, the KM adaptively controlled multi-window display driver control method according to the present invention may be embodied in the form of a software product, which may be stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk, etc.), and includes instructions for causing a first micro-control processor and a second micro-control processor to execute the steps of the KM adaptively controlled multi-window display driver control method.
Example four
Referring to fig. 7, a schematic diagram of an on-screen display window of an interactive interface of the system 100 connected to four terminal devices according to the fourth embodiment of the present invention is shown; the main difference from the second embodiment and the third embodiment is that the video signal input module 14 of the Scaler display driving module 10 is connected with four terminal devices, the KM switching output module 24 of the KM control processing module 20 is also connected with KM control devices of the four terminal devices, the same KM adaptive control-based multi-window display driving control method realizes the same screen display of the interactive interface of the four terminal devices and the roaming operation of KM, and the display window 1, the display window 2, the display window 3 and the display window 4 respectively correspond to the video signal output of the four terminal devices, which is the same as that in the second embodiment and the third embodiment, and is not repeated herein.
EXAMPLE five
Please refer to fig. 8, which is a schematic diagram of a split-screen POP display window of four terminal devices connected to the system 100 according to the fifth embodiment of the present invention, and is different from the fourth embodiment in that the display window 1, the display window 2, the display window 3, and the display window 4 are in a four-split-screen POP stacked mode, the display window 1 and the display window 3 are distributed on the display screen in a non-intersecting manner, the display window 2 and the display window 4 are respectively stacked thereon, it is assumed that the mouse starts to operate the interactive interface of the fourth terminal device in the display window 4 interval, when the mouse slides to the upper left and passes through the upper left boundary, the mouse cursor automatically roams to the interactive interface interval of the second terminal device in the display window 2, at this time, the mouse and the keyboard can operate the second terminal device, and then the mouse cursor can roam to the display window 1 after the mouse slides to the upper boundary and slides over the upper left boundary, at this time, the mouse and the keyboard can operate the first terminal device, and similarly, when the mouse continues to slide to the right side, the mouse cursor may roam to an interactive interface region of the third terminal device of the display window 3, and at this time, the mouse and the keyboard can operate the third terminal device.
Compared with the prior art, the invention provides a KM self-adaptive control multi-window display drive control method and system, which controls multi-window co-screen display through a Scaler display drive module, controls operation signal management of KM control equipment through a KM control processing module, realizes display and KM control equipment operation synchronization through a first communication interface and a second communication interface, realizes multi-layer combination modes of separation and overlapping of interaction interfaces of a plurality of source video signals on one or more displays, realizes roaming operation of the KM control equipment in a plurality of windows, realizes multi-window co-screen display modes, one-to-one correspondence of display windows and source video signals and modification of display window parameters, and when the window combination mode or the exchange window is changed, the source video signal interface corresponding to each window display and the KM control equipment operation are self-adaptively updated according to the position spatial relationship of the display windows, the method and the device have the advantages that the source video signal interaction interface corresponding to the display window is replaced or adjusted without manual adjustment on hardware, meanwhile, after splicing or stacking relation among the display windows is adjusted, the display windows and operation of the KM control equipment are not affected, and roaming operation of the KM control equipment under multi-window stacking or multi-window composite split screen mode application is achieved.
While the foregoing is directed to embodiments of the present invention, it will be understood by those skilled in the art that various changes may be made without departing from the spirit and scope of the invention.

Claims (4)

1. A KM self-adaptive control multi-window display driving control method is characterized by comprising the following steps:
step S10: receiving a KM input control signal through a KM input interface;
step S20: processing the KM input control signal by a first HID codec processor to obtain first KM window data, and sending the first KM window data to a second microcontroller processor; the first KM window data comprises an HID-Scaler control command and an HID-Common control command;
step S30: the second micro-control processor acquires the HID-Common control command in the first KM window data and calculates the relative displacement information of the effective window; specifically, the calculating of the relative displacement information of the effective window includes:
step S310: the second micro control processor acquires the display window multi-dimensional structure state data sent by the first micro control processor;
step S320: the second micro-control processor analyzes the display window multi-dimensional structure state data to obtain the number of display windows, the size of the windows, the positions of the windows, the window hierarchy right and a video signal source corresponding to the windows in a terminal display plane;
step S330: the second micro-control processor analyzes the HID-Common control command to obtain the position information of a current KM cursor relative to a terminal display plane;
step S340: the second micro-control processor combines the display window multi-dimensional structure state data and the position information of the current KM cursor relative to the terminal display plane to perform effective window selection and cursor relative displacement calculation so as to obtain a window table to which the KM cursor belongs;
step S350: calculating to obtain a window with the highest cursor level weight according to the window table to which the KM cursor belongs and the window level relation; the window with the highest cursor level weight is a current effective window;
step S360: calculating the relative displacement information of the cursor relative to the current effective window according to the position information of the current KM cursor relative to the terminal display plane and the position and size relation of the current effective window relative to the terminal display plane;
step S370: determining effective terminal equipment acted by a current cursor according to the mapping relation between the window video signal source and the KM output interface and the current effective window;
step S380: generating second KM window data and effective window control data, and sending the second KM window data and the effective window control data to a second HID coding and decoding processor;
step S390: reading the HID-Scaler control command in the first KM window data to generate HID-Scaler control information, and sending the HID-Scaler control information to a first micro control processor;
step S40: the first microcontroller receives the HID-Scaler control information and updates the multidimensional structural state data of the display window according to the HID-Scaler control information;
step S50: the second HID codec processor encodes and decodes the second KM window data and the effective window control data to obtain effective window relative coordinate displacement data, and sends the effective window relative coordinate displacement data to effective terminal equipment through a KM output interface; the effective terminal equipment is determined according to the mapping relation between the window video signal source and the KM output interface;
step S60: according to the mapping relation between the window video signal source and the KM output interface, the effective terminal equipment receives the relative coordinate displacement data of the effective window and timely and synchronously outputs a source video signal;
step S70: the window management and signal processing module processes each source video signal according to the display window multi-dimensional structure state data acquired from the first micro control processor to obtain a terminal display video signal, and outputs the terminal display video signal to the display signal output module; the terminal display video signals comprise integral window video display signals formed by combining source video signals corresponding to each opened display window on a two-dimensional screen according to display window multi-dimensional structure state data;
step S80: the display signal output module determines the composition form of the terminal display signal and the composition form of the ring-out display signal according to the display control signal and outputs the display signal through the first display terminal and/or the second display terminal; the terminal display signal comprises a terminal display video signal and an OSD interactive interface signal; the loop-out display signal comprises a terminal display signal or any source video signal.
2. The KM adaptively controlled multi-window display driver control method according to claim 1, further comprising:
modifying the display window multidimensional structure state data through an OSD interaction control module so as to realize data adjustment of KM control equipment and a display window;
and the first micro control processor returns the modification result of the display window multi-dimensional structure state data.
3. The KM adaptively controlled multi-window display driving control method according to claim 1, wherein the display window multi-dimensional structure state data includes the number of display windows, display window parameters, the hierarchical right of the window, and a window video signal source; the display window multi-dimensional structure state data is updated and stored by the first micro-control processor.
4. The KM adaptively controlled multi-window display driving control method according to claim 3, wherein the hierarchical weight of the window includes a window effective value, and the window with the largest window effective value among all windows is a current effective window; the display window parameters include any one or more of window resolution, window position coordinates, and window shape.
CN202110634273.1A 2021-06-08 2021-06-08 KM self-adaptive control multi-window display driving control method and system Active CN113093963B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110634273.1A CN113093963B (en) 2021-06-08 2021-06-08 KM self-adaptive control multi-window display driving control method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110634273.1A CN113093963B (en) 2021-06-08 2021-06-08 KM self-adaptive control multi-window display driving control method and system

Publications (2)

Publication Number Publication Date
CN113093963A CN113093963A (en) 2021-07-09
CN113093963B true CN113093963B (en) 2021-10-22

Family

ID=76665904

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110634273.1A Active CN113093963B (en) 2021-06-08 2021-06-08 KM self-adaptive control multi-window display driving control method and system

Country Status (1)

Country Link
CN (1) CN113093963B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106528012A (en) * 2015-09-10 2017-03-22 微窗科技股份有限公司 Seamless switching method and system for computer hosts
CN110007888A (en) * 2019-04-16 2019-07-12 深圳市湘凡科技有限公司 A kind of single screen shared control system and shared device

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7627186B2 (en) * 2003-11-14 2009-12-01 Avocent California Corporation Compression systems and methods
US9665525B2 (en) * 2014-06-09 2017-05-30 High Sec Labs Ltd. Multi-host docking device
CN206270889U (en) * 2016-12-16 2017-06-20 广东视腾电子科技有限公司 The KVM switcher that a kind of many computers are split with picture
CN108845783B (en) * 2018-08-01 2024-01-09 广州魅视电子科技有限公司 Multi-window combined screen display method and system based on KVM device
CN111078020A (en) * 2019-12-09 2020-04-28 北京小鸟科技股份有限公司 Multi-specification display screen layout system and method for KVM system and coordinate mapping method
CN112035085A (en) * 2020-11-05 2020-12-04 深圳市嘉利达专显科技有限公司 KM self-adaptive control multi-window same-screen display

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106528012A (en) * 2015-09-10 2017-03-22 微窗科技股份有限公司 Seamless switching method and system for computer hosts
CN110007888A (en) * 2019-04-16 2019-07-12 深圳市湘凡科技有限公司 A kind of single screen shared control system and shared device

Also Published As

Publication number Publication date
CN113093963A (en) 2021-07-09

Similar Documents

Publication Publication Date Title
CN103841442B (en) A kind of method for controlling air mouse, air mouse and Set Top Box
US9152253B2 (en) Remote control method, remote control apparatus, and display apparatus
US9489066B2 (en) Computer system including mouse device with adjustable DPI based on screen size and operating method thereof
CN102591493B (en) Mouse cursor synchronization method for internet protocol K virtual machine (IPKVM) system
JPH0922339A (en) Remote controller
CN108845783B (en) Multi-window combined screen display method and system based on KVM device
CN110990089B (en) IDV-based double-screen double-desktop intelligent system and method
US20110066971A1 (en) Method and apparatus for providing application interface portions on peripheral computing devices
CN112181342B (en) Display method, display device, electronic equipment and storage medium
US20120120320A1 (en) System and method for on-the-fly key color generation
CN112230836B (en) Object moving method and device, storage medium and electronic device
US11120617B2 (en) Method and apparatus for switching panoramic scene
US20180293042A1 (en) Method of Switching Display of a Terminal and a Terminal
WO2014201841A1 (en) Editing method of the three-dimensional shopping platform display interface for users
JP2013131990A (en) Information processor and program
US10929083B2 (en) Resource sharing device and control method thereof
TWI497400B (en) Method of auto-recognizing for cursor in monitors
CN102957875A (en) Image processing method, device and system
JP2024509069A (en) Game screen display methods and devices, electronic devices, and computer programs
CN113093963B (en) KM self-adaptive control multi-window display driving control method and system
JP2003316490A (en) Remote control system and method thereof
CN112035085A (en) KM self-adaptive control multi-window same-screen display
WO2022184029A1 (en) Collage display system and data processing method thereof
CN211654269U (en) Display control circuit of double-screen LED all-in-one machine and double-screen LED all-in-one machine
RU2566975C2 (en) System, method and computer programme for operating multiple computing devices

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant