CN116501201A - Software development split-screen processing method and system - Google Patents
Software development split-screen processing method and system Download PDFInfo
- Publication number
- CN116501201A CN116501201A CN202210083843.7A CN202210083843A CN116501201A CN 116501201 A CN116501201 A CN 116501201A CN 202210083843 A CN202210083843 A CN 202210083843A CN 116501201 A CN116501201 A CN 116501201A
- Authority
- CN
- China
- Prior art keywords
- virtual
- screen
- physical
- mouse
- keyboard
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims description 23
- 238000000034 method Methods 0.000 claims abstract description 55
- 238000012545 processing Methods 0.000 claims abstract description 46
- 238000005516 engineering process Methods 0.000 claims abstract description 18
- 230000001360 synchronised effect Effects 0.000 claims abstract description 15
- 238000004590 computer program Methods 0.000 claims description 13
- 238000010276 construction Methods 0.000 claims description 7
- 230000009471 action Effects 0.000 claims description 5
- 238000013507 mapping Methods 0.000 claims description 5
- 238000011161 development Methods 0.000 abstract description 17
- 230000008569 process Effects 0.000 abstract description 10
- 238000010586 diagram Methods 0.000 description 9
- 230000008859 change Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 239000011521 glass Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/02—Input arrangements using manually operated switches, e.g. using keyboards or dials
- G06F3/0202—Constructional details or processes of manufacture of the input device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/0354—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
- G06F3/03543—Mice or pucks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention provides a method and a system for processing split screens in software development, wherein the method comprises the following steps: constructing a virtual scene required by software development split-screen processing based on a Unity technology; displaying a virtual mouse keyboard in the virtual scene, and synchronizing information of the virtual mouse keyboard and the physical mouse keyboard; displaying a virtual screen window in the virtual scene, and synchronizing output information of the virtual screen window with output information of a physical screen of the software development terminal; switching among a plurality of display layers and synchronous movement across screen areas are performed on the virtual scene for which information synchronization has been completed. According to the invention, the VR is applied to software development, so that content information which can be obtained by a developer in the development process is not limited by a development terminal screen, and the virtual scene is synchronized with the software development terminal information, so that the developer can use the VR to perform software development on the basis of keeping the existing operation habit to the greatest extent, and the learning cost required by the developer is reduced.
Description
Technical Field
The invention relates to the technical field of software development, in particular to a split screen processing method and a split screen processing system for software development.
Background
With the increasing complexity of software development, developers often need to switch among multiple software in a software development process, and a single software can open multiple copies for use, for example, the developers may open several or even tens of engineering projects to develop simultaneously when using a code editor.
Existing solutions generally fall into two categories.
The screen is divided into a plurality of block areas by conventional split screen software, and the window to be displayed is moved to the designated screen area position for placement.
Two or more display screens are connected by a host device by means of a screen splitter or the like, and contents to be displayed are distributed to a plurality of screens for display.
The existing split-screen mode and virtual reality VR display mode have the following defects in the software development process:
1. the content that can be displayed by a single screen is too little, the content that can be contained by the single screen is often a window, the displayed content has great limitation, and the size of the screen is limited by the size of the display.
2. The prior VR technology does not solve the problem of the need of the software development scene by the display method of the keyboard and the mouse in the virtual scene and the control method of a plurality of display windows.
3. There is a limitation to displaying multiple contents of multiple windows, in the process of software development, a developer often needs to monitor the change condition of the multiple windows, for example, a front-end developer needs to check the change of a front-end page while modifying codes, or the developer needs to detect multiple windows such as a server running state, a page display state, a database state and the like at the same time when monitoring a project running state.
Disclosure of Invention
The method and the system for processing the split screen of the software development are used for solving at least one problem in the prior art, the VR is applied to the software development, so that content information which can be obtained by a developer in the development process is not limited by a development terminal screen, and by synchronizing a virtual scene with the information of the software development terminal, the developer can use the VR to carry out the software development on the basis of keeping the existing operation habit to the greatest extent, and the learning cost required by the developer is reduced.
The invention provides a split screen processing method for software development, which comprises the following steps:
constructing a virtual scene required by software development split-screen processing based on a Unity technology;
displaying a virtual mouse keyboard in the virtual scene, and synchronizing information of the virtual mouse keyboard and the physical mouse keyboard;
displaying a virtual screen window in the virtual scene, and synchronizing output information of the virtual screen window with output information of a physical screen of a software development terminal;
switching among a plurality of display layers and synchronously moving across screen areas for the virtual scene with information synchronization completed;
the virtual mouse keyboard comprises a virtual keyboard and a virtual mouse;
the physical mouse keyboard comprises a physical keyboard and a physical mouse;
the display layer is built with the virtual screen window.
According to the method for processing the split screen of the software development, which is provided by the invention, the virtual mouse keyboard is displayed in the virtual scene, and the information of the virtual mouse keyboard and the information of the physical mouse keyboard are synchronized, and the method comprises the following steps:
collecting input signals of the physical mouse and keyboard, and displaying a virtual keyboard and a virtual mouse through a first model and a second model which are placed in the virtual scene;
synchronizing the information of the virtual keyboard and the physical keyboard based on a preset first model action when receiving an input signal of the physical keyboard by adjusting the position posture of the first model in the virtual scene layout to correspond to the position posture of the physical keyboard;
determining the moving distance of the physical mouse in the virtual scene according to the pixel point of the pointer of the physical mouse moving on the physical screen of the software development terminal and the positioning precision of the physical mouse;
and simulating the position and the movement track of the physical mouse in the virtual scene according to the movement distance so as to synchronize the information of the virtual mouse and the physical mouse.
According to the method for processing the split screen of the software development, which is provided by the invention, the virtual screen window is displayed in the virtual scene, and the output information of the virtual screen window is synchronized with the output information of the physical screen of the software development terminal, and the method comprises the following steps:
constructing a plurality of display layers in the virtual scene according to the distance from a developer;
displaying the virtual screen window on a plurality of display layers;
and mapping the output information of the physical screen captured in real time onto the virtual screen window to finish synchronization of the output information of the virtual screen window and the output information of the software development terminal.
According to the method for processing the split screen of the software development, which is provided by the invention, the virtual scene which has completed information synchronization is switched among a plurality of display layers, and the method comprises the following steps:
and controlling switching among a plurality of display layers through the preset shortcut key of the physical keyboard among different display layers of the virtual scene with the information synchronization.
According to the method for processing the split screen of the software development, which is provided by the invention, the virtual scene which has completed the information synchronization is synchronously moved across the screen area, and the method comprises the following steps:
and in the same display layer of the virtual scene with the information synchronization completed, synchronously moving a plurality of virtual screen windows in the same display layer through an inter-screen spanning method.
According to the method for processing the split screen of the software development, which is provided by the invention, the synchronous movement of a plurality of virtual screen windows in the same display layer is carried out by the inter-screen spanning method, and the method comprises the following steps:
respectively acquiring first positions of boundary points in a plurality of virtual screen windows in the same display layer;
connecting the plurality of virtual screen windows into a whole according to the first position of each boundary point in the plurality of virtual screen windows;
acquiring a second position of a pointer of a virtual mouse in a current virtual screen window, and judging whether the second position touches the boundary of the current virtual screen window or not;
if so, when a target virtual screen window exists in the extending direction of the boundary, the pointer of the virtual mouse is crossed from the current virtual screen window to the target virtual screen window, so that synchronous movement of a plurality of virtual screen windows in the same display layer is completed.
The invention also provides a software development split-screen processing system, which comprises: the system comprises a virtual scene construction module, a first information synchronization module, a second information synchronization module and a split screen processing module;
the virtual scene construction module is used for constructing a virtual scene required by the split screen processing of the software development based on the Unity technology;
the first information synchronization module is used for displaying a virtual mouse keyboard in the virtual scene and synchronizing information of the virtual mouse keyboard and the physical mouse keyboard;
the second information synchronization module is used for displaying a virtual screen window in the virtual scene and synchronizing output information of the virtual screen window with output information of a physical screen of the software development terminal;
the split screen processing module is used for switching among a plurality of display layers and synchronously moving across screen areas for the virtual scene with information synchronization completed;
the virtual mouse keyboard comprises a virtual keyboard and a virtual mouse;
the physical mouse keyboard comprises a physical keyboard and a physical mouse;
the display layer is built with the virtual screen window.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the steps of the software development split-screen processing method according to any one of the above when executing the program.
The present invention also provides a processor-readable storage medium storing a computer program for causing a processor to execute the steps of the software development split-screen processing method as any one of the above.
The invention also provides a computer program product comprising a computer program which when executed by a processor implements the steps of a software development split-screen processing method as described in any one of the above.
According to the software development split-screen processing method and system, the VR is applied to software development, so that content information which can be obtained by a developer in the development process is not limited by a development terminal screen, and by synchronizing a virtual scene with the software development terminal information, the developer can use the VR to perform software development on the basis of keeping the existing operation habit to the greatest extent, and the learning cost required by the developer is reduced.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a software development split screen processing method provided by the invention;
FIG. 2 is a schematic diagram of a system architecture of a virtual scenario provided by the present invention;
FIG. 3 is a schematic diagram of manipulating a virtual scene according to the present invention;
FIG. 4 is a second schematic diagram of manipulating a virtual scene according to the present invention;
FIG. 5 is a schematic diagram of a software development split-screen processing system provided by the invention;
fig. 6 is a schematic diagram of the physical structure of the electronic device provided by the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Fig. 1 is a flow chart of a software development split-screen processing method provided by the invention, and as shown in fig. 1, the method comprises the following steps:
s1, constructing a virtual scene required by software development split-screen processing based on a Unity technology;
s2, displaying a virtual mouse keyboard in the virtual scene, and synchronizing information of the virtual mouse keyboard and the physical mouse keyboard;
s3, displaying a virtual screen window in the virtual scene, and synchronizing output information of the virtual screen window with output information of a physical screen of the software development terminal;
s4, switching among a plurality of display layers of the virtual scene with the information synchronization completed, and synchronously moving across screen areas;
the virtual mouse keyboard comprises a virtual keyboard and a virtual mouse;
the physical mouse keyboard comprises a physical keyboard and a physical mouse;
the display layer is built with a virtual screen window.
It should be noted that, the execution subject of the above method may be a computer device.
Optionally, the split screen processing of the software development is realized based on the virtual reality VR technology, so that the split screen processing is used for replacing the split screen effect achieved by using a plurality of displays, and the problem in the traditional software development is solved by utilizing the characteristics of virtual reality.
The Unity technology is used for constructing a virtual scene required by the software development split-screen processing, specifically, the Unity 3D technology can be used for completing the construction of the virtual scene, a plurality of screen output images are constructed through the virtual display card and the image interception of the multi-screen output of the display card based on the screen expansion of the windows platform, and finally the virtual scene system architecture diagram shown in fig. 2 is obtained.
As shown in fig. 2, the system architecture of the virtual scene may specifically include a presentation layer, an application architecture layer, a system library and a kernel layer, where the presentation layer may be used for multi-screen presentation applications, virtual environment applications, development mode applications, and the like; the application architecture layer mainly comprises a screen manager, an audio manager, a resource manager and the like; the system library mainly comprises a 2D image engine, a 3D image engine, a media library and the like; the kernel layer may be based on VR glasses drivers, virtualized graphics card drivers, base station drivers, etc.
And displaying a virtual mouse keyboard (comprising a virtual mouse and a virtual keyboard) in the constructed virtual scene, and synchronizing the information of the virtual mouse keyboard and the physical mouse keyboard (comprising a physical mouse and a physical keyboard), so that a developer can see the position of the physical mouse keyboard when using VR equipment such as VR glasses, and subsequent operation is convenient.
And displaying the virtual screen window in the constructed virtual scene, and synchronizing the output information of the virtual screen window and the output information of the physical screen window by acquiring the output information (such as output images) of the physical screen of a software development terminal such as a PC (personal computer) and mapping the acquired output information of the physical screen to the virtual screen window.
And controlling the virtual scene after the information synchronization of the virtual mouse keyboard and the physical mouse keyboard and the output information synchronization of the virtual screen window and the output information of the physical screen of the software development terminal are completed, so as to realize the switching among a plurality of display layers and the synchronous movement across the screen area.
According to the split-screen processing method for software development, the VR is applied to software development, so that content information which can be obtained by a developer in the development process is not limited by a development terminal screen, and by synchronizing a virtual scene with the software development terminal information, the developer can use the VR to perform software development on the basis of keeping the existing operation habit to the greatest extent, and the learning cost required by the developer is reduced.
Further, in one embodiment, step S2 may specifically include:
s21, acquiring input signals of a physical mouse and keyboard, and displaying the virtual keyboard and the virtual mouse through a first model and a second model which are placed in a virtual scene;
s22, adjusting the position and the posture of the first model in the virtual scene layout to correspond to the position and the posture of the physical keyboard, and synchronizing the information of the virtual keyboard and the physical keyboard based on a preset first model action when receiving an input signal of the physical keyboard;
s23, determining the moving distance of the physical mouse in the virtual scene according to the positioning accuracy of the pixel points of the pointer of the physical mouse moving on the physical screen of the software development terminal and the physical mouse;
s24, simulating the position and the moving track of the physical mouse in the virtual scene according to the moving distance so as to synchronize the information of the virtual mouse and the physical mouse.
Optionally, input signals of a physical mouse keyboard are collected, a first model is placed in a built virtual scene to display the virtual keyboard, a second model is placed in the built virtual scene to display the virtual mouse, the placement position of the physical keyboard is generally fixed for software, a first model of one keyboard such as a keyboard 3D model is placed, the position and the posture of the physical keyboard are corresponding through adjusting the recorded coordinate position and placement posture of the keyboard 3D model in the virtual scene layout, when the physical keyboard signals are received, the prefabricated keyboard 3D model is used for action, and the corresponding keys of the virtual keyboard flash and lighten briefly to enable the virtual keyboard to be related with the physical keyboard, so that information synchronization of the virtual keyboard and the physical keyboard is completed.
The physical mouse is generally placed on the right side of the physical keyboard, a second model of the virtual mouse such as a 3D model of the mouse is adjusted to obtain an initial position, the moving distance of the physical mouse in the virtual scene is calculated by obtaining a pixel point of a pointer corresponding to the physical mouse moving on a physical screen of a software development terminal such as a PC (personal computer) and the positioning precision dpi of the physical mouse, and the position and the moving track of the physical mouse in the virtual scene are simulated according to the moving distance, so that the information synchronization of the virtual mouse and the physical mouse is completed, wherein the formula for calculating the moving distance is as follows:
moving pixel/physical mouse current dpi 2.54cm
In this way, physical mouse movements can be simulated in the virtual scene, and additional processing of the VR device and physical mouse is not required.
According to the split-screen processing method for software development, the VR is applied to software development, so that content information which can be obtained by a developer in the development process is not limited by a development terminal screen, and by synchronizing a virtual scene with the software development terminal information, the developer can use the VR to perform software development on the basis of keeping the existing operation habit to the greatest extent, the learning cost required by the developer is reduced, and meanwhile, a solution is provided for how to display the physical mouse and keyboard position in the VR mode.
Further, in one embodiment, step S3 may specifically include:
s31, constructing a plurality of display layers in the virtual scene according to the distance from the developer;
s32, displaying the virtual screen window on a plurality of display layers;
and S33, mapping the output information of the physical screen captured in real time onto the virtual screen window to finish synchronization of the output information of the virtual screen window and the output information of the software development terminal.
Optionally, when the Unity 3D is used for manufacturing the virtual scene, a plurality of display layers are divided according to the distance from a developer, the virtual screen window built in the virtual scene is displayed on the plurality of display layers in the virtual scene, and a display card output image is captured in real time through a PC computer windows system Application Program Interface (API) to map a windows system desktop onto the virtual screen window.
As shown in fig. 3, first, a system main desktop (i.e., a physical screen) of a PC computer to be used is mapped to the main desktop shown in fig. 3, in the main desktop, functions of windows extended desktop are used to add an extended screen, a method of creating a virtual screen window is provided by a Unity 3D component, and then information of the extended screen acquired through a windows system API is mapped to the virtual screen window to complete creation of the virtual screen window. And the resolution of the virtual screen window is modified by adjusting the configuration of the windows expansion screen, the size of the virtual screen window is stretched by calling a preset control method in the Unity 3D, and the display position of the virtual screen window is changed to adjust the virtual screen window, so that the front-end development can quickly modify the proper screen resolution according to the display requirement of the front-end development. The method has the advantages that the limitation of a physical display is avoided, the number of virtual screen windows can be expanded infinitely in a virtual scene, developers can increase the number of the virtual screen windows to be displayed according to requirements, the content to be checked in software development is matched, and the limitation of the traditional split-screen mode in the software development is broken through.
According to the split-screen processing method for software development, provided by the invention, the virtual scene of the software development working space is infinitely expanded through one VR device, so that the content which can be seen by a developer at the same time is greatly improved, and the split-screen processing method is free from the constraint of physical space, and compared with the traditional mode of improving the number of screens which can be simultaneously controlled by the developer in the real physical space, the restriction of the space to the device is greatly reduced.
Further, in an embodiment, in step S3, switching between a plurality of display layers for the virtual scene for which information synchronization has been completed may specifically include:
s31, switching among a plurality of display layers of the virtual scene with information synchronization completed is controlled through shortcut keys of a preset physical keyboard.
Optionally, by using a method of controlling multiple display windows by using a virtual mouse and keyboard in VR, a method of simulating a position of a physical mouse and keyboard in a virtual scene, so that characteristics of VR can be used in software development to realize simulating a physical display, and break through limitations of number and size of the physical display, in particular:
as shown in fig. 3, two display layers, namely a display layer 1 and a display layer 2, are preset in the virtual scene, and different display layers are switched and controlled by shortcut keys of a preset physical keyboard.
For example, a shortcut key of ctrl+alt+tab may be used to implement switching between multiple display layers.
The split-screen processing method for software development provided by the invention provides a method for controlling a plurality of display layers of a virtual scene by using a physical keyboard by VR, solves the problem of controlling the virtual scene when the VR performs software development, and ensures that the software development can be realized by means of a virtual reality technology, thereby providing convenience for the software development.
Further, in an embodiment, in step S3, the performing the synchronous movement across the screen area on the virtual scene that has completed the information synchronization may specifically include:
s32, in the same display layer of the virtual scene with the information synchronization completed, synchronous movement of a plurality of virtual screen windows in the same display layer is performed through an inter-screen spanning method.
Further, in one embodiment, step S32 may specifically include:
s321, respectively acquiring first positions of boundary points in a plurality of virtual screen windows in the same display layer;
s322, connecting the plurality of virtual screen windows into a whole according to the first position of each boundary point in the plurality of virtual screen windows;
s323, acquiring a second position of a pointer of the virtual mouse in the current virtual screen window, and judging whether the second position touches the boundary of the current virtual screen window;
and S324, if the target virtual screen window exists in the extending direction of the boundary, crossing the pointer of the virtual mouse from the current virtual screen window to the target virtual screen window so as to finish synchronous movement of a plurality of virtual screen windows in the same display layer.
Optionally, in the same display layer of the virtual scene for which the information synchronization has been completed, the synchronized movement of a plurality of virtual screen windows in the same display layer is performed by an inter-screen spanning method, specifically:
as shown in fig. 4, for the virtual screen windows (e.g., screens 1 to 4) of the same display layer, coordinates of boundary points in the screens 1 to 4 are acquired respectively, the first positions of the boundary points of each screen are determined according to the coordinates of each screen, and the boundaries of each screen can be determined according to the first positions of the boundary points of each screen, based on which the respective screens (screens 1 to 4) having different sizes can be connected as a whole, and the pointer of the virtual mouse can move only in the virtual screen window as in the physical space split screen.
The coordinate values of all boundary points of all screens (screen 1-screen 4) in the currently controlled display layer are obtained, the largest and smallest coordinate values are found and used as the boundary values of the whole after connection, the coordinate values of the pointer of the virtual mouse in the second position of the current virtual screen window are obtained through a windows system API, for example, the coordinate values (x 3, y 3) of the pointer of the virtual mouse in the left screen 1 of fig. 4 are not the coordinate values in the virtual environment, but the coordinate values of the pointer in the virtual scene are the coordinate values (x 1+x3, y1+y3) of the pointer in the left screen 1 of fig. 4 in the virtual environment, the coordinate values of all boundary points of the screen 1 obtained through the Unity API are the coordinate values (x 1, y 1), (x 2, y 1) and the coordinate values (x 2, y 2) respectively, and the coordinate values of the pointer of the virtual mouse are obtained through the coordinates (x 1, y 1) of the left lower boundary point and the coordinate values (x 3, y 3) of the pointer of the virtual mouse in the current virtual screen window, for example, the coordinate values of the pointer in the virtual screen 1 of the left screen 1 of fig. 4 in the virtual environment are the coordinate values of (x 1+x1+y3), the coordinate of the pointer in the virtual window of the current screen, and the coordinate of the current position of the virtual mouse can be determined after touching the second position of the virtual mouse window, and the current position of the virtual mouse window can be determined. If so, when the target virtual screen window exists in the extending direction of the boundary, the pointer of the virtual mouse is crossed from the current virtual screen window to the target virtual screen window, so that synchronous movement of a plurality of virtual screen windows in the same display layer is completed.
For example, as shown in the right diagram of fig. 4, the pointer of the virtual mouse in the screen 3 touches the upper boundary of the current virtual screen window, such as the screen 3, at this time, the target virtual screen window screen 1 exists in the y-value direction along this coordinate, which extends upward, at this time, the windows system API is called to change the pointer position of the virtual mouse across to the screen 1 and enter from the upper left boundary point (x 1, y 2), if there is no other screen in the time delay direction in touching the boundary, the pointer stays at the boundary of the screen, as shown in the right diagram of fig. 4, the pointer of the virtual mouse in the screen 2 touches the upper boundary of the screen 2, and if no other virtual screen window exists beyond the boundary value of the whole display layer, the pointer of the virtual mouse stays at the original boundary, that is, the upper boundary of the screen 2.
According to the current display and control method, the problem that the current VR does not have a crossing method for displaying a plurality of screens and mouse pointers among the plurality of screens in a far-near layering mode is solved, the control mode is different from that of a physical display split screen when the pointers cross the boundary of the screens, and the position value of the display screen can be acquired in the VR, so that the control of the screen in the VR is more convenient, and the use habit of a developer is more met.
The split-screen processing method for software development provided by the invention provides a method for controlling a plurality of screens by using a physical mouse and keyboard by VR, solves the problems of how to display the screens and match the positions of the physical mouse and keyboard when the VR performs software development, and ensures that the software development can be realized by means of a virtual reality technology, thereby providing convenience for the software development.
The software development split-screen processing system provided by the invention is described below, and the software development split-screen processing system described below and the software development split-screen processing method described above can be correspondingly referred to each other.
Fig. 5 is a schematic structural diagram of a software development split-screen processing system provided by the present invention, as shown in fig. 5, including: the virtual scene construction module 510, the first information synchronization module 511, the second information synchronization module 512 and the split screen processing module 513;
the virtual scene construction module 510 is configured to construct a virtual scene required by the software development split-screen processing based on the Unity technology;
a first information synchronization module 511, configured to display a virtual mouse keyboard in a virtual scene, and synchronize information of the virtual mouse keyboard and a physical mouse keyboard;
a second information synchronization module 512, configured to display a virtual screen window in the virtual scene, and synchronize output information of the virtual screen window with output information of a physical screen of the software development terminal;
the split screen processing module 513 is configured to switch between multiple display layers of the virtual scene that has completed information synchronization, and perform synchronization movement across screen areas;
the virtual mouse keyboard comprises a virtual keyboard and a virtual mouse;
the physical mouse keyboard comprises a physical keyboard and a physical mouse;
the display layer is built with a virtual screen window.
According to the software development split-screen processing system provided by the invention, the VR is applied to software development, so that content information which can be obtained by a developer in the development process is not limited by a development terminal screen, and by synchronizing a virtual scene with the software development terminal information, the developer can use the VR to perform software development on the basis of keeping the existing operation habit to the greatest extent, and the learning cost required by the developer is reduced.
Further, in one embodiment, the first information synchronization module 511 may be further specifically configured to:
collecting input signals of a physical mouse keyboard, and displaying the virtual keyboard and the virtual mouse through a first model and a second model which are placed in a virtual scene;
the method comprises the steps of adjusting the position and the posture of a first model in the virtual scene layout to correspond to the position and the posture of a physical keyboard, and synchronizing the information of the virtual keyboard and the physical keyboard based on a preset first model action when an input signal of the physical keyboard is received;
determining the moving distance of the physical mouse in the virtual scene according to the pixel points of the pointer of the physical mouse moving on the physical screen of the software development terminal and the positioning precision of the physical mouse;
and simulating the position and the moving track of the physical mouse in the virtual scene according to the moving distance so as to synchronize the information of the virtual mouse and the physical mouse.
According to the software development split-screen processing system provided by the invention, through applying VR to software development, content information which can be obtained by a developer in a development process is not limited by a development terminal screen, and through synchronizing a virtual scene with the software development terminal information, the developer can use VR to perform software development on the basis of keeping the existing operation habit to the greatest extent, so that the learning cost required by the developer is reduced, and meanwhile, a solution is provided for how to display the physical mouse and keyboard position in a VR mode.
Further, in one embodiment, the second information synchronization module 512 may be further specifically configured to:
constructing a plurality of display layers in the virtual scene according to the distance from a developer;
displaying the virtual screen window on a plurality of display layers;
and mapping the output information of the physical screen captured in real time onto the virtual screen window to finish synchronization of the output information of the virtual screen window and the output information of the software development terminal.
According to the software development split-screen processing system, the virtual scene of the software development working space is infinitely expanded through one VR device, so that the content which can be seen by a developer at the same time is greatly improved, the system is not limited by physical space, and compared with the traditional mode of improving the number of screens which can be simultaneously controlled by the developer in the real physical space, the system greatly reduces the restriction of space to the device.
Further, in one embodiment, the split screen processing module 513 may be further specifically configured to:
and controlling switching among a plurality of display layers through shortcut keys of a preset physical keyboard among different display layers of the virtual scene with information synchronization.
The split-screen processing system for software development provided by the invention provides a method for controlling a plurality of display layers of a virtual scene by using a physical keyboard by VR, solves the problem of controlling the virtual scene when the VR performs software development, and ensures that the software development can be realized by means of a virtual reality technology, thereby providing convenience for the software development.
Further, in one embodiment, the split screen processing module 513 may be further specifically configured to:
and in the same display layer of the virtual scene with the information synchronization completed, synchronously moving a plurality of virtual screen windows in the same display layer through an inter-screen spanning method.
Further, in one embodiment, the split screen processing module 513 may be further specifically configured to:
respectively acquiring first positions of boundary points in a plurality of virtual screen windows in the same display layer;
connecting the plurality of virtual screen windows into a whole according to the first position of each boundary point in the plurality of virtual screen windows;
acquiring a second position of a pointer of the virtual mouse in a current virtual screen window, and judging whether the second position touches the boundary of the current virtual screen window;
if so, when the target virtual screen window exists in the extending direction of the boundary, the pointer of the virtual mouse is crossed from the current virtual screen window to the target virtual screen window, so that synchronous movement of a plurality of virtual screen windows in the same display layer is completed.
The split-screen processing system for software development provided by the invention provides a method for controlling a plurality of screens by using a physical mouse and keyboard by VR, solves the problems of how to display the screens and match the positions of the physical mouse and keyboard when the VR performs software development, and ensures that the software development can be realized by means of a virtual reality technology, thereby providing convenience for the software development.
Fig. 6 is a schematic physical structure of an electronic device according to the present invention, as shown in fig. 6, the electronic device may include: a processor (processor) 610, a communication interface (communication interface) 611, a memory (memory) 612 and a bus (bus) 613, wherein the processor 610, the communication interface 611, and the memory 612 communicate with each other via the bus 613. The processor 610 may call logic instructions in the memory 612 to perform the following method:
constructing a virtual scene required by software development split-screen processing based on a Unity technology;
displaying a virtual mouse keyboard in the virtual scene, and synchronizing information of the virtual mouse keyboard and the physical mouse keyboard;
displaying a virtual screen window in the virtual scene, and synchronizing output information of the virtual screen window with output information of a physical screen of the software development terminal;
switching among a plurality of display layers and synchronously moving across screen areas for the virtual scene with information synchronization completed;
the virtual mouse keyboard comprises a virtual keyboard and a virtual mouse;
the physical mouse keyboard comprises a physical keyboard and a physical mouse;
the display layer is built with the virtual screen window.
Further, the logic instructions in the memory described above may be implemented in the form of software functional units and stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer power supply screen (which may be a personal computer, a server, or a network power supply screen, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Further, the present invention discloses a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, are capable of executing the software development split-screen processing method provided in the above method embodiments, for example, comprising:
constructing a virtual scene required by software development split-screen processing based on a Unity technology;
displaying a virtual mouse keyboard in the virtual scene, and synchronizing information of the virtual mouse keyboard and the physical mouse keyboard;
displaying a virtual screen window in the virtual scene, and synchronizing output information of the virtual screen window with output information of a physical screen of the software development terminal;
switching among a plurality of display layers and synchronously moving across screen areas for the virtual scene with information synchronization completed;
the virtual mouse keyboard comprises a virtual keyboard and a virtual mouse;
the physical mouse keyboard comprises a physical keyboard and a physical mouse;
the display layer is built with the virtual screen window.
Further, the present invention also provides a processor-readable storage medium storing a computer program for causing the processor to execute the method provided in the above embodiments, for example, including:
constructing a virtual scene required by software development split-screen processing based on a Unity technology;
displaying a virtual mouse keyboard in the virtual scene, and synchronizing information of the virtual mouse keyboard and the physical mouse keyboard;
displaying a virtual screen window in the virtual scene, and synchronizing output information of the virtual screen window with output information of a physical screen of the software development terminal;
switching among a plurality of display layers and synchronously moving across screen areas for the virtual scene with information synchronization completed;
the virtual mouse keyboard comprises a virtual keyboard and a virtual mouse;
the physical mouse keyboard comprises a physical keyboard and a physical mouse;
the display layer is built with the virtual screen window.
The system embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer power screen (which may be a personal computer, a server, or a network power screen, etc.) to perform the method described in the various embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. The split screen processing method for software development is characterized by comprising the following steps:
constructing a virtual scene required by software development split-screen processing based on a Unity technology;
displaying a virtual mouse keyboard in the virtual scene, and synchronizing information of the virtual mouse keyboard and the physical mouse keyboard;
displaying a virtual screen window in the virtual scene, and synchronizing output information of the virtual screen window with output information of a physical screen of a software development terminal;
switching among a plurality of display layers and synchronously moving across screen areas for the virtual scene with information synchronization completed;
the virtual mouse keyboard comprises a virtual keyboard and a virtual mouse;
the physical mouse keyboard comprises a physical keyboard and a physical mouse;
the display layer is built with the virtual screen window.
2. The method for processing the split screen of the software development according to claim 1, wherein displaying a virtual mouse keyboard in the virtual scene and synchronizing information of the virtual mouse keyboard and a physical mouse keyboard comprises:
collecting input signals of the physical mouse and keyboard, and displaying a virtual keyboard and a virtual mouse through a first model and a second model which are placed in the virtual scene;
synchronizing the information of the virtual keyboard and the physical keyboard based on a preset first model action when receiving an input signal of the physical keyboard by adjusting the position posture of the first model in the virtual scene layout to correspond to the position posture of the physical keyboard;
determining the moving distance of the physical mouse in the virtual scene according to the pixel point of the pointer of the physical mouse moving on the physical screen of the software development terminal and the positioning precision of the physical mouse;
and simulating the position and the movement track of the physical mouse in the virtual scene according to the movement distance so as to synchronize the information of the virtual mouse and the physical mouse.
3. The software development split-screen processing method according to claim 1, wherein the displaying a virtual screen window in the virtual scene and synchronizing output information of the virtual screen window with output information of a physical screen of a software development terminal comprises:
constructing a plurality of display layers in the virtual scene according to the distance from a developer;
displaying the virtual screen window on a plurality of display layers;
and mapping the output information of the physical screen captured in real time onto the virtual screen window to finish synchronization of the output information of the virtual screen window and the output information of the software development terminal.
4. The method for processing a split screen for software development according to claim 3, wherein said switching between a plurality of display layers for a virtual scene for which synchronization of information has been completed comprises:
and controlling switching among a plurality of display layers through the preset shortcut key of the physical keyboard among different display layers of the virtual scene with the information synchronization.
5. The method for processing split-screen software development according to claim 3, wherein said synchronizing movement of the virtual scene for which the synchronization of the information has been completed across the screen area comprises:
and in the same display layer of the virtual scene with the information synchronization completed, synchronously moving a plurality of virtual screen windows in the same display layer through an inter-screen spanning method.
6. The method for processing split-screen software development according to claim 5, wherein said performing synchronous movement of a plurality of virtual screen windows in the same display layer by an inter-screen spanning method comprises:
respectively acquiring first positions of boundary points in a plurality of virtual screen windows in the same display layer;
connecting the plurality of virtual screen windows into a whole according to the first position of each boundary point in the plurality of virtual screen windows;
acquiring a second position of a pointer of a virtual mouse in a current virtual screen window, and judging whether the second position touches the boundary of the current virtual screen window or not;
if so, when a target virtual screen window exists in the extending direction of the boundary, the pointer of the virtual mouse is crossed from the current virtual screen window to the target virtual screen window, so that synchronous movement of a plurality of virtual screen windows in the same display layer is completed.
7. A software development split-screen processing system, comprising: the system comprises a virtual scene construction module, a first information synchronization module, a second information synchronization module and a split screen processing module;
the virtual scene construction module is used for constructing a virtual scene required by the split screen processing of the software development based on the Unity technology;
the first information synchronization module is used for displaying a virtual mouse keyboard in the virtual scene and synchronizing information of the virtual mouse keyboard and the physical mouse keyboard;
the second information synchronization module is used for displaying a virtual screen window in the virtual scene and synchronizing output information of the virtual screen window with output information of a physical screen of the software development terminal;
the split screen processing module is used for switching among a plurality of display layers and synchronously moving across screen areas for the virtual scene with information synchronization completed;
the virtual mouse keyboard comprises a virtual keyboard and a virtual mouse;
the physical mouse keyboard comprises a physical keyboard and a physical mouse;
the display layer is built with the virtual screen window.
8. An electronic device comprising a processor and a memory storing a computer program, characterized in that the processor implements the steps of the software development split-screen processing method of any one of claims 1 to 6 when executing the computer program.
9. A processor-readable storage medium, wherein the processor-readable storage medium stores a computer program for causing a processor to execute the steps of the software development split-screen processing method according to any one of claims 1 to 6.
10. A computer program product comprising a computer program which, when executed by a processor, implements the steps of the software development split-screen processing method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210083843.7A CN116501201A (en) | 2022-01-21 | 2022-01-21 | Software development split-screen processing method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210083843.7A CN116501201A (en) | 2022-01-21 | 2022-01-21 | Software development split-screen processing method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116501201A true CN116501201A (en) | 2023-07-28 |
Family
ID=87323613
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210083843.7A Pending CN116501201A (en) | 2022-01-21 | 2022-01-21 | Software development split-screen processing method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116501201A (en) |
-
2022
- 2022-01-21 CN CN202210083843.7A patent/CN116501201A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107958480B (en) | Image rendering method and device and storage medium | |
| KR101379074B1 (en) | An apparatus system and method for human-machine-interface | |
| US20220249949A1 (en) | Method and apparatus for displaying virtual scene, device, and storage medium | |
| EP4120199A1 (en) | Image rendering method and apparatus, and electronic device and storage medium | |
| CN108776544B (en) | Interaction method and device in augmented reality, storage medium and electronic equipment | |
| CN109298806B (en) | Remote quick interface interaction method and device based on object recognition | |
| CN112380357B (en) | Method for realizing interactive navigation of knowledge graph visualization | |
| CN112882637A (en) | Interaction method for multi-layer animation display and browser | |
| CN112634414A (en) | Map display method and device | |
| CN113359995A (en) | Man-machine interaction method, device, equipment and storage medium | |
| CN107491289B (en) | Window rendering method and device | |
| CN107992242A (en) | A kind of switching method of suspension windows, device, equipment and storage medium | |
| CN111190826A (en) | Testing method and device for virtual reality immersive tracking environment, storage medium and equipment | |
| CN109598672B (en) | Map road rendering method and device | |
| CN113345108A (en) | Augmented reality data display method and device, electronic equipment and storage medium | |
| JP2005149175A (en) | Display controller and program | |
| US20230206573A1 (en) | Method of learning a target object by detecting an edge from a digital model of the target object and setting sample points, and method of augmenting a virtual model on a real object implementing the target object using the learning method | |
| CN116501201A (en) | Software development split-screen processing method and system | |
| CN115965731A (en) | Rendering interaction method, device, terminal, server, storage medium and product | |
| EP4325344A1 (en) | Multi-terminal collaborative display update method and apparatus | |
| CN114655240A (en) | Information display method and device, electronic equipment and storage medium | |
| US10080963B2 (en) | Object manipulation method, object manipulation program, and information processing apparatus | |
| US20210303744A1 (en) | Computer aided design (cad) model connection propagation | |
| JP2024506879A (en) | Methods and apparatus, computer equipment and storage media for presenting virtual representations | |
| CN112085814A (en) | Electronic map display method, device, equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |