CN114020400A

CN114020400A - NoVNC-based remote virtual simulation method, system, device and medium

Info

Publication number: CN114020400A
Application number: CN202111194995.6A
Authority: CN
Inventors: 陈旻琪; 吴车; 廖江帆; 左志军; 贺毅; 姚维兵
Original assignee: Guangzhou Mino Equipment Co Ltd
Current assignee: Guangzhou Mino Equipment Co Ltd
Priority date: 2021-10-12
Filing date: 2021-10-12
Publication date: 2022-02-08

Abstract

The invention provides a remote virtual simulation method, a system, equipment and a medium based on a NoVNC, wherein the method comprises the following steps: acquiring debugging information of target debugging equipment; determining that the target debugging equipment is primary debugging equipment according to the debugging information, deploying a NoVNC client, and accessing the primary debugging equipment through the NoVNC client; acquiring a debugging instruction, connecting to a controller of the primary debugging equipment through a remote gateway in a virtual environment of the NoVNC client according to the debugging instruction, and debugging the controller; the scheme carries out remote debugging on the device controller through the remote gateway route, greatly improves debugging efficiency, breaks through geographical limitation, replaces the existing mode of centralized debugging by debugging personnel, saves labor cost and financial cost, and can be widely applied to the technical field of computers.

Description

NoVNC-based remote virtual simulation method, system, device and medium

Technical Field

The invention relates to the technical field of computers, in particular to a remote virtual simulation method, system, equipment and medium based on a NoVNC.

Background

Virtual simulation is a widely popular automobile virtual debugging technology at present, and can perform a series of configurations by introducing digital models used for equipment on site, such as a rolling machine, a robot, personnel, an automobile body, a clamp and a Programmable Logic Controller (PLC) program, into special software, so that real production can be simulated in a system, and debugging personnel can debug programs and layouts in a virtual environment in advance. And meanwhile, the production is monitored by a manager in a one-to-one mode through digital analogy.

Virtual debugging technology has been widely applied to the early stage design of the automation line body in the manufacturing industry as a pre-integration means. In the process of virtual debugging, the virtual debugging process generally includes virtual debugging which requires the running of the entity PLC and virtual debugging which does not require the running of the entity PLC according to different brands and protocols of the PLC. In the prior art, an operator needs to conduct simulation debugging in a virtual production environment by conducting a series of configuration after importing a digital-analog and a program into software; because the data exchange requirement of the PLC-virtual software system exists in the system, the PLC and the software can only be transmitted in a wired mode, and therefore the geographical limitation factor of virtual debugging is formed.

Disclosure of Invention

In view of the above, to at least partially solve one of the above technical problems, embodiments of the present invention provide a remote virtual simulation method based on a nonvc, which can break through wired data transmission, and a system, a device, and a medium capable of implementing the method.

In a first aspect, a technical solution of the present application provides a remote virtual simulation method based on a noVNC, which includes:

acquiring debugging information of target debugging equipment;

determining that the target debugging equipment is primary debugging equipment according to the debugging information, deploying a NoVNC client, and accessing the primary debugging equipment through the NoVNC client;

and acquiring a debugging instruction, and connecting to a controller of the primary debugging equipment through a remote gateway in the virtual environment of the NoVNC client according to the debugging instruction to debug the controller.

In a possible embodiment of the present disclosure, after the step of accessing the primary debug device through the nonvnc client, the remote virtual simulation method further includes the following steps: and acquiring a monitoring instruction, and performing remote virtual simulation monitoring on debugging equipment of the virtual environment according to the monitoring instruction.

In a possible embodiment of the present disclosure, the step of deploying the nonvnc client includes:

deploying and starting a virtual network control service;

loading an agent tool, wherein the agent tool is used for converting TCP Sockets of the virtual network control service into Web Sockets;

deploying the NOVNC client in a computer environment in which the agent is loaded.

In a possible embodiment of the present disclosure, the step of accessing the primary debug apparatus by the nonvc client includes:

acquiring user information, and starting a daemon process example of the virtual network control service according to the user information;

and in the daemon process example, starting the virtual network control service of the NoVNC client, and accessing the processing debugging equipment through the virtual network control service.

In a possible embodiment of the solution of the present application, the remote gateway is an open source gateway, and the open source gateway is at least one of OpenResty, Kong, Zuul2, and springcloudggateway.

In a possible embodiment of the solution of the present application, the step of connecting to the controller of the primary commissioning device through a remote gateway includes:

analyzing the debugging instruction to obtain an IP address and a port number of the NoVNC client;

and generating a first access link according to the IP address and the port number, and determining a forwarding path of the Web sockets according to the first access link and the open source gateway.

In a possible embodiment of the present disclosure, after the step of debugging the controller, the remote virtual simulation method further includes the following steps:

and simulating and building an automatic production line in the virtual environment, and driving the automatic production line through the debugged controller.

In a second aspect, a technical solution of the present invention further provides a remote virtual simulation system based on a noVNC, where the system includes:

the data receiving unit is used for acquiring debugging information of the target debugging equipment;

the virtual simulation unit is used for determining that the target debugging equipment is primary debugging equipment according to the debugging information, deploying a NOVNC client, accessing the primary debugging equipment through the NOVNC client, and connecting to a controller of the primary debugging equipment through a remote gateway in a virtual environment of the NOVNC client according to the debugging instruction to debug the controller;

and the user interaction unit is used for acquiring the debugging instruction.

In a third aspect, a technical solution of the present invention further provides a remote virtual simulation device based on a noVNC, including:

at least one processor;

at least one memory for storing at least one program;

when the at least one program is executed by the at least one processor, the at least one processor is caused to perform the method of the first aspect.

In a fourth aspect, the present invention further provides a storage medium, in which a processor-executable program is stored, and the processor-executable program is used to execute the nonvnc-based remote virtual simulation method according to the first aspect.

Advantages and benefits of the present invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention:

according to the technical scheme, firstly, software and hardware used through virtual debugging are used, the primary debugging equipment determined in debugging information is aimed at, a NOVNC client side is installed and configured, after a virtual simulation environment is built, equipment needing debugging can be directly accessed through a browser page, remote virtual production line simulation is carried out, debugging and monitoring are carried out, the scheme further adopts a remote gateway routing technology, an engineer can have the capability of remotely carrying out remote debugging on the PLC through a computer as a gateway, the debugging efficiency is greatly improved, geographic limitation is broken through, the existing centralized debugging mode of debugging personnel is replaced, and the human cost and the financial cost are saved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a flowchart illustrating steps of a remote virtual simulation method based on a noVNC according to an embodiment of the present invention;

fig. 2 is a flowchart illustrating steps of another remote virtual simulation method based on a noVNC according to an embodiment of the present invention;

FIG. 3 is a diagram illustrating a simulation operation performed according to an embodiment of the present invention.

Detailed Description

Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention. The step numbers in the following embodiments are provided only for convenience of illustration, the order between the steps is not limited at all, and the execution order of each step in the embodiments can be adapted according to the understanding of those skilled in the art.

The terms "first," "second," and the like in the description and in the claims of the present application and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances and are merely descriptive of the various embodiments of the application and how objects of the same nature can be distinguished. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of elements is not necessarily limited to those elements, but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.

A Virtual Network Console (VNC) is an open source software developed by the european AT & T research laboratory, based on UNIX and LINUX operating systems. The server comprises a server part and a viewer part, namely a server and a client. The VNC server can remotely visualize a system interface through graphical desktop components such as ghome and the like, require the server to give the picture display control right to a vnceser through XProtecol, and remotely release the desktop environment to a viewer end through socket and simplethtp, so that the purpose of remotely operating the NOVNC which is a browser-based vncvnvviewer with a pitubb upper source is achieved, and node. He can solve the complex operation that the vncvnviewer needs to install clients on different systems in the past and solve the problem of cross-system access.

The noVNC in the embodiment is a VNC client of HTML5, and may be implemented by JavaScript programming. The main function of the remote VNC Server is to communicate with the remote VNC Server, and through encoding and decoding RFB (remote Frame buffer) protocol data, on one hand, data sent by the remote VNC Server is received, and after decoding, the data is drawn on a client side through a Cancas technology, and on the other hand, terminal input of the client side is encoded into RFB data and sent to the remote VNC Server. That is, based on the noVNC, in the embodiment, a Web browser supporting HTML5, such as Chrome, may be directly used to access the configuration interface of the remote machine on which the VNC Server is installed, without additionally installing a native VNC client.

Based on the above principle, in a first aspect, as shown in fig. 1, the technical solution of the present application provides a remote virtual simulation method based on a noVNC, which may include steps S100 to S300:

s100, obtaining debugging information of target debugging equipment;

specifically, the debugging information in the embodiment refers to information for debugging a certain event, which is obtained through an interactive interface of an electronic device terminal capable of performing human-computer interaction, such as a smart phone and a computer, where the event may be an automobile welding production line.

S200, determining that the target debugging equipment is primary debugging equipment according to the debugging information, deploying a NoVNC client, and accessing the primary debugging equipment through the NoVNC client;

specifically, in the embodiment, the debugging records of the devices stored in the local device or the cloud database are first obtained, matching is performed according to the unique identifiers such as the names of the debugging devices or the device IDs in the records, and when the same debugging records are not obtained through matching, it can be determined that the target debugging device in the debugging information is the initial debugging device, that is, the debugging of the device is the first debugging. After the first debugging is determined, the embodiment deploys the nonVNC client, wherein the deployed nonVNC client process includes but is not limited to installing a VNC Server in a processor of the debugging device; since the NoVNC establishes a link through the Websocket and the VNC server does not support the Websocket, the Websocket proxy needs to be started to convert between the Websockets and the TCP sockets. After the installation and deployment are completed, objects such as equipment needing debugging and the like can be constructed and formed in the virtual environment through the NoVNC client, and an operator can access a control panel or a parameter interface of the equipment needing debugging in the virtual environment to acquire parameter information, working state and the like of the equipment, so that the access to the equipment needing debugging is completed. It should be noted that the target debugging devices, which are objects constructed in the virtual environment, are all virtual devices, and the information such as the relevant control parameters and the operating states of the virtual devices is obtained from the real devices by means of sensors, for example, and is modeled in the virtual environment according to the control parameters and the state parameters to form the virtual devices.

S300, acquiring a debugging instruction, connecting to a controller of the primary debugging equipment through a remote gateway in a virtual environment of the NoVNC client according to the debugging instruction, and debugging the controller;

the debugging instruction is instruction information which is obtained through an interactive interface of the equipment terminal and is different from the debugging information; the instructional information contains the content of specific debugging of the target equipment, such as the motion track of the mechanical arm in the automobile welding production line, and the debugging content is written into the PLC of the debugging equipment, so as to realize the programmed control and the automatic operation of the mechanical arm. In the embodiment, the device with the nonVNC client is capable of issuing the virtual devices in the virtual environment to the real scene according to the corresponding communication protocol in a manner that the host computer controls the lower computer, and writing the control parameters into the device PLC in a manner of program writing.

In some optional embodiments, after the step S200 is a process of accessing the primary debug device through the noVNC client, the method of this embodiment may further include a step S210:

s210, acquiring a monitoring instruction, and performing remote virtual simulation monitoring on debugging equipment in a virtual environment according to the monitoring instruction;

for example, as shown in fig. 2, after the device in the real scene is debugged through remote simulation, the embodiment may further obtain the running condition of the device in the real environment in real time through a target device (virtual device) obtained through virtual simulation modeling in the nonvnc client, so as to implement more efficient remote real-time monitoring.

In some alternative embodiments, the process of deploying a noVNC client in step S200 may include steps S220-S240:

s220, deploying and starting a virtual network control service;

s230, loading an agent tool, wherein the agent tool is used for converting TCP Sockets of the virtual network control service into Web Sockets;

s240, deploying a NoVNC client in a computer environment loaded with an agent tool;

specifically, in the embodiment, unlike the native VNC client, RFB data of the native client is directly carried on tcp (raw tcp), and RFB data processed by the noVNC is carried on Websocket. Since most VNC servers do not support WebSockets at present, the noVNC cannot directly connect to the VNC server, and therefore, the embodiment needs a proxy to implement the conversion between WebSockets and Raw TCP, and the proxy is websocket. A typical websocket-mediated web service from a client browser to a VNC server is as follows, where a noVNC is a client of HTML5, and although initially stored on a proxy server where the websocket is located, the main js code of the noVNC is downloaded to the browser of the client for execution when the client browser accesses the websocket service.

Illustratively, the embodiment first installs the UltraVNC in the remote host, and starts the service, sets the password; js, and selecting npm package manager during installation; after the node.js is installed, installing ws and optimal modules (which are needed for executing websocket.js files); after the ws and optimists are installed, a node _ modules directory is generated under C \ Users \ Adminstrator \. In the embodiment, a nonVNC.zip is decompressed to a node _ modules directory, and then a websocket-master.zip is decompressed to the nonVNC directory; newly establishing an inbound rule, and executing websocks. The http of the 9000 port is forwarded and linked to the 5900 port; finally, the address bar of the browser is input with http://10.70.117.185: 9000/; and (5) searching for filename + (index)/html in websocket.

In some alternative embodiments, the step of accessing the primary commissioning device through the noVNC client in step S200 may include steps S250-S260:

s250: acquiring user information, and starting a daemon example of the virtual network control service according to the user information;

s260: in the daemon example, starting a virtual network control service of a NoVNC client, and accessing a processing debugging device through the virtual network control service;

specifically, in the embodiment, each user may start a separate VNC service daemon instance; so in embodiments each user connected through a VNC must start a new instance of a daemon process; the user information obtained in the step may be a unique identifier of the user, and is used to determine whether the user is a new user. Illustratively, the CentOS7 uses a system daemon to start other services, each service running under the local system has a service unit file stored under the/lib/system/directory, and the automatically started process has a link to the service unit file at the time of start.

In some alternative embodiments, the remote gateway in the embodiment system is an open source gateway, wherein the open source gateway is at least one of OpenResty, Kong, Zuul2, and springcloudggateway.

In some alternative embodiments, the process of connecting to the controller of the primary debugging device through the remote gateway in step S200 may further include steps S270 to S280:

s270, analyzing the debugging instruction to obtain an IP address and a port number of the NOVNC client;

s280, generating a first access link according to the IP address and the port number, and determining a forwarding path of the Web sockets according to the first access link and the open source gateway.

Exemplarily, the middle-layer microservice in the embodiment receives an access request initiated by a client, and calls an IP and a port number matched with a nonvc server corresponding to the access request from a server; then generating a unique token according to the access request and binding the unique token with the access path only exposed to the server side to generate a new access path; the new access path saves https and WebSocket to determine a routing relation and pushes the routing relation to the client; and re-inputting a new URL to the client, modifying Host information and Origin information in an http request header through the open source gateway, and determining a forwarding path of https and WebSocket according to the new URL and the routing relation.

In addition, in some optional embodiments, after the step of debugging the controller, the remote virtual simulation method further includes the following step S400 of building an automation line in a virtual environment by simulation, and driving the automation line by the debugged controller.

For example, as shown in fig. 2, the embodiment simulates an automobile welding production line, that is, virtual debugging of an automobile automation production line can be implemented in virtual simulation software, and a virtual automation production line is built up in a one-to-one ratio in a virtual simulation environment. The virtual environment is driven by the PLC, and a remote engineer can access an operating system of the field computer through a browser at any time when a network is unobstructed by using the remote engineer as a NoVNC proxy server according to NoVNC viewer software on a remote computer to perform virtual simulation verification.

the virtual simulation unit is used for determining the target debugging equipment as primary debugging equipment according to the debugging information, deploying the NoVNC client, accessing the primary debugging equipment through the NoVNC client, connecting to a controller of the primary debugging equipment through a remote gateway in a virtual environment of the NoVNC client according to a debugging instruction, and debugging the controller;

and the user interaction unit is used for acquiring the debugging instruction.

Various changes and specific examples of the content recommendation method based on the remote virtual simulation of the nonvnc described in detail in the first aspect are also applicable to the remote virtual simulation system based on the nonvnc of the present embodiment, and a person skilled in the art can clearly know an implementation method of the remote virtual simulation system based on the nonvnc in the present embodiment through the foregoing detailed description of the remote virtual simulation method based on the nonvnc, so for the brevity of the description, details are not described here.

In a third aspect, as shown in fig. 3, an embodiment of the present application provides a remote virtual simulation device based on a noVNC, which includes: at least one processor; at least one memory for storing at least one program; when the at least one program is executed by the at least one processor, the at least one processor is caused to perform the method of the first aspect.

The device in an embodiment may be a removable device itself, or a stand-alone device independent thereof, having stored thereon a computer program which, when executed by a processor, performs any of the methods or steps of the first aspect.

The device may contain one or more processors and memory, which may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and which may control other components in the electronic device to perform the desired functions. The memory may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. Volatile memory can include, for example, Random Access Memory (RAM), cache memory (or the like). The non-volatile memory may include, for example, Read Only Memory (ROM), a hard disk, flash memory, and the like. One or more computer program instructions may be stored on a computer-readable storage medium and executed by a processor to implement the methods of the various embodiments of the first aspect and/or other desired functions.

From the above specific implementation process, it can be concluded that the technical solution provided by the present invention has the following advantages or advantages compared to the prior art:

1. the technical scheme of this application can be through the equipment that browser page direct access needs the debugging, the long-range virtual production line emulation of carrying out, debugging and control, the scheme still adopts remote gateway routing technique, let the engineer have the ability long-rangely to do the gateway through the computer and carry out the remote debugging to PLC, increase substantially debugging efficiency, break through geographical limitation, replaced the mode that current debugging personnel concentrated the debugging, saved human cost and financial cost

In alternative embodiments, the functions/acts noted in the block diagrams may occur out of the order noted in the operational illustrations. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality/acts involved. Furthermore, the embodiments presented and described in the flow charts of the present invention are provided by way of example in order to provide a more thorough understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and in which sub-operations described as part of larger operations are performed independently.

Furthermore, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise stated to the contrary, one or more of the functions and/or features may be integrated in a single physical device and/or software module, or one or more of the functions and/or features may be implemented in a separate physical device or software module. It will also be appreciated that a detailed discussion of the actual implementation of each module is not necessary for an understanding of the present invention. Rather, the actual implementation of the various functional modules in the apparatus disclosed herein will be understood within the ordinary skill of an engineer, given the nature, function, and internal relationship of the modules. Accordingly, those skilled in the art can, using ordinary skill, practice the invention as set forth in the claims without undue experimentation. It is also to be understood that the specific concepts disclosed are merely illustrative of and not intended to limit the scope of the invention, which is defined by the appended claims and their full scope of equivalents.

The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions.

In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

While embodiments of the invention have been shown and described, it will be understood by those of ordinary skill in the art that: various changes, modifications, substitutions and alterations can be made to the embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

While the preferred embodiments of the present invention have been illustrated and described, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

1. A remote virtual simulation method based on a NoVNC is characterized by comprising the following steps:

acquiring debugging information of target debugging equipment;

2. A nonvnc-based remote virtual simulation method according to claim 1, wherein after the step of accessing the primary commissioning device through the nonvnc client, the remote virtual simulation method further comprises the steps of:

and acquiring a monitoring instruction, and performing remote virtual simulation monitoring on debugging equipment of the virtual environment according to the monitoring instruction.

3. A nonvnc-based remote virtual simulation method according to claim 1, wherein the step of deploying a nonvnc client includes:

deploying and starting a virtual network control service;

4. The method according to claim 1, wherein the step of accessing the primary debugging device through the NoVNC client comprises:

5. The nonVNC-based remote virtual simulation method of claim 3, wherein the remote gateway is an open-source gateway, and the open-source gateway is at least one of OpenResty, Kong, Zuu 2 and SpringCloudGateway.

6. The nonVNC-based remote virtual simulation method according to claim 5, wherein the step of connecting to the controller of the primary debugging device through a remote gateway includes:

7. A NOVNC-based remote virtual simulation method according to any one of claims 1 to 6, wherein after the step of commissioning the controller, the remote virtual simulation method further comprises the steps of:

8. A remote virtual simulation system based on a noVNC, comprising:

and the user interaction unit is used for acquiring the debugging instruction.

9. A remote virtual simulation device based on a noVNC, comprising:

at least one processor;

at least one memory for storing at least one program; when executed by the at least one processor, the at least one program causes the at least one processor to execute a method for remote virtual simulation based on a NOVNC according to any one of claims 1 to 7.

10. A storage medium having stored therein a processor-executable program, wherein the processor-executable program, when executed by a processor, is configured to execute a nonvc-based remote virtual simulation method according to any of claims 1 to 7.