CN116501608A

CN116501608A - Information physical comprehensive test bed frame structure based on Dajiang machine armor master

Info

Publication number: CN116501608A
Application number: CN202310289507.2A
Authority: CN
Inventors: 秦逸; 许畅; 毛心怡
Original assignee: Nanjing University
Current assignee: Nanjing University
Priority date: 2023-03-23
Filing date: 2023-03-23
Publication date: 2023-07-28

Abstract

The invention relates to an information physical integrated test bed structure based on a Dajiang machine tool master, which comprises an application support frame and an environment synchronous execution frame, and provides execution support for a software support component and an operation environment in an information physical fusion system respectively. The invention combines the software in the CPS application development process with the hardware in the loop running environment, maintains the real software-hardware interaction process of the hardware in the loop, combines the characteristics of low construction cost and safe running of the software in the loop virtual environment, and ensures the safety, observability and operability of the software execution process in CPS application development.

Description

Information physical comprehensive test bed frame structure based on Dajiang machine armor master

Technical Field

The invention relates to the field of development and application of information physical fusion systems, in particular to an information physical comprehensive test bed structure based on a large-scale mechanical armor master.

Background

An information Physical fusion system (CPS) is an engineering system consisting of a set of highly fused software components and Physical environments. Unlike traditional software, CPS application takes on interaction task of connection information and physical space of "perception" and "execution" besides functional task, the perception task can be regarded as reading CPS application from environment, and the execution task can be regarded as outputting CPS application to environment. Because the physical space has the characteristic of high dynamic uncertainty, the interaction behavior of CPS application and environment has universal uncertainty, and the development process of CPS application is challenged.

In order to develop CPS applications efficiently and safely, the existing software development process of CPS can be divided into two stages of Software In Loop (SiL) and Hardware In Loop (HiL) according to the development environment, and the characteristics are as follows: the software builds the running environment of CPS application in the loop stage usually by using a simulation or emulation mode, so that the cost of building the running scene is reduced, but the reliability of the software cannot be ensured because the simulator and the real hardware have larger difference and only use the virtual environment for development; the hardware runs the software in the physical environment of final deployment in the ring stage, so that the interaction between the software and the real hardware is ensured, but the physical development environment of the ring by using the hardware is often required to bear larger construction cost and debugging risk.

Because of the advantages and disadvantages of both environments, there have been some attempts to combine both to design a comprehensive development framework that is intermediate to software-on-loop and hardware-on-loop, but these frameworks all suffer from the following problems:

(1) In the development process, CPS applications run with insufficient observability and operability.

(2) The hardware has higher construction cost in the development environment of the ring, and the operation scene can not be quickly constructed.

(3) When combining the virtual environment of software in the ring and the physical environment of hardware in the ring, the set virtual simulation scene is limited by the current physical environment.

Disclosure of Invention

The invention aims to provide an information physical comprehensive test bed architecture based on a Dajiang machine tool, which is a tool for assisting a software development process in an information physical fusion system (CPS), reserves the advantages of high observability and operability and low running risk and construction cost of software in a ring development process, and simultaneously uses real physical hardware to execute so as to avoid the problem of insufficient cognition of the software on real software-hardware interaction behavior in the ring development.

In order to achieve the above purpose, the technical scheme provided by the invention is as follows:

the information physical comprehensive test bed architecture based on the Dajiang machine tool master comprises an application support frame and an environment synchronous execution frame, and provides execution support for a software support component and an operation environment in an information physical fusion system CPS respectively; the application support framework is used for receiving CPS application to be executed uploaded by a user, providing a software support component for application execution and simulating execution, dividing an execution task into two classes of perception type tasks and action execution type tasks through the software support component during the simulation execution, transmitting the perception type tasks to software in the environment synchronous execution framework for processing in a ring operation mode, and transmitting the action execution type tasks to hardware in the environment synchronous execution framework for processing in the ring operation mode; the environment synchronous execution framework is used for maintaining a virtual environment in which software runs in a ring and a real physical environment in which hardware runs in the ring, executing a perception class task by using an execution scene in the virtual environment in which the software runs in the ring as input of CPS application, outputting a real hardware execution result of an action execution class task by using the real hardware execution result of the action execution class task in the real physical environment in which the hardware runs in the ring as output of CPS application, and generating a comprehensive running environment between the hardware in the ring and the software in the ring through synchronizing scenes in the virtual environment and the real physical environment.

In order to optimize the technical scheme, the specific measures adopted further comprise:

the application supporting frame comprises an analog operation module, a perception processing module and an execution processing module; the simulation running module is responsible for initializing CPS application loading and application execution supporting environment SDK loading in configuration, performing simulation execution on CPS application uploaded by a user by combining with the application execution supporting environment SDK, and performing local recording decision; the perception processing module is responsible for packaging a perception class task call request and then sending the request to a virtual environment of software running in a ring; the execution processing module is responsible for sending an action execution class task call request to a real physical environment of hardware running in a ring, and analyzing and packaging return value information of the real physical environment as an execution result.

The environment synchronous execution framework comprises a software on-loop running module, a hardware on-loop running module and a scene synchronization module; the software in-loop running module is responsible for maintaining the virtual environment of the software in-loop, and after receiving the perception class task, executing the perception class task as input of CPS application by using the execution scene of the software in the virtual environment in-loop running; the hardware-in-loop running module is responsible for maintaining the real physical environment of the hardware-in-loop, and takes the real hardware execution result of the action execution class task as the output of CPS application in the real physical environment of the hardware-in-loop running after receiving the action execution class task; the scene synchronization module is responsible for calculating a real hardware execution result in a real physical environment, and synchronizing scenes in a virtual environment and the real physical environment to generate a comprehensive running environment between a hardware in-loop and a software in-loop.

Further, the user CPS application execution flow includes the following specific steps:

(1) Initializing configuration: uploading CPS application to be executed by a user through an application supporting framework;

(2) Execution environment synchronization: the synchronous environment synchronously executes the virtual environment and the real physical environment in the framework;

(3) The software executes in a loop: executing a perception task by the virtual environment to acquire CPS application input information;

(4) The application performs the simulation: giving CPS application input information to an application support framework, analyzing and generating an action execution class task as CPS application output;

(5) Hardware performs in the loop: CPS application output is performed in a real physical environment.

Wherein, the initialization configuration of the step (1) and the application execution simulation of the step (4) are carried out by the application support framework; the step (2) of executing the environment synchronization, (3) of executing the software in the ring and (5) of executing the hardware in the ring are responsible for execution by the environment synchronization execution framework.

Further, the perception class task is a perception class API, and the action execution class task is an action execution class API.

Further, the software supporting component is an extended SDK for carrying out secondary development on a primary SDK of a Dajiang machine A, modifies and encapsulates key APIs of CPS application interaction with an external environment, and divides the request and execution process of the API call so as to realize the noninductive switching of the API call request of the CPS application between a virtual environment and a real physical environment.

As a preferred solution, the application of the support frame is performed as follows:

loading CPS application and execution environment: initializing an execution environment by using an application support framework, and loading the latest CPS application submitted by a user into the execution environment by using a python module dynamic loading function, wherein the execution environment is an extended SDK after a patch library is introduced into a primary SDK of a Dajiang machine master;

executing a CPS application: the application supporting framework sequentially executes the sentences in the CPS application, and if the sentences are the perception class API or the action execution class API, the application supporting framework jumps to the extended SDK to execute the execution process;

executing a perceptual class API in a CPS application: calling a perception class API code in the extended SDK, wherein the code decouples the data subscription of specific hardware of a specific sensor in a physical world hardware component, reads a virtual sensor result from a virtual world, provides a unified interface for interaction with an external environment, and ensures flexible virtual environment configuration; the unified interface for interaction with the external environment is a virtual sensor modeling interface configured in a virtual environment, and is hereinafter referred to as a data reading interface.

Executing an action execution class API in a CPS application: the method comprises the steps of executing class API codes by using actions in an extended SDK, wherein the codes provide task splitting execution functions and are used for coping with API context switching or interrupt triggering conditions during hardware execution, so that a safe hardware interaction process is ensured;

And (3) local recording: context data of CPS application and integrated running environment interaction combining virtual environment and real physical environment are recorded in a local database, and calling parameters, executing data and return values of API calling in the extended SDK are recorded.

As a preferred scheme, the execution process of the environment synchronization execution framework is as follows:

maintaining a virtual operating environment of software in a ring: setting up a virtual scene and a machine tool master car model in a third-party virtual engine, and providing a scene setting interface to update the scene, such as the addition of obstacles, the movement of the car model and the like;

maintaining the physical running environment of hardware in a ring: setting an experiment site in the physical world, and connecting a large-scale machine armor master so as to obtain an operable hardware component;

processing a perception class API: when a sensing request is received, reading current scene data in a virtual scene through a data reading interface, and generating CPS application input from a virtual environment;

processing action execution class API: when receiving the action execution class API, controlling a large-scale robot master to safely execute actions on an experimental field;

the change result of the physical scene is measured in a numerical mode: generating real-time positioning data of a large-scale mechanical master in a physical scene, and calculating the relative position of the large-scale mechanical master on an experimental field in real time by using a four-way sensing positioning algorithm;

Positioning result synchronization and scene synthesis: synchronizing an execution result of the CPS application in the loop scene by the hardware to a virtual scene and displaying a comprehensive operation environment, mapping position change data of a Dajiang machine tool during operation of the action execution class API to the virtual scene, generating a comprehensive operation environment containing the action execution class API result of the hardware in the loop and the perception class API result of the software in the loop, and displaying the behavior of the CPS application in the current scene for a user.

The hardware adopts a boundary detection mechanism and a gesture adjustment mechanism when the ring running module processes action to execute the class API; the hardware-in-loop running module consists of a data stream processing module, an instruction analysis module, an execution state control module, a boundary monitoring module, an attitude scheduling module and a positioning computer module; the data flow processing module receives an input action execution class API and forwards a request of the action execution class API to the instruction analysis module; the instruction analysis module receives action execution API calls from the data stream processing module, the boundary monitoring module and the gesture scheduling module, and finally analyzes the action execution API calls into a bottom instruction for the Dajiang machine staff and interacts with the Dajiang machine staff; the execution state control module controls the execution state of the hardware in the whole loop module to be an execution state or a scheduling state; the boundary monitoring module is used for realizing a boundary monitoring mechanism; the gesture scheduling module is used for realizing a gesture adjustment mechanism; the positioning calculator module is used for supporting the numerical measurement of physical scene change;

The scene synchronization module consists of a scene synchronization data filter and a comprehensive scene updating module, wherein the scene synchronization data filter and hardware interact with each other in a loop execution module and are responsible for counting output results of CPS application execution in a real physical environment to the real physical environment; the comprehensive scene updating module and the software interact in the loop execution module, and the action execution result of the CPS application is mapped to the virtual environment built by the virtual engine.

Compared with the prior art, the invention has the beneficial effects that:

the interaction process of the information physical integrated test bed architecture based on the Dajiang machine tool master and the CPS application is that the software generally obtains the environment input in the virtual environment in the ring, and meanwhile, the hardware generally executes output on real hardware in the ring, and the execution of the software in the ring and the hardware in the ring stage is intermediate.

The information physical integrated test bed architecture based on the Dajiang machine tool master is split into an application support frame and an environment synchronous execution frame, and corresponds to two parts of a software component and an operating environment in an information physical fusion system (CPS) respectively.

The application supporting framework is a software framework for supporting CPS application uploading, simulating execution and assisting in configuring an execution environment. The application supporting framework accepts CPS applications to be executed uploaded by users, provides software supporting components (software development kit SDKs) for application execution, and simulates the execution, during the execution, the software supporting components split the CPS applications into tasks interacting with environments, and the tasks interacting with the environments are usually API calls in the software development kit, namely a perception class API and an execution class API, and the perception class API and the execution class API are transferred to corresponding running environment processes. Through the steps, a user does not need to configure a software supporting environment for CPS application execution, and the construction cost of a software assembly end is reduced.

The environment synchronous execution framework is a software framework for connecting the information physical integrated test bed framework based on the Dajiang Jijia master and the running environment used by CPS application development. The environment synchronous execution framework maintains a virtual operation environment built based on a unit virtual engine and a physical operation environment built based on unmanned vehicle hardware of a large-scale master EP model, generates input of the environment to CPS application in the virtual operation environment, executes output of the CPS application to the environment in the physical operation environment, synchronizes execution data of the CPS application and displays the CPS application in combination with an operation scene. Through the steps, the module generates a comprehensive operation environment integrating virtual operation scenes and real hardware behaviors, the construction cost and risk of a physical scene are reduced while the rapid iteration of the scenes and the feedback of the real hardware are ensured, and the observability is improved.

Compared with the prior art, the invention has the remarkable advantages that:

1. the information physical comprehensive test bed architecture based on the Dajiang machine tool master divides the execution process of the information physical fusion system into an application support frame for maintaining CPS application execution and an environment synchronous execution frame for maintaining an execution scene in the environment; the application support framework has behavior records with API granularity, and the hardware state is controlled in real time in the environment synchronous execution framework; therefore, the observability and operability of the whole architecture are better;

2. The environment synchronous execution framework based on the information physical integrated test bed architecture of the Dajiang machine tool master still provides an execution result of CPS application on real hardware; meanwhile, the cost of scene construction in the physical world is reduced by using a mode of constructing a simple experiment field and constructing a virtual scene.

3. For the problem that virtual scene construction is limited by a physical experiment field, the information physical comprehensive test bed architecture based on the Dajiang Jijia master supports action execution type API execution of any parameter through expanding SDK in an application support frame, maintains a virtual-real combined operation environment in an environment synchronous execution frame, builds the experiment field in the physical scene for execution, and displays a complete execution result in the virtual scene, thereby solving the scene limitation of the action execution type API.

Drawings

FIG. 1 is a diagram of the information physical integrated test bed frame structure based on the Dajiang Jijia master.

Fig. 2 is a block diagram of the module design of the present invention.

Fig. 3 is a flow chart of an implementation of the application support framework.

Fig. 4 is a general pattern of CPS application code.

FIG. 5 is a native software development kit using a Patch Patch library.

FIG. 6 is an expanded perceptual class API execution.

FIG. 7 is an expanded action execution class API execution.

FIG. 8 is an execution flow diagram of an environment synchronized execution framework.

FIG. 9 is a block diagram of the software in loop module design in the information physical integrated test bed architecture based on the Massa Medicata Fermentata.

FIG. 10 is an example of a virtual execution environment for software in a loop.

FIG. 11 is a block diagram of the hardware-in-the-loop module design in the information physical integrated test bed architecture based on the Massa Medicata Fermentata.

FIG. 12 is an example of a physical operating environment for hardware in the loop.

FIG. 13 is an action class API execution under a combination of a boundary detection mechanism and a gesture adjustment mechanism.

Fig. 14 is a diagram illustrating a data format of a four-way sensing technology.

FIG. 15 is a diagram of the design of the scene synchronization module in the information physical integrated test bed frame based on the Dajiang machine staff.

Detailed Description

The above-described matters of the present invention will be further described in detail by way of examples, but it should not be construed that the scope of the above-described subject matter of the present invention is limited to the following examples, and all techniques realized based on the above-described matters of the present invention are within the scope of the present invention.

As shown in fig. 1, the invention provides an information physical integrated test bed architecture based on a large-scale mechanical master, which comprises an application support frame and an environment synchronous execution frame, and provides execution support for a software support component and an operation environment in an information physical fusion system CPS respectively; the application support frame is used for receiving CPS application to be executed uploaded by a user, providing a software support component for application execution and simulating execution, dividing an execution task into two classes of perception type tasks and action execution type tasks through the software support component during the simulation execution, transmitting the perception type tasks to software in the environment synchronous execution frame to perform processing in ring operation, and transmitting the action execution type tasks to hardware in the environment synchronous execution frame to perform processing in ring operation; the environment synchronous execution framework is used for maintaining a virtual environment in which software runs in a ring and a real physical environment in which hardware runs in the ring, executing a perception class task by using an execution scene of the software in the virtual environment in which the software runs in the ring as input of the CPS application, outputting a real hardware execution result of the action execution class task by using the real hardware execution result of the action execution class task in the real physical environment in which the hardware runs in the ring as output of the CPS application, and generating a comprehensive running environment between the hardware in the ring and the software in the ring through synchronizing the scenes in the virtual environment and the real physical environment.

The invention combines the software in the loop running environment and the hardware in the loop running environment in the CPS application development process by respectively providing support for the software components and the execution environment in the information physical fusion system, so that the characteristics of low construction cost and safe running of the software in the loop virtual environment are combined while the real software and hardware interaction process of the hardware in the loop is reserved, and the safety, observability and operability of the software execution process in the CPS application development process are ensured.

The application support frame comprises a simulation running module, a perception processing module and an execution processing module; the simulation running module is responsible for initializing CPS application loading and application execution supporting environment SDK loading in configuration, performing simulation execution on CPS application uploaded by a user by combining the application execution supporting environment SDK, and performing local recording decision; the perception processing module is responsible for packaging the perception class task call request and then sending the request to a virtual environment of the software running in a ring; the execution processing module is responsible for sending an action execution class task call request to a real physical environment of hardware running in a ring, and analyzing and packaging return value information of the real physical environment as an execution result.

The environment synchronous execution framework comprises a software on-loop running module, a hardware on-loop running module and a scene synchronous module; the software in-loop running module is responsible for maintaining the virtual environment of the software in-loop, and executing the perception class task as input of the CPS application by using the execution scene of the software in the virtual environment of the in-loop running after receiving the perception class task; the hardware-in-loop running module is responsible for maintaining the real physical environment of the hardware-in-loop, and takes the real hardware execution result of the action execution class task as the output of the CPS application in the real physical environment of the hardware-in-loop running after receiving the action execution class task; the scene synchronization module is responsible for calculating a real hardware execution result in a real physical environment, and synchronizing scenes in a virtual environment and the real physical environment to generate a comprehensive running environment between a hardware in-loop and a software in-loop.

In an embodiment, the aware class task is a aware class API, and the action execution class task is an action execution class API.

In the embodiment, the software supporting component is an extended SDK for secondary development of a primary SDK of a Dajiang machine A, modifies and encapsulates a key API for interaction between the CPS application and an external environment, and divides the request and execution process of the API call so as to realize the noninductive switching of the request of the API call of the CPS application between the virtual environment and the real physical environment.

Specifically, in the application support framework, the simulation running module is responsible for user CPS application loading, application execution environment configuration and simulation execution functions; configuring an extended SDK environment, receiving CPS application and executing the CPS application sentence by sentence, and recording an execution result; and if the current execution statement is a perception class API or an action execution class API, the jump corresponding module processes. The perception processing module is responsible for executing a perception class API in the CPS application, generating a request of the perception class API and processing an execution result returned by the environment end, and the execution processing module is responsible for executing an action execution class API in the CPS application, sending an execution command of the action execution class API and waiting for the execution result of the environment end;

specifically, in the environment synchronous execution framework, the software in-loop running module is responsible for maintaining the virtual running environment of the software in-loop and processing the perception class API, and the hardware in-loop running module is responsible for maintaining the real physical environment of the hardware in-loop and processing the action execution class API. The scene synchronization module is responsible for calculating action execution results and generating a comprehensive operation environment.

The user CPS application execution flow comprises the following specific steps:

(1a) Loading CPS application and execution environment: initializing an execution environment by using an application support framework, and loading the latest CPS application submitted by a user into the execution environment by using a python module dynamic loading function, wherein the execution environment is an extended SDK after a patch library is introduced into a primary SDK of a Dajiang machine master;

(4a) Executing a CPS application: the application supporting framework sequentially executes the sentences in the CPS application, and if the sentences are the perception class API or the action execution class API, the application supporting framework jumps to the extended SDK to execute the execution process;

(4b) Executing a perceptual class API in a CPS application: calling a perception class API code in the extended SDK, reading a virtual sensor result from a virtual world, and providing a unified interface for interaction with a virtual environment, namely a data reading interface, so as to ensure flexible virtual environment configuration;

(4c) Executing an action execution class API in a CPS application: the method comprises the steps of executing class API codes by using actions in an extended SDK, wherein the codes provide task splitting execution functions and are used for coping with API context switching or interrupt triggering conditions during hardware execution, so that a safe hardware interaction process is ensured;

(1b) And (3) local recording: context data of CPS application and integrated running environment interaction combining virtual environment and real physical environment are recorded in a local database, and calling parameters, executing data and return values of API calling in the extended SDK are recorded.

The simulation running module is responsible for (1 a) loading user CPS application and execution environment, (4 a) executing CPS application and (1 b) local recording function, and the module executes the program file uploaded by the user by combining with the extended SDK, and simultaneously makes a local recording decision; the simulation running module carries out rapid expansion of the SDK execution environment configuration by introducing a patch library and modifying the function pointer under the condition of not modifying the official SDK source code.

The simulation running module is responsible for (1 b) the local recording function. Because the extended SDK code and the official SDK code maintain consistent parameters, callback processes and return values, the local recording function of the Dajiang machine staff is multiplexed, the data recorded by the simulation running module is consistent with the local recording function of the Dajiang machine staff, but the recorded data only comprises sensing and executing results related to CPS application.

Specifically, the simulation running module is responsible for loading the user CPS application and the execution environment (4 a), and the extension perception type API code and the extension action type API code are executed by other modules of the jump application supporting framework in the extension SDK code.

The perception processing module is responsible for (4 b) executing a perception class API in the CPS application, sending an API call request to a virtual running environment of software in a ring, and waiting for an execution result; the perception processing module adds external data synchronous codes through a pub-sub mechanism of rewriting data subscription and receives perception data from the outside.

The execution processing module is responsible for (4 c) executing the action execution class API in the CPS application, sending the API call request to the physical running environment of the hardware in the ring, and processing the execution record. The execution processing module adds the context data and the platform timer of the action execution class API through the action execution segmentation mechanism, and ensures the complete execution of the action execution class API by supporting context switching.

Referring to fig. 4, the user CPS application loaded by the simulation run module can be written as a decision program having a general mode "CPS application periodically checks the state of the dolly (status=call aware class API) and performs a specified action (Call action execution class API)" when the dolly satisfies the corresponding condition.

Referring to fig. 5, the application execution environment of the simulation run module configuration is as follows. The Dajiang machine tool university provides a software development kit SDK for supporting interactive functions of applications and environments, and the kit is called a native SDK; the simulation running module introduces a Patch library, modifies function pointers of a perception class API and an action execution class API, and calls codes of an extended SDK in the Patch library when the execution CPS application encounters a perception class API and an action execution class API statement; the module divides the request and execution process of the API call, and realizes the noninductive switching of the API call request of CPS application between the virtual environment and the real environment;

referring to fig. 6, after extension, the specific procedure performed by the perceptual class API is described as follows:

the perception processing module receives the request of the extended perception type API from the simulation execution module, and periodically recalls according to the user perception frequency set in the perception type API parameters;

If the perception class API is the first subscription, initializing a perception processing module: sending a sensing request, subscribing the maximum sensing frequency to a software loop running module, and receiving response data; the transmitted awareness requests and received response data are referred to herein as awareness-type data streams.

The execution processing module of the application supporting framework is responsible for (4 c) executing the action execution class API in the CPS application, sending the API call request to the physical running environment of the hardware in the ring, and processing the execution record. The execution processing module adds the context data and the platform timer of the action execution class API through the action execution segmentation mechanism, and ensures the complete execution of the action execution class API by supporting context switching.

Referring to fig. 7, after extension, the specific procedure executed by the action execution class API is divided into a plurality of parts, and is described as follows:

the execution processing module receives the action execution class API call from the simulation execution module, records the action execution class API call as a complete action, sets an action context and monitors the execution progress;

generating sub-actions (also an action execution class API), sending the sub-actions to a hardware-in-loop running module, calling physical hardware to run, receiving sub-action execution results given by the hardware-in-loop running module, and updating the execution progress of the complete actions; the transmitted sub-actions and the received sub-action execution results are referred to herein as an action execution data stream.

If the current execution progress is incomplete, the execution processing module continues to generate a new sub-action and sends the new sub-action to the hardware-in-the-loop module, and if the current execution progress is completed, the execution processing module returns a final execution result to the simulation execution module.

(2a) Maintaining a virtual operating environment of software in a ring: setting up a virtual scene and a machine tool master car model in a third-party virtual engine, and providing a scene setting interface to update the scene;

(2b) Maintaining the physical running environment of hardware in a ring: setting an experiment site in the physical world, and connecting a large-scale machine armor master so as to obtain an operable hardware component;

(3a) Processing a perception class API: when a sensing request is received, reading current scene data in a virtual scene through a data reading interface, and generating CPS application input from a virtual environment;

(5a) Processing action execution class API: when receiving the action execution class API, controlling a large-scale robot master to safely execute actions on an experimental field;

(5b) The change result of the physical scene is measured in a numerical mode: generating real-time positioning data of a large-scale mechanical master in a physical scene, and calculating the relative position of the large-scale mechanical master on an experimental field in real time by using a four-way sensing positioning algorithm;

(2c) Positioning result synchronization and scene synthesis: synchronizing an execution result of the CPS application in the loop scene by the hardware to a virtual scene and displaying a comprehensive operation environment, mapping position change data of a Dajiang machine tool during operation of the action execution class API to the virtual scene, generating a comprehensive operation environment containing the action execution class API result of the hardware in the loop and the perception class API result of the software in the loop, and displaying the behavior of the CPS application in the current scene for a user.

The software in the environment synchronous execution framework is responsible for (2 a) maintaining the virtual running environment of the software in the ring and (3 a) processing the perception class API, and fig. 9 is a design structure diagram of the software in the ring running module.

Further, the software in-loop running module is responsible for (2 a) maintaining a virtual running environment of the software in-loop, including two parts of virtual environment building and virtual environment maintenance:

the virtual running environment is built on a third-party virtual engine Unity, and the building content is shown in fig. 10, wherein the virtual running scene is a virtual field (left) containing obstacles, and the virtual running hardware is modeled by a machine tool master simulator (right); here, the virtual running scene is not fixed to a specific map, and can be quickly switched and reset.

The maintenance of the virtual running environment is realized through a coupling interface (figure 9) of a definition and a third party platform, the interface ensures that any virtual engine platform can be accessed into an information physical integrated test bed architecture (simply called a test bed) based on a Dajiang machine tool university after simple configuration, and an execution result of CPS software in the test bed is obtained, so that the complexity of building and updating the virtual environment is reduced;

taking access of a Unity 3D virtual engine as an example, a unified configuration interface writes a C# script at a Unity end, a communication plaintext format of a socket is written at a test bed end, and the interface comprises two types: a scene setting interface and a data reading interface;

the scene setting interface receives scene setting data, taking the road finding software development scene in the field as an example, wherein the scene setting data is maze map data comprising the number of obstacles and the real-time vehicle position; the interface can be used for rapidly switching the operation scene at low cost and displaying the currently-facing execution environment of the CPS application.

The data reading interface generates environment sensing data, taking sensing data of an infrared ranging sensor in front as an example, wherein the environment sensing data is a floating point number variable, and the distance from a vehicle model to a front obstacle model in a virtual scene is stored; the sensing type interface predefines sensing data formats of a large number of sensors, including data such as cameras (stream data), ranging sensors (floating point numbers), angle sensors (integers) and the like used by a Dajiang Jijia master, and the interface can be used for converting scenes in a virtual engine into CPS application input in real time;

Further, the software is responsible (3 a) for handling the awareness class API at the loop running module; referring to fig. 9, the processing procedure of the software on the received perception class API at the ring running module is as follows:

the software receives a perception type API subscription request sent by a perception processing module in a ring module, and reads the maximum perception frequency set in subscription parameters;

● Periodically calling a data reading interface coupled with a third-party virtual engine by using the maximum perception frequency to obtain environment perception data in a virtual environment as environment output;

● For the obtained environment output information, adjusting data (such as adding ranging noise and the like) according to an uncertainty model of the current environment to generate input information of CPS application;

the hardware in the environment synchronous execution framework is responsible for (2 b) maintaining the virtual running environment of the software in the ring, (5 a) processing action execution class API and (5 b) numerical measurement physical scene change, and FIG. 11 is a design structure diagram of the hardware in-ring running module.

Further, the hardware-in-loop running module is responsible for (2 b) maintaining the physical running environment of the hardware-in-loop, including two parts of physical environment building and physical environment maintenance:

the environment synchronous execution framework comprises a software in-loop running module, a hardware in-loop running module and a scene synchronous module. The software is in charge of (2 a) maintaining a virtual running environment of the software in a ring and (3 a) processing a perception class API, the module models an execution scene (comprising an initial position of a large-scale driver cart, a scene obstacle and the like) on a third-party virtual engine, processes a perception class API request, reads perception data from the environment and generates input of CPS application; the software can be inserted into the platform after simple configuration by defining the coupling interface between the software and the third-party platform in the ring running module, so that the cost of scene construction is reduced.

Further, the hardware-in-loop running module is responsible for (2 b) maintaining the physical running environment of the hardware-in-loop and (5 a) processing action execution class API, and is connected with a Dajiang machine tool master trolley to control chassis hardware to process action execution class API requests in square open experimental fields and output execution results to the environment; the hardware-in-loop running module reduces the probability of physical collision of the API through a gesture adjustment mechanism, adds running environment monitoring during execution through a boundary monitoring mechanism, and terminates interaction behaviors possibly causing physical collision in advance, so that the safety of the hardware-in-loop interaction process is ensured.

Further, the scene synchronization module is responsible for (5 b) numerically measuring the change result of the physical scene and (2 c) positioning result synchronization and scene synthesis, and the module positions a Dajiang machine tool master cart (short for a cart) in real time, defines the physical scene change as cart position change data during operation, measures and maps the physical scene change data into a virtual scene, thereby displaying a comprehensive operation environment containing both the action execution type API result of hardware in a ring and the perception type API result of software in the ring, and displaying the behavior of CPS application in the current scene for a user; the scene synchronization module calculates the position of the trolley in the physical field in real time through a four-way sensing positioning technology, restores the execution result of the API executed by the current action by combining the execution flow data of the environment synchronization execution framework, synchronizes the execution result into the virtual environment, and displays the behavior of CPS application in the current scene to a user, thereby ensuring the observability during the operation.

Compared with the existing CPS application test bed, the CPS application test bed has the advantages that CPS application and real hardware in the framework are interacted, so that the defect of cognition deficiency of CPS application to hardware behavior caused by using simulation or emulation means in a loop running environment of the pure software is avoided; compared with a test bed with pure hardware in a ring, CPS application in the framework uses the virtual scene as an application input source, so that huge expenditure and equipment damage risk of building a real scene when the hardware runs in the ring are avoided.

As shown in fig. 12, the construction of the physical operation environment depends on manual labor, and the construction structure of the constructed environment is as follows: the operation scene is a square simple experiment field with a partition plate as a boundary, and the operation hardware is a trolley carrying a four-way infrared ranging sensor; the maintenance of the physical operation environment needs to ensure that the connection of the local area network formed by the trolley and the computer is stable, and the ranging information of the infrared sensor can be transmitted to the computer.

Further, the hardware-in-loop execution module is responsible (5 a) for processing action execution class APIs;

the key point of the hardware-in-loop running module (5 a) for processing the action execution class API is to ensure the safety of the interaction behavior of the trolley hardware; the existing hardware interaction behavior is unsafe, and due to the lack of a guarantee mechanism, the module can collide with the boundary of the field and the trolley hardware in the process of executing the action execution class API, so that the damage of the trolley hardware is caused; the boundary detection mechanism and the gesture adjustment mechanism can improve the safety of interaction behavior;

The boundary detection mechanism is a pessimistic safety guarantee mechanism, the mechanism judges the area close to the edge in the experimental field as a dangerous area, and when the API operates the trolley to touch the area at any moment, the module can stop the action of the trolley, so that primary collision is prevented in advance, even if the trolley can stop at the position close to the boundary in time;

the gesture adjustment mechanism is an optimistic security guarantee mechanism, which delineates a security area at the center of the experimental site, and the API executed from the area is more likely to not collide, or will not collide in the initial period of time when the API is executed;

FIG. 13 illustrates a one-time action execution class API execution process under a boundary detection mechanism and a gesture adjustment mechanism:

assuming the cart is at point P, the hardware receives an action execution API call at the ring module that represents the progress.

According to the posture adjustment mechanism, when the motion starts, the dolly should be in a safe area (mesh part), so the dolly is scheduled from point P to point Q, and starts to perform the motion, at which time the dolly starts to move in the QR direction.

According to the boundary monitoring mechanism, when the cart reaches a dangerous area (grid part), the execution is suspended.

The hardware-in-loop running module feeds back the interrupt reason and the execution progress to an API initiator;

at this time, the execution processing module in the application support framework recognizes that this is an incomplete API execution, generates an action execution class API representing the remaining progress, and then submits the action execution class API to the hardware in-loop execution module again for execution.

Referring to fig. 11, the hardware-in-loop running module is composed of a data stream processing module, an instruction parsing module, an execution state control module, a boundary monitoring module, an attitude scheduling module and a positioning computer module;

the data flow processing module receives the input action execution class API and forwards the request to the instruction analysis module;

the instruction analysis module receives action execution API calls from the data stream processing module, the boundary monitoring module and the gesture scheduling module, and finally analyzes the action execution API calls into a bottom instruction for the Dajiang machine staff and interacts with the Dajiang machine staff; if the API call request comes from the data processing module, the analysis module returns an execution state and an execution progress after the interaction is finished;

the execution state control module controls the execution state of the hardware in the whole loop module to be an execution state or a scheduling state. The execution state represents that the module is executing an API call from the CPS application, and the boundary monitoring mechanism performs execution monitoring; the dispatching desk represents the action API of the module being executed by the module, and at the moment, the gesture adjustment mechanism is used for adjusting the trolley position; the execution state module receives positioning data of the trolley in the physical field, and is used for judging whether the current trolley is in a safety area or a dangerous area or not and performing corresponding module jump.

The boundary monitoring module realizes a boundary monitoring mechanism, namely when the module is in an execution state, if the module monitors that the trolley is in a dangerous area, the module sends an abort instruction, stops the action of the current hardware equipment and switches to a scheduling state;

the gesture scheduling module realizes a gesture adjustment mechanism, namely when the module is in a scheduling state, if the trolley is not in a safety area, the module sends a scheduling instruction sequence until the trolley reaches the safety area and is switched to an execution state;

further, the hardware-in-the-loop running module is responsible for (5 b) numerically measuring physical scene changes;

the function is supported by the positioning calculator module; the numerical measurement of the physical scene change requires real-time and rapid calculation of the trolley position in the physical field; position measurement is performed using a four-way sensing positioning technique that combines a set of equations for a solution target (trolley positioning), optimizes a solution process to combat a solution failure problem caused by noisy input data, and simultaneously selects an optimal solution from a plurality of candidate solutions using historical positioning data and current motion trends.

FIG. 14 specifically defines the data format and equation set-up procedure for the four-way sensor location technique:

FIG. 14 illustrates the input of the technique, using which four binocular infrared ranging sensors are required to be installed for the cart, the ranging values of the four sensors being the input;

The middle part of FIG. 14 depicts the output of this technique, the positioning technique calculates the position of the cart in the physical field in real time, and the output position information is calculated from the cart coordinates (x ₀ ，y ₀ ) And orientation θ, the triplet is specifically defined as follows: the known experimental field is a square field with a side length of E meters, and an x-axis and a y-axis which are parallel to the boundary are made by taking the center of the square field as an origin to establish a coordinate system. X is x ₀ And y ₀ The horizontal and vertical coordinates of the center point of the trolley in the coordinate system are shown, and theta is the included angle between the front direction of the trolley and the positive x axis;

FIG. 15 illustrates to the right the process of using the simultaneous equations of input and output data for a four-way sensing localization technique; in the coordinate system, four boundaries of the square may be considered as four line segments, and the ranging process may be considered as over-trolley coordinates (x ₀ ，y ₀ ) Two straight lines parallel or perpendicular to the trolley orientation θ are made:

since the trolley is inside the square field, the coordinates (x ₀ ，y ₀ ) The two straight lines and four boundary line segments of the line pair must generate four intersection points, and the distance measurement value Dist in four directions of front, back, left and right _dir Is equal to the intersection point coordinate (x _dir ，y _dir ) To the centroid coordinates (x) ₀ ，y ₀ ) Is a euclidean distance of (2):

(x _dir -x ₀ ) ² +(y _dir -y ₀ ) ² ＝Dist _dir

if the boundary line segment where each intersection point is located is known, the intersection point coordinates (x _dir ，y _dir ) Can be further passed through (x ₀ ，y ₀ ) And θ, thereby achieving simultaneous equations and solutions;

the specific flow of the hardware in the loop running module for positioning the calculator module (5 b) for numerically measuring the scene change is as follows:

simultaneous equation set G, reading in ranging values of four sensor hardware on the cart, reading in environmental side length, enumerating the intersection point condition pair coordinates (x ₀ ，y ₀ ) And orientation θ, simultaneous equation set G _i Wherein i refers to the i-th intersection situation; the equation set has four sub-equations, using G _i，1 ～G _i，4 Numbering;

solving the group, solving the equation group G _i Split into { G' _i，k |G′ _i，k ＝G _i -G _i，k Four sets of sub-equations, k=1, 2,3, 4; the constraint condition is reduced, so that the sub-equation set can still be solved successfully under the condition that four ranging input values contain noise; after the outliers are removed from the solutions of the four sub-equation sets, merging the solutions into a plurality of feasible solutions S;

determining an optimal solution, selecting the optimal candidate position by weighted sorting if at least one feasible solution exists, otherwise jumping to a solution-free process; this stepFirst, the historical position information (x) is obtained from the historical positioning database _h ，y _h ，θ _h ) Obtaining the current action trend (x from HiL Wrapper _δ ，y _δ ，θ _δ ) The resulting ideal position (x _α ，y _α ，θ _α ) The method comprises the following steps:

(x _α ，y _α )＝(x _h +x _δ ，y _h +yδ)

θ _α ＝θ _h +θ _δ

then for each candidate position (x _i ，y _i ，θ _i ) Calculating and ideal positioning deviation values, wherein v and u are two positive numbers, representing the distance deviation and the threshold value of the angle deviation:

S _i ＝v*dist((x _a ，y _α )，(x _i ，y _i ))+u*|θ _α -θ _i |

The smaller the deviation value Si, the closer the candidate position and the ideal position are, and the higher the solution score is;

and finally, updating the solution with the highest score as the optimal solution into a historical positioning database, wherein the scene change result is the difference value of the two adjacent time stamp solving results.

Carrying out solution-free processing, if the solution process is solution-free, repeating the complete solution step k times (k is the number of self-set retries), and trying whether the solution is successful under the condition of different input noise; if no solution exists in k times, resetting the real physical environment, and returning back to the last successfully-calculated positioning node in the virtual scene according to the historical positioning data;

the method comprises the steps that through grouping solution and non-solution processing, a positioning calculator module generates a series of positioning records of a physical trolley relative to a square experimental field on a time line, and the positioning records are used as observation results of a scene synchronization module on a hardware in-loop execution environment;

the scene synchronization module of the environment synchronization execution framework is responsible for (2 c) positioning result synchronization and scene synthesis; fig. 15 is a design architecture diagram of a scene synchronization module.

Further, the scene synchronization module is responsible for (2 c) positioning result synchronization and scene synthesis; the method comprises the steps of calculating the execution output of the CPS application to the real physical environment and mapping the execution output to two parts in the virtual scene:

The scene synchronization module consists of a scene synchronization data filter and a comprehensive scene updating module:

the scene synchronous data filter and the hardware interact in the loop execution module, and are responsible for counting the output result of CPS application execution on the environment in a real physical environment, wherein input data are a trolley positioning result and a module execution state, a filtering rule only keeps a position change result in the execution state, and position change in the scheduling state is discarded; thereby segmenting scene changes caused by CPS application action execution API and scene changes caused by platform scheduling.

The comprehensive scene updating module and the software interact in the loop execution module, the action execution result of the CPS application is mapped to a virtual environment built by a virtual engine, and the complete CPS application behavior is displayed to a user.

The scene synchronization module executes the synchronization process of the API execution result for a certain action, and the specific steps are as follows:

retaining positioning change information of hardware when the ring module is in an execution state, including position change information (Deltax _p ，Δy _p ) And angle variation information delta theta _p Generating location update data for the CPS application:

(Δx _p ，Δy _p )＝(x _p，i+1 -x _p，i ，y _p，i+1 -y _p，i )

Δθ _p ＝θ _p，i+1 -θ _p，i

wherein (x) _p，i ，，y _p，i ，θ _p，i ) The i-th positioning result of the positioning calculator on the physical trolley p.

At the virtual scene execution module, the latest positioning information P of the virtual vehicle s is read from the third party engine by using the uniformly configured data reading interface _s，k Comprising position information (x _s，k ，y _s，k ) And current vehicle angle information θ _s，k ；

Position change information (Deltax) _p ，Δy _p ) And correcting the angle of the trolley orientation. The change information vector (Deltax _p ，Δy _p ) Anticlockwise rotation about origin by θ _p，i Then rotate theta clockwise _s，i The method comprises the steps of carrying out a first treatment on the surface of the Generating relative virtual positioning information P _s，i Position change (Deltax) _s ，Δy _s )；

In the computing virtual environment, the (x) th time (k+1) of the vehicle positioning result _s，k+1 ，y _s，k+1 ，θ _s，k+1 ) And updating the scene setting interface into the virtual scene by using the uniformly configured scene setting interface.

(x _s，k+l ，y _s，k+1 )＝(x _s，i +Δx _s ，y _s，i +Δy _s )

θ _s，k+1 ＝θ _s，k +Δθ _s

The execution flow of the present invention is further illustrated based on an exemplary CPS application. The application is a trolley labyrinth path-finding program, and the application reads the distance measurement values of the sensors at the front and the left side of the trolley, carries out movement decision and controls the chassis to run; the scene is a labyrinth composed of wood board barriers in the transverse and vertical directions, and in the virtual scene, the labyrinth is modeled by units, and the trolley model is initially placed at the right lower corner of the labyrinth; in a physical scene, no maze modeling is performed, an experimental site is a square bounded site with a side length of 2.4m, and a large-scale machine armor master trolley which is running is placed inside the experimental site. After the CPS application has been started to run,

for a front sensor reading API and the like of CPS application, a test bed reads the distance from a vehicle model to a front obstacle model through a data reading interface uniformly configured with units, and uncertainty noise is added; at this time, the CPS application obtains scene input from the virtual scene, and the virtual scene has lower scene configuration cost compared with the construction of the physical maze.

For chassis forward motion API and the like performed by CPS application, a test bed firstly controls a trolley to advance in an experimental field, calculates the forward distance and moves the position of the trolley in a virtual scene; due to the fact that the physical world execution uncertainty exists, the actual execution result of the trolley and the expected deviation of a user exist, for example, if a developer does not consider forward inertia (namely, the actual forward distance is usually larger than the set forward distance parameter) when writing CPS application, the trolley and the virtual obstacle in the virtual scene collide due to the movement, and errors are reported to the developer, so that the display of real software interaction behavior is ensured; in the process, the physical trolley and the experimental site are not collided, and the platform is safer to operate.

In the test bed, the interaction process with CPS application is as the software in the ring method, the application input in the virtual environment is obtained, and the application output is executed on the real hardware as the hardware in the ring method; the software in the CPS application development process is combined with the hardware in the loop running environment, so that the real software-hardware interaction process of the hardware in the loop is reserved, and the safety, observability and operability of the software execution process in CPS development are ensured by combining the characteristics of low construction cost and safe operation of the software in the loop virtual environment.

The present invention is not limited to the preferred embodiments, and any simple modification, equivalent replacement, and improvement made to the above embodiments by those skilled in the art without departing from the technical scope of the present invention, will fall within the scope of the present invention.

Claims

1. Information physical comprehensive test bed structure based on Dajiang Jijia master, its characterized in that: the system comprises an application support frame and an environment synchronous execution frame, wherein execution support is respectively provided for a software support component and an operation environment in the information physical fusion system; the application support framework is used for receiving CPS application to be executed uploaded by a user, providing a software support component for application execution and simulating execution, dividing an execution task into two classes of perception type tasks and action execution type tasks through the software support component during the simulation execution, transmitting the perception type tasks to software in the environment synchronous execution framework for processing in a ring operation mode, and transmitting the action execution type tasks to hardware in the environment synchronous execution framework for processing in the ring operation mode; the environment synchronous execution framework is used for maintaining a virtual environment in which software runs in a ring and a real physical environment in which hardware runs in the ring, executing a perception class task by using an execution scene in the virtual environment in which the software runs in the ring as input of CPS application, outputting a real hardware execution result of an action execution class task by using the real hardware execution result of the action execution class task in the real physical environment in which the hardware runs in the ring as output of CPS application, and generating a comprehensive running environment between the hardware in the ring and the software in the ring through synchronizing scenes in the virtual environment and the real physical environment.

2. The information physical integrated test bed structure based on the Dajiang machine tool university according to claim 1, the method is characterized in that: the application supporting frame comprises an analog operation module, a perception processing module and an execution processing module; the simulation running module is responsible for initializing CPS application loading and application execution supporting environment SDK loading in configuration, performing simulation execution on CPS application uploaded by a user by combining with the application execution supporting environment SDK, and performing local recording decision; the perception processing module is responsible for packaging a perception class task call request and then sending the request to a virtual environment of software running in a ring; the execution processing module is responsible for sending an action execution class task call request to a real physical environment of hardware running in a ring, and analyzing and packaging return value information of the real physical environment as an execution result.

3. The information physical integrated test bed architecture based on the large-scale machine tool master as claimed in claim 2, wherein: the environment synchronous execution framework comprises a software on-loop running module, a hardware on-loop running module and a scene synchronization module; the software in-loop running module is responsible for maintaining the virtual environment of the software in-loop, and after receiving the perception class task, executing the perception class task as input of CPS application by using the execution scene of the software in the virtual environment in-loop running; the hardware-in-loop running module is responsible for maintaining the real physical environment of the hardware-in-loop, and takes the real hardware execution result of the action execution class task as the output of CPS application in the real physical environment of the hardware-in-loop running after receiving the action execution class task; the scene synchronization module is responsible for calculating a real hardware execution result in a real physical environment, and synchronizing scenes in a virtual environment and the real physical environment to generate a comprehensive running environment between a hardware in-loop and a software in-loop.

4. The information physical integrated test bed architecture based on the large-scale machine tool master according to claim 3, wherein: the user CPS application execution flow comprises the following specific steps:

(5) Hardware performs in the loop: executing CPS application output in a real physical environment;

5. The information physical integrated test bed architecture based on the large-scale machine tool master as claimed in claim 4, wherein: the perception class task is a perception class API, and the action execution class task is an action execution class API.

6. The information physical integrated test bed architecture based on the large-scale machine tool master as claimed in claim 5, wherein: the software supporting component is an extended SDK for secondary development of a primary SDK of a Dajiang machine A, modifies and encapsulates key APIs of CPS application interaction with an external environment, and divides the request and execution process of the API call, so that the noninductive switching of the API call request of the CPS application between a virtual environment and a real physical environment is realized.

7. The information physical integrated test bed architecture based on the large-scale machine tool master as claimed in claim 6, wherein: the application support frame is implemented as follows:

executing a perceptual class API in a CPS application: calling a perception class API code in the extended SDK, reading a virtual sensor result from a virtual world, providing a unified interface for interaction with a virtual environment, and ensuring flexible virtual environment configuration; the interface is a data reading interface;

8. The information physical integrated test bed architecture based on the large-scale machine tool master as claimed in claim 7, wherein: the execution process of the environment synchronous execution framework is as follows:

maintaining a virtual operating environment of software in a ring: setting up a virtual scene and a machine tool master car model in a third-party virtual engine, and providing a scene setting interface to update the scene;

9. The information physical integrated test bed architecture based on the large-scale machine tool master according to claim 3, wherein: the hardware adopts a boundary detection mechanism and a gesture adjustment mechanism when the ring running module processes action to execute the class API; the hardware-in-loop running module consists of a data stream processing module, an instruction analysis module, an execution state control module, a boundary monitoring module, an attitude scheduling module and a positioning computer module; the data flow processing module receives an input action execution class API and forwards a request of the action execution class API to the instruction analysis module; the instruction analysis module receives action execution API calls from the data stream processing module, the boundary monitoring module and the gesture scheduling module, and finally analyzes the action execution API calls into a bottom instruction for the Dajiang machine staff and interacts with the Dajiang machine staff; the execution state control module controls the execution state of the hardware in the whole loop module to be an execution state or a scheduling state; the boundary monitoring module is used for realizing a boundary monitoring mechanism; the gesture scheduling module is used for realizing a gesture adjustment mechanism; the positioning calculator module is used for supporting the numerical measurement of the physical scene change.

10. The information physical integrated test bed architecture based on the large-scale machine tool master according to claim 3, wherein: the scene synchronization module consists of a scene synchronization data filter and a comprehensive scene updating module, wherein the scene synchronization data filter and hardware interact with each other in a loop execution module and are responsible for counting output results of CPS application execution in a real physical environment to the real physical environment; the comprehensive scene updating module and the software interact in the loop execution module, and the action execution result of the CPS application is mapped to the virtual environment built by the virtual engine.