CN115168928A - Intelligent window system of machine tool and construction method - Google Patents

Abstract

The invention provides an intelligent window system of a machine tool and a construction method thereof, wherein the intelligent window system comprises an intelligent window physical system positioned on a physical layer, a network communication, data processing and analyzing system positioned on a middle layer, and an intelligent window three-dimensional virtual scene, live video, augmented reality and running state billboard display system positioned on a service layer. The construction method of the intelligent window system service layer comprises the following steps: (1) collecting data; (2) geometric modeling; (3) a human-machine interface; (4) constructing a scene; (5) a scene model; and (6) optimizing the scene. Through the intelligent window system provided by the invention, a user of the machine tool can monitor the production running condition in the machine tool in real time, check the real-time production information and the production statistical data, improve the working efficiency of machine tool setting adjustment, process optimization, fault diagnosis and new workpiece trial production, and is particularly suitable for running on the machine tool frequently facing product frequent replacement and small-batch production.

Description

Intelligent window system of machine tool and construction method

Technical Field

The invention relates to a digital twinning technology oriented to the field of machine tool production and machining, in particular to an intelligent window system of a machine tool and a construction method.

Background

When a new product is manufactured or an order is changed on a machine tool, a user of the machine tool, such as a worker for process design, operation, maintenance, management, and the like, needs to check the working state inside the machine tool in order to correctly combine production components in the machine tool, such as raw materials, clamping devices, positioning points, tools, numerical control machining programs, and the like. Since the machine tool must be equipped with raw materials, clamping devices and numerical control machining programs that are suitable for the product order, and appropriate setpoints, tool sets, etc. must be provided in the machine tool, the user of the machine tool must deal with a complex range of tasks when changing product models. Most tasks in these machine tools, such as parameter, workpiece, tool etc. setting, process adjustment and optimization, fault diagnosis, etc., will be performed manually by the relevant personnel in the foreseeable future. Although work has been advanced in the past to assist in this regard and further developments are still in progress, steps that are performed manually on the machine tool are unavoidable. In view of the shorter and shorter lead times faced by businesses in market competition, and the higher and higher demands of customers on product complexity and flexibility, the users of machine tools must handle more complex tasks in the same or shorter time periods. Therefore, in order to enhance the market competitiveness of enterprises in the case that machine tool type processing equipment is not fully automated, people need to support the manual operation process of a user of a machine tool by adopting a more effective means.

In order to support the manual operation process of the machine tool user, on the one hand, from the perspective of the machine tool manufacturer, machine tool machining equipment is generally provided with a viewing window, which serves as an isolation device for protecting the user during the operation of the machine tool and can also be used for monitoring the production process of the machine tool by the user. When a machine tool starts a new machining process, the visibility of its working area is an important indicator for the machine tool user to monitor the production process. In order to provide a good overview of the machining situation, the machine tool is generally equipped with a viewing window which is as large as possible. However, it is not uncommon for cooling lubricants to impair workpiece visibility during machining. In particular, when machining at high pressures, well above 10bar, the visibility of the direct machining process on the tool is seriously impaired by the spraying of cooling lubricants, or the formation of mist, by the machine tool. Even in dry machining, when there is a long machining process, a chip stack is formed, resulting in reduced visibility of machining at the cutting edge of the tool.

On the other hand, there is a trend of using smart phones, tablet computers, industrial computers or other types of screens in the machine tool working environment of production and processing enterprises, so as to assist the machine tool user in checking information of relevant systems such as production plans, process parameters and the like, for example, synchronizing tool data with an ERP system or acquiring process data from an MES database, and the like, so that the machine tool user can adjust the equipment accordingly. This trend is enhanced by the current popularity of such smart devices (tablet computers, smart phones, etc.), but the information they present is not updated in full and real-time, such as the remaining machine tool run time, workpiece counters, process simulation, etc., thus increasing the number of these distributed displays while displaying information required by the machine tool user. Each of these numerous display methods attempts to make the workflow for supporting the machine tool user more efficient. However, at the same time, a burden is also imposed on the machine tool user. Because the machine tool user must divert attention to additional screens, and these screens have not provided consistent user operating guidance information. If the machine tool user no longer navigates through the operating interface that requires menu driving, but directly views it using augmented reality technology in the digital twin domain, higher efficiencies in software operation and learning, obtaining information from the digital virtual world can be achieved.

The digital twin technology can symmetrically map all related entity objects of the physical world and virtual object information of the digital world in the working process of the machine tool, and realize real-time interaction of state data of the two objects. The automatic transparent screen is provided for the machine tool observation window, and the screen interface of manual operation can be integrated for a machine tool user by combining the digital twin technology, so that the multiple burden of dispersing to other screens when the machine tool user manually operates the machine tool is relieved. After the machine tool observation window is used as a screen, a software system with specific functions required in the process of machine tool preparation and operation can be displayed on the visual window of the machine tool in a binding mode, so that the visual window becomes the screen of the intelligent window system.

There are also machines equipped with special windows, such as one-piece turning-milling machines, where the installation of windows suitable for occupational safety is a costly task, since it is characteristic of such machines that the turning operation can also be carried out on the milling centre. This leads to a change in safety considerations, requiring the use of laminated safety glass as a viewing window. Under the condition, the machine tool adopting the intelligent window can directly use the sheet metal shell of the machine tool as a screen carrier of the intelligent window without setting a complex and expensive observation window, and the working state of the whole working area in the machine tool is displayed through an augmented reality technology. By utilizing the augmented reality technology, the intelligent window system can provide real-time three-dimensional view display of the state of a workpiece in the machining process, so that the visibility of the machining process can be improved, the influence of cooling lubricant or chip stacks is avoided, and an observation window does not need to be arranged on a machine tool.

Other independent software systems, such as order management information, etc., may also be displayed in the smart window screen of the machine tool and interlocked with the machining process. There are several ways in which the user of the machine tool can more easily compare the order data, for example, with the parameter settings of the machine tool or the actual production process. And the simulation processing process superposed on the actual operation process of the machine tool is displayed in the screen of the intelligent window system, so that a machine tool user can clearly check whether the currently used tool is consistent with the planned tool. By combining numerical control machining process simulation with screen display on an intelligent window system, information about the actual state of a machine tool, which is available for a machine tool control system, can be evaluated in real time, such as collision detection, tool breakage risk and the like in the process of observing and predicting the tool in real time. And once the machine tool is abnormal, a machine tool user can call and compare the simulation of the numerical control machining process and the motion history of the machine tool, so that the analysis, diagnosis and evaluation of the reasons of the machine tool problems are facilitated, the problems are quickly solved, and the normal operation of the machine tool is recovered.

Disclosure of Invention

The invention aims to overcome the defects of the prior art and provides an intelligent window system of a machine tool and a construction method thereof.

In order to achieve the purpose, the invention adopts the technical scheme that: the intelligent window system of the machine tool comprises three parts: (1) The intelligent window physical system in the physical layer module is flexible, is suitable for automatic information display and control system with different machine tool observation window configuration requirements, and may be used in different types of machine tools after replacing corresponding display hardware and control software modules. (2) And the network communication, data processing and analysis system is positioned in the middle layer module and is used for realizing the data acquisition, processing, storage and analysis processes of the intelligent window system. (3) The machine tool located in the service layer module is used for processing and manufacturing a three-dimensional virtual scene, live video, augmented reality and state billboard display system, can bring additional values for real-time production monitoring, setting adjustment, process optimization, fault diagnosis and trial production of new workpieces of the machine tool, and is particularly suitable for running on the machine tool frequently facing product replacement and small-batch production.

The method for constructing the intelligent window system of the machine tool comprises the following steps:

step 1, a data collecting module: and summarizing all data information of the entity object and the virtual object in the operation process of the machine tool and then dividing the data information into non-model driving data and model driving data. The non-model-driven data are static type data and are stored and displayed in the Internet of things platform tool in a table form. The model driving data is dynamic data, the WebSocket protocol is used for sending the data to the platform tool of the Internet of things for storage, and the data is transmitted to the three-dimensional visualization engine in a JSON character string format for data linkage and binding. The data collected by the data collecting module are respectively provided for the geometric modeling and human-computer interface module through the step 2 and the step 3;

step 2, a geometric modeling module: and a digital model is constructed by adopting three-dimensional modeling software according to the geometric dimension of the entity object, and necessary light, materials, texture rendering models and special effects are added, so that the three-dimensional model has real texture and is more vivid. In order to reduce the operating pressure of hardware, the model is simplified appropriately and exported to a lightweight format file, and the file is imported to the scene construction module in step 4;

step 3, a human-computer interface module: the system comprises the functions of man-machine interaction design, external input event response and the like, and aims to realize scene roaming in a three-dimensional virtual environment, so that a user has immersive visual experience. Exporting a file realizing the functions of the human-computer interface module into a lightweight format file in a three-dimensional modeling software environment, and importing the file into a scene construction module in step 4;

step 4, a scene construction module: the geometric modeling module and the function file of the human-computer interface module are further improved in a three-dimensional visualization engine development tool, namely the design effect of the human-computer interface module and the model file of the geometric modeling module are combined into one scene, scene management, scene roaming, user interface design, performance optimization and the like are carried out, the combined effect and the model file are exported to a lightweight format file, and the file is imported into the scene model module in step 5;

step 5, a scene model module: carrying out data binding on information provided by the Internet of things platform development tool in a JSON character string format in a three-dimensional visualization engine development tool, realizing real-time updating of the state information of a virtual scene, a state billboard, a live video and an augmented reality function module driven by real-time data of an entity object in the physical layer module, carrying out operation test on the function modules and feeding back the test result to a scene optimization module in step 6;

step 6, a scene optimization module: and feeding back the test operation result of the scene model module to the geometric modeling and human-computer interface module, respectively performing design optimization in the two modules, issuing the model file of the virtual scene module into a WebGL format in a three-dimensional visualization engine development tool after iteration, and integrating the model file into an Internet of things platform tool.

The Internet of things platform tool is a ThinsBoard, the three-dimensional visualization engine is Unreal, the three-dimensional modeling software comprises blend, siemens NX and Unreal, and the lightweight format file is an FBX file.

The invention is based on a digital twin technology, and further describes an intelligent window system construction process of a machine tool and an implementation method thereof by constructing a physical object system and a virtual object system in the manual and automatic operation process of the machine tool. The intelligent window system has the beneficial effects that:

the invention can integrate various screen interfaces required by manual operation for a machine tool user by providing the digital transparent screen for the machine tool observation window and combining the digital twin technology, thereby reducing the burden of dispersing other screens when the machine tool user manually operates the machine tool. Through the augmented reality technology, the intelligent window system can provide real-time three-dimensional scene display of the state of the internal working part of the machine tool in the automatic machining process, so that the visibility of the internal working part of the machine tool can be improved without being influenced by cooling lubricant or chip stacks, and even on some machine tools with special safety requirements, an original observation window does not need to be arranged, and therefore the dual purposes of considering the safety of the machine tool and the visibility of the working part are achieved. Other independent software systems, such as order management information in ERP and data such as tool settings, recipe settings, process parameters in MES, can also be displayed in the smart window screen of the machine tool and interlocked with the machining process.

The invention can also comprise value-added services such as numerical control simulation, namely, the simulation of the numerical control machining process is combined with the screen display on the intelligent window system, and the simulated machining process superposed on the actual operation process of the machine tool is displayed in the screen of the intelligent window system, so that the information about the actual state of the machine tool, which is available for the machine tool control system, can be evaluated in real time, such as the risk of cutter breakage when the cutter in the machine tool moves is observed and predicted in real time, and the functions of assisting a machine tool user to check whether the currently used cutter is consistent with a planned cutter at a glance are realized. And once the machine tool is abnormal, a machine tool user can call and compare the simulation of the numerical control machining process and the movement history of the machine tool, so that the analysis, diagnosis and evaluation of the reasons of the machine tool problems are facilitated, the problems are solved quickly, and the normal operation of the machine tool is recovered. Therefore, the intelligent window system can bring additional values for real-time production monitoring, setting adjustment, process optimization, fault diagnosis and trial production of new workpieces for machine tool users, and is particularly suitable for running on machine tools frequently facing frequent product replacement and small-batch production.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

fig. 1 schematically shows an overall architecture diagram of an intelligent window system of a machine tool according to an embodiment of the present application.

Fig. 2 schematically shows a physical layer structure layout of an intelligent window system of a machine tool according to an embodiment of the present application.

Fig. 3 schematically illustrates an intelligent window system data mining and network communication architecture of a machine tool according to an embodiment of the present application.

Fig. 4 schematically shows a process diagram for constructing an intelligent window virtual system of a machine tool according to an embodiment of the present application.

Detailed Description

To make the objects, technical solutions and advantages of the present application more clear, the present application is further described in detail below with reference to the accompanying drawings and specific embodiments.

In an effort to provide a concise description of these embodiments, all features of an actual embodiment may not be described in the specification. It should be appreciated that in the development of any such actual implementation, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developers' specific goals, such as compliance with system-related and business-related constraints, which may vary from one implementation to another.

In the following description, references to "one embodiment", "an embodiment", "one example", "an example", etc., indicate that the embodiment or example so described may include a particular feature, structure, characteristic, property, element, or limitation, but not every embodiment or example necessarily includes the particular feature, structure, characteristic, property, element, or limitation. Moreover, repeated use of the phrase "in accordance with an embodiment of the present application" although it may possibly refer to the same embodiment, does not necessarily refer to the same embodiment.

Certain features that are well known to those skilled in the art have been omitted from the following description for simplicity.

When introducing elements of various embodiments of the present invention, the articles "a," "an," and "the" are intended to mean that there are one or more of the elements. The terms "comprising," "including," "and" having an inclusive meaning may be used to specify the presence of stated elements.

According to one embodiment of the present application, as shown in fig. 1, the general architecture 10 of the smart window system of a machine tool is a three-layer structure comprising: (1) The intelligent window physical system positioned in the physical layer module 100 is a flexible automatic information display and control system which is suitable for different machine tool observation window configuration requirements, and can be used for different types of machine tools after corresponding display hardware and control software modules are replaced; (2) The network communication, data processing and analysis system is positioned in the middle layer module 200 and is used for realizing the data acquisition, processing, storage and analysis processes of the intelligent window system; (3) The machine tool located in the service layer module 300 produces and processes a three-dimensional virtual scene, a live video, an augmented reality and an operating state billboard display system. The real object data information of the machine tool running in the physical layer module 100 is mapped to the simulation object in the virtual space of the service layer module 300 in real time through the data processing and conversion of the middle layer module 200. The virtual object in the service layer module 300 completely corresponds to the physical object in the physical layer module 100 in terms of dynamic and static information, and the two become virtual and real twins of the smart window system.

(1) Physical layer construction method of intelligent window system of machine tool

The physical layer module 100 is a physical object in the physical space of the smart window system, and is composed of five parts, a tracking system 105, a backlight 110, a machine body 115, a camera 120, and a display 125.

The tracking system 105 is intended to record the exact position of a machine tool user on the machine tool body 115 to simulate the perspective of the machine tool user so as to show the state of the scene inside the machine tool in perspective with respect to the machine tool user on the display 125. In one embodiment of the present application, as shown in FIG. 2, a set of tracking systems 105 is mounted above the display 125 on the exterior 1155 of the machine tool door for tracking the viewing position of a machine tool user when standing at different angles relative to the display 125, enabling the display 125 to present a perspective representation of the state of the interior 1150 of the machine tool door, whether in a purely virtual or a virtual-to-real superimposed state.

The position of the machine tool user may be detected by using a tracking system 105 that tracks the head or eyes of the machine tool user. Such tracking systems 105 are commercially available and have different characteristics, such as microsoft's markerless system Kinect, which can record not only the body and head of a person, but also facial features such as the eyes, mouth and nose of a person completely markerless.

Depending on the tracking system 105 used, the eye position can be determined directly (eye tracking) or by recording the head position of the machine tool user (head tracking). Each tracking system 105 provides the current machine tool user's eye position EP0 in the same data output format and transmits the position data EP0 to the data processing system 220 via the network communication link 215 of the middle tier module 200.

The data processing system 220 further processes the machine tool user position data EP0 provided by the tracking system 105, such as by smoothing the received data EP0 using an averaging filter, to remove the noise effects of the machine tool user pose recorded by the head tracking system. The data processing system 220 then transmits the processed machine tool user position data EP0 to the data analysis system 230 for storage and analysis via the network communication link 225.

The data analysis system 230 can simulate the screen effects of any number, size of the display 125 using a computer digital model of the traditional display 125 and the physical objects inside the machine tool door 1150 by taking into account the perspective of the machine tool user and overlaying the physical objects inside the machine tool door 1150 with virtual content on the screen of the display 125. Therefore, the machine tool can be completely closed by the metal shell to protect a machine tool user; the display 125 used may take the form of a different number and size of screens mounted anywhere on the machine tool housing that is easily viewable by the machine tool user, even if the components on the machine tool would restrict the field of view of the machine tool user.

As shown in fig. 2, a plurality of cameras 120 are installed in the interior 1150 of the machine tool door, and are responsible for capturing images of the physical object state and pre-processing the images, and then transmitting the pre-processed images to the data processing system 220 through the network communication link 215 of the middle layer module 200. The data processing system 220 is responsible for further data processing and calibration of the images provided by the camera 120 and for transferring the processed image data CD0 to the data analysis system 230 for storage and analysis via the network communication link 225. The data analysis system 230, on the one hand, performs three-dimensional reconstruction of the physical object in the interior 1150 of the machine tool door based on the image data CD0 provided by the camera 120 to form a virtual computer digital model; on the other hand, the complete coverage of the scene inside 1150 the machine tool door is realized by combining the real object image provided by the camera 120 and adopting a virtual and real image superposition mode.

In addition to using a pure virtual visualization of the computer digital model and full coverage of the scene inside 1150 of the machine tool door based on the camera 120, a transparent display can also be used to provide a perspective coverage of the scene inside 1150 of the machine tool door. In a particular embodiment, the display 125 may employ two technologies for see-through coverage of the interior 1150 of the machine tool door, namely a Liquid Crystal Display (LCD) and an Organic Light Emitting Diode (OLED) based display.

Conventional LCDs can be disassembled and converted into transparent displays, for example, by removing the back panel, electronics, and backlight needed for illumination from the liquid crystal layer, and then mounting the electronics and liquid crystal layer side by side in a new housing to form a transparent LCD. The LCD is then mounted in front of the existing viewing window on the exterior 1155 of the machine tool door, as shown in fig. 2, and the backlight, which is backlight illumination 110, is mounted on the wall opposite the viewing window in the interior 1150 of the machine tool door. An Organic Light Emitting Diode (OLED) display has a self-light emitting characteristic, and thus, does not require additional backlight 110.OLED displays also possess better optical properties in terms of transparency and contrast. Thus, in a particular embodiment, one of the two schemes, transparent LCD or OLED, or a mixture thereof, may be selected for use on the display 125 according to the particular needs of the machine tool customer.

A simulation model of the real object in the machine tool door interior 1150 can be seen on the display 125, which moves simultaneously with the real machine tool according to the position setting value PS0 of the real object. In addition to static and dynamic machine parts, the simulation model also contains a spindle 115A and a tool 115B in a tool magazine, as well as a clamping device 115C and a workpiece 115D. In the simulation model, the tool changing process is performed as on the real machine tool body 115, because the simulation model receives the same control signals (e.g., rotating the tool magazine, changing the tool 115B, clamping the tool 115B in the spindle 115A, etc.). By simulating material removal, the machine tool user can also see real-time, detailed machining progress on the display 125.

The middle module 200 implements the data collection, processing, storage and analysis processes of the smart window system, and comprises three functional modules, namely a network communication link 210, 215, 225, 235, a network communication system 205, a data processing system 220 and a data analysis system 230. The network communication links 210, 215, 225, 235 are distributed among the network communication system 205, the data processing system 220, the data analysis system 230, the physical layer module 100, and the service layer module 300. Real-time data information of physical objects in the smart window system passes through the network communication links 210, 215 and the network communication system 205 to the data processing system 220. The data processing system 220 classifies, converts, and transmits the real-time data information to the data analysis system 230 for storage and analysis via the network communication link 225 according to the application requirements of the service layer module 300. The data analysis system 230 then feeds back the data analysis result to the service layer module 300 through the network communication link 235 according to the request information of the specific application service in the service layer module 300.

The service layer module 300 is a simulation object in a virtual space of the smart window system, corresponds to an entity object in a physical space through a three-dimensional modeled model and a data information presentation tool, and is responsible for providing application-specific services to the physical layer. The service layer module 300 includes 4 functional modules, namely a status signboard 305, a virtual scene 310, a live video 320 and an augmented reality 325.

The display effect of the status signboard 305 is not as direct as that of the three-dimensional model, but plays an irreplaceable role in the aspect of statistical information attributes (time, text, quantity, etc.), and can display the statistical information and detailed production resource status of each workpiece 115D in real time. In one embodiment of the present application, the status billboard 305 displays the real-time status of the physical object in the physical layer module 100 and the historical data generated by the data processing system 220 in different forms, such as text, numbers, tables, dot diagrams, broken line diagrams, dashboards, etc., and also has the function of summarizing data in different periods (such as displaying according to seconds, minutes, hours, days, weeks, months, etc.). The three-dimensional virtual scene module 310 is a main monitoring mode, and is combined with the live video module 320 to perform real-time monitoring and numerical control machining process simulation on internal components of the machine tool main body 115, such as the tool 115B, the clamping device 115C, the machined workpiece 115D and the like, so as to realize three-dimensional visual control of the processes of machine tool setting adjustment, process optimization, fault diagnosis, trial production of new workpieces and the like. The live broadcast video module 320 realizes the visual monitoring of the real-time production state of the key links of the machine tool adjustment and the machining process through a plurality of industrial cameras 120 arranged in the inner part 1150 of the machine tool door. The augmented reality module 325 comprehensively utilizes technical means such as three-dimensional modeling, real-time tracking and registration, intelligent interaction, a tracking system 105 and a camera 120 to superimpose information of a virtual object on a real physical object in the machine tool main body 115, so that the internal data information display capability of the real object is enhanced, for example, by detecting whether barriers such as a stop block attached to a clamping device 115C exist on a target path and an area to which the tool 115B is to be moved, the intelligent window system can warn a machine tool user that the tool 115B is at risk of tool collision and tool breakage, and stop the tool 115B from moving in time.

According to an embodiment of the present application, the smart window system of the machine tool further includes the following software and hardware components in addition to the physical objects of the tracking system 105, the backlight 110, the machine tool body 115, the camera 120 and the display 125 included in the physical layer module 100: (1) A workstation computer with higher graphic processing performance is arranged in a product design office, and a computer digital modeling and simulation software system is operated; (2) A server with higher data processing and analyzing capacity is arranged in a production workshop, and an Internet of things platform system with data processing and analyzing functions is operated; (3) Installing visual electronic signboards and terminal computers with higher graphic processing performance in a production workshop and a production scheduling room, and operating a three-dimensional scene visualization system in the production and processing process of a machine tool; and (4) the systems are interconnected through industrial Ethernet. The personnel of production management, process, maintenance and design of the machine tool can master the running state, the yield statistical information, the progress information, the key data historical curve and the like of the machine tool in real time in a dispatching room and an office like putting the machine tool in a production workshop field.

The visual electronic billboard adopts a high-resolution spliced large screen and comprises a matrix type multi-block spliced screen, a large screen controller, a multiplexer, a three-dimensional visual server, a video switch and the like, wherein the three-dimensional visual server is used for installing and deploying an intelligent window system service application program of a machine tool, and the large screen controller is used for receiving multi-channel video signals from the three-dimensional visual server and the video switch, processing, deploying and matrix mapping the video signals and displaying the video signals to different areas of the spliced screen.

(2) Interlayer construction method of intelligent window system of machine tool

When the machine tool body 115 is in a set state or in a production process, real-time operation state data of physical objects such as the tool 115B, the clamping device 115C, and the workpiece 115D to be processed, and image data CD1 captured by the tracking system 105 and the camera 120 reach the intermediate layer module 200 through the network communication link 215 to perform data format conversion, storage, and analysis. As shown in fig. 3, the network communication system 205 is an important link for implementing data information interconnection in the smart window system, and includes two functional modules, namely, bus data 205A and discrete data 205B. The two communication modules are responsible for transferring real-time data of the physical layer module 100 into the data processing system 220. Data processing system 220 includes four functional modules: data collection software 220A, data collection tool 220B, data concentrator 220C, and data processing server 220D. Some data may be passed through the data processing system 220 and then directly transmitted to the particular application of the service layer module 300 by the ethernet communication link 315; there are also data that needs to be transmitted to the data analysis system 230 for further analysis. The data analysis system 230 includes three functional modules, namely, a real-time database 230A, a historical database 230B, and a data analysis server 230C, and completes data analysis, mining, and preparation according to specific application requirements of the service layer module 300, and provides the data to the service layer module 300 through an interface unified with the data processing system 220.

Because of the large number of manufacturers of the machine tool main body 115 and the large number of standards of the communication protocol of the machine tool automation system, it is necessary to integrate the software and hardware data communication of different manufacturers in a flexible manner. OPC UA is a standardized technical framework that integrates information model definitions, services and communication standards. The OPC UA server can be connected to field devices such as programmable control devices and smart meters of most machine tool manufacturers through an industrial ethernet, so in one embodiment of the present application, the bus data module 205A uses OPC UA protocol to obtain data information of sensors, actuators, controllers, and the like from the control system OPC UA server of the machine tool body 115, and transmits the data to the data acquisition software module 220A through the network communication links 210 and 215. The data collection software module 220A employs an OPC UA client to transmit data information obtained from the bus data module 205A to the data processing server module 220D via the network communication link 210.

Some physical objects in the physical layer module 100 do not support the OPC UA protocol but support other fieldbus communication protocols, such as in one embodiment of the present application, the tracking system 105 and the video camera 120 respectively use cameras with two interface types of USB and GbE, and the bus data module 205A may communicate image data from the tracking system 105 and the video camera 120 to the data collection tool module 220B via the network communication links 210, 215 using two bus protocols, USB and GbE. The data acquisition tool 220B module transmits the acquired image data to the data processing server module 220D via the network communication link 210 using USB and GbE communication protocols corresponding to the bus data module 205A.

There are also some physical objects in the physical layer module 100 that do not support the communication method of industrial fieldbus, such as 0-10V status signals of some meters on the machine tool main body 115, 4-20 mA output signals of temperature controllers for cooling lubricant, or some switching value signals, etc., so it is necessary to transmit these data signals from the physical objects in the physical layer module 100 to the data concentrator module 220C via the network communication links 210, 215 in the discrete data module 205B. The data concentrator module 220C transmits the acquired discrete data to the data processing server module 220D via the network communication link 210.

Twin data in the physical layer module 100, which represents the observation position of the machine tool user and the real and complete operating state of the machine tool main body 115, is heterogeneous and multi-source data stream, and needs to be classified, summarized, uniformly characterized, and the like in the data processing server module 220D, and then the result data is transmitted to the data analysis system module 230 through the ethernet link 225, and the data is further analyzed, so as to provide data support for the application program of the service layer 300.

According to the specific application requirements of the service layer module 300, some data passes through the data processing server module 220D and is directly transmitted to the specific application of the service layer module 300 by the ethernet communication link 315; still other data needs to be transmitted to the real-time database 230A and the historical database 230B by the ethernet communication link 225, then transmitted to the data analysis server module 230C by the ethernet communication link 225, further analyzed, mined and prepared according to the specific application requirements of the service layer module 300, and provided to the service layer module 300 by the ethernet communication link 315 through an interface unified with the data processing system 220.

In an embodiment of the application, the middle layer realizes equipment connection, data acquisition, data storage, data analysis, application service development and Web front-end system integration based on the strong ubiquitous network connection, flexible data management and rapid application development capability of an Internet of things platform tool thingsBoard.

In one embodiment of the present application, the network communication system 205 employs network switches, wired and wireless routers, and the like; the data processing system 220 employs personal computers PC, raspberry Pi and Arduino, etc. small edge computing devices; the data analysis system 230 is a commercial workstation computer, and realizes partition selection, storage, cataloguing and indexing of machine tool data based on a data management engine composed of CentOS7.0, postgreSQL and a timescaleDB time sequence database; and finally, realizing information interaction between the service layer application and the industrial Internet of things platform based on a Get interface mode of the Restful architecture.

(3) Service layer construction method of intelligent window system of machine tool

By constructing an intelligent window system of the machine tool and researching a three-dimensional scene data processing technology, motion simulation, real-time analysis and optimization of the running state of the machine tool are carried out on the working process of machining the workpiece 115D by the machine tool, and the flexibility and the intelligent capacity of the machine tool during working and running are improved. Meanwhile, live-action video monitoring is carried out on a key working area in the running process of the machine tool, and the augmented reality technology is combined for application, so that dynamic information of the production process of the machine tool, the internal condition of the machine tool and the historical action process are displayed in real time, and abnormal conditions of a production site, such as abnormal temperature of a cooling lubricant, abnormal current of a motor of a main shaft 115A, overlarge torque, cutter breaking of a cutter 115B and other production emergency conditions, are responded in time.

Besides assisting the manual work of the machine tool user through the display 125, the intelligent window system of the machine tool also comprises the preparation, setting, monitoring and optimization work during the machine tool machining process, and further presents more value-added service assisting functions which are not possessed by the machine tool development process to the machine tool user through the contents displayed by the service layer module 300. For example, there are two methods of preparing a machine tool numerical control program (G code) to be run while machining the workpiece 115D, one is created using CAD and CAM software in a process preparation department, and the other is directly programmed on the machine tool by a machine tool user by viewing drawings.

In the first case, the machine tool user receives the numerical control program in the form of a "black box", in which case the machine tool user has little knowledge of the working content thereof, and the path, movement trajectory and working steps of the tool 115B are not obvious; possible collisions (e.g. between the tool 115B and the clamping device 115C) cannot be deduced in advance and checked visually; it is not possible to check against the target workpiece whether the original workpiece 115D used is correct. In the second case, the machine tool user needs to program directly on the machine tool to convert the geometric information in the drawing into program code. This task is not intuitive and error prone, and simulated motion, visual impact, depth of cut and feed of the tool 115B can be helpful to the machine tool user who also needs machine program optimization functions (e.g., fast movement).

Therefore, it is necessary to provide the machine tool user with value-added service assistance functions in the service layer module 300 that are adapted to the above task. The building service layer module 300 is a system project for collaborative development of multiple software and mutual cooperation of multifunctional modules. In an embodiment of the application, an internet of things platform such as ThingsBoard is used as a data application bus, and the service layer module 300 is designed and developed by using blend, siemens NX and urea multi-software collaborative modeling. As shown in fig. 4, the construction process of the service layer module 300 mainly includes 6 functional modules, i.e., data collection 3105, geometric modeling 3120, scene optimization 3140, human-machine interface 3145, scene construction 3160, and scene model 3170.

In order to construct a three-dimensional virtual scene, first, in the collected data 3105 module, all data information of the physical object of the physical layer module 100 and the virtual object of the service layer module 300 are summarized, where the data includes static types, such as the geometric dimensions GS0, mechanical properties MP0, physical properties PP0, machining indexes TS0, etc. of the tool 115B and the workpiece 115D in the machine tool body 115, the communication address CA0 of the machine tool body 115 system and the types DT0, the quantity DQ0, and the frequency DF0 of the transmitted data, the communication address CA1 and the communication protocol CP0 of the tracking system 105 and the camera 120, the chart types CT0, the data types DT1, and the quantity DQ1 displayed by the status signboard 305 and the augmented reality 325, and the like; there are dynamic types such as automatic, manual, and stop state information IS0 of the machine tool body 115, alarm information IA0, machining count WC0 of the workpiece 115D, cycle data CT1, and data such as motor state data MS0 of the spindle 115A during operation of the machine tool body 115, and cooling and lubricating system temperature state TS 0.

The collect data module 3105 is to separate all data into non-model driven data and model driven data. The non-model driven data is static type data, stored and presented in ThingsBoard in tabular form. The model driving data is dynamic data, the data is sent to an internet of things platform such as a ThingsBoard for storage by using a WebSocket protocol in the data processing system module 220, and the data is transmitted to a real-time three-dimensional engine such as Unreal for data linking and binding in a JSON character string format. The aggregated data from the collect data module 3105 is passed through steps 3115 and 3110 to provide information to the geometry module 3120 and the human interface module 3145, respectively.

The geometric modeling module 3120 is a basis for virtual scene construction, and constructs a digital model by using three-dimensional modeling software blend and the like according to the geometric dimensions of the entity objects in the physical layer module 100, and adds necessary light, material, texture rendering models and special effects, so that the three-dimensional model has real texture and is more vivid. In order to reduce the operating pressure of hardware, the model is appropriately simplified and exported as a lightweight format file such as FBX. The model file is imported into the scene building module 3160 in step 3125.

The human-computer interface module 3145 includes functions of human-computer interaction design, external input event response, and the like, and aims to implement scene roaming in a three-dimensional virtual environment, so that a user has an immersive visual experience. Files for realizing the functions of the human-computer interface module 3145 are exported in a three-dimensional modeling software environment such as blend and the like into a lightweight format such as FBX and the like, and are imported into the scene building module 3160 in step 3155.

The scene construction module 3160 further improves the functions of the geometric modeling module 3120 and the human-computer interface module 3145 in a real-time three-dimensional engine such as universal, that is, the design effect of the human-computer interface module 3145 and the model file of the geometric modeling module 3120 are combined into one scene, and scene management, scene roaming, human-computer interface design, performance optimization and the like are performed. The resulting file output by the scene construction module 3160 is then imported into the scene model module 3170 in step 3165.

The scene model module 3170 is used for performing data binding on information provided by an internet of things platform such as a ThingsBoard in a JSON string format in a real-time three-dimensional engine such as an unregeal engine, so as to realize real-time updating of the state information of the real-time data-driven state signboard 305, the virtual scene 310, the live video 320 and the augmented reality 325 based on the physical object of the physical layer module 100. In order to balance the contradiction between the complexity of virtual scene rendering and real-time rendering and ensure the fluency of the system in large-scale scenes, the virtual scene needs to be tested and optimized in operation, so the scene model module 3170 feeds back the result of the test operation to the scene optimization module 3135 in step 3140.

The scene optimization module 3135 feeds back the test operation result of the scene model module 3170 to the geometric modeling 3120 and the human-machine interface 3145 module in steps 3130 and 3150, respectively performs design optimization in the two modules, and after iteration, publishes the model file of the scene model module 3170 in a real-time three-dimensional engine such as Unreal into a WebGL format and integrates the model file into an internet of things platform such as thinsgoard.

The above-mentioned embodiments only show some embodiments of the present invention, and the description thereof is more specific and detailed, but should not be construed as limiting the scope of the present invention. It should be noted that various changes and modifications can be made by those skilled in the art without departing from the spirit of the invention, and these changes and modifications are all within the scope of the invention. Therefore, the protection scope of the present invention should be subject to the claims.

Claims (10)

1. An intelligent window system of a machine tool and a construction method are characterized by comprising the following steps:

the physical layer module is an entity object in the physical space of the intelligent window system, is used for constructing an automatic information display and control system which is flexible and adapts to the configuration requirements of observation windows of different machine tools, and can be used for different types of machine tools after corresponding display hardware and control software modules are replaced;

the middle layer module is used for constructing a network system for communicating the physical layer module and the service layer module, realizing the data acquisition, processing, storage and analysis processes of the intelligent window system and quickly establishing a real-time database and an expert knowledge base during the production and processing of the machine tool;

and the service layer module is a simulation object in the virtual space of the intelligent window system, corresponds to the entity object in the physical space through the three-dimensional modeled model and the data information display tool, and is responsible for providing the service specific to the application program for the machine tool user.

2. The intelligent window system of the machine tool and the construction method thereof according to claim 1, wherein:

the entity object of the physical layer module of the intelligent window system of the machine tool comprises a tracking system, backlight illumination, a machine tool main body, a camera and a display;

the machine body includes within it a spindle, a tool, a clamping device, a workpiece, and other machine components of interest to a machine user.

3. The intelligent window system and the construction method of the machine tool according to claim 2, wherein: the tracking system detects the position of a machine tool user by using a sensor that tracks the head or eyes, etc. of the machine tool user.

4. The intelligent window system of the machine tool and the construction method thereof as claimed in claim 2, wherein:

the backlight illumination is adapted to a transparent display converted from a traditional Liquid Crystal Display (LCD) after disassembly;

the backlight is arranged on the wall inside the machine tool door and opposite to the observation window.

5. The intelligent window system and the construction method of the machine tool according to claim 2, wherein: the machine tool main body is a machine or equipment with a viewing window with different sizes, or a machine or equipment without the viewing window and with a completely closed shell for protecting a machine tool user.

6. The intelligent window system and the construction method of the machine tool according to claim 2, wherein: the camera is installed in the machine tool main body and is responsible for acquiring a real object state image in the machine tool main body and preprocessing the image to realize complete coverage of a scene in the machine tool main body.

7. The intelligent window system of the machine tool and the construction method thereof as claimed in claim 2, wherein: the display can adopt screens with different numbers and sizes, and is arranged at any position on the shell of the machine tool body, which is convenient for a machine tool user to view, even if the machine part of the machine tool body can limit the visual field of the machine tool user.

8. The intelligent window system and the construction method of the machine tool according to claim 1, wherein: the middle layer module comprises a network communication link, a network communication system, a data processing system and a data analysis system.

9. The intelligent window system and the construction method of the machine tool according to claim 1, wherein: the intelligent window system of the machine tool is based on the machine tool adjusting state and the actual business process of production and processing, and the construction process comprises 6 functional modules including data collection, geometric modeling, scene optimization, a human-computer interface, scene construction and a scene model, and 6 steps among the modules.

10. The intelligent window system and construction method of a machine tool according to claim 9, comprising the steps of:

step 1, the data collecting module collects all data information of the entity object and the virtual object when the machine tool runs and then divides the data information into non-model driving data and model driving data. The non-model-driven data is static type data and is stored and displayed in the Internet of things platform tool in a table form. The model driving data is dynamic data, the WebSocket protocol is used for sending the data to the platform tool of the Internet of things for storage, and the data is transmitted to the three-dimensional visualization engine in a JSON character string format for data linkage and binding. The data collected by the data collecting module are respectively provided for the geometric modeling and human-computer interface module through the step 2 and the step 3;

and 2, the geometric modeling module constructs a digital model by adopting three-dimensional modeling software according to the geometric dimension of the entity object, and adds necessary light, materials, texture rendering models and special effects, so that the three-dimensional model has real texture and is more vivid. In order to reduce the operating pressure of hardware, the model is simplified appropriately and exported into a lightweight format file, and the file is imported into the scene construction module in step 4;

and 3, the human-computer interface module comprises functions of human-computer interaction design, external input event response and the like, and aims to realize scene roaming in a three-dimensional virtual environment and enable a user to have immersive visual experience. Exporting a file realizing the functions of the human-computer interface module into a lightweight format file in a three-dimensional modeling software environment, and importing the file into a scene construction module in step 4;

step 4, the scene construction module further improves the function files of the geometric modeling module and the human-computer interface module in the three-dimensional visualization engine development tool, namely, the design effect of the human-computer interface module and the model file of the geometric modeling module are combined into one scene, scene management, scene roaming, user interface design, performance optimization and the like are carried out, and the file is led into the scene model module in step 5;

step 5, the scene model module performs data binding on information provided by the Internet of things platform development tool in a JSON character string format in a three-dimensional visualization engine development tool, realizes real-time updating of the state information of the virtual scene, the state signboard, the live video and the augmented reality module driven by real-time data of the entity object in the physical layer module, performs operation test on the modules and feeds back the test result to the scene optimization module in step 6;

and 6, feeding back the test operation result of the scene model module in the step 5 to the geometric modeling and human-computer interface module by the scene optimization module, respectively performing design optimization in the geometric modeling and human-computer interface module, publishing the model file of the virtual scene module into a WebGL format in a three-dimensional visualization engine development tool after iteration, and integrating the model file into an Internet of things platform tool.

Images