CN108109460A - Equipment is visited in a kind of teaching with augmented reality chemical plant - Google Patents

Abstract

The invention discloses a kind of chemical plant teaching with augmented reality to visit equipment.The equipment is made of moving cart, host computer and the augmented reality helmet, moving cart is walked according to specific track in factory, its upper part installs camera, instead of scene in eye-observation factory, by the data transmission of acquisition to host computer, augmented reality is utilized in host computer, the threedimensional model of virtual text information and each equipment is added in real factory's contextual data, the augmented reality helmet receives the actual situation mixing contextual data from host computer, and a kind of immersion experience walked in factory is simulated by wearing the augmented reality helmet.The present invention avoids the various dangerous situations really visited in factory using augmented reality, a kind of very three-dimensional immersion experience walked in factory of simulation, student while actual experience is obtained.

Description

A teaching and visiting equipment with augmented reality function chemical factory

technical field

The invention relates to a teaching and visiting equipment with augmented reality functional chemical factory, which is a teaching and visiting equipment with augmented reality immersive experience and remote learning operation.

Background technique

At present, most of the teaching of chemical engineering courses in engineering colleges is only in the form of simple graphics and texts. This teaching mode is boring and cannot stimulate students' interest in learning. Without a comprehensive and intuitive understanding of the operating principles and internal structure of various large-scale equipment, graduates lack corresponding professional skills, and it is difficult to meet the needs of enterprises for talents. Although some universities organize students to visit factories while teaching graphics and texts, the visit time is short and the entire production process cannot be understood. Moreover, the chemical production process has a high risk factor and it is difficult to guarantee the personal safety of students. In addition, the visit of a large number of students indirectly affects the production efficiency of the factory.

Contents of the invention

The object of the present invention is to provide a chemical plant teaching and visiting equipment with augmented reality function in order to overcome the shortcomings of the prior art. The device consists of a mobile car, a host computer and an augmented reality helmet. The mobile car walks in the factory according to a specific trajectory. A camera is installed on the upper part of the car to replace the human eye to observe the scene in the factory, and the collected data is transmitted to the host computer. Using augmented reality technology, the virtual text information and the 3D model of each device are superimposed on the real factory scene data. The augmented reality helmet receives the virtual and real mixed scene data from the host computer, and simulates a kind of factory environment by wearing the augmented reality helmet. An immersive experience of walking.

In order to solve the above-mentioned technical problems, the present invention is achieved through the following technical solutions:

A chemical plant teaching and visiting equipment with augmented reality function, which is composed of a mobile car, a host computer and an augmented reality helmet; the mobile car is equipped with a camera for collecting scene data in the factory, which is transmitted to the The upper computer, the upper computer uses augmented reality technology to superimpose virtual information on the real scene data in the factory, and then transmits it to the augmented reality helmet, and the augmented reality helmet can enable the wearer to simulate a kind of walking in the factory The real three-dimensional immersive experience, and the virtual information superimposed on the scene can further learn the operating status and internal structure of each device in the factory; the wireless network is built by ZigBee and covers the entire factory.

The mobile trolley is controlled by the STM32 single-chip microcomputer to drive the trolley motor, moves in the factory according to the set trajectory, and controls the movement and stop of the mobile trolley through the host computer; the camera on the mobile trolley is controlled by the host computer Control its rotation angle to collect scene data in the factory from different perspectives; the mobile car transmits the collected scene data in the factory to the host computer in real time through the wireless network.

The upper computer includes a first upper computer and a second upper computer, the first upper computer is located on the mobile car, and is used to receive the real-time video data collected by the mobile car; the second upper computer is located on the augmented reality helmet, and the second upper computer is located on the augmented reality helmet. The cloud server is used for data transmission and command transmission between the first host computer and the second host computer, and remote control can be realized; after receiving the real-time data in the factory, the second host computer recognizes the identification objects on the equipment in the factory, Identify and locate each device, use three-dimensional registration technology to superimpose virtual information around the real device, obtain augmented reality virtual and real mixed data, and transmit it to the augmented reality helmet through the wireless network; the movement and stop of the mobile car, The rotation angle of the camera and the display of the virtual information in the augmented reality helmet are controlled by the second host computer; the virtual information includes the text introduction of the factory equipment and the three-dimensional model of the equipment; the three-dimensional model is based on the factory equipment Modeling of the real internal structure can simulate the operating state of the device, and one-click disassembly of the 3D model can be performed to intuitively display its internal structure.

The augmented reality helmet receives superimposed and mixed scene data in the factory from the second host computer through the wireless network, and projects it into the lens in front of the helmet. Trainees simulate a real three-dimensional immersive experience of walking in the factory by wearing an augmented reality helmet, and further learn the operating status and internal structure of each equipment in the factory through superimposed virtual information, such as the three-dimensional model of the equipment.

The teaching and visiting equipment of a chemical plant with augmented reality function described in the present invention also has a teacher system, and the teacher system controls the display of the virtual information of the three-dimensional model in the augmented reality helmet through the second host computer, and controls the simulated equipment in the augmented reality helmet operating status, and can disassemble the 3D model with one click; enable students to learn more intuitively and clearly the operating principle of each equipment in the chemical production process and the internal structure of each equipment; on the other hand, the teacher system through the second The upper computer can also control the movement and stop of the moving car and the rotation angle of the camera so as to observe the equipment of interest in the factory.

The chemical plant teaching and visiting equipment with augmented reality function described in the present invention also has a remote learning function. The first upper computer and the second upper computer perform data transmission and command control through the cloud server to realize remote operation; at the same time, multiple groups located at The first host computer and the second host computer in different locations can be paired and connected through the cloud server, instead of only one-to-one connection, so that resource sharing can be realized to the greatest extent, resources can be utilized to the greatest extent, and students learn More professional skills.

Due to the adoption of the above technical solution, the present invention provides a chemical plant teaching and visiting equipment with augmented reality function, which has the following beneficial effects compared with the prior art:

Using augmented reality technology to simulate a true three-dimensional immersive experience of walking in the factory, students can avoid various dangerous situations in the factory while gaining real experience;

Using augmented reality technology, the virtual equipment model is superimposed and displayed in the real environment, which can make the teaching more vivid and stimulate students' interest in learning;

With distance learning function, teaching activities and students' independent learning are not limited by time and space, making learning more convenient and free;

Realize the sharing of resources, make the most use of resources, and at the same time, students can obtain more learning resources.

Description of drawings

Fig. 1 is the overall schematic diagram of the present invention;

Fig. 2 is a structural representation of the present invention;

Fig. 3 is a flowchart of the three-dimensional registration technology of the present invention;

Fig. 4 is a conversion diagram of the coordinate relationship of the three-dimensional registration technology of the present invention;

Fig. 5 is the scene rendering of the augmented reality of the present invention;

Fig. 6 is a dismantling effect diagram of the three-dimensional equipment model of the present invention.

Detailed ways

Below in conjunction with accompanying drawing and specific embodiment the present invention is described in further detail:

A chemical plant teaching and visiting equipment with augmented reality function of the present invention, as shown in Figure 1, consists of a mobile car, a host computer and an augmented reality helmet. The mobile car is equipped with a camera to collect scene data in the factory, and transmits it to the host computer through a wireless network. The host computer uses augmented reality technology to superimpose virtual information on the real scene data in the factory, and then transmits it to the real scene data in the factory. In the augmented reality helmet, trainees simulate a true three-dimensional immersive experience of walking in the factory by wearing the augmented reality helmet, and further learn the operating status and internal structure of each equipment in the factory through the virtual information superimposed on the scene. The wireless network is built by ZigBee and covers the entire factory.

The mobile trolley is controlled by the STM32 single-chip microcomputer to drive the trolley motor, moves in the factory according to the set trajectory, and controls the movement and stop of the mobile trolley through the host computer; the camera on the mobile trolley is controlled by the host computer Control its rotation angle to collect scene data in the factory from different perspectives; the mobile car transmits the collected scene data in the factory to the host computer in real time through the wireless network.

Described upper computer comprises first upper computer and second upper computer, and first upper computer is positioned at mobile dolly, is used to receive the real-time video data that mobile dolly collects; Second upper computer is positioned at augmented reality helmet, first upper computer and The second upper computer uses the cloud server for data transmission and command transmission, which can realize remote control. After receiving the real-time data in the factory, the second upper computer can identify each device by identifying the markers on the equipment in the factory. Positioning, using three-dimensional registration technology to superimpose virtual information around the real device, get augmented reality virtual and real mixed data, and transmit it to the augmented reality helmet through the wireless network. The movement and stop of the mobile car, the rotation angle of the camera and the display of virtual information in the augmented reality helmet are all controlled by the second host computer;

The virtual information includes a text introduction of the equipment and a three-dimensional model of the equipment.

The 3D model is modeled according to the real structure inside the device, which can simulate the running state of the device. In addition, the 3D model can be disassembled with one key to intuitively display its internal structure.

In the present invention, the visual tracking and registration technology based on artificial markers is adopted to complete the identification and positioning of each device by identifying different markers placed on each device. The three-dimensional registration technology, its implementation flowchart is shown in Figure 3, and its entire three-dimensional registration process includes the following steps:

The first step: image feature detection and description, mainly to extract the feature information of the marker in the image, and describe its features. In the present invention, the FAST (Features from Accelerated Segment Test) algorithm is used to detect feature points, and the pixel points to be detected and The gray value of the pixels in the neighborhood, these pixels are divided into three categories:

Where d is the neighborhood point of the point C to be detected, I _d is the gray value of the point, I _C is the gray value of the point C to be detected, t is a custom threshold, and -1, 0, 1 represent points respectively d is darker, similar or brighter than detection point C. If there are n consecutive pixel points detected as dark points (d=-1) or bright points (d=1), it is determined that the point is a corner point. In addition, the present invention uses the ORB (oriented FASTandrotated BRIEF) algorithm to describe the corner points, which defines the main direction of the feature points as:

θ＝a tan2(m ₀₁ ,m ₁₀ )

in, is the area gray level sum of the pixel in the x direction, is the area gray level sum of the pixel in the y direction. Define the descriptor as:

g _n (p,θ)=f _n (p)|( _xi ,y _i )∈S _θ

Wherein, S is a matrix with a size of 2xn defined according to the position combination ( _xi , y _i ) of the central pixel of the selected n groups of pixel block pairs.

The second step: perform feature matching, and search for corresponding matching identifiers from the feature information template library.

Step 3: Calculate the camera pose, identify the marker in the image through the above steps, take the plane where the marker is located as the Z plane, and establish a right-handed coordinate system, according to the formula

Calculate the external parameter rotation matrix R and translation vector T of the camera and the internal parameters f _u , f _v , u ₀ , v ₀ , s to obtain the external parameter matrix M ₁ and the internal parameter matrix M ₂ , and determine the camera pose to obtain the projection matrix M.

The fourth step: virtual and real registration, using the image as the projection plane, according to the external parameters and internal parameters of the camera calculated in the previous step, and the coordinate conversion sequence shown in Figure 4, perform corresponding projection transformation on the virtual information, and superimpose it on the target object , to achieve an augmented reality effect.

Figure 2 is a schematic diagram of the structure of the present invention. The mobile car moves in the factory according to a specific trajectory, and uses the camera installed on its upper part to replace the human eyes to observe the factory environment, and transmits the collected scene data in the factory to the first host computer through the wireless network; the first host computer and the second host computer The two host computers use the cloud server for data transmission and command transmission to realize remote control. At the same time, the first host computer located in different factories and the second host computer at different locations can be connected to each other, instead of only one-to-one connection, to achieve Maximum sharing of resources; after the second host computer receives the real-time data in the factory, it identifies and locates each device by identifying the markers on the equipment in the factory, and uses three-dimensional registration technology to superimpose virtual information around the real equipment , get the virtual and real mixed data of augmented reality, and transmit it to the augmented reality helmet through the wireless network, on the other hand, set the course and remotely control the mobile car through the second host computer, such as controlling the display of the virtual 3D model in the augmented reality helmet, The start and stop of the mobile car and the rotation angle of the camera, etc.; the augmented reality helmet projects the received image onto the lens in front of it, and the students wearing the augmented reality helmet will see the scene in the factory after the augmented reality, and the mobile car replaces the students Walking in the factory, the camera replaces human eyes to observe, simulates an immersive experience of walking in the factory, and learns the internal structure of each device through the virtual three-dimensional model of the device to understand its operating status.

The augmented reality helmet receives the mixed virtual and real scene data in the factory through the wireless network, and projects it into the lens in front of the helmet. By wearing the augmented reality helmet, the trainees simulate a true three-dimensional immersive experience of walking in the factory, and pass The superimposed virtual information, such as the three-dimensional model of the equipment, further learns the operating status and internal structure of each equipment in the factory.

Fig. 5 shows the scene effect of the augmented reality of the present invention. Put a virtual three-dimensional engine model into the real environment through augmented reality technology, and students can see the engine model is actually placed in the environment by wearing the augmented reality helmet, and can simulate the running state, just like a real engine in the It works the same as before. In addition, Figure 6 shows the dismantling effect diagram of the three-dimensional equipment model of the present invention. The three-dimensional model of the virtual equipment is disassembled with one key, and the various components of the equipment are displayed, so that students can understand the internal structure of the equipment more intuitively. .

Claims (6)

1. A chemical plant teaching and visiting device with augmented reality function, characterized in that: the device is composed of a mobile car, a host computer and an augmented reality helmet; the mobile car is equipped with a camera for collecting scene data in the factory, through The wireless network is transmitted to the host computer, and the host computer uses augmented reality technology to superimpose virtual information on the real scene data in the factory, and then transmits it to the augmented reality helmet, which enables the wearer to simulate A true three-dimensional immersive experience of walking in the factory, and through the virtual information superimposed on the scene, it is possible to further learn the operating status and internal structure of each device in the factory; the wireless network is built by ZigBee and covers the entire factory. 2. A chemical plant teaching and visiting equipment with augmented reality function according to claim 1, characterized in that: the mobile trolley is controlled by an STM32 single-chip microcomputer to drive the trolley motor, and moves in the factory according to the set track, passing The upper computer controls the moving and stopping of the mobile car; the camera on the mobile car controls its rotation angle through the upper computer, and collects scene data in the factory from different perspectives; the factory will be collected by the mobile car The scene data is transmitted to the host computer in real time through the wireless network. 3. A chemical plant teaching and visiting equipment with augmented reality function according to claim 1, characterized in that: the host computer includes a first host computer and a second host computer, and the first host computer is located on the mobile trolley on, used to receive the real-time video data collected by the mobile car; the second host computer is located on the augmented reality helmet, and the cloud server is used between the first host computer and the second host computer to perform data transmission and command transmission, and can realize remote Control; after the second host computer receives the real-time data in the factory, it recognizes the markers on the equipment in the factory to identify and locate each equipment, and uses three-dimensional registration technology to superimpose virtual information around the real equipment to obtain augmented reality. The virtual and real mixed data are transmitted to the augmented reality helmet through the wireless network, the movement and stop of the mobile car, the rotation angle of the camera and the display of virtual information in the augmented reality helmet are all controlled by the second host computer ; The virtual information includes the text introduction of the factory equipment, and the three-dimensional model of the equipment; the three-dimensional model is modeled according to the real structure inside the factory equipment, can simulate the operating state of the equipment, and can carry out a one-key operation on the three-dimensional model Disassemble to show its internal structure intuitively. 4. A chemical plant teaching and visiting device with augmented reality function according to claim 1, characterized in that: the augmented reality helmet receives superimposed and mixed scene data in the factory from the second host computer through the wireless network, and It is projected into the lens on the front of the helmet. 5. A chemical plant teaching and visiting device with augmented reality function according to claim 1, characterized in that: the device also has a teacher system, and the teacher system controls the three-dimensional model in the augmented reality helmet through the second host computer The display of virtual information can control the running state of the simulated equipment in the augmented reality helmet, and can dismantle the 3D model with one key; the teacher system can also control the movement and stop of the mobile car and control the movement of the camera through the second host computer. angle of rotation. 6. A chemical industry teaching and visiting device with augmented reality function according to claim 1, characterized in that: the device also has a distance learning function, and the first host computer and the second host computer perform data transmission through the cloud server And command control to realize remote operation; at the same time, multiple groups of the first host computer and the second host computer located in different locations can be paired and connected through the cloud server.