CN113566864A - Distributed machine vision system based on 5G and edge calculation - Google Patents
Distributed machine vision system based on 5G and edge calculation Download PDFInfo
- Publication number
- CN113566864A CN113566864A CN202111032544.2A CN202111032544A CN113566864A CN 113566864 A CN113566864 A CN 113566864A CN 202111032544 A CN202111032544 A CN 202111032544A CN 113566864 A CN113566864 A CN 113566864A
- Authority
- CN
- China
- Prior art keywords
- plug
- detection
- instruction
- module
- light source
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004364 calculation method Methods 0.000 title claims abstract description 12
- 238000001514 detection method Methods 0.000 claims abstract description 113
- 230000002093 peripheral effect Effects 0.000 claims abstract description 49
- 238000012545 processing Methods 0.000 claims description 46
- 230000000875 corresponding effect Effects 0.000 claims description 32
- 230000007547 defect Effects 0.000 claims description 22
- 238000004458 analytical method Methods 0.000 claims description 13
- 230000001276 controlling effect Effects 0.000 claims description 12
- 238000013136 deep learning model Methods 0.000 claims description 12
- 238000010801 machine learning Methods 0.000 claims description 10
- 238000004519 manufacturing process Methods 0.000 claims description 10
- 230000002950 deficient Effects 0.000 claims description 9
- 230000003993 interaction Effects 0.000 claims description 8
- 238000000034 method Methods 0.000 claims description 7
- 230000009471 action Effects 0.000 claims description 3
- 230000008569 process Effects 0.000 claims description 3
- 238000012797 qualification Methods 0.000 claims description 3
- 238000012216 screening Methods 0.000 claims description 3
- 230000000007 visual effect Effects 0.000 claims description 3
- 238000011161 development Methods 0.000 abstract description 27
- 230000006870 function Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000001960 triggered effect Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 230000008571 general function Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 238000009517 secondary packaging Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01D—MEASURING NOT SPECIALLY ADAPTED FOR A SPECIFIC VARIABLE; ARRANGEMENTS FOR MEASURING TWO OR MORE VARIABLES NOT COVERED IN A SINGLE OTHER SUBCLASS; TARIFF METERING APPARATUS; MEASURING OR TESTING NOT OTHERWISE PROVIDED FOR
- G01D11/00—Component parts of measuring arrangements not specially adapted for a specific variable
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/70—Software maintenance or management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- General Factory Administration (AREA)
Abstract
The invention discloses a distributed machine vision system based on 5G and edge calculation, which comprises: the system comprises a cloud server, a local host, a user terminal and edge computing boxes on n detection system nodes; each edge computing box is connected with peripheral detection equipment; and the industrial camera module, the light source controller module, the motion control module and the machine vision algorithm module are independent into independent plug-ins for connecting/controlling peripheral equipment and algorithm deployment, and a set of complete vision detection system can be constructed only by selecting the plug-ins matched with the peripheral detection equipment by a user. The invention can improve the flexibility and adaptability of the edge computing machine vision system, reduce the project development difficulty, shorten the project development period and reduce the project development cost and the development difficulty.
Description
Technical Field
The invention relates to the field of machine vision, in particular to a distributed machine vision system for edge calculation.
Background
There have been many 5G relevant edge computing products in the market at present, every set of edge computing product can set up a set of machine vision detecting system, but the function of most products is all single, only to current project, when carrying out other different tasks, need begin again to develop the project, when current project demand changes even, need make huge adjustment to this system again, the development degree of difficulty is big, the flexibility is low, the development cycle length, project achievement reuse rate is low. For example, when switching industrial cameras of different manufacturers, the conventional edge computing system needs to perform secondary development again according to sdk of a new product, recompile software, in addition, at present, deep learning models are multiple and diverse (such as yolo series, R-CNN series, mobilenet series and other series), deployment modes are different (such as libtorch, Tenstort, Openviro and other well-known deployment frames), if the edge computing vision detection system in the market is adopted, each time when the models are switched due to project requirements, a great deal of adjustment is needed, the development period is prolonged in the midway due to version conflict or compatibility problems among development kits, the development difficulty is high, the adaptability is low, the development period is long, the flexibility is poor, and the secondary development experience of developers is poor.
Disclosure of Invention
The invention aims to solve the defects of the prior art and provides a distributed machine vision system based on 5G and edge computing, and the control of peripheral equipment and the construction of the vision detection system can be finished only by selecting a plug-in which is matched with peripheral detection equipment by a user, so that the flexibility and the adaptability of the edge computing machine vision system can be improved, the project development difficulty is reduced, and the project development period is shortened.
In order to achieve the purpose, the invention adopts the following technical scheme:
the invention relates to a distributed machine vision system based on 5G and edge calculation, which is characterized by being applied to a production line of industrial products and comprising the following components: the system comprises a cloud server, a local host, a user terminal and edge computing boxes on n detection system nodes; each edge computing box is connected with peripheral detection equipment;
the user terminal sends a control instruction to the cloud server through 4G/5G or Ethernet, and the cloud server sends the control instruction to the local host through 4G/5G or Ethernet; the content of the control instruction comprises: the method comprises the steps of remote power on and power off instructions, detection starting instructions, detection suspending instructions, detection stopping instructions, plug-in replacement instructions, parameter setting instructions and query instructions;
the local host analyzes the control instruction to obtain a control object and instruction contents and sends the corresponding instruction contents to 1 or more edge computing boxes appointed by the control object through 4G/5G or Ethernet;
any of the edge calculation boxes includes: the system comprises a central processing unit, an industrial camera module, a light source controller module, a motion control module, a machine vision algorithm module, a network module and a display module;
the industrial camera module includes: the system comprises a USB2.0 interface plug-in, a USB3.0 interface plug-in, a GIGE gigabit network interface plug-in and a CameraLink interface plug-in, wherein the USB2.0 interface plug-in, the USB3.0 interface plug-in, the GIGE gigabit network interface plug-in and the CameraLink interface plug-in are used for connecting and controlling one or more industrial cameras in peripheral detection equipment;
the light source controller module includes: the device comprises a collimation light source plug-in, an annular light source plug-in and a strip light source plug-in, wherein the collimation light source plug-in, the annular light source plug-in and the strip light source plug-in are used for connecting and controlling one or more light source controllers in peripheral detection equipment;
the motion control module includes: the PLC control unit plug-in and the motion control card plug-in are used for connecting and controlling one or more PLC controllers and motion control cards in peripheral detection equipment;
the machine vision algorithm module includes: the system comprises a traditional image processing algorithm plug-in, a machine learning algorithm plug-in and a deep learning model plug-in, wherein the traditional image processing algorithm plug-in, the machine learning algorithm plug-in and the deep learning model plug-in are used for carrying out visual detection and image processing on data in peripheral detection equipment connected with an edge computing box of the system;
the network module includes: the 4G/5G, Ethernet and WiFi are used for data interaction with the local host;
the corresponding edge computing box utilizes a central processing unit to analyze according to the received instruction content, if the remote power on/off instruction is received, an instruction operation object is obtained, and the peripheral detection equipment and a local host which are connected with the edge computing box are remotely powered on and powered off;
if the plug-in replacing instruction is received, plug-in information is obtained so as to replace or select the corresponding industrial camera plug-in, light source controller plug-in and/or motion control unit plug-in to connect with corresponding peripheral detection equipment;
if the parameter setting instruction is received, parameter information is obtained so as to set parameters of the corresponding plug-in;
if the detection start instruction, the detection pause instruction and the detection stop instruction exist, obtaining an instruction operation object so as to control the start, pause and stop of peripheral detection equipment connected with the edge computing box through the plug-in;
if the query instruction is the query instruction, obtaining a query object so as to obtain system parameters, plug-in parameters, detection parameters and detection results of the corresponding query object;
the central processing unit calls corresponding plug-in modules in the industrial camera module and the light source controller module according to an instruction operation object of the detection starting instruction to control an industrial camera and a light source controller in peripheral detection equipment to acquire images of industrial products on the production line and send the acquired images to the display module for original image display; simultaneously sending the acquired images to the machine vision algorithm module;
the machine vision algorithm module processes the acquired image by using a plug-in of the machine vision algorithm module, obtains a processing result, sends the processing result to the display module for result display, and simultaneously sends the processing result to the motion control module;
the motion control module calls a corresponding plug-in unit according to the processing result to control a PLC (programmable logic controller) and/or a motion control card in the peripheral detection equipment to execute corresponding actions;
the n edge computing boxes send the detection result of the current industrial product to a local host through 4G/5G or Ethernet;
the local host computer collects and analyzes all execution results to obtain analysis results, locally stores the analysis results, and sends the analysis results to the cloud server through 4G/5G or Ethernet; and the cloud server distributes the analysis result to each user terminal so as to complete real-time feedback.
The distributed machine vision system based on 5G and edge calculation is also characterized in that: the processing result comprises: the number, the detection time, the number of defects, the type, the size, the shape and the position of each defect of the currently detected industrial product; if the number of the defects in the processing result is 0 or the number, the size, the type and the shape of the defects are within the qualified standards allowed by the industrial product, judging the currently detected industrial product to be a good product, otherwise judging the currently detected industrial product to be a defective product, adding the judgment result of whether the currently detected industrial product is the good product into the processing result, thus obtaining the detection result of the currently detected industrial product and sending the detection result to the display module, and simultaneously sending the serial number of the currently detected industrial product and the judgment result to the motion control module;
the display module displays the serial number, the detection time, the defect number, the type, the size, the shape and the position of each defect and a judgment result of the current industrial product according to the obtained detection result of the current industrial product, and updates the qualification rate of the industrial product and the total number of the detected industrial product;
the motion control module calls a current PLC control unit plug-in or motion control card plug-in according to the obtained serial number and judgment result of the current industrial product, controls a PLC controller or a motion control card in the connected peripheral equipment, and performs screening action on the current industrial product, namely: if the current industrial workpiece is a good product, the PLC and/or the motion control card controls electric steam equipment and/or mechanical structures in the peripheral equipment to place the good product in the assembly line, and if not, the defective product in the assembly line is placed in the assembly line.
Compared with the prior art, the invention has the beneficial effects that:
1. according to the invention, flexible networking of equipment and cross-platform data fusion processing are realized through a 5G industrial internet technology, and management is convenient; the edge computing boxes can be used independently, and each edge computing box controls one camera, one light source controller and one motion control unit and is matched with a display to form a set of machine vision detection system; a plurality of edge computing boxes can be used for replacing a lower PC with higher traditional cost, and the lower PC and the summary host form a master-slave distributed system together, the detection data of different nodes are fused, the evaluation detection result is integrated, the distributed detection is realized, the cost can be effectively saved, and the problem of difficult wiring is solved.
2. The industrial camera module, the light source controller module, the motion control module and the machine vision algorithm module are independent to form independent plug-ins for connecting/controlling peripheral equipment and algorithm deployment, and a set of complete vision detection system can be built only by selecting the plug-ins which are matched with the peripheral detection equipment, so that a user can flexibly use the existing modules to build a detection system like building blocks, project development is completed, the use is flexible, the development period is short, the adaptability is strong, and the project development cost and the development difficulty are reduced.
3. The industrial cameras sdk or transmission protocols of a plurality of manufacturers are made into plug-in forms and are embedded into the control software of the edge computing box, so that the detection system can use cameras of different manufacturers, a proper camera and a software plug-in thereof can be conveniently selected according to different project requirements, the cost is flexible and controllable, and the camera is not limited by supply shortage of camera manufacturers;
4. the invention makes the instruction communication protocol of the motion control units (such as PLC motion control system and motion control card) and light source controller of a plurality of manufacturers into plug-in form and embeds the plug-in into the software of the edge computing box, so that the detection system can use the control equipment of different manufacturers, users can freely match according to the actual situation, the development difficulty is greatly reduced, and the development period is reduced;
5. according to the invention, a plurality of machine vision algorithms are compiled into plug-ins and are embedded into software of the edge computing box, so that a user can select a proper image processing algorithm according to actual conditions, and simultaneously, a proper deep learning model deployment plug-in (such as libtorch, Tensort and the like) can be selected according to project requirements, and the method is flexible, simple, convenient and fast in deployment, high in practicability and adaptability and greatly reduces the deployment difficulty of a neural network model.
Drawings
FIG. 1 is a schematic diagram of the communication components of a distributed machine vision system based on 5G and edge computing;
FIG. 2 is a schematic diagram of the module components of the control software of the 5G edge computing box provided by the present invention;
FIG. 3 is an overall architecture diagram of the edge computing box system of the present invention;
FIG. 4 is a system board structure diagram of the edge computing box of the present invention.
Detailed Description
In this embodiment, the distributed machine vision system based on 5G and edge calculation is a distributed machine vision system that combines and applies 5G industrial network, edge calculation, master-slave distribution, and machine vision to industry, and is mainly applied to a detection production line of magnetic ring industrial products. The whole distributed system mainly comprises a cloud server, a local host, a user terminal and edge computing boxes on n detection system nodes; each edge computing box is connected with peripheral detection equipment; as shown in fig. 1, the n edge computing boxes are connected to the local host through a 5G industrial network, and the local host collects the processing results of the edge computing boxes and sends the collected processing results to the cloud server for access and real-time control of the user terminal.
The user terminal sends a control instruction to the cloud server through the 4G/5G or the Ethernet, and the cloud server sends the control instruction to the local host through the 4G/5G or the Ethernet; wherein, the content of the control instruction comprises: the method comprises the steps of remote power on and power off instructions, detection starting instructions, detection suspending instructions, detection stopping instructions, plug-in replacement instructions, parameter setting instructions and query instructions;
the local host analyzes the control instruction to obtain a control object and instruction contents and sends the corresponding instruction contents to 1 or more edge computing boxes appointed by the control object through 4G/5G or Ethernet;
as shown in fig. 2, any one of the edge calculation boxes includes: the system comprises a central processing unit, an industrial camera module, a light source controller module, a motion control module, a machine vision algorithm module, a network module and a display module; in the implementation case, the Qt cross-platform framework is used for compiling edge computing box control software to serve as a central processing unit, basic functions such as picture display, data storage, data transmission, man-machine interaction and the like are completed on the control software, a uniform industrial camera plug-in interface, a light source controller plug-in interface, a PLC motion control unit plug-in interface and a machine vision algorithm plug-in interface are compiled on the control software, and a plug-in manager is compiled on the Qt framework and used for identifying, managing and loading the industrial camera plug-in, the light source controller plug-in, the PLC motion control unit plug-in and the machine vision algorithm plug-in; finally, compiling plug-in classes by utilizing a Qt framework, and combining the corresponding plug-in classes to form an industrial camera module, a light source controller module, a motion control module and a machine vision algorithm module; wherein, each plug-in class inherits the four interface classes respectively, and writes the general function as a function interface, thereby facilitating the calling of users and realizing the corresponding purpose,
the industrial camera module includes: the system comprises a USB2.0 interface plug-in, a USB3.0 interface plug-in, a GIGE gigabit network interface plug-in and a CameraLink interface plug-in, wherein the USB2.0 interface plug-in, the USB3.0 interface plug-in, the GIGE gigabit network interface plug-in and the CameraLink interface plug-in are used for connecting and controlling one or more industrial cameras in peripheral detection equipment; in the implementation case, sdk development kits corresponding to multiple types of industrial cameras are developed for the second time by utilizing a Qt cross-platform framework and are compiled into the plug-in classes, the multiple types of plug-in classes are inherited from industrial camera interface classes, and the general functions of the industrial cameras are compiled into function interfaces for realizing the connection and control of industrial cameras of different manufacturers, different types and different interfaces;
the light source controller module includes: the device comprises a collimation light source plug-in, an annular light source plug-in and a strip light source plug-in, wherein the collimation light source plug-in, the annular light source plug-in and the strip light source plug-in are used for connecting and controlling one or more light source controllers in peripheral detection equipment; in the embodiment, the Qt cross-platform framework is utilized to encapsulate the Ethernet instruction protocols of different light source controllers, and the Ethernet instruction protocols are compiled into the light source controller plug-ins which are divided into the three types according to the types of light sources and are used for realizing the connection and control of the light source controllers of different manufacturers;
the motion control module includes: the PLC control unit plug-in and the motion control card plug-in are used for connecting and controlling one or more PLC controllers and motion control cards in peripheral detection equipment; in the implementation case, a Qt cross-platform framework is utilized to encapsulate Ethernet instruction protocols of various different PLC controllers and compile the encapsulated Ethernet instruction protocols into PLC control unit plug-ins for realizing the connection and control of the PLC controllers of different manufacturers; the Qt cross-platform framework is utilized to carry out secondary development on the motion control cards sdk of different manufacturers, and the motion control cards are compiled into a plurality of motion control card plug-ins for realizing the connection and control of the motion control cards of the corresponding manufacturers;
the machine vision algorithm module comprises: the system comprises a traditional image processing algorithm plug-in, a machine learning algorithm plug-in and a deep learning model plug-in, wherein the traditional image processing algorithm plug-in, the machine learning algorithm plug-in and the deep learning model plug-in are used for carrying out visual detection and image processing on data in peripheral detection equipment connected with an edge computing box of the system; in the implementation case, a Qt cross-platform framework is utilized to combine/package a plurality of OpenCV basic image processing algorithms, and the OpenCV basic image processing algorithms are compiled into a plug-in of a traditional image processing algorithm and used for completing simple image processing tasks; packaging the machine learning algorithm in the OpenCV by using a uniform interface, and compiling the machine learning algorithm into a machine learning algorithm plug-in for finishing deployment and use of the machine learning algorithm; finally, performing secondary packaging on Tensorrt deployment and/or libtorch deployment of multiple deep learning models by using a Qt cross-platform framework and a uniform interface, and compiling into corresponding deep learning model plug-ins for realizing deployment and use of multiple deep learning model algorithms;
the network module includes: 4G/5G, Ethernet and WiFi, which are used for data interaction with the local host.
As shown in fig. 3, the hardware of the edge computing box is composed of five parts, namely a display, a system motherboard, an external interface, an external card slot and a 5G module;
the display is used for displaying a human-computer interaction software interface, and is convenient for a user to operate.
As shown in fig. 4, the system main board of the edge computing box includes: ARM framework core board and FPGA core board. The ARM architecture core board takes a Jetson TX2 super-computation module of NVIDIA company as a core, and comprises a CPU and a GPU processing unit, wherein the CPU and the GPU processing unit are used for deploying algorithms, processing image data acquired by an industrial camera, and processing requests of a PLC (programmable logic controller), a light source controller and a user terminal and control instructions of a cloud processor; expanding a plurality of peripheral interfaces by taking Jetson TX2 as a core, wherein the peripheral interfaces are used for connecting a plurality of peripheral equipment; the FPGA core board is used for expanding the CameraLink interface, so that the edge computing box can use the camera of the CameraLink interface.
As shown in fig. 3, the types of industrial camera interfaces that can be used by the edge computing box are GIGE gigabit network interface, USB interface, and CameraLink interface, and are used to send the acquired workpiece pictures to the edge computing box.
As shown in fig. 4, the external interface of the edge control box includes: the USB interface, the Ethernet interface, the CameraLink interface, the RS485 interface, the RS23 interface and the TTL serial port are respectively used for data interaction with the cloud processor, the local host, the PLC, the light source controller and the service terminal.
As shown in fig. 4, the peripheral card slot of the edge control box includes: the system comprises a Wifi clamping groove, a 4G clamping groove and an expansion clamping groove, wherein a corresponding protocol board card is inserted in a hot-plug and replaceable use mode, and the system is used for completing analysis of various data uploaded by a collection node under various transmission protocols, a cloud processor, a local host, a PLC, a light source controller and a service terminal.
The 5G module of the edge control box is a network port or a USB3.0 interface 5G module and is used for data interaction with a local host.
The corresponding edge computing box utilizes the central processing unit to analyze according to the received instruction content, if the instruction is a remote power on/off instruction, an instruction operation object is obtained, and the peripheral detection equipment and the local host computer which are connected with the edge computing box are remotely powered on and powered off;
if the command is a plug-in replacement command, plug-in information is obtained so as to replace or select the corresponding industrial camera plug-in, light source controller plug-in and/or motion control unit plug-in to connect with corresponding peripheral detection equipment;
if the command is a parameter setting command, obtaining parameter information to set parameters of the corresponding plug-in;
if the detection start instruction, the detection pause instruction and the detection stop instruction are obtained, obtaining an instruction operation object so as to control the start, pause and stop of peripheral detection equipment connected with the edge computing box through the plug-in;
if the query instruction is the query instruction, obtaining a query object so as to obtain system parameters, plug-in parameters, detection parameters and detection results of the corresponding query object;
and the central processing unit calls corresponding plug-in modules in the industrial camera module and the light source controller module according to the instruction operation object of the detection starting instruction to control the industrial camera and the light source controller in the peripheral detection equipment to acquire images of the magnetic ring industrial products on the production line.
In a detection sub-node formed by each edge computing box, when each magnetic ring industrial product reaches the designated position of a detection production line, a pulse signal of a photoelectric sensor in peripheral equipment is triggered and is detected by an industrial camera and/or a light source controller in the peripheral equipment, and the industrial camera is triggered to acquire images; each time the acquisition is triggered, the industrial camera plug-in obtains the current magnetic ring industrial product picture from the industrial camera and sends the obtained magnetic ring industrial product picture to a display module of the edge computing box and a depth learning model plug-in the machine vision algorithm module; the display module is used for displaying the original image and recording the serial number of the current magnetic ring industrial product.
Processing the acquired image by a deep learning model plug-in a machine vision algorithm module to obtain the defect number, the type, the size, the shape and the position of each defect of the magnetic ring workpiece, if the defect number in the result is 0 or the number, the size, the type and the shape of each defect are within the qualified standards allowed by industrial products, regarding the currently detected industrial product as a good product, and otherwise, regarding the currently detected industrial product as a defective product, adding the information of whether the current industrial product is a good product into the processing result to obtain the detection result of the current industrial product, sending the detection result to a display module, and sending two information of the number and the number of the magnetic ring industrial product to a motion control module;
after the display module obtains the detection result of the current industrial product, the serial number, the detection time, the number of the defects, the type, the size, the shape and the position of each defect and whether the product is qualified or not of the current magnetic ring industrial product are displayed according to the specific content of the detection result of the current magnetic ring product, and the product qualification rate and the total number of the detected magnetic ring industrial products are updated;
the motion control module calls a current PLC control unit plug-in unit according to the information of the product number of the current industrial product and whether the industrial product is a good product or not, and controls a PLC controller in the connected peripheral equipment to perform screening action; if the current workpiece is a good product, the PLC controls a mechanical baffle in the peripheral equipment to be opened, so that the workpiece enters a good product production line; otherwise, if the defective products exist, the mechanical baffle is closed, the defective products are intercepted, and the defective products are replaced to a defective product production line to be removed.
Finally, the n edge computing boxes respectively send the final detection result of the magnetic ring industrial product to a local host through 4G/5G or Ethernet;
the local host computer collects and analyzes all execution results to obtain analysis results, locally stores the analysis results, and sends the analysis results to the cloud server through 4G/5G or Ethernet; and the cloud server distributes the analysis result to each user terminal for displaying, such as computer client software, websites, mobile phone APP and the like, and finally, the functions of magnetic ring defect detection and user service in the whole distributed magnetic ring defect detection system are completed. And if the user terminal does not disagree with the detection result, observing the detection data in real time. And if the user terminal disagrees the detection result, sending a control instruction to the cloud server through a 3G/4G/5G protocol or an Ethernet for adjusting and/or controlling the edge computing box detection child node with the disagreeable detection result. And after the cloud server takes the instruction, the cloud server carries out interaction to the local host, the local host sends a control instruction to the edge computing box with the objection, and the appointed edge computing box carries out corresponding action according to the instruction after receiving the control instruction, so that the adjustment and/or control of the user terminal on the edge computing box with the objection are completed.
The industrial camera module, the light source controller module, the motion control module and the machine vision algorithm module which are required in machine vision detection are independent into independent plug-in units and are used for connecting/controlling peripheral equipment and algorithm deployment, so that a user can flexibly use the existing modules to build a detection system of the user like building blocks, and project development is completed. When the task amount is large, the task is complex, and a subsystem formed by one edge computing box is difficult to process in real time, the task can be decomposed, the connection and communication of a plurality of edge computing devices and a cloud server are established by utilizing the 5G industrial Internet technology, the detection data of different nodes are fused, the detection result is comprehensively evaluated, a distributed detection system is formed, the distributed defect detection is completed, the popularization and the application on a production line are facilitated, the integral management of a project is completed, the use is flexible, the development period is short, the adaptability is high, the project development cost and the development difficulty are reduced, the reusability of the edge computing box and the detection system is enhanced, and the project development cost is saved.
Claims (2)
1. A distributed machine vision system based on 5G and edge calculation is characterized by being applied to a production line of industrial products and comprising: the system comprises a cloud server, a local host, a user terminal and edge computing boxes on n detection system nodes; each edge computing box is connected with peripheral detection equipment;
the user terminal sends a control instruction to the cloud server through 4G/5G or Ethernet, and the cloud server sends the control instruction to the local host through 4G/5G or Ethernet; the content of the control instruction comprises: the method comprises the steps of remote power on and power off instructions, detection starting instructions, detection suspending instructions, detection stopping instructions, plug-in replacement instructions, parameter setting instructions and query instructions;
the local host analyzes the control instruction to obtain a control object and instruction contents and sends the corresponding instruction contents to 1 or more edge computing boxes appointed by the control object through 4G/5G or Ethernet;
any of the edge calculation boxes includes: the system comprises a central processing unit, an industrial camera module, a light source controller module, a motion control module, a machine vision algorithm module, a network module and a display module;
the industrial camera module includes: the system comprises a USB2.0 interface plug-in, a USB3.0 interface plug-in, a GIGE gigabit network interface plug-in and a CameraLink interface plug-in, wherein the USB2.0 interface plug-in, the USB3.0 interface plug-in, the GIGE gigabit network interface plug-in and the CameraLink interface plug-in are used for connecting and controlling one or more industrial cameras in peripheral detection equipment;
the light source controller module includes: the device comprises a collimation light source plug-in, an annular light source plug-in and a strip light source plug-in, wherein the collimation light source plug-in, the annular light source plug-in and the strip light source plug-in are used for connecting and controlling one or more light source controllers in peripheral detection equipment;
the motion control module includes: the PLC control unit plug-in and the motion control card plug-in are used for connecting and controlling one or more PLC controllers and motion control cards in peripheral detection equipment;
the machine vision algorithm module includes: the system comprises a traditional image processing algorithm plug-in, a machine learning algorithm plug-in and a deep learning model plug-in, wherein the traditional image processing algorithm plug-in, the machine learning algorithm plug-in and the deep learning model plug-in are used for carrying out visual detection and image processing on data in peripheral detection equipment connected with an edge computing box of the system;
the network module includes: the 4G/5G, Ethernet and WiFi are used for data interaction with the local host;
the corresponding edge computing box utilizes a central processing unit to analyze according to the received instruction content, if the remote power on/off instruction is received, an instruction operation object is obtained, and the peripheral detection equipment and a local host which are connected with the edge computing box are remotely powered on and powered off;
if the plug-in replacing instruction is received, plug-in information is obtained so as to replace or select the corresponding industrial camera plug-in, light source controller plug-in and/or motion control unit plug-in to connect with corresponding peripheral detection equipment;
if the parameter setting instruction is received, parameter information is obtained so as to set parameters of the corresponding plug-in;
if the detection start instruction, the detection pause instruction and the detection stop instruction exist, obtaining an instruction operation object so as to control the start, pause and stop of peripheral detection equipment connected with the edge computing box through the plug-in;
if the query instruction is the query instruction, obtaining a query object so as to obtain system parameters, plug-in parameters, detection parameters and detection results of the corresponding query object;
the central processing unit calls corresponding plug-in modules in the industrial camera module and the light source controller module according to an instruction operation object of the detection starting instruction to control an industrial camera and a light source controller in peripheral detection equipment to acquire images of industrial products on the production line and send the acquired images to the display module for original image display; simultaneously sending the acquired images to the machine vision algorithm module;
the machine vision algorithm module processes the acquired image by using a plug-in of the machine vision algorithm module, obtains a processing result, sends the processing result to the display module for result display, and simultaneously sends the processing result to the motion control module;
the motion control module calls a corresponding plug-in unit according to the processing result to control a PLC (programmable logic controller) and/or a motion control card in the peripheral detection equipment to execute corresponding actions;
the n edge computing boxes send the detection result of the current industrial product to a local host through 4G/5G or Ethernet;
the local host computer collects and analyzes all execution results to obtain analysis results, locally stores the analysis results, and sends the analysis results to the cloud server through 4G/5G or Ethernet; and the cloud server distributes the analysis result to each user terminal so as to complete real-time feedback.
2. The distributed machine vision system based on 5G and edge computing of claim 1, wherein: the processing result comprises: the number, the detection time, the number of defects, the type, the size, the shape and the position of each defect of the currently detected industrial product; if the number of the defects in the processing result is 0 or the number, the size, the type and the shape of the defects are within the qualified standards allowed by the industrial product, judging the currently detected industrial product to be a good product, otherwise judging the currently detected industrial product to be a defective product, adding the judgment result of whether the currently detected industrial product is the good product into the processing result, thus obtaining the detection result of the currently detected industrial product and sending the detection result to the display module, and simultaneously sending the serial number of the currently detected industrial product and the judgment result to the motion control module;
the display module displays the serial number, the detection time, the defect number, the type, the size, the shape and the position of each defect and a judgment result of the current industrial product according to the obtained detection result of the current industrial product, and updates the qualification rate of the industrial product and the total number of the detected industrial product;
the motion control module calls a current PLC control unit plug-in or motion control card plug-in according to the obtained serial number and judgment result of the current industrial product, controls a PLC controller or a motion control card in the connected peripheral equipment, and performs screening action on the current industrial product, namely: if the current industrial workpiece is a good product, the PLC and/or the motion control card controls electric steam equipment and/or mechanical structures in the peripheral equipment to place the good product in the assembly line, and if not, the defective product in the assembly line is placed in the assembly line.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111032544.2A CN113566864A (en) | 2021-09-03 | 2021-09-03 | Distributed machine vision system based on 5G and edge calculation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111032544.2A CN113566864A (en) | 2021-09-03 | 2021-09-03 | Distributed machine vision system based on 5G and edge calculation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113566864A true CN113566864A (en) | 2021-10-29 |
Family
ID=78173594
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111032544.2A Pending CN113566864A (en) | 2021-09-03 | 2021-09-03 | Distributed machine vision system based on 5G and edge calculation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113566864A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113821009A (en) * | 2021-11-24 | 2021-12-21 | 北京和利时系统工程有限公司 | Edge controller and quality detection system |
| CN114363368A (en) * | 2021-12-17 | 2022-04-15 | 中国电子科技集团公司第四十一研究所 | Multi-vision detection networking centralized management method |
| CN114826869A (en) * | 2022-03-04 | 2022-07-29 | 阿里巴巴（中国）有限公司 | Device management method and device management system |
| WO2024007206A1 (en) * | 2022-07-06 | 2024-01-11 | 宁德时代新能源科技股份有限公司 | Debugging method and apparatus for production line devices, and production line system |
| CN117427892A (en) * | 2023-12-21 | 2024-01-23 | 福州城投德正数字科技有限公司 | Machine vision garbage classification recycling system based on edge collaborative computing |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030139848A1 (en) * | 2002-01-18 | 2003-07-24 | Christopher Cifra | System and method for invoking execution of a sequence of operations that includes motion control, machine vision, and data acquisition (DAQ) functionality |
| CN107310795A (en) * | 2017-07-04 | 2017-11-03 | 上海烟草机械有限责任公司 | Product external packaging detector and detecting system based on machine vision technique |
| US20180052815A1 (en) * | 2016-08-19 | 2018-02-22 | Cognex Corporation | Apparatuses, systems, and methods for providing a visual program for machine vision systems |
| CN108322548A (en) * | 2018-03-07 | 2018-07-24 | 浙江大学 | A kind of industrial process data analyzing platform based on cloud computing |
| CN110221558A (en) * | 2019-06-05 | 2019-09-10 | 镇江四联机电科技有限公司 | A kind of electrohydraulic servo valve on-line fault diagnosis gateway based on edge calculations technology |
| CN111273550A (en) * | 2020-03-12 | 2020-06-12 | 成都英孚克斯科技有限公司 | Pig house environment intelligent control system |
| CN111816252A (en) * | 2020-07-21 | 2020-10-23 | 腾讯科技（深圳）有限公司 | Drug screening method and device and electronic equipment |
| EP3798817A1 (en) * | 2019-09-27 | 2021-03-31 | Rockwell Automation Technologies, Inc. | User interface logical and execution view navigation and shifting |
| CN112671898A (en) * | 2020-12-23 | 2021-04-16 | 浙江工业大学 | Remote communication control system based on embedded equipment |
| CN113310897A (en) * | 2021-05-31 | 2021-08-27 | 广东骏亚电子科技股份有限公司 | AI visual inspection system application based on 5G algorithm |
-
2021
- 2021-09-03 CN CN202111032544.2A patent/CN113566864A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030139848A1 (en) * | 2002-01-18 | 2003-07-24 | Christopher Cifra | System and method for invoking execution of a sequence of operations that includes motion control, machine vision, and data acquisition (DAQ) functionality |
| US20180052815A1 (en) * | 2016-08-19 | 2018-02-22 | Cognex Corporation | Apparatuses, systems, and methods for providing a visual program for machine vision systems |
| CN107310795A (en) * | 2017-07-04 | 2017-11-03 | 上海烟草机械有限责任公司 | Product external packaging detector and detecting system based on machine vision technique |
| CN108322548A (en) * | 2018-03-07 | 2018-07-24 | 浙江大学 | A kind of industrial process data analyzing platform based on cloud computing |
| CN110221558A (en) * | 2019-06-05 | 2019-09-10 | 镇江四联机电科技有限公司 | A kind of electrohydraulic servo valve on-line fault diagnosis gateway based on edge calculations technology |
| EP3798817A1 (en) * | 2019-09-27 | 2021-03-31 | Rockwell Automation Technologies, Inc. | User interface logical and execution view navigation and shifting |
| CN111273550A (en) * | 2020-03-12 | 2020-06-12 | 成都英孚克斯科技有限公司 | Pig house environment intelligent control system |
| CN111816252A (en) * | 2020-07-21 | 2020-10-23 | 腾讯科技（深圳）有限公司 | Drug screening method and device and electronic equipment |
| CN112671898A (en) * | 2020-12-23 | 2021-04-16 | 浙江工业大学 | Remote communication control system based on embedded equipment |
| CN113310897A (en) * | 2021-05-31 | 2021-08-27 | 广东骏亚电子科技股份有限公司 | AI visual inspection system application based on 5G algorithm |
Non-Patent Citations (1)
| Title |
|---|
| 俞庆平: "影像测量设备软件框架构建技术的研究", CNKI优秀硕士学位论文全文库, vol. 2013, no. 3 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113821009A (en) * | 2021-11-24 | 2021-12-21 | 北京和利时系统工程有限公司 | Edge controller and quality detection system |
| CN114363368A (en) * | 2021-12-17 | 2022-04-15 | 中国电子科技集团公司第四十一研究所 | Multi-vision detection networking centralized management method |
| CN114826869A (en) * | 2022-03-04 | 2022-07-29 | 阿里巴巴（中国）有限公司 | Device management method and device management system |
| CN114826869B (en) * | 2022-03-04 | 2023-11-28 | 阿里巴巴（中国）有限公司 | Device management method and device management system |
| WO2024007206A1 (en) * | 2022-07-06 | 2024-01-11 | 宁德时代新能源科技股份有限公司 | Debugging method and apparatus for production line devices, and production line system |
| CN117427892A (en) * | 2023-12-21 | 2024-01-23 | 福州城投德正数字科技有限公司 | Machine vision garbage classification recycling system based on edge collaborative computing |
| CN117427892B (en) * | 2023-12-21 | 2024-04-12 | 福州城投德正数字科技有限公司 | Machine vision garbage classification recycling system based on edge collaborative computing |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113566864A (en) | Distributed machine vision system based on 5G and edge calculation | |
| CN110399642B (en) | Digital twin body aiming at production line and construction method and application thereof | |
| CN105045970B (en) | Analogue system and its implementation based on Unity dynamic generation three-dimensional scenic | |
| CN109933035A (en) | A kind of production line control system, method and the production system twin based on number | |
| CN111797521B (en) | Three-dimensional simulation debugging and monitoring method for automatic production line | |
| CN108108403B (en) | Spacer layer based equipment logic configuration visualization platform system and construction method thereof | |
| KR102536763B1 (en) | Programmable Logic Controller-Based Modular Acceleration Modules for Artificial Intelligence | |
| CN104009882B (en) | The method of testing and system of the equivalent satellite power-supply system based on distributed structure/architecture | |
| CN106295081A (en) | Flexible manufacturing system Simulation on Decision system | |
| CN113176759A (en) | Information management system applied to treatment of medium-low radioactive nuclear waste liquid cement solidified body | |
| CN116127821A (en) | Three-dimensional visual presentation method and platform for operation and maintenance data | |
| Ye et al. | Implementation of a production-control system using integrated automation ML and OPC UA | |
| Caiza et al. | Digital twin for monitoring an industrial process using augmented reality | |
| CN106487595A (en) | Power transformation O＆M big data network control platform | |
| CN108227520A (en) | A kind of control system and control method of the smart machine based on panorama interface | |
| Lu et al. | A general asset administration shell platform for production line design | |
| CN219351760U (en) | Programmable intelligent industrial control gateway | |
| CN104932467A (en) | Method for controlling program-controlled equipment in centralized way | |
| Hu et al. | The scheme and system architecture of product quality inspection based on software-defined edge intelligent controller (SD-EIC) in industrial internet of things | |
| CN111660294B (en) | Augmented reality control system of hydraulic heavy-duty mechanical arm | |
| CN112289107A (en) | Industrial Internet bidirectional simulation mapping training system | |
| CN207557362U (en) | A kind of full production cycle automatic test device of distribution terminal | |
| CN202230379U (en) | Data acquisition and monitoring system based on terminal touch screen | |
| CN115439006A (en) | Intelligent production line simulation system and method based on Unity3D | |
| Xiao et al. | STEP-NC enabled edge–cloud collaborative manufacturing system for compliant CNC machining |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |