CN111401154A - Accurate delivery transparentization auxiliary operation device of commodity circulation based on AR - Google Patents
Accurate delivery transparentization auxiliary operation device of commodity circulation based on AR Download PDFInfo
- Publication number
- CN111401154A CN111401154A CN202010131920.2A CN202010131920A CN111401154A CN 111401154 A CN111401154 A CN 111401154A CN 202010131920 A CN202010131920 A CN 202010131920A CN 111401154 A CN111401154 A CN 111401154A
- Authority
- CN
- China
- Prior art keywords
- logistics
- workshop
- information
- module
- equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012384 transportation and delivery Methods 0.000 title claims description 39
- 238000009826 distribution Methods 0.000 claims abstract description 54
- 238000012545 processing Methods 0.000 claims abstract description 42
- 238000004519 manufacturing process Methods 0.000 claims abstract description 40
- 238000000034 method Methods 0.000 claims abstract description 31
- 230000008569 process Effects 0.000 claims abstract description 20
- 238000000605 extraction Methods 0.000 claims abstract description 17
- 238000013500 data storage Methods 0.000 claims abstract description 11
- 230000004927 fusion Effects 0.000 claims abstract description 9
- 238000007781 pre-processing Methods 0.000 claims abstract description 7
- 238000009877 rendering Methods 0.000 claims abstract description 6
- 239000000463 material Substances 0.000 claims description 28
- 238000003860 storage Methods 0.000 claims description 9
- 239000013598 vector Substances 0.000 claims description 9
- 230000010354 integration Effects 0.000 claims description 6
- 238000001514 detection method Methods 0.000 claims description 5
- 238000009825 accumulation Methods 0.000 claims description 4
- 238000004140 cleaning Methods 0.000 claims description 2
- 230000000007 visual effect Effects 0.000 abstract description 18
- 238000005516 engineering process Methods 0.000 abstract description 9
- 238000012800 visualization Methods 0.000 abstract description 3
- 238000007689 inspection Methods 0.000 abstract description 2
- 238000012544 monitoring process Methods 0.000 description 10
- 238000004458 analytical method Methods 0.000 description 9
- 239000002994 raw material Substances 0.000 description 7
- 238000002474 experimental method Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000013523 data management Methods 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 238000007726 management method Methods 0.000 description 3
- 230000000903 blocking effect Effects 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000003754 machining Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004040 coloring Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005538 encapsulation Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000007654 immersion Methods 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000004886 process control Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000013468 resource allocation Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
- G06Q10/083—Shipping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/04—Manufacturing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/757—Matching configurations of points or features
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Economics (AREA)
- Multimedia (AREA)
- Tourism & Hospitality (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Business, Economics & Management (AREA)
- Evolutionary Computation (AREA)
- Strategic Management (AREA)
- Health & Medical Sciences (AREA)
- Marketing (AREA)
- Human Resources & Organizations (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Databases & Information Systems (AREA)
- Manufacturing & Machinery (AREA)
- Primary Health Care (AREA)
- Medical Informatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computing Systems (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- Development Economics (AREA)
- Entrepreneurship & Innovation (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to an AR-based logistics accurate distribution transparentization auxiliary operation device which comprises a data acquisition and processing module, a data storage module, an equipment feature extraction module, an information fusion module and an AR display module, wherein the data acquisition and processing module is used for acquiring and preprocessing logistics distribution information; the equipment feature extraction module is used for constructing a three-dimensional virtual workshop model, and rendering and feature extraction are carried out on the three-dimensional virtual workshop model; the information fusion module superimposes virtual information to a workshop environment based on the characteristics of the workshop physical equipment; the AR display module is used for displaying logistics information corresponding to actual workshop equipment in a virtual reality mode. Compared with the prior art, the method can acquire real-time data of production and logistics of products in the ordering, processing, assembling and quality inspection processes, realize visual guidance of logistics of the whole process from ordering to manufacturing of workshop products, transmit workshop information to operators comprehensively, in real time and accurately by the aid of a visualization technology, and improve processing precision.
Description
Technical Field
The invention relates to intelligent production equipment, in particular to an AR-based auxiliary operation device for precise logistics distribution and transparence.
Background
With the rapid development of the internet of things technology and the digital manufacturing technology, the manufacturing industry also makes obvious progress, and the production workshops of more and more modeling enterprises are changed to digital workshops so as to achieve the purposes of cost reduction, quality improvement, efficiency improvement and rapid response to the market. The digital workshop is a computer virtual environment which is based on hardware facilities such as production equipment and production facilities, designs, manages, simulates, optimizes and visually operates production resources and a production process through means such as digitalization, networking, intellectualization and the like on the basis of physics of links such as process design, production organization, process control and the like. Because workshop production order form insertion phenomenon ubiquitous, in the face of emergency such as line limit storehouse material backlog, upstream and downstream material misdelivery and warehouse lack of material among the workshop commodity circulation operation process, lack real-time visual guide information, commodity circulation dispatch personnel can't adjust accurately, lack visual man-machine interaction equipment simultaneously, can't accurate reflection in process of goods in process and the relation that the logistics distribution links up.
In order to solve the above technical problems, some innovations have been made in the prior art. For example, patent application CN109976296A discloses a visual workshop production process based on virtual sensors, which mainly comprises the following steps: firstly, acquiring the layout of a production workshop, the size of equipment and monitoring data of monitoring points, carrying out visual modeling on the workshop to form a model base, and importing the model base into a visual management and control system to complete the layout of the workshop; then acquiring monitoring data, abstracting the monitoring data into a virtual sensor, and realizing remote communication based on an OPC technology; then, establishing layered monitoring on the production workshop according to the acquired monitoring data; and finally, analyzing the monitoring data, and predicting and analyzing the fault. However, the visual workshop cannot guide material distribution in the production workshop, and the experience of human-computer interaction is poor.
For another example, patent application CN109375595A discloses a visual workshop monitoring method, device and apparatus, which mainly comprises the following steps: and constructing a workshop resource model, wherein the workshop resource model comprises an equipment model, a personnel model, a workpiece model, a storage model and a workshop environment model, and establishing a virtual workshop three-dimensional scene model according to the workshop resource model. And then establishing a workshop production system model and a data management model, and controlling the operation logic operation contained in the production model of the workshop three-dimensional scene model according to the real-time workshop data in the data management model. And establishing a virtual workshop operation mode according to the production system model and the workshop real-time data, controlling the state change of each module in the virtual workshop operation mode according to the updated workshop real-time data in the data management model, and mapping the state change into a workshop three-dimensional scene model. However, the plant logistics guidance lacks certain prediction, for example, which station the plant production materials should be distributed to, and the visual monitoring modeling process for a plant is complex, has many modules, and has various types of data to be collected, and is complex to implement.
For another example, patent application CN109697584A discloses a visual logistics optimization method, which mainly comprises the following steps: firstly, connecting an internet + resource module in a system with a data cloud module through big data acquisition and capture; the intelligent analysis centralized control module is connected with the data cloud module and performs data exchange, data mining and data analysis with each other; the intelligent analysis centralized control module is connected with the intelligent logistics center module and performs automatic control and data processing between the intelligent analysis centralized control module and the intelligent logistics center module; the intelligent analysis centralized control module is connected with the knowledge base system and the service module, and performs automatic control and data processing between the intelligent analysis centralized control module and the knowledge base system; the intelligent logistics center module is connected with the client side to perform intelligent resource allocation; the client is connected with the intelligent analysis centralized control module, and information feedback and customized service are carried out among the client and the intelligent analysis centralized control module; the knowledge base system and the business module are connected with the marketing end and carry out data support and business evaluation mutually. However, the method is a commercial logistics management mode, logistics is optimized by visualization rather than guidance, and if the technology is put into a workshop, processing information of workshop materials and the working state of equipment cannot be displayed in real time, and accurate distribution of the workshop materials cannot be achieved.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide an AR-based logistics accurate distribution transparentization auxiliary operation device.
The purpose of the invention can be realized by the following technical scheme:
an AR-based logistics accurate distribution transparentization auxiliary operation device comprises a data acquisition and processing module, a data storage module, an equipment feature extraction module, an information fusion module and an AR display module,
the data acquisition and processing module is used for acquiring and preprocessing logistics distribution information;
the data storage module is used for storing the logistics distribution information;
the equipment feature extraction module is used for constructing a three-dimensional virtual workshop model, and performing rendering and feature extraction on the three-dimensional virtual workshop model to obtain workshop physical equipment features;
the information fusion module is used for drawing real-time production information and logistics distribution information of a workshop into virtual information and superposing the virtual information to a workshop environment based on the characteristics of physical equipment of the workshop;
the AR display module is used for displaying logistics information corresponding to actual workshop equipment in a virtual reality mode.
Further, the logistics distribution information includes part processing start time, process processing duration, product order information, line side warehouse material accumulation condition of a distribution start station, target station requirement, basic information of equipment, working procedure of a product, a distribution path, equipment positioning information between two stations and positioning information of a logistics specialist
Further, the pre-processing includes cleaning, integration and standardization.
Further, the data storage module stores data by adopting a corresponding storage mode according to the data integration characteristics, wherein the storage mode comprises a relational database, a non-relational database and a file system.
Further, in the device feature extraction module, the feature extraction specifically includes: and extracting the features of the interest points and the corner points of the three-dimensional virtual workshop model based on equipment by using an SIFT feature detection algorithm, and describing the features of the collected interest points and the corner points by using an SIFT feature point description algorithm.
Further, the virtual information is represented by a logistics guide three-dimensional information box.
Further, the overlaying of the virtual information to the workshop environment specifically includes:
and (3) extracting the physical equipment characteristics of the workshop from the field equipment characteristics and the three-dimensional virtual workshop model by using a matching algorithm to perform characteristic point matching, and superposing after judging to be consistent.
Further, in the matching algorithm, for the matching of high-dimensional features, Euclidean distance is adopted as a feature point to carry out similarity judgment; and carrying out similarity judgment by adopting a Hamming distance in the low-dimensional binary descriptor vector.
Further, the AR display module is a logistics headset AR device.
Further, the information displayed by the AR display module comprises an order number, a material type, a delivery quantity, a delivery starting station, a delivery target station, a processing ending time, a predicted delivery and transportation time and a next delivery task.
Compared with the prior art, the invention can acquire real-time data of production and logistics of products in the ordering, processing, assembling and quality inspection processes, realizes the logistics visual guidance of the whole process from ordering to manufacturing of workshop products, transmits workshop information to operators comprehensively, accurately in real time by using the visualization technology, changes the interaction mode of people and equipment, and has the following beneficial effects:
1) and the production information and the logistics information of the workshop are transparent. The virtual entity projected by the AR equipment can reflect the processing state and the logistics state of the products on line, accurately pushes production information and logistics information to logistics specialists on a production line, and determines material distribution tasks.
2) By utilizing the AR technology, a centralized retrieval material distribution task is not needed. The logistics specialist carries the head-mounted AR equipment without needing to go to a workshop computer terminal to search the current logistics distribution tasks layer by layer, the head-mounted AR equipment faces the workshop equipment to display the distribution tasks required by the work center, and meanwhile, both hands of the logistics specialist are liberated.
3) And the virtual guide information is fused with the physical entities of the workshop in real time. The transparency means of the workshop manufacturing information is combined with the virtual registration technology of the AR, the transparency information is overlaid on workshop equipment in real time, and the immersion feeling is strong.
Drawings
FIG. 1 is a schematic structural view of the present invention;
FIG. 2 is a flow chart of the present invention;
FIG. 3 is a schematic diagram of detecting Gaussian difference pyramid SIFT feature points in the embodiment;
FIG. 4 is a flow chart of the processing technology of the production line of the workshop CD L in the embodiment.
Detailed Description
The invention is described in detail below with reference to the figures and specific embodiments. The present embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation manner and a specific operation process are given, but the scope of the present invention is not limited to the following embodiments.
The embodiment provides an AR-based logistics accurate distribution transparentization auxiliary operation device, which comprises a data acquisition and processing module a, a data storage module B, an equipment feature extraction module C, an information fusion module D and an AR display module E, as shown in fig. 1.
The data acquisition and processing module A mainly has the tasks of data acquisition, filtration, encapsulation and processing. Common workshop data acquisition modes include RFID, sensors, intelligent terminals and the like. The data processing and analyzing is a data center of the system and mainly used for preprocessing data. If the data collected in real time or the data accessed by the external software system is directly used without being processed, a large amount of invalid data may exist, so that the data needs to be preprocessed before being used, and the data is processed and filtered according to specific requirements, so that the data which effectively meets the requirements is obtained.
The data storage module B mainly aims at the problem of workshop data storage, and different storage modes such as a relational database (typically MySQ L), a non-relational database (e.g. HBase), a file system (e.g. HDFS) and the like need to be adopted according to the integration characteristics of workshop data because the workshop data is diverse and the storage characteristics of structured data and unstructured data are different.
And the equipment feature extraction module C is mainly used for constructing a three-dimensional virtual workshop model according to the digital workshop, rendering and extracting features of the three-dimensional virtual workshop model, and realizing the one-to-one mapping relation of the features from the workshop equipment to the three-dimensional virtual equipment model.
The information fusion module D is mainly used for drawing the collected real-time production information and logistics information of the workshop into virtual information and overlaying the virtual information to the workshop environment. And meanwhile, the three-dimensional logistics guidance information frame is bound with the features extracted from the module C, so that the three-dimensional virtual guidance information can be bound according to the features of the workshop physical equipment in the actual guidance application.
And the AR display module E is mainly used for monitoring and guiding the logistics distribution of the workshop and receiving effective data of the logistics distribution. The virtual indication information projected from the logistics head-mounted AR equipment can reflect relevant data such as the working state of the equipment, the completion condition of a current order, the line side inventory condition and the like. The logistics special staff combines the actual situation to carry out certain management and control to the workshop logistics.
Taking logistics distribution among certain workshop stations as an example, a CD L production line is selected on a production site of the workshop for visual processing and distribution guidance, and a logistics specialist can obtain the following information, namely, an order number, a material type, a distribution quantity, a distribution starting station, a distribution target station, a processing ending time, a predicted distribution and transportation time and a next distribution task through AR equipment.
The blockage in the embodiment refers to the fact that parts at the current station are not timely distributed to the next station after production is completed, so that the station cannot normally operate, and the misdistribution refers to the fact that a material to be sent to the station A is misdistributed to the station B by a logistics special worker.
As shown in fig. 2, the specific implementation process using the above device is as follows:
and S01, analyzing logistics guidance requirements, firstly, carrying out systematic logistics guidance service requirement analysis on two raw materials, namely an oil cylinder shell and a piston rod on a CD L assembly line, and determining information required to be transparent for workshop logistics distribution tasks, such as processing time, distribution paths, time and the like.
S02: and (6) collecting logistics data. When a part is machined, working condition data of equipment, such as real-time information of equipment coordinates, machining time, order progress of machining in the equipment and the like, and material and personnel positioning information are acquired.
Through the production information system, the logistics information system and through sensor acquisition of transferring the workshop, the data that mainly acquire have: the method comprises the following steps of starting time of part processing, process processing time, order information of products, line side warehouse material accumulation condition of a distribution starting station, target station requirements, basic information of equipment, working procedures of the products, a distribution path, equipment positioning information between two stations and positioning information of logistics specialists.
S03: and (4) preprocessing logistics data. In order to improve the quality of data analysis, the data collected in S02 is cleaned, integrated, normalized and analyzed, and useful data is retained.
And S04, storing logistics data, storing the processed data, and storing order information in MySQ L according to the integration characteristics of workshop data, wherein the basic information of the equipment is stored in HBase, the requirements of target stations, the working procedures of products, the storage of distribution paths, the processing start time, the processing duration of the process, and the storage condition of the line side library materials of the distribution start stations in the HDFS.
S05: and (5) constructing logistics guide information. The method comprises the steps of deducing a processing end time according to processing start time and process processing time, deducing a delivery quantity according to a line side library material accumulation condition of a delivery start station and a target station requirement, deducing a delivery start station according to basic information of equipment, deducing a target station according to a working procedure of a product, deducing a predicted delivery time according to a delivery path, equipment positioning between two stations and positioning information of a logistics specialist, carrying out logic programming on the distribution path, the equipment positioning between two stations and the predicted delivery time by utilizing C # language, systematically constructing a logistics guidance three-dimensional information frame model in Unity, and providing related delivery guidance information for the logistics specialist.
S11: and (5) three-dimensional modeling of workshop equipment. And (3) utilizing SolidWorks to perform three-dimensional geometric modeling on stations of workshop equipment, namely GW-2053, GW-3619, GW-3384, GW-2113, GW-2275 and GW-9000.
S12: and rendering a three-dimensional model of the workshop. Rendering the three-dimensional geometric equipment model constructed in the S11 at Unity, ensuring that the virtual workshop model can be consistent with the actual workshop in terms of coloring, light source, appearance and the like, and ensuring that the virtual three-dimensional model is consistent with workshop equipment in terms of interest points and corner point characteristics.
S13: and extracting the characteristics of the three-dimensional model of the workshop. The characteristic points are the parts with the most severe change in the local area of the image, and the key to obtaining a sufficient amount of characteristic point information with robustness is whether the image can be successfully matched. The feature representations of different models are different, i.e. unique. In this embodiment, an SIFT feature detection algorithm is used to perform feature extraction of interest points and corner points based on equipment on the workshop three-dimensional model built in S11, and an SIFT feature point description algorithm is used to describe the collected interest points and corner points.
1) Feature point detection algorithm
The SIFT feature point detection algorithm detects feature points on a gaussian difference scale space, firstly, N layers of gaussian scale space image pyramids need to be established, and then, adjacent difference is carried out on each layer to establish N-1 layers of gaussian difference pyramids, as shown in fig. 3. And removing the images of the first layer and the last layer in the DOG pyramid, and comparing the pixel values of 26 points in total for each pixel point in each remaining layer with 8 adjacent points around the pixel point and 9 points in the neighborhoods of corresponding positions in two layers of images with adjacent scales up and down in the pyramid.
2) Feature point description algorithm
The SIFT descriptor can assign a direction feature main direction to each key point by using the gradient direction distribution characteristic of the pixels in the neighborhood of the key point, so that the descriptor has invariance to image rotation.
The main direction is determined using the gradient of the feature points, the gradient of a point being expressed as:
the gradient amplitude is:
the gradient direction is as follows:
in actual calculation, sampling is carried out in a neighborhood window with the feature point as the center, and the gradient direction of a neighborhood pixel is counted by using a histogram. The histogram typically has one bin per 10 degrees for a total of 36 bins. And adding the amplitude of the point in the neighborhood into the histogram according to the column corresponding to the angle. The peak of the histogram represents the main direction of the neighborhood gradient at the feature point, i.e., the direction of the feature point.
S14: and developing an AR application scene. And binding the three-dimensional information frame with the workshop three-dimensional virtual model built in the S11 and the extracted equipment characteristics in the Unity according to the logistics guidance three-dimensional information frame built in the S05, and then integrating the three-dimensional information frame with the workshop three-dimensional virtual model into an AR display scene, namely displaying the static data of the workshop through AR equipment.
S15: and superposing and fusing virtual and actual information. And matching the characteristics of the field equipment and the characteristics of the virtual equipment model by using a matching algorithm, and if the characteristics are consistent, superposing the constructed logistics guide three-dimensional information frame on the field equipment of the workshop.
1) And matching the two high-dimensional features, and adopting the Euclidean distance as similarity judgment of feature points in the two images. When SIFT feature vectors of two images are generated, it is assumed that the feature vector X in the candidate image is [ X1, X2 … xn ], and the feature vector Y in the reference image is [ Y1, Y2 … yn ], which is expressed as follows:
where n is the dimension of the feature point descriptor, in the SIFT algorithm, n is typically 128.
2) Hamming distance is employed in low-dimensional binary descriptor vectors to measure similarity between vectors. The hamming distance between two equal-length strings is defined as the number of different characters at the same position of the two strings.
Where x, y are binary descriptor vectors, k is the dimension of the vector,
is an exclusive or instruction.
S16: and displaying logistics information. The logistics specialist aims at the workshop equipment by utilizing the AR equipment, and information such as order numbers, material types, delivery quantity, delivery starting stations, delivery target stations, processing ending time, predicted delivery and transportation time, next delivery tasks and the like can be displayed on the AR equipment to complete the logistics delivery tasks. The logistics special staff can conduct real-time logistics guidance according to the field information, and when the situations of material distribution errors and the like occur, the logistics special staff is reminded to correct the logistics special staff.
In the above steps, the main steps of the data acquisition processing module a are S01, S02 and S03, the main step of the data storage module B is S04, the main steps of the device feature extraction module C are S11, S12 and S13, the main steps of the information fusion module D are S05, S14 and S15, and the main step of the AR display module E is S16. In the steps, the steps S01-S05 are synchronously performed with the steps S11-S13, and the sequence is not sequential.
GW-2053 and GW-3619 stations for processing raw materials of the piston rod are selected for experimental verification, the experiments are divided into two groups for comparison, namely AR-free visual guidance of the group A and AR visual guidance of the group B, and the results are shown in tables 1 and 2. Under the condition that raw materials are all 5, in the group A experiment, the number of actual supplies on a GW-2053 station is 4, and the actual receiving amount on a GW-3619 station is 3, because a piston rod raw material is damaged in the processing process of the GW-2053, and in the distribution process, a special worker distributes the first piston rod, the station is wrongly fed, the time is occupied in the distribution process, the subsequent processing stations are blocked, and the processing and distribution tasks are not completed in the preset time. And can find in B group's experiment, when the specialist utilized AR equipment to carry out visual guidance, can realize the accurate delivery of piston rod, do not have the error in processing and delivery process.
TABLE 1 visual guidance distribution information of piston rod without AR
TABLE 2 visual guidance of dispensing information for piston rod shells
In the embodiment, production data of two weeks of raw materials, namely the oil cylinder shell and the piston rod, on a CD L production line are selected for comparison, the production process in the 19 th week is operated in a traditional manual distribution mode, and the production process in the 20 th week is performed by adopting the device disclosed by the invention, so that the good product rate of the piston rod is increased to 92% from the original 78%, the blocking rate is reduced by 8%, the misdistribution rate is reduced by 6%, the distribution time is shortened by 1.2h, the good product rate of the oil cylinder shell is increased to 95% from the original 81%, the blocking rate is reduced by 6%, the misdistribution rate is reduced by 8%, and the distribution time is shortened by 1.3 h.
Therefore, the invention can greatly reduce the cost of workshop material distribution, show the processing time, position information and the like of the materials between the stations to material specialists through the AR equipment, and guide the material distribution between the stations of the material specialists. Through table 3 and table 4 contrast, can discover to utilize visual logistics to guide, shorten the delivery time of material, reduce the blockage rate of material on the station, improve the accurate degree of delivery.
TABLE 3 production data of the piston rod and cylinder housing at the 19 th cycle
| Piston rod | Oil cylinder shell | |
| Raw material/ | 100 | 100 |
| Good product/piece | 78 | 81 |
| Is blocked/an | 13 | 9 |
| Mis-delivery/one | 9 | 10 |
| Total transport time/h | 3.5 | 3.8 |
TABLE 4 production data of piston rod and cylinder housing at 20 th cycle
| Piston rod | Oil cylinder shell | |
| Raw material/ | 100 | 100 |
| Good product/piece | 92 | 95 |
| Is blocked/an | 5 | 3 |
| Mis-delivery/one | 3 | 2 |
| Total transport time/h | 2.3 | 2.5 |
The foregoing detailed description of the preferred embodiments of the invention has been presented. It should be understood that numerous modifications and variations could be devised by those skilled in the art in light of the present teachings without departing from the inventive concepts. Therefore, the technical solutions that can be obtained by a person skilled in the art through logic analysis, reasoning or limited experiments based on the prior art according to the concept of the present invention should be within the protection scope determined by the present invention.
Claims (10)
1. An AR-based logistics accurate distribution transparentization auxiliary operation device is characterized by comprising a data acquisition and processing module, a data storage module, an equipment feature extraction module, an information fusion module and an AR display module,
the data acquisition and processing module is used for acquiring and preprocessing logistics distribution information;
the data storage module is used for storing the logistics distribution information;
the equipment feature extraction module is used for constructing a three-dimensional virtual workshop model, and performing rendering and feature extraction on the three-dimensional virtual workshop model to obtain workshop physical equipment features;
the information fusion module is used for drawing real-time production information and logistics distribution information of a workshop into virtual information and superposing the virtual information to a workshop environment based on the characteristics of physical equipment of the workshop;
the AR display module is used for displaying logistics information corresponding to actual workshop equipment in a virtual reality mode.
2. The AR-based logistics precise delivery transparency auxiliary operation device according to claim 1, wherein the logistics delivery information comprises part processing starting time, process processing time, product order information, line side warehouse material accumulation condition of a delivery starting station, target station requirement, basic information of equipment, working procedure of a product, delivery path, equipment positioning information between two stations and positioning information of logistics specialists.
3. The AR-based logistics precision delivery transparency aid as claimed in claim 1 wherein the pre-processing comprises cleaning, integration and standardization.
4. The AR-based logistics precise delivery transparency auxiliary operation device as claimed in claim 1, wherein the data storage module is used for storing data according to data integration characteristics by adopting corresponding storage modes, and the storage modes comprise a relational database, a non-relational database and a file system.
5. The AR-based logistics precise distribution transparentization auxiliary operation device according to claim 1, wherein in the equipment feature extraction module, the feature extraction is specifically as follows: and extracting the features of the interest points and the corner points of the three-dimensional virtual workshop model based on equipment by using an SIFT feature detection algorithm, and describing the features of the collected interest points and the corner points by using an SIFT feature point description algorithm.
6. The AR-based logistics precise distribution transparentization auxiliary work apparatus according to claim 1, wherein the virtual information is represented by a logistics guidance three-dimensional information frame.
7. The AR-based logistics accurate delivery transparentization auxiliary operation device according to claim 1, wherein the overlaying of the virtual information into the workshop environment specifically comprises:
and (3) extracting the physical equipment characteristics of the workshop from the field equipment characteristics and the three-dimensional virtual workshop model by using a matching algorithm to perform characteristic point matching, and superposing after judging to be consistent.
8. The AR-based logistics precise distribution transparentization assisting apparatus according to claim 7, wherein in the matching algorithm, for the matching of high-dimensional features, the euclidean distance is adopted as a feature point to perform similarity determination; and carrying out similarity judgment by adopting a Hamming distance in the low-dimensional binary descriptor vector.
9. The AR-based logistics precise distribution transparentization auxiliary work apparatus according to claim 1, wherein the AR display module is a logistics head-mounted AR device.
10. The AR-based logistics precise delivery transparency aid as claimed in claim 1, wherein the information displayed by the AR display module comprises order number, material type, delivery quantity, delivery start station, delivery target station, processing end time, estimated delivery transportation time and next delivery task.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010131920.2A CN111401154B (en) | 2020-02-29 | 2020-02-29 | AR-based logistics accurate auxiliary operation device for transparent distribution |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010131920.2A CN111401154B (en) | 2020-02-29 | 2020-02-29 | AR-based logistics accurate auxiliary operation device for transparent distribution |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111401154A true CN111401154A (en) | 2020-07-10 |
| CN111401154B CN111401154B (en) | 2023-07-18 |
Family
ID=71430449
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010131920.2A Active CN111401154B (en) | 2020-02-29 | 2020-02-29 | AR-based logistics accurate auxiliary operation device for transparent distribution |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111401154B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113034668A (en) * | 2021-03-01 | 2021-06-25 | 中科数据(青岛)科技信息有限公司 | AR-assisted mechanical simulation operation method and system |
| CN117908498A (en) * | 2024-03-18 | 2024-04-19 | 西安易诺敬业电子科技有限责任公司 | Workshop production monitoring system based on MBS |
| CN118095995A (en) * | 2024-04-26 | 2024-05-28 | 运易通科技有限公司 | Logistics distribution method and system based on AR technology |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107392519A (en) * | 2017-06-07 | 2017-11-24 | 海航创新科技研究有限公司 | Processing method, device and the logistics system of logistics system |
| CN107885336A (en) * | 2017-11-30 | 2018-04-06 | 成都飞机工业（集团）有限责任公司 | Map the virtual workshop model and information fusion display method of the true condition of production |
| CN108090572A (en) * | 2017-12-01 | 2018-05-29 | 大唐国信滨海海上风力发电有限公司 | A kind of marine wind electric field augmented reality system and its control method |
| CN110505464A (en) * | 2019-08-21 | 2019-11-26 | 佳都新太科技股份有限公司 | A kind of number twinned system, method and computer equipment |
| CN110599603A (en) * | 2019-09-20 | 2019-12-20 | 上海大学 | Mechanical equipment visual interaction and equipment state monitoring system and method based on augmented reality |
-
2020
- 2020-02-29 CN CN202010131920.2A patent/CN111401154B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107392519A (en) * | 2017-06-07 | 2017-11-24 | 海航创新科技研究有限公司 | Processing method, device and the logistics system of logistics system |
| CN107885336A (en) * | 2017-11-30 | 2018-04-06 | 成都飞机工业（集团）有限责任公司 | Map the virtual workshop model and information fusion display method of the true condition of production |
| CN108090572A (en) * | 2017-12-01 | 2018-05-29 | 大唐国信滨海海上风力发电有限公司 | A kind of marine wind electric field augmented reality system and its control method |
| CN110505464A (en) * | 2019-08-21 | 2019-11-26 | 佳都新太科技股份有限公司 | A kind of number twinned system, method and computer equipment |
| CN110599603A (en) * | 2019-09-20 | 2019-12-20 | 上海大学 | Mechanical equipment visual interaction and equipment state monitoring system and method based on augmented reality |
Non-Patent Citations (4)
| Title |
|---|
| LUYAO XIA等: ""Development and Application of Workshop Virtual Monitoring System Based on Unity"", 《DDCLS》 * |
| 牟少敏等: "《模式识别与机器学习技术》", 30 June 2019, 北京冶金工业出版社 * |
| 陶飞等: "数字孪生及其应用探索", 《计算机集成制造系统》 * |
| 龚涛: "《摄影测量学》", 30 April 2014, 西南交通大学出版社 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113034668A (en) * | 2021-03-01 | 2021-06-25 | 中科数据(青岛)科技信息有限公司 | AR-assisted mechanical simulation operation method and system |
| CN113034668B (en) * | 2021-03-01 | 2023-04-07 | 中科数据(青岛)科技信息有限公司 | AR-assisted mechanical simulation operation method and system |
| CN117908498A (en) * | 2024-03-18 | 2024-04-19 | 西安易诺敬业电子科技有限责任公司 | Workshop production monitoring system based on MBS |
| CN117908498B (en) * | 2024-03-18 | 2024-05-31 | 西安易诺敬业电子科技有限责任公司 | Workshop production monitoring system based on MBS |
| CN118095995A (en) * | 2024-04-26 | 2024-05-28 | 运易通科技有限公司 | Logistics distribution method and system based on AR technology |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111401154B (en) | 2023-07-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Liu et al. | Probing an intelligent predictive maintenance approach with deep learning and augmented reality for machine tools in IoT-enabled manufacturing | |
| CN111401154B (en) | AR-based logistics accurate auxiliary operation device for transparent distribution | |
| CN110705666A (en) | Artificial intelligence cloud computing display rack goods and label monitoring and goods storage method | |
| WO2020124247A1 (en) | Automated inspection system and associated method for assessing the condition of shipping containers | |
| Sommer et al. | Automatic generation of digital twin based on scanning and object recognition | |
| Browne | Artificial intelligence data-driven internet of things systems, real-time process monitoring, and sustainable industrial value creation in smart networked factories | |
| CN115330734A (en) | Automatic robot repair welding system based on three-dimensional target detection and point cloud defect completion | |
| CN114266514A (en) | Informationized intelligent warehousing management method and system | |
| Szajna et al. | The production quality control process, enhanced with augmented reality glasses and the new generation computing support system | |
| Beckett | Virtual retail algorithms, behavioral predictive analytics, and geospatial mapping technologies in the decentralized metaverse | |
| US11865740B2 (en) | Systematic disposal, classification and dynamic procurement of recyclable resin | |
| Bugaj et al. | Virtual navigation and digital twin simulation tools, internet of things-based decision support and big data computing systems, and image recognition and geolocation data processing algorithms in the virtual economy of the metaverse | |
| Sommer et al. | Automated generation of a digital twin of a manufacturing system by using scan and convolutional neural networks | |
| CN116822159B (en) | Digital twin workshop rapid modeling method for dynamic and static fusion of man-machine environment | |
| Daei Niaki et al. | A Review of The Concept of ‘Supply Chain Digital Twin’in The Era of Industry 4.0 | |
| Nissanka et al. | IoT based automatic storing and retrieval system | |
| Carey | Deep Learning-based Ambient Sound Processing and Visual Imagery Tools, Spatial Computing and Immersive Technologies, and Decision and Control Algorithms in the Metaverse Interactive Environment | |
| Hayward et al. | Machine Learning Image Analysis for Asset Inspection | |
| CN111222941B (en) | Online purchase information classification system based on big data and handheld operation terminal | |
| CN114723713A (en) | Product detection system applying big data to industrial internet | |
| Zhao et al. | Recognition results classification and post-processing methods for painted characters on billet surface | |
| Woodward | Digital Twin Simulation and Modeling Tools, Deep Learning-based Sensing and Technologies, and Computer Vision Algorithms in Big Data-driven Urban Geopolitics | |
| Ji et al. | A Computer Vision-Based System for Metal Sheet Pick Counting. | |
| Ratusny et al. | Characterizing customer ordering Behaviors in semiconductor supply chains with convolutional neural networks | |
| Woodward | Remote big data management and visual imagery tools, multisensor fusion and dynamic routing technologies, and 3D space mapping and object recognition algorithms on blockchain-based metaverse platforms |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |