CN116882070A - Industrial digital twin management system for whole vehicle manufacturing - Google Patents
Industrial digital twin management system for whole vehicle manufacturing Download PDFInfo
- Publication number
- CN116882070A CN116882070A CN202311123643.0A CN202311123643A CN116882070A CN 116882070 A CN116882070 A CN 116882070A CN 202311123643 A CN202311123643 A CN 202311123643A CN 116882070 A CN116882070 A CN 116882070A
- Authority
- CN
- China
- Prior art keywords
- image
- manufacturing
- digital twin
- unit
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000004519 manufacturing process Methods 0.000 title claims abstract description 173
- 230000002159 abnormal effect Effects 0.000 claims abstract description 81
- 230000005856 abnormality Effects 0.000 claims abstract description 38
- 238000003860 storage Methods 0.000 claims abstract description 34
- 238000012545 processing Methods 0.000 claims abstract description 19
- 238000012937 correction Methods 0.000 claims abstract description 8
- 238000003384 imaging method Methods 0.000 claims abstract description 5
- 238000000034 method Methods 0.000 claims description 49
- 238000004891 communication Methods 0.000 claims description 7
- 230000001360 synchronised effect Effects 0.000 claims description 4
- 238000012544 monitoring process Methods 0.000 description 10
- 238000010438 heat treatment Methods 0.000 description 5
- 238000012394 real-time manufacturing Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000001125 extrusion Methods 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/10—Geometric CAD
- G06F30/15—Vehicle, aircraft or watercraft design
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
- G06T17/205—Re-meshing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Geometry (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Mathematical Optimization (AREA)
- Mathematical Analysis (AREA)
- Computational Mathematics (AREA)
- Pure & Applied Mathematics (AREA)
- Computer Hardware Design (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Computer Graphics (AREA)
- Software Systems (AREA)
- Automation & Control Theory (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of image processing, in particular to an industrial digital twin management system for whole car manufacturing. The system comprises: an imaging unit for imaging a mold image of a manufacturing mold and an object image in the target area in the whole vehicle manufacturing process; a creation unit for creating a digital twin model from the mold image and the object image; an acquisition unit configured to acquire variation information of a manufacturing mold and abnormality information of an object; a processing unit for correcting parameters of corresponding mold parts of the manufacturing mold according to the change information of the manufacturing mold and the abnormal information of the object; a storage unit configured to store a second image of the manufacturing mold and a fourth image of the object; the invention solves the problem that the digital twin model can not correct the mould part according to the mould change information and the object abnormality information, and realizes the correction of the mould part aiming at the mould change information and the object abnormality information.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to an industrial digital twin management system for whole car manufacturing.
Background
With the development of digital twin technology and the application in the field of industrial manufacturing, the real-time management of industrial production states by a digital twin model system has become a more common technical means, for example: the invention discloses a China patent CN114609983, which discloses a digital twin monitoring system, comprising a 3D vision model management module, an acquisition module, an intelligent manufacturing module, an archiving database, an environment monitoring module, an access control monitoring module, a monitoring platform video monitoring module, a monitoring system personnel positioning management module and a digital twin management platform; the beneficial effects of the invention are as follows: the remote monitoring system provides an immersive 3D feeling for an operator during remote monitoring, realizes unmanned or unattended remote operation, and achieves the purposes of reducing staff and enhancing efficiency; when a major alarm occurs, judgment is quickly and accurately made and measures are taken, so that loss caused by production stopping is avoided; the manager can clearly know the activity condition of the personnel during the working period, and the personnel can be immediately positioned for timely treatment when the emergency occurs; monitoring production process data of equipment and instruments in real time and fault conditions of the equipment and the instruments; any quality problems and accidents occurring in the future are guaranteed to be traceable to the greatest extent. Also for example: korean patent KR102562828B1 the present invention relates to a heat treatment process system based on a digital twin/virtual physical system, which implements a virtual heat treatment furnace identical to that of a heat treatment line, generates and provides heat treatment production data by performing data analysis and simulation production of key process variables affecting product production in the heat treatment furnace. Therefore, the production efficiency of the product is improved, the reject ratio is reduced, various process variables of the heat treatment production process are measured and monitored in real time through the virtual heat treatment furnace, the manufacturing process is controlled and managed, and various variables generated in the manufacturing process are reduced in real time. Both of the above patents solve the monitoring effect of the production process, but when the production and the manufacture are completed through the mold, the relationship between the abnormality and the deformation cannot be obtained through the digital twin system when the abnormality of the manufactured object is caused by the deformation of the mold, and the purpose of automatically correcting the mold parameters and eliminating the abnormality cannot be achieved by adjusting the parameters of the components corresponding to the abnormality and the deformation.
Disclosure of Invention
In order to better solve the problems, the invention provides an industrial digital twin management system for whole vehicle manufacture, which comprises:
the image pick-up unit is used for picking up a mould image of a manufacturing mould in a target area corresponding to each procedure in the whole car manufacturing process and an object image in the target area; the camera shooting unit is an unmanned aerial vehicle shooting unit;
a creation unit configured to create a digital twin model based on a positional relationship between the manufacturing mold in the target area and the object in the target area from the mold image and the object image, wherein the digital twin model is synchronized with a space and time for manufacturing the mold and the object in the target area corresponding to each process;
the acquisition unit is used for comparing the first image of the manufacturing die shot by the shooting unit with the second image stored in the storage unit to acquire the change information of the manufacturing die, and comparing the third image of the object in the target area shot by the shooting unit with the fourth image stored in the storage unit to acquire the abnormal information of the object;
a processing unit for correcting parameters of corresponding mold parts of the manufacturing mold in the target area according to the change information of the manufacturing mold in the target area and the abnormal information of the object;
the storage unit is used for storing a second image of each procedure corresponding to the manufacturing mould in the normal production process in the whole vehicle manufacturing process and a fourth image of the object when the corresponding procedure is normally executed at the manufacturing mould;
the image pickup unit periodically shoots a manufacturing mould image and an object image in the target area, processes the manufacturing mould image and the object image by the processing unit and updates the manufacturing mould image and the object image into the digital twin model.
As a preferable aspect of the present invention, the image capturing unit is further configured to: dividing the surface space of the object in each target area into grids, wherein the size of a single grid is 1/M of the surface area of the object, M is a positive integer greater than or equal to 2, and the shooting angle is adjusted according to the position of each grid to obtain a third image of each grid;
the acquisition unit is configured to: searching a fourth image corresponding to the third image position in the storage unit for the third image, comparing the fourth image with the fourth image position to obtain a first comparison result, and normally executing a procedure corresponding to the target area by using a grid corresponding to the third image when the first comparison result is smaller than or equal to a first threshold value; and when the first comparison result is larger than a first threshold value, reducing the shooting height of the shooting unit, improving the shooting resolution of the shooting unit, re-acquiring a third image, comparing the re-acquired third image with the corresponding fourth image to acquire a second comparison result, when the second comparison result is continuously larger than a second threshold value for T times, recording the position information and the abnormal state of the abnormal grid as the corresponding grid of the third image, and adding 1 to the number of the abnormal grids, otherwise, normally executing the procedure corresponding to the target area by the corresponding grid of the third image, and recovering the default shooting height and the default shooting resolution of the shooting unit after judging the second comparison result.
As a preferable technical scheme of the present invention, the acquiring unit is configured to acquire a third comparison result based on a first image of the target area captured by the imaging unit, and search the second image with highest similarity to the first image in the storage unit based on the first image, where the first image and the second image include the same reference object;
the processing unit is configured to: and acquiring a first position of the manufacturing mould relative to the reference object in the first image and a second position of the manufacturing mould relative to the reference object in the second image through the third comparison result, adjusting position information of a fourth image in the storage unit based on the first difference value when the difference value between the first position and the second position is larger than the first difference value, shooting a third image of each grid through the shooting unit again, recalculating the number of abnormal grids and the positions of the abnormal grids based on the fourth image with the position information adjusted, and updating the first image and the third image acquired again into the digital twin model.
As a preferred embodiment of the present invention, the processing unit is configured to: when the difference value between the first position and the second position is smaller than or equal to the first difference value, when the third comparison result is larger than or equal to a third threshold value, when the number of abnormal grids is equal to 0, the first image is acquired again, the second image is searched again to acquire the third comparison result, when the number of abnormal grids is larger than 0, a stopping instruction is sent to a control unit of the manufacturing die through a communication unit, the change position of the first image relative to the second image is acquired according to the third comparison result, the first image is updated into the digital twin model, the change position is marked, and meanwhile the third image is updated into the digital twin model, and the position of the abnormal grid is marked into the digital twin model;
when the difference value between the first position and the second position is smaller than or equal to the first difference value, and when the third comparison result is smaller than a third threshold value, under the condition that the number of abnormal grids is 0, directly updating the second image into the digital twin model, and acquiring the change position of the first image relative to the second image according to the third comparison result to identify in the digital twin model; when the number of the abnormal grids is greater than 0 and the number of the abnormal grids is smaller than or equal to a fourth threshold value, acquiring a change position of the first image relative to the second image according to the third comparison result, updating the first image into the digital twin model, marking the change position, and simultaneously updating the third image into the digital twin model and marking the position of the abnormal grids into the digital twin model;
and when the number of abnormal grids is greater than 0 and the number of abnormal grids is greater than a fourth threshold, determining a changed object type by searching a second image with highest similarity with the first image in the storage unit, searching the fourth image again through the object type and the third image, and determining whether grids in the third image are abnormal grids and the number of abnormal grids again based on the third image and the fourth image which is searched again.
As a preferred embodiment of the present invention, the processing unit is configured to: determining the position of the change information in the manufacturing mold and a mold component corresponding to the change information based on the change information of the manufacturing mold, determining the position of the abnormality information in the object according to the abnormality information of the object, carrying out reverse reasoning on the abnormality cause of the object according to the position of the change information, the corresponding relation between the change and the position of the abnormality of the object, the change information, the content of the abnormality information and the mold component information, generating correction information based on the abnormality cause and the adjustment rule of the component information corresponding to the change information, acquiring the adjustment parameters of the mold component according to the correction information, and sending the adjustment parameters to a control unit corresponding to the mold component through a communication unit.
Compared with the prior art, the invention has the following beneficial effects:
the digital twin model is created according to the position relation between the model image in the target area and the object image corresponding to the vehicle in the manufacturing process, and the real-time manufacturing state of the corresponding vehicle in each target area and the working state of the manufacturing die in the target area can be known in real time through the digital twin model, so that a manager can conveniently master and manage the manufacturing state of each production process in the vehicle manufacturing process. The method comprises the steps of obtaining a first image of a manufacturing mould, searching a second image with highest similarity with the first image in a storage unit based on the first image, comparing the first image with the second image, obtaining the change information of the manufacturing mould according to the comparison result, judging whether the manufacturing mould is in a normal state or not according to the change information of the manufacturing mould, searching a fourth image closest to the third image in the target area in the storage unit according to the third image of the object in the target area, comparing the fourth image closest to the third image in the storage unit, obtaining the change information of the object deviating from the normal state in the process of executing the manufacturing mould, and determining parameter adjustment of a mould component corresponding to the manufacturing mould according to the change information of the manufacturing mould and the change information of the object in the process of executing the process, so that the change information detected in a virtual digital twin management system is used for guiding and correcting the adjustment of the mould parameters in a real manufacturing environment, and is not only the real manufacturing environment. The change type of the manufacturing mould is determined according to the change information of the manufacturing mould and the change information of the object, and the adjustment of the corresponding component parameters of the manufacturing mould is determined according to the change type, so that the quality problem or deviation caused by the manufacturing mould when the corresponding process is executed is corrected.
Drawings
FIG. 1 is a block diagram of an industrial digital twin management system for whole vehicle manufacturing.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
The invention provides an industrial digital twin management system for whole vehicle manufacture, as shown in fig. 1, the system comprises:
the image pick-up unit is used for picking up a mould image of a manufacturing mould in a target area corresponding to each procedure in the whole car manufacturing process and an object image in the target area; the camera shooting unit is an unmanned aerial vehicle shooting unit;
specifically, the image capturing unit captures a mold image of the manufacturing mold in the target area to obtain a real-time image of the manufacturing mold, the mold image may be an orthogonal image, and the process and state of the object in the target area may be obtained by capturing a mold object image in the target area, wherein the object is a vehicle performing a processing process in the target area, a scene image and an object image are provided for creating a digital twin model by the mold image and the object image, and the image capturing unit further performs image preprocessing on the mold image and the object image to improve the image quality, thereby improving the accuracy of the digital twin model.
A creation unit configured to create a digital twin model based on a positional relationship between the mold in the target area and the object in the target area from the mold image and the object image, wherein the digital twin model is synchronized with a space and time in which the mold and the object are manufactured in the target area corresponding to each process;
specifically, the digital twin model is created by the position relationship between the mold image in the target area and the object image corresponding to the vehicle in the manufacturing process, and because the digital twin model is synchronous in time and space with the target area, the real-time manufacturing state of the corresponding vehicle in each target area and the working state of the manufacturing mold in the target area can be known in real time through the digital twin model, so that a manager can conveniently grasp and manage the manufacturing state of each production process in the vehicle manufacturing process.
The acquisition unit is used for comparing the first image of the manufacturing die shot by the shooting unit with the second image stored in the storage unit to acquire the change information of the manufacturing die, and comparing the third image of the object in the target area shot by the shooting unit with the fourth image stored in the storage unit to acquire the abnormal information of the object;
specifically, a first image of the manufacturing mold is photographed by a photographing unit, the second image having the highest similarity with the first image is searched in the storage unit based on the first image, and the change information of the manufacturing mold is obtained according to the comparison result by comparing the first image with the second image, wherein the change information of the manufacturing mold is the change of the manufacturing mold relative to the normal working state, and whether the manufacturing mold is in the normal state can be judged according to the change information of the manufacturing mold, and meanwhile, the fourth image corresponding to the position of the third image is searched in the storage unit according to the third image of the object in the target area, so that the abnormal information of the object relative to the normal state in the process of executing the manufacturing mold is obtained.
A processing unit for correcting parameters of corresponding mold parts of the manufacturing mold in the target area according to the change information of the manufacturing mold in the target area and the abnormal information of the object;
specifically, the change type of the manufacturing mold is determined according to the change information of the manufacturing mold in the target area and the abnormal information of the object, and the adjustment of the parameters of the corresponding part of the manufacturing mold is determined according to the change type, so that the parameters of the manufacturing mold when the corresponding process is executed are corrected, and the quality problem or deviation caused by the parameters of the manufacturing mold is avoided.
The storage unit is used for storing a second image of each procedure corresponding to the manufacturing mould in the normal production process in the whole car manufacturing process and a fourth image of an object when the corresponding procedure is normally executed at the manufacturing mould;
specifically, the second image of the manufacturing mold and the fourth image of the object, which are stored in the storage unit when the process is normally executed, provide a reference basis for judging the mold information change and the object information change body when the process is actually executed by the manufacturing mold, wherein the second image and the fourth image also comprise corresponding vehicle object model information.
The image pickup unit periodically shoots a manufacturing mould image and an object image in the target area, processes the manufacturing mould image and the object image by the processing unit and updates the manufacturing mould image and the object image into the digital twin model.
Specifically, the manufacturing mold image and the object image in the target area are periodically shot through the shooting unit, and after being processed through the processing unit, the manufacturing mold image and the object image are updated into the digital twin model when the mold change information and the object abnormality information do not influence the normal execution of the working procedure, and when the manufacturing model change information influences the normal production of the vehicle object, the parameters of the corresponding mold parts are adjusted or the abnormal prompt is carried out through the digital twin model, so that the real production process is corrected through the virtual digital twin model.
Further, the image capturing unit is further configured to: dividing the surface space of the object in each target area into grids, wherein the size of a single grid is 1/M of the surface area of the object, M is a positive integer greater than or equal to 2, and a shooting angle is adjusted according to the position of each grid to obtain a third image of each grid;
specifically, the surface space of the object is subjected to grid division, and 1/M of the surface area of the object is taken as the minimum grid unit, so that the image capturing unit can be used for capturing the grid image in a refined manner, and as the surface of the vehicle is not a plane, the image capturing angle of the image capturing unit is adjusted according to the position and angle of each grid, so that a grid image which can be more accurate is obtained, and a good foundation is laid for further judging whether the object correctly executes the process at each grid.
The acquisition unit is configured to: searching a fourth image corresponding to the third image position in the storage unit for the third image, comparing the fourth image with the fourth image position to obtain a first comparison result, and normally executing a procedure corresponding to the target area by using a grid corresponding to the third image when the first comparison result is smaller than or equal to a first threshold value; and when the first comparison result is larger than a first threshold value, reducing the shooting height of the shooting unit, improving the shooting resolution of the shooting unit, re-acquiring a third image, comparing the re-acquired third image with the corresponding fourth image to acquire a second comparison result, when the second comparison result is continuously larger than a second threshold value for T times, recording the position information and the abnormal state of the abnormal grid as the corresponding grid of the third image, and adding 1 to the number of the abnormal grids, otherwise, normally executing the procedure corresponding to the target area by the corresponding grid of the third image, and recovering the default shooting height and the default shooting resolution of the shooting unit after judging the second comparison result.
Specifically, by comparing a third image corresponding to each grid with a fourth image corresponding to the third image position stored in the storage unit, wherein the third image and the fourth image are both images captured after the process corresponding to the target area is performed, and determining a difference between the third image corresponding to each grid and the fourth image for normally performing the process according to a first comparison result, when the difference between the third image and the fourth image is less than or equal to a first threshold value, the process corresponding to the target area is accurately performed by the third image corresponding to the grid due to the small difference; when the difference between the third image and the fourth image is larger than the first threshold, the difference is larger, which may be caused by unclear shooting of the third image, so that the shooting height and shooting definition of the shooting unit are reduced, more clear third images are acquired by improving shooting resolution, when the second comparison result of the third image and the fourth image acquired for T times continuously is larger than the second threshold, abnormal conditions are judged to occur when the corresponding procedure of the target area is executed, the positions of abnormal grids corresponding to the third image are recorded, the number of abnormal grids is accumulated, so that the abnormal types are further judged, and after the judgment of the second comparison result is finished, the shooting height and shooting definition of the shooting unit are recovered, more grid images are conveniently segmented from single images shot by the shooting unit, and the grid image recognition efficiency is improved.
Further, the acquiring unit is configured to acquire a third comparison result based on a first image of the target area captured by the imaging unit, and search the second image with the highest similarity to the first image in the storage unit based on the first image, where the first image and the second image include the same reference object;
specifically, a first image of a mold is manufactured in a target area shot by a shooting unit, and the second image with the highest similarity with the first image is searched in the storage unit, so that the first image and the second image are compared in the same state to obtain image changes, and the image acquisition is periodically and continuously acquired, so that the change between adjacent images is relatively small, and more accurate mold changes can be obtained through the technical scheme.
The processing unit is configured to: acquiring a first position of a manufacturing die relative to the reference object in the first image and a second position of the manufacturing die relative to the reference object in the second image through the third comparison result, adjusting position information of a fourth image in the storage unit based on the first difference value when the difference value between the first position and the second position is larger than the first difference value, shooting a third image of each grid through the shooting unit again, recalculating the number of abnormal grids and the positions of the abnormal grids based on the fourth image with the position information adjusted, and updating the first image and the third image acquired again into the digital twin model;
specifically, by comparing the first position of the manufacturing mold in the first image with the second position of the manufacturing mold in the second image with respect to the reference object, it is possible to determine whether or not the position of the manufacturing mold in the first image with respect to the manufacturing mold in the second image has changed, and when the position of the manufacturing mold has changed significantly due to the difference between the positions of the manufacturing molds being larger than the first difference, that is, when the position of the manufacturing mold has changed significantly, the position of the vehicle changes and the position of each grid changes when the corresponding process is completed, and if the position information of the fourth image in the storage unit is not adjusted, the position information of the third image and the fourth image does not correspond, so that it is not possible to accurately determine whether or not each grid is abnormal, and by adjusting the position of the fourth image based on the first difference, the position information of the third image and the fourth image corresponds, and further the correct information whether or not each grid is abnormal is obtained.
When the difference value between the first position and the second position is smaller than or equal to the first difference value, when the third comparison result is larger than or equal to a third threshold value, when the number of abnormal grids is equal to 0, the first image is acquired again, the second image is searched again to acquire the third comparison result, when the number of abnormal grids is larger than 0, a stopping instruction is sent to a control unit of the manufacturing die through a communication unit, the change position of the first image relative to the second image is acquired according to the third comparison result, the first image is updated into the digital twin model, the change position is marked, and meanwhile the third image is updated into the digital twin model, and the position of the abnormal grid is marked into the digital twin model;
specifically, when the third comparison result is greater than or equal to the third threshold, that is, the manufacturing mold is greatly deformed, in this case, the number of abnormal grids may be large, if the number of abnormal grids is 0, the first image may be inaccurate, the more accurate first image is obtained again, the second image is searched again, the third comparison result is obtained, and the judgment is again performed, when the number of abnormal grids is greater than 0, it is indicated that the deformation of the manufacturing mold has caused an abnormality, and the first image may be accurate, so that in order to avoid causing greater deformation to the manufacturing mold, a stop instruction is sent to the control unit of the manufacturing mold through the communication unit, and at the same time, deformation information and abnormality information are updated to the digital twin model, so that management personnel is prompted to perform maintenance.
When the third comparison result is smaller than a third threshold value, under the condition that the number of abnormal grids is 0, directly updating the second image into the digital twin model, and acquiring a change position of the first image relative to the second image according to the third comparison result to be identified in the digital twin model; when the number of the abnormal grids is greater than 0 and the number of the abnormal grids is smaller than or equal to a fourth threshold value, acquiring a change position of the first image relative to the second image according to the third comparison result, updating the first image into the digital twin model, marking the change position, and simultaneously updating the third image into the digital twin model and marking the position of the abnormal grids into the digital twin model;
and when the number of abnormal grids is greater than 0 and the number of abnormal grids is greater than a fourth threshold, determining a changed object type by searching a second image with highest similarity with the first image in the storage unit, searching the fourth image again through the object type and the third image, and determining whether grids in the third image are abnormal grids and the number of abnormal grids again based on the third image and the fourth image which is searched again.
Specifically, when the difference in the position change of the manufacturing mold is less than or equal to the first difference, that is, when the position of the manufacturing mold is not substantially changed, it is necessary to further determine whether the manufacturing mold is deformed, that is, when the third comparison result is less than the third threshold, that is, when the manufacturing mold is deformed, it is further determined whether the deformation affects the normal operation of the process according to whether the vehicle object is abnormal, that is, when the number of abnormal grids is 0, that is, the vehicle object can be normal, the deformation of the manufacturing mold does not affect the normal operation of the process, so that only the third image needs to be updated into the digital twin model, and the deformation is identified, which attracts attention of a manager, and at the same time, the deformation is reflected to have no relationship with the abnormal grids due to the identification of the non-abnormal grids; on the contrary, when the number of the abnormal grids is greater than 0 and the number of the abnormal grids is smaller than or equal to the fourth threshold value, namely the deformation possibly causes the occurrence of the abnormality, so that the position of the abnormal grids is updated into the digital twin model and the position of the abnormal grids is updated into the digital twin model, a manager can intuitively see the position of the manufacturing mould and the position of the abnormality of a vehicle queue, and a foundation is laid for further correcting the parameters of the manufacturing mould to eliminate the grid abnormality; when the number of abnormal meshes is greater than 0 and the number of abnormal meshes is greater than a fourth threshold, the number of abnormal meshes causing abnormality of the vehicle object is relatively small, but at this time, there is a possibility that the parameters of the manufacturing mold may change, and when the manufactured vehicle type changes, the second image having the highest similarity with the first image can be found from the first image of the manufacturing mold after the parameters are modified, and therefore, when the third comparison result is smaller than the third threshold and when the number of abnormal meshes is greater than 0 and the number of abnormal meshes is greater than the fourth threshold, there is a possibility that the manufactured vehicle type changes, and therefore, the vehicle type being manufactured is acquired by the vehicle type corresponding to the second image, and the abnormal meshes are acquired again by searching the fourth image corresponding to the third image based on the vehicle type.
Further, the processing unit is configured to: determining the position of the change information in the manufacturing die and the die component corresponding to the change information based on the change information of the manufacturing die, determining the position of the abnormality information in the object according to the abnormality information of the object, carrying out reverse reasoning on the abnormality cause of the object according to the position of the change information, the corresponding relation between the change and the position of the abnormality of the object and the change information, the content of the abnormality information and the die component information, generating correction information based on the abnormality cause and the adjustment rule of the component information corresponding to the change information, acquiring the adjustment parameters of the die component according to the correction information, sending the adjustment parameters to the control unit corresponding to the die component through the communication unit, and carrying out correction on the abnormality position again.
Specifically, the position of the change in the manufacturing mold, that is, the deformation position, the position of the abnormality information, and the correspondence relationship between the two are determined, for example: when the positioning slide block in the mold component is deformed or worn due to extrusion, a certain abnormality is generated in the corresponding position of the vehicle object corresponding to the position of the positioning slide block, so that whether the abnormality in the corresponding position of the vehicle object is caused by the deformation or the wear or not can be inferred based on the deformation or the wear of the positioning slide block, when the abnormality is caused by the deformation or the wear, the positioning slide block is adjusted to the normal position through the adjustment rule of the positioning slide block, and the adjusted positioning slide block is used for correcting the abnormal position corresponding to the vehicle object again.
In summary, the digital twin model is created by the position relationship between the model image in the target area and the object image corresponding to the vehicle in the manufacturing process, and the real-time manufacturing state of the corresponding vehicle in each target area and the working state of the manufacturing model in the target area can be known in real time through the digital twin model, so that a manager can conveniently master and manage the manufacturing state of each manufacturing process in the vehicle manufacturing process. The method comprises the steps of obtaining a first image of a manufacturing mould, searching a second image with highest similarity with the first image in a storage unit based on the first image, comparing the first image with the second image, obtaining the change information of the manufacturing mould according to the comparison result, judging whether the manufacturing mould is in a normal state or not according to the change information of the manufacturing mould, searching a fourth image closest to the third image in the target area in the storage unit according to the third image of the object in the target area, comparing the fourth image closest to the third image in the storage unit, obtaining the change information of the object deviating from the normal state in the process of executing the manufacturing mould, and determining parameter adjustment of a mould component corresponding to the manufacturing mould according to the change information of the manufacturing mould and the change information of the object in the process of executing the process, so that the change information detected in a virtual digital twin management system is used for guiding and correcting the adjustment of the mould parameters in a real manufacturing environment, and is not only the real manufacturing environment. The change type of the manufacturing mould is determined according to the change information of the manufacturing mould and the change information of the object, and the adjustment of the corresponding component parameters of the manufacturing mould is determined according to the change type, so that the quality problem or deviation caused by the manufacturing mould when the corresponding process is executed is corrected.
The technical features of the foregoing embodiments may be arbitrarily combined, and for brevity, all of the possible combinations of the technical features of the foregoing embodiments are not described, however, they should be considered as the scope of the disclosure as long as there is no contradiction between the combinations of the technical features.
The foregoing examples illustrate only a few embodiments of the invention and are described in detail herein without thereby limiting the scope of the invention. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention. Accordingly, the scope of protection of the present invention is to be determined by the appended claims.
The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather is intended to cover all modifications, equivalents, and alternatives falling within the spirit and principles of the invention.
Claims (5)
1. An industrial digital twin management system for whole vehicle manufacturing, the system comprising:
the image pick-up unit is used for picking up a mould image of a manufacturing mould in a target area corresponding to each procedure in the whole car manufacturing process and an object image in the target area; the camera shooting unit is an unmanned aerial vehicle shooting unit;
a creation unit configured to create a digital twin model based on a positional relationship between the manufacturing mold in the target area and the object in the target area from the mold image and the object image, wherein the digital twin model is synchronized with a space and time for manufacturing the mold and the object in the target area corresponding to each process;
the acquisition unit is used for comparing the first image of the manufacturing die shot by the shooting unit with the second image stored in the storage unit to acquire the change information of the manufacturing die, and comparing the third image of the object in the target area shot by the shooting unit with the fourth image stored in the storage unit to acquire the abnormal information of the object;
a processing unit configured to correct a mold component parameter corresponding to variation information of the manufacturing mold in the target area according to the variation information of the manufacturing mold in the target area and the abnormality information of the object;
the storage unit is used for storing a second image of each procedure corresponding to the manufacturing mould in the normal production process in the whole vehicle manufacturing process and a fourth image of the object when the corresponding procedure is normally executed at the manufacturing mould;
the image pickup unit periodically shoots a manufacturing mould image and an object image in the target area, processes the manufacturing mould image and the object image by the processing unit and updates the manufacturing mould image and the object image into the digital twin model.
2. The industrial digital twin management system for whole vehicle manufacturing according to claim 1, wherein the camera unit is further configured to: dividing the surface space of an object in each target area into grids, wherein the size of a single grid is 1/M of the surface area of the object, M is a positive integer greater than or equal to 2, and the shooting angle is adjusted according to the position of each grid to obtain a third image of each grid;
the acquisition unit is configured to: searching a fourth image corresponding to the third image position in the storage unit for the third image, comparing the fourth image with the fourth image position to obtain a first comparison result, and normally executing a procedure corresponding to the target area by using a grid corresponding to the third image when the first comparison result is smaller than or equal to a first threshold value; and when the first comparison result is larger than a first threshold value, reducing the shooting height of the shooting unit, improving the shooting resolution of the shooting unit, re-acquiring a third image, comparing the re-acquired third image with the corresponding fourth image to acquire a second comparison result, when the second comparison result is continuously larger than a second threshold value for T times, recording the position information and the abnormal state of the abnormal grid as the corresponding grid of the third image, and adding 1 to the number of the abnormal grids, otherwise, normally executing the procedure corresponding to the target area by the corresponding grid of the third image, and recovering the default shooting height and the default shooting resolution of the shooting unit after judging the second comparison result.
3. The industrial digital twin management system for whole vehicle manufacturing according to claim 2, wherein the acquiring unit is configured to search the second image with highest similarity to the first image in the storage unit based on the first image of the target area captured by the imaging unit, and compare the first image with the second image to obtain a third comparison result, where the first image and the second image include the same reference object;
the processing unit is configured to: and acquiring a first position of the manufacturing mould relative to the reference object in the first image and a second position of the manufacturing mould relative to the reference object in the second image through the third comparison result, adjusting position information of a fourth image in the storage unit based on the first difference value when the difference value between the first position and the second position is larger than the first difference value, shooting a third image of each grid through the shooting unit again, recalculating the number of abnormal grids and the positions of the abnormal grids based on the fourth image with the position information adjusted, and updating the first image and the third image acquired again into the digital twin model.
4. An industrial digital twin management system for whole vehicle manufacturing according to claim 3,
the processing unit is configured to: when the difference value between the first position and the second position is smaller than or equal to the first difference value, when the third comparison result is larger than or equal to a third threshold value, when the number of abnormal grids is equal to 0, the first image is acquired again, the second image is searched again to acquire the third comparison result, when the number of abnormal grids is larger than 0, a stopping instruction is sent to a control unit of the manufacturing die through a communication unit, the change position of the first image relative to the second image is acquired according to the third comparison result, the first image is updated into the digital twin model, the change position is marked, and meanwhile the third image is updated into the digital twin model, and the position of the abnormal grid is marked into the digital twin model;
when the difference value between the first position and the second position is smaller than or equal to the first difference value, and when the third comparison result is smaller than a third threshold value, under the condition that the number of abnormal grids is 0, directly updating the second image into the digital twin model, and acquiring the change position of the first image relative to the second image according to the third comparison result to identify in the digital twin model; when the number of the abnormal grids is greater than 0 and the number of the abnormal grids is smaller than or equal to a fourth threshold value, acquiring a change position of the first image relative to the second image according to the third comparison result, updating the first image into the digital twin model, marking the change position, and simultaneously updating the third image into the digital twin model and marking the position of the abnormal grids into the digital twin model;
and when the number of abnormal grids is greater than 0 and the number of abnormal grids is greater than a fourth threshold, determining a changed object type by searching a second image with highest similarity with the first image in the storage unit, searching the fourth image again through the object type and the third image, and determining whether grids in the third image are abnormal grids and the number of abnormal grids again based on the third image and the fourth image which is searched again.
5. The industrial digital twin management system for whole vehicle manufacturing according to claim 1, wherein the processing unit is configured to: determining the position of the change information in the manufacturing mold and a mold component corresponding to the change information based on the change information of the manufacturing mold, determining the position of the abnormality information in the object according to the abnormality information of the object, carrying out reverse reasoning on the abnormality cause of the object according to the position of the change information, the corresponding relation between the change and the position of the abnormality of the object, the change information, the content of the abnormality information and the mold component information, generating correction information based on the abnormality cause and the adjustment rule of the component information corresponding to the change information, acquiring the adjustment parameters of the mold component according to the correction information, and sending the adjustment parameters to a control unit corresponding to the mold component through a communication unit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311123643.0A CN116882070B (en) | 2023-09-01 | 2023-09-01 | Industrial digital twin management system for whole vehicle manufacturing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311123643.0A CN116882070B (en) | 2023-09-01 | 2023-09-01 | Industrial digital twin management system for whole vehicle manufacturing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116882070A true CN116882070A (en) | 2023-10-13 |
| CN116882070B CN116882070B (en) | 2023-11-14 |
Family
ID=88257135
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311123643.0A Active CN116882070B (en) | 2023-09-01 | 2023-09-01 | Industrial digital twin management system for whole vehicle manufacturing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116882070B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2012091374A (en) * | 2010-10-26 | 2012-05-17 | Sigumakkusu Kk | Apparatus for monitoring injection molding machine |
| JP2013206267A (en) * | 2012-03-29 | 2013-10-07 | Jsol Corp | Information processing system, server device, terminal equipment, information processor, information processing method and program |
| CN108919760A (en) * | 2018-07-05 | 2018-11-30 | 长安大学 | A kind of intelligent workshop autonomy production process dynamic linkage control method twin based on number |
| JP2019087050A (en) * | 2017-11-07 | 2019-06-06 | 株式会社ベイシスコンサルティング | Structure maintenance management job support system |
| JP2020135660A (en) * | 2019-02-22 | 2020-08-31 | パナソニックＩｐマネジメント株式会社 | Information management device and information management method |
| CN113094867A (en) * | 2021-03-01 | 2021-07-09 | 广州铁路职业技术学院（广州铁路机械学校） | Train compartment noise environment modeling method based on digital twins |
| US20230125890A1 (en) * | 2020-03-27 | 2023-04-27 | Hitachi Kokusai Electric Inc. | Image analysis system, image analysis method, and image analysis program |
| CN116128320A (en) * | 2023-01-04 | 2023-05-16 | 杭州有泰信息技术有限公司 | Visual control method and platform for power transmission and transformation of power grid |
| KR102547748B1 (en) * | 2022-11-04 | 2023-06-28 | 주식회사 스탠스 | Server, method and computer program for managing digital twin model |
| US20230230312A1 (en) * | 2022-01-20 | 2023-07-20 | Qingdao university of technology | Digital twin modeling method and system for assembling a robotic teleoperation environment |
-
2023
- 2023-09-01 CN CN202311123643.0A patent/CN116882070B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2012091374A (en) * | 2010-10-26 | 2012-05-17 | Sigumakkusu Kk | Apparatus for monitoring injection molding machine |
| JP2013206267A (en) * | 2012-03-29 | 2013-10-07 | Jsol Corp | Information processing system, server device, terminal equipment, information processor, information processing method and program |
| JP2019087050A (en) * | 2017-11-07 | 2019-06-06 | 株式会社ベイシスコンサルティング | Structure maintenance management job support system |
| CN108919760A (en) * | 2018-07-05 | 2018-11-30 | 长安大学 | A kind of intelligent workshop autonomy production process dynamic linkage control method twin based on number |
| JP2020135660A (en) * | 2019-02-22 | 2020-08-31 | パナソニックＩｐマネジメント株式会社 | Information management device and information management method |
| US20230125890A1 (en) * | 2020-03-27 | 2023-04-27 | Hitachi Kokusai Electric Inc. | Image analysis system, image analysis method, and image analysis program |
| CN113094867A (en) * | 2021-03-01 | 2021-07-09 | 广州铁路职业技术学院（广州铁路机械学校） | Train compartment noise environment modeling method based on digital twins |
| US20230230312A1 (en) * | 2022-01-20 | 2023-07-20 | Qingdao university of technology | Digital twin modeling method and system for assembling a robotic teleoperation environment |
| KR102547748B1 (en) * | 2022-11-04 | 2023-06-28 | 주식회사 스탠스 | Server, method and computer program for managing digital twin model |
| CN116128320A (en) * | 2023-01-04 | 2023-05-16 | 杭州有泰信息技术有限公司 | Visual control method and platform for power transmission and transformation of power grid |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116882070B (en) | 2023-11-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11688056B2 (en) | Method for predicting defects in assembly units | |
| CN112837302B (en) | Method and device for monitoring state of die, industrial personal computer, storage medium and system | |
| CN115631449B (en) | Intelligent video identification management method and system | |
| GB2428930A (en) | Generating a depth map using both focal and zoom data | |
| CN105979143B (en) | Method and device for adjusting shooting parameters of dome camera | |
| CN113723325B (en) | Frock defect detecting system of prefabricated component of assembled | |
| CN110648319B (en) | Equipment image acquisition and diagnosis system and method based on double cameras | |
| CN102109841A (en) | Monitoring system of a dynamical arrangement of pieces taking part in a process related to a manufacturing executing system | |
| CN116428984A (en) | Hardware mould stamping processing intelligent detection system | |
| US20220375056A1 (en) | Method for predicting defects in assembly units | |
| CN116882070B (en) | Industrial digital twin management system for whole vehicle manufacturing | |
| CN115880296A (en) | Machine vision-based prefabricated part quality detection method and device | |
| CN115861173A (en) | Automatic detection system and method for accuracy of optical splitter resources based on digital twin and AI | |
| CN115937218B (en) | Surface defect detection method and system for cold stamping part | |
| CN116958264A (en) | Bolt hole positioning and pose estimation method based on three-dimensional vision | |
| CN112016354B (en) | Method for detecting loading state of grain tank of grain transporting vehicle based on visual identification | |
| WO2017218718A1 (en) | Method and system for articulation of a visual inspection device | |
| CN113723841B (en) | On-line detection method for tool missing in assembled prefabricated part | |
| CN112380704B (en) | Digital twin model correction method and system based on machine vision | |
| CN115578706A (en) | Lane line correction method and device, vehicle control unit and readable storage medium | |
| CN115937773A (en) | Intelligent image recognition system based on ai recognition | |
| CN116405532B (en) | Industrial control and automation method and device based on Internet of things and electronic equipment | |
| CN116760955B (en) | Information tracking system for seamless steel pipe production | |
| CN117032147B (en) | Production line machine tool digital regulation system and method based on artificial intelligence | |
| CN113689467B (en) | Feature point optimization method and system suitable for plane tracking |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |