WO2024016283A1 - 组装辅助方法及装置 - Google Patents

组装辅助方法及装置 Download PDF

Info

Publication number
WO2024016283A1
WO2024016283A1 PCT/CN2022/107191 CN2022107191W WO2024016283A1 WO 2024016283 A1 WO2024016283 A1 WO 2024016283A1 CN 2022107191 W CN2022107191 W CN 2022107191W WO 2024016283 A1 WO2024016283 A1 WO 2024016283A1
Authority
WO
WIPO (PCT)
Prior art keywords
assembly
physical components
assembly step
product
physical
Prior art date
Application number
PCT/CN2022/107191
Other languages
English (en)
French (fr)
Inventor
王子健
赵旭
范顺杰
Original Assignee
西门子股份公司
西门子（中国）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 西门子股份公司, 西门子（中国）有限公司 filed Critical 西门子股份公司
Priority to PCT/CN2022/107191 priority Critical patent/WO2024016283A1/zh
Publication of WO2024016283A1 publication Critical patent/WO2024016283A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F30/00—Computer-aided design [CAD]
    • G06F30/20—Design optimisation, verification or simulation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling

Definitions

  • the present invention mainly relates to the field of industrial automation, and in particular, to an assembly assistance method and device.
  • An assembly assistance system is based on augmented reality, which identifies the parts of the current assembly step by scanning the barcode attached to the part, and provides corresponding assembly instructions to the worker. However, it is necessary to attach the barcode to the part and prepare the assembly instructions manually.
  • the production efficiency is still not high; another assembly assistance system is targeted at specific application scenarios, such as tightening screws. Augmented reality is used to indicate the position marks of the screws, so workers can easily tighten the screws. However, it can only be targeted at specific application scenarios. It cannot be applied to other application scenarios.
  • the present invention provides an assembly assistance method and device to instruct workers to complete assembly tasks, improve production efficiency, and is suitable for various application scenarios.
  • the assembly assistance method includes: acquiring data from the product life cycle management system to construct an assembly information database, and based on the assembly information database and assembly tasks obtained from the manufacturing execution system. Determine the assembly process of the product to be assembled; scan all physical components of the product to be assembled to obtain a reconstructed model of each physical component; match the assembly process with the reconstructed model of the physical component, determine and indicate the steps involved in the assembly The physical components and the assembly method of the front and rear physical components in the assembly step until the assembly process of the product to be assembled is completed.
  • the assembly process of the product to be assembled is determined from the data in the product life cycle management system and manufacturing execution system, the physical parts are scanned to obtain the reconstructed model, and the reconstructed model and the assembly process are matched in real time to instruct the user to complete the assembly according to the assembly process. There is no need to attach marks to physical parts, which improves production efficiency and is widely applicable to various application scenarios.
  • indicating the physical components involved in the assembly step includes: projecting light through a projector onto the physical components involved in the assembly step.
  • the physical components involved in the assembly steps can be intuitively indicated to the user by projecting light through a projector.
  • the assembly method of determining and indicating the physical components involved in the assembly step and the preceding and following physical components in the assembly step includes: providing an operation button, and in response to the user pressing the operation button, indicating the entities involved in the next assembly step. Parts and methods of assembling the front and rear physical parts in the next assembly step. To this end, by providing an operation button to indicate the next set of assembly steps in response to the user pressing the operation button, the assembly process can be made more flexible and convenient.
  • the assembly method for determining and indicating the physical components involved in the assembly step and the front and rear physical components in the assembly step includes: providing a human-machine interface device, and displaying the front and rear entities in the assembly step on the human-machine interface device. How to assemble the parts. To this end, by displaying the assembly method of the front and rear physical components in the assembly step on the human-machine interface device, the assembly method in the assembly step can be intuitively instructed to the user.
  • the method further includes: after the current assembly step is completed, scanning the current physical component, performing quality verification on the current physical component and the corresponding digital model in the assembly information database, and entering the next step when the quality verification passes.
  • One assembly step the assembly quality of components can be automatically verified in real time, improving production efficiency.
  • the present invention also proposes an assembly auxiliary device.
  • the assembly auxiliary device includes: an acquisition module, which acquires data from the product life cycle management system to construct an assembly information database, and determines the assembly task based on the assembly information database and the assembly task acquired from the manufacturing execution system.
  • the assembly process of the product to be assembled the modeling module, scans all the physical components of the product to be assembled, and obtains a reconstructed model of each of the physical components; the instruction module, matches the assembly process with the reconstructed model of the physical component, and determines And indicate the physical components involved in the assembly step and the assembly method of the front and rear physical components in the assembly step until the end of the assembly process of the product to be assembled.
  • indicating the physical components involved in the assembly step includes: projecting light through a projector onto the physical components involved in the assembly step.
  • the assembly method of determining and indicating the physical components involved in the assembly step and the preceding and following physical components in the assembly step includes: providing an operation button, and in response to the user pressing the operation button, indicating the entities involved in the next assembly step. Parts and methods of assembling the front and rear physical parts in the next assembly step.
  • the assembly method for determining and indicating the physical components involved in the assembly step and the front and rear physical components in the assembly step includes: providing a human-machine interface device, and displaying the front and rear entities in the assembly step on the human-machine interface device. How to assemble the parts.
  • the device further includes: after the current assembly step is completed, scan the current physical component, perform quality verification on the current physical component and the corresponding digital model in the assembly information database, and enter the next step when the quality verification passes.
  • One assembly step scan the current physical component, perform quality verification on the current physical component and the corresponding digital model in the assembly information database, and enter the next step when the quality verification passes.
  • the present invention also proposes an electronic device, including a processor, a memory and instructions stored in the memory, wherein when the instructions are executed by the processor, the above method is implemented.
  • the present invention also proposes a computer-readable storage medium on which computer instructions are stored, which execute the method as described above when executed.
  • Figure 1 is a flow chart of an assembly assistance method according to an embodiment of the present invention
  • Figure 2 is an exemplary schematic diagram of an implementation process of an assembly assistance method according to an embodiment of the present invention
  • Figure 3 is a flow chart of a state machine in an assembly assistance method according to an embodiment of the present invention.
  • Figure 4 is a schematic diagram of an assembly auxiliary device according to an embodiment of the present invention.
  • Figure 5 is a schematic diagram of an electronic device according to an embodiment of the present invention.
  • Figure 1 is a flow chart of an assembly assistance method 100 according to an embodiment of the present invention. As shown in Figure 1, the assembly assistance method 100 includes:
  • Step 110 Obtain data from the product life cycle management system to build an assembly information database, and determine the assembly process of the product to be assembled based on the assembly information database and the assembly tasks obtained from the manufacturing execution system.
  • the product lifecycle management system integrates product-related information, such as CAD files of multiple products, assembly methods, resources, etc.
  • the data in PLM can be obtained through the API of PLM to build an assembly information database.
  • Manufacturing Execution System MES
  • the plan can include assembly tasks, that is, items to be assembled product and the quantity of the product to be assembled.
  • the assembly task includes the product to be assembled and the quantity of the product to be assembled. Search the product to be assembled in the assembly information database to determine the assembly process of the product to be assembled, that is, the CAD of the product to be assembled.
  • the assembly task obtained from the MES system is to assemble a piston, and the CAD file, assembly method, assembly tool, etc. of the piston are obtained by searching in the assembly information database.
  • Figure 2 is an exemplary schematic diagram of the implementation process of an assembly assistance method according to an embodiment of the present invention.
  • the edge computing device 23 includes an assembly information database building unit 232a.
  • the assembly information database building unit 232a obtains products.
  • the data of the life cycle management system 21 constructs an assembly information database, and the assembly process of the product to be assembled is determined based on the assembly information database and the assembly tasks obtained from the manufacturing execution system 22 .
  • Step 120 Scan all physical components of the product to be assembled to obtain a reconstructed model of each physical component.
  • 3D sensors can be used to scan all physical parts of the product to be assembled. It can be understood that all physical parts are laid flat on a surface, and there is no overlap between the physical parts to avoid mutual interference between the physical parts. Modeling the scanned data can A reconstructed model of each solid component is obtained. Preferably, the reconstructed model is a point cloud model.
  • the assembled product is a piston.
  • the parts 27a, 27b, 27c, and 27d of the piston are laid flat on a plane.
  • the 3D sensor 26 detects the parts 27a, 27b, 27c, and 27d of the piston. Scan, and the scanned data is sent to the component recognition unit 233a. In the component recognition unit 233a, the background and noise of the scanned point cloud data are removed, the point cloud corresponding to each component is segmented, and the point cloud data model of each component is reconstructed.
  • Step 130 Match the assembly process with the reconstructed model of the physical components, determine and instruct the physical components involved in the assembly step and the assembly methods of the front and rear physical components in the assembly step, until the assembly process of the product to be assembled is completed.
  • the reconstructed model that matches the assembly process and the physical parts can be generated based on the CAD files of each assembled part to generate corresponding sample point cloud data, rotated at different angles along the XYZ axis to generate a training data set labeled with the part ID, and a deep learning algorithm is used for training
  • the data set trains the point cloud classification model.
  • the point cloud data obtained by scanning the physical parts is divided into multiple groups and then filtered to remove noise. Each group of point cloud data is input into the point cloud classification model to predict the corresponding part ID, thereby determining the entities involved in the assembly step. components, and indicates to the user the physical component, as well as the assembly method of the front and rear physical components in the assembly step.
  • the user can assemble according to the instructions, and after the assembly step is completed, enter the next assembly step, and continue the instructions in the next assembly step. Regarding the physical components and assembly methods involved, the user continues to assemble according to the instructions until the assembly process of the product to be assembled is completed.
  • the component search unit 233b searches for the component currently involved in the assembly process
  • the digital model extraction unit 233c extracts the CAD model of the component from the assembly information database
  • the matching unit 233d matches the component in the reconstructed model of all physical components.
  • the CAD model of the component can determine the ID of the physical component involved in the current assembly step, determine the position of the physical component through the component positioning unit 233e, and finally indicate the position of the physical component through the indication generation unit 233f.
  • the assembly method extraction unit 231a extracts the assembly method from the assembly information database
  • the instruction generation unit 231b generates an operation instruction according to the extracted assembly method to instruct the user to assemble according to the operation instruction.
  • indicating the physical components involved in the assembly step includes projecting light through a projector onto the physical components involved in the assembly step.
  • the instruction generation unit 233f sends the component positioning to the projector 25, and the projector projects light to the physical component at the target position according to the component positioning.
  • the physical component involved is 27b, and the projector 25 casts light on the solid part 27b.
  • the physical components involved in the assembly steps can be intuitively indicated to the user by projecting light through a projector.
  • an assembly method for determining and indicating the physical components involved in the assembly step and the preceding and following physical components in the assembly step includes: providing an operation button, and in response to the user pressing the operation button, indicating the entities involved in the next assembly step. Parts and methods of assembling the front and rear physical parts in the next assembly step. Specifically, as shown in Figure 2, an operation button 24 is provided. After the user finds the physical components involved according to the instructions and completes the assembly according to the assembly method, he presses the operation button 24, and the system instructs the physical components involved in the next assembly step and the following steps. An assembly method of front and rear physical components in an assembly step. To this end, by providing an operation button to indicate the next set of assembly steps in response to the user pressing the operation button, the assembly process can be made more flexible and convenient.
  • an assembly method for determining and indicating the physical components involved in the assembly step and the front and rear physical components in the assembly step includes: providing a human-machine interface device, and displaying the physical components before and after the assembly step on the human-machine interface device.
  • Assembly method Specifically, as shown in Figure 2, a human-machine interface device 28 is provided, and the assembly method of the front and rear physical components in the assembly steps is displayed on the human-machine interface device 28. The user can follow the assembly steps in the assembly steps displayed on the human-machine interface device 28. Assemble the front and rear physical parts using the same assembly method. To this end, by displaying the assembly method of the front and rear physical components in the assembly step on the human-machine interface device, the assembly method in the assembly step can be intuitively instructed to the user.
  • the method further includes: after the current assembly step is completed, scanning the current physical component, performing quality verification on the current physical component and the corresponding digital model in the assembly information database, and entering the next assembly when the quality verification is passed. step.
  • the component 27e is a real-time assembly result.
  • the current physical component 27e is scanned, the component identification unit 234a reconstructs and identifies the component 27e, and the comparison unit 234b reconstructs the physical component 27e.
  • the reconstructed recognition model is compared with the CAD model in the assembly information database, that is, the quality of the current physical component is verified with the corresponding digital model in the assembly information database, and the next assembly step is entered when the quality verification is passed.
  • the edge computing device 23 also includes a state machine 232b and an operation identification unit 232c, through which the user action and assembly process can be identified.
  • Figure 3 is a flow chart of a state machine in an assembly assistance method according to an embodiment of the present invention. As shown in Figure 3, the state machine includes:
  • Step 301 event of operating button 24
  • Step 302 process the button signal of the operation button 24;
  • Step 303 update assembly steps
  • Step 304 extract assembly information
  • Step 305 scan the physical components
  • Step 306, indicate physical components
  • Step 307 generate operation instructions and display
  • Step 308 scan the assembled components
  • Step 309 assembly quality verification
  • Step 310 whether it passes the assembly quality verification, if so, go to step 311, if not, go to step 312;
  • Step 31 check the assembly process
  • Step 313 whether the assembly is completed, if yes, go to step 314, if not, go to step 302;
  • the auxiliary assembly method of the present invention will be described below by taking the piston as the product to be assembled.
  • the assembly information database was constructed by acquiring data from the PLM system.
  • the product to be assembled was determined to be a piston from the MES system.
  • the CAD file of the piston was extracted from the assembly information database.
  • the assembly method, the piston components 27a, Parts 27b, 27c, and 27d are laid flat on a plane.
  • the 3D sensor 26 scans the parts 27a, 27b, 27c, and 27d of the piston.
  • the scan data is sent to the part identification unit 233a.
  • the state machine b determines the current In the assembly step, the search unit 233b searches for the physical component 27a involved in the current assembly step.
  • the digital model extraction unit 233c extracts the digital model of the physical component 27a.
  • the matching unit 233d matches the digital model of the physical component 27a in all reconstructed models to obtain the entity.
  • the ID of the component 27a, the component positioning unit 233e determines the position of the physical component 27a, the instruction generation unit 233f generates an instruction, the projector 25 projects light on the component 27a according to the instruction, the user finds the component 27a according to the light projection instruction, and then presses the operation button 24 to update Assembling step, the component involved in this assembly step is 27b.
  • the projector 25 projects light on the component 27b.
  • the human-machine interface device 28 displays the assembly method of the assembly step. The user finds the component 27b according to the lighting instructions and follows the instructions.
  • the assembly method showing the assembly step in the machine interface device 28 assembles the component 27b to the component 27a, scans the components of the component 27b and the component 27a, the component identification unit 234a identifies and reconstructs the real-time component, and the comparison unit 234b compares the identified and reconstructed model with The digital models in the system are compared for quality verification. After passing the quality verification, the user presses the operation button 24 to update the assembly steps, and so on, until the last component 27d is assembled, thereby completing the assembly process of the product to be assembled.
  • Embodiments of the present invention provide an assembly assistance method that determines the assembly process of the product to be assembled from data in the product life cycle management system and the manufacturing execution system, scans the physical components to obtain a reconstructed model, and matches the reconstructed model with the assembly process in real time to Users are instructed to complete the assembly according to the assembly process. This process does not require marking on the physical parts, improves production efficiency, and is widely applicable to various application scenarios.
  • FIG. 4 is a schematic diagram of an assembly auxiliary device 400 according to an embodiment of the present invention. As shown in Figure 4, the assembly auxiliary device 400 includes:
  • the acquisition module 410 acquires data from the product life cycle management system to construct an assembly information database, and determines the assembly process of the product to be assembled based on the assembly information database and the assembly tasks obtained from the manufacturing execution system.
  • the modeling module 420 scans all physical components of the product to be assembled and obtains reconstructed models of each physical component.
  • the instruction module 430 matches the assembly process with the reconstruction model of the physical components, determines and instructs the physical components involved in the assembly step and the assembly devices of the front and rear physical components in the assembly step, until the assembly process of the product to be assembled is completed.
  • indicating the physical components involved in the assembly step includes projecting light through a projector onto the physical components involved in the assembly step.
  • an assembly method for determining and indicating the physical components involved in the assembly step and the preceding and following physical components in the assembly step includes: providing an operation button, and in response to the user pressing the operation button, indicating the entities involved in the next assembly step. Parts and methods of assembling the front and rear physical parts in the next assembly step.
  • an assembly method for determining and indicating the physical components involved in the assembly step and the front and rear physical components in the assembly step includes: providing a human-machine interface device, and displaying the physical components before and after the assembly step on the human-machine interface device. Assembly method.
  • the device further includes: after the current assembly step is completed, scan the current physical component, perform quality verification on the current physical component and the corresponding digital model in the assembly information database, and enter the next assembly when the quality verification is passed. step.
  • FIG. 5 is a schematic diagram of an electronic device 500 according to an embodiment of the present invention.
  • the electronic device 500 includes a processor 510 and a memory 520 .
  • the memory 520 stores instructions, and when the instructions are executed by the processor 510 , the method 100 as described above is implemented.
  • the present invention also proposes a computer-readable storage medium on which computer instructions are stored, and when executed, the computer instructions execute the method 100 as described above.
  • Some aspects of the method and device of the present invention may be executed entirely by hardware, may be entirely executed by software (including firmware, resident software, microcode, etc.), or may be executed by a combination of hardware and software.
  • the above hardware or software may be referred to as “data block”, “module”, “engine”, “unit”, “component” or “system”.
  • the processor may be one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DAPDs), programmable logic devices (PLCs), field programmable gate arrays (FPGAs), processors , controller, microcontroller, microprocessor or combination thereof.
  • ASICs application specific integrated circuits
  • DSPs digital signal processors
  • DAPDs digital signal processing devices
  • PLCs programmable logic devices
  • FPGAs field programmable gate arrays
  • processors controller, microcontroller, microprocessor or combination thereof.
  • aspects of the invention may be embodied as a computer product embodied in one or more computer-readable media, the product including computer-readable program code.
  • computer-readable media may include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic tapes, etc.), optical disks (e.g., compact disks (CD), digital versatile disks (DVD), ...), smart cards and flash memory devices (e.g. cards, sticks, key drives).
  • a flowchart is used here to illustrate operations performed by methods according to embodiments of the present application. It should be understood that the preceding operations are not necessarily performed in exact order. Instead, the various steps can be processed in reverse order or simultaneously. At the same time, other operations may be added to these processes, or a step or steps may be removed from these processes.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Strategic Management (AREA)
  • Economics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Resources & Organizations (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Game Theory and Decision Science (AREA)
  • Development Economics (AREA)
  • Educational Administration (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Geometry (AREA)
  • Computer Hardware Design (AREA)
  • Marketing (AREA)
  • Operations Research (AREA)
  • Quality & Reliability (AREA)
  • Tourism & Hospitality (AREA)
  • General Business, Economics & Management (AREA)
  • Automatic Assembly (AREA)

Abstract

本发明提出了一种组装辅助方法，所述组装辅助方法包括：获取产品生命周期管理系统的数据构建组装信息库，根据所述组装信息库和从制造执行系统获取的组装任务确定待组装产品的组装过程；扫描所述待组装产品的所有实体部件，得到各个所述实体部件的重建模型；匹配所述组装过程与所述实体部件的重建模型，确定并指示组装步骤中涉及的实体部件以及组装步骤中的前后实体部件的组装方法，直至所述待组装产品的组装过程结束。

Description

组装辅助方法及装置 技术领域
本发明主要涉及工业自动化领域，尤其涉及一种组装辅助方法及装置。
背景技术
针对传统的批量生成组装过程，工人接收到的培训是执行对应于有限类型产品的固定操作。对于定制化的小批量的柔性制造，也就是在同一条产线上生成不同的产品，工人需要在不同的产品之间切换，频繁的切换将会导致较高的组装错误率和较低的生成效率。为了使工人适应新的组装流程，提高生产效率，组装辅助系统应运而生。一种组装辅助系统是基于增强现实的，通过扫描附在部件上的条形码识别当前组装步骤的部件，并且向工人提供相应的组装指令，然而需要在部件上附上条形码，并且手动准备组装指令，生产效率依然不高；另一种组装辅助系统是针对特定的应用场景的，例如拧螺丝，通过增强现实来指示螺丝的位置标记，工人可以轻松地拧紧螺丝，然而只能针对特定的应用场景，无法适用于其它的应用场景。
发明内容
为了解决上述技术问题，本发明提供一种组装辅助方法及装置，以指示工人完成组装任务，提高生产效率，且适用于各种应用场景。
为实现上述目的，本发明提出了一种组装辅助方法，所述组装辅助方法包括：获取产品生命周期管理系统的数据构建组装信息库，根据所述组装信息库和从制造执行系统获取的组装任务确定待组装产品的组装过程；扫描所述待组装产品的所有实体部件，得到各个所述实体部件的重建模型；匹配所述组装过程与所述实体部件的重建模型，确定并指示组装步骤中涉及的实体部件以及组装步骤中的前后实体部件的组装方法，直至所述待组装产品的组装过程结束。为此，从产品生命周期管理系统和制造执行系统中的数据确定待组装产品的组装过程，扫描实体部件得到重建模型，实时匹配重建模型与组装过程，以指示用户按照组装过程完成组装，该过程无需在实体部件上附上标记，提高了生产效率，且广泛适用于各种应用场景。
可选地，指示组装步骤中涉及的实体部件包括：通过投影仪向组装步骤中涉及的实体部件投光。为此，通过投影仪向组装步骤中涉及的实体部件投光，可以直观地向用户指示涉及到的实体部件。
可选地，确定并指示组装步骤中涉及的实体部件以及组装步骤中的前后实体部件的组装方法包括：提供一操作按钮，响应于用户按压所述操作按钮，指示下一组装步骤中涉及的实体部件以及下一组装步骤中的前后实体部件的组装方法。为此，通过提供一操作按钮，响应于用户按压操作按钮指示下一组组装步骤，可以使得组装过程更加灵活便利。
可选地，确定并指示组装步骤中涉及的实体部件以及组装步骤中的前后实体部件的组装方法包括：提供一人机接口设备，在所述人机接口设备上显示所述组装步骤中的前后实体部件的组装方法。为此，通过在人机接口设备上显示组装步骤中的前后实体部件的组装方法，可以直观地向用户指示组装步骤中的组装方法。
可选地，所述方法还包括：在当前组装步骤结束之后，对当前实体组件进行扫描，将所述当前实体组件与组装信息库中对应的数字模型进行质量验证，在质量验证通过时进入下一组装步骤。为此，可以自动实时验证组件的组装质量，提高了生产效率。
本发明还提出了一种组装辅助装置，所述组装辅助装置包括：获取模块，获取产品生命周期管理系统的数据构建组装信息库，根据所述组装信息库和从制造执行系统获取的组装任务确定待组装产品的组装过程；建模模块，扫描所述待组装产品的所有实体部件，得到各个所述实体部件的重建模型；指示模块，匹配所述组装过程与所述实体部件的重建模型，确定并指示组装步骤中涉及的实体部件以及组装步骤中的前后实体部件的组装方法，直至所述待组装产品的组装过程结束。
可选地，指示组装步骤中涉及的实体部件包括：通过投影仪向组装步骤中涉及的实体部件投光。
可选地，确定并指示组装步骤中涉及的实体部件以及组装步骤中的前后实体部件的组装方法包括：提供一操作按钮，响应于用户按压所述操作按钮，指示下一组装步骤中涉及的实体部件以及下一组装步骤中的前后实体部件的组装方法。
可选地，确定并指示组装步骤中涉及的实体部件以及组装步骤中的前后实体部件的组装方法包括：提供一人机接口设备，在所述人机接口设备上显示所述组装步骤中的前后实体部件的组装方法。
可选地，所述装置还包括：在当前组装步骤结束之后，对当前实体组件进行扫描，将所述当前实体组件与组装信息库中对应的数字模型进行质量验证，在质量验证通过时进入下一组装步骤。
本发明还提出了一种电子设备，包括处理器、存储器和存储在所述存储器中的指令，其中所述指令被所述处理器执行时实现如上所述的方法。
本发明还提出了一种计算机可读存储介质，其上存储有计算机指令，所述计算机指令 在被运行时执行如上所述的方法。
附图说明
以下附图仅旨在于对本发明做示意性说明和解释，并不限定本发明的范围。其中，
图1是根据本发明的一实施例的一种组装辅助方法的流程图；
图2是根据本发明的一实施例的一种组装辅助方法的实施过程的示范性示意图；
图3是根据本发明的一实施例的一种组装辅助方法中状态机的流程图；
图4是根据本发明的一实施例的一种组装辅助装置的示意图；
图5是根据本发明的一实施例的一种电子设备的示意图。
附图标记说明
100组装辅助方法
110-130步骤
21产品生命周期管理系统
22制造执行系统
23边缘计算设备
231a组装方法提取单元
231b指令生成单元
232a组装信息库构建单元
232b状态机
232c操作识别单元
233a部件识别单元
233b搜索单元
233c数字模型提取单元
233d匹配单元
233e定位单元
233f指示生成单元
234a组件识别单元
234b比较单元
24操作按钮
25投影仪
26 3D摄像头
27a-27d部件
27e组件
28人机接口设备
301-314步骤
400组装辅助装置
410获取模块
420建模模块
430指示模块
500电子设备
510处理器
520存储器
具体实施方式
为了对本发明的技术特征、目的和效果有更加清楚的理解，现对照附图说明本发明的具体实施方式。
在下面的描述中阐述了很多具体细节以便于充分理解本发明，但是本发明还可以采用其它不同于在此描述的其它方式来实施，因此本发明不受下面公开的具体实施例的限制。
如本申请和权利要求书中所示，除非上下文明确提示例外情形，“一”、“一个”、“一种”和/或“该”等词并非特指单数，也可包括复数。一般说来，术语“包括”与“包含”仅提示包括已明确标识的步骤和元素，而这些步骤和元素不构成一个排它性的罗列，方法或者设备也可能包含其他的步骤或元素。
本发明提出了一种组装辅助方法，图1是根据本发明的一实施例的一种组装辅助方法100的流程图，如图1所示，组装辅助方法100包括：
步骤110，获取产品生命周期管理系统的数据构建组装信息库，根据组装信息库和从制造执行系统获取的组装任务确定待组装产品的组装过程。
产品生命周期管理系统(Product Lifecycle Management，PLM)中集成有与产品相关的信息，例如多种产品的CAD文件，组装方法、资源等，通过PLM的API可以获取PLM中的数据构建组装信息库。制造执行系统(Manufacturing Execution System，MES)为操作人员/管理人员提供计划的执行、跟踪以及所有资源(人、设备、物料、客户需求等)的当前状态，计划可以包括组装任务，即待组装的产品以及待组装产品的数量。从MES系统 获取组装任务，组装任务包括待组装的产品以及待组装产品的数量，将该待组装的产品在组装信息库中进行搜索，可以确定待组装产品的组装过程，即待组装产品的CAD文件，组装方法，组装工具等。例如，从MES系统获取组装任务为组装活塞，在组装信息库中搜索得到活塞的CAD文件，组装方法，组装工具等。图2是根据本发明的一实施例的一种组装辅助方法的实施过程的示范性示意图，如图2所示，边缘计算设备23包括组装信息库构建单元232a，组装信息库构建单元232a获取产品生命周期管理系统21的数据构建组装信息库，根据组装信息库和从制造执行系统22获取的组装任务确定待组装产品的组装过程。
步骤120，扫描待组装产品的所有实体部件，得到各个实体部件的重建模型。
可以使用3D传感器扫描待组装产品的所有实体部件，可以理解，所有实体部件平铺一表面上，实体部件之间相互没有重叠，以避免实体部件之间的相互干扰，对扫描数据进行建模可以得到各个实体部件的重建模型，优选地，重建模型是点云模型。如图2所示，带组装产品是活塞，活塞的部件27a，部件27b，部件27c，部件27d平铺在一平面上，3D传感器26对活塞的部件27a，部件27b，部件27c，部件27d进行扫描，扫描数据被发送至部件识别单元233a，在部件识别单元233a中，去掉扫描的点云数据的背景和噪声，分割各个部件对应的点云，并重建各个部件的点云数据模型。
步骤130，匹配组装过程与实体部件的重建模型，确定并指示组装步骤中涉及的实体部件以及组装步骤中的前后实体部件的组装方法，直至待组装产品的组装过程结束。
匹配组装过程与实体部件的重建模型可以是基于各个组装部件的CAD文件生成对应的样本点云数据，沿着XYZ轴旋转不同的角度以产生标记部件ID的训练数据集，采用深度学习算法使用训练数据集训练点云分类模型。扫描实体部件得到的点云数据被分割成多组，然后过滤以去除噪声，每组点云数据被输入到点云分类模型中以预测对应的部件ID，由此确定了组装步骤中涉及的实体部件，并向用户指示该实体部件，以及指示组装步骤中的前后实体部件的组装方法，用户可以按照指示进行组装，在该组装步骤结束之后，进入下一组装步骤，在下一组装步骤中继续指示涉及的实体部件以及组装方法，用户继续按照指示进行组装，直至待组装产品的组装过程结束。
如图2所示，部件搜索单元233b搜索到组装过程中当前涉及的部件，数字模型提取单元233c从组装信息库中提取该部件的CAD模型，匹配单元233d在全部实体部件的重建模型中匹配该部件的CAD模型，可以确定当前组装步骤中涉及的实体部件的ID，通过部件定位单元233e确定实体部件的位置，最后通过指示生成单元233f指示该实体部件的位置。同时，组装方法提取单元231a从组装信息库中提取组装方法，指令生成单元231b 根据提取的组装方法生成操作指令，以指示用户按照操作指令进行组装。
在一些实施例中，指示组装步骤中涉及的实体部件包括：通过投影仪向组装步骤中涉及的实体部件投光。具体地，如图2所示，指示生成单元233f将部件定位发送至投影仪25，投影仪根据部件定位向目标位置的实体部件投光，在图2中，涉及的实体部件为27b，投影仪25向实体部件为27b投光。为此，通过投影仪向组装步骤中涉及的实体部件投光，可以直观地向用户指示涉及到的实体部件。
在一些实施例中，确定并指示组装步骤中涉及的实体部件以及组装步骤中的前后实体部件的组装方法包括：提供一操作按钮，响应于用户按压操作按钮，指示下一组装步骤中涉及的实体部件以及下一组装步骤中的前后实体部件的组装方法。具体地，如图2所示，提供一操作按钮24，用户在按照指示找到涉及的实体部件以及按照组装方法完成组装之后，按压操作按钮24，系统指示下一组装步骤中涉及的实体部件以及下一组装步骤中的前后实体部件的组装方法。为此，通过提供一操作按钮，响应于用户按压操作按钮指示下一组组装步骤，可以使得组装过程更加灵活便利。
在一些实施例中，确定并指示组装步骤中涉及的实体部件以及组装步骤中的前后实体部件的组装方法包括：提供一人机接口设备，在人机接口设备上显示组装步骤中的前后实体部件的组装方法。具体地，如图2所示，提供一人机接口设备28，在人机接口设备28上显示组装步骤中的前后实体部件的组装方法，用户可以根据人机接口设备28上显示的组装步骤中的前后实体部件的组装方法进行组装。为此，通过在人机接口设备上显示组装步骤中的前后实体部件的组装方法，可以直观地向用户指示组装步骤中的组装方法。
在一些实施例中，方法还包括：在当前组装步骤结束之后，对当前实体组件进行扫描，将当前实体组件与组装信息库中对应的数字模型进行质量验证，在质量验证通过时进入下一组装步骤。具体地，如图2所示，组件27e为实时的组装结果，在当前组装步骤结束之后，对当前实体组件27e进行扫描，组件识别单元234a对组件27e进行重建识别，比较单元234b将实体组件27e的重建识别模型与组装信息库中的CAD模型进行比较，即将当前实体组件与组装信息库中对应的数字模型进行质量验证，在质量验证通过时进入下一组装步骤。
边缘计算设备23还包括状态机232b和操作识别单元232c，通过状态机232b和操作识别单元232c可以识别用户动作和组装过程。图3是根据本发明的一实施例的一种组装辅助方法中状态机的流程图，如图3所示，状态机包括：
步骤301，操作按钮24的事件；
步骤302，处理操作按钮24的按钮信号；
步骤303，更新组装步骤；
步骤304，提取组装信息；
步骤305，扫描实体部件；
步骤306，指示实体部件；
步骤307，生成操作指令和显示；
步骤308，扫描组装的组件；
步骤309，组装质量验证；
步骤310，是否通过组装质量验证，如果是，进入步骤311，如果否，进入步骤312；
步骤311，检查组装过程；
步骤312，重新组装；
步骤313，组装是否完成，如果是，进入步骤314，如果否，进入步骤302；
步骤314，结束。
下面以活塞作为待组装产品对本发明的组装辅助方法进行说明。
如图2所示，从PLM系统中获取数据构建了组装信息库，从MES系统中确定待组装产品为活塞，从组装信息库中提取了活塞的CAD文件，组装方法，将活塞的部件27a，部件27b，部件27c，部件27d平铺在一平面上，3D传感器26对活塞的部件27a，部件27b，部件27c，部件27d进行扫描，扫描数据被发送至部件识别单元233a，状态机b确定当前组装步骤，搜索单元233b搜到当前组装步骤涉及到的实体部件27a，数字模型提取单元233c提取实体部件27a的数字模型，匹配单元233d在所有的重建模型中匹配实体部件27a的数字模型，获得实体部件27a的ID，部件定位单元233e确定实体部件27a的位置，指示生成单元233f生成指示，投影仪25根据指示向部件27a投光，用户根据该投光指示找到部件27a，然后按压操作按钮24更新组装步骤，该组装步骤涉及到的部件是27b，投影仪25向部件27b投光，同时人机接口设备28中显示该组装步骤的组装方法，用户根据该投光指示找到部件27b，并按照人机接口设备28中显示该组装步骤的组装方法将部件27b组装至部件27a上，扫描部件27b和部件27a的组件，组件识别单元234a对实时组件进行识别重建，由比较单元234b对识别重建模型与系统中的数字模型进行比较以进行质量验证，在通过质量验证之后，用户按压操作按钮24更新组装步骤，依次类推，直至组装好最后一个部件27d，从而完成待组装产品的组装过程。
本发明的实施例提供了一种组装辅助方法，从产品生命周期管理系统和制造执行系统中的数据确定待组装产品的组装过程，扫描实体部件得到重建模型，实时匹配重建模型与组装过程，以指示用户按照组装过程完成组装，该过程无需在实体部件上附上标记，提高 了生产效率，且广泛适用于各种应用场景。
本发明还提出了一种组装辅助装置，图4是根据本发明的一实施例的一种组装辅助装置400的示意图，如图4所示，组装辅助装置400包括：
获取模块410，获取产品生命周期管理系统的数据构建组装信息库，根据组装信息库和从制造执行系统获取的组装任务确定待组装产品的组装过程。
建模模块420，扫描待组装产品的所有实体部件，得到各个实体部件的重建模型。
指示模块430，匹配组装过程与实体部件的重建模型，确定并指示组装步骤中涉及的实体部件以及组装步骤中的前后实体部件的组装装置，直至待组装产品的组装过程结束。
在一些实施例中，指示组装步骤中涉及的实体部件包括：通过投影仪向组装步骤中涉及的实体部件投光。
在一些实施例中，确定并指示组装步骤中涉及的实体部件以及组装步骤中的前后实体部件的组装方法包括：提供一操作按钮，响应于用户按压操作按钮，指示下一组装步骤中涉及的实体部件以及下一组装步骤中的前后实体部件的组装方法。
在一些实施例中，确定并指示组装步骤中涉及的实体部件以及组装步骤中的前后实体部件的组装方法包括：提供一人机接口设备，在人机接口设备上显示组装步骤中的前后实体部件的组装方法。
在一些实施例中，装置还包括：在当前组装步骤结束之后，对当前实体组件进行扫描，将当前实体组件与组装信息库中对应的数字模型进行质量验证，在质量验证通过时进入下一组装步骤。
本发明还提出一种电子设备500。图5是根据本发明的一实施例的一种电子设备500的示意图。如图5所示，电子设备500包括处理器510和存储器520，存储器520存储中存储有指令，其中指令被处理器510执行时实现如上文所述的方法100。
本发明还提出一种计算机可读存储介质，其上存储有计算机指令，计算机指令在被运行时执行如上文所述的方法100。
本发明的方法和装置的一些方面可以完全由硬件执行、可以完全由软件(包括固件、常驻软件、微码等)执行、也可以由硬件和软件组合执行。以上硬件或软件均可被称为“数据块”、“模块”、“引擎”、“单元”、“组件”或“系统”。处理器可以是一个或多个专用集成电路(ASIC)、数字信号处理器(DSP)、数字信号处理器件(DAPD)、可编程逻辑器件(PLC)、现场可编程门阵列(FPGA)、处理器、控制器、微控制器、微处理器或者其组合。此外，本发明的各方面可能表现为位于一个或多个计算机可读介质中的计算机产品，该产品包括计算机可读程序编码。例如，计算机可读介质可包括，但不限于，磁性存储设备(例如，硬盘、 软盘、磁带……)、光盘(例如，压缩盘(CD)、数字多功能盘(DVD)……)、智能卡以及闪存设备(例如，卡、棒、键驱动器……)。
在此使用了流程图用来说明根据本申请的实施例的方法所执行的操作。应当理解的是，前面的操作不一定按照顺序来精确地执行。相反，可以按照倒序或同时处理各种步骤。同时，或将其他操作添加到这些过程中，或从这些过程移除某一步或数步操作。
应当理解，虽然本说明书是按照各个实施例描述的，但并非每个实施例仅包含一个独立的技术方案，说明书的这种叙述方式仅仅是为清楚起见，本领域技术人员应当将说明书作为一个整体，各实施例中的技术方案也可以经适当组合，形成本领域技术人员可以理解的其他实施方式。
以上所述仅为本发明示意性的具体实施方式，并非用以限定本发明的范围。任何本领域的技术人员，在不脱离本发明的构思和原则的前提下所作的等同变化、修改与结合，均应属于本发明保护的范围。

Claims (12)

  1. 一种组装辅助方法(100)，其特征在于，所述组装辅助方法(100)包括：
    获取产品生命周期管理系统的数据构建组装信息库，根据所述组装信息库和从制造执行系统获取的组装任务确定待组装产品的组装过程(110)；
    扫描所述待组装产品的所有实体部件，得到各个所述实体部件的重建模型(120)；
    匹配所述组装过程与所述实体部件的重建模型，确定并指示组装步骤中涉及的实体部件以及组装步骤中的前后实体部件的组装方法，直至所述待组装产品的组装过程结束(130)。
  2. 根据权利要求所述1的组装辅助方法(100)，其特征在于，指示组装步骤中涉及的实体部件包括：通过投影仪向组装步骤中涉及的实体部件投光。
  3. 根据权利要求1或2所述的组装辅助方法(100)，其特征在于，确定并指示组装步骤中涉及的实体部件以及组装步骤中的前后实体部件的组装方法包括：提供一操作按钮，响应于用户按压所述操作按钮，指示下一组装步骤中涉及的实体部件以及下一组装步骤中的前后实体部件的组装方法。
  4. 根据权利要求1所述的组装辅助方法(100)，其特征在于，确定并指示组装步骤中涉及的实体部件以及组装步骤中的前后实体部件的组装方法包括：提供一人机接口设备，在所述人机接口设备上显示所述组装步骤中的前后实体部件的组装方法。
  5. 根据权利要求1所述的组装辅助方法(100)，其特征在于，所述方法(100)还包括：在当前组装步骤结束之后，对当前实体组件进行扫描，将所述当前实体组件与组装信息库中对应的数字模型进行质量验证，在质量验证通过时进入下一组装步骤。
  6. 一种组装辅助装置(400)，其特征在于，所述组装辅助装置(400)包括：
    获取模块(410)，获取产品生命周期管理系统的数据构建组装信息库，根据所述组装信息库和从制造执行系统获取的组装任务确定待组装产品的组装过程；
    建模模块(420)，扫描所述待组装产品的所有实体部件，得到各个所述实体部件的重建模型；
    指示模块(430)，匹配所述组装过程与所述实体部件的重建模型，确定并指示组装步骤中涉及的实体部件以及组装步骤中的前后实体部件的组装方法，直至所述待组装产品的组装过程结束。
  7. 根据权利要求所述6的组装辅助装置(400)，其特征在于，所述指示模块(430)指示组装步骤中涉及的实体部件包括：通过投影仪向组装步骤中涉及的实体部件投光。
  8. 根据权利要求6或7所述的组装辅助装置(400)，其特征在于，所述指示模块(430)确定并指示组装步骤中涉及的实体部件以及组装步骤中的前后实体部件的组装方法包括：提供一操作按钮，响应于用户按压所述操作按钮，指示下一组装步骤中涉及的实体部件以及下一组装步骤中的前后实体部件的组装方法。
  9. 根据权利要求6所述的组装辅助装置(400)，其特征在于，所述指示模块(430)确定并指示组装步骤中涉及的实体部件以及组装步骤中的前后实体部件的组装方法包括：提供一人机接口设备，在所述人机接口设备上显示所述组装步骤中的前后实体部件的组装方法。
  10. 根据权利要求6所述的组装辅助装置(400)，其特征在于，所述装置(400)还包括：在当前组装步骤结束之后，对当前实体组件进行扫描，将所述当前实体组件与组装信息库中对应的数字模型进行质量验证，在质量验证通过时进入下一组装步骤。
  11. 一种电子设备(500)，包括处理器(510)、存储器(520)和存储在所述存储器(520)中的指令，其中所述指令被所述处理器(510)执行时实现如权利要求1-5任一项所述的方法。
  12. 一种计算机可读存储介质，其上存储有计算机指令，所述计算机指令在被运行时执行根据权利要求1-5中任一项所述的方法。
PCT/CN2022/107191 2022-07-21 2022-07-21 组装辅助方法及装置 WO2024016283A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/107191 WO2024016283A1 (zh) 2022-07-21 2022-07-21 组装辅助方法及装置

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/107191 WO2024016283A1 (zh) 2022-07-21 2022-07-21 组装辅助方法及装置

Publications (1)

Publication Number Publication Date
WO2024016283A1 true WO2024016283A1 (zh) 2024-01-25

Family

ID=89616789

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/107191 WO2024016283A1 (zh) 2022-07-21 2022-07-21 组装辅助方法及装置

Country Status (1)

Country Link
WO (1) WO2024016283A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2006027339A2 (en) * 2004-09-06 2006-03-16 The European Community, Represented By The European Commission Method and system for 3d scene change detection
CN108098293A (zh) * 2017-12-22 2018-06-01 大连运明自动化技术有限公司 一种用于装配生产线的智能电子拣选方法
CN110065068A (zh) * 2019-04-08 2019-07-30 浙江大学 一种基于逆向工程的机器人装配作业演示编程方法及装置
CN110909439A (zh) * 2018-09-14 2020-03-24 中国商用飞机有限责任公司 一种基于ar的装配方法、装置及终端

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2006027339A2 (en) * 2004-09-06 2006-03-16 The European Community, Represented By The European Commission Method and system for 3d scene change detection
CN108098293A (zh) * 2017-12-22 2018-06-01 大连运明自动化技术有限公司 一种用于装配生产线的智能电子拣选方法
CN110909439A (zh) * 2018-09-14 2020-03-24 中国商用飞机有限责任公司 一种基于ar的装配方法、装置及终端
CN110065068A (zh) * 2019-04-08 2019-07-30 浙江大学 一种基于逆向工程的机器人装配作业演示编程方法及装置

Similar Documents

Publication Publication Date Title
US11163999B2 (en) Augmented reality-based capture, processing and transfer of occupational knowledge
US9008370B2 (en) Methods, systems and processor-readable media for tracking history data utilizing vehicle and facial information
KR20200111617A (ko) 제스처 인식 방법, 장치, 전자 기기 및 저장 매체
US11648621B2 (en) Systems and methods to design part weld processes using media libraries
JP2020201772A (ja) 姿勢分析プログラム、および、姿勢分析装置
US20200125839A1 (en) Method and system for automatic repetitive step and cycle detection for manual assembly line operations
CN107924585B (zh) 作业辅助装置、作业辅助方法以及存储介质
CN113779144B (zh) 大数据整合处理方法、系统和存储介质
CN113538631A (zh) 通过基于用户输入所生成的人工智能模型来生成虚拟缺陷图像的计算机程序、方法和装置
JP6488417B1 (ja) ワークショップ支援システムおよびワークショップ支援方法
WO2024016283A1 (zh) 组装辅助方法及装置
KR102597376B1 (ko) 의류 매칭 방법 및 장치, 컴퓨팅 장치 및 매체
US20230297109A1 (en) Method for Remote Assistance and Device
US20210048805A1 (en) Method and system for automatic work instruction creation
CN111368674B (zh) 图像识别方法及装置
CN113811908B (zh) 用于确定生产设备的生产周期的方法和装置
WO2023136029A1 (ja) 機器のメンテナンス方法、装置及びシステム
CN114167993B (zh) 信息处理方法及装置
WO2023184185A1 (zh) 应用程序的编排方法及装置
CN108132802B (zh) 一种配置模型在系统开发中的应用方法
JP2005056272A (ja) 作業時間把握システムおよび作業指示作成システム
US20240193919A1 (en) Machine learning device, classification device, and control device
CN117377929A (zh) 在增强现实视图中通过物理对象的3d模型来标识物理对象上的感兴趣地方
US10599710B2 (en) System and method for generating digital information and altering digital models of components with same
JP6149812B2 (ja) 情報処理システム、その制御方方法、及びプログラム、並びに情報処理装置、その制御方法、及びプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22951556

Country of ref document: EP

Kind code of ref document: A1