WO2024055396A1 - Système et procédé d'enseignement interactif d'assemblage de pièces - Google Patents

Système et procédé d'enseignement interactif d'assemblage de pièces Download PDF

Info

Publication number
WO2024055396A1
WO2024055396A1 PCT/CN2022/129722 CN2022129722W WO2024055396A1 WO 2024055396 A1 WO2024055396 A1 WO 2024055396A1 CN 2022129722 W CN2022129722 W CN 2022129722W WO 2024055396 A1 WO2024055396 A1 WO 2024055396A1
Authority
WO
WIPO (PCT)
Prior art keywords
assembly
unit
parts
site terminal
interactive teaching
Prior art date
Application number
PCT/CN2022/129722
Other languages
English (en)
Chinese (zh)
Inventor
梁钊铭
付傲然
Original Assignee
上海智能制造功能平台有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海智能制造功能平台有限公司 filed Critical 上海智能制造功能平台有限公司
Publication of WO2024055396A1 publication Critical patent/WO2024055396A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/903—Querying
    • G06F16/9035—Filtering based on additional data, e.g. user or group profiles
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/10—Services
    • G06Q50/20—Education
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B9/00—Simulators for teaching or training purposes

Definitions

  • the present invention relates to the field of intelligent manufacturing, and specifically to an interactive teaching system and method for component assembly.
  • Parts assembly is a crucial part of the product manufacturing process. Its assembly quality often directly affects the service life and performance of the product. When faced with small batches and multiple varieties of assembled products, it is difficult to assemble them through automated methods. Only manual assembly can be used. The recognition of product parts in manual assembly is generally done by human eyes and manual installation. Its performance depends heavily on the experience and skills of the assembler, which consumes a lot of human resources. At the same time, there are missing installations, mixed installations, etc., which affect the assembly efficiency; and due to the assembly The steps are complicated and the efficiency of manual one-to-one practical training is low.
  • the Chinese patent application number is CN201810991964.5, a three-dimensional animation assembly teaching method and device based on virtual reality, including step one: import all component models of the product to be assembled, and form a virtual assembly scene; step two :Plan the initial positions of all components mentioned in step one; Step three: Establish a mathematical model of the running path and running speed adaptive algorithm of each component based on the characteristic position one by one; Step four: According to steps two and three, form the steps 1.
  • the invention forms a high-precision model for three-dimensional animation assembly teaching, effectively reducing teaching costs, improving students' learning efficiency, and achieving rapid familiarity with each part and assembly. process effects.
  • it lacks the interactivity of teaching and the teaching efficiency is not ideal.
  • the purpose of the present invention is to provide an interactive teaching system and method for component assembly.
  • an interactive teaching system for parts assembly including:
  • Assembly site terminal the assembly site terminal performs interactive teaching and performs component assembly
  • a backend server which receives information during the assembly process of the assembly site terminal and performs assembly supervision.
  • the assembly site terminal includes:
  • a feeding unit the feeding unit includes a plurality of containers for storing different parts, and each container is provided with a prompting device;
  • An assembly unit which is used to assemble parts
  • a visual inspection unit that collects images of the assembly process of components of the assembly unit in real time
  • a visual feedback unit is used to project teaching or error correction animations to the assembly unit.
  • the backend server includes:
  • the image processing unit receives the image collected by the visual detection unit,
  • An assembly process determination unit which determines whether the current component assembly is correct and whether the current step is completed based on the processed image of the image processing unit;
  • the artificial intelligence error correction unit after receiving the determination result of the current component assembly from the assembly process determination unit, the artificial intelligence error correction unit sends a control instruction to the visual feedback unit to play the corresponding video.
  • the visual feedback unit plays a demonstration video of the next assembly link
  • the visual feedback unit plays the error correction video.
  • the prompt device of the feeding unit includes guidance and error correction prompts, wherein,
  • the guidance prompt includes: the image assembly process determination unit in the background server determines the assembly step, and prompts the container where the material corresponding to the step is located through a long-lit indicator light;
  • the error correction prompt includes: the image assembly process determination unit in the backend server determines whether the assembler selected the correct material in the received image in the image processing unit; if incorrect, an error correction prompt is provided by flashing.
  • the backend server also includes:
  • a process data management unit stores different assembly processes of different parts and assembly processes of different assemblers.
  • the backend server is connected to the on-site terminal, basic data is imported from the on-site terminal connection to the back-end server, and control instructions are transmitted back to the on-site terminal connection from the back-end server.
  • the on-site terminal imports basic data into the back-end server through wireless means, wherein the on-site terminal is configured with a download function to implement the import;
  • the on-site terminal imports basic data into the back-end server through a wired manner, wherein the on-site terminal is connected to the back-end server to implement the import.
  • the backend server performs unified data management through the process data management unit, including:
  • the establishment or change of the artificial intelligence error correction model for the assembly process guidance can be realized.
  • an interactive teaching method for parts assembly is provided, using the interactive teaching system for parts assembly described in any one of the above, including:
  • S1 play the correct assembly method guidance video of the current installation step, and follow the guidance video to perform assembly at the assembly unit;
  • the background server determines whether the assembly is completed based on the real-time area image. If it is completed, it goes to S5; if it is not completed, it returns to S1;
  • the background server determines whether the assembly is correct, and if it is correct, it goes to S6; if it is incorrect, it plays the assembly method error correction video and returns to S2;
  • the present invention has the following beneficial effects:
  • the interactive teaching system and method for parts assembly interactively performs installation of each step, completes detection and supervision, realizes intelligent error correction, and ensures assembly quality.
  • the interactive teaching system and method for parts assembly can guide assemblers to perform assembly through animated teaching videos, effectively improving the efficiency of assembler training.
  • the parts assembly interactive teaching system and method of the embodiment of the present invention realizes the establishment of error correction models through artificial intelligence.
  • the working mode is flexible and can be adapted to different materials and different assembly scenarios. It can meet the requirements with a small amount of improvements. Interactive teaching of different parts assembly processes.
  • Figure 1 is a schematic structural diagram of an interactive teaching system for parts assembly according to an embodiment of the present invention
  • Figure 2 is a schematic three-dimensional structural diagram of an interactive teaching system for parts assembly according to a preferred embodiment of the present invention
  • FIG. 3 is a schematic structural diagram of a feeding unit according to a preferred embodiment of the present invention.
  • Figure 4 is a schematic structural diagram of an assembly unit according to a preferred embodiment of the present invention.
  • Figure 5 is a flow chart of an interactive teaching method for parts assembly in a preferred embodiment of the present invention.
  • Figure 6 is a schematic diagram of the projection guidance video of the interactive parts assembly teaching system and method corresponding to Figure 5, in which a is the actual rendering and b is the corresponding black and white line drawing;
  • Figure 7 is a schematic diagram of the correction video of the interactive parts assembly teaching system and method corresponding to Figure 5, where a is the actual rendering and b is the corresponding black and white line drawing.
  • the present invention provides an embodiment, an interactive teaching system for parts assembly, including an assembly site terminal and a backend server.
  • the assembly site terminal conducts interactive teaching and performs parts assembly; the backend server receives the assembly site terminal. information during the assembly process and perform assembly supervision.
  • the on-site terminal includes a feeding unit, an assembly unit, a visual detection unit and a visual feedback unit.
  • cooperation between each unit of the assembly site terminal and the backend server is used to assist parts assembly teaching in a real-time interactive manner, which can improve teaching efficiency and help assemblers become familiar with the assembly process faster.
  • the feeding unit is used to store a variety of different materials in multiple containers, and each container is equipped with an indicator light device.
  • This indicator light serves as a prompt and warning.
  • the assembly unit consists of the target assembly component model (assembly semi-finished product); the assembler assembles the corresponding materials into the assembly unit based on the instructions of the indicator light of the supply unit, the assembly process steps and the guidance of the on-site terminal.
  • the visual inspection unit includes an image acquisition device and a fixed bracket of the image acquisition device.
  • the visual inspection unit is adapted to obtain photos of the assembly unit or videos of the operation process of each assembly step in real time, and transmit the collected image or video data to the backend server.
  • the visual feedback unit projects the teaching animation of the corresponding assembly steps near the assembly unit through the projector, instructing the assembler to perform the corresponding assembly operation.
  • the backend server includes an image processing unit, an assembly process determination unit and an artificial intelligence error correction unit.
  • the image processing unit receives images or videos collected by the visual detection unit; the assembly process determination unit determines whether the current parts are assembled correctly and whether the current step is completed based on the processed images of the image processing unit; the artificial intelligence error correction unit receives data from After the assembly process determines the assembly error information of the unit, it sends control instructions to the visual feedback unit to play the correction video.
  • a preferred prompting method for an indicator light is provided.
  • Its prompting function is: the indicator light device is suitable for the backend server to determine the assembly step, and prompts the container where the material corresponding to the step is located through the long-lit indicator light; its warning function is: the indicator light device is suitable for each step in the assembly process according to the backend server.
  • the return data of the real-time monitoring of the assembly step is used to determine whether the assembler selected the correct material in this step. If it is incorrect, an error correction prompt will be provided by flashing.
  • the assembler when the visual detection unit combined with the assembly process determination unit detects a specific assembly error, the assembler is instructed to correct the assembly error steps by projecting a specific teaching animation.
  • the specific assembly error refers to the error type defined in advance during training of the machine learning model for correctness and error judgment in this embodiment.
  • the model will not and cannot point out the error, it can only detect that the step is not completed.
  • the data of the on-site terminal during the assembly process will be uploaded to the back-end server in real time through the cyber-physical system; the back-end server determines whether the assembly of the corresponding parts is completed based on the collected images, and projects it through the assembly on-site terminal
  • the assembly personnel can be guided by animations, indicator lights, or flashing lights.
  • the backend server is connected to the on-site terminal in a wired manner to realize the basic data import function.
  • the backend server also performs unified data management, that is, it manages the different assembly processes of different parts and the assembly processes of different assemblers.
  • the corresponding assembly process guidance error correction model is retrieved; or based on the data collected by the visual inspection mechanism of the on-site terminal, the establishment or change of the assembly process guidance artificial intelligence error correction model is achieved.
  • the above-mentioned embodiments of the present invention use the cooperation between each unit of the assembly site terminal and the backend server to assist parts assembly teaching in a real-time interactive manner, which can improve teaching efficiency and help assemblers become familiar with the assembly process faster.
  • the on-site terminal includes a feeding unit, an assembly unit, a visual detection mechanism, a visual feedback mechanism, and a storage backend server. Space for hardware devices.
  • the corresponding parts for each step required for assembly are arranged in the order of the assembly process and placed in the corresponding material trays by category.
  • Indicator lights are arranged at the entrance of each material pallet as a visual feedback device to guide the assembler.
  • this embodiment uses a three-dimensional printed assembly model of the engine surface pipelines, and reserves material interfaces corresponding to each step of the assembly process.
  • a Hikvision industrial camera is installed as a visual inspection mechanism, which can be adjusted to an appropriate configuration state according to the environment of the use scene.
  • a projector is installed in this embodiment as one of the visual feedback mechanisms. Based on the background server's processing of the images collected by the visual detection mechanism, it is determined whether the corresponding component assembly steps are completed or whether the specific component assembly steps are completed correctly. And the visual feedback mechanism projects animation to guide the assembly personnel.
  • the present invention provides an interactive teaching method for parts assembly.
  • the visual detection mechanism collects images in real time and transmits them to the background server for processing and returns to the assembly Success/assembly error/not assembled, etc. status, and use the installed projection device to play the corresponding teaching video, or proceed to the next step.
  • the specific process is shown in Figure 5, including:
  • the background server determines whether the assembly is completed based on the real-time area image obtained in S4. If it is completed, it goes to S5; if it is not completed, it returns to S1;
  • the background server determines whether the assembly is correct, and if it is correct, it goes to S6; if it is incorrect, see Figure 7, plays the assembly method error correction video, and returns to S2;
  • This embodiment assists parts assembly teaching in a real-time interactive manner through the cooperation between each unit of the assembly site terminal and the backend server. It uses indicator lights to prompt the materials that should be picked up, and uses projection teaching animations to guide the assembly process. Through manual Intelligent models and machine vision identify the assembly status of corresponding steps and provide timely correction guidance for specific assembly errors, which can improve teaching efficiency and help assemblers become familiar with the assembly process faster.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Educational Administration (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Tourism & Hospitality (AREA)
  • Educational Technology (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Health & Medical Sciences (AREA)
  • Economics (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Resources & Organizations (AREA)
  • Marketing (AREA)
  • Primary Health Care (AREA)
  • Strategic Management (AREA)
  • General Business, Economics & Management (AREA)
  • Electrically Operated Instructional Devices (AREA)

Abstract

L'invention concerne un système d'enseignement interactif d'assemblage de pièces. Le système comprend : un terminal de site d'assemblage et un serveur dorsal ; le terminal de site d'assemblage effectue un enseignement interactif et effectue un assemblage de pièces ; et le serveur dorsal reçoit des informations dans le processus d'assemblage du terminal de site d'assemblage et effectue une supervision d'assemblage. Le montage de chaque étape est effectué de manière interactive, et l'inspection et la supervision sont achevées, ce qui permet de mettre en œuvre une correction d'erreur intelligente et d'assurer une qualité d'assemblage ; une vidéo d'enseignement d'animation est utilisée pour guider un assembleur pour effectuer un assemblage, et l'efficacité d'entraînement d'assembleur est efficacement améliorée ; et l'établissement d'un modèle de correction d'erreur est mis en œuvre par intelligence artificielle. Le mode de fonctionnement est flexible, et peut être adapté à différents matériaux et différents scénarios d'assemblage, et l'enseignement interactif de processus d'assemblage pour différentes parties peut être mis en œuvre au moyen de minuscules améliorations.
PCT/CN2022/129722 2022-09-14 2022-11-04 Système et procédé d'enseignement interactif d'assemblage de pièces WO2024055396A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211113311.XA CN115841773A (zh) 2022-09-14 2022-09-14 一种零部件装配互动式教学系统和方法
CN202211113311.X 2022-09-14

Publications (1)

Publication Number Publication Date
WO2024055396A1 true WO2024055396A1 (fr) 2024-03-21

Family

ID=85575441

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/129722 WO2024055396A1 (fr) 2022-09-14 2022-11-04 Système et procédé d'enseignement interactif d'assemblage de pièces

Country Status (2)

Country Link
CN (1) CN115841773A (fr)
WO (1) WO2024055396A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116021250B (zh) * 2023-03-29 2023-06-06 清华大学 智能装配系统

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6477437B1 (en) * 1998-11-27 2002-11-05 Sumitomo Wiring Systems, Ltd. Assembly work support system
CN102663916A (zh) * 2012-04-24 2012-09-12 河海大学常州校区 虚拟漫游场景下机电设备模型拆装培训系统及其开发方法
US20160314704A1 (en) * 2015-04-22 2016-10-27 Sap Se Interactive product assembly and repair
CN108630053A (zh) * 2018-08-29 2018-10-09 常州天眼星图光电科技有限公司 一种基于虚拟现实技术的三维动画装配教学方法和装置
CN108646926A (zh) * 2018-08-29 2018-10-12 常州天眼星图光电科技有限公司 机械制造模具虚拟装配培训系统及培训方法
CN108806347A (zh) * 2018-06-13 2018-11-13 福建中科智汇数字科技有限公司 基于vr设备的医疗教学系统
CN113110742A (zh) * 2021-04-14 2021-07-13 杭州杭钢炽橙智能科技有限公司 一种基于slam定位技术的ar多人交互的工业机器人教学系统
CN113205607A (zh) * 2021-05-08 2021-08-03 山西晓雯文化艺术发展有限公司 一种建筑装配的沉浸式虚拟现实互动教学系统及使用方法
CN114444856A (zh) * 2021-12-22 2022-05-06 上海智能制造功能平台有限公司 一种辅助装配系统

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6477437B1 (en) * 1998-11-27 2002-11-05 Sumitomo Wiring Systems, Ltd. Assembly work support system
CN102663916A (zh) * 2012-04-24 2012-09-12 河海大学常州校区 虚拟漫游场景下机电设备模型拆装培训系统及其开发方法
US20160314704A1 (en) * 2015-04-22 2016-10-27 Sap Se Interactive product assembly and repair
CN108806347A (zh) * 2018-06-13 2018-11-13 福建中科智汇数字科技有限公司 基于vr设备的医疗教学系统
CN108630053A (zh) * 2018-08-29 2018-10-09 常州天眼星图光电科技有限公司 一种基于虚拟现实技术的三维动画装配教学方法和装置
CN108646926A (zh) * 2018-08-29 2018-10-12 常州天眼星图光电科技有限公司 机械制造模具虚拟装配培训系统及培训方法
CN113110742A (zh) * 2021-04-14 2021-07-13 杭州杭钢炽橙智能科技有限公司 一种基于slam定位技术的ar多人交互的工业机器人教学系统
CN113205607A (zh) * 2021-05-08 2021-08-03 山西晓雯文化艺术发展有限公司 一种建筑装配的沉浸式虚拟现实互动教学系统及使用方法
CN114444856A (zh) * 2021-12-22 2022-05-06 上海智能制造功能平台有限公司 一种辅助装配系统

Also Published As

Publication number Publication date
CN115841773A (zh) 2023-03-24

Similar Documents

Publication Publication Date Title
WO2024055396A1 (fr) Système et procédé d'enseignement interactif d'assemblage de pièces
CN109242736A (zh) 一种协助老师了解学生的学习情况的方法和系统
CN109032362A (zh) 一种基于vr的教学系统及其控制方法
CN106875767B (zh) 在线学习系统及方法
CN109062884A (zh) 一种交互式微课的控制方法及交互式微课
CN105139713A (zh) 机车及轨道监控装置仿真培训系统
CN110148331A (zh) 一种基于虚拟现实的教学培训平台
CN105844997A (zh) 一种互动电子舞蹈教练系统
CN110533987A (zh) 一种实景交互模拟驾驶系统及方法
WO2020177318A1 (fr) Système d'opération de saccharification de bière artisanale basé sur la réalité virtuelle et procédé
CN108961876B (zh) 一种用于舞蹈在线学习以及教学的网络平台
KR20210032121A (ko) 전력설비 점검용 가상 드론 훈련 시뮬레이션 장치 및 방법
CN112800606A (zh) 数字孪生产线构建方法、系统、电子设备及存储介质
CN111905356A (zh) 适用于棋类游戏教学指导的智能棋具显示方法及系统
CN111905355A (zh) 适用于棋类游戏教学演练的智能棋盘显示方法及系统
CN116843866A (zh) 基于ar设备养殖场智能巡检方法、系统
CN109118848A (zh) 一种基于光通信的多媒体信息系统
Wiesen et al. Learning by Doing-Mobile Robotics in the FH Aachen ROS Summer School.
CN108847063A (zh) 一种投影机器人互动教学方法及其系统
CN110136242B (zh) 无人机模型系统及电子设备
CN113110151A (zh) 一种单兵移动靶控制系统
CN110364047A (zh) 基于声光电技术的虚拟教学系统
CN114973835B (zh) 一种数字化深度沉浸式互动学习模型训练系统及实现方法
CN110444067A (zh) 一种基于虚拟现实的精酿啤酒成品配方的教学系统及方法
CN110264812A (zh) 基于虚拟现实的cip清洗啤酒发酵罐操作系统及方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22958584

Country of ref document: EP

Kind code of ref document: A1