WO2020001464A1 - 模型展现方法及装置、模型提供方法及装置、终端、服务器、系统及存储介质 - Google Patents

模型展现方法及装置、模型提供方法及装置、终端、服务器、系统及存储介质 Download PDF

Info

Publication number
WO2020001464A1
WO2020001464A1 PCT/CN2019/092956 CN2019092956W WO2020001464A1 WO 2020001464 A1 WO2020001464 A1 WO 2020001464A1 CN 2019092956 W CN2019092956 W CN 2019092956W WO 2020001464 A1 WO2020001464 A1 WO 2020001464A1
Authority
WO
WIPO (PCT)
Prior art keywords
point cloud
real
model
identification
maps
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/092956
Other languages
English (en)
French (fr)
Inventor
徐敏
李尼格
于海
彭林
侯战胜
王刚
何志敏
朱亮
鲍兴川
王鹤
张泽浩
韩海韵
周强
徐郁
田文锋
孙辰军
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
State Grid Chongqing Electric Power Co Ltd
Global Energy Interconnection Research Institute Co Ltd
State Grid Corp of China SGCC
Original Assignee
State Grid Chongqing Electric Power Co Ltd
Global Energy Interconnection Research Institute Co Ltd
State Grid Corp of China SGCC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by State Grid Chongqing Electric Power Co Ltd, Global Energy Interconnection Research Institute Co Ltd, State Grid Corp of China SGCC filed Critical State Grid Chongqing Electric Power Co Ltd
Publication of WO2020001464A1 publication Critical patent/WO2020001464A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/251—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving models
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence

Definitions

  • the present application relates to the field of information technology applications, for example, to a model display method and device, a model providing method and device, a terminal, a server, a system, and a storage medium.
  • the operation inspection personnel cannot complete the tasks independently, and is highly dependent on manufacturers. Dangerous situations such as misoperations and misintervals are prone to accidents during the operation, resulting in personal injury and death.
  • the actual operation time of the inspection operation is less than half, and more or even 70% of the time is spent on preparation and confirmation communication.
  • the traditional three-dimensional model display is only a flat display, which requires the user to manually control the wearable helmet device for mobile model interaction. Due to the inconvenience of human-machine interaction of the augmented reality helmet device, the on-site model display is more abstract, the model display is not intuitive, and the data fusion Not in place, it is difficult to assist field personnel to quickly grasp the internal and external structure of the device.
  • the present application provides a model display method and device, a model supply method and device, a terminal, a server, a system, and a storage medium, and solves the problems that the on-site power equipment model display is not intuitive and data fusion is not in place.
  • an embodiment of the present application provides a model display method, including:
  • a reverse algorithm correction is performed on the three-dimensional model after superimposed data, so that the superimposed data is displayed in the opposite direction to the movement direction of the workers on the job site Post 3D model.
  • an embodiment of the present application provides a model providing method, including:
  • the point cloud recognition map includes multiple point cloud recognition maps with different lighting conditions and angles;
  • an embodiment of the present application provides a model display device, including:
  • a sending module configured to send a download identification file request, and obtain multiple point cloud identification maps according to the download identification file request
  • An acquisition module configured to acquire a real-time video stream of a power physical device at a job site, and decomposing the real-time video stream into real-time point cloud data through a point cloud segmentation algorithm;
  • the comparison module is configured to compare the real-time point cloud data with the point cloud data in the multiple point cloud identification maps respectively, determine a point cloud identification map file name that matches the power entity device, and based on the Obtaining a three-dimensional model corresponding to the power entity device with a file name of a point cloud identification map;
  • a calculation module configured to superimpose the real-time operation data of the electric power physical device on the three-dimensional model, and calculate the real-time displacement and rotation coordinate values of workers on the job site in a preset coordinate system;
  • a correction module is set to perform a reverse algorithm correction on the three-dimensional model after superimposing data based on the real-time displacement and rotation coordinate values of the job site workers to achieve a relative inversion with the movement direction of the job site workers
  • the three-dimensional model after displaying the superimposed data is displayed.
  • an embodiment of the present application provides a model providing apparatus, including:
  • the first providing module is configured to receive a download identification file request and return multiple point cloud identification maps based on the download identification file request, wherein the multiple point cloud identification maps include point cloud identification maps of multiple power entity devices
  • the point cloud identification map of each power entity device includes multiple point cloud identification maps with different lighting conditions and angles;
  • the second providing module is configured to receive a model download request and return a three-dimensional model of the power entity device corresponding to the model download request according to the three-dimensional models corresponding to the multiple power entity devices, wherein the model download request is based on The point cloud identification map file name is determined, and the point cloud identification map file name is determined based on the multiple point cloud identification maps and a real-time video stream of a power entity device at the job site.
  • an embodiment of the present application provides a terminal, including:
  • One or more processors are One or more processors;
  • a storage device configured to store one or more programs
  • the one or more programs are executed by the one or more processors, so that the one or more processors implement the foregoing model presentation method.
  • an embodiment of the present application provides a server, including:
  • One or more processors are One or more processors;
  • a storage device configured to store one or more programs
  • the one or more programs are executed by the one or more processors, so that the one or more processors implement the above-mentioned model providing method.
  • an embodiment of the present application provides a system including: a terminal and a server;
  • the terminal is configured to implement the above model display method
  • the server is provided to provide a method for implementing the above model.
  • an embodiment of the present application provides a computer-readable storage medium.
  • the computer-readable storage medium stores a computer program, and the computer program implements the foregoing method when executed by a processor.
  • FIG. 1 is a flowchart of a model display method provided by an embodiment of the present application
  • FIG. 2 is a flowchart of a model providing method according to an embodiment of the present application.
  • FIG. 3 is a technical overview diagram of a model panoramic display provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of head movement posture position collection provided by an embodiment of the present application.
  • Figure 5 is a schematic diagram of the SLAM short-distance estimation displacement algorithm
  • Figure 6 is a schematic diagram of the SLAM short-time rotation angle algorithm
  • FIG. 7 is a schematic structural diagram of a model display device according to an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of a model providing apparatus according to an embodiment of the present application.
  • FIG. 9 is a schematic structural diagram of a terminal according to an embodiment of the present application.
  • FIG. 10 is a schematic structural diagram of a server according to an embodiment of the present application.
  • FIG. 11 is a schematic structural diagram of a system provided by an embodiment of the present application.
  • Environmental intelligent identification is the active identification of on-site operation information, identifying operation targets and analyzing the operation environment through augmented reality technology to achieve automatic acquisition of operation and equipment information; high-precision stereo positioning based on computer vision, and dynamic updating of model and business data coordinates. Realize accurate positioning of models and data. Users can observe 360 degrees of the model from multiple angles by themselves to achieve a clearer and more vivid 3D model display effect; integrate the operating environment and task task scenarios, and then combine information with experience to realize information initiative. The visual push provides accurate and timely auxiliary guidance for operators.
  • SLAM refers to moving from an unknown location in an unknown environment, and positioning itself based on position estimates and maps during the movement. At the same time, an incremental map is built on the basis of its own positioning to achieve autonomous positioning and navigation.
  • SLAM short-distance estimation displacement algorithm The core idea of the short-distance estimation displacement algorithm is to calculate two time points based on the angle between the obstacle surface in front and the vertical line directly in front of the vision, and the maximum distance between the two time points before and after the same laser. Move distance between.
  • SLAM short-time rotation angle algorithm The core idea of the short-time estimation rotation angle algorithm is to use the angle between the forward obstacle plane and the robot's vision sensor measured in a short time to estimate the robot's rotation angle.
  • this application proposes a model display method, which is applied to a terminal and includes:
  • Step 110 Send a download identification file request, and obtain multiple point cloud identification maps according to the download identification file request;
  • Step 120 Obtain a real-time video stream of a power physical device at the job site, and decompose the real-time video stream into real-time point cloud data by using a point cloud segmentation algorithm;
  • Step 130 Compare the real-time point cloud data with the point cloud data in the multiple point cloud recognition maps respectively, determine a file name of the point cloud recognition map that matches the power entity device, and based on the point cloud Obtaining a three-dimensional model corresponding to the power entity device by using the identification map file name;
  • Step 140 superimpose the real-time operation data of the electric entity equipment on the three-dimensional model, and calculate the real-time displacement and rotation coordinate values of the workers on the job site in a preset coordinate system;
  • Step 150 Based on the real-time displacement and rotation coordinate values of the workers on the job site, by performing a reverse algorithm correction on the three-dimensional model after superimposing the data, a reverse display of the movement direction of the workers on the job site is achieved.
  • the three-dimensional model after superimposing data is described.
  • calculating the real-time displacement and coordinate values of the workers on the job site includes:
  • acquiring the three-dimensional model corresponding to the power entity device based on the point cloud identification map file name includes:
  • the method further includes:
  • the superimposing the real-time running data of the power entity device on the three-dimensional model includes:
  • An augmented reality tracking registration algorithm is used to superimpose the three-dimensional model and the real-time operation data of the electric power physical device.
  • the multiple point cloud identification maps include point cloud identification maps of multiple power entity devices, and the point cloud identification map of each power entity device includes multiple point cloud identification maps with different lighting conditions and angles.
  • the three-dimensional model does not only refer to the model, but also includes data related to the model.
  • the terminal may be a separate device connected to the wearable device.
  • the terminal may also be a wearable device, such as a head wearable device.
  • an embodiment of the present application provides a model providing method, which is applied to a server and includes:
  • Step 210 Receive a request for downloading an identification file, and return multiple point cloud identification maps based on the download identification file request, where the multiple point cloud identification maps include point cloud identification maps of multiple power physical devices, and each power The point cloud recognition map of the physical device includes multiple point cloud recognition maps with different lighting conditions and angles;
  • Step 220 Receive a model download request, and return a three-dimensional model of a power entity device corresponding to the model download request according to the three-dimensional models corresponding to the multiple power entity devices, where the model download request is based on a point cloud identification map.
  • the file name is determined, and the file name of the point cloud identification map is determined based on the multiple point cloud identification maps and a real-time video stream of a power entity device at a job site.
  • the manner of obtaining the point cloud identification map of each power entity device includes multiple point cloud identification maps with different lighting conditions and angles includes:
  • the method before the using the point cloud segmentation algorithm to generate multiple point cloud recognition maps with different lighting conditions and angles according to pictures of the power entity device under different lighting conditions and angles, the method further includes:
  • the method further includes:
  • This application uses the augmented reality image recognition technology to realize the rapid identification of the power physical equipment on the job site, and retrieves the three-dimensional panoramic model corresponding to the power physical equipment in real time.
  • a one-to-many association algorithm between power physical devices and point cloud recognition maps is used to realize fast recognition of power physical devices under different angles and different lighting conditions.
  • the smart helmet depth of field camera is used to quickly perform scene scanning and positioning to obtain the trajectory coordinate offset of the user's head movement, so as to dynamically adjust the model's coordinates in the world coordinate system and move the model in the opposite direction as the user moves
  • the user can observe 360 degrees of each angle of the model in all directions through his own movement, to achieve a clearer and more vivid three-dimensional model display effect.
  • This makes the field operators more intuitive and convenient to understand the basic situation of the operation of the physical power equipment, which greatly improves the work efficiency of the field operators.
  • the technical architecture is shown in Figure 3 and includes the following steps:
  • Step 1 Take photos of different lighting conditions and angles at the substation site according to the physical power equipment and upload them to the cloud recognition background server.
  • the server uses the point cloud segmentation algorithm to create a point cloud identification map file based on the device photos, and query the device's unique Numbering, sorting in order of photo shooting time, associating the device with multiple point cloud identification map files.
  • the substation has device A, which took n photos for different lighting conditions and angles, and uploaded it to the background cloud recognition server.
  • the background server queries the database to obtain the unique number of the device as N, and sorts the photos according to the time sequence of the shooting.
  • the cloud identification map files are named N_1, N_2 ... N_n and stored locally on the server.
  • Step 2 When the terminal runs the recognition program, it sends a download identification file request to the server to complete the download of the point cloud identification map file.
  • the real-time video stream data is collected using the helmet of the field operator, and the video stream is decomposed into real-time points by the point cloud segmentation algorithm.
  • the cloud data is compared with the point cloud data in the point cloud recognition diagram of the device block by block. When the difference is small, it is matched, otherwise it is not matched.
  • D (k) be the field real-time point cloud data and the point cloud identification map at block (i, j).
  • the comparison result (result 0 is matched, result 1 is not matched), and N (i, j) is the live real-time point.
  • Cloud data Is the point cloud data in the point cloud recognition map
  • T is the matching difference size threshold
  • the corresponding image model is identified according to the threshold.
  • the matching result of the feature block (i, j) is:
  • the real-time point cloud data in the field is compared with the point cloud data in the point cloud recognition map to obtain the final recognition result.
  • h is the dot matrix height
  • w is the dot matrix width
  • A is the number of standard image dot matrix sets.
  • Step 3 The terminal obtains the file name of the point cloud identification map corresponding to the device to be identified, obtains the unique number of the device by analyzing the file name, and then sends a request to the server to download the three-dimensional model of the device and related ledger data.
  • the file name resolution process is as follows: Assuming that the device to be identified corresponds to the current point cloud identification map file name N_3, only the number N before the file name “_” is parsed, and the content after the “_” is discarded to obtain the true number of the device.
  • N_3 the current point cloud identification map file name
  • the function of a single power entity device corresponding to multiple identification maps on the site is realized, that is, no matter how many identification maps are recognized on the site, the real number of the device can be deduced.
  • Step 4 As shown in Figure 4, after the terminal downloads the 3D model and data of the corresponding device, the real-time point cloud data is obtained through the deep vision camera on the smart helmet, and the point cloud data in the current frame of the video stream and the point cloud data in the previous frame are analyzed.
  • the offset amount is calculated by using the SLAM short-distance estimation displacement algorithm and the SLAM short-time rotation angle algorithm to calculate the real-time displacement and rotation coordinate values of the workers in the field.
  • the core of the SLAM short-distance estimation displacement algorithm is to calculate the moving distance between two time points based on the angle ⁇ between the front object plane and the vertical line directly in front of the user's vision, and the maximum distance between the two time points before and after the same laser.
  • the principle is shown in Figure 5.
  • the core of the SLAM short-time rotation angle algorithm is to use the angle between the forward obstacle plane and the user's visual sensor measured in a short time to estimate the rotation angle of the head.
  • the principle is shown in Figure 6. Assuming that the operator's position was at point O, the plane of the obstacle is a straight line AB, the direction of the head vision sensor is OC, and OA and OB are the two laser ranging paths, the angle ⁇ 1 can be calculated:
  • the direction of the vision sensor is OC ′, OA ′ and OB ′ are the laser ranging paths after rotation.
  • the angle ⁇ 2 can be calculated:
  • the final angle of head rotation in a short time is ⁇ 2 - ⁇ 1 .
  • Step 5 According to the real-time calculation of the displacement and rotation of the on-site operator, the three-dimensional model and the superimposed data displayed by the smart helmet are reversely corrected to achieve the virtual information such as the three-dimensional model and data. Effect, the operator can achieve 360-degree observation of the three-dimensional model and data of the observation device without dead angles.
  • the reverse correction algorithm is as follows: Assume that the real-time displacement and coordinate values before and after rotation of the field operator are (x 0 , y 0 , z 0 ), (x 1 , y 1 , z 1 ), the coordinate values of the three-dimensional model and the data.
  • the correction factor ⁇ ranges from 0.5-1, which can be adjusted manually according to the actual environment on site.
  • the coordinates of the three-dimensional model and data are reversely modified to realize the dynamic adjustment of the three-dimensional model and data following the movement of people, thereby achieving the effect of positioning the virtual information such as the three-dimensional model and data in place.
  • an embodiment of the present application provides a model display device, which is applied to a terminal and includes:
  • the sending module 710 is configured to send a download identification file request, and obtain multiple point cloud identification maps according to the download identification file request;
  • the obtaining module 720 is configured to obtain a real-time video stream of a power entity device at a job site, and decompose the real-time video stream into real-time point cloud data by using a point cloud segmentation algorithm;
  • the comparison module 730 is configured to compare the real-time point cloud data with the point cloud data in the multiple point cloud identification maps respectively, determine a file name of the point cloud identification map that matches the power entity device, and based on the Obtaining a three-dimensional model corresponding to the power entity device by using the point cloud recognition map file name;
  • the calculation module 740 is configured to superimpose the real-time running data of the electric solid equipment on the three-dimensional model, and calculate the real-time displacement and rotation coordinate values of the workers on the job site in a preset coordinate system;
  • a correction module 750 is configured to perform a reverse algorithm correction on the 3D model of the superimposed data based on the real-time displacement and rotation coordinate values of the job site workers to achieve a relative inversion with the movement direction of the job site workers. Show the three-dimensional model after the superimposed data.
  • the calculation module 740 is configured to: obtain the offset between the point cloud data of the current frame and the point cloud data of the previous frame of the video stream, and construct a SLAM short-distance estimation displacement algorithm and SLAM using real-time positioning and maps.
  • the short-time rotation angle algorithm calculates the real-time displacement and coordinate values of the on-site workers.
  • the comparison module 730 is configured to:
  • the apparatus further includes a photographing module configured to take and upload pictures of the power entity device under different lighting conditions and angles in advance.
  • the calculation module 740 is configured to:
  • An augmented reality tracking registration algorithm is used to superimpose the three-dimensional model and the real-time operation data of the electric power physical device.
  • the multiple point cloud identification maps include point cloud identification maps of multiple power entity devices, and the point cloud identification map of each power entity device includes multiple point cloud identification maps with different lighting conditions and angles.
  • an embodiment of the present application provides a model providing apparatus, including:
  • the first providing module 810 is configured to receive a download identification file request and return multiple point cloud identification maps based on the download identification file request, wherein the multiple point cloud identification maps include point cloud identifications of multiple power entity devices Figure, the point cloud identification map of each power entity device includes multiple point cloud identification maps with different lighting conditions and angles;
  • the second providing module 820 is configured to receive a model download request, and return a three-dimensional model of a power entity device corresponding to the model download request according to the three-dimensional models corresponding to the multiple power entity devices, wherein the model download request It is determined according to the point cloud identification map file name, and the point cloud identification map file name is determined based on the multiple point cloud identification maps and a real-time video stream of a power entity device at the job site.
  • the apparatus further includes a first binding module configured to: receive pictures of the power entity device under different lighting conditions and angles; and use a point cloud segmentation algorithm according to the power entity device under different lighting conditions.
  • the pictures under the conditions and angles generate multiple point cloud recognition maps with different lighting conditions and angles; and associate and bind the multiple point cloud recognition maps with different lighting conditions and angles with the power entity device.
  • the apparatus further includes a sorting module configured to sort pictures of the power entity device under different lighting conditions and angles according to the shooting time.
  • the apparatus further includes a second binding module configured to: query a unique number of the power device in the system; and compare the unique number with the plurality of point clouds of different lighting conditions and angles Recognition graphs are associated and bound.
  • an embodiment of the present application provides a terminal, including:
  • One or more first processors 910 are One or more first processors 910;
  • a first storage device 920 configured to store one or more programs
  • the one or more programs are executed by the one or more first processors 910, so that the one or more first processors 910 implement the model display method described in any one of the foregoing embodiments.
  • an embodiment of the present application provides a server, including:
  • One or more second processors 1010 are One or more second processors 1010;
  • a second storage device 1020 configured to store one or more programs
  • the one or more programs are executed by the one or more second processors 1010, so that the one or more second processors 1010 implement the model providing method described in any one of the above embodiments.
  • an embodiment of the present application provides a system, including: a terminal 1110 and a server 1120;
  • the terminal 1110 is configured to implement the model display method described in any one of the foregoing embodiments;
  • the server 1120 is configured to implement the model providing method according to any one of the foregoing embodiments.
  • an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the implementation of any one of the foregoing embodiments is implemented. method.
  • Storage medium any one or more types of memory devices or storage devices.
  • the foregoing storage medium may be a volatile memory (for example, Random-Access Memory (RAM); or a non-volatile memory (for example, read-only) Read-only memory (ROM), flash memory, hard disk drive (HDD) or solid-state drive (SSD); or a combination of the above types of memory Provide instructions and data.
  • RAM Random-Access Memory
  • ROM Read-only memory
  • flash memory flash memory
  • HDD hard disk drive
  • SSD solid-state drive
  • the above storage medium may further include: a compact disc read-only memory (CD-ROM), a floppy disk or a magnetic tape device; a computer system memory or a random access memory such as a dynamic random access memory (Dynamic Random Access Memory, DRAM), Double Rate Random Access Memory (DDR Random Access Memory, DDR RAM), Static Random Access Memory (Static Random Access Memory, SRAM), Extended Data Output Random Access Memory (Extended Data Output Random Access Memory (EDO, RAM), Rambus Random Access Memory (Rambus RAM), etc .; non-volatile memory, such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory Components, etc.
  • the storage medium may further include other types of memory or a combination thereof.
  • the above processor may be an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), or a Programmable Logic Device (Programmable Logic). Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), controller, microcontroller, and microprocessor.
  • ASIC Application Specific Integrated Circuit
  • DSP Digital Signal Processor
  • DSPD Digital Signal Processing Device
  • PLD Programmable Logic Device
  • FPGA Field Programmable Gate Array
  • CPU Central Processing Unit
  • controller microcontroller, and microprocessor.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Graphics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Geometry (AREA)
  • Image Analysis (AREA)
  • Processing Or Creating Images (AREA)

Abstract

本文公开了一种模型展现方法，包括：获取多幅点云识别图；获取作业现场的电力实体设备的实时视频流，通过点云分割算法将实时视频流分解为实时点云数据；根据实时点云数据与多幅点云识别图中的点云数据的比较结果，获取与电力实体设备对应的三维模型；在三维模型上叠加电力实体设备实时运行数据，并计算出预设坐标系下作业现场工作人员的实时位移和旋转后的坐标值；基于作业现场工作人员的实时位移和旋转后的坐标值，通过对叠加数据后的三维模型进行反向算法修正，实现与作业现场工作人员的移动方向相对反向展现叠加数据后的三维模型。本文还公开了一种模型展现装置，模型提供方法及装置、终端、服务器、系统及存储介质。

Description

模型展现方法及装置、模型提供方法及装置、终端、服务器、系统及存储介质
本申请要求在2018年6月26日提交中国专利局、申请号为201810674183.3的中国专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本申请涉及信息技术应用领域，例如涉及一种模型展现方法及装置、模型提供方法及装置、终端、服务器、系统及存储介质。
背景技术
传统电力巡检作业方式下，由于电力设备操作趋于繁杂，组件逻辑不可视，例如智能站的高度集成模块和更复杂的逻辑关系，运检人员无法独立完成任务，对厂家依赖度高，在作业过程中易发生误操作、误入间隔等危险情况，造成人员伤亡事件，同时巡检作业实际操作时间不足一半，花费较多甚至70％的时间在准备工作和确认沟通上。传统的三维模型展现仅是平面的展示，需要用户手动控制可穿戴头盔设备进行模型的移动交互，由于增强现实头盔设备人机交互的不便，现场模型的展现较为抽象，模型展示不直观，数据融合不到位，难以辅助现场人员快捷掌握设备的内外部构造。
发明内容
本申请提供一种模型展现方法及装置、模型提供方法及装置、终端、服务器、系统及存储介质，解决了现场电力设备模型展示不直观，数据融合不到位的问题。
在一实施例中，本申请实施例提供了一种模型展现方法，包括：
发送下载识别文件请求，根据所述下载识别文件请求获取多幅点云识别图；
获取作业现场的电力实体设备的实时视频流，通过点云分割算法将所述实时视频流分解为实时点云数据；
将所述实时点云数据与所述多幅点云识别图中的点云数据分别进行比较，确定与所述电力实体设备匹配的点云识别图文件名，并基于所述点云识别图文 件名获取与所述电力实体设备对应的三维模型；
在所述三维模型上叠加所述电力实体设备实时运行数据，并计算出预设坐标系下作业现场工作人员的实时位移和旋转后的坐标值；
基于所述作业现场工作人员的实时位移和旋转后的坐标值，通过对叠加数据后的三维模型进行反向算法修正，实现与所述作业现场工作人员的移动方向相对反向展现所述叠加数据后的三维模型。
在一实施例中，本申请实施例提供了一种模型提供方法，包括：
接收下载识别文件请求，并基于所述下载识别文件请求返回多幅点云识别图，其中，所述多幅点云识别图包括多个电力实体设备的点云识别图，每个电力实体设备的点云识别图包括多幅不同光照条件和角度的点云识别图；
接收模型下载请求，根据所述多个电力实体设备分别对应的三维模型，返回与所述模型下载请求对应的电力实体设备的三维模型，其中，所述模型下载请求根据点云识别图文件名确定，所述点云识别图文件名基于所述多幅点云识别图和作业现场的电力实体设备的实时视频流确定。
在一实施例中，本申请实施例提供了一种模型展现装置，包括：
发送模块，设置为发送下载识别文件请求，根据所述下载识别文件请求获取多幅点云识别图；
获取模块，设置为获取作业现场的电力实体设备的实时视频流，通过点云分割算法将所述实时视频流分解为实时点云数据；
比较模块，设置为将所述实时点云数据与所述多幅点云识别图中的点云数据分别进行比较，确定与所述电力实体设备匹配的点云识别图文件名，并基于所述点云识别图文件名获取与所述电力实体设备对应的三维模型；
计算模块，设置为在所述三维模型上叠加所述电力实体设备实时运行数据，并计算出预设坐标系下作业现场工作人员的实时位移和旋转后的坐标值；
修正模块，设置为基于所述作业现场工作人员的实时位移和旋转后的坐标值，通过对叠加数据后的三维模型进行反向算法修正，实现与所述作业现场工作人员的移动方向相对反向展现所述叠加数据后的三维模型。
在一实施例中，本申请实施例提供了一种模型提供装置，包括：
第一提供模块，设置为接收下载识别文件请求，并基于所述下载识别文件请求返回多幅点云识别图，其中，所述多幅点云识别图包括多个电力实体设备的点云识别图，每个电力实体设备的点云识别图包括多幅不同光照条件和角度的点云识别图；
第二提供模块，设置为接收模型下载请求，根据所述多个电力实体设备分别对应的三维模型，返回与所述模型下载请求对应的电力实体设备的三维模型，其中，所述模型下载请求根据点云识别图文件名确定，所述点云识别图文件名基于所述多幅点云识别图和作业现场的电力实体设备的实时视频流确定。
在一实施例中，本申请实施例提供了一种终端，包括：
一个或多个处理器；
存储装置，设置为存储一个或多个程序；
所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现上述模型展现方法。
在一实施例中，本申请实施例提供了一种服务器，包括：
一个或多个处理器；
存储装置，设置为存储一个或多个程序；
所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现上述模型提供方法。
在一实施例中，本申请实施例提供了一种系统，包括：终端和服务器；
所述终端设置为实现上述模型展现方法；
所述服务器设置为实现上述模型提供方法。
在一实施例中，本申请实施例提供了一种计算机可读存储介质，所述计算机可读存储介质上存储有计算机程序，所述计算机程序被处理器执行时实现上述的方法。
附图说明
图1是本申请实施例提供的一种模型展现方法的流程图；
图2是本申请实施例提供的一种模型提供方法的流程图；
图3是本申请实施例提供的模型全景展现技术架构图；
图4是本申请实施例提供的头部运动姿态位置采集原理图；
图5是SLAM短距离估算位移算法原理图；
图6是SLAM短时间旋转角度算法原理图；
图7是本申请实施例提供的一种模型展现装置的结构示意图；
图8是本申请实施例提供的一种模型提供装置的结构示意图；
图9是本申请实施例提供的一种终端的结构示意图；
图10是本申请实施例提供的一种服务器的结构示意图；
图11是本申请实施例提供的一种系统的结构示意图。
具体实施方式
下面结合附图对本申请的具体实施方式作进一步的详细说明。
以下描述和附图充分地示出本申请的具体实施方案，以使本领域的技术人员能够实践它们。其他实施方案可以包括结构的、逻辑的、电气的、过程的以及其他的改变。实施例仅代表可能的变化。除非明确要求，否则单独的组件和功能是可选的，并且操作的顺序可以变化。一些实施方案的部分和特征可以被包括在或替换其他实施方案的部分和特征。本申请的实施方案的范围包括权利要求书的整个范围，以及权利要求书的所有可获得的等同物。在本文中，本申请的这些实施方案可以被单独地或总地用术语“申请”来表示，这仅仅是为了方便，并且如果事实上公开了超过一个的申请，不是要自动地限制该应用的范围为任何单个申请或构思。
随着信息技术(Information Technology，IT)的发展，利用多种传感技术和高效的图像处理算法，与可穿戴技术相结合，无需在变电站额外部署基站或者传感器，通过佩戴穿戴设备在变电站直接扫描环境即可形成三维模型，并可基 于该模型的高精度定位和经验知识固化，为作业人员开展相关作业任务提供主动安全防护和作业任务指导。视觉即时定位与地图构建(simultaneous localization and mapping，SLAM)可以解决传感器与环境均未知情况下的同时定位与地图创建问题。基于深度视觉的SLAM算法能够扫描创建全局环境三维地图也能准确获取用户的头部细微运动姿态变化量。
环境智能识别是对现场作业信息主动辨识，通过增强现实技术识别作业目标、分析作业环境，实现作业及设备信息的自动获取；基于计算机视觉高精度立体定位，实现模型及业务数据的坐标动态更新，实现模型及数据的精确定位，用户可以通过自身移动360度全方位观察模型多个角度，实现更为清晰生动的三维模型展现效果；综合作业环境和作业任务情景，进而与经验知识结合实现信息主动的可视化推送，提供作业人员准确、及时的辅助引导。
本申请用到的技术术语说明如下：
SLAM是指在未知环境中从一个未知位置开始移动，在移动过程中根据位置估计和地图进行自身定位，同时在自身定位的基础上建造增量式地图，实现自主定位和导航。
SLAM短距离估算位移算法：短距离估算位移算法的核心思想是根据前方障碍物平面与视觉正前方的垂直线的夹角，以及同一束激光前后两个时间点的最大距离来计算两个时间点之间的移动距离。
SLAM短时间旋转角度算法：短时间估算旋转角度算法的核心思想是利用短时间内测得的前方障碍物平面与机器人视觉传感器的夹角，来估算机器人的旋转角度。
在一实施例中，如图1所示，本申请提出了一种模型展现方法，应用于终端，包括：
步骤110，发送下载识别文件请求，根据所述下载识别文件请求获取多幅点云识别图；
步骤120，获取作业现场的电力实体设备的实时视频流，通过点云分割算法将所述实时视频流分解为实时点云数据；
步骤130，将所述实时点云数据与所述多幅点云识别图中的点云数据分别进行比较，确定与所述电力实体设备匹配的点云识别图文件名，并基于所述点云识别图文件名获取与所述电力实体设备对应的三维模型；
步骤140，在所述三维模型上叠加所述电力实体设备实时运行数据，并计算出预设坐标系下作业现场工作人员的实时位移和旋转后的坐标值；
步骤150，基于所述作业现场工作人员的实时位移和旋转后的坐标值，通过对叠加数据后的三维模型进行反向算法修正，实现与所述作业现场工作人员的移动方向相对反向展现所述叠加数据后的三维模型。
在一实施例中，所述计算出作业现场工作人员的实时位移和旋转后的坐标值，包括：
获取所述视频流的当前帧点云数据和上一帧点云数据的偏移量，利用即时定位与地图构建SLAM短距离估算位移算法和SLAM短时间旋转角度算法计算出所述现场作业人员的实时位移和旋转后的坐标值。
在一实施例中，基于所述点云识别图文件名获取与所述电力实体设备对应的三维模型，包括：
通过对所述点云识别图文件名解析获取所述电力实体设备的唯一编号；
发送模型下载请求，并获取所述模型下载请求对应的三维模型；其中，所述模型下载请求中携带所述电力实体设备的唯一编号。
在一实施例中，该方法还包括：
预先拍摄并上传所述电力实体设备在不同光照条件和角度下的图片。
在一实施例中，所述在所述三维模型上叠加所述电力实体设备实时运行数据，包括：
利用增强现实跟踪注册算法将所述三维模型和所述电力实体设备实时运行数据进行叠加。
在一实施例中，所述多幅点云识别图包括多个电力实体设备的点云识别图，每个电力实体设备的点云识别图包括多幅不同光照条件和角度的点云识别图。
在一实施例中，三维模型不仅仅指模型，还包括与模型相关的数据。
在一实施例中，终端可以是一个单独的设备，与可穿戴设备连接。终端也可以是可穿戴设备，例如头部可穿戴设备。
如图2所示，本申请实施例提供了一种模型提供方法，应用于服务器，包括：
步骤210、接收下载识别文件请求，并基于所述下载识别文件请求返回多幅点云识别图，其中，所述多幅点云识别图包括多个电力实体设备的点云识别图，每个电力实体设备的点云识别图包括多幅不同光照条件和角度的点云识别图；
步骤220、接收模型下载请求，根据所述多个电力实体设备分别对应的三维模型，返回与所述模型下载请求对应的电力实体设备的三维模型，其中，所述模型下载请求根据点云识别图文件名确定，所述点云识别图文件名基于所述多幅点云识别图和作业现场的电力实体设备的实时视频流确定。
在一实施例中，获取所述每个电力实体设备的点云识别图包括多幅不同光照条件和角度的点云识别图的方式包括：
接收所述电力实体设备在不同光照条件和角度下的图片；
利用点云分割算法根据所述电力实体设备在不同光照条件和角度下的图片 生成多幅不同光照条件和角度的点云识别图；
将所述多幅不同光照条件和角度的点云识别图与所述电力实体设备进行关联绑定。
在一实施例中，在所述利用点云分割算法根据所述电力实体设备在不同光照条件和角度下的图片生成多幅不同光照条件和角度的点云识别图之前，还包括：
根据拍摄时间，对所述电力实体设备在不同光照条件和角度下的图片顺序排序。
在一实施例中，该方法还包括：
查询所述电力设备在系统内的唯一编号；
将所述唯一编号与所述多幅不同光照条件和角度的点云识别图进行关联绑定。
本申请利用增强现实图像识别技术实现对作业现场电力实体设备的快速识别，实时调取对应电力实体设备的三维全景模型。利用电力实体设备与点云识别图一对多关联算法实现电力实体设备多角度不同光照条件下的快速识别。借助计算机SLAM算法利用智能头盔景深摄像头快速进行场景扫描定位，获取用户头部运动的轨迹坐标偏移量，从而动态调整模型在世界坐标系内的坐标，将模型随着用户移动而反方向移动，从而达到模型固定在某个位置的效果，利用该方法用户可以通过自身移动360度全方位观察模型每个角度，实现更为清晰生动的三维模型展现效果，通过将现场电力设备运行数据的实时叠加，使得现场作业人员更为直观方便的了解电力实体设备运行的基本情况，大幅提高现场作业人员工作效率。技术架构如图3所示，包括下述步骤：
步骤一：在变电站现场根据电力实体设备拍摄不同光照条件和角度的照片， 上传至云识别后台服务器，服务器利用点云分割算法根据设备照片制作点云识别图文件，并查询设备在系统内唯一的编号，按照片拍摄时间顺序排序，将设备与多张点云识别图文件进行关联绑定。例如变电站有设备A，针对不同光照条件和角度拍摄了n张照片，上传至后台云识别服务器，后台服务器查询数据库获取该设备唯一编号为N，按照片拍摄时间顺序排序，分别将该设备的点云识别图文件命名为N_1,N_2……N_n，保存在服务器本地。
步骤二：终端运行识别程序时，向服务器发送下载识别文件请求，完成点云识别图文件的下载，利用采集现场作业人员头盔采集实时视频流数据，通过点云分割算法将视频流分解为实时点云数据，并与设备点云识别图中的点云数据逐块进行比对，当差别较小时为匹配，否则为不匹配。设D(k)为现场实时点云数据与点云识别图在块(i，j)处匹配比较结果(结果0为匹配，结果1为不匹配)，N(i，j)为现场实时点云数据，
Figure PCTCN2019092956-appb-000001
为点云识别图中的点云数据，T为匹配差别大小阈值，根据阈值识别出对应的图像模型，特征块(i,j)匹配结果为:
Figure PCTCN2019092956-appb-000002
将现场实时点云数据与点云识别图中的点云数据进行完整比较后，得到最终识别结果
Figure PCTCN2019092956-appb-000003
其中h是点阵高度，w是点阵宽度，A是标准图像点阵集的个数，最终确定待识别设备的点云识别图文件。
步骤三：终端获取待识别设备对应的点云识别图文件名，通过对文件名的解析获取设备的唯一编号，进而向服务器发送下载设备三维模型及相关台账数据的请求。文件名解析过程如下：假设待识别设备对应当前点云识别图文件名为N_3，则只解析文件名“_”前的编号N，放弃“_”后的内容，从而获取设 备的真实编号。通过该文件名的解析，实现了现场单个电力实体设备对应多个识别图的功能，即无论现场识别到设备的第几个识别图，均能倒推出设备的真实编号。
步骤四：如图4所示，终端下载对应设备的三维模型及数据后，通过智能头盔上的深度视觉摄像头获取点云实时数据，分析视频流当前帧中点云数据和上一帧点云数据的偏移量，利用SLAM短距离估算位移算法和SLAM短时间旋转角度算法计算出现场作业人员的实时位移和旋转后的坐标值。
其中，SLAM短距离估算位移算法核心是根据前方物体平面与用户视觉正前方的垂直线的夹角α以及同一束激光前后两个时间点的最大距离来计算两个时间点之间的移动距离，原理如图5所示。
SLAM短时间旋转角度算法核心是利用短时间内测得的前方障碍物平面与用户视觉传感器的夹角来估算头部的旋转角度，原理如图6所示。假设上个时间点作业人员位置在O点，障碍物平面为直线AB，头部视觉传感器的正前方方向为OC，OA、OB为两束激光测距的路线，则可计算出角α 1:
Figure PCTCN2019092956-appb-000004
当头部转过一定角度后，视觉传感器正前方方向为OC′，OA′和OB′为旋转后的激光测距路线，同理可计算出角α 2:
Figure PCTCN2019092956-appb-000005
最终短时间头部旋转的角度即为α 2-α 1。
步骤五：根据实时计算出的现场作业人员位移和旋转后的坐标值，对智能头盔显示出的三维模型和叠加数据进行反向算法修正，从而实现将三维模型和数据这些虚拟信息定在原地的效果，作业人员可以360度无死角的观察设备三 维模型和数据，达到形象生动的观察效果。反向修正算法如下：假设现场作业人员的实时位移和旋转前后的坐标值分别为(x 0，y 0，z 0)，(x 1，y 1，z 1)，三维模型和数据的坐标值为(a 0，b 0，c 0)，其修正后的坐标值为(a 1，b 1，c 1)，修正因子为α，则反向修正后三维模型和数据的坐标值：
(a 1，b 1，c 1)＝((a 0-α*(x 0-x 1))，(a 0-α*(x 0-x 1))，(a 0-α*(x 0-x 1)))
其中修正因子α取值范围为0.5-1，可根据现场实际环境进行手动调整。通过对三维模型和数据的坐标反向修正，实现三维模型和数据跟随人员的移动而动态调整，从而实现将三维模型和数据这些虚拟信息定在原地的效果。
在一实施例中，如图7所示，本申请实施例提供一种模型展现装置，应用于终端，包括：
发送模块710，设置为发送下载识别文件请求，根据所述下载识别文件请求获取多幅点云识别图；
获取模块720，设置为获取作业现场的电力实体设备的实时视频流，通过点云分割算法将所述实时视频流分解为实时点云数据；
比较模块730，设置为将所述实时点云数据与所述多幅点云识别图中的点云数据分别进行比较，确定与所述电力实体设备匹配的点云识别图文件名，并基于所述点云识别图文件名获取与所述电力实体设备对应的三维模型；
计算模块740，设置为在所述三维模型上叠加所述电力实体设备实时运行数据，并计算出预设坐标系下作业现场工作人员的实时位移和旋转后的坐标值；
修正模块750，设置为基于所述作业现场工作人员的实时位移和旋转后的坐标值，通过对叠加数据后的三维模型进行反向算法修正，实现与所述作业现场工作人员的移动方向相对反向展现所述叠加数据后的三维模型。在一实施例中， 计算模块740是设置为：获取所述视频流的当前帧点云数据和上一帧点云数据的偏移量，利用即时定位与地图构建SLAM短距离估算位移算法和SLAM短时间旋转角度算法计算出所述现场作业人员的实时位移和旋转后的坐标值。
在一实施例中，比较模块730是设置为：
通过对所述点云识别图文件名解析获取所述电力实体设备的唯一编号；
发送模型下载请求，并获取所述模型下载请求对应的三维模型；其中，所述模型下载请求中携带所述电力实体设备的唯一编号。
在一实施例中，该装置还包括拍摄模块，设置为预先拍摄并上传所述电力实体设备在不同光照条件和角度下的图片。
在一实施例中，计算模块740是设置为：
利用增强现实跟踪注册算法将所述三维模型和所述电力实体设备实时运行数据进行叠加。
在一实施例中，所述多幅点云识别图包括多个电力实体设备的点云识别图，每个电力实体设备的点云识别图包括多幅不同光照条件和角度的点云识别图。
在一实施例中，如图8所示，本申请实施例提供一种模型提供装置，包括：
第一提供模块810，设置为接收下载识别文件请求，并基于所述下载识别文件请求返回多幅点云识别图，其中，所述多幅点云识别图包括多个电力实体设备的点云识别图，每个电力实体设备的点云识别图包括多幅不同光照条件和角度的点云识别图；
第二提供模块820，设置为接收模型下载请求，根据所述多个电力实体设备分别对应的三维模型，返回与所述模型下载请求对应的电力实体设备的三维模型，其中，所述模型下载请求根据点云识别图文件名确定，所述点云识别图文件名基于所述多幅点云识别图和作业现场的电力实体设备的实时视频流确定。
在一实施例中，所述装置还包括第一绑定模块，设置为：接收所述电力实体设备在不同光照条件和角度下的图片；利用点云分割算法根据所述电力实体设备在不同光照条件和角度下的图片生成多幅不同光照条件和角度的点云识别图；将所述多幅不同光照条件和角度的点云识别图与所述电力实体设备进行关联绑定。
在一实施例中，所述装置还包括：排序模块，设置为根据拍摄时间，对所述电力实体设备在不同光照条件和角度下的图片顺序排序。
在一实施例中，所述装置还包括第二绑定模块，设置为：查询所述电力设备在系统内的唯一编号；将所述唯一编号与所述多幅不同光照条件和角度的点云识别图进行关联绑定。
在一实施例中，如图9所示，本申请实施例提供一种终端，包括：
一个或多个第一处理器910；
第一存储装置920，设置为存储一个或多个程序；
所述一个或多个程序被所述一个或多个第一处理器910执行，使得所述一个或多个第一处理器910实现上述任一实施例所述的模型展示方法。
在一实施例中。如图10所示，本申请实施例提供一种服务器，包括：
一个或多个第二处理器1010；
第二存储装置1020，设置为存储一个或多个程序；
所述一个或多个程序被所述一个或多个第二处理器1010执行，使得所述一个或多个第二处理器1010实现上述任一实施例所述的模型提供方法。
在一实施例中，如图11所示，本申请实施例提供一种系统，包括：终端1110和服务器1120；
所述终端1110设置为实现上述任一实施例所述的模型展示方法；
所述服务器1120设置为实现上述任一实施例所述的模型提供方法。
在一实施例中，本申请实施例提供一种计算机可读存储介质，所述计算机可读存储介质上存储有计算机程序，所述计算机程序被处理器执行时实现上述任一实施例所述的方法。
存储介质—任何的一种或多种类型的存储器设备或存储设备。在实际应用中，上述的存储介质可以是易失性存储器(volatile memory)，例如随机存取存储器(Random-Access Memory，RAM)；或者非易失性存储器(non-volatile memory)，例如只读存储器(Read-Only Memory，ROM)，快闪存储器(flash memory)，硬盘(Hard Disk Drive，HDD)或固态硬盘(Solid-State Drive，SSD)；或者上述种类的存储器的组合，并向处理器提供指令和数据。
上述存储介质还可以包括：光盘只读存储器(Compact Disc Read-Only Memory，CD-ROM)、软盘或磁带装置；计算机系统存储器或随机存取存储器，诸如动态随机存取存储器(Dynamic Random Access Memory，DRAM)、双倍速率随机存取存储器(Double Data Rate Random Access Memory，DDR RAM)、静态随机存取存储器(Static Random-Access Memory，SRAM)、扩展数据输出随机存取存储器(Extended Data Output Random Access Memory，EDO RAM)，兰巴斯随机存取存储器(Rambus Random Access Memory，Rambus RAM)等；非易失性存储器，诸如闪存、磁介质(例如硬盘或光存储)；寄存器或其它相似类型的存储器元件等。存储介质可以还包括其它类型的存储器或其组合。
上述处理器可以为特定用途集成电路(Application Specific Integrated Circuit，ASIC)、数字信号处理器(Digital Signal Processor，DSP)、数字信号处理装置(Digital Signal Processing Device，DSPD)、可编程逻辑装置(Programmable Logic Device，PLD)、现场可编程门阵列(Field Programmable Gate Array，FPGA)、 中央处理器(Central Processing Unit，CPU)、控制器、微控制器、微处理器中的至少一种。
本申请提供的技术方案具有的有益效果包括：
(1)针对关键设备进行特征图像采集，创新地将增强现实识别算法引入变电站实体设备识别中，通过图像实时识别并查询特征图像对应的设备三维模型，使得现场作业人员看到实体设备便能快速获取设备的三维模型及实时运行数据。
(2)创新地引入增强现实识别图文件与实体设备一对多的匹配关联算法，实现实体设备在不同光照条件下的多角度快速识别，提高了传统实体设备与识别图一对一关联的识别率，大幅提高增强现实算法的实用度。
(3)创新地将计算机即时建模定位技术引入变电站进行应用，对变电站电力设备进行快速扫描建模，通过SLAM识别算法获取用户精确的位移信息。计算机即时建模定位算法中的点云识别算法用于头部姿态偏移量的计算，通过点云数据的识别，分析出人员头部运动的坐标偏移量，通过对偏移量的处理，对三维模型进行反向动态修正，达到三维模型全景展现的效果。
(4)创新地将增强现实识别技术与计算机即时定位技术相结合，通过获取现场用户的实时位移数据实现设备数据及模型的真实环境坐标固定，通过增强现实技术对现场电力设备进行识别下载对应的设备数据及模型信息。用户通过自身移动360度全方位观察模型每个角度，实现更为清晰生动的三维模型展现效果。

Claims (16)

  1. 一种模型展现方法，包括：
    发送下载识别文件请求，根据所述下载识别文件请求获取多幅点云识别图；
    获取作业现场的电力实体设备的实时视频流，通过点云分割算法将所述实时视频流分解为实时点云数据；
    将所述实时点云数据与所述多幅点云识别图中的点云数据分别进行比较，确定与所述电力实体设备匹配的点云识别图文件名，并基于所述点云识别图文件名获取与所述电力实体设备对应的三维模型；
    在所述三维模型上叠加所述电力实体设备实时运行数据，并计算出预设坐标系下作业现场工作人员的实时位移和旋转后的坐标值；
    基于所述作业现场工作人员的实时位移和旋转后的坐标值，通过对叠加数据后的三维模型进行反向算法修正，实现与所述作业现场工作人员的移动方向相对反向展现所述叠加数据后的三维模型。
  2. 根据权利要求1所述的方法，其中，所述计算出作业现场工作人员的实时位移和旋转后的坐标值，包括：
    获取所述视频流的当前帧点云数据和上一帧点云数据的偏移量，利用即时定位与地图构建SLAM短距离估算位移算法和SLAM短时间旋转角度算法计算出所述现场作业人员的实时位移和旋转后的坐标值。
  3. 根据权利要求1所述的方法，其中，基于所述点云识别图文件名获取与所述电力实体设备对应的三维模型，包括：
    通过对所述点云识别图文件名解析获取所述电力实体设备的唯一编号；
    发送模型下载请求，并获取所述模型下载请求对应的三维模型；其中，所述模型下载请求中携带所述电力实体设备的唯一编号。
  4. 根据权利要求1所述的方法，还包括：
    预先拍摄并上传所述电力实体设备在不同光照条件和角度下的图片。
  5. 根据权利要求1所述的方法，其中，所述在所述三维模型上叠加所述电力实体设备实时运行数据，包括
    利用增强现实跟踪注册算法将所述三维模型和所述电力实体设备实时运行 数据进行叠加。
  6. 根据权利要求1所述的方法，其中，所述多幅点云识别图包括多个电力实体设备的点云识别图，每个电力实体设备的点云识别图包括多幅不同光照条件和角度的点云识别图。
  7. 一种模型提供方法，包括：
    接收下载识别文件请求，并基于所述下载识别文件请求返回多幅点云识别图，其中，所述多幅点云识别图包括多个电力实体设备的点云识别图，每个电力实体设备的点云识别图包括多幅不同光照条件和角度的点云识别图；
    接收模型下载请求，根据所述多个电力实体设备分别对应的三维模型，返回与所述模型下载请求对应的电力实体设备的三维模型，其中，所述模型下载请求根据点云识别图文件名确定，所述点云识别图文件名基于所述多幅点云识别图和作业现场的电力实体设备的实时视频流确定。
  8. 根据权利要求7所述的方法，其中，获取所述每个电力实体设备的点云识别图包括多幅不同光照条件和角度的点云识别图的方式包括：
    接收所述电力实体设备在不同光照条件和角度下的图片；
    利用点云分割算法根据所述电力实体设备在不同光照条件和角度下的图片生成多幅不同光照条件和角度的点云识别图；
    将所述多幅不同光照条件和角度的点云识别图与所述电力实体设备进行关联绑定。
  9. 根据权利要求7所述的方法，在所述利用点云分割算法根据所述电力实体设备在不同光照条件和角度下的图片生成多幅不同光照条件和角度的点云识别图之前，还包括：
    根据拍摄时间，对所述电力实体设备在不同光照条件和角度下的图片顺序排序。
  10. 根据权利要求7或8所述的方法，还包括：
    查询所述电力设备在系统内的唯一编号；
    将所述唯一编号与所述多幅不同光照条件和角度的点云识别图进行关联绑定。
  11. 一种模型展现装置，包括：发送模块，设置为发送下载识别文件请求，根据所述下载识别文件请求获取多幅点云识别图；
    获取模块，设置为获取作业现场的电力实体设备的实时视频流，通过点云分割算法将所述实时视频流分解为实时点云数据；
    比较模块，设置为将所述实时点云数据与所述多幅点云识别图中的点云数据分别进行比较，确定与所述电力实体设备匹配的点云识别图文件名，并基于所述点云识别图文件名获取与所述电力实体设备对应的三维模型；
    计算模块，设置为在所述三维模型上叠加所述电力实体设备实时运行数据，并计算出预设坐标系下作业现场工作人员的实时位移和旋转后的坐标值；
    修正模块，设置为基于所述作业现场工作人员的实时位移和旋转后的坐标值，通过对叠加数据后的三维模型进行反向算法修正，实现与所述作业现场工作人员的移动方向相对反向展现所述叠加数据后的三维模型。
  12. 一种模型提供装置，包括：
    第一提供模块，设置为接收下载识别文件请求，并基于所述下载识别文件请求返回多幅点云识别图，其中，所述多幅点云识别图包括多个电力实体设备的点云识别图，每个电力实体设备的点云识别图包括多幅不同光照条件和角度的点云识别图；
    第二提供模块，设置为接收模型下载请求，根据所述多个电力实体设备分别对应的三维模型，返回与所述模型下载请求对应的电力实体设备的三维模型，其中，所述模型下载请求根据点云识别图文件名确定，所述点云识别图文件名基于所述多幅点云识别图和作业现场的电力实体设备的实时视频流确定。
  13. 一种终端，包括：
    一个或多个处理器；
    存储装置，设置为存储一个或多个程序；
    所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如权利要求1-6中任一项所述的方法。
  14. 一种服务器，包括：
    一个或多个处理器；
    存储装置，设置为存储一个或多个程序；
    所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现如权利要求7-10中任一项所述的方法。
  15. 一种系统，包括：终端和服务器；
    所述终端设置为实现如权利要求1-6中任一项所述的方法；
    所述服务器设置为实现如权利要求7-10中任一项所述的方法。
  16. 一种计算机可读存储介质，所述计算机可读存储介质上存储有计算机程序，所述计算机程序被处理器执行时实现如权利要求1-6或7-10中任一项所述的方法。
PCT/CN2019/092956 2018-06-26 2019-06-26 模型展现方法及装置、模型提供方法及装置、终端、服务器、系统及存储介质 Ceased WO2020001464A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810674183.3 2018-06-26
CN201810674183.3A CN109118515B (zh) 2018-06-26 2018-06-26 一种电力设备的视频跟踪方法及装置

Publications (1)

Publication Number Publication Date
WO2020001464A1 true WO2020001464A1 (zh) 2020-01-02

Family

ID=64821965

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/092956 Ceased WO2020001464A1 (zh) 2018-06-26 2019-06-26 模型展现方法及装置、模型提供方法及装置、终端、服务器、系统及存储介质

Country Status (2)

Country Link
CN (1) CN109118515B (zh)
WO (1) WO2020001464A1 (zh)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112102490A (zh) * 2020-09-25 2020-12-18 中国电建集团福建工程有限公司 一种用于变电站三维模型的建模方法
CN112631379A (zh) * 2021-01-06 2021-04-09 Oppo广东移动通信有限公司 盖板装配方法及电子设备
CN114078191A (zh) * 2020-08-18 2022-02-22 腾讯科技（深圳）有限公司 一种点云媒体的数据处理方法、装置、设备及介质
CN115424299A (zh) * 2022-09-01 2022-12-02 国网江苏省电力有限公司技能培训中心 一种电力作业人员工器具所属人员确定方法及系统
CN116404561A (zh) * 2023-06-08 2023-07-07 威海双城电气有限公司 一种电力设备智能识别装置
CN118657897A (zh) * 2024-08-21 2024-09-17 浙江黄氏建设科技股份有限公司 基于点云数据的建筑物轮廓三维建模监测方法及系统

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109118515B (zh) * 2018-06-26 2022-04-01 全球能源互联网研究院有限公司 一种电力设备的视频跟踪方法及装置
CN110288234B (zh) * 2019-06-26 2021-10-22 西安西拓电气股份有限公司 数据处理方法及装置
CN111488819B (zh) * 2020-04-08 2023-04-18 全球能源互联网研究院有限公司 电力设备的灾损监控感知采集方法及装置
CN112348967A (zh) * 2020-10-29 2021-02-09 国网浙江省电力有限公司 一种电力设备三维模型与实时视频无缝融合的方法
CN112698615A (zh) * 2020-11-10 2021-04-23 四川省东宇信息技术有限责任公司 一种基于云平台的设备故障反馈系统
CN112432669A (zh) * 2020-11-27 2021-03-02 贵州电网有限责任公司 一种电力二次设备运行状态移动视频实时监视方法及系统
CN112509148B (zh) * 2020-12-04 2024-12-24 全球能源互联网研究院有限公司 一种基于多特征识别的交互方法、装置及计算机设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106845502A (zh) * 2017-01-23 2017-06-13 东南大学 一种用于设备检修的穿戴式辅助装置及设备检修可视化指导方法
CN106910244A (zh) * 2017-02-20 2017-06-30 广东电网有限责任公司教育培训评价中心 电力设备内部构造现场认知方法和装置
WO2017177019A1 (en) * 2016-04-08 2017-10-12 Pcms Holdings, Inc. System and method for supporting synchronous and asynchronous augmented reality functionalities
CN109118515A (zh) * 2018-06-26 2019-01-01 全球能源互联网研究院有限公司 一种电力设备的视频跟踪方法及装置

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
ITBO20130466A1 (it) * 2013-08-29 2015-03-01 Umpi Elettronica Societa A Respo Nsabilita Lim Metodo di ispezione e/o manutenzione di una parte di un impianto industriale mediante realta' aumentata, e corrispondente sistema per guidare l'ispezione e/o la manutenzione della parte di impianto industriale
CN103823935B (zh) * 2014-02-28 2016-09-14 武汉大学 一种风电场三维远程监控系统
CN107193375B (zh) * 2017-05-17 2020-11-06 李福荣 一种基于虚拟现实的电力安全生产场景交互系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017177019A1 (en) * 2016-04-08 2017-10-12 Pcms Holdings, Inc. System and method for supporting synchronous and asynchronous augmented reality functionalities
CN106845502A (zh) * 2017-01-23 2017-06-13 东南大学 一种用于设备检修的穿戴式辅助装置及设备检修可视化指导方法
CN106910244A (zh) * 2017-02-20 2017-06-30 广东电网有限责任公司教育培训评价中心 电力设备内部构造现场认知方法和装置
CN109118515A (zh) * 2018-06-26 2019-01-01 全球能源互联网研究院有限公司 一种电力设备的视频跟踪方法及装置

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114078191A (zh) * 2020-08-18 2022-02-22 腾讯科技（深圳）有限公司 一种点云媒体的数据处理方法、装置、设备及介质
CN112102490A (zh) * 2020-09-25 2020-12-18 中国电建集团福建工程有限公司 一种用于变电站三维模型的建模方法
CN112102490B (zh) * 2020-09-25 2023-05-26 中国电建集团福建工程有限公司 一种用于变电站三维模型的建模方法
CN112631379A (zh) * 2021-01-06 2021-04-09 Oppo广东移动通信有限公司 盖板装配方法及电子设备
CN115424299A (zh) * 2022-09-01 2022-12-02 国网江苏省电力有限公司技能培训中心 一种电力作业人员工器具所属人员确定方法及系统
CN116404561A (zh) * 2023-06-08 2023-07-07 威海双城电气有限公司 一种电力设备智能识别装置
CN116404561B (zh) * 2023-06-08 2023-08-15 威海双城电气有限公司 一种电力设备智能识别装置
CN118657897A (zh) * 2024-08-21 2024-09-17 浙江黄氏建设科技股份有限公司 基于点云数据的建筑物轮廓三维建模监测方法及系统

Also Published As

Publication number Publication date
CN109118515B (zh) 2022-04-01
CN109118515A (zh) 2019-01-01

Similar Documents

Publication Publication Date Title
WO2020001464A1 (zh) 模型展现方法及装置、模型提供方法及装置、终端、服务器、系统及存储介质
KR102145109B1 (ko) 지도 생성 및 운동 객체 위치 결정 방법 및 장치
CN110568447B (zh) 视觉定位的方法、装置及计算机可读介质
US10068344B2 (en) Method and system for 3D capture based on structure from motion with simplified pose detection
WO2019196478A1 (zh) 机器人定位
CN112258567A (zh) 物体抓取点的视觉定位方法、装置、存储介质及电子设备
CA3137709A1 (en) Image-based localization
WO2019062619A1 (zh) 对图像内目标物体进行自动标注的方法、装置及系统
CN110458897A (zh) 多摄像头自动标定方法及系统、监控方法及系统
US10127667B2 (en) Image-based object location system and process
CN110361005A (zh) 定位方法、定位装置、可读存储介质及电子设备
CN106959691A (zh) 可移动电子设备和即时定位与地图构建方法
EP3330928A1 (en) Image generation device, image generation system, and image generation method
WO2016082797A1 (zh) 一种基于单幅图像的三维场景结构建模与注册方法
WO2021036587A1 (zh) 面向电力巡检场景的定位方法及系统
CN113920263A (zh) 地图构建方法、装置、设备及存储介质
CN112991440B (zh) 车辆的定位方法和装置、存储介质和电子装置
CN112146647B (zh) 一种地面纹理的双目视觉定位方法及芯片
CN111340942A (zh) 一种基于无人机的三维重建系统及其方法
CN115797451A (zh) 一种穴位识别方法、装置、设备及可读存储介质
CN113597568A (zh) 数据处理方法、控制设备及存储介质
WO2023025175A1 (zh) 用于空间定位的方法及装置
CN113465600A (zh) 一种导航方法、装置及电子设备和存储介质
CN116129087A (zh) 定位方法、视觉地图的生成方法及其装置
CN117557931A (zh) 一种基于三维场景的表计最优巡检点的规划方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19824761

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19824761

Country of ref document: EP

Kind code of ref document: A1