WO2020140524A1 - 芯片结构的增强现实显示方法、装置和可读存储介质 - Google Patents

芯片结构的增强现实显示方法、装置和可读存储介质 Download PDF

Info

Publication number
WO2020140524A1
WO2020140524A1 PCT/CN2019/110180 CN2019110180W WO2020140524A1 WO 2020140524 A1 WO2020140524 A1 WO 2020140524A1 CN 2019110180 W CN2019110180 W CN 2019110180W WO 2020140524 A1 WO2020140524 A1 WO 2020140524A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
circuit model
chip
target structure
mapping relationship
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/110180
Other languages
English (en)
French (fr)
Inventor
伍衍亮
梁博
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Gree Electric Appliances Inc of Zhuhai
Original Assignee
Gree Electric Appliances Inc of Zhuhai
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Gree Electric Appliances Inc of Zhuhai filed Critical Gree Electric Appliances Inc of Zhuhai
Priority to US17/419,884 priority Critical patent/US20220084291A1/en
Publication of WO2020140524A1 publication Critical patent/WO2020140524A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30148—Semiconductor; IC; Wafer

Definitions

  • the present disclosure relates to the field of display, and in particular to an augmented reality display method, device, and readable storage medium of a chip structure.
  • the current chip application development is carried out based on the specifications described by the text. There are many functional modules of the chip, and the functional parameters involved in each module are also very many.
  • the specifications that form the text are often hundreds of thousands of pages, resulting in
  • the application of the chip is very difficult, not intuitive, and it is also very difficult to develop and verify the chip, and the efficiency is low; under the current circumstances, the application of the chip or the developer verifier will spend a lot of energy to verify one by one Compare each parameter of the chip specification and make accurate application only after the understanding is correct.
  • Augmented reality is a technology that calculates the position and angle of camera images in real time and adds corresponding images, videos, and 3D models.
  • the goal of this technology is to put the virtual world on the screen.
  • Interactive display in the real world to improve user experience. If the operating principle and process of the chip can be displayed in combination with the AR technology, it will be convenient for users to understand the operating principle and process of the chip and improve development efficiency.
  • the present disclosure provides an augmented reality display method, device and readable storage medium of a chip structure to facilitate people to understand the operation principle and process of the chip.
  • an augmented reality display method of a chip structure including:
  • circuit model diagram corresponding to the chip structure, and the circuit model diagram shows a three-dimensional circuit model
  • the target frame image grab the target area showing the target structure from the target frame image, the target structure is part or all of the chip structure displayed on the target frame image;
  • a target circuit model corresponding to the target structure is generated and displayed.
  • mapping relationship between the target structure and the circuit model diagram including:
  • the feature point matching algorithm is used to match the first point group and the second point group to determine the model view matrix between the target structure and the circuit model;
  • the model view matrix is used to represent the mapping relationship between the target structure and the circuit model.
  • grabbing multiple second feature points from the target structure as the second point group includes:
  • a target circuit model corresponding to the target structure is generated and displayed, including:
  • Optional also includes:
  • One or more of the current information, voltage information, resistance information and switch on-off state when the target structure is working are displayed on the target circuit model.
  • the present disclosure also proposes an augmented reality display device with a chip structure, including:
  • the first acquiring unit is used to acquire a circuit model diagram corresponding to the chip structure, and a three-dimensional circuit model is displayed on the circuit model diagram;
  • the second acquisition unit is used to acquire the target frame image and grab the target area showing the target structure from the target frame image; wherein, the target structure is a part or all of the chip structure displayed on the target frame image;
  • the affine transformation unit is used to determine the mapping relationship between the target structure and the circuit model
  • the model generating unit is used to generate and display a target circuit model corresponding to the target structure according to the mapping relationship and the target structure.
  • the affine transformation unit determines the mapping relationship between the target structure and the circuit model diagram, including:
  • the feature point matching algorithm is used to match the first point group and the second point group to determine the model view matrix between the target structure and the circuit model;
  • the model view matrix is used to represent the mapping relationship between the target structure and the circuit model.
  • the affine transformation unit grabs multiple second feature points from the target structure as the second point group, including:
  • the model generating unit generates and displays the target circuit model corresponding to the target structure according to the mapping relationship and the target structure, including:
  • an analog circuit unit also includes: an analog circuit unit;
  • It is used to display one or more of current information, voltage information, resistance information and switch on-off status when the target structure is working on the target circuit model.
  • the present disclosure also proposes a readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of any of the methods proposed in the present disclosure are implemented.
  • the present disclosure proposes a chip structure augmented reality display method, device and readable storage medium, and correspondingly generates a target circuit model according to the target structure on the target frame image, thereby facilitating people to understand the operating principle of the chip and improving chip development efficiency And chip verification efficiency.
  • FIG. 1 is a flowchart of an augmented reality display method of a chip structure in an embodiment of the present disclosure
  • FIG. 2 is a composition diagram of an augmented reality display device with a chip structure in an embodiment of the present disclosure.
  • the microstructure photos of the chip structure taken by a camera are usually used, and the R&D personnel compare the microstructure photos and the chip specifications used when designing the chip to understand the operation principle and process of the chip.
  • the R&D personnel compare the microstructure photos and the chip specifications used when designing the chip to understand the operation principle and process of the chip.
  • an augmented reality display method of a chip structure including:
  • the chip structure is an internal structure of any chip manufactured according to a manufacturing specification, and may include multiple components such as resistors, inductors, capacitors, diodes, transistors, and thin film transistors, and wires connecting the various components.
  • a three-dimensional circuit model is shown on the circuit model diagram.
  • the circuit model is a three-dimensional model made according to the manufacturing specifications based on which the chip structure is manufactured.
  • the circuit model corresponds to the chip structure.
  • each component in the chip is in the circuit
  • the model is represented by the physical graphic symbols corresponding to each component, for example, the capacitance is represented by "
  • the length of each connecting wire in the circuit model, the relative position of each component and The relative positions of the components in the chip structure fabricated in the real environment are the same.
  • the length of the connecting wire between the two resistors in the chip structure is 1mm, and the wire between the two resistors in the circuit model is also 1mm, that is
  • the circuit model may be an equal-scale three-dimensional schematic diagram of the same size as the chip structure.
  • the purpose of establishing a three-dimensional circuit model is to facilitate users to understand the operating principles and processes of various components in the chip structure.
  • S12 Acquire the target frame image, and grab the target area showing the target structure from the target frame image.
  • the internal structure of the manufactured chip will be photographed with the camera's photographic equipment.
  • the photo taken by the camera is the target frame image
  • the internal structure of the chip is the one in this disclosure
  • the chip structure usually requires a high-magnification camera to capture the internal structure of the chip.
  • the target frame image here is any photo that captures the internal structure of the chip produced in the real environment.
  • the chip has the chip described in this disclosure Structure, therefore part or all of the chip structure must be displayed in the captured target frame image, the area showing the chip structure is the target area, and the chip structure displayed on the target frame image is called the target structure in this disclosure
  • the target structure is a part or all of the chip structure displayed on the target frame image.
  • the reason for extracting the target area is that there may be areas on the target frame image that show the non-chip structure, and these areas will interfere with the calculation of the mapping relationship later. Therefore, it is necessary to exclude only the target area that shows the target structure.
  • develop Personnel usually shoot multiple target frame images, each of which shows part of the chip structure, and then compares with the manufacturing specifications used in making the chip.
  • the design specifications write the design parameters of each component on the chip , For example, write the parameters of the resistance of the chip, the inductance of the inductance, the capacitive reactance of the capacitor, the cut-off voltage of the diode and the transistor, the switching status of each component during normal operation, the voltage and current carried by each component, etc.
  • S14 Generate and display a target circuit model corresponding to the target structure according to the mapping relationship and the target structure.
  • the target structure is a part of the chip structure, and the chip structure corresponds to the circuit model, so the target structure corresponds to the partial model in the circuit model, and the circuit model of the part corresponding to the chip structure is the target Circuit model.
  • the circuit model is drawn according to the chip structure.
  • the parameters in the circuit model are the same as the geometric dimensions of the chip structure, but the angle to which the circuit model is drawn is usually different from the angle at which the target frame image is taken. It can be easily understood as follows: when making the circuit model, camera A is used to shoot the chip structure. Camera A automatically generates a circuit model diagram proportional to the chip structure according to the chip structure. The picture taken by camera A is the circuit model diagram, and Camera B is used to capture the target frame image.
  • the circuit model in camera A must be translated and rotated to be able to Corresponds to the chip structure in the target frame image taken by camera B, and the specific translation direction, translation distance, rotation direction and rotation angle are the mapping relationship between the target structure and the circuit model.
  • This mapping relationship can be expressed by a matrix. Graphically, this matrix is usually called the model view matrix. After determining the mapping relationship, you know how to translate and rotate the circuit model. After the above translation and rotation, the developer will see the circuit model at the same angle of view as when shooting the target frame image. The target circuit model corresponding to the target structure is sufficient.
  • the target circuit model may be displayed to the user only, or the target circuit model may be displayed in the target area of the target frame image and displayed to the user.
  • the established The circuit model is better understood, especially when the name and symbol of each component are displayed on the circuit model, as well as the physical parameters of each component, so it can be convenient for developers to understand and improve the efficiency and accuracy of secondary development on the basis of the chip
  • the verification efficiency of the chip for verification uses augmented reality technology to display a circuit model corresponding to the target structure in the target frame image, which is convenient for understanding the operating principle of the chip.
  • the structure of the chip is not easy to understand when viewing the target frame image.
  • the present disclosure displays the target circuit model to facilitate people to understand the operation principle of the chip and improve the chip development efficiency and chip verification efficiency.
  • mapping relationship between the target structure and the circuit model diagram including:
  • the feature point matching algorithm is used to match the first point group and the second point group to determine the model view matrix between the target structure and the circuit model;
  • the model view matrix is used to represent the mapping relationship between the target structure and the circuit model.
  • grabbing multiple first feature points here refers to determining each first feature point first, and then determining the coordinates of the first feature point on the circuit model diagram, where the first point group refers to each first feature The coordinate set of points on the circuit model diagram.
  • the second point group refers to the coordinate set of the coordinates of the second feature point on the target frame image.
  • the image coordinate system of the target frame image and the image coordinate system on the circuit model diagram are preferably the same, That is, the directions of the abscissa and the ordinate are consistent, the scale is consistent, and the coordinate origin is located at the same position.
  • the coordinate origin of the image coordinate system is set in the upper left corner of the image.
  • the first feature point and the second feature point are preferably the intersection of the circuit of the chip structure, that is, the cross connection point of the two connecting wires in the chip structure, because although the circuit model is an equal-scale three-dimensional schematic diagram of the chip structure, each circuit model
  • the size of the components may be slightly different from the components in the real chip structure, and the point coordinates of the connection point of the two wires, that is, the intersection of the circuits, are usually easier to determine and the error is smaller.
  • the first feature point set can be regarded as a first curved surface
  • the second feature point set can be regarded as a second curved surface
  • the feature point matching algorithm is used to align the two overlapping surfaces.
  • the feature points here match It is preferable to use the ICP algorithm (Iterative Closest Point), the specific calculation process is related technology, and this disclosure will not repeat them here.
  • the resulting mapping relationship is expressed by the model view matrix (Model View Matrix, MVM), which is The translation and rotation matrix (Rotation&Translation, RT) of the second point group and the first point group.
  • MVM Model View Matrix
  • the model view matrix is used to map the points on the circuit model diagram to the target frame image.
  • the coordinates of any pixel on the circuit model of the circuit model diagram Multiplying with the model feature matrix to get the coordinates of the pixel on the target frame image.
  • capturing multiple second feature points from the target structure as the second point group includes: converting the target frame image into a grayscale image, and capturing multiple second features from the target structure displayed in the grayscale image Point as the second point group.
  • the purpose of converting the target frame image into a grayscale image is to facilitate capturing the second feature point.
  • a target circuit model corresponding to the target structure is generated and displayed, including:
  • the entire chip structure corresponds to the entire circuit model
  • the target structure may only be a part of the chip structure.
  • the original circuit model refers to the circuit model corresponding to the chip structure.
  • the target structure is the left half of the chip structure
  • the original circuit model is the left half of the circuit model, and then the coordinates of various points on the original circuit model on the plane of the target frame image are calculated according to the mapping relationship, thereby reconstructing to form the target circuit model.
  • each component can be identified with a physical graphic symbol that is easy to understand (for example, the corresponding physical symbol is drawn above each component), and in the circuit model
  • Each component displays the corresponding physical parameters, such as the resistance value displayed on the resistance, and the inductive reactance displayed on the inductance, so it is convenient for the user to clearly and simply understand the principle of the chip structure.
  • the method proposed in the present disclosure further includes: displaying one or more of current information, voltage information, resistance information, and switch on-off status when the target structure is in operation on the target circuit model.
  • the current information includes the current size and current direction.
  • the moving direction of the current can be displayed on the target circuit model to realize the dynamic display of the current movement process.
  • the voltage information includes the voltage size and any component in the chip structure The voltage value at both ends.
  • the method proposed in the present disclosure can be applied to AR devices.
  • the AR device When users use the AR device to view the captured target frame pictures, the corresponding target circuit model is displayed on the target frame image, so that the user can unnecessarily read hundreds of thousands of pages of text production Specification, and can directly develop the chip, and it is convenient to verify whether the produced chip is given the production specification.
  • the present disclosure also proposes an augmented reality display device with a chip structure, including: a first acquisition unit 10, a second acquisition unit 20, an affine transformation unit 30 and a model generation unit 40.
  • the first acquiring unit 10 is used to acquire a circuit model diagram corresponding to the chip structure, and a three-dimensional circuit model is displayed on the circuit model diagram;
  • the second acquiring unit 20 is configured to acquire a target frame image and grab a target area displaying the target structure from the target frame image; wherein, the target structure is a part or all of the chip structure displayed on the target frame image;
  • the affine transformation unit 30 is used to determine the mapping relationship between the target structure and the circuit model
  • the model generating unit 40 is configured to generate and display a target circuit model corresponding to the target structure according to the mapping relationship and the target structure.
  • the target frame image here is any photo that captures the internal structure of the chip produced in the real environment.
  • the chip has the chip structure described in this disclosure. Therefore, part of the captured target frame image must be displayed. Or all of the chip structures, the area where the chip structure is displayed is the target area, and the chip structure displayed on the target frame image is called the target structure in the present disclosure, that is, the target structure is the part or the part displayed on the target frame image All of the chip structure.
  • the circuit model is drawn according to the chip structure.
  • the parameters in the circuit model are the same as the geometric dimensions of the chip structure, but the angle to which the circuit model is drawn is usually different from the angle at which the target frame image is taken.
  • the model generating unit 40 may display only the target circuit model or display the target circuit model in the target area of the target frame image. Compared with the captured target frame image, the established circuit model is better understood, especially when the circuit model displays the names and symbols of the various components in the chip structure and the physical parameters of each component in the chip structure, so it can be easily developed The personnel understand that the efficiency of secondary development on the basis of the chip and the verification efficiency of verifying the chip are improved.
  • the device proposed in the present disclosure may be an AR display device.
  • the model generating unit 40 includes a display for displaying a target frame image and a target circuit model.
  • the model generating unit 40 displays the corresponding on the target structure of the target frame image Circuit model diagram, or draw a check box on the target frame image to mark the identified target area, and display the circuit model corresponding to the target structure on the side of the target area to facilitate the user to compare the target structure and the target circuit model. That is, in the present disclosure, an augmented reality technology may be used to display a circuit model corresponding to the target structure in the target frame image, which is convenient for understanding the operating principle of the chip.
  • the affine transformation unit 30 determines the mapping relationship between the target structure and the circuit model diagram, including: grabbing a plurality of first feature points from the circuit model as the first point group;
  • the feature point matching algorithm is used to match the first point group and the second point group to determine the model view matrix between the target structure and the circuit model;
  • the model view matrix is used to represent the mapping relationship between the target structure and the circuit model.
  • grabbing multiple first feature points here refers to determining each first feature point first, and then determining the coordinates of the first feature point on the circuit model diagram, where the first point group refers to each first feature The coordinate set of points on the circuit model diagram.
  • the second point group refers to the coordinate set of the coordinates of the second feature point on the target frame image.
  • the image coordinate system of the target frame image and the image coordinate system on the circuit model diagram are preferably the same, That is, the directions of the abscissa and the ordinate are consistent, the scale is consistent, and the coordinate origin is located at the same position.
  • the coordinate origin of the image coordinate system is set in the upper left corner of the image.
  • the first feature point and the second feature point are preferably the intersection of the circuit of the chip structure, that is, the cross connection point of the two connecting wires in the chip structure, because although the circuit model is an equal-scale three-dimensional schematic diagram of the chip structure, each circuit model
  • the size of the components may be slightly different from the components in the real chip structure, and the point coordinates of the connection point of the two wires, that is, the intersection of the circuits, are usually easier to determine and the error is smaller. After obtaining the MVM matrix, you can determine how to project the circuit model from the circuit model diagram onto the target frame image.
  • the affine transformation unit 30 captures multiple second feature points from the target structure as the second point group, including: converting the target frame image into a grayscale image, and capturing from the target structure displayed in the grayscale image A plurality of second feature points are used as the second point group.
  • the model generating unit 40 generates a target circuit model corresponding to the target structure according to the mapping relationship and the target structure, including:
  • the target circuit model corresponding to the target structure is generated according to the target coordinates.
  • the entire chip structure corresponds to the entire circuit model, and the target structure may only be a part of the chip structure.
  • the original circuit model refers to the circuit model corresponding to the chip structure, because the original circuit model may be proportional to the target structure Schematic diagram.
  • each component can be identified with a physical graphic symbol that is easy to understand, and each component in the circuit model displays the corresponding physical parameters, such as the resistance value on the resistance and the inductance on the inductor, so It is convenient for users to clearly and simply understand the principle of chip structure.
  • the optional device proposed in the present disclosure further includes: an analog circuit unit; for displaying one or more of current information, voltage information, resistance information, and switch on-off status when the target structure is in operation on the target circuit model.
  • the current information includes the current size and current direction.
  • the moving direction of the current can be displayed on the target circuit model to realize the dynamic display of the current movement process.
  • the voltage information includes the voltage size and any component in the chip structure The voltage value at both ends.
  • the present disclosure also proposes a readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of any of the methods proposed in the present disclosure are implemented.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Processing Or Creating Images (AREA)

Abstract

一种芯片结构的增强现实显示方法、装置和可读存储介质，其中方法包括：获取芯片结构对应的电路模型图（S11），电路模型图上显示有三维的电路模型；获取目标帧图像，从目标帧图像内抓取显示有目标结构的目标区域（S12），目标结构为目标帧图像上显示的部分或全部的芯片结构；确定目标结构与电路模型之间的映射关系（S13）；根据映射关系和目标结构，生成并显示与目标结构相对应的目标电路模型（S14）。从而方便人们理解芯片的运行原理，提高了芯片开发效率和芯片验证效率。

Description

芯片结构的增强现实显示方法、装置和可读存储介质
本公开以2019年1月2日递交的、申请号为201910001623.3且名称为“芯片结构的增强现实显示方法、装置和可读存储介质”的专利文件为优先权文件，其全部内容通过引用结合在本公开中。
技术领域
本公开涉及显示领域，特别涉及一种芯片结构的增强现实显示方法、装置和可读存储介质。
背景技术
现在的芯片应用开发均是基于文字描述的规格书来开展的，芯片的功能模块非常多，并且每一个模块涉及的功能参数也非常多，形成文字的规格书往往是几百上千页，导致芯片应用的难度非常大，不直观，并且对芯片研发验证也有很大难度，效率低；目前的情况下，芯片的应用者或者是开发验证者，都是要花费很大的精力，逐一的核实对照芯片规格书的每一项参数，并在理解无误后才能做准确的应用。
综上所述，在相关技术中，人们理解芯片运行原理费时费力，造成开发效率低。增强现实技术(Augmented Reality，简称AR)，是一种实时地计算摄影机影像的位置及角度并加上相应图像、视频、3D模型的技术，这种技术的目标是在屏幕上把虚拟世界套在现实世界并进行互动显示，以提高用户体验，如果能够结合AR技术显示芯片运行原理和过程，就能够方便用户理解芯片的运行原理和过程，提高开发效率。
因此，方便人们理解芯片的运行原理和过程，是相关技术中亟待解决的问题。
发明内容
本公开提供了一种芯片结构的增强现实显示方法、装置和可读存储介质，以方便人们理解芯片的运行原理和过程。
为了解决上述问题，作为本公开的一个方面，提供了一种芯片结构的增强现实显示方法，包括：
获取芯片结构对应的电路模型图，电路模型图上显示有三维的电路模型；
获取目标帧图像，从目标帧图像内抓取显示有目标结构的目标区域，目标结构为目标帧图像上显示的部分或全部的芯片结构；
确定目标结构与电路模型之间的映射关系；
根据映射关系和目标结构，生成并显示与目标结构相对应的目标电路模型。
可选的，确定目标结构与电路模型图之间的映射关系，包括：
从电路模型上抓取多个第一特征点作为第一点群；
从目标结构上抓取多个第二特征点作为第二点群；
采用特征点匹配算法对第一点群和第二点群进行匹配，以确定目标结构和电路模型之间的模型视图矩阵；
采用模型视图矩阵表征目标结构与电路模型之间的映射关系。
可选的，从目标结构上抓取多个第二特征点作为第二点群，包括：
将目标帧图像转换成灰度图，从灰度图显示的目标结构上抓取多个第二特征点作为第二点群。
可选的，根据映射关系和目标结构，生成并显示与目标结构相对应的目标电路模型，包括：
从电路模型图中获取与目标结构相对应的原始电路模型；
根据映射关系对原始电路模型进行仿射变换，得到原始电路模型上各个点在目标帧图像视平面上的目标坐标；
根据目标坐标生成并显示目标结构对应的目标电路模型。
可选的，还包括：
在目标电路模型上显示目标结构工作时的电流信息、电压信息、电阻信息和开关通断状态中的一个或多个。
本公开还提出一种芯片结构的增强现实显示装置，包括：
第一获取单元，用于获取芯片结构对应的电路模型图，电路模型图上显示有三维的电路模型；
第二获取单元，用于获取目标帧图像，从目标帧图像内抓取显示有目标结 构的目标区域；其中，目标结构为目标帧图像上显示的部分或全部的芯片结构；
仿射变换单元，用于确定目标结构与电路模型之间的映射关系；
模型生成单元，用于根据映射关系和目标结构，生成并显示与目标结构相对应的目标电路模型。
可选的，仿射变换单元确定目标结构与电路模型图之间的映射关系，包括：
从电路模型上抓取多个第一特征点作为第一点群；
从目标结构上抓取多个第二特征点作为第二点群；
采用特征点匹配算法对第一点群和第二点群进行匹配，以确定目标结构和电路模型之间的模型视图矩阵；
采用模型视图矩阵表征目标结构与电路模型之间的映射关系。
可选的，仿射变换单元从目标结构上抓取多个第二特征点作为第二点群，包括：
将目标帧图像转换成灰度图，从灰度图显示的目标结构上抓取多个第二特征点作为第二点群。
可选的，模型生成单元根据映射关系和目标结构，生成并显示与目标结构相对应的目标电路模型，包括：
从电路模型图中获取与目标结构相对应的原始电路模型；
根据映射关系对原始电路模型进行仿射变换，得到原始电路模型上各个点在目标帧图像视平面上的目标坐标；
根据目标坐标生成并显示目标结构对应的目标电路模型。
可选的，还包括：模拟电路单元；
用于在目标电路模型上显示目标结构工作时的电流信息、电压信息、电阻信息和开关通断状态中的一个或多个。
本公开还提出一种可读存储介质，其上存储有计算机程序，程序被处理器执行时实现本公开提出的任一项方法的步骤。
本公开提出了一种芯片结构的增强现实显示方法、装置和可读存储介质，根据目标帧图像上的目标结构，对应生成目标电路模型，从而方便人们理解芯片的运行原理，提高了芯片开发效率和芯片验证效率。
附图说明
图1为本公开实施例中一种芯片结构的增强现实显示方法流程图；
图2为本公开实施例中一种芯片结构的增强现实显示装置组成图。
具体实施方式
为使本公开的目的、技术方案和优点更加清楚，下面将结合本公开具体实施例及相应的附图对本公开技术方案进行清楚、完整地描述。显然，所描述的实施例仅是本公开一部分实施例，而不是全部的实施例。基于本公开中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本公开保护的范围。
需要说明的是，本公开的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本公开的实施例能够以除了在这里图示或描述的那些以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
在相关技术中，在芯片制作完毕后，通常利用照相机拍摄的芯片结构的微观结构照片，研发人员对照微观结构照片和设计芯片时使用的芯片规格书，以理解芯片的运行原理和过程。芯片的模块非常多，且每个模块的参数也非常多，因此这种开发模式的开发效率较低，研发人员理解芯片的运行原理和过程费时费力。
为了解决上述技术问题，如图1所示，本公开提出了一种芯片结构的增强现实显示方法，包括：
S11：获取芯片结构对应的电路模型图；
具体的，芯片结构是按照制作规格书制作的任意一种芯片的内部结构，其可以包括：电阻、电感、电容、二极管、三极管、薄膜晶体管等多个部件以及 连接各个部件的导线。所述电路模型图上显示有三维的电路模型，电路模型是根据制造芯片结构时所依据的制作规格书制作的三维模型，电路模型与芯片结构相对应，优选地，芯片中的各个部件在电路模型中用各个部件对应的物理图形符号进行表示，例如电容用“||”进行表示，电阻用长方形进行表示，优选地，电路模型中各个连接导线的长度、各部件的彼此间的相对位置与现实环境中制作出的芯片结构中各部件的相对位置相同，例如芯片结构中两个电阻之间的连接导线的长度为1mm，而在电路模型中这两个电阻之间的导线也是1mm，即电路模型可以是与芯片结构大小相同的等比例三维原理图。更优选地，在电路模型的各个部件上标识有对应的物理字母符号和/或中文名称，例如在电阻上显示“R”和/或显示“电阻”，优选地同时显示有各个部件的物理参数，例如在电阻上显示有阻值，又例如在电容上显示“电容＝10”来表示部件类型和电容大小，以方便人们理解。建立三维的电路模型的用途在于方便用户理解芯片结构中各个部件的运行原理和过程。
S12：获取目标帧图像，从目标帧图像内抓取显示有目标结构的目标区域。
具体的，在现实环境中制作了具有上述芯片结构的芯片后，会用相机的照相设备拍摄制作出的芯片的内部结构，相机拍摄的照片就是目标帧图像，芯片的内部结构就是本公开中的芯片结构，通常需要用高倍率的相机拍摄芯片的内部结构，此处的目标帧图像就是任意一张拍摄了现实环境中制作出的芯片的内部结构的照片，该芯片具有本公开所述的芯片结构，因此拍摄的目标帧图像中必然显示有部分或全部的所述芯片结构，显示有芯片结构的区域就是目标区域，目标帧图像上显示的芯片结构在本公开中称为目标结构，即所述目标结构为所述目标帧图像上显示的部分或全部所述芯片结构。提取目标区域的原因是，目标帧图像上可能有显示有非芯片结构的区域，这些区域会对之后计算映射关系造成干扰，因此需要排除只获取显示目标结构的目标区域，在相关技术中，开发人员通常会拍摄多张目标帧图像，每一张目标帧图像上显示部分所述芯片结构，然后和制作芯片时使用的制作规格书进行对比，制作规格书上写有芯片上各个部件的设计参数，例如写有芯片中电阻的阻止、电感的感抗、电容的容抗、二极管和三极管的截止电压、正常工作时各个部件的开关情况、各个部件所承载的电压、电流等参数，开发人员用多张目标帧图片对照制作规格书以理解制 作出的芯片的运行原理和运行过程，制作规格书往往较长且都是文字介绍，开发人员难以理解，降低了基于芯片进行二次开发的效率和验证芯片是否符合制作规格书的效率。
S13：确定目标结构与电路模型之间的映射关系。
S14：根据映射关系和目标结构生成并显示与目标结构相对应的目标电路模型。
具体的，如前所述，目标结构是芯片结构的一部分，而芯片结构是与电路模型相对应的，因此目标结构与电路模型中的部分模型相对应，芯片结构对应的部分的电路模型就是目标电路模型。电路模型是依据芯片结构进行绘制的，电路模型中的参数与芯片结构的几何尺寸相同，但是绘制电路模型时所依照的角度与拍摄目标帧图像的角度通常是不一样的。可以采用如下方式简单的理解：制作电路模型时是使用了相机A拍摄芯片结构，相机A根据芯片结构自动生成了与芯片结构等比例的电路模型图，相机A拍摄的图片就是电路模型图，而拍摄目标帧图像时使用了相机B，虽然相机A和相机B拍摄的是同一个芯片结构，但是相机A和相机B拍摄照片时的位置不同，相机A中的电路模型必须经过平移、旋转才能够与相机B拍摄的目标帧图像中的芯片结构相对应，而具体的平移方向和平移距离、旋转方向和旋转角度就是目标结构和电路模型之间的映射关系，这个映射关系可以用矩阵表示，在图形学上通常称该矩阵为模型视图矩阵。在确定了映射关系后，就知道了如何对电路模型进行平移、旋转，经过上述平移、旋转后，开发人员看到的电路模型就会和拍摄目标帧图像时的视角相同，此时只显示与目标结构相对应的目标电路模型即可。在本公开中，在生成目标电路模型后，可以只向用户显示目标电路模型，也可以将目标电路模型显示在目标帧图像的目标区域展示给用户，与拍摄的目标帧图像相比，建立的电路模型更好理解，特别是当电路模型上显示有各个部件的名称和符号，以及各部件的物理参数时，因此可以方便开发人员理解，提高在芯片的基础上进行二次开发的效率和对芯片进行验证的验证效率。即本公开采用增强现实技术显示与目标帧图像中的目标结构相对应的电路模型，方便理解芯片的运行原理。相关技术中查看目标帧图像时，芯片结构不易理解，本公开通过显示目标电路模型，从而方便人们理解芯片的运行原理，提高了芯片开发效率和芯片验证效率。
可选的，确定目标结构与电路模型图之间的映射关系，包括：
从电路模型上抓取多个第一特征点作为第一点群；
从目标结构上抓取多个第二特征点作为第二点群；
采用特征点匹配算法对第一点群和第二点群进行匹配，以确定目标结构和电路模型之间的模型视图矩阵；
采用模型视图矩阵表征目标结构与电路模型之间的映射关系。
具体的，此处抓取多个第一特征点是指先确定各个第一特征点，再确定第一特征点在电路模型图上的坐标，此处的第一点群指的是各个第一特征点在电路模型图上的坐标集合。同理，第二点群指的是第二特征点在目标帧图像上坐标的坐标集合，需要注意的是，目标帧图像的图像坐标系以及电路模型图上的图像坐标系优选是相同的，即横坐标方向一致、纵坐标方向一致，标尺一致且坐标原点位于同一位置，例如都将图像坐标系的坐标原点设置在图像的左上角。第一特征点和第二特征点优选为芯片结构的电路交汇点，即芯片结构中两个连接导线的交叉连接点，因为电路模型虽然是芯片结构的等比例三维原理图，但电路模型上各部件的大小可能与真实的芯片结构中各部件略有差异，而两个导线的连接点，即电路交汇点，的点坐标通常比较容易确定且误差较小。第一特征点的集合可以看做是一个第一曲面，第二特征点的集合可以看作是第二曲面，因为电路模型和芯片结构是相同的，因此，第二曲面必然与第一曲面的一部分相交叠，而要将第二曲面与第一曲面相交叠就需要对第二曲面进行平移、旋转和缩放，特征点匹配算法用于对齐两个相互交叠的曲面，此处的特征点匹配算优选采用ICP算法(Iterative Closest Point，迭代最近点算法)，具体的计算过程为相关技术，本公开不再赘述，最终得到的映射关系采用模型视图矩阵(Model View Matrix、MVM)表示，也就是第二点群和第一点群的平移旋转矩阵(Rotation&Translation，RT)，模型视图矩阵用于将电路模型图上的点映射到目标帧图像上，电路模型图的电路模型上任一像素点的坐标与模型特征矩阵相左乘即得到该像素在目标帧图像上的坐标。在得到MVM矩阵后就可以确定如何将电路模型从电路模型图投影至目标帧图像上。
可选的，从目标结构上抓取多个第二特征点作为第二点群，包括：将目标帧图像转换成灰度图，从灰度图显示的目标结构上抓取多个第二特征点作为第 二点群。具体的，将目标帧图像转换为灰度图的目的在于便于抓取第二特征点。
可选的，根据映射关系和目标结构，生成并显示与目标结构相对应的目标电路模型，包括：
从电路模型图中获取与目标结构相对应的原始电路模型；
根据映射关系对原始电路模型进行仿射变换，得到原始电路模型上各个点在目标帧图像视平面上的目标坐标；
根据目标坐标生成并显示目标结构对应的目标电路模型。
具体的，整个芯片结构与整个电路模型相对应，而目标结构可能只是芯片结构的一部分，原始电路模型是指与芯片结构相对应的电路模型，例如当目标结构为芯片结构的左半部时，原始电路模型为电路模型的左半部，然后按照映射关系计算出原始电路模型上各个点在目标帧图像的平面上的坐标，从而重构形成目标电路模型。因为，原始电路模型可以是与目标结构的等比例原理图，在电路模型中各个部件可以用便于理解的物理图形符号标识(例如在各部件的上方绘制对应的物理符号)，并且在电路模型中各部件显示对应的物理参数，例如在电阻上显示阻值大小，在电感上显示感抗大小，因此可以方便用户清楚而简单的理解芯片结构的原理。
可选的，本公开提出的方法，还包括：在目标电路模型上显示目标结构工作时的电流信息、电压信息、电阻信息和开关通断状态中的一个或多个。具体的，电流信息包括电流大小、电流方向，例如可以在目标电路模型上用移动的箭头显示电流的流动方向，以实现动态显示电流的运动过程，电压信息包括电压大小，芯片结构中任一部件两端的电压值。
本公开提出的方法可以用于AR设备，用户在使用AR设备查看拍摄的目标帧图片时，在目标帧图像上显示对应的目标电路模型，让用户可以不必要阅读几百上千页的文字制作规格书，而可以直接开展芯片开发工作，并方便验证制作出的芯片是否赋予制作规格书。
如图2所示，本公开还提出一种芯片结构的增强现实显示装置，包括：第一获取单元10、第二获取单元20、仿射变换单元30和模型生成单元40。
第一获取单元10，用于获取芯片结构对应的电路模型图，电路模型图上显示有三维的电路模型；
第二获取单元20，用于获取目标帧图像，从目标帧图像内抓取显示有目标结构的目标区域；其中，目标结构为目标帧图像上显示的部分或全部芯片结构；
仿射变换单元30，用于确定目标结构与电路模型之间的映射关系；
模型生成单元40，用于根据映射关系和目标结构，生成并显示与目标结构相对应的目标电路模型。
具体的，此处的目标帧图像就是任意一张拍摄了现实环境中制作出的芯片的内部结构的照片，该芯片具有本公开所述的芯片结构，因此拍摄的目标帧图像中必然显示有部分或全部所述芯片结构，显示有芯片结构的区域就是目标区域，目标帧图像上显示的芯片结构在本公开中称为目标结构，即所述目标结构为所述目标帧图像上显示的部分或全部的所述芯片结构。电路模型是依据芯片结构进行绘制的，电路模型中的参数与芯片结构的几何尺寸相同，但是绘制电路模型时所依照的角度与拍摄目标帧图像的角度通常是不一样的。因此需要用仿射变换单元30确定映射关系，以对电路模型进行仿射变换。模型生成单元40可以只显示目标电路模型或者将目标电路模型显示在目标帧图像的目标区域。与拍摄的目标帧图像相比，建立的电路模型更好理解，特别是当电路模型上显示有芯片结构中各个部件的名称和符号，以及芯片结构中各部件的物理参数时，因此可以方便开发人员理解，提高在芯片的基础上进行二次开发的效率和对芯片进行验证的验证效率。本公开提出的装置可以是AR显示装置，模型生成单元40包括显示器用于显示目标帧图像和目标电路模型，当载入目标帧图像后，模型生成单元40在目标帧图像的目标结构上显示对应的电路模型图，或者，在目标帧图像上绘制选取框，用于标注被识别的目标区域，在目标区域一侧显示与所述目标结构对应的电路模型，以方便用户对比目标结构和目标电路模型。即本公开中可以采用增强现实技术显示与目标帧图像中的目标结构相对应的电路模型，方便理解芯片的运行原理。
可选的，仿射变换单元30确定目标结构与电路模型图之间的映射关系，包括：从电路模型上抓取多个第一特征点作为第一点群；
从目标结构上抓取多个第二特征点作为第二点群；
采用特征点匹配算法对第一点群和第二点群进行匹配，以确定目标结构和电路模型之间的模型视图矩阵；
采用模型视图矩阵表征目标结构与电路模型之间的映射关系。
具体的，此处抓取多个第一特征点是指先确定各个第一特征点，再确定第一特征点在电路模型图上的坐标，此处的第一点群指的是各个第一特征点在电路模型图上的坐标集合。同理，第二点群指的是第二特征点在目标帧图像上坐标的坐标集合，需要注意的是，目标帧图像的图像坐标系以及电路模型图上的图像坐标系优选是相同的，即横坐标方向一致、纵坐标方向一致，标尺一致且坐标原点位于同一位置，例如都将图像坐标系的坐标原点设置在图像的左上角。第一特征点和第二特征点优选为芯片结构的电路交汇点，即芯片结构中两个连接导线的交叉连接点，因为电路模型虽然是芯片结构的等比例三维原理图，但电路模型上各部件的大小可能与真实的芯片结构中各部件略有差异，而两个导线的连接点，即电路交汇点，的点坐标通常比较容易确定且误差较小。在得到MVM矩阵后就可以确定如何将电路模型从电路模型图投影至目标帧图像上。
可选的，仿射变换单元30从目标结构上抓取多个第二特征点作为第二点群，包括：将目标帧图像转换成灰度图，从灰度图显示的目标结构上抓取多个第二特征点作为第二点群。
可选的，模型生成单元40根据映射关系和目标结构，生成与目标结构相对应的目标电路模型，包括：
从电路模型图中获取与目标结构相对应的原始电路模型；
根据映射关系对原始电路模型进行仿射变换，得到原始电路模型上各个点在目标帧图像视平面上的目标坐标；
根据目标坐标生成目标结构对应的目标电路模型。
具体的，整个芯片结构与整个电路模型相对应，而目标结构可能只是芯片结构的一部分，原始电路模型是指与芯片结构相对应的电路模型，因为，原始电路模型可以是与目标结构的等比例原理图，在电路模型中各个部件可以用便于理解的物理图形符号标识，并且在电路模型中各部件显示对应的物理参数，例如在电阻上显示阻值大小，在电感上显示感抗大小，因此可以方便用户清楚而简单的理解芯片结构的原理。
可选的本公开提出的装置，还包括：模拟电路单元；用于在目标电路模型上显示目标结构工作时的电流信息、电压信息、电阻信息和开关通断状态中的 一个或多个。具体的，电流信息包括电流大小、电流方向，例如可以在目标电路模型上用移动的箭头显示电流的流动方向，以实现动态显示电流的运动过程，电压信息包括电压大小，芯片结构中任一部件两端的电压值。
本公开还提出一种可读存储介质，其上存储有计算机程序，程序被处理器执行时实现本公开提出的任一项方法的步骤。
以上所述仅为本公开的优选实施例而已，并不用于限制本公开，对于本领域的技术人员来说，本公开可以有各种更改和变化。凡在本公开的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本公开的保护范围之内。

Claims (11)

  1. 一种芯片结构的增强现实显示方法，其特征在于，包括：
    获取所述芯片结构对应的电路模型图，所述电路模型图上显示有三维的电路模型；
    获取目标帧图像，从所述目标帧图像内抓取显示有目标结构的目标区域，所述目标结构为所述目标帧图像上显示的部分或全部的所述芯片结构；
    确定所述目标结构与所述电路模型之间的映射关系；
    根据所述映射关系和所述目标结构，生成并显示与所述目标结构相对应的目标电路模型。
  2. 根据权利要求1所述的芯片结构的增强现实显示方法，其特征在于，确定所述目标结构与所述电路模型图之间的映射关系，包括：
    从所述电路模型上抓取多个第一特征点作为第一点群；
    从所述目标结构上抓取多个第二特征点作为第二点群；
    采用特征点匹配算法对所述第一点群和所述第二点群进行匹配，以确定所述目标结构和所述电路模型之间的模型视图矩阵；
    采用所述模型视图矩阵表征所述目标结构与电路模型之间的映射关系。
  3. 根据权利要求2所述的芯片结构的增强现实显示方法，其特征在于，从所述目标结构上抓取多个第二特征点作为第二点群，包括：
    将所述目标帧图像转换成灰度图，从所述灰度图显示的所述目标结构上抓取多个所述第二特征点作为所述第二点群。
  4. 根据权利要求1-3任一项所述的芯片结构的增强现实显示方法，其特征在于，根据所述映射关系和所述目标结构，生成并显示与所述目标结构相对应的目标电路模型，包括：
    从所述电路模型图中获取与所述目标结构相对应的原始电路模型；
    根据所述映射关系对所述原始电路模型进行仿射变换，得到所述原始电路模型上各个点在所述目标帧图像的视平面上的目标坐标；
    根据所述目标坐标生成并显示所述目标结构对应的目标电路模型。
  5. 根据权利要求1-3任一项所述的芯片结构的增强现实显示方法，其特征在于，还包括：
    在所述目标电路模型上显示所述目标结构工作时的电流信息、电压信息、电阻信息和开关通断状态中的一个或多个。
  6. 一种芯片结构的增强现实显示装置，其特征在于，包括：
    第一获取单元，用于获取所述芯片结构对应的电路模型图，所述电路模型图上显示有三维的电路模型；
    第二获取单元，用于获取目标帧图像，从所述目标帧图像内抓取显示有目标结构的目标区域；其中，所述目标结构为所述目标帧图像上显示的部分或全部的所述芯片结构；
    仿射变换单元，用于确定所述目标结构与所述电路模型之间的映射关系；
    模型生成单元，用于根据所述映射关系和所述目标结构，生成并显示与所述目标结构相对应的目标电路模型。
  7. 根据权利要求6所述的芯片结构的增强现实显示装置，其特征在于，所述仿射变换单元确定所述目标结构与所述电路模型图之间的映射关系，包括：
    从所述电路模型上抓取多个第一特征点作为第一点群；
    从所述目标结构上抓取多个第二特征点作为第二点群；
    采用特征点匹配算法对所述第一点群和所述第二点群进行匹配，以确定所述目标结构和所述电路模型之间的模型视图矩阵；
    采用所述模型视图矩阵表征所述目标结构与电路模型之间的映射关系。
  8. 根据权利要求7所述的芯片结构的增强现实显示装置，其特征在于，所述仿射变换单元从所述目标结构上抓取多个第二特征点作为第二点群，包括：
    将所述目标帧图像转换成灰度图，从所述灰度图显示的所述目标结构上抓取多个所述第二特征点作为所述第二点群。
  9. 根据权利要求6-8任一项所述的芯片结构的增强现实显示装置，其特征在于，所述模型生成单元根据所述映射关系和所述目标结构，生成并显示与所述目标结构相对应的目标电路模型，包括：
    从所述电路模型图中获取与所述目标结构相对应的原始电路模型；
    根据所述映射关系对所述原始电路模型进行仿射变换，得到所述原始电路模型上各个点在所述目标帧图像的视平面上的目标坐标；
    根据所述目标坐标生成并显示所述目标结构对应的目标电路模型。
  10. 根据权利要求6-8任一项所述的芯片结构的增强现实显示装置，其特征在于，还包括：
    模拟电路单元，用于在所述目标电路模型上显示所述目标结构工作时的电流信息、电压信息、电阻信息和开关通断状态中的一个或多个。
  11. 一种可读存储介质，其特征在于，其上存储有计算机程序，所述程序被处理器执行时实现权利要求1-5任一项所述方法的步骤。
PCT/CN2019/110180 2019-01-02 2019-10-09 芯片结构的增强现实显示方法、装置和可读存储介质 Ceased WO2020140524A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/419,884 US20220084291A1 (en) 2019-01-02 2019-10-09 Augmented reality display method and device for chip structure, and readable storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910001623.3A CN111402424A (zh) 2019-01-02 2019-01-02 芯片结构的增强现实显示方法、装置和可读存储介质
CN201910001623.3 2019-01-02

Publications (1)

Publication Number Publication Date
WO2020140524A1 true WO2020140524A1 (zh) 2020-07-09

Family

ID=71407253

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/110180 Ceased WO2020140524A1 (zh) 2019-01-02 2019-10-09 芯片结构的增强现实显示方法、装置和可读存储介质

Country Status (3)

Country Link
US (1) US20220084291A1 (zh)
CN (1) CN111402424A (zh)
WO (1) WO2020140524A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114330180A (zh) * 2021-12-30 2022-04-12 广东芯聚能半导体有限公司 芯片封装方法、装置、计算机设备和存储介质
CN117853559A (zh) * 2024-03-05 2024-04-09 上海楷领科技有限公司 一种基于芯片二维数据获取三维数据的方法及其相关装置

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11420649B2 (en) * 2020-03-24 2022-08-23 Here Global B.V. Method, apparatus, and computer program product for generating turn paths through an intersection
CN116167333B (zh) * 2023-04-25 2023-07-14 苏州浪潮智能科技有限公司 芯片验证方法和装置、电子设备和存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150379778A1 (en) * 2013-01-30 2015-12-31 Sean Finn Coordinate geometry augmented reality process for internal elements concealed behind an external element
CN106249881A (zh) * 2016-07-21 2016-12-21 江苏奥格视特信息科技有限公司 增强现实视场空间和虚拟三维目标动态配准方法
CN106530410A (zh) * 2016-11-04 2017-03-22 大连文森特软件科技有限公司 一种基于增强现实技术的汽车组装演示系统
CN108711188A (zh) * 2018-02-24 2018-10-26 石化盈科信息技术有限责任公司 一种基于ar的工厂实时数据展示方法及系统
CN108765575A (zh) * 2018-02-24 2018-11-06 石化盈科信息技术有限责任公司 一种基于ar的工业设备图鉴展示方法和系统

Family Cites Families (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7616807B2 (en) * 2005-02-24 2009-11-10 Siemens Corporate Research, Inc. System and method for using texture landmarks for improved markerless tracking in augmented reality applications
WO2007092316A2 (en) * 2006-02-03 2007-08-16 The University Of Florida Research Foundation, Inc. Image guidance system for deep brain stimulation
JP4668220B2 (ja) * 2007-02-20 2011-04-13 ソニー株式会社 画像処理装置および画像処理方法、並びにプログラム
US8898618B2 (en) * 2009-03-26 2014-11-25 Altera Corporation Interactive simplification of schematic diagram of integrated circuit design
FR2951565B1 (fr) * 2009-10-20 2016-01-01 Total Immersion Procede, programme d'ordinateur et dispositif de suivi hybride de representations d'objets, en temps reel, dans une sequence d'images
US9280850B2 (en) * 2010-11-08 2016-03-08 Sony Corporation Augmented reality system for communicating tagged video and data on a network
US9070216B2 (en) * 2011-12-14 2015-06-30 The Board Of Trustees Of The University Of Illinois Four-dimensional augmented reality models for interactive visualization and automated construction progress monitoring
JP6145966B2 (ja) * 2012-05-09 2017-06-14 ソニー株式会社 表示装置
US10824310B2 (en) * 2012-12-20 2020-11-03 Sri International Augmented reality virtual personal assistant for external representation
US9477312B2 (en) * 2012-11-05 2016-10-25 University Of South Australia Distance based modelling and manipulation methods for augmented reality systems using ultrasonic gloves
US10025486B2 (en) * 2013-03-15 2018-07-17 Elwha Llc Cross-reality select, drag, and drop for augmented reality systems
CN105741290B (zh) * 2016-01-29 2018-06-19 中国人民解放军国防科学技术大学 一种基于增强现实技术的印制电路板信息指示方法及装置
EP3242228A1 (en) * 2016-05-02 2017-11-08 Artag SARL Managing the display of assets in augmented reality mode
CN106919915B (zh) * 2017-02-22 2020-06-12 武汉极目智能技术有限公司 基于adas系统的地图道路标记及道路质量采集装置及方法
US10740613B1 (en) * 2017-04-20 2020-08-11 Digimarc Corporation Hybrid feature point/watermark-based augmented reality
JP2020522827A (ja) * 2017-04-21 2020-07-30 ザ ユナイテッド ステイツ ガバメント アズ リプレゼンティド バイ ザ デパートメント オブ ヴェテランズ アフェアーズ 外科ナビゲーションにおける拡張現実の使用
CN109492608B (zh) * 2018-11-27 2019-11-05 腾讯科技（深圳）有限公司 图像分割方法、装置、计算机设备及存储介质
US10871377B1 (en) * 2019-08-08 2020-12-22 Phiar Technologies, Inc. Computer-vision based positioning for augmented reality navigation

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150379778A1 (en) * 2013-01-30 2015-12-31 Sean Finn Coordinate geometry augmented reality process for internal elements concealed behind an external element
CN106249881A (zh) * 2016-07-21 2016-12-21 江苏奥格视特信息科技有限公司 增强现实视场空间和虚拟三维目标动态配准方法
CN106530410A (zh) * 2016-11-04 2017-03-22 大连文森特软件科技有限公司 一种基于增强现实技术的汽车组装演示系统
CN108711188A (zh) * 2018-02-24 2018-10-26 石化盈科信息技术有限责任公司 一种基于ar的工厂实时数据展示方法及系统
CN108765575A (zh) * 2018-02-24 2018-11-06 石化盈科信息技术有限责任公司 一种基于ar的工业设备图鉴展示方法和系统

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114330180A (zh) * 2021-12-30 2022-04-12 广东芯聚能半导体有限公司 芯片封装方法、装置、计算机设备和存储介质
CN117853559A (zh) * 2024-03-05 2024-04-09 上海楷领科技有限公司 一种基于芯片二维数据获取三维数据的方法及其相关装置
CN117853559B (zh) * 2024-03-05 2024-06-04 上海楷领科技有限公司 一种基于芯片二维数据获取三维数据的方法及其相关装置

Also Published As

Publication number Publication date
CN111402424A (zh) 2020-07-10
US20220084291A1 (en) 2022-03-17

Similar Documents

Publication Publication Date Title
CN109584295B (zh) 对图像内目标物体进行自动标注的方法、装置及系统
US9807263B2 (en) Mobile document capture assistance using augmented reality
TWI587205B (zh) Method and system of three - dimensional interaction based on identification code
JP6264834B2 (ja) ガイド方法、情報処理装置およびガイドプログラム
CN110111388B (zh) 三维物体位姿参数估计方法及视觉设备
JP6056319B2 (ja) 画像処理装置、画像処理方法および画像処理プログラム
US20220084291A1 (en) Augmented reality display method and device for chip structure, and readable storage medium
JP6096634B2 (ja) 仮想現実を用いた３次元地図表示システム
JP6293386B2 (ja) データ処理装置、データ処理方法及びデータ処理プログラム
CN115035032B (zh) 神经网络训练方法、相关方法、设备、终端及存储介质
CN112950528B (zh) 证件姿态确定方法、模型训练方法、装置、服务器及介质
TW202011353A (zh) 深度資料處理系統的操作方法
KR20100065918A (ko) 사진의 촬영 위치 및 방향 정보 태깅 방법과 그 장치
CN116206320B (zh) 图纸特征与数字匹配的方法、系统、设备及介质
CN105719586A (zh) 透明显示方法和透明显示装置
CN112529097B (zh) 样本图像生成方法、装置以及电子设备
CN105741290A (zh) 一种基于增强现实技术的印制电路板信息指示方法及装置
JP6017343B2 (ja) データベース生成装置、カメラ姿勢推定装置、データベース生成方法、カメラ姿勢推定方法、およびプログラム
JP6304815B2 (ja) 画像処理装置ならびにその画像特徴検出方法、プログラムおよび装置
WO2020220832A1 (zh) 实现投影画面拼接的方法、装置及投影系统
JP2015194424A (ja) 情報処理装置、情報処理方法、情報処理プログラムおよびその情報処理プログラムが記録された記録媒体
CN115830135B (zh) 一种图像处理方法、装置及电子设备
CN116843565B (zh) 一种图像透视变换还原的方法、装置、电子设备及介质
CN117253022A (zh) 一种对象识别方法、装置及查验设备
CN116862992A (zh) 确定监测相机可视域的方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19907917

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19907917

Country of ref document: EP

Kind code of ref document: A1