WO2023029122A1 - Method and apparatus for restoring image occlusion area, device, and storage medium - Google Patents
Method and apparatus for restoring image occlusion area, device, and storage medium Download PDFInfo
- Publication number
- WO2023029122A1 WO2023029122A1 PCT/CN2021/119729 CN2021119729W WO2023029122A1 WO 2023029122 A1 WO2023029122 A1 WO 2023029122A1 CN 2021119729 W CN2021119729 W CN 2021119729W WO 2023029122 A1 WO2023029122 A1 WO 2023029122A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- occluded
- area
- occlusion
- feature points
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims abstract description 64
- 230000009466 transformation Effects 0.000 claims abstract description 58
- 239000011159 matrix material Substances 0.000 claims abstract description 34
- 238000004590 computer program Methods 0.000 claims description 12
- 239000013598 vector Substances 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 7
- 208000006440 Open Bite Diseases 0.000 claims description 5
- 238000010586 diagram Methods 0.000 description 6
- 230000003190 augmentative effect Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000001404 mediated effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
Disclosed in the present invention are a method and apparatus for restoring an image occlusion area, a device, and a computer readable storage medium. The method comprises: performing affine transformation on an occluded image and a non-occluded image according to the acquired matching feature points in the occluded image and the non-occluded image to obtain an affine transformation matrix; determining pixel point coordinates of an occlusion area in the occluded image according to occlusion area selection information corresponding to the occluded image; and determining pixel point coordinates of a restoration area in the non-occluded image according to the pixel point coordinates of the occlusion area and the affine transformation matrix, and restoring the pixel points of the occlusion area by using the pixel points of the restoration area to obtain a restored image corresponding to the occluded image. According to the present invention, information of the occluded part is determined by using affine transformation established according to the correspondence between the feature points in two images, so that the restored image required by a user is obtained, the occlusion area in the occluded image is restored, and the user experience is improved.
Description
本申请要求于2021年8月30日提交中国专利局、申请号为202111006783.0、发明名称为“一种图像的遮挡区域还原方法、装置、设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。This application claims the priority of the Chinese patent application submitted to the China Patent Office on August 30, 2021, with the application number 202111006783.0, and the title of the invention is "A Method, Device, Equipment, and Storage Medium for Restoring Occlusion Areas of Images", all of which The contents are incorporated by reference in this application.
本发明涉及图像处理计算领域，特别涉及一种图像的遮挡区域还原方法、装置、设备及计算机可读存储介质。The present invention relates to the field of image processing and computing, and in particular to a method, device, equipment and computer-readable storage medium for restoring an occluded area of an image.
目前，如智能手机、VR(Virtual Reality，虚拟现实)设备、AR(Augmented Reality，增强现实)设备和MR(Mediated Reality，混合现实)设备等智能设备有着强大的拍照功能。在人们日常生活中或者外出旅行中，大多会用照片记录那些美好的时刻和风景；在拍照中，尤其是广角拍摄，可能会出现有些照片中有遮挡(如被手指遮挡)的情况，但是这些照片除了遮挡区域外又恰好是被需要的。因此，如何能够还原出图像中被遮挡部分的信息，提升用户体验，是现今急需解决的问题。At present, smart devices such as smartphones, VR (Virtual Reality, virtual reality) devices, AR (Augmented Reality, augmented reality) devices, and MR (Mediated Reality, mixed reality) devices have powerful camera functions. In people's daily life or when traveling, most of them use photos to record those beautiful moments and scenery; when taking pictures, especially wide-angle shooting, there may be some occlusions in some photos (such as being blocked by fingers), but these The photo happens to be needed in addition to the occluded area. Therefore, how to restore the information of the occluded part of the image and improve the user experience is an urgent problem to be solved today.
发明内容Contents of the invention
本发明的目的是提供一种图像的遮挡区域还原方法、装置、设备及计算机可读存储介质，以利用同一场景下的拍摄到的无遮挡图像来还原存被遮挡图像中因遮挡丢失的信息，从而得到用户所需的图像，提升用户体验。The object of the present invention is to provide a method, device, device and computer-readable storage medium for restoring an occluded area of an image, so as to restore the information lost due to occlusion in the occluded image by using the unoccluded image captured in the same scene. In this way, the image required by the user can be obtained, and the user experience can be improved.
为解决上述技术问题，本发明提供一种图像的遮挡区域还原方法，包括：In order to solve the above-mentioned technical problems, the present invention provides a method for restoring an occluded area of an image, including:
根据获取的被遮挡图像和无遮挡图像中的匹配特征点，对所述被遮挡图像和所述无遮挡图像进行仿射变换，得到仿射变换矩阵；Performing affine transformation on the occluded image and the unoccluded image according to the matching feature points in the acquired occluded image and the unoccluded image to obtain an affine transformation matrix;
根据所述被遮挡图像对应的遮挡区域选择信息，确定所述被遮挡图像中的遮挡区域的像素点坐标；Determine the pixel coordinates of the occlusion area in the occlusion image according to the occlusion area selection information corresponding to the occlusion image;
根据所述遮挡区域的像素点坐标和所述仿射变换矩阵，确定所述无遮挡 图像中的还原区域的像素点坐标，并利用所述还原区域的像素点对所述遮挡区域的像素点进行还原，得到所述被遮挡图像对应的还原图像。According to the pixel point coordinates of the occlusion area and the affine transformation matrix, determine the pixel point coordinates of the restoration area in the non-occlusion image, and use the pixels of the restoration area to carry out the pixel points of the occlusion area Restore to obtain a restored image corresponding to the occluded image.
可选的，所述根据获取的被遮挡图像和无遮挡图像中匹配特征点，对所述被遮挡图像和所述无遮挡图像进行仿射变换，得到仿射变换矩阵，包括：Optionally, the affine transformation is performed on the occluded image and the unoccluded image according to the matching feature points in the acquired occluded image and the unoccluded image to obtain an affine transformation matrix, including:
对所述被遮挡图像和所述无遮挡图像中的特征点进行匹配，获取所述匹配特征点；Matching the feature points in the occluded image and the unoccluded image to obtain the matching feature points;
根据所述匹配特征点，对所述被遮挡图像和所述无遮挡图像进行仿射变换，得到仿射变换矩阵；其中，所述仿射变换矩阵包括所述匹配特征点中的正确匹配特征点和所述正确匹配特征点之外的未匹配特征点。Perform affine transformation on the occluded image and the unoccluded image according to the matching feature points to obtain an affine transformation matrix; wherein the affine transformation matrix includes correct matching feature points among the matching feature points and the unmatched feature points other than the correct matching feature points.
可选的，所述对所述被遮挡图像和所述无遮挡图像中的特征点进行匹配，获取所述匹配特征点，包括：Optionally, the matching the feature points in the occluded image and the unoccluded image, and obtaining the matching feature points include:
提取所述被遮挡图像和所述无遮挡图像中的特征点；extracting feature points in the occluded image and the unoccluded image;
根据所述特征点，生成被遮挡图像和所述无遮挡图像的特征向量；Generate feature vectors of the occluded image and the unoccluded image according to the feature points;
根据所述特征向量，确定所述被遮挡图像和所述无遮挡图像中的所述匹配特征点。Determine the matching feature points in the occluded image and the unoccluded image according to the feature vector.
可选的，所述提取所述被遮挡图像和所述无遮挡图像中的特征点，包括：Optionally, the extracting the feature points in the occluded image and the unoccluded image includes:
利用SURF特征提取算法，提取所述被遮挡图像和所述无遮挡图像中的特征点。Using a SURF feature extraction algorithm to extract feature points in the occluded image and the unoccluded image.
可选的，该方法还包括：Optionally, the method also includes:
获取第一图像和第二图像；acquire the first image and the second image;
根据所述第一图像对应的遮挡区域选择信息，将所述第一图像确定为所述被遮挡图像，并将所述第二图像确定为所述无遮挡图像。Determining the first image as the occluded image and determining the second image as the unoccluded image according to the occlusion area selection information corresponding to the first image.
可选的，所述根据所述被遮挡图像对应的遮挡区域选择信息，确定所述被遮挡图像中的遮挡区域的像素点坐标，包括：Optionally, the determining the pixel coordinates of the occlusion area in the occlusion image according to the occlusion area selection information corresponding to the occlusion image includes:
根据所述被遮挡图像对应的遮挡区域选择点坐标，确定所述遮挡区域的像素点坐标。Determine the pixel point coordinates of the occluded area according to the selected point coordinates of the occluded area corresponding to the occluded image.
可选的，所述根据所述被遮挡图像对应的遮挡区域选择点坐标，确定所述遮挡区域的像素点坐标，包括：Optionally, the selecting point coordinates according to the occlusion area corresponding to the occluded image, and determining the pixel point coordinates of the occlusion area include:
利用区域生长算法，将所述被遮挡图像中所述遮挡区域选择点坐标对应的区域确定为所述遮挡区域，并获取所述所述遮挡区域的像素点坐标。Using a region growing algorithm, determine the area corresponding to the selected point coordinates of the occluded area in the occluded image as the occluded area, and obtain the pixel point coordinates of the occluded area.
本发明还提供了一种图像的遮挡区域还原装置，包括：The present invention also provides an image occlusion area restoring device, comprising:
特征仿射模块，用于根据获取的被遮挡图像和无遮挡图像中的匹配特征点，对所述被遮挡图像和所述无遮挡图像进行仿射变换，得到仿射变换矩阵；A feature affine module, configured to perform affine transformation on the occluded image and the unoccluded image according to the acquired matched feature points in the occluded image and the unoccluded image, to obtain an affine transformation matrix;
遮挡确定模块，用于根据所述被遮挡图像对应的遮挡区域选择信息，确定所述被遮挡图像中的遮挡区域的像素点坐标；An occlusion determination module, configured to determine the pixel point coordinates of the occlusion area in the occlusion image according to the occlusion area selection information corresponding to the occlusion image;
遮挡还原模块，用于根据所述遮挡区域的像素点坐标和所述仿射变换矩阵，确定所述无遮挡图像中的还原区域的像素点坐标，并利用所述还原区域的像素点对所述遮挡区域的像素点进行还原，得到所述被遮挡图像对应的还原图像。An occlusion restoration module, configured to determine the pixel coordinates of the restoration area in the non-occlusion image according to the pixel coordinates of the occlusion area and the affine transformation matrix, and use the pixels of the restoration area to map the The pixels in the occluded area are restored to obtain a restored image corresponding to the occluded image.
本发明还提供了一种图像的遮挡区域还原设备，包括：The present invention also provides an image occlusion area restoration device, including:
存储器，用于存储计算机程序；memory for storing computer programs;
处理器，用于执行所述计算机程序时实现如上述所述的图像的遮挡区域还原方法的步骤。The processor is configured to realize the steps of the method for restoring an occluded area of an image as described above when executing the computer program.
此外，本发明还提供了一种计算机可读存储介质，所述计算机可读存储介质上存储有计算机程序，所述计算机程序被处理器执行时实现如上述所述的图像的遮挡区域还原方法的步骤。In addition, the present invention also provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method for restoring an occluded area of an image as described above is implemented. step.
本发明所提供的一种图像的遮挡区域还原方法，包括：根据获取的被遮挡图像和无遮挡图像中的匹配特征点，对被遮挡图像和无遮挡图像进行仿射变换，得到仿射变换矩阵；根据被遮挡图像对应的遮挡区域选择信息，确定被遮挡图像中的遮挡区域的像素点坐标；根据遮挡区域的像素点坐标和仿射变换矩阵，确定无遮挡图像中的还原区域的像素点坐标，并利用还原区域的像素点对遮挡区域的像素点进行还原，得到被遮挡图像对应的还原图像；A method for restoring an occluded area of an image provided by the present invention includes: performing affine transformation on the occluded image and the unoccluded image according to the matching feature points in the acquired occluded image and the unoccluded image to obtain an affine transformation matrix ; Determine the pixel point coordinates of the occluded area in the occluded image according to the occluded area selection information corresponding to the occluded image; determine the pixel point coordinates of the restored area in the unoccluded image according to the pixel point coordinates of the occluded area and the affine transformation matrix , and use the pixels in the restored area to restore the pixels in the occluded area to obtain a restored image corresponding to the occluded image;
可见，本发明通过根据获取的被遮挡图像和无遮挡图像中的匹配特征点，对被遮挡图像和无遮挡图像进行仿射变换，得到仿射变换矩阵，利用由两张图像中特征点的对应关系建立的仿射变换来确定被遮挡部分的信息，从而得到用户所需要的还原图像，实现了对被遮挡图像中遮挡区域的还原，提升了用户体验。此外，本发明还提供了一种图像的遮挡区域还原装置、设备及计算机可读存储介质，同样具有上述有益效果。It can be seen that the present invention performs affine transformation on the occluded image and the unoccluded image according to the matching feature points in the acquired occluded image and the unoccluded image to obtain an affine transformation matrix, and utilizes the correspondence between the feature points in the two images The affine transformation established by the relationship is used to determine the information of the occluded part, so as to obtain the restored image required by the user, realize the restoration of the occluded area in the occluded image, and improve the user experience. In addition, the present invention also provides an image occlusion area restoration device, equipment and computer-readable storage medium, which also have the above beneficial effects.
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据提供的附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only It is an embodiment of the present invention, and those skilled in the art can also obtain other drawings according to the provided drawings without creative work.
图1为本发明实施例所提供的一种图像的遮挡区域还原方法的流程图；FIG. 1 is a flowchart of a method for restoring an occluded area of an image provided by an embodiment of the present invention;
图2为本发明实施例所提供的一种图像的遮挡区域还原装置的结构框图；FIG. 2 is a structural block diagram of an image occlusion region restoration device provided by an embodiment of the present invention;
图3为本发明实施例所提供的一种图像的遮挡区域还原设备的结构示意图；FIG. 3 is a schematic structural diagram of an image occlusion region restoration device provided by an embodiment of the present invention;
图4为本发明实施例所提供的一种图像的遮挡区域还原设备的具体结构示意图。FIG. 4 is a schematic structural diagram of an image occlusion region restoration device provided by an embodiment of the present invention.
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.
请参考图1，图1为本发明实施例所提供的一种图像的遮挡区域还原方法的流程图。该方法可以包括：Please refer to FIG. 1 , which is a flow chart of a method for restoring an occluded area of an image provided by an embodiment of the present invention. The method can include:
步骤101：根据获取的被遮挡图像和无遮挡图像中的匹配特征点，对被遮挡图像和无遮挡图像进行仿射变换，得到仿射变换矩阵。Step 101: Perform affine transformation on the occluded image and the unoccluded image according to the acquired matching feature points in the occluded image and the unoccluded image to obtain an affine transformation matrix.
其中，本步骤中的被遮挡图像可以包括需要被还原的遮挡区域的图像，如被手指遮挡的部分区域的照片；本步骤中的无遮挡图像可以为包括被遮挡图像中遮挡区域对应的无遮挡区域(即还原区域)的图像，如与被遮挡图像在同一场景下的遮挡区域未被遮挡的图像，例如与被遮挡图像在同一场景下拍摄的未手指遮挡的照片。Wherein, the occluded image in this step may include an image of an occluded area that needs to be restored, such as a photo of a partial area covered by a finger; the unoccluded image in this step may include an unoccluded The image of the area (that is, the restored area), such as the unoccluded image of the occluded area in the same scene as the occluded image, for example, the unoccluded photo taken in the same scene as the occluded image.
具体的，本步骤中的匹配特征点可以为被遮挡图像和无遮挡图像中存在匹配关系的特征点。本步骤中通过对被遮挡图像和无遮挡图像进行仿射变换，可以将被遮挡图像和无遮挡图像中的共有信息利用仿射变换对应起来，从而去除掉被遮挡图像和无遮挡图像之间的特征点匹配关系(即匹配特征点)中的不正确的匹配特征点，得到仿射变换矩阵，以利用仿射变换矩阵得到被遮 挡图像中的遮挡区域对应的还原信息，即无遮挡图像中还原区域的像素点坐标。Specifically, the matching feature points in this step may be feature points in a matching relationship between the occluded image and the unoccluded image. In this step, by performing affine transformation on the occluded image and the unoccluded image, the shared information in the occluded image and the unoccluded image can be mapped using affine transformation, thereby removing the gap between the occluded image and the unoccluded image. Incorrect matching feature points in the feature point matching relationship (that is, matching feature points) to obtain an affine transformation matrix, so as to use the affine transformation matrix to obtain the restoration information corresponding to the occlusion area in the occluded image, that is, the restoration information in the unoccluded image The pixel coordinates of the region.
需要说明的是，对于本步骤中处理器根据获取的被遮挡图像和无遮挡图像中的匹配特征点，对被遮挡图像和无遮挡图像进行仿射变换，得到仿射变换矩阵的具体方式，可以由设计人员根据实用场景和用户需求自行设置，如处理器可以对被遮挡图像和无遮挡图像中的特征点进行匹配，获取匹配特征点；根据匹配特征点，对被遮挡图像和无遮挡图像进行仿射变换，得到仿射变换矩阵；其中，仿射变换矩阵包括匹配特征点中的正确匹配特征点和正确匹配特征点之外的未匹配特征点。处理器也可以直接接收被遮挡图像、无遮挡图像和被遮挡图像与无遮挡图像之间的匹配特征点，本实施例对此不做任何限制。It should be noted that, in this step, the processor performs affine transformation on the occluded image and the unoccluded image according to the matching feature points in the acquired occluded image and the unoccluded image to obtain the affine transformation matrix. It is set by the designer according to practical scenarios and user needs. For example, the processor can match the feature points in the occluded image and the unoccluded image to obtain the matching feature points; according to the matching feature points, the occluded image and the unoccluded image Affine transformation to obtain an affine transformation matrix; wherein, the affine transformation matrix includes correct matching feature points among the matching feature points and unmatched feature points other than the correct matching feature points. The processor may also directly receive the occluded image, the unoccluded image, and the matching feature points between the occluded image and the unoccluded image, which is not limited in this embodiment.
对应的，对于上述处理器对被遮挡图像和无遮挡图像中的特征点进行匹配，获取匹配特征点的具体方式，可以由设计人员自行设置，如处理器可以提取被遮挡图像和无遮挡图像中的特征点；根据特征点，生成被遮挡图像和无遮挡图像的特征向量；根据特征向量，确定被遮挡图像和无遮挡图像中的匹配特征点。处理器也可以直接接收被遮挡图像、无遮挡图像和被遮挡图像与无遮挡图像中的特征点。本实施例对此不做任何限制。Correspondingly, for the above-mentioned processor to match the feature points in the occluded image and the unoccluded image, the specific way to obtain the matching feature points can be set by the designer. For example, the processor can extract the occluded image and the unoccluded image. The feature points; according to the feature points, generate the feature vectors of the occluded image and the unoccluded image; according to the feature vectors, determine the matching feature points in the occluded image and the unoccluded image. The processor may also directly receive feature points in the occluded image, the unoccluded image, and the occluded image and the unoccluded image. This embodiment does not impose any limitation on this.
同样的，对于上述处理器提取被遮挡图像和无遮挡图像中的特征点的具体方式，即特征点的具体类型，可以由设计人员自行设置，如特征点可以为SURF(Speeded Up Robust Features，加速稳健特征)特征点，即处理器可以利用SURF特征提取算法，提取被遮挡图像和无遮挡图像中的特征点；特征点也可以HOG(Histogram of Oriented Gradient，方向梯度直方图)特征点或LBP(Local Binary Pattern，局部二值模式)特征点的其他类型特征点，例如处理器可以利用HOG特征提取算法，提取被遮挡图像和无遮挡图像中的特征点。只要处理器可以从获取的被遮挡图像和无遮挡图像中提取出相应的特征点，本实施例对此不做任何限制。Similarly, the specific method for the above-mentioned processor to extract the feature points in the occluded image and the unoccluded image, that is, the specific type of the feature point, can be set by the designer, such as the feature point can be SURF (Speeded Up Robust Features, accelerated Robust features) feature points, that is, the processor can use the SURF feature extraction algorithm to extract feature points in occluded images and unoccluded images; feature points can also be HOG (Histogram of Oriented Gradient, histogram of oriented gradient) feature points or LBP ( Local Binary Pattern, other types of feature points of feature points, for example, the processor can use the HOG feature extraction algorithm to extract feature points in occluded images and unoccluded images. As long as the processor can extract corresponding feature points from the acquired occluded image and the unoccluded image, this embodiment does not impose any limitation on this.
其中，本实施例并不限定步骤101与步骤102的具体先后顺序，如可以先进行步骤101再进行步骤102，也可以先进行步骤102再进行步骤101，还可以步骤101和步骤102同时进行。只要步骤103可以利用步骤101和步骤102得到的仿射变换矩阵和遮挡区域的像素点坐标，确定无遮挡图像中的还原 区域的像素点坐标，本实施例对此不做任何限制。Wherein, this embodiment does not limit the specific sequence of steps 101 and 102, for example, step 101 can be performed first and then step 102 can be performed, step 102 can be performed first and then step 101 can be performed, or step 101 and step 102 can be performed simultaneously. As long as step 103 can use the affine transformation matrix obtained in step 101 and step 102 and the pixel coordinates of the occluded area to determine the pixel coordinates of the restoration area in the unoccluded image, this embodiment does not impose any restrictions on this.
对应的，本实施例所提供的方法在步骤101和步骤102之前，还可以包括识别被遮挡图像和无遮挡图像的过程；如处理器可以获取第一图像和第二图像；根据第一图像对应的遮挡区域选择信息，将第一图像确定为被遮挡图像，并将第二图像确定为无遮挡图像。也就是说，处理器可以在接收到两张图像(即第一图像和第二图像)后，将遮挡区域选择信息对应的图像(即第一图像)确定为被遮挡图像，将另一图像(即第二图像)确定为无遮挡图像，实现被遮挡图像和无遮挡图像的自动识别。Correspondingly, before step 101 and step 102, the method provided by this embodiment may also include the process of identifying the occluded image and the unoccluded image; for example, the processor may acquire the first image and the second image; The occlusion area selection information of , determines the first image as the occluded image, and determines the second image as the unoccluded image. That is to say, after receiving two images (ie, the first image and the second image), the processor may determine the image corresponding to the occlusion region selection information (ie, the first image) as the occluded image, and determine the other image (ie, the first image) as the occluded image. That is, the second image) is determined as an unoccluded image to realize automatic identification of an occluded image and an unoccluded image.
步骤102：根据被遮挡图像对应的遮挡区域选择信息，确定被遮挡图像中的遮挡区域的像素点坐标。Step 102: Determine the pixel point coordinates of the occluded area in the occluded image according to the occluded area selection information corresponding to the occluded image.
其中，本步骤中的遮挡区域选择信息可以为被遮挡图像中的遮挡区域的选择信息，如用户点击遮挡区域时的遮挡区域中用户的选择点坐标(即遮挡区域选择点坐标)，或者用户圈选遮挡区域时的圈选区域的坐标(即圈选区域坐标)。Wherein, the occlusion area selection information in this step can be the selection information of the occlusion area in the occluded image, such as the coordinates of the user's selection point in the occlusion area when the user clicks on the occlusion area (i.e. the coordinates of the selection point of the occlusion area), or the user circle The coordinates of the circled area when the masked area is selected (that is, the coordinates of the circled area).
具体的，对于本步骤中处理器根据被遮挡图像对应的遮挡区域选择信息，确定被遮挡图像中的遮挡区域的像素点坐标的具体方式，可以由设计人员根据实用场景和用户需求自行设置，如遮挡区域选择信息为遮挡区域选择点坐标时，处理器可以利用区域生长算法，将被遮挡图像中遮挡区域选择点坐标对应的区域确定为遮挡区域，并获取遮挡区域的像素点坐标；也就是说，处理器可以使用区域生长算法来确定被遮挡图像中遮挡区域(如手指遮挡区域)的具体范围，以遮挡区域内用户选择的一个像素点(即遮挡区域选择点坐标)作为种子点，然后从该种子点进行生长，直到边缘处停止生长，这样就可以得到整个遮挡区域的像素点的坐标。遮挡区域选择信息为遮挡区域选择点坐标时，处理器也可以利用如分水岭算法或超像素分割算法的其他数字图像处理算法，将被遮挡图像中遮挡区域选择点坐标对应的分割区域确定为遮挡区域，并获取遮挡区域的像素点坐标；例如处理器可以利用分水岭算法，将分割得到的包含遮挡区域选择点坐标的区域确定为遮挡区域，并获取遮挡区域的像素点坐标。遮挡区域选择信息为圈选区域坐标时，处理器可以分水岭算法，将分割得到的圈选区域坐标对应的区域内的分割区域确定为遮挡区域，并获取遮挡区域的像素点坐标。本实施例对此不做任何限制。Specifically, in this step, the processor determines the pixel coordinates of the occluded area in the occluded image according to the occluded area selection information corresponding to the occluded image, which can be set by the designer according to practical scenarios and user needs, such as When the occlusion area selection information is the coordinates of the selection point of the occlusion area, the processor can use the region growing algorithm to determine the area corresponding to the coordinates of the selection point of the occlusion area in the occluded image as the occlusion area, and obtain the pixel point coordinates of the occlusion area; , the processor can use the region growing algorithm to determine the specific range of the occluded area (such as the finger occluded area) in the occluded image, and use a pixel point selected by the user in the occluded area (ie, the coordinates of the selected point of the occluded area) as the seed point, and then from The seed point grows until it stops growing at the edge, so that the coordinates of the pixels in the entire occlusion area can be obtained. When the occlusion area selection information is the occlusion area selection point coordinates, the processor can also use other digital image processing algorithms such as watershed algorithm or superpixel segmentation algorithm to determine the segmentation area corresponding to the occlusion area selection point coordinates in the occluded image as the occlusion area , and obtain the pixel coordinates of the occluded area; for example, the processor can use the watershed algorithm to determine the segmented area containing the selected point coordinates of the occluded area as an occluded area, and obtain the pixel coordinates of the occluded area. When the occlusion area selection information is the coordinates of the encircled area, the processor may use a watershed algorithm to determine the segmented area in the area corresponding to the obtained encircled area coordinates as an occluded area, and obtain pixel coordinates of the occluded area. This embodiment does not impose any limitation on this.
步骤103：根据遮挡区域的像素点坐标和仿射变换矩阵，确定无遮挡图像中的还原区域的像素点坐标，并利用还原区域的像素点对遮挡区域的像素点进行还原，得到被遮挡图像对应的还原图像。Step 103: According to the pixel point coordinates of the occluded area and the affine transformation matrix, determine the pixel point coordinates of the restored area in the unoccluded image, and use the pixels in the restored area to restore the pixels in the occluded area to obtain the corresponding restore image.
可以理解的是，本步骤中处理器可以通过遮挡区域的像素点坐标与仿射变换矩阵的运算，得到遮挡区域的像素点对应的在无遮挡图像中的像素点坐标，以通过无遮挡图像中的像素点还原遮挡区域中对应的像素点，从而将被遮挡图像中遮挡区域的像素点还原为无遮挡图像中还原区域相应的像素点，得到还原图像，即遮挡区域的像素点被还原为还原区域的像素点的被遮挡图像。It can be understood that in this step, the processor can obtain the pixel coordinates in the unoccluded image corresponding to the pixel points in the occluded area through the operation of the pixel point coordinates of the occluded area and the affine transformation matrix, so as to pass through the unoccluded image. The corresponding pixels in the occluded area are restored by the pixels of the occluded area, so that the pixels in the occluded area in the occluded image are restored to the corresponding pixels in the unoccluded image in the restored area, and the restored image is obtained, that is, the pixels in the occluded area are restored to restore The occluded image of the pixels of the region.
本实施例中，本发明实施例通过根据获取的被遮挡图像和无遮挡图像中的匹配特征点，对被遮挡图像和无遮挡图像进行仿射变换，得到仿射变换矩阵，利用由两张图像中特征点的对应关系建立的仿射变换来确定被遮挡部分的信息，从而得到用户所需要的还原图像，实现了对被遮挡图像中遮挡区域的还原，提升了用户体验。In this embodiment, the embodiment of the present invention performs affine transformation on the occluded image and the unoccluded image according to the matching feature points in the acquired occluded image and the unoccluded image to obtain an affine transformation matrix, and utilizes the two images The affine transformation established by the corresponding relationship of the feature points in the image is used to determine the information of the occluded part, so as to obtain the restored image required by the user, realize the restoration of the occluded area in the occluded image, and improve the user experience.
相应于上面的方法实施例，本发明实施例还提供了一种图像的遮挡区域还原装置，下文描述的一种图像的遮挡区域还原装置与上文描述的一种图像的遮挡区域还原方法可相互对应参照。Corresponding to the above method embodiment, the embodiment of the present invention also provides an image occlusion area restoration device, the image occlusion area restoration device described below and the image occlusion area restoration method described above can be mutually Corresponding reference.
请参考图2，图2为本发明实施例所提供的一种图像的遮挡区域还原装置的结构框图。该装置可以包括：Please refer to FIG. 2 , which is a structural block diagram of an apparatus for restoring an image occlusion area provided by an embodiment of the present invention. The device can include:
特征仿射模块10，用于根据获取的被遮挡图像和无遮挡图像中的匹配特征点，对被遮挡图像和无遮挡图像进行仿射变换，得到仿射变换矩阵；The characteristic affine module 10 is used for carrying out affine transformation to the occluded image and the unoccluded image according to the matching feature points in the acquired occluded image and the unoccluded image, to obtain an affine transformation matrix;
遮挡确定模块20，用于根据被遮挡图像对应的遮挡区域选择信息，确定被遮挡图像中的遮挡区域的像素点坐标；The occlusion determination module 20 is configured to determine the pixel coordinates of the occlusion area in the occlusion image according to the occlusion area selection information corresponding to the occlusion image;
遮挡还原模块30，用于根据遮挡区域的像素点坐标和仿射变换矩阵，确定无遮挡图像中的还原区域的像素点坐标，并利用还原区域的像素点对遮挡区域的像素点进行还原，得到被遮挡图像对应的还原图像。The occlusion restoration module 30 is used to determine the pixel coordinates of the restoration area in the non-occlusion image according to the pixel point coordinates of the occlusion area and the affine transformation matrix, and use the pixels of the restoration area to restore the pixels of the occlusion area to obtain The restored image corresponding to the occluded image.
可选的，特征仿射模块10可以包括：Optionally, the feature affine module 10 may include:
特征匹配子模块，用于对被遮挡图像和无遮挡图像中的特征点进行匹配，获取匹配特征点；The feature matching submodule is used to match the feature points in the occluded image and the unoccluded image, and obtain the matching feature points;
仿射变换子模块，用于根据匹配特征点，对被遮挡图像和无遮挡图像进行仿射变换，得到仿射变换矩阵；其中，仿射变换矩阵包括匹配特征点中的正确匹配特征点和正确匹配特征点之外的未匹配特征点。The affine transformation sub-module is used to perform affine transformation on the occluded image and the unoccluded image according to the matching feature points to obtain an affine transformation matrix; wherein, the affine transformation matrix includes the correct matching feature points and the correct matching feature points in the matching feature points Unmatched feature points other than matched feature points.
可选的，特征匹配子模块可以包括：Optionally, the feature matching submodule may include:
特征点提取单元，用于提取被遮挡图像和无遮挡图像中的特征点；A feature point extraction unit is used to extract feature points in the occluded image and the non-occluded image;
特征向量生成单元，用于根据特征点，生成被遮挡图像和无遮挡图像的特征向量；A feature vector generation unit is used to generate feature vectors of the occluded image and the unoccluded image according to the feature points;
特征匹配单元，用于根据特征向量，确定被遮挡图像和无遮挡图像中的匹配特征点。The feature matching unit is configured to determine matching feature points in the occluded image and the unoccluded image according to the feature vectors.
可选的，特征点提取单元可以具体用于利用SURF特征提取算法，提取被遮挡图像和无遮挡图像中的特征点。Optionally, the feature point extraction unit may be specifically configured to use the SURF feature extraction algorithm to extract feature points in the occluded image and the unoccluded image.
可选的，该装置还可以包括：Optionally, the device may also include:
图像获取单元，用于获取第一图像和第二图像；an image acquisition unit, configured to acquire the first image and the second image;
图像确定单元，用于根据第一图像对应的遮挡区域选择信息，将第一图像确定为被遮挡图像，并将第二图像确定为无遮挡图像。The image determination unit is configured to determine the first image as the occluded image and the second image as the unoccluded image according to the occlusion area selection information corresponding to the first image.
可选的，遮挡确定模块20可以包括：Optionally, the occlusion determination module 20 may include:
点击确定子模块，用于根据被遮挡图像对应的遮挡区域选择点坐标，确定遮挡区域的像素点坐标。The click determination sub-module is used to select point coordinates according to the occluded area corresponding to the occluded image, and determine the pixel point coordinates of the occluded area.
可选的，点击确定子模块可以具体用于利用区域生长算法，将被遮挡图像中遮挡区域选择点坐标对应的区域确定为遮挡区域，并获取遮挡区域的像素点坐标。Optionally, the click determination sub-module may be specifically configured to use a region growing algorithm to determine the area corresponding to the selected point coordinates of the occluded area in the occluded image as the occluded area, and obtain the pixel point coordinates of the occluded area.
本实施例中，本发明实施例通过特征仿射模块10根据获取的被遮挡图像和无遮挡图像中的匹配特征点，对被遮挡图像和无遮挡图像进行仿射变换，得到仿射变换矩阵，利用由两张图像中特征点的对应关系建立的仿射变换来确定被遮挡部分的信息，从而得到用户所需要的还原图像，实现了对被遮挡图像中遮挡区域的还原，提升了用户体验。In this embodiment, the embodiment of the present invention uses the characteristic affine module 10 to perform affine transformation on the occluded image and the unoccluded image according to the matching feature points in the acquired occluded image and the unoccluded image to obtain an affine transformation matrix, Using the affine transformation established by the corresponding relationship between the feature points in the two images to determine the information of the occluded part, so as to obtain the restored image required by the user, realize the restoration of the occluded area in the occluded image, and improve the user experience.
相应于上面的方法实施例，本发明实施例还提供了一种图像的遮挡区域还原设备，下文描述的一种图像的遮挡区域还原设备与上文描述的一种图像的遮挡区域还原方法可相互对应参照。Corresponding to the above method embodiment, the embodiment of the present invention also provides an image occlusion area restoration device, the image occlusion area restoration device described below and the image occlusion area restoration method described above can be mutually Corresponding reference.
请参考图3，图3为本发明实施例所提供的一种图像的遮挡区域还原设备的结构示意图。该检测设备可以包括：Please refer to FIG. 3 . FIG. 3 is a schematic structural diagram of an image occlusion area restoring device provided by an embodiment of the present invention. The detection equipment may include:
存储器D1，用于存储计算机程序；memory D1 for storing computer programs;
处理器D2，用于执行计算机程序时实现上述方法实施例所提供的图像的遮挡区域还原方法的步骤。The processor D2 is configured to implement the steps of the method for restoring an occluded area of an image provided by the above method embodiments when executing a computer program.
具体的，请参考图4，图4为本发明实施例所提供的一种图像的遮挡区域还原设备的具体结构示意图，该图像的遮挡区域还原设备210可因配置或性能不同而产生比较大的差异，可以包括一个或一个以上处理器(central processing units，CPU)222(例如，一个或一个以上处理器)和存储器232，一个或一个以上存储应用程序242或数据244的存储介质230(例如一个或一个以上海量存储设备)。其中，存储器232和存储介质230可以是短暂存储或持久存储。存储在存储介质230的程序可以包括一个或一个以上模块(图示没标出)，每个模块可以包括对数据处理设备中的一系列指令操作。更进一步地，中央处理器222可以设置为与存储介质230通信，在图像的遮挡区域还原设备210上执行存储介质230中的一系列指令操作。Specifically, please refer to FIG. 4. FIG. 4 is a schematic structural diagram of an image occlusion area restoration device provided by an embodiment of the present invention. The image occlusion area restoration device 210 may generate relatively large difference, may include one or more processors (central processing units, CPUs) 222 (for example, one or more processors) and memory 232, one or more storage media 230 for storing applications 242 or data 244 (for example, one or more than one mass storage device). Wherein, the memory 232 and the storage medium 230 may be temporary storage or persistent storage. The program stored in the storage medium 230 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the data processing device. Furthermore, the central processing unit 222 may be configured to communicate with the storage medium 230 , and execute a series of instruction operations in the storage medium 230 on the image occlusion area restoration device 210 .
图像的遮挡区域还原设备210还可以包括一个或一个以上电源226，一个或一个以上有线或无线网络接口250，一个或一个以上输入输出接口258，和/或，一个或一个以上操作系统241。例如，Windows系统。The image occlusion region restoration device 210 may also include one or more power sources 226 , one or more wired or wireless network interfaces 250 , one or more input/output interfaces 258 , and/or, one or more operating systems 241 . For example, Windows system.
上文所描述的图像的遮挡区域还原方法中的步骤可以由图像的遮挡区域还原设备的结构实现。The steps in the method for restoring an image occlusion area described above can be implemented by the structure of an image occlusion area restoration device.
相应于上面的方法实施例，本发明实施例还提供了一种计算机可读存储介质，下文描述的一种计算机可读存储介质与上文描述的一种图像的遮挡区域还原方法可相互对应参照。Corresponding to the above method embodiment, the embodiment of the present invention also provides a computer-readable storage medium. The computer-readable storage medium described below and the method for restoring an occlusion area of an image described above can be referred to in correspondence with each other. .
一种计算机可读存储介质，该计算机可读存储介质上存储有计算机程序，计算机程序被处理器执行时实现上述方法实施例所提供的图像的遮挡区域还原方法的步骤。A computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method for restoring an occluded area of an image provided by the above method embodiments are implemented.
该计算机可读存储介质具体可以为U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可存储程序代码的可读存储介质。The computer-readable storage medium can specifically be a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, etc., which can store various program codes. readable storage media.
说明书中各个实施例采用递进的方式描述，每个实施例重点说明的都是与其他实施例的不同之处，各个实施例之间相同相似部分互相参见即可。对于实施例公开的装置、设备及计算机可读存储介质而言，由于其与实施例公开的方法相对应，所以描述的比较简单，相关之处参见方法部分说明即可。Each embodiment in the description is described in a progressive manner, each embodiment focuses on the difference from other embodiments, and the same and similar parts of each embodiment can be referred to each other. As for the device, equipment and computer-readable storage medium disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and for relevant details, please refer to the description of the method part.
以上对本发明所提供的一种图像的遮挡区域还原方法、装置、设备及计算机可读存储介质进行了详细介绍。本文中应用了具体个例对本发明的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本发明的方法及其核心思想。应当指出，对于本技术领域的普通技术人员来说，在不脱离本发明原理的前提下，还可以对本发明进行若干改进和修饰，这些改进和修饰也落入本发明权利要求的保护范围内。The method, device, equipment, and computer-readable storage medium for restoring an occluded area of an image provided by the present invention are described above in detail. In this paper, specific examples are used to illustrate the principle and implementation of the present invention, and the descriptions of the above embodiments are only used to help understand the method and core idea of the present invention. It should be pointed out that for those skilled in the art, without departing from the principle of the present invention, some improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.
Claims (10)
- 一种图像的遮挡区域还原方法，其特征在于，包括：A method for restoring an occluded area of an image, comprising:根据获取的被遮挡图像和无遮挡图像中的匹配特征点，对所述被遮挡图像和所述无遮挡图像进行仿射变换，得到仿射变换矩阵；Performing affine transformation on the occluded image and the unoccluded image according to the matching feature points in the acquired occluded image and the unoccluded image to obtain an affine transformation matrix;根据所述被遮挡图像对应的遮挡区域选择信息，确定所述被遮挡图像中的遮挡区域的像素点坐标；Determine the pixel coordinates of the occlusion area in the occlusion image according to the occlusion area selection information corresponding to the occlusion image;根据所述遮挡区域的像素点坐标和所述仿射变换矩阵，确定所述无遮挡图像中的还原区域的像素点坐标，并利用所述还原区域的像素点对所述遮挡区域的像素点进行还原，得到所述被遮挡图像对应的还原图像。According to the pixel point coordinates of the occlusion area and the affine transformation matrix, determine the pixel point coordinates of the restoration area in the non-occlusion image, and use the pixels of the restoration area to carry out the pixel points of the occlusion area Restore to obtain a restored image corresponding to the occluded image.
- 根据权利要求1所述的图像的遮挡区域还原方法，其特征在于，所述根据获取的被遮挡图像和无遮挡图像中匹配特征点，对所述被遮挡图像和所述无遮挡图像进行仿射变换，得到仿射变换矩阵，包括：The method for restoring an occluded area of an image according to claim 1, wherein, according to matching feature points in the acquired occluded image and the unoccluded image, affine is performed on the occluded image and the unoccluded image Transform to get an affine transformation matrix, including:对所述被遮挡图像和所述无遮挡图像中的特征点进行匹配，获取所述匹配特征点；Matching the feature points in the occluded image and the unoccluded image to obtain the matching feature points;根据所述匹配特征点，对所述被遮挡图像和所述无遮挡图像进行仿射变换，得到仿射变换矩阵；其中，所述仿射变换矩阵包括所述匹配特征点中的正确匹配特征点和所述正确匹配特征点之外的未匹配特征点。Perform affine transformation on the occluded image and the unoccluded image according to the matching feature points to obtain an affine transformation matrix; wherein the affine transformation matrix includes correct matching feature points among the matching feature points and the unmatched feature points other than the correct matching feature points.
- 根据权利要求2所述的图像的遮挡区域还原方法，其特征在于，所述对所述被遮挡图像和所述无遮挡图像中的特征点进行匹配，获取所述匹配特征点，包括：The method for restoring an occluded area of an image according to claim 2, wherein the matching of feature points in the occluded image and the unoccluded image, and obtaining the matching feature points includes:提取所述被遮挡图像和所述无遮挡图像中的特征点；extracting feature points in the occluded image and the unoccluded image;根据所述特征点，生成被遮挡图像和所述无遮挡图像的特征向量；Generate feature vectors of the occluded image and the unoccluded image according to the feature points;根据所述特征向量，确定所述被遮挡图像和所述无遮挡图像中的所述匹配特征点。Determine the matching feature points in the occluded image and the unoccluded image according to the feature vector.
- 根据权利要求3所述的图像的遮挡区域还原方法，其特征在于，所述提取所述被遮挡图像和所述无遮挡图像中的特征点，包括：The method for restoring an occluded area of an image according to claim 3, wherein the extracting the feature points in the occluded image and the unoccluded image comprises:利用SURF特征提取算法，提取所述被遮挡图像和所述无遮挡图像中的特征点。Using a SURF feature extraction algorithm to extract feature points in the occluded image and the unoccluded image.
- 根据权利要求1所述的图像的遮挡区域还原方法，其特征在于，还包括：The occlusion region restoration method of an image according to claim 1, further comprising:获取第一图像和第二图像；acquire the first image and the second image;根据所述第一图像对应的遮挡区域选择信息，将所述第一图像确定为所述被遮挡图像，并将所述第二图像确定为所述无遮挡图像。Determining the first image as the occluded image and determining the second image as the unoccluded image according to the occlusion area selection information corresponding to the first image.
- 根据权利要求1至5任一项所述的图像的遮挡区域还原方法，其特征在于，所述根据所述被遮挡图像对应的遮挡区域选择信息，确定所述被遮挡图像中的遮挡区域的像素点坐标，包括：The image occlusion area restoration method according to any one of claims 1 to 5, characterized in that, according to the occlusion area selection information corresponding to the occlusion image, the pixels of the occlusion area in the occlusion image are determined Point coordinates, including:根据所述被遮挡图像对应的遮挡区域选择点坐标，确定所述遮挡区域的像素点坐标。Determine the pixel point coordinates of the occluded area according to the selected point coordinates of the occluded area corresponding to the occluded image.
- 根据权利要求6所述的图像的遮挡区域还原方法，其特征在于，所述根据所述被遮挡图像对应的遮挡区域选择点坐标，确定所述遮挡区域的像素点坐标，包括：The method for restoring an occlusion area of an image according to claim 6, wherein said selecting point coordinates according to the occlusion area corresponding to the occluded image to determine the pixel point coordinates of the occlusion area comprises:利用区域生长算法，将所述被遮挡图像中所述遮挡区域选择点坐标对应的区域确定为所述遮挡区域，并获取所述所述遮挡区域的像素点坐标。Using a region growing algorithm, determine the area corresponding to the selected point coordinates of the occluded area in the occluded image as the occluded area, and obtain the pixel point coordinates of the occluded area.
- 一种图像的遮挡区域还原装置，其特征在于，包括：An image occlusion area restoration device, characterized in that it comprises:特征仿射模块，用于根据获取的被遮挡图像和无遮挡图像中的匹配特征点，对所述被遮挡图像和所述无遮挡图像进行仿射变换，得到仿射变换矩阵；A feature affine module, configured to perform affine transformation on the occluded image and the unoccluded image according to the acquired matched feature points in the occluded image and the unoccluded image, to obtain an affine transformation matrix;遮挡确定模块，用于根据所述被遮挡图像对应的遮挡区域选择信息，确定所述被遮挡图像中的遮挡区域的像素点坐标；An occlusion determination module, configured to determine the pixel point coordinates of the occlusion area in the occlusion image according to the occlusion area selection information corresponding to the occlusion image;遮挡还原模块，用于根据所述遮挡区域的像素点坐标和所述仿射变换矩阵，确定所述无遮挡图像中的还原区域的像素点坐标，并利用所述还原区域的像素点对所述遮挡区域的像素点进行还原，得到所述被遮挡图像对应的还原图像。An occlusion restoration module, configured to determine the pixel coordinates of the restoration area in the non-occlusion image according to the pixel coordinates of the occlusion area and the affine transformation matrix, and use the pixels of the restoration area to map the The pixels in the occluded area are restored to obtain a restored image corresponding to the occluded image.
- 一种图像的遮挡区域还原设备，其特征在于，包括：An image occlusion region restoration device, characterized in that it comprises:存储器，用于存储计算机程序；memory for storing computer programs;处理器，用于执行所述计算机程序时实现如权利要求1至7任一项所述的图像的遮挡区域还原方法的步骤。A processor configured to implement the steps of the method for restoring an occluded area of an image according to any one of claims 1 to 7 when executing the computer program.
- 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质上存储有计算机程序，所述计算机程序被处理器执行时实现如权利要求1至7任一项所述的图像的遮挡区域还原方法的步骤。A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the occlusion of the image according to any one of claims 1 to 7 is realized Steps of the region restore method.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111006783.0 | 2021-08-30 | ||
| CN202111006783.0A CN115731096A (en) | 2021-08-30 | 2021-08-30 | Method, device and equipment for restoring image occlusion area and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2023029122A1 true WO2023029122A1 (en) | 2023-03-09 |
Family
ID=85291048
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2021/119729 WO2023029122A1 (en) | 2021-08-30 | 2021-09-23 | Method and apparatus for restoring image occlusion area, device, and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN115731096A (en) |
| WO (1) | WO2023029122A1 (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130128121A1 (en) * | 2010-09-14 | 2013-05-23 | Aseem O. Agarwala | Methods and Apparatus for Video Completion |
| CN106056534A (en) * | 2016-05-31 | 2016-10-26 | 中国科学院深圳先进技术研究院 | Obstruction perspective method and device based on smart glasses |
| CN110866863A (en) * | 2018-08-27 | 2020-03-06 | 天津理工大学 | Automobile A-pillar perspective algorithm |
-
2021
- 2021-08-30 CN CN202111006783.0A patent/CN115731096A/en active Pending
- 2021-09-23 WO PCT/CN2021/119729 patent/WO2023029122A1/en unknown
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130128121A1 (en) * | 2010-09-14 | 2013-05-23 | Aseem O. Agarwala | Methods and Apparatus for Video Completion |
| CN106056534A (en) * | 2016-05-31 | 2016-10-26 | 中国科学院深圳先进技术研究院 | Obstruction perspective method and device based on smart glasses |
| CN110866863A (en) * | 2018-08-27 | 2020-03-06 | 天津理工大学 | Automobile A-pillar perspective algorithm |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115731096A (en) | 2023-03-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6789402B2 (en) | Method of determining the appearance of an object in an image, equipment, equipment and storage medium | |
| CN111368685B (en) | Method and device for identifying key points, readable medium and electronic equipment | |
| WO2019042419A1 (en) | Image tracking point acquisition method and device, and storage medium | |
| CN111640181A (en) | Interactive video projection method, device, equipment and storage medium | |
| CN110517214B (en) | Method and apparatus for generating image | |
| EP2863362B1 (en) | Method and apparatus for scene segmentation from focal stack images | |
| CN110349161B (en) | Image segmentation method, image segmentation device, electronic equipment and storage medium | |
| WO2022166258A1 (en) | Behavior recognition method and apparatus, terminal device, and computer-readable storage medium | |
| KR20060115123A (en) | Apparatus and method for extracting moving image | |
| CN110598139A (en) | Web browser augmented reality real-time positioning method based on 5G cloud computing | |
| CN114187333A (en) | Image alignment method, image alignment device and terminal equipment | |
| CN108322658B (en) | Photographing method and device | |
| CN108229281B (en) | Neural network generation method, face detection device and electronic equipment | |
| CN103955713B (en) | A kind of icon-based programming method and apparatus | |
| WO2023029122A1 (en) | Method and apparatus for restoring image occlusion area, device, and storage medium | |
| CN111612063A (en) | Image matching method, device and equipment and computer readable storage medium | |
| CN111783677A (en) | Face recognition method, face recognition device, server and computer readable medium | |
| CN110880160A (en) | Picture frame super-division method and device, terminal equipment and computer readable storage medium | |
| CN108270973B (en) | Photographing processing method, mobile terminal and computer readable storage medium | |
| CN112487943B (en) | Key frame de-duplication method and device and electronic equipment | |
| WO2021164329A1 (en) | Image processing method and apparatus, and communication device and readable storage medium | |
| US9361540B2 (en) | Fast image processing for recognition objectives system | |
| CN110942065B (en) | Text box selection method, text box selection device, terminal equipment and computer readable storage medium | |
| Sari et al. | An Approach For Stitching Satellite Images In A Bigdata Mapreduce Framework | |
| CN115861361A (en) | Matting method, matting device, electronic equipment, storage medium and computer program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21955652 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |