WO2020024744A1 - 一种图像特征点检测方法、终端设备及存储介质 - Google Patents

一种图像特征点检测方法、终端设备及存储介质 Download PDF

Info

Publication number
WO2020024744A1
WO2020024744A1 PCT/CN2019/093685 CN2019093685W WO2020024744A1 WO 2020024744 A1 WO2020024744 A1 WO 2020024744A1 CN 2019093685 W CN2019093685 W CN 2019093685W WO 2020024744 A1 WO2020024744 A1 WO 2020024744A1
Authority
WO
WIPO (PCT)
Prior art keywords
initial
image
feature points
current category
feature point
Prior art date
Application number
PCT/CN2019/093685
Other languages
English (en)
French (fr)
Inventor
张弓
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Publication of WO2020024744A1 publication Critical patent/WO2020024744A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks

Definitions

  • the present application relates to the field of image recognition technology, and in particular, to an image feature point detection method, a terminal device, and a computer-readable storage medium.
  • the present application provides an image feature point detection method, a terminal device, and a computer-readable storage medium, which are used to solve the problem of low detection accuracy of a feature point detection method of a current scene.
  • a first aspect of the embodiments of the present application provides an image feature point detection method, including:
  • a second aspect of the embodiments of the present application provides a terminal device, including:
  • An initial image acquisition module configured to acquire initial images of multiple categories of natural scenes, where the natural scenes of each category include multiple initial images
  • the initial feature point acquisition module is used to extract the initial feature points from the initial image of the current category for each category of natural scenes;
  • a correspondence relationship acquisition module configured to obtain a correspondence relationship between the initial feature points in each initial image of a current category
  • a target feature point acquisition module configured to obtain an initial feature point that meets a preset condition from an initial feature point of an initial image of the current category as a target feature point of the current category based on the corresponding relationship;
  • a training image acquisition module configured to use an initial image of a target feature point of the current category in the initial image of each category as a training image to obtain training sample sets of natural scenes in multiple categories;
  • a training module configured to train a constructed deep neural network through training images in the training sample set to obtain a trained deep neural network
  • a detection module is configured to detect an image to be detected based on the trained deep neural network, and obtain a feature point in the image to be detected.
  • a third aspect of the embodiments of the present application provides a terminal device including a memory, a processor, and a computer program stored in the memory and executable on the processor.
  • the processor executes the computer program, Implement the steps of the method provided by the first aspect of the embodiments of the present application.
  • a fourth aspect of the embodiments of the present application provides a computer-readable storage medium.
  • the computer-readable storage medium stores a computer program, and the computer program is implemented by one or more processors to implement the first embodiment of the present application. Steps of the method provided by aspects.
  • a computer program product includes a computer program that, when executed by one or more processors, implements a payment application management method mentioned in the first aspect of the present application. .
  • FIG. 1 is a schematic flowchart of an image feature point detection method provided by an embodiment of the present application
  • FIG. 2 is a schematic flowchart of another method for detecting image feature points according to an embodiment of the present application
  • FIG. 3 is a schematic block diagram of a terminal device according to an embodiment of the present application.
  • FIG. 4 is a schematic block diagram of another terminal device according to an embodiment of the present application.
  • the category of the scene can be preset to be earth, stream, cloud, and after rain. , Snow mountain, etc.
  • the scene that detects an image is the category of the natural scene given in the image.
  • the detection of scene categories requires detection based on feature points.
  • face detection facial features with obvious features can be used as feature points.
  • Obtaining feature points based on specific features in face detection can achieve face detection. In scene detection, it is difficult to manually calibrate feature points with obvious special features for training images.
  • a certain response value of the image pixels is calculated one by one in the image scale space, and the local extreme value is obtained in the three-dimensional space composed of the pixel position and the scale to obtain the feature point detection result.
  • the detection of such feature points is not accurate enough and may not be representative of the features of the scene.
  • the embodiment of the present application first obtains target feature points that can represent the characteristics of the scene, and then calibrates the target feature points in the image.
  • the deep neural network is trained by the image with the target feature points calibrated, and the deep neural network is detected by the training The scene feature points of the image to obtain the scene of the image.
  • FIG. 1 is a schematic flowchart of an image feature point detection method provided by an embodiment of the present application. As shown in the figure, the method may include the following steps:
  • Step S101 Obtain initial images of a plurality of categories of natural scenes, where the natural scenes of each category include a plurality of initial images.
  • the acquired training image needs to include images corresponding to the multiple types of natural scenes.
  • a deep neural network can also be trained on images of a single category of natural scenes. In this way, the deep neural network obtained through training can only detect feature points of images of a single category of natural scenes when performing scene detection.
  • initial images of natural scenes in multiple categories can be obtained. For example, if five natural scenes are set, a large number of natural scenes need to be collected. Initial image.
  • step S102 for each category of natural scene, an initial feature point is extracted from the initial image of the current category.
  • a method for extracting an initial feature point from an initial image includes, but is not limited to, Harris, SUSAN, SIFT, SURF, FAST, MSER, and the like. Taking Harris corners as an example, the image is first divided into M ⁇ M small blocks, Harris corner response calculation is performed for each small block, and N points with the largest corner response value in each small block are extracted as feature points. The graph extracts at most M ⁇ M ⁇ N feature points. It can be understood that, in practical applications, other methods for extracting image feature points may also be used.
  • Step S103 Obtain a corresponding relationship between the initial feature points in each initial image of the current category.
  • the initial feature points of the initial image in the scene may be the same or different.
  • initial feature point a1, initial feature point a2, initial feature point a3, and initial feature point a4 are extracted from initial image A
  • initial feature point b1, initial feature point b2, and initial feature point b3 are extracted from initial image B.
  • the initial feature points a1 and the initial feature points b2 may belong to the same type of feature points. That is, the initial feature point a1 in the initial image A and the initial feature point b2 in the initial image B are in a corresponding relationship. In this way, the initial feature points a1 and b2 can be labeled as the same type of initial feature points.
  • the corresponding relationship between the initial feature points in different initial images is determined according to the feature information of the initial feature points.
  • Step S104 Based on the corresponding relationship, obtain an initial feature point that meets a preset condition from an initial feature point of an initial image of the current category as a target feature point of the current category.
  • initial feature point a1 left eye
  • initial feature point a2 right eye
  • initial feature point a3 tip of the nose
  • Initial feature point a4 face point
  • initial feature point b1 nose tip
  • initial feature point b2 left eye
  • initial feature point b3 right eye
  • initial feature points are not: initial feature points a1, initial feature points a2, initial feature points a3, initial feature points a4, initial feature points b1, initial feature points b2, initial feature points b3; initial features of the current natural scene
  • the types of points should be: left eye, right eye, nose tip, and face points. This is because the initial feature point a1 and the initial feature point b2 are in a corresponding relationship, both representing the left eye, and the initial feature point a2 and the initial feature point b3 are in a corresponding relationship, both are representing the right eye, the initial feature point a3 and the initial feature point b1.
  • the initial feature point a4 represents a point on the face.
  • the feature points in the natural scene image do not have obvious features like the face image, so if the corresponding relationship is uncertain, the problem of representing the same scene feature point with different initial feature points will occur.
  • an initial feature point that meets a preset condition may be selected from the current initial feature points as a target feature point. For example, an initial feature point having a higher frequency in different initial images is selected as the target feature point, and an initial feature point that matches a preset feature among the initial feature points may also be used as the target feature point.
  • the process of obtaining the target feature point from the initial feature point is actually to obtain the initial feature point that can represent the feature of the current natural scene as the target feature point.
  • an initial feature point that is different from an initial feature point in other natural scenes than a threshold value from the initial feature points of the current scene may also be selected as the target feature point of the current scene.
  • other preset conditions can also be set in practice to obtain the target feature points.
  • Step S105 Use the initial image containing the target feature points of the current category as the training image in the initial image of each category to obtain training sample sets of natural scenes in multiple categories.
  • the obtained target feature point is a feature point capable of representing the current natural scene, so the target feature point can be marked in the initial image containing the target feature point, and the initial feature point is marked with the target feature point.
  • the image is used as a training image.
  • the natural scenes of each category need to go through the process of selecting target feature points from the initial feature points, then the training images corresponding to each natural scene can be obtained, and the training sample set of multiple categories of natural scenes can be obtained.
  • step S106 the constructed deep neural network is trained through the training images in the training sample set to obtain a trained deep neural network.
  • the deep neural network may be a VGG neural network model.
  • the process of training a deep neural network by calibrating the target feature point training image may be: inputting the training image into the deep neural network to obtain an output image, constructing a loss function based on the difference between the detected feature points in the output image and the target feature points, and based on the The loss function updates the parameters of each layer in the deep neural network inversely. Until the deep neural network detects that the feature points tend to be calibrated to the target feature points, that is, the deep neural network converges, the trained deep neural network can be obtained. Of course, in practical applications, other training methods can also be used.
  • the method before training the constructed deep neural network through the training images in the training sample set, and before obtaining the trained deep neural network, the method further includes:
  • the natural scene and target feature points of the training image are calibrated.
  • the target feature points can be calibrated for the training image, but also the natural scene corresponding to the training image can be calibrated.
  • a classifier can be added at the end for detecting The feature points classify the natural scene of the image.
  • the natural scene of the image to be detected can be obtained when detecting the feature points of the image by using a deep neural network with a classifier added.
  • Step S107 Detect an image to be detected based on the trained deep neural network to obtain feature points in the image to be detected.
  • the trained deep neural network has the ability to detect feature points that approach the target feature points infinitely. Therefore, after the image to be detected is input into the trained deep neural network, the Feature points in the detection image that can characterize the scene of the image to be detected.
  • detection and recognition of image scenes are mostly based on calculating a certain response value of image pixels one by one in the image scale space, and obtaining local extreme values based on the pixel position and scale to obtain feature point detection. result.
  • this method of image feature point detection has lower detection accuracy.
  • the solution of the present application first obtains initial images of natural scenes in multiple categories, and for each category of natural scenes, extracts initial feature points, and then obtains the corresponding relationship between the initial feature points in different initial images.
  • a target feature point capable of characterizing the current natural scene is selected from the initial image, and the initial image including the target feature point is used as a training image to train a constructed deep neural network model and a trained deep neural network model. It has the ability to detect the feature points of the image scene.
  • the training image for training the deep neural network model in the embodiment of the present application is an image where the target feature points of the natural scene of each category can be selected by filtering from the initial feature points. Therefore, the detection accuracy of feature points in scene detection can be improved.
  • FIG. 2 is a schematic flowchart of another image feature point detection method provided by an embodiment of the present application.
  • the embodiment of the present application is a process for describing how to obtain a target feature point based on the embodiment shown in FIG.
  • Step S201 Acquire initial images of a plurality of categories of natural scenes, where the natural scenes of each category include a plurality of initial images.
  • step S202 for each category of natural scene, an initial feature point is extracted from the initial image of the current category.
  • steps S201 to S202 is consistent with the content of steps S101 to S102.
  • steps S101 to S102 are not described herein again.
  • Step S203 Obtain a three-dimensional model of the natural scene in the current category.
  • the three-dimensional model of the natural scene may be established in advance, or may be established according to an initial image of the current natural scene.
  • the acquiring a three-dimensional model of a natural scene in a current category includes:
  • a three-dimensional model of the natural scene of the current category is established based on the initial image of the current category.
  • the three-dimensional model of the natural scene of the current category is established based on the initial image, and the three-dimensional model of the natural scene of the current category may be established according to an image sequence composed of multiple initial images.
  • the initial images are sorted according to the similarity between any two initial images, so that the initial image has the highest similarity with the two adjacent images.
  • the SIFT features of each initial image can be obtained, and the SIFT features of each initial image are matched to obtain the first one.
  • the 3D reconstruction of the second initial image and then based on the SIFT feature matching between the second initial image and the third initial image, the 3D reconstruction of the first and second initial images is modified and expanded to obtain the first Three-dimensional reconstruction between an initial image, a second initial image, and a third initial image.
  • the SIFT feature matching between the third initial image and the fourth initial image the first initial image, the second The three-dimensional reconstruction between the initial image and the third initial image is modified and expanded to obtain the three-dimensional reconstruction between the first to the fourth initial graphics, ..., and so on, to obtain all the initial images in the current natural scene. 3D reconstruction results.
  • step S204 based on a projection matrix of the initial image of the current category in the three-dimensional model, a corresponding relationship of the initial feature points in each initial image of the current category is obtained.
  • a natural scene is taken as an example.
  • the initial image of the current natural scene can be mapped into a three-dimensional model to obtain a projection matrix of each initial image. It can also be understood as imaging the three-dimensional model from a perspective.
  • An initial image can be obtained.
  • After obtaining the projection matrix of each initial image in the three-dimensional model since the initial feature points are located in the initial image, according to the projection matrix of the initial image in the three-dimensional model, all the The corresponding relationship between the initial feature points in each initial image of the current category is described.
  • the process of obtaining the corresponding relationship between the initial feature points in each initial image of the current category may also be It is a process of matching the initial feature points, and the matching can be performed according to information such as features and positions of the initial feature points.
  • obtaining a corresponding relationship between the initial feature points in each initial image of the current category based on a projection matrix of the initial image of the current category in the three-dimensional model includes:
  • matching may be performed based on the positions of the initial feature points.
  • the positions of the initial feature points in the initial image and the projection matrix of the initial image in the three-dimensional model may be used to obtain the initial feature points in the three-dimensional model.
  • the position in the model is based on the position of each initial feature point in the three-dimensional model, and the corresponding relationship between the initial feature points in each initial image of the current category is obtained.
  • Step S205 Based on the corresponding relationship, obtain the frequency of occurrence of each initial feature point in the initial image of the current category.
  • step S206 the initial feature points that meet the preset conditions are used as the target feature points of the current category.
  • the frequency of the initial feature points appearing in the initial image of the current category can be used as a condition for filtering the target feature points. It can also be understood that the initial feature points a1 appear in N initial images, and the initial The frequency of the feature points a1 is recorded as the number N of initial images in which the initial feature points a1 appear.
  • the using the initial feature points that meet the preset conditions as the target feature points of the current category includes:
  • the initial feature points of the current category are sorted according to the frequency, and a preset number of initial feature points are sequentially selected from the high frequency to the low frequency as the target feature points of the current category.
  • an initial feature point with the same initial feature point appearing in different initial images more than a preset number of times may be used as the target feature point, or a preset number may be set, and the pre-selection is performed from high frequency to low frequency.
  • Let the number of initial feature points be the target feature points of the current category.
  • FIG. 3 is a schematic block diagram of a terminal device according to an embodiment of the present application. For ease of description, only parts related to the embodiment of the present application are shown.
  • the terminal device 3 may be a software unit, a hardware unit, or a combination of software and hardware, which is built into a terminal device such as a mobile phone, a tablet computer, and a notebook, or may be integrated as an independent pendant into the terminal device such as a mobile phone, tablet, notebook in.
  • the terminal device 3 includes:
  • An initial image acquisition module 31 is configured to acquire initial images of a plurality of categories of natural scenes, where the natural scenes of each category include a plurality of initial images;
  • the initial feature point acquisition module 32 is configured to extract an initial feature point from an initial image of the current category for each category of natural scenes;
  • a correspondence relationship acquisition module 33 configured to obtain a correspondence relationship between the initial feature points in each initial image of a current category
  • a target feature point acquisition module 34 configured to obtain an initial feature point that meets a preset condition from an initial feature point of an initial image of the current category as a target feature point of the current category based on the corresponding relationship;
  • the training image acquisition module 35 is configured to use the initial image of the target feature points of the current category in the initial image of each category as a training image to obtain training sample sets of natural scenes in multiple categories;
  • a training module 36 configured to train a constructed deep neural network by using the training images in the training sample set to obtain a trained deep neural network
  • a detection module 37 is configured to detect an image to be detected based on the trained deep neural network, and obtain a feature point in the image to be detected.
  • the corresponding relationship acquisition module 33 includes:
  • a correspondence relationship acquiring unit 332 is configured to obtain a correspondence relationship between the initial feature points in each initial image of the current category based on a projection matrix of the initial image of the current category in the three-dimensional model.
  • the three-dimensional model obtaining unit 331 is further configured to:
  • a three-dimensional model of the natural scene of the current category is established based on the initial image of the current category.
  • the correspondence acquiring unit 332 includes:
  • An initial feature point position acquisition subunit configured to obtain a position of each initial feature point in the three-dimensional model based on a projection matrix of an initial image of a current category in the three-dimensional model;
  • a correspondence relationship acquisition subunit is configured to obtain a correspondence relationship between the initial feature points in each initial image of a current category based on a position of each initial feature point in the three-dimensional model.
  • the target feature point acquisition module 34 includes:
  • An initial feature point frequency obtaining unit 341 is configured to obtain, based on the corresponding relationship, the frequency at which each initial feature point appears in the initial image of the current category;
  • the target feature point acquisition unit 342 is configured to use the initial feature points that meet the preset conditions as the target feature points of the current category.
  • the target feature point obtaining unit 342 is further configured to:
  • the initial feature points of the current category are sorted according to the frequency, and a preset number of initial feature points are sequentially selected from the high frequency to the low frequency as the target feature points of the current category.
  • the terminal device 3 further includes:
  • a calibration module is configured to calibrate the natural scene and target feature points of the training image for each training image before training the constructed deep neural network through the training images in the training sample set and obtaining the trained deep neural network.
  • FIG. 4 is a schematic block diagram of a terminal device according to another embodiment of the present application.
  • the terminal device 4 of this embodiment includes one or more processors 40, a memory 41, and a computer program 42 stored in the memory 41 and executable on the processor 40.
  • the processor 40 executes the computer program 42, the steps in the embodiments of the image feature point detection method described above are implemented, for example, steps S101 to S107 shown in FIG.
  • the processor 40 executes the computer program 42
  • the functions of the modules / units in the foregoing embodiment of the terminal device are implemented, for example, the functions of modules 31 to 37 shown in FIG. 3.
  • the computer program 42 may be divided into one or more modules / units, and the one or more modules / units are stored in the memory 41 and executed by the processor 40 to complete This application.
  • the one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 42 in the terminal device 4.
  • the computer program 42 may be divided into an initial image acquisition module, an initial feature point acquisition module, a correspondence relationship acquisition module, a target feature point acquisition module, a training image acquisition module, a training module, and a detection module.
  • An initial image acquisition module configured to acquire initial images of multiple categories of natural scenes, where the natural scenes of each category include multiple initial images
  • the initial feature point acquisition module is used to extract the initial feature points from the initial image of the current category for each category of natural scenes;
  • a correspondence relationship acquisition module configured to obtain a correspondence relationship between the initial feature points in each initial image of a current category
  • a target feature point acquisition module configured to obtain an initial feature point that meets a preset condition from an initial feature point of an initial image of the current category as a target feature point of the current category based on the corresponding relationship;
  • a training image acquisition module configured to use an initial image of a target feature point of the current category in the initial image of each category as a training image to obtain training sample sets of natural scenes in multiple categories;
  • a training module configured to train a constructed deep neural network through training images in the training sample set to obtain a trained deep neural network
  • a detection module is configured to detect an image to be detected based on the trained deep neural network, and obtain a feature point in the image to be detected.
  • the terminal device includes, but is not limited to, a processor 40 and a memory 41.
  • FIG. 4 is only an example of the terminal device 4, and does not constitute a limitation on the terminal device 4. It may include more or fewer components than shown in the figure, or combine some components, or different Components, for example, the terminal device may further include an input device, an output device, a network access device, a bus, and the like.
  • the processor 40 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), Ready-made programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • a general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • the memory 41 may be an internal storage unit of the terminal device 4, such as a hard disk or a memory of the terminal device 4.
  • the memory 41 may also be an external storage device of the terminal device 4, such as a plug-in hard disk, a Smart Media Card (SMC), and a Secure Digital (SD) provided on the terminal device 4. Card, flash card, etc.
  • the memory 41 may include both an internal storage unit of the terminal device 4 and an external storage device.
  • the memory 41 is configured to store the computer program and other programs and data required by the terminal device.
  • the memory 41 may also be used to temporarily store data that has been output or is to be output.
  • the disclosed terminal device and method may be implemented in other manners.
  • the terminal device embodiments described above are only schematic.
  • the division of the modules or units is only a logical function division.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, which may be electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objective of the solution of this embodiment.
  • the functional units in the embodiments of the present application may be integrated into one processing unit, or each of the units may exist separately physically, or two or more units may be integrated into one unit.
  • the above integrated unit may be implemented in the form of hardware or in the form of software functional unit.
  • the integrated module / unit When the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, this application implements all or part of the processes in the method of the above embodiment, and can also be completed by a computer program instructing related hardware.
  • the computer program can be stored in a computer-readable storage medium.
  • the computer When the program is executed by a processor, the steps of the foregoing method embodiments can be implemented.
  • the computer program includes computer program code, and the computer program code may be in a source code form, an object code form, an executable file, or some intermediate form.
  • the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, Random Access Memory), electric carrier signals, telecommunication signals, and software distribution media.
  • ROM Read-Only Memory
  • RAM Random Access Memory
  • electric carrier signals telecommunication signals
  • software distribution media any entity or device capable of carrying the computer program code
  • a recording medium a U disk, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, Random Access Memory), electric carrier signals, telecommunication signals, and software distribution media.

Abstract

本申请适用于图像识别技术领域，提供了一种图像特征点检测方法终端设备及计算机可读存储介质，所述方法包括：获取多个类别的自然场景的初始图像，针对每个类别的自然场景，从初始图像中提起初始特征点，根据初始特征点在当前自然场景的初始图像中的对应关系，将符合预设条件的初始特征点作为当前类别的目标特征点，将包含目标特征点的初始图像作为当前类别的训练图像，通过所述训练样本集中的训练图像，训练构建的深度神经网络，获得训练后的深度神经网络，基于所述训练后的深度神经网络，对待检测图像进行检测，获得所述待检测图像中的特征点，通过本申请可以提高场景检测中的检测精度。

Description

一种图像特征点检测方法、终端设备及存储介质 技术领域
本申请涉及图像识别技术领域，具体涉及一种图像特征点检测方法、终端设备及计算机可读存储介质。
背景技术
随着计算机视觉的不断发展，以及用户需求的不断提高，出现了很多图像处理技术。在对图像进行各种处理时，为了获得较好的处理效果，有时需要识别图像的场景。
发明内容
有鉴于此，本申请提供一种图像特征点检测方法、终端设备及计算机可读存储介质，用于解决目前场景的特征点检测方式检测精度较低的问题。
本申请实施例的第一方面提供了一种图像特征点检测方法，包括：
获取多个类别的自然场景的初始图像，其中，每个类别的自然场景包括多个初始图像；
对于每个类别自然场景，从当前类别的初始图像中分别提取初始特征点；
获取所述初始特征点在当前类别的每个初始图像中的对应关系；
基于所述对应关系，从当前类别的初始图像的初始特征点中获取符合预设条件的初始特征点作为当前类别的目标特征点；
将每个类别的初始图像中包含当前类别的目标特征点的初始图像作为训练图像，获得多个类别的自然场景的训练样本集；
通过所述训练样本集中的训练图像，训练构建的深度神经网络，获得训练后的深度神经网络；
基于所述训练后的深度神经网络，对待检测图像进行检测，获得所述待检测图像中的特征点。
本申请实施例的第二方面提供了一种终端设备，包括：
初始图像获取模块，用于获取多个类别的自然场景的初始图像，其中，每个类别的自然场景包括多个初始图像；
初始特征点获取模块，用于对于每个类别自然场景，从当前类别的初始图像中分别提取初始特征点；
对应关系获取模块，用于获取所述初始特征点在当前类别的每个初始图像中的对应关系；
目标特征点获取模块，用于基于所述对应关系，从当前类别的初始图像的初始特征点中获取符合预设条件的初始特征点作为当前类别的目标特征点；
训练图像获取模块，用于将每个类别的初始图像中包含当前类别的目标特征点的初始图像作为训练图像，获得多个类别的自然场景的训练样本集；
训练模块，用于通过所述训练样本集中的训练图像，训练构建的深度神经网络，获得训练后的深度神经网络；
检测模块，用于基于所述训练后的深度神经网络，对待检测图像进行检测，获得所述待检测图像中的特征点。
本申请实施例的第三方面提供了一种终端设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现本申请实施例第一方面提供的所述方法的步骤。
本申请实施例的第四方面提供了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，所述计算机程序被一个或多个处理器执行时实现本申请实施例第一方面提供的所述方法的步骤。
本申请实第五方面提供一种计算机程序产品，所述计算机程序产品包括计算机程序，所述计算机程序被一个或多个处理器执行时实现本申请第一方面提及的支付类应用程序管理方法。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1是本申请实施例提供的一种图像特征点检测方法的实现流程示意图；
图2是本申请实施例提供的另一种图像特征点检测方法的实现流程示意图；
图3是本申请实施例提供的一种终端设备的示意框图；
图4是本申请实施例提供的另一种终端设备的示意框图。
具体实施方式
为使得本申请的发明目的、特征、优点能够更加的明显和易懂，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而非全部实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
为了说明本申请所述的技术方案，首先介绍本申请实施例的应用场景，本申请可应用于对图像的场景检测，例如，可以预设设定场景的类别为大地、溪水、云朵、雨后、雪山 等，当然，在实际应用中，也可以是其它对自然场景的分类方式，在此不做限制，检测图像的场景就是检测出图像中给的自然场景的类别。对场景类别的检测需要基于特征点的检测，然而，场景检测中不像人脸检测可以将具有明显特征的五官作为特征点，人脸检测中基于特定的五官获取特征点可以实现人脸检测，场景检测中对于训练图像很难手动标定出具有明显特殊特征的特征点。所以，通常是在图像尺度空间逐个计算图像像素的某种响应值，并在像素位置与尺度联合组成的三维空间求取局部极值以得到特征点检测结果。这种特征点的检测不够精确，并且有可能并不能代表场景的特征。本申请实施例是先获取到能够代表场景的特征的目标特征点，然后对图像中的目标特征点进行标定，通过标定了目标特征点的图像训练深度神经网络，通过训练后的深度神经网络检测图像的场景特征点，从而获得图像的场景。下面通过具体实施例来进行说明。
图1是本申请实施例提供的一种图像特征点检测方法的实现流程示意图，如图所示该方法可以包括以下步骤：
步骤S101，获取多个类别的自然场景的初始图像，其中，每个类别的自然场景包括多个初始图像。
在本申请实施例中，为了使得训练的深度神经网络能够识别多种类别的自然场景，所以，获取的训练图像需要包含多个类别的自然场景对应的图像。实际应用中，也可以通过单一类别的自然场景的图像训练深度神经网络，这样，训练获得的深度神经网络在进行场景检测时就只能够检测出单一类别的自然场景的图像的特征点。
若需要训练后的深度神经网络能够对多个类别的场景进行特征点检测，可以获取多个类别的自然场景的初始图像，例如，设置了5个自然场景，需要收集每个自然场景对应的大量的初始图像。
步骤S102，对于每个类别自然场景，从当前类别的初始图像中分别提取初始特征点。
在本申请实施例中，从初始图像中提取初始特征点的方法包括但不限于Harris、SUSAN、SIFT、SURF、FAST、MSER等。以Harris角点为例，先将图像分成M×M个小块，对每个小块进行Harris角点响应计算，提取每个小块内角点响应值最大的N个点作为特征点，一张图最多提取M×M×N个特征点。可以理解，实际应用中，还可以是其它提取图像特征点的方法。
步骤S103，获取所述初始特征点在当前类别的每个初始图像中的对应关系。
在本申请实施例中，对于某个类别的自然场景，可能该场景中初始图像的初始特征点之间存在相同，也存在不同。例如，初始图像A中提取了初始特征点a1、初始特征点a2、初始特征点a3、初始特征点a4，初始图像B中提取了初始特征点b1、初始特征点b2、初始特征点b3。由于是从不同的初始图像中提取的特征点，因此，初始特征点a1和初始特 征点b2可能属于同一类型的特征点。即初始图像A中的初始特征点a1与初始图像B中的初始特征点b2是对应关系。这样就可以将初始特征点a1和初始特征点b2标记为同一类型的初始特征点。具体地，根据初始特征点的特征信息判断初始特征点在不同初始图像中的对应关系。
步骤S104，基于所述对应关系，从当前类别的初始图像的初始特征点中获取符合预设条件的初始特征点作为当前类别的目标特征点。
在本申请实施例中，在确定了初始特征点的对应关系之后，就可以获得从当前自然场景的初始图像中总共获得了多少种类型的初始特征点。
为了更通俗的理解，我们以人脸检测为例，假设从当前自然场景的初始图像A中提取了初始特征点a1(左眼)、初始特征点a2(右眼)、初始特征点a3(鼻尖)、初始特征点a4(脸部的点)，初始图像B中提取了初始特征点b1(鼻尖)、初始特征点b2(左眼)、初始特征点b3(右眼)，那么，当前自然场景的初始特征点的种类并不是：初始特征点a1、初始特征点a2、初始特征点a3、初始特征点a4、初始特征点b1、初始特征点b2、初始特征点b3；当前自然场景的初始特征点的种类应该是：左眼、右眼、鼻尖、脸部的点。这是因为，初始特征点a1与初始特征点b2为对应的关系，均表示左眼，初始特征点a2和初始特征点b3为对应关系，均表示右眼，初始特征点a3和初始特征点b1为对应关系，均表示鼻尖，初始特征点a4表示脸部的点。自然场景图像中的特征点不像人脸图像具有明显的特征，所以如果不确定对应关系，就会出现用不同的初始特征点表示同一场景特征点的问题。
在确定了当前自然场景的初始图像中总共获得了多少种类型的初始特征点之后，可以从当前初始特征点中选取符合预设条件的初始特征点作为目标特征点。例如，选取在不同初始图像中出现的频次较高的初始特征点作为目标特征点，还可以将初始特征点中符合预设特征的初始特征点作为目标特征点。从初始特征点中获取目标特征点的过程实际上是获取能够代表当前自然场景的特征的初始特征点作为目标特征点。例如，还可以是从当前场景的初始特征点中选取与其它自然场景中的初始特征点的差异大于阈值的初始特征点作为当前场景的目标特征点。当然，实际应用中还可以设置其它的预设条件以获得目标特征点。
步骤S105，将每个类别的初始图像中包含当前类别的目标特征点的初始图像作为训练图像，获得多个类别的自然场景的训练样本集。
在本申请实施例中，获得的目标特征点为能够代表当前自然场景的特征点，所以可以在包含了目标特征点的初始图像中将目标特征点标记出来，并将标记了目标特征点的初始图像作为训练图像。每个类别的自然场景均需要经过从初始特征点中选取目标特征点的过程，那么就可以获得每个自然场景对应的训练图像，这样就可以获得多个类别的自然场景的训练样本集。
步骤S106，通过所述训练样本集中的训练图像，训练构建的深度神经网络，获得训练后的深度神经网络。
在本申请实施例中，所述深度神经网络可以是VGG神经网络模型。通过标定了目标特征点训练图像训练深度神经网络的过程可以是：将训练图像输入深度神经网络获得输出图像，根据输出图像中检测到的特征点与目标特征点的差异构建损失函数，基于所述损失函数，反向更新深度神经网络中各层的参数，直到通过深度神经网络检测到特征点趋向于标定的目标特征点，即深度神经网络收敛，就可以获得训练后的深度神经网络。当然，实际应用中，还可以是其它训练方式。
作为本申请又一实施例，在通过所述训练样本集中的训练图像，训练构建的深度神经网络，获得训练后的深度神经网络之前，还包括：
为每个训练图像标定所述训练图像的自然场景和目标特征点。
在本申请实施例中，不仅可以为训练图像标定目标特征点，还可以标定出训练图像对应的自然场景，这样在设置深度神经网络的时候可以在最后增加一个分类器，用于根据检测到的特征点对图像的自然场景进行分类，这样，通过增加了分类器的深度神经网络检测图像特征点的时候就可以相应的获得待检测图像的自然场景。
步骤S107，基于所述训练后的深度神经网络，对待检测图像进行检测，获得所述待检测图像中的特征点。
在本申请实施例中，训练后的深度神经网络，具有了能够检测到无限逼近目标特征点的特征点的能力，因此，在将待检测图像输入训练后的深度神经网络之后，就可以获得待检测图像中能够表征待检测图像的场景的特征点。
另外，还需要说明的是，现有技术中，对图像场景的检测识别大多是在图像尺度空间逐个计算图像像素的某种响应值，并基于像素位置和尺度获得局部极值以得到特征点检测结果。然而，这种图像特征点检测的方式检测精度较低。
因此，本申请方案为解决上述技术问题，首先获取多个类别的自然场景的初始图像，对于每个类别的自然场景，提取初始特征点，然后将获取初始特征点在不同初始图像中的对应关系，根据所述对应关系，从初始图像中筛选出能够表征当前自然场景的目标特征点，将包括目标特征点的初始图像作为训练图像，训练构建的深度神经网络模型，训练后的深度神经网络模型就具有了检测图像场景特征点的能力，由于本申请实施例中训练深度神经网络模型的训练图像是通过从初始特征点中筛选出的能够表征每个类别的自然场景的目标特征点所在的图像，因此，能够提高场景检测中特征点的检测精度。
图2是本申请实施例提供的另一种图像特征点检测方法的流程示意图，本申请实施例 是在图1所示实施例的基础上描述如何获取目标特征点的过程，可以包括以下步骤：
步骤S201，获取多个类别的自然场景的初始图像，其中，每个类别的自然场景包括多个初始图像。
步骤S202，对于每个类别自然场景，从当前类别的初始图像中分别提取初始特征点。
步骤S201至步骤S202的内容和步骤S101至步骤S102的内容一致，具体可参照步骤S101至步骤S102的描述，在此不再赘述。
步骤S203，获取当前类别的自然场景的三维模型。
在本申请实施例中，所述自然场景的三维模型可以是预先建立的，也可以是根据当前自然场景的初始图像建立的。
作为本申请又一实施例，所述获取当前类别的自然场景的三维模型包括：
基于图像重建算法，根据当前类别的初始图像建立当前类别的自然场景的三维模型。
在本申请实施例中，基于所述初始图像建立当前类别的自然场景的三维模型，可以是根据多个初始图像组成的图像序列建立当前类别的自然场景的三维模型。首先根据任意两个初始图像之间的相似度，对所述初始图像进行排序，使得初始图像与前后相邻的两个图像的相似度最高。然后，从图像序列的头部开始，对于相邻的第一个和第二个初始图像，可以获取每个初始图像的SIFT特征，对每个初始图像的SIFT特征进行匹配，从而得到第一个和第二个初始图像的三维重建，然后根据第二个初始图像和第三个初始图像之间的SIFT特征匹配，对第一个和第二个初始图像的三维重建进行修正和扩充，得到第一个初始图像、第二个初始图像和第三个初始图像之间的三维重建，根据第三个初始图像和第四个初始图像之间的SIFT特征匹配，对第一个初始图像、第二个初始图像和第三个初始图像之间的三维重建进行修正和扩充，得到第一个至第四个初始图形之间的三维重建，……，依次类推，获得当前自然场景下的所有初始图像的三维重建结果。
需要说明的是，上述对多个初始图像进行三维重建获得三维模型的过程仅用于举例，实际应用中，还可以是其它三维重建方法。
步骤S204，基于当前类别的初始图像在所述三维模型中的投影矩阵，获得所述初始特征点在当前类别的每个初始图像中的对应关系。
在本申请实施例中，以一个自然场景为例，当前自然场景的初始图像可以映射到三维模型中，获得每个初始图像的投影矩阵，也可以理解为以一个视角对所述三维模型进行成像可以获得一个初始图像。在获得了每个初始图像在所述三维模型中的投影矩阵之后，由于所述初始特征点位于所述初始图像中，所以，根据初始图像在所述三维模型中的投影矩阵，就可以获得所述初始特征点在当前类别的每个初始图像中的对应关系，如图1所示实施例中的描述，获得所述初始特征点在当前类别的每个初始图像中的对应关系的过程也可 以是对所述初始特征点进行匹配的过程，可以根据初始特征点的特征、位置等信息进行匹配。
作为本申请又一实施例，所述基于当前类别的初始图像在所述三维模型中的投影矩阵，获得所述初始特征点在当前类别的每个初始图像中的对应关系包括：
基于当前类别的初始图像在所述三维模型中的投影矩阵，获得每个初始特征点在所述三维模型中的位置；
基于每个初始特征点在所述三维模型中的位置，获得所述初始特征点在当前类别的每个初始图像中的对应关系。
在本申请实施例中，可以基于初始特征点的位置进行匹配，例如，根据初始特征点在初始图像中的位置，以及初始图像在三维模型中的投影矩阵，可获得初始特征点在所述三维模型中的位置，基于每个初始特征点在所述三维模型中的位置，获得所述初始特征点在当前类别的每个初始图像中的对应关系。
步骤S205，基于所述对应关系，获取每个初始特征点在当前类别的初始图像中出现的频次。
步骤S206，将所述频次符合预设条件的初始特征点作为当前类别的目标特征点。
在本申请实施例中，可以根据初始特征点在当前类别的初始图像中出现的频次作为筛选目标特征点的条件，也可以理解为，有N个初始图像中出现了初始特征点a1，则初始特征点a1的频次就记录为出现初始特征点a1的初始图像的个数N。
作为本申请又一实施例，所述将所述频次符合预设条件的初始特征点作为当前类别的目标特征点包括：
将当前类别的初始特征点中，所述频次大于预设频次的初始特征点作为当前类别的目标特征点；
或，按照所述频次将当前类别的初始特征点进行排序，从高频次到低频次依次选取预设数量的初始特征点作为当前类别的目标特征点。
在本申请实施例中，可以将同一初始特征点在不同初始图像中出现的次数大于预设次数的初始特征点作为目标特征点，也可以预先设置数量，从高频次到低频次依次选取预设数量的初始特征点作为当前类别的目标特征点。
应理解，上述实施例中各步骤的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。
图3是本申请一实施例提供的终端设备的示意框图，为了便于说明，仅示出与本申请 实施例相关的部分。
该终端设备3可以是内置于手机、平板电脑、笔记本等终端设备内的软件单元、硬件单元或者软硬结合的单元，也可以作为独立的挂件集成到所述手机、平板电脑、笔记本等终端设备中。
所述终端设备3包括：
初始图像获取模块31，用于获取多个类别的自然场景的初始图像，其中，每个类别的自然场景包括多个初始图像；
初始特征点获取模块32，用于对于每个类别自然场景，从当前类别的初始图像中分别提取初始特征点；
对应关系获取模块33，用于获取所述初始特征点在当前类别的每个初始图像中的对应关系；
目标特征点获取模块34，用于基于所述对应关系，从当前类别的初始图像的初始特征点中获取符合预设条件的初始特征点作为当前类别的目标特征点；
训练图像获取模块35，用于将每个类别的初始图像中包含当前类别的目标特征点的初始图像作为训练图像，获得多个类别的自然场景的训练样本集；
训练模块36，用于通过所述训练样本集中的训练图像，训练构建的深度神经网络，获得训练后的深度神经网络；
检测模块37，用于基于所述训练后的深度神经网络，对待检测图像进行检测，获得所述待检测图像中的特征点。
可选的，所述对应关系获取模块33包括：
三维模型获取单元331，用于获取当前类别的自然场景的三维模型；
对应关系获取单元332，用于基于当前类别的初始图像在所述三维模型中的投影矩阵，获得所述初始特征点在当前类别的每个初始图像中的对应关系。
可选的，所述三维模型获取单元331还用于：
基于图像重建算法，根据当前类别的初始图像建立当前类别的自然场景的三维模型。
可选的，所述对应关系获取单元332包括：
初始特征点位置获取子单元，用于基于当前类别的初始图像在所述三维模型中的投影矩阵，获得每个初始特征点在所述三维模型中的位置；
对应关系获取子单元，用于基于每个初始特征点在所述三维模型中的位置，获得所述初始特征点在当前类别的每个初始图像中的对应关系。
可选的，所述目标特征点获取模块34包括：
初始特征点频次获取单元341，用于基于所述对应关系，获取每个初始特征点在当前类 别的初始图像中出现的频次；
目标特征点获取单元342，用于将所述频次符合预设条件的初始特征点作为当前类别的目标特征点。
可选的，所述目标特征点获取单元342还用于：
将当前类别的初始特征点中，所述频次大于预设频次的初始特征点作为当前类别的目标特征点；
或，按照所述频次将当前类别的初始特征点进行排序，从高频次到低频次依次选取预设数量的初始特征点作为当前类别的目标特征点。
可选的，所述终端设备3还包括：
标定模块，用于在通过所述训练样本集中的训练图像，训练构建的深度神经网络，获得训练后的深度神经网络之前，为每个训练图像标定所述训练图像的自然场景和目标特征点。
所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，仅以上述各功能单元、模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能单元、模块完成，即将所述终端设备的内部结构划分成不同的功能单元或模块，以完成以上描述的全部或者部分功能。实施例中的各功能单元、模块可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中，上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。另外，各功能单元、模块的具体名称也只是为了便于相互区分，并不用于限制本申请的保护范围。上述终端设备中单元、模块的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
图4是本申请又一实施例提供的终端设备的示意框图。如图4所示，该实施例的终端设备4包括：一个或多个处理器40、存储器41以及存储在所述存储器41中并可在所述处理器40上运行的计算机程序42。所述处理器40执行所述计算机程序42时实现上述各个图像特征点检测方法实施例中的步骤，例如图1所示的步骤S101至S107。或者，所述处理器40执行所述计算机程序42时实现上述终端设备实施例中各模块/单元的功能，例如图3所示模块31至37的功能。
示例性的，所述计算机程序42可以被分割成一个或多个模块/单元，所述一个或者多个模块/单元被存储在所述存储器41中，并由所述处理器40执行，以完成本申请。所述一个或多个模块/单元可以是能够完成特定功能的一系列计算机程序指令段，该指令段用于描 述所述计算机程序42在所述终端设备4中的执行过程。例如，所述计算机程序42可以被分割成初始图像获取模块、初始特征点获取模块、对应关系获取模块、目标特征点获取模块、训练图像获取模块、训练模块、检测模块。
初始图像获取模块，用于获取多个类别的自然场景的初始图像，其中，每个类别的自然场景包括多个初始图像；
初始特征点获取模块，用于对于每个类别自然场景，从当前类别的初始图像中分别提取初始特征点；
对应关系获取模块，用于获取所述初始特征点在当前类别的每个初始图像中的对应关系；
目标特征点获取模块，用于基于所述对应关系，从当前类别的初始图像的初始特征点中获取符合预设条件的初始特征点作为当前类别的目标特征点；
训练图像获取模块，用于将每个类别的初始图像中包含当前类别的目标特征点的初始图像作为训练图像，获得多个类别的自然场景的训练样本集；
训练模块，用于通过所述训练样本集中的训练图像，训练构建的深度神经网络，获得训练后的深度神经网络；
检测模块，用于基于所述训练后的深度神经网络，对待检测图像进行检测，获得所述待检测图像中的特征点。
其它模块或者单元可参照图3所示的实施例中的描述，在此不再赘述。
所述终端设备包括但不仅限于处理器40、存储器41。本领域技术人员可以理解，图4仅仅是终端设备4的一个示例，并不构成对终端设备4的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件，例如所述终端设备还可以包括输入设备、输出设备、网络接入设备、总线等。
所述处理器40可以是中央处理单元(Central Processing Unit，CPU)，还可以是其他通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现成可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
所述存储器41可以是所述终端设备4的内部存储单元，例如终端设备4的硬盘或内存。所述存储器41也可以是所述终端设备4的外部存储设备，例如所述终端设备4上配备的插接式硬盘，智能存储卡(Smart Media Card,SMC)，安全数字(Secure Digital,SD)卡， 闪存卡(Flash Card)等。进一步地，所述存储器41还可以既包括所述终端设备4的内部存储单元也包括外部存储设备。所述存储器41用于存储所述计算机程序以及所述终端设备所需的其他程序和数据。所述存储器41还可以用于暂时地存储已经输出或者将要输出的数据。
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述或记载的部分，可以参见其它实施例的相关描述。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
在本申请所提供的实施例中，应该理解到，所揭露的终端设备和方法，可以通过其它的方式实现。例如，以上所描述的终端设备实施例仅仅是示意性的，例如，所述模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通讯连接可以是通过一些接口，装置或单元的间接耦合或通讯连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的模块/单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请实现上述实施例方法中的全部或部分流程，也可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一计算机可读存储介质中，该计算机程序在被处理器执行时，可实现上述各个方法实施例的步骤。其中，所述计算机程序包括计算机程序代码，所述计算机程序代码可以为源代码形式、对象代码形式、可执行文件或某些中间形式等。所述计算机可读介质可以包括：能够携带所述计算机程序代码的任何实体或装置、记录介质、U盘、移动硬盘、磁碟、光盘、计算机存储器、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、电载波信号、电信信号以及软件分发介质等。需要说 明的是，所述计算机可读介质包含的内容可以根据司法管辖区内立法和专利实践的要求进行适当的增减，例如在某些司法管辖区，根据立法和专利实践，计算机可读介质不包括是电载波信号和电信信号。
以上所述实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围，均应包含在本申请的保护范围之内。

Claims (20)

  1. 一种图像特征点检测方法，其特征在于，包括：
    获取多个类别的自然场景的初始图像，其中，每个类别的自然场景包括多个初始图像；
    对于每个类别自然场景，从当前类别的初始图像中分别提取初始特征点；
    获取所述初始特征点在当前类别的每个初始图像中的对应关系；
    基于所述对应关系，从当前类别的初始图像的初始特征点中获取符合预设条件的初始特征点作为当前类别的目标特征点；
    将每个类别的初始图像中包含当前类别的目标特征点的初始图像作为训练图像，获得多个类别的自然场景的训练样本集；
    通过所述训练样本集中的训练图像，训练构建的深度神经网络，获得训练后的深度神经网络；
    基于所述训练后的深度神经网络，对待检测图像进行检测，获得所述待检测图像中的特征点。
  2. 如权利要求1所述的图像特征点检测方法，其特征在于，所述获取所述初始特征点在当前类别的每个初始图像中的对应关系包括：
    获取当前类别的自然场景的三维模型；
    基于当前类别的初始图像在所述三维模型中的投影矩阵，获得所述初始特征点在当前类别的每个初始图像中的对应关系。
  3. 如权利要求2所述的图像特征点检测方法，其特征在于，所述获取当前类别的自然场景的三维模型包括：
    基于图像重建算法，根据当前类别的初始图像建立当前类别的自然场景的三维模型。
  4. 如权利要求2所述的图像特征点检测方法，其特征在于，所述基于当前类别的初始图像在所述三维模型中的投影矩阵，获得所述初始特征点在当前类别的每个初始图像中的对应关系包括：
    基于当前类别的初始图像在所述三维模型中的投影矩阵，获得每个初始特征点在所述三维模型中的位置；
    基于每个初始特征点在所述三维模型中的位置，获得所述初始特征点在当前类别的每个初始图像中的对应关系。
  5. 如权利要求1所述的图像特征点检测方法，其特征在于，所述基于所述对应关系，从当前类别的初始图像的初始特征点中获取符合预设条件的初始特征点作为当前类别的目标特征点包括：
    基于所述对应关系，获取每个初始特征点在当前类别的初始图像中出现的频次；
    将所述频次符合预设条件的初始特征点作为当前类别的目标特征点。
  6. 如权利要求5所述的图像特征点检测方法，其特征在于，所述将所述频次符合预设条件的初始特征点作为当前类别的目标特征点包括：
    将当前类别的初始特征点中，所述频次大于预设频次的初始特征点作为当前类别的目标特征点；
    或，按照所述频次将当前类别的初始特征点进行排序，从高频次到低频次依次选取预设数量的初始特征点作为当前类别的目标特征点。
  7. 如权利要求1所述的图像特征点检测方法，其特征在于，所述基于所述对应关系，从当前类别的初始图像的初始特征点中获取符合预设条件的初始特征点作为当前类别的目标特征点包括：
    基于所述对应关系，从当前类别的初始图像的初始特征点中获取符合预设特征的初始特征点作为当前类别的目标特征点。
  8. 如权利要求1所述的图像特征点检测方法，其特征在于，所述基于所述对应关系，从当前类别的初始图像的初始特征点中获取符合预设条件的初始特征点作为当前类别的目标特征点包括：
    基于所述对应关系，从当前类别的初始图像的初始特征点中获取与其它自然场景中的初始特征点的差异大于阈值的初始特征点作为当前类别的目标特征点。
  9. 如权利要求1所述的图像特征点检测方法，其特征在于，在通过所述训练样本集中的训练图像，训练构建的深度神经网络，获得训练后的深度神经网络之前，还包括：
    为每个训练图像标定所述训练图像的自然场景和目标特征点。
  10. 一种终端设备，其特征在于，包括：
    初始图像获取模块，用于获取多个类别的自然场景的初始图像，其中，每个类别的自然场景包括多个初始图像；
    初始特征点获取模块，用于对于每个类别自然场景，从当前类别的初始图像中分别提取初始特征点；
    对应关系获取模块，用于获取所述初始特征点在当前类别的每个初始图像中的对应关系；
    目标特征点获取模块，用于基于所述对应关系，从当前类别的初始图像的初始特征点中获取符合预设条件的初始特征点作为当前类别的目标特征点；
    训练图像获取模块，用于将每个类别的初始图像中包含当前类别的目标特征点的初始图像作为训练图像，获得多个类别的自然场景的训练样本集；
    训练模块，用于通过所述训练样本集中的训练图像，训练构建的深度神经网络，获得训练后的深度神经网络；
    检测模块，用于基于所述训练后的深度神经网络，对待检测图像进行检测，获得所述待检测图像中的特征点。
  11. 如权利要求10所述的终端设备，其特征在于，所述对应关系获取模块包括：
    三维模型获取单元，用于获取当前类别的自然场景的三维模型；
    对应关系获取单元，用于基于当前类别的初始图像在所述三维模型中的投影矩阵，获得所述初始特征点在当前类别的每个初始图像中的对应关系。
  12. 如权利要求11所述的终端设备，其特征在于，所述三维模型获取单元具体用于：
    基于图像重建算法，根据当前类别的初始图像建立当前类别的自然场景的三维模型。
  13. 如权利要求11所述的终端设备，其特征在于，所述对应关系获取单元包括：
    初始特征点位置获取子单元，用于基于当前类别的初始图像在所述三维模型中的投影矩阵，获得每个初始特征点在所述三维模型中的位置；
    对应关系获取子单元，用于基于每个初始特征点在所述三维模型中的位置，获得所述初始特征点在当前类别的每个初始图像中的对应关系。
  14. 如权利要求10所述的终端设备，其特征在于，所述目标特征点获取模块包括：
    初始特征点频次获取单元，用于基于所述对应关系，获取每个初始特征点在当前类别的初始图像中出现的频次；
    目标特征点获取单元，用于将所述频次符合预设条件的初始特征点作为当前类别的目标特征点。
  15. 如权利要求14所述的终端设备，其特征在于，所述目标特征点获取单元还用于：
    将当前类别的初始特征点中，所述频次大于预设频次的初始特征点作为当前类别的目标特征点；
    或，按照所述频次将当前类别的初始特征点进行排序，从高频次到低频次依次选取预设数量的初始特征点作为当前类别的目标特征点。
  16. 如权利要求10所述的终端设备，其特征在于，所述目标特征点获取模块具体用于：
    基于所述对应关系，从当前类别的初始图像的初始特征点中获取符合预设特征的初始特征点作为当前类别的目标特征点。
  17. 如权利要求10所述的终端设备，其特征在于，所述目标特征点获取模块具体用于：
    基于所述对应关系，从当前类别的初始图像的初始特征点中获取与其它自然场景中的初始特征点的差异大于阈值的初始特征点作为当前类别的目标特征点。
  18. 如权利要求10所述的终端设备，其特征在于，所述终端设备还包括：
    标定模块，用于在通过所述训练样本集中的训练图像，训练构建的深度神经网络，获得训练后的深度神经网络之前，为每个训练图像标定所述训练图像的自然场景和目标特征点。
  19. 一种终端设备，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时实现如权利要求1至9任一项所述方法的步骤。
  20. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有计算机程序，所述计算机程序被一个或多个处理器执行时实现如权利要求1至9任一项所述方法的步骤。
PCT/CN2019/093685 2018-08-01 2019-06-28 一种图像特征点检测方法、终端设备及存储介质 WO2020024744A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810865350.2 2018-08-01
CN201810865350.2A CN109117773B (zh) 2018-08-01 2018-08-01 一种图像特征点检测方法、终端设备及存储介质

Publications (1)

Publication Number Publication Date
WO2020024744A1 true WO2020024744A1 (zh) 2020-02-06

Family

ID=64863925

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/093685 WO2020024744A1 (zh) 2018-08-01 2019-06-28 一种图像特征点检测方法、终端设备及存储介质

Country Status (2)

Country Link
CN (1) CN109117773B (zh)
WO (1) WO2020024744A1 (zh)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112907726A (zh) * 2021-01-25 2021-06-04 重庆金山医疗器械有限公司 一种图像处理方法、装置、设备及计算机可读存储介质
CN113361363A (zh) * 2021-05-31 2021-09-07 北京百度网讯科技有限公司 人脸图像识别模型的训练方法、装置、设备及存储介质
CN113470110A (zh) * 2020-03-30 2021-10-01 北京四维图新科技股份有限公司 测距方法和装置
CN114898354A (zh) * 2022-03-24 2022-08-12 中德（珠海）人工智能研究院有限公司 基于三维模型的测量方法、装置、服务器及可读存储介质
CN115953567A (zh) * 2023-03-14 2023-04-11 广州市玄武无线科技股份有限公司 一种堆箱数量的检测方法、装置、终端设备及存储介质

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109117773B (zh) * 2018-08-01 2021-11-02 Oppo广东移动通信有限公司 一种图像特征点检测方法、终端设备及存储介质
CN110322399B (zh) * 2019-07-05 2023-05-05 深圳开立生物医疗科技股份有限公司 一种超声图像调整方法、系统、设备及计算机存储介质
CN110942063B (zh) * 2019-11-21 2023-04-07 望海康信(北京)科技股份公司 证件文字信息获取方法、装置以及电子设备
CN113807451B (zh) * 2021-05-25 2022-11-01 中德（珠海）人工智能研究院有限公司 全景图像特征点匹配模型的训练方法、装置以及服务器

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100183228A1 (en) * 2009-01-20 2010-07-22 Seiko Epson Corporation Specifying position of characteristic portion of face image
CN103310445A (zh) * 2013-06-01 2013-09-18 吉林大学 用于虚拟视点绘制的虚拟视点摄像机参数估计方法
CN103617432A (zh) * 2013-11-12 2014-03-05 华为技术有限公司 一种场景识别方法及装置
CN105184271A (zh) * 2015-09-18 2015-12-23 苏州派瑞雷尔智能科技有限公司 一种基于深度学习的车辆自动检测方法
CN105488541A (zh) * 2015-12-17 2016-04-13 上海电机学院 增强现实系统中基于机器学习的自然特征点识别方法
CN106127748A (zh) * 2012-07-18 2016-11-16 成都理想境界科技有限公司 一种图像特征样本数据库及其建立方法
CN109117773A (zh) * 2018-08-01 2019-01-01 Oppo广东移动通信有限公司 一种图像特征点检测方法、终端设备及存储介质

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101859326B (zh) * 2010-06-09 2012-04-18 南京大学 一种图像检索方法
CN104008400A (zh) * 2014-06-16 2014-08-27 河南科技大学 结合sift和bp网络进行物体识别的方法
KR20170024303A (ko) * 2015-08-25 2017-03-07 영남대학교 산학협력단 얼굴의 특징점 검출 시스템 및 방법
CN106446930B (zh) * 2016-06-28 2019-11-22 沈阳工业大学 基于深层卷积神经网络的机器人工作场景识别方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100183228A1 (en) * 2009-01-20 2010-07-22 Seiko Epson Corporation Specifying position of characteristic portion of face image
CN106127748A (zh) * 2012-07-18 2016-11-16 成都理想境界科技有限公司 一种图像特征样本数据库及其建立方法
CN103310445A (zh) * 2013-06-01 2013-09-18 吉林大学 用于虚拟视点绘制的虚拟视点摄像机参数估计方法
CN103617432A (zh) * 2013-11-12 2014-03-05 华为技术有限公司 一种场景识别方法及装置
CN105184271A (zh) * 2015-09-18 2015-12-23 苏州派瑞雷尔智能科技有限公司 一种基于深度学习的车辆自动检测方法
CN105488541A (zh) * 2015-12-17 2016-04-13 上海电机学院 增强现实系统中基于机器学习的自然特征点识别方法
CN109117773A (zh) * 2018-08-01 2019-01-01 Oppo广东移动通信有限公司 一种图像特征点检测方法、终端设备及存储介质

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113470110A (zh) * 2020-03-30 2021-10-01 北京四维图新科技股份有限公司 测距方法和装置
CN112907726A (zh) * 2021-01-25 2021-06-04 重庆金山医疗器械有限公司 一种图像处理方法、装置、设备及计算机可读存储介质
CN112907726B (zh) * 2021-01-25 2022-09-20 重庆金山医疗技术研究院有限公司 一种图像处理方法、装置、设备及计算机可读存储介质
CN113361363A (zh) * 2021-05-31 2021-09-07 北京百度网讯科技有限公司 人脸图像识别模型的训练方法、装置、设备及存储介质
CN113361363B (zh) * 2021-05-31 2024-02-06 北京百度网讯科技有限公司 人脸图像识别模型的训练方法、装置、设备及存储介质
CN114898354A (zh) * 2022-03-24 2022-08-12 中德（珠海）人工智能研究院有限公司 基于三维模型的测量方法、装置、服务器及可读存储介质
CN115953567A (zh) * 2023-03-14 2023-04-11 广州市玄武无线科技股份有限公司 一种堆箱数量的检测方法、装置、终端设备及存储介质

Also Published As

Publication number Publication date
CN109117773A (zh) 2019-01-01
CN109117773B (zh) 2021-11-02

Similar Documents

Publication Publication Date Title
WO2020024744A1 (zh) 一种图像特征点检测方法、终端设备及存储介质
CN108765278B (zh) 一种图像处理方法、移动终端及计算机可读存储介质
WO2020125216A1 (zh) 一种行人重识别方法、装置、电子设备及计算机可读存储介质
CN109325954B (zh) 图像分割方法、装置及电子设备
WO2021004180A1 (zh) 一种纹理特征提取方法、纹理特征提取装置及终端设备
CN110276411B (zh) 图像分类方法、装置、设备、存储介质和医疗电子设备
CN110363091B (zh) 侧脸情况下的人脸识别方法、装置、设备及存储介质
CN112381775B (zh) 一种图像篡改检测方法、终端设备及存储介质
CN109002820B (zh) 一种车牌识别方法、装置及相关设备
CN110781911B (zh) 一种图像匹配方法、装置、设备及存储介质
US20210133980A1 (en) Image processing apparatus, training apparatus, image processing method, training method, and storage medium
CN108961267B (zh) 图片处理方法、图片处理装置及终端设备
CN108921181B (zh) 一种局部图像特征提取方法、装置、系统及可读存储介质
CN108986125B (zh) 物体边缘提取方法、装置及电子设备
WO2020143165A1 (zh) 一种翻拍图像的识别方法、系统及终端设备
CN107909569B (zh) 一种花屏检测方法、花屏检测装置及电子设备
WO2023273411A1 (zh) 一种多光谱数据的获取方法、装置及设备
CN113706472A (zh) 公路路面病害检测方法、装置、设备及存储介质
CN111598176A (zh) 一种图像匹配处理方法及装置
CN113642639B (zh) 活体检测方法、装置、设备和存储介质
CN113158773B (zh) 一种活体检测模型的训练方法及训练装置
JP6785181B2 (ja) 物体認識装置、物体認識システム、及び物体認識方法
CN109919164B (zh) 用户界面对象的识别方法及装置
CN112287905A (zh) 车辆损伤识别方法、装置、设备及存储介质
CN111783677A (zh) 人脸识别方法、装置、服务器和计算机可读介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19845309

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19845309

Country of ref document: EP

Kind code of ref document: A1