CN110489587A - The tire trace image characteristic extracting method of three value mode of Local gradient direction - Google Patents

The tire trace image characteristic extracting method of three value mode of Local gradient direction Download PDF

Info

Publication number
CN110489587A
CN110489587A CN201910699380.5A CN201910699380A CN110489587A CN 110489587 A CN110489587 A CN 110489587A CN 201910699380 A CN201910699380 A CN 201910699380A CN 110489587 A CN110489587 A CN 110489587A
Authority
CN
China
Prior art keywords
image
gradient direction
lgdtp
tire
feature
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910699380.5A
Other languages
Chinese (zh)
Other versions
CN110489587B (en
Inventor
刘颖
董海涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xian University of Posts and Telecommunications
Original Assignee
Xian University of Posts and Telecommunications
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xian University of Posts and Telecommunications filed Critical Xian University of Posts and Telecommunications
Priority to CN201910699380.5A priority Critical patent/CN110489587B/en
Publication of CN110489587A publication Critical patent/CN110489587A/en
Application granted granted Critical
Publication of CN110489587B publication Critical patent/CN110489587B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/5862Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using texture
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/40Image enhancement or restoration using histogram techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/13Edge detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30108Industrial image inspection

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Library & Information Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

一种局部梯度方向三值模式的轮胎痕迹图像特征提取方法,由图像预处理、特征提取、确定特征向量、图像检索步骤组成。本发明提出一种适用于轮胎痕迹图像的局部梯度方向三值模式特征,采用更加稳定的梯度方向值代替灰度值进行局部纹理编码,对中心像素梯度方向角进行阈值量化,产生高质量的纹理边缘信息,提高了检索准确度;将特征向量描述后的特征用曼哈顿距离进行相似度计算,得到检索结果,检索准确率明显优于其他纹理特征。具有轮胎痕迹图像纹理边缘信息清晰、检索准确度高、平均查准率高、适用于大样本数据等优点,可用于轮胎痕迹图像特征提取。

A tire trace image feature extraction method in local gradient direction ternary mode is composed of image preprocessing, feature extraction, determination of feature vector, and image retrieval steps. The invention proposes a local gradient direction ternary pattern feature suitable for tire track images, uses a more stable gradient direction value instead of gray value to perform local texture encoding, and performs threshold quantization on the gradient direction angle of the central pixel to generate high-quality texture. The edge information improves the retrieval accuracy; the Manhattan distance is used to calculate the similarity of the features described by the feature vector, and the retrieval results are obtained, and the retrieval accuracy is significantly better than other texture features. It has the advantages of clear texture edge information of tire track images, high retrieval accuracy, high average precision, and is suitable for large sample data, and can be used for feature extraction of tire track images.

Description

局部梯度方向三值模式的轮胎痕迹图像特征提取方法Tire Track Image Feature Extraction Method Based on Ternary Mode of Local Gradient Direction

技术领域technical field

本发明属于图像处理技术领域,具体涉及到图像特征提取方法。The invention belongs to the technical field of image processing, and specifically relates to an image feature extraction method.

背景技术Background technique

通常交通事故中会利用现场留下的车胎压痕痕迹和物证之间的关系解释事故的过程,判断双方责任,而地面轮胎痕迹往往是最有用的痕迹物证之一,所以轮胎痕迹检索常用于公安破案或交通事故处理中的线索获取。Usually in a traffic accident, the relationship between the tire indentation marks left on the scene and the physical evidence will be used to explain the process of the accident and judge the responsibility of both parties. The tire traces on the ground are often one of the most useful traces of physical evidence, so tire trace retrieval is often used in public security. Clue acquisition in case solving or traffic accident handling.

我国对轮胎痕迹图像检索相关的研究起步较晚,在轮胎痕迹方面的研究更加鲜有成果,并且在轮胎痕迹图像检索与分类研究领域没有标准的轮胎痕迹图像测试数据库。目前主要通过传统特征结合的方法对轮胎痕迹图像进行分类检索,并没有针对轮胎痕迹图像的特点进行系统分析。例如:通过分析SIFT变换和Gabor小波原理,提出了基于SIFT-Gabor变换的轮胎痕迹图像模式识别方法。结合无监督学习和层次化提取特征,提出了一种基于稀疏表示和概率潜在语义分析的轮胎痕迹图像检索方法。一种基于非下采样Contourlet变换(Non-Subsampled Contourlet Transform,NSCT)和灰度共生矩阵方法(Gray Level Co-occurrence matrix,GLCM)的组合特征提取方法,并利用多级支持向量机(Support VectorMachine,SVM)训练分类器。本方法通过分析得出轮胎与地面形成的轮胎痕迹图像包括主线沟槽和缘纹理线基本特征。同时,边缘纹理线很复杂,不同类型轮胎的边缘纹理线具有不同的宽度和方向,并且纹理线互连和缠绕,痕迹颜色相对单一。The research related to tire mark image retrieval in my country started late, and the research on tire marks is even less fruitful, and there is no standard tire mark image test database in the field of tire mark image retrieval and classification research. At present, the classification and retrieval of tire track images is mainly carried out by the method of combining traditional features, and there is no systematic analysis of the characteristics of tire track images. For example, by analyzing the principle of SIFT transform and Gabor wavelet, a pattern recognition method of tire tracks based on SIFT-Gabor transform is proposed. Combining unsupervised learning and hierarchical feature extraction, a tire track image retrieval method based on sparse representation and probabilistic latent semantic analysis is proposed. A combined feature extraction method based on Non-Subsampled Contourlet Transform (NSCT) and Gray Level Co-occurrence matrix (GLCM), and uses a multi-level support vector machine (Support Vector Machine, SVM) to train the classifier. In this method, the tire track image formed by the tire and the ground includes the basic features of the main line groove and the edge grain line through analysis. At the same time, the edge grain lines are very complex. The edge grain lines of different types of tires have different widths and directions, and the grain lines are interconnected and intertwined, and the trace color is relatively single.

在各种纹理图像描述符中,局部二值模式(Local Binary Patterns,LBP)是一种流行且功能强大的图像描述符。其计算复杂度低,无需训练学习,易于工程实现所以在计算机视觉和图像处理领域收到了广泛的关注。基于原始的LBP方法,学者们又提出了许多改进LBP方法。例如:局部方向模式(Local Directional Patterns,LDP)。增强的局部方向模式(Enhanced Local Directional Patterns,ELDP。局部方向数(Local DirectionalNumber,LDN)等。Among various texture image descriptors, Local Binary Patterns (LBP) is a popular and powerful image descriptor. It has low computational complexity, no training and learning, and easy engineering implementation, so it has received extensive attention in the fields of computer vision and image processing. Based on the original LBP method, scholars have proposed many improved LBP methods. For example: Local Directional Patterns (LDP). Enhanced Local Directional Patterns (Enhanced Local Directional Patterns, ELDP. Local Directional Number (LDN), etc.

在图像处理技术领域,当前需迫切解决的一个技术问题是提供一种轮胎痕迹图像特征提取方法。In the field of image processing technology, a technical problem that needs to be urgently solved at present is to provide a method for extracting the image features of tire tracks.

发明内容SUMMARY OF THE INVENTION

本发明所要解决的技术问题在于克服上述现有技术的缺点,提供一种轮胎痕迹图像纹理边缘信息清晰、检索准确度高、平均查准率高、适用于大样本数据的局部梯度方向三值模式的轮胎痕迹图像特征提取方法。The technical problem to be solved by the present invention is to overcome the above-mentioned shortcomings of the prior art, and to provide a local gradient direction ternary model with clear tire trace image texture edge information, high retrieval accuracy, high average precision, and suitable for large sample data. A method for feature extraction of tire track images.

解决上述技术所采用的技术方案是由以下步骤组成:The technical solution adopted to solve the above-mentioned technology is composed of the following steps:

(1)图像预处理(1) Image preprocessing

从轮胎痕迹图像数据库中选取轮胎痕迹样本图像30类每类50~80张进行大小归一化、灰度化处理。From the tire track image database, 50 to 80 tire track sample images of 30 categories are selected for size normalization and grayscale processing.

(2)特征提取(2) Feature extraction

1)用Sobel边缘检测方法确定图像沿x方向的图像差分Gx和y方向的图像差分Gy,用公式(1)确定图像中每个像素的梯度方向角α(x,y)1) Use the Sobel edge detection method to determine the image difference G x along the x direction and the image difference G y in the y direction, and use the formula (1) to determine the gradient direction angle α(x, y) of each pixel in the image

α(x,y)=arctan(Gy/Gx) (1)α(x,y)=arctan(G y /G x ) (1)

2)在3×3的邻域滑窗中用局部梯度方向三值模式LGDTP方法确定每一个梯度方向角值,局部梯度方向三值模式LGDTP方法增加自定义阈值t,gi大于区间[gc-t,gc+t]}时为1,属于此区间为0,小于此区间为-1,得到三值编码值。2) In the 3×3 neighborhood sliding window, use the local gradient direction ternary mode LGDTP method to determine each gradient direction angle value, and the local gradient direction ternary mode LGDTP method increases the custom threshold t, g i is greater than the interval [g c -t,g c +t]} is 1, belongs to this interval is 0, is smaller than this interval is -1, and obtains a three-valued encoded value.

式中P是邻域像素的数目,R是邻域的半径,0<t<2π,gc是中心像素的梯度方向角,gi是其邻域像素的梯度方向角。where P is the number of pixels in the neighborhood, R is the radius of the neighborhood, 0 < t < 2π, g c is the gradient direction angle of the central pixel, and gi is the gradient direction angle of its neighborhood pixels.

3)将LGDTP特征值分解为正负编码特征值,编码值不为1修改为0,得到正编码特征值LGDTPP,编码值为-1修改为1,其余编码值修改为0,得到负编码特征值LGDTPM3) Decompose the LGDTP eigenvalue into positive and negative coding eigenvalues, modify the coding value to 0 if the coding value is not 1, obtain the positive coding eigenvalue LGDTP P , modify the coding value -1 to 1, and modify the remaining coding values to 0 to obtain the negative coding Eigenvalue LGDTP M .

4)正编码特征值LGDTPP代替图像中的像素值构造正编码图像,负编码特征值LGDTPM代替图像中的像素值构造负编码图像。4) The positive coded feature value LGDTP P replaces the pixel values in the image to construct a positive coded image, and the negative coded feature value LGDTP M replaces the pixel values in the image to construct a negative coded image.

(3)确定特征向量(3) Determine the feature vector

1)将正编码特征值LGDTPP构成的正编码图像和负编码特征值LGDTPM构成的负编码图像各自均匀地分割为3×3子块,按顺序从上往下,从左至右标号1~9子区域,每个子区域的像素为m×n,m、n为128或256。1) Divide the positive coded image composed of the positive coding eigenvalue LGDTP P and the negative coded image composed of the negative coding eigenvalue LGDTP M into 3×3 sub-blocks uniformly, from top to bottom in order, and labelled 1 from left to right ~9 sub-regions, the pixels of each sub-region are m×n, and m and n are 128 or 256.

2)对正编码图像的每个子块中像素的梯度方向角进行直方图统计,级联所有子块直方图;对负编码图像中的每个子块中像素的梯度方向角进行直方图统计,级联所有子块直方图,将正编码图像和负编码图像的子块直方图级联。2) Perform histogram statistics on the gradient direction angles of the pixels in each sub-block of the positive-coded image, and cascade all the sub-block histograms; perform histogram statistics on the gradient direction angles of the pixels in each sub-block in the negative-coded image, level Concatenate all sub-block histograms, and concatenate the sub-block histograms of the positive and negative coded images.

3)将级联后的直方图作为整幅轮胎痕迹图像的特征,正编码图像LGDTPM和负编码图像LGDTPP特征的长度分别为256,最终编码图像LGDTP的特征长度为256×2×5,表示为FLGDTP3) The concatenated histogram is used as the feature of the entire tire track image, the length of the features of the positive encoding image LGDTP M and the negative encoding image LGDTP P are 256 respectively, and the feature length of the final encoded image LGDTP is 256 × 2 × 5, Denoted as F LGDTP .

4)将FLGDTP特征按(4)式进行归一化处理,得到归一化后的特征向量值:4) Normalize the F LGDTP feature according to formula (4) to obtain the normalized eigenvector value:

其中Fc(t)是特征c的第t个分量,是归一化后的特征向量值。where F c (t) is the t-th component of feature c, is the normalized eigenvector value.

(4)图像检索(4) Image retrieval

1)确定每一幅轮胎痕迹图像的特征向量值分别将每一幅图像作为查询图像与轮胎痕迹图像数据库中的每一幅图片按(5)用曼哈顿距离度量方法进行相似度度量:1) Determine the eigenvector value of each tire track image Take each image as a query image and each image in the tire track image database to measure the similarity using the Manhattan distance metric method according to (5):

其中d是两幅轮胎痕迹图像特征间的距离长度,Xi,Xj表示每一幅轮胎痕迹的特征向量;where d is the length of the distance between the features of two tire track images, X i , X j represent the feature vector of each tire track image;

2)用平均查准率P作为检索性能评价指标,按下式(6)确定:2) Using the average precision rate P as the retrieval performance evaluation index, it is determined by the following formula (6):

其中S为查询结果中包含正确图像数目,K为查询结果的图像总数,K小于轮胎痕迹图像数据库中每一类的张数。Where S is the number of correct images included in the query result, K is the total number of images in the query result, and K is less than the number of sheets of each category in the tire track image database.

在本发明的确定特征向量步骤(3)的步骤1)中,所述的m和n为能整除3的正整数、且相等。In the step 1) of the step (3) of determining the feature vector of the present invention, the m and n are positive integers that are divisible by 3 and are equal.

在本发明的特征提取步骤(2)的步骤2)中,式(3)中的t最佳为π/6。In step 2) of the feature extraction step (2) of the present invention, t in formula (3) is preferably π/6.

在本发明的确定特征向量步骤(3)的步骤2)中,根据轮胎痕迹图像纹理信息的空间分布特殊性,所述的子块直方图是标号分别为2,4,5,6,8子区域的直方图。In the step 2) of the step (3) of determining the feature vector of the present invention, according to the particularity of the spatial distribution of the texture information of the tire track image, the sub-block histograms are labeled 2, 4, 5, 6, and 8 respectively. Histogram of the area.

本发明针对轮胎痕迹图像数据库图像和查询图像,提取图像特征,用查询图像在轮胎痕迹图像数据库中查询相关种类轮胎痕迹图像,确定轮胎种类。通过分析得出轮胎与地面形成的轮胎痕迹图像包括主线沟槽和的边缘纹理线基本特征。本发明提出一种适用于轮胎痕迹图像的局部梯度方向三值模式特征,采用更加稳定的梯度方向值代替灰度值进行局部纹理编码,通过对中心像素梯度方向角进行阈值量化,产生高质量的纹理边缘信息,提高了检索准确度;将特征向量描述后的特征用曼哈顿距离进行相似度计算,得到检索结果,检索准确率明显优于其他纹理特征。本发明具有轮胎痕迹图像纹理边缘信息清晰、检索准确度高、平均查准率高、适用于大样本数据等优点,可用于轮胎痕迹图像特征提取。The invention extracts the image features according to the tire track image database image and the query image, and uses the query image to query the tire track image database of related types of tire track images to determine the tire type. Through the analysis, it is concluded that the tire track image formed by the tire and the ground includes the basic features of the main line groove and the edge texture line. The invention proposes a local gradient direction ternary pattern feature suitable for tire track images, uses a more stable gradient direction value instead of gray value to perform local texture coding, and quantizes the gradient direction angle of the central pixel by thresholding, producing high-quality images. The texture edge information improves the retrieval accuracy; the Manhattan distance is used to calculate the similarity of the features described by the feature vector, and the retrieval results are obtained, and the retrieval accuracy is significantly better than other texture features. The invention has the advantages of clear texture edge information of tire track images, high retrieval accuracy, high average precision, and is suitable for large sample data, and can be used for feature extraction of tire track images.

附图说明Description of drawings

图1是本发明实施例1的流程图。FIG. 1 is a flow chart of Embodiment 1 of the present invention.

图2是实施例1方法与6种现有的纹理特征提取方法的对比实验结果图。FIG. 2 is a comparison experiment result diagram of the method of Embodiment 1 and 6 existing texture feature extraction methods.

图3是实施例1方法与6种现有的纹理特征提取方法的对比实验曲线。FIG. 3 is a comparative experiment curve between the method of Embodiment 1 and 6 existing texture feature extraction methods.

具体实施方式Detailed ways

下面结合附图和实例对本发明进行进一步详细说明,但本发明不限于下述的实施例。The present invention will be further described in detail below with reference to the accompanying drawings and examples, but the present invention is not limited to the following embodiments.

实施例1Example 1

本实施例的图像来自申请人自建的轮胎痕迹图像数据库,包括30类每类80幅图像共计2400幅,进行了实验,局部梯度方向三值模式的轮胎痕迹图像特征提取方法步骤如下(参见图1):The images in this example come from the tire track image database built by the applicant, including 30 categories of 80 images in each category, a total of 2,400 images. Experiments have been carried out. The steps of the tire track image feature extraction method in the local gradient direction ternary mode are as follows (see Fig. 1):

(1)图像预处理(1) Image preprocessing

从轮胎痕迹图像数据库中选取轮胎痕迹样本图像30类每类80幅共2400幅,进行大小归一化为384×384、灰度化处理。From the tire track image database, select 2400 tire track sample images in 30 categories, 80 for each category, and normalize the size to 384 × 384 and grayscale.

(2)特征提取(2) Feature extraction

1)用Sobel边缘检测方法确定图像沿x方向的图像差分Gx和y方向的图像差分Gy,用公式(1)确定图像中每个像素的梯度方向角α(x,y)1) Use the Sobel edge detection method to determine the image difference G x along the x direction and the image difference G y in the y direction, and use the formula (1) to determine the gradient direction angle α(x, y) of each pixel in the image

α(x,y)=arctan(Gy/Gx) (7)α(x,y)=arctan(G y /G x ) (7)

2)在3×3的邻域滑窗中用局部梯度方向三值模式LGDTP方法确定每一个梯度方向角值,局部梯度方向三值模式LGDTP方法增加自定义阈值t,gi大于区间[gc-t,gc+t]}时为1,属于此区间为0,小于此区间为-1,得到三值编码值:2) In the 3×3 neighborhood sliding window, use the local gradient direction ternary mode LGDTP method to determine each gradient direction angle value, and the local gradient direction ternary mode LGDTP method adds a custom threshold t, g i is greater than the interval [g c -t,g c +t]} is 1, belongs to this interval is 0, is smaller than this interval is -1, and obtains a three-valued encoded value:

式中P是邻域像素的数目,R是邻域的半径,本实施例的t为2π,gc是中心像素的梯度方向角,gi是其邻域像素的梯度方向角。where P is the number of pixels in the neighborhood, R is the radius of the neighborhood, t in this embodiment is 2π, g c is the gradient direction angle of the central pixel, and gi is the gradient direction angle of its neighborhood pixels.

3)将LGDTP特征值分解为正负编码特征值,编码值不为1修改为0,得到正编码特征值LGDTPP,编码值为-1修改为1,其余编码值修改为0,得到负编码特征值LGDTPM3) Decompose the LGDTP eigenvalue into positive and negative coding eigenvalues, modify the coding value to 0 if the coding value is not 1, obtain the positive coding eigenvalue LGDTP P , modify the coding value -1 to 1, and modify the remaining coding values to 0 to obtain the negative coding Eigenvalue LGDTP M .

4)正编码特征值LGDTPP代替图像中的像素值构造正编码图像,负编码特征值LGDTPM代替图像中的像素值构造负编码图像。4) The positive coded feature value LGDTP P replaces the pixel values in the image to construct a positive coded image, and the negative coded feature value LGDTP M replaces the pixel values in the image to construct a negative coded image.

(3)确定特征向量(3) Determine the feature vector

1)将正编码特征值LGDTPP构成的正编码图像和负编码特征值LGDTPM构成的负编码图像各自均匀地分割为3×3子块,按顺序从上往下,从左至右标号1~9子区域,每个子区域的像素为m×n,本实施例的m、n为128。1) Divide the positive coded image composed of the positive coding eigenvalue LGDTP P and the negative coded image composed of the negative coding eigenvalue LGDTP M into 3×3 sub-blocks uniformly, from top to bottom in order, and labelled 1 from left to right ~9 sub-regions, the pixels of each sub-region are m×n, and m and n in this embodiment are 128.

2)对正编码图像的每个子块中像素的梯度方向角进行直方图统计,级联所有子块直方图;对负编码图像中的每个子块中像素的梯度方向角进行直方图统计,级联所有子块直方图,将正编码图像和负编码图像的子块直方图级联。根据轮胎痕迹图像纹理信息的空间分布特殊性,上述的子块直方图是标号分别为2,4,5,6,8子区域的直方图。2) Perform histogram statistics on the gradient direction angles of the pixels in each sub-block of the positive-coded image, and cascade all the sub-block histograms; perform histogram statistics on the gradient direction angles of the pixels in each sub-block in the negative-coded image, level Concatenate all sub-block histograms, and concatenate the sub-block histograms of the positive and negative coded images. According to the particularity of the spatial distribution of the texture information of the tire track image, the above-mentioned sub-block histograms are histograms of sub-regions numbered 2, 4, 5, 6, and 8 respectively.

3)将级联后的直方图作为整幅轮胎痕迹图像的特征,正编码图像LGDTPM和负编码图像LGDTPP特征的长度分别为256,最终编码图像LGDTP的特征长度为256×2×5,表示为FLGDTP3) The concatenated histogram is used as the feature of the entire tire track image, the length of the features of the positive encoding image LGDTP M and the negative encoding image LGDTP P are 256 respectively, and the feature length of the final encoded image LGDTP is 256 × 2 × 5, Denoted as F LGDTP .

4)将FLGDTP特征按(10)式进行归一化处理,得到归一化后的特征向量值:4) Normalize the F LGDTP feature according to formula (10) to obtain the normalized eigenvector value:

其中Fc(t)是特征c的第t个分量,是归一化后的特征向量值。where F c (t) is the t-th component of feature c, is the normalized eigenvector value.

(4)图像检索(4) Image retrieval

1)确定每一幅轮胎痕迹图像的特征向量值分别将每一幅图像作为查询图像与轮胎痕迹图像数据库中的每一幅图片按(11)用曼哈顿距离度量方法进行相似度度量:1) Determine the eigenvector value of each tire track image Take each image as a query image and each image in the tire track image database to measure the similarity using the Manhattan distance metric method according to (11):

其中d是两幅轮胎痕迹图像特征间的距离长度,Xi,Xj表示每一幅轮胎痕迹的特征向量。where d is the length of the distance between the features of two tire track images, and X i and X j represent the feature vector of each tire track image.

2)用平均查准率P作为检索性能评价指标,按下式(12)确定:2) Using the average precision rate P as the retrieval performance evaluation index, it is determined by the following formula (12):

其中S为查询结果中包含正确图像数目,K为查询结果的图像总数。Where S is the number of correct images contained in the query result, and K is the total number of images in the query result.

在本实施例中,采用本实施例方法与局部二值模式LBP方法、局部方向模式LDP方法、增强型局部方向模式ELDP方法、局部方向数模式LDN方法、最优化局部方向模式OLDP方法、局部三值模式LTP方法在轮胎痕迹图像数据库利用曼哈顿距离进行对比检索实验,局部二值模式LBP方法、局部方向模式LDP方法、增强型局部方向模式ELDP方法、局部方向数模式LDN方法、最优化局部方向模式OLDP方法特征维度为256×5,局部三值模式LTP方法是将LTP特征值分解为正负编码特征值,正编码特征值LTPP代替图像中的像素值构造正编码图像,负编码特征值LTPM代替图像中的像素值构造负编码图像,正编码特征值LTPP和负编码特征值LTPM级联后组成特征维度为256×5×2,按照步骤(4)中的2)计算每种方法的平均查准率,实验结果见表1和图2。几种方法的编码图像见图3,在图3中给出了原始图片、6种方法、本实施例的编码图像,其中本实施例分为正编码特征图像LGDTPP和负编码特征图像LGDTPMIn this embodiment, the method of this embodiment and the local binary mode LBP method, the local direction mode LDP method, the enhanced local direction mode ELDP method, the local direction number mode LDN method, the optimized local direction mode OLDP method, and the local three-dimensional mode are adopted. The value mode LTP method uses the Manhattan distance to carry out comparative retrieval experiments in the tire track image database, the local binary mode LBP method, the local direction mode LDP method, the enhanced local direction mode ELDP method, the local direction number mode LDN method, and the optimized local direction mode. The feature dimension of the OLDP method is 256×5, and the LTP method of the local three-value mode is to decompose the LTP eigenvalues into positive and negative coded eigenvalues. M replaces the pixel values in the image to construct a negative coded image, the positive coded feature value LTP P and the negative coded eigenvalue LTP M are concatenated to form a feature dimension of 256×5×2, according to 2) in step (4) to calculate each The average precision of the method, the experimental results are shown in Table 1 and Figure 2. The coded images of several methods are shown in Fig. 3. In Fig. 3, the original picture, the 6 methods, and the coded images of this embodiment are given, wherein this embodiment is divided into a positive coded feature image LGDTP P and a negative coded feature image LGDTP M .

表1 本实施例与6种方法的对比实验结果(K=10)Table 1 Comparative experimental results between this embodiment and 6 methods (K=10)

序号serial number 特征feature 维度dimension 平均查准率(K=10)Average precision (K=10) 11 LBPLBP 12801280 43.1%,43.1%, 22 LDPLDP 12801280 45%45% 33 ELDPELDP 12801280 49.5%,49.5%, 44 LDNLDN 12801280 47.3%47.3% 55 OLDPOLDP 12801280 45.6%,45.6%, 66 LTPLTP 25602560 52.1%52.1% 77 本实施例This embodiment 25602560 63.3%63.3%

由表1、图2可见,K为10时,本实施例的平均查准率为63.3%,局部二值模式LBP方法、局部方向模式LDP方法、增强型局部方向模式ELDP方法、局部方向数模式LDN方法、最优化局部方向模式OLDP方法、局部三值模式LTP方法的平均查准率分别为43.1%、45%、49.5%、47.3%、45.6%、52.1%,可以看出本方法获得最高的平均查准率。本实施例方法的编码图像更加清晰。It can be seen from Table 1 and Figure 2 that when K is 10, the average precision of this embodiment is 63.3%. The local binary mode LBP method, the local direction mode LDP method, the enhanced local direction mode ELDP method, and the local direction number mode The average precision rates of the LDN method, the optimized local direction mode OLDP method, and the local ternary mode LTP method are 43.1%, 45%, 49.5%, 47.3%, 45.6%, and 52.1%, respectively. It can be seen that this method achieves the highest accuracy. average precision. The encoded image of the method of this embodiment is clearer.

实施例2Example 2

本实施例的图像来自申请人自建的轮胎痕迹图像数据库,包括30类每类80幅图像共计2400幅,进行实验,局部梯度方向三值模式的轮胎痕迹图像特征提取方法步骤如下:The images in this embodiment are from the tire track image database built by the applicant, including 30 categories of 80 images in each category, a total of 2,400 images. Experiments are carried out. The steps of the tire track image feature extraction method in the local gradient direction ternary mode are as follows:

(1)图像预处理(1) Image preprocessing

该步骤与实施例1相同。This procedure is the same as in Example 1.

(2)特征提取(2) Feature extraction

1)用Sobel边缘检测方法确定图像沿x方向的图像差分Gx和y方向的图像差分Gy,用公式(1)确定图像中每个像素的梯度方向角α(x,y)1) Use the Sobel edge detection method to determine the image difference G x along the x direction and the image difference G y in the y direction, and use the formula (1) to determine the gradient direction angle α(x, y) of each pixel in the image

α(x,y)=arctan(Gy/Gx) (13)α(x,y)=arctan(G y /G x ) (13)

2)在3×3的邻域滑窗中用局部梯度方向三值模式LGDTP方法确定每一个梯度方向角值,局部梯度方向三值模式LGDTP方法增加自定义阈值t,gi大于区间[gc-t,gc+t]}时为1,属于此区间为0,小于此区间为-1,得到三值编码值:2) In the 3×3 neighborhood sliding window, use the local gradient direction ternary mode LGDTP method to determine each gradient direction angle value, and the local gradient direction ternary mode LGDTP method adds a custom threshold t, g i is greater than the interval [g c -t,g c +t]} is 1, belongs to this interval is 0, is smaller than this interval is -1, and obtains a three-valued encoded value:

式中P是邻域像素的数目,R是邻域的半径,本实施例的t为π/18,gc是中心像素的梯度方向角,gi是其邻域像素的梯度方向角。该步骤中的其它步骤与实施例1相同。where P is the number of pixels in the neighborhood, R is the radius of the neighborhood, t in this embodiment is π/18, g c is the gradient direction angle of the central pixel, and gi is the gradient direction angle of its neighborhood pixels. The other steps in this step are the same as in Example 1.

其它步骤与实施例1相同。Other steps are the same as in Example 1.

实施例3Example 3

本实施例的图像来自申请人自建的轮胎痕迹图像数据库,包括30类每类80幅图像共计2400幅,进行实验,局部梯度方向三值模式的轮胎痕迹图像特征提取方法步骤如下:The images in this embodiment are from the tire track image database built by the applicant, including 30 categories of 80 images in each category, a total of 2,400 images. Experiments are carried out. The steps of the tire track image feature extraction method in the local gradient direction ternary mode are as follows:

(1)图像预处理(1) Image preprocessing

该步骤与实施例1相同。This procedure is the same as in Example 1.

(2)特征提取(2) Feature extraction

1)用Sobel边缘检测方法确定图像沿x方向的图像差分Gx和y方向的图像差分Gy,用公式(1)确定图像中每个像素的梯度方向角α(x,y)1) Use the Sobel edge detection method to determine the image difference G x along the x direction and the image difference G y in the y direction, and use the formula (1) to determine the gradient direction angle α(x, y) of each pixel in the image

α(x,y)=arctan(Gy/Gx) (16)α(x,y)=arctan(G y /G x ) (16)

2)在3×3的邻域滑窗中用局部梯度方向三值模式LGDTP方法确定每一个梯度方向角值,局部梯度方向三值模式LGDTP方法增加自定义阈值t,gi大于区间[gc-t,gc+t]}时为1,属于此区间为0,小于此区间为-1,得到三值编码值:2) In the 3×3 neighborhood sliding window, use the local gradient direction ternary mode LGDTP method to determine each gradient direction angle value, and the local gradient direction ternary mode LGDTP method adds a custom threshold t, g i is greater than the interval [g c -t,g c +t]} is 1, belongs to this interval is 0, is less than this interval is -1, and obtains a three-value code value:

式中P是邻域像素的数目,R是邻域的半径,本实施例的t为3π/2,gc是中心像素的梯度方向角,gi是其邻域像素的梯度方向角。该步骤中的其它步骤与实施例1相同。where P is the number of pixels in the neighborhood, R is the radius of the neighborhood, t in this embodiment is 3π/2, g c is the gradient direction angle of the center pixel, and gi is the gradient direction angle of its neighborhood pixels. The other steps in this step are the same as in Example 1.

其它步骤与实施例1相同。Other steps are the same as in Example 1.

实施例4Example 4

在以上的实施例1-3的图像预处理步骤(1)中,从轮胎痕迹图像数据库中选取轮胎痕迹样本图像30类每类80幅共2400幅,进行大小归一化为768×768、灰度化处理。In the image preprocessing step (1) of the above embodiment 1-3, 2400 tire track sample images of 30 categories and 80 images of each category are selected from the tire track image database, and the size is normalized to 768×768, gray Scale processing.

本实施例的确定特征向量步骤(3)的1)步骤为:将正编码特征值LGDTPP构成的正编码图像和负编码特征值LGDTPM构成的负编码图像各自均匀地分割为3×3子块,按顺序从上往下,从左至右标号1~9子区域,每个子区域的像素为m×n,本实施例的m、n为256。该步骤中的其它步骤与实施例1相同。Step 1) of the step (3) of determining the feature vector in this embodiment is: dividing the positive coded image composed of the positive coded feature value LGDTP P and the negative coded image composed of the negative coded feature value LGDTP M into 3×3 subsections uniformly, respectively. The blocks are numbered 1 to 9 sub-regions from top to bottom in order, and the pixels of each sub-region are m×n, and m and n in this embodiment are 256. The other steps in this step are the same as in Example 1.

其它步骤与实施例1相同。Other steps are the same as in Example 1.

Claims (4)

1.一种局部梯度方向三值模式的轮胎痕迹图像特征提取方法,其特征在于由以下步骤组成:1. a tire trace image feature extraction method of a local gradient direction ternary pattern is characterized in that being made up of the following steps: (1)图像预处理(1) Image preprocessing 从轮胎痕迹图像数据库中选取轮胎痕迹样本图像30类每类50~80张进行大小归一化、灰度化处理;Select 30 types of tire trace sample images from the tire trace image database, 50 to 80 for each type, and perform size normalization and grayscale processing; (2)特征提取(2) Feature extraction 1)用Sobel边缘检测方法确定图像沿x方向的图像差分Gx和y方向的图像差分Gy,用公式(1)确定图像中每个像素的梯度方向角α(x,y)1) Use the Sobel edge detection method to determine the image difference G x along the x direction and the image difference G y in the y direction, and use the formula (1) to determine the gradient direction angle α(x, y) of each pixel in the image α(x,y)=arctan(Gy/Gx) (1)α(x,y)=arctan(G y /G x ) (1) 2)在3×3的邻域滑窗中用局部梯度方向三值模式LGDTP方法确定每一个梯度方向角值,局部梯度方向三值模式LGDTP方法增加自定义阈值t,gi大于区间[gc-t,gc+t]}时为1,属于此区间为0,小于此区间为-1,得到三值编码值:2) In the 3×3 neighborhood sliding window, use the local gradient direction ternary mode LGDTP method to determine each gradient direction angle value, and the local gradient direction ternary mode LGDTP method increases the custom threshold t, g i is greater than the interval [g c -t,g c +t]} is 1, belongs to this interval is 0, is smaller than this interval is -1, and obtains a three-valued encoded value: 式中P是邻域像素的数目,R是邻域的半径,0<t<2π,gc是中心像素的梯度方向角,gi是其邻域像素的梯度方向角;where P is the number of neighborhood pixels, R is the radius of the neighborhood, 0 < t < 2π, g c is the gradient direction angle of the central pixel, and gi is the gradient direction angle of its neighborhood pixels; 3)将LGDTP特征值分解为正负编码特征值,编码值不为1修改为0,得到正编码特征值LGDTPP,编码值为-1修改为1,其余编码值修改为0,得到负编码特征值LGDTPM3) Decompose the LGDTP eigenvalue into positive and negative coding eigenvalues, modify the coding value to 0 if the coding value is not 1, obtain the positive coding eigenvalue LGDTP P , modify the coding value -1 to 1, and modify the remaining coding values to 0 to obtain the negative coding eigenvalue LGDTP M ; 4)正编码特征值LGDTPP代替图像中的像素值构造正编码图像,负编码特征值LGDTPM代替图像中的像素值构造负编码图像;4) positive coding feature value LGDTP P replaces the pixel value in the image to construct a positive coding image, and negative coding feature value LGDTP M replaces the pixel value in the image to construct a negative coding image; (3)确定特征向量(3) Determine the feature vector 1)将正编码特征值LGDTPP构成的正编码图像和负编码特征值LGDTPM构成的负编码图像各自均匀地分割为3×3子块,按顺序从上往下,从左至右标号1~9子区域,每个子区域的像素为m×n,m、n为128或256;1) Divide the positive coded image composed of the positive coding eigenvalue LGDTP P and the negative coded image composed of the negative coding eigenvalue LGDTP M into 3×3 sub-blocks uniformly, from top to bottom in order, and labelled 1 from left to right ~9 sub-regions, the pixels of each sub-region are m×n, and m and n are 128 or 256; 2)对正编码图像的每个子块中像素的梯度方向角进行直方图统计,级联所有子块直方图;对负编码图像中的每个子块中像素的梯度方向角进行直方图统计,级联所有子块直方图,将正编码图像和负编码图像的子块直方图级联;2) Perform histogram statistics on the gradient direction angles of the pixels in each sub-block of the positive-coded image, and cascade all the sub-block histograms; perform histogram statistics on the gradient direction angles of the pixels in each sub-block in the negative-coded image, level Concatenate all the sub-block histograms, and concatenate the sub-block histograms of the positive coded image and the negative coded image; 3)将级联后的直方图作为整幅轮胎痕迹图像的特征,正编码图像LGDTPM和负编码图像LGDTPP特征的长度分别为256,最终编码图像LGDTP的特征长度为256×2×5,表示为FLGDTP3) The concatenated histogram is used as the feature of the entire tire track image, the length of the features of the positive encoding image LGDTP M and the negative encoding image LGDTP P are 256 respectively, and the feature length of the final encoded image LGDTP is 256 × 2 × 5, Denoted as F LGDTP ; 4)将FLGDTP特征按(4)式进行归一化处理,得到归一化后的特征向量值:4) Normalize the F LGDTP feature according to formula (4) to obtain the normalized eigenvector value: 其中Fc(t)是特征c的第t个分量,是归一化后的特征向量值;where F c (t) is the t-th component of feature c, is the normalized eigenvector value; (4)图像检索(4) Image retrieval 1)确定每一幅轮胎痕迹图像的特征向量值分别将每一幅图像作为查询图像与轮胎痕迹图像数据库中的每一幅图片按(5)用曼哈顿距离度量方法进行相似度度量:1) Determine the eigenvector value of each tire track image Take each image as a query image and each image in the tire track image database to measure the similarity using the Manhattan distance metric method according to (5): 其中d是两幅轮胎痕迹图像特征间的距离长度,Xi,Xj表示每一幅轮胎痕迹的特征向量;where d is the distance between the features of the two tire track images, X i , X j represent the feature vector of each tire track image; 2)用平均查准率P作为检索性能评价指标,按下式(6)确定:2) Using the average precision rate P as the retrieval performance evaluation index, it is determined by the following formula (6): 其中S为查询结果中包含正确图像数目,K为查询结果的图像总数,K小于轮胎痕迹图像数据库中每一类的张数。Where S is the number of correct images contained in the query result, K is the total number of images in the query result, and K is less than the number of sheets of each category in the tire track image database. 2.根据权利要求1所述的局部梯度方向三值模式的轮胎痕迹图像特征提取方法,其特征在于:在确定特征向量步骤(3)的步骤1)中,所述的m和n为能整除3的正整数、且相等。2. The tire trace image feature extraction method of local gradient direction ternary mode according to claim 1, is characterized in that: in step 1) of determining feature vector step (3), described m and n are divisible 3 is a positive integer and is equal. 3.根据权利要求1所述的局部梯度方向三值模式的轮胎痕迹图像特征提取方法,其特征在于:在特征提取步骤(2)的步骤2)中,式(3)中的t为π/6。3. The tire trace image feature extraction method of the local gradient direction ternary pattern according to claim 1, is characterized in that: in step 2) of feature extraction step (2), t in formula (3) is π/ 6. 4.根据权利要求1所述的局部梯度方向三值模式的轮胎痕迹图像特征提取方法,其特征在于:在确定特征向量步骤(3)的步骤2)中,根据轮胎痕迹图像纹理信息的空间分布特殊性,所述的子块直方图是标号分别为2,4,5,6,8子区域的直方图。4. The tire track image feature extraction method of the local gradient direction ternary pattern according to claim 1, is characterized in that: in the step 2) of determining the feature vector step (3), according to the spatial distribution of tire track image texture information The particularity, the sub-block histogram is the histogram of sub-regions labeled 2, 4, 5, 6, and 8 respectively.
CN201910699380.5A 2019-07-31 2019-07-31 Feature Extraction Method of Tire Trace Image Based on Local Gradient Orientation Ternary Mode Active CN110489587B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910699380.5A CN110489587B (en) 2019-07-31 2019-07-31 Feature Extraction Method of Tire Trace Image Based on Local Gradient Orientation Ternary Mode

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910699380.5A CN110489587B (en) 2019-07-31 2019-07-31 Feature Extraction Method of Tire Trace Image Based on Local Gradient Orientation Ternary Mode

Publications (2)

Publication Number Publication Date
CN110489587A true CN110489587A (en) 2019-11-22
CN110489587B CN110489587B (en) 2023-04-28

Family

ID=68548845

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910699380.5A Active CN110489587B (en) 2019-07-31 2019-07-31 Feature Extraction Method of Tire Trace Image Based on Local Gradient Orientation Ternary Mode

Country Status (1)

Country Link
CN (1) CN110489587B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112918956A (en) * 2021-02-20 2021-06-08 陆伟凤 Garbage classification system based on image recognition technology

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100105865A1 (en) * 2000-10-27 2010-04-29 John Telford Nucleic acids and proteins from streptococcus groups a & b
CN105389573A (en) * 2015-12-23 2016-03-09 山东大学 Face recognition method based on stratified reconstruction in local ternary pattern
EP3076367A1 (en) * 2013-07-22 2016-10-05 Zhejiang University Method for road detection from one image
CN106228163A (en) * 2016-07-25 2016-12-14 长安大学 The local poor ternary sequential image feature that a kind of feature based selects describes method
CN106529504A (en) * 2016-12-02 2017-03-22 合肥工业大学 Dual-mode video emotion recognition method with composite spatial-temporal characteristic
CN107122396A (en) * 2017-03-13 2017-09-01 西北大学 Three-dimensional model searching algorithm based on depth convolutional neural networks
CN109035317A (en) * 2018-07-04 2018-12-18 重庆邮电大学 Illumination reversion and invariable rotary texture expression based on three value mode of gradient local
WO2019100436A1 (en) * 2017-11-22 2019-05-31 Zhejiang Dahua Technology Co., Ltd. Methods and systems for face recognition
WO2019136894A1 (en) * 2018-01-10 2019-07-18 Zhejiang Dahua Technology Co., Ltd. Methods and systems for face alignment

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100105865A1 (en) * 2000-10-27 2010-04-29 John Telford Nucleic acids and proteins from streptococcus groups a & b
EP3076367A1 (en) * 2013-07-22 2016-10-05 Zhejiang University Method for road detection from one image
CN105389573A (en) * 2015-12-23 2016-03-09 山东大学 Face recognition method based on stratified reconstruction in local ternary pattern
CN106228163A (en) * 2016-07-25 2016-12-14 长安大学 The local poor ternary sequential image feature that a kind of feature based selects describes method
CN106529504A (en) * 2016-12-02 2017-03-22 合肥工业大学 Dual-mode video emotion recognition method with composite spatial-temporal characteristic
CN107122396A (en) * 2017-03-13 2017-09-01 西北大学 Three-dimensional model searching algorithm based on depth convolutional neural networks
WO2019100436A1 (en) * 2017-11-22 2019-05-31 Zhejiang Dahua Technology Co., Ltd. Methods and systems for face recognition
WO2019136894A1 (en) * 2018-01-10 2019-07-18 Zhejiang Dahua Technology Co., Ltd. Methods and systems for face alignment
CN109035317A (en) * 2018-07-04 2018-12-18 重庆邮电大学 Illumination reversion and invariable rotary texture expression based on three value mode of gradient local

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
FAISAL AHMED: "Automated Facial Expression Recognition Using Gradient-Based Ternary Texture Patterns", 《CHINESE JOURNAL OF ENGINEERING》 *
MD AZHER UDDIN: "Dynamic Scene Recognition Using Spatiotemporal Based DLTP on Spark", 《IEEE ACCESS》 *
ZEYNAB SHOKOOHI: "Expression recognition using directional gradient local pattern and gradient-based ternary texture patterns", 《2015 2ND INTERNATIONAL CONFERENCE ON PATTERN RECOGNITION AND IMAGE ANALYSIS (IPRIA)》 *
赵飞飞: "基于视频的人脸追踪与识别", 《中国优秀硕士学位论文全文数据库 (信息科技辑)》 *
齐美彬: "改进特征与GPU加速的行人检测", 《中国图象图形学报》 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112918956A (en) * 2021-02-20 2021-06-08 陆伟凤 Garbage classification system based on image recognition technology

Also Published As

Publication number Publication date
CN110489587B (en) 2023-04-28

Similar Documents

Publication Publication Date Title
CN102254303B (en) Methods for segmenting and searching remote sensing image
Pietikäinen et al. Two decades of local binary patterns: A survey
CN104809481B (en) A kind of natural scene Method for text detection based on adaptive Color-based clustering
CN106021330B (en) A kind of method for searching three-dimension model for mixing under contour line view
CN104778457B (en) Video face identification method based on multi-instance learning
CN103218621B (en) The recognition methods of multiple dimensioned vehicle in a kind of life outdoor videos monitoring
CN104182538B (en) Image search method based on semi-supervised Hash
CN105956560A (en) Vehicle model identification method based on pooling multi-scale depth convolution characteristics
CN101847163A (en) Design patent image retrieval method with multi-characteristics fusion
CN106355577A (en) Method and system for quickly matching images on basis of feature states and global consistency
Wang et al. DCT inspired feature transform for image retrieval and reconstruction
CN104036012A (en) Dictionary learning method, visual word bag characteristic extracting method and retrieval system
CN103778434A (en) Face recognition method based on multi-resolution multi-threshold local binary pattern
CN110706235A (en) Far infrared pedestrian detection method based on two-stage cascade segmentation
CN112560858B (en) Character and picture detection and rapid matching method combining lightweight network and personalized feature extraction
CN106503694A (en) Digit recognition method based on eight neighborhood feature
CN106203373A (en) A kind of human face in-vivo detection method based on deep vision word bag model
Huang et al. Local image region description using orthogonal symmetric local ternary pattern
CN104778472A (en) Extraction method for facial expression feature
CN110599478A (en) Image area copying and pasting tampering detection method
CN108805183B (en) An Image Classification Method Fusing Local Aggregated Descriptors and Local Linear Coding
Chen et al. Image retrieval based on quadtree classified vector quantization
CN110489587B (en) Feature Extraction Method of Tire Trace Image Based on Local Gradient Orientation Ternary Mode
CN107578005A (en) A LBP Face Recognition Method in Complex Wavelet Transform Domain
CN110705569A (en) Image local feature descriptor extraction method based on texture features

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant