CN107633038A - Tealeaves recognition methods and its system based on image recognition technology - Google Patents

Tealeaves recognition methods and its system based on image recognition technology Download PDF

Info

Publication number
CN107633038A
CN107633038A CN201710821319.4A CN201710821319A CN107633038A CN 107633038 A CN107633038 A CN 107633038A CN 201710821319 A CN201710821319 A CN 201710821319A CN 107633038 A CN107633038 A CN 107633038A
Authority
CN
China
Prior art keywords
tealeaves
image
module
title
identified
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201710821319.4A
Other languages
Chinese (zh)
Inventor
范方媛
李舒江
郭昊蔚
李春霖
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University ZJU
Original Assignee
Zhejiang University ZJU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University ZJU filed Critical Zhejiang University ZJU
Priority to CN201710821319.4A priority Critical patent/CN107633038A/en
Publication of CN107633038A publication Critical patent/CN107633038A/en
Pending legal-status Critical Current

Links

Landscapes

  • Image Analysis (AREA)

Abstract

The present invention, which provides one kind, can identify unknown tealeaves immediately, and the tealeaves identifying system based on image recognition technology of tealeaves characteristic information is corresponded to user's science popularization, including image capture module, feature recognition module, image processor module, database and display module;The present invention also proposes that one kind carries out tealeaves according to the system and knows method for distinguishing, and the tealeaves information to be identified of input is scanned for and matched using database after treatment by feature recognition module or image processor module first;Feature recognition module or image processor module will be searched for and matched obtained tealeaves title and shown by display module afterwards.Wherein image processor module includes classified inquiry module and name search module, while so as to enable the invention to carry out image recognition to unknown tealeaves by feature recognition module, additionally it is possible to be made instantly available tealeaves characteristic information corresponding to certain accurate class or certain tealeaves by inquiring module or name search module by classification.

Description

Tealeaves recognition methods and its system based on image recognition technology
Technical field
The present invention relates to the image recognition technology of mobile terminal, and in particular to a kind of tealeaves based on image recognition technology is known Other method and its system.
Background technology
With the continuous improvement that the fast development of economic society and people are required healthy living, tealeaves is as natural health Drink is favored by more and more consumers.China is the country for finding and drinking earliest tealeaves, with tea making technology not Disconnected development, the tealeaves in China are divided into six big teas according to the difference of tea-manufacturing technology;It is various however as the deep and product of research The development of change trend, all kinds of tealeaves are in a great variety, and the rich choice of products of in the market is various, and this provides the same of more more options for consumer When, also increase the difficulty that consumer's identification judges tea quality.In order to obtain more accurate information, consumer can pass through tea The multiple channels such as leaf pertinent texts, magazine, network;With the application popularization of smart mobile phone, tealeaves correlation science popularization software meet the tendency of and It is raw, provide a more easily channel for the tea knowledge acquisition of consumer.But investigation display, although mobile phone application at present The cell phone software that the tealeaves of in the market application is related is abundant in content, but the following defect of generally existing:(1), content broad covered area, it is right It is weaker in the guided bone of beginner from the superficial to the deep;(2) it is, weaker for the instant application of common primary consumer, it is seen that tealeaves without Method obtains rapidly relevant information by software application;(3), tealeaves feature recognition does not correspond to chain effectively with tealeaves property data base Connect.
Data for images processing is a kind of specific manifestation using computer digital animation, is a kind of digital picture to be entered Row automation computing and a kind of means of processing.Image processing techniques has operability in terms of the identification immediately of tealeaves feature, There is realistic meaning for the lifting instant application of software, and lack identify tealeaves and phase immediately by image processing techniques now The method and system of user's science popularization tealeaves characteristic information.
Convolutional neural networks (Convolutional Neural Network, CNN) are a kind of feedforward neural networks, it Artificial neuron can respond the surrounding cells in a part of coverage, have outstanding performance for large-scale image procossing.
The establishment of convolutional neural networks model, optimization, there are a variety of workable algorithms (increase income or non-increase income), similarly for " component " (weights are shared, pond) of convolutional neural networks model realizes that there is also a variety of different realizations now with code Code form, such as existing Caffe (refer to convolutional neural networks framework, full name Convolutional Architecture for Fast Feature Embedding), TensorFlow (refer to Google based on the artificial intelligence of the second generation that DistBelief is researched and developed Can learning system), Theano (deep learning instrument) etc. be ripe convolutional neural networks model build tool, its code is equal Can be increased income acquisition;
MSGD (small lot stochastic gradient descent) algorithm is one of convolutional neural networks Model Parameter Optimization algorithm.
The content of the invention
Unknown tealeaves can be identified immediately it is an object of the invention to provide one kind and corresponds to tealeaves feature to user's science popularization The tealeaves identifying system based on image recognition technology of information, and the tealeaves recognition methods that the system is carried out.
In order to solve the above-mentioned technical problem, the present invention provides a kind of tealeaves identifying system based on image recognition technology;
The mobile terminal includes image capture module, feature recognition module, image processor module, database and display mould Block;Signal is connected successively for described image acquisition module, feature recognition module and database;Described image acquisition module, feature are known Other module and image processor module are connected with display module signal, and wherein image processor module is connected with data field signal;
Described image acquisition module is used to obtain tealeaves image to be identified, and tealeaves image to be identified is sent to feature and known Other module;
The feature recognition module is used to the tealeaves image to be identified acquired in image capture module is handled and divided Analysis, and extract the characteristic vector of the tealeaves image to be identified;The feature recognition module uses image recognition model;
The image processor module is used to input the title or classification of tealeaves to be identified, and scanned for using database and Matching;
The database is used to store tealeaves sample image data collection and tealeaves characteristic information;The feature recognition module will The tealeaves image feature vector to be identified of extraction is concentrated in tealeaves sample image data using database and scans for and match, special Identification module is levied to arrange the tealeaves title corresponding to the sample tealeaves image to match according to the order of matching degree from high to low Shown by display module;
The image processor module will input or the tealeaves classification of selection (including oolong tea, white tea, green tea, black tea, yellow tea And black tea) matched using database with tealeaves classification in tealeaves characteristic information, and will match corresponding to tealeaves classification Tealeaves title is exported to display module;
The image processor module will input or the tealeaves title of selection utilizes database and tealeaves in tealeaves characteristic information Title is matched, and the tealeaves characteristic information corresponding to the tealeaves title to match is exported to display module;
The display module is used for the tealeaves image information to be identified of display image acquisition module collection, and shows by feature The tealeaves title that identification module is sent;The display module is additionally operable to show the tealeaves title or classification of image processor module input Information, and show the tealeaves title sent by image processor module or tealeaves characteristic information.
Improvement as tealeaves identifying system of the present invention based on image recognition technology:
Described image identification model is convolutional neural networks model;
The database includes the sample image database and tealeaves knowledge data base that signal is connected;
The name search module and classification that the image processor module includes with tealeaves knowledge data base signal being connected are looked into Ask module;
The sample image database is connected with feature recognition module signal, and sample image database is used to store tealeaves name Claim, and the sample tealeaves image data set being stored under tealeaves title, the sample tealeaves image data set comprises at least pair Answer the characteristic vector of sample tealeaves image;The feature recognition module is by the characteristic vector of the tealeaves image to be identified of extraction in sample Matched in this image data base, i.e. feature recognition module utilizes sample image database and sample tealeaves image data set The characteristic vector of middle sample tealeaves image is matched;
The tealeaves knowledge data base is connected with name search module and classified inquiry module by signal respectively, tealeaves knowledge number It is used to store tealeaves characteristic information according to storehouse, tealeaves characteristic information comprises at least tealeaves title and tealeaves classification;The classified inquiry Module is inputted or the tealeaves classification of selection is matched with tealeaves classification in tealeaves knowledge data base, i.e. classified inquiry module profit Matched with tealeaves knowledge data base with tealeaves classification in tealeaves characteristic information, classified inquiry module by tealeaves categorical match into The tealeaves title of work(is shown by display module;The tealeaves title of the input of name search module or selection is known with tealeaves The tealeaves title known in database matches, i.e. name search module is using in tealeaves knowledge data base and tealeaves characteristic information Tealeaves title is matched, tealeaves characteristic information of the name search module corresponding to by the successful tealeaves title of tealeaves categorical match Shown by display module.
In order to solve the above-mentioned technical problem, the present invention proposes a kind of tealeaves recognition methods based on image recognition technology, according to It is secondary to follow the steps below:
1), feature recognition module or the image processor module utilizes the tealeaves information to be identified of input after treatment Database is scanned for and matched;
2), feature recognition module or image processor module will search for and match in step 1) obtained tealeaves title pass through it is aobvious Show that module is shown.
Improvement as tealeaves recognition methods of the present invention based on image recognition technology:
Described tealeaves information to be identified is included by the tealeaves image of image capture module collection and by image processor module Input or the tealeaves title and tealeaves classification of selection.
Further improvement as tealeaves recognition methods of the present invention based on image recognition technology:
When the tealeaves information to be identified is tealeaves image, follow the steps below successively:
1.1) tealeaves image to be identified, is gathered by image capture module;
2.1), the tealeaves image to be identified described in step 1.1) is sent to feature recognition module and carried out by image capture module Image procossing and analysis, feature recognition module extract the characteristic parameter of the tealeaves image to be identified;Feature recognition module is by institute The characteristic parameter of the tealeaves image to be identified obtained (is stored in database using the characteristic vector of database and sample tealeaves image Tealeaves sample image data collection) scan for and match, the tealeaves title and matching degree to be matched;
3.1), the tealeaves name that feature recognition module will match in step 2.1) with the characteristic parameter of tealeaves image to be identified Claim according to matching degree being arranged sequentially on display module and show from high to low;When user selects tealeaves title, image processor mould Block recalls such tealeaves characteristic information of corresponding tealeaves title from database and shown by display module.
Further improvement as tealeaves recognition methods of the present invention based on image recognition technology:
The feature recognition module uses convolutional neural networks model;
The concrete operation step of the step 2.1) is as follows:
2.1.1) pre-process:Feature recognition module carries out noise abatement to the tealeaves image to be identified received by noise filter Pretreatment;
2.1.2) extraction image features:Described image characteristic parameter is obtained by multilayer convolution algorithm, comprising whole The global information of tealeaves image to be identified;Concretely comprise the following steps:
Feature recognition module is extracted the tealeaves image to be identified of gained after above-mentioned pretreatment by convolution filter Image local feature, weights are shared and convolutional layer pondization extracts characteristics of image at least twice, are linked afterwards using full articulamentum All characteristics of image obtain image feature vector;
2.1.3) by step 2.1.2) in known using the characteristic vector of multilayer convolution algorithm extraction by the feature built Simultaneously prediction is identified based on the tealeaves sample image data collection in database in other module, and last feature recognition module is according to input Image feature vector and database in tealeaves sample image data concentrate the image feature vector similarity that matches (to weigh The matching degree of tealeaves to be identified is calculated again).
Further improvement as tealeaves recognition methods of the present invention based on image recognition technology:
When tealeaves information to be identified is tealeaves classification, follow the steps below successively:
1.2) tealeaves classification to be identified, is inputted or selected by the classified inquiry module in image processor module;
2.2), the tealeaves classification to be identified that step 1.2) inputs or selects is utilized database and tea by classified inquiry module Tealeaves classification is scanned for and matched in leaf characteristic information, obtains the tealeaves title that classification matches;
3.2) module, is inquired by classification by all tealeaves titles to match with tealeaves classification to be identified obtained by step 2.2) It is arranged in order, is shown by display module;When user selects tealeaves title, name search module in image processor module is from number Shown according to such tealeaves characteristic information that corresponding tealeaves title is recalled in storehouse and by display module.
Further improvement as tealeaves recognition methods of the present invention based on image recognition technology:
Tealeaves information to be identified is tealeaves title;Follow the steps below successively:
1.3) tealeaves title to be identified, is inputted by the name search module in image processor module, or directly selected aobvious Show the tealeaves title to be identified shown in module;
2.3), name search module believes the tealeaves title to be identified described in step 1.3) using database and tealeaves feature Tealeaves title is scanned for and matched in breath, the tealeaves title matched;
3.3), name search module is right by the tealeaves title institute to match with tealeaves title to be identified obtained by step 2.2) Tealeaves characteristic information is answered to be shown by display module.
Further improvement as tealeaves recognition methods of the present invention based on image recognition technology:
The database includes the sample image database and tealeaves knowledge data base that signal is connected;
The sample image database includes tealeaves title, and the sample tealeaves view data being stored under tealeaves title Collection, the sample tealeaves image data set comprise at least the characteristic vector of corresponding sample tealeaves image;
The step 2.1.3) in after feature recognition module extracts the characteristic vector of tealeaves image to be identified, feature recognition Module is carried out the characteristic vector of tealeaves image to be identified and the characteristic vector of sample tealeaves image using sample image database Search and matching;Feature recognition module is according to the tealeaves image feature vector to be identified of input and sample image number after the completion of matching The matching degree of tealeaves to be identified, feature are calculated according to the similarity (i.e. weight) of the sample tealeaves image feature vector to match in storehouse Identification module is arranged the order of corresponding tealeaves title from high to low according to matching degree, and is shown by display module Show;
After classified inquiry module inputs or selected the classification of tealeaves to be identified in the step 2.2), classified inquiry module will Tealeaves classification to be identified is scanned for and matched with the classification of tealeaves characteristic information using tealeaves knowledge data base, is matched Tealeaves title;The tealeaves title is arranged in order and shown by display module;
After name search module inputs or selected the title of tealeaves to be identified in the step 2.3), name search module will Tealeaves title to be identified is scanned for and matched using the title of tealeaves knowledge data base tealeaves characteristic information, is matched Tealeaves title, and by display module show corresponding to tealeaves characteristic information.
Further improvement as tealeaves recognition methods of the present invention based on image recognition technology:
Title of the tealeaves characteristic information including tealeaves, classification, producing region, processing technology, organoleptic quality feature are represented, with And dry tea, millet paste and tea residue figure.
Compared with prior art, technical advantage of the invention is:
1st, the present invention is directed to unknown Tea Samples, and the dry tea photo of the tealeaves need to be only gathered by image capture module, is System provides the tealeaves title and such tealeaves characteristic information for having certain matching degree with unknown sample immediately, convenient to use;
2nd, tealeaves characteristic information provided by the present invention includes such tealeaves title, represents producing region, processing technology, sense organ product Matter feature and corresponding dry tea, millet paste and tea residue picture, it is detailed that parameter describes system;
3rd, the present invention can exist in a manner of effectively solving the problems, such as existing tealeaves science popularization, by mobile phone photograph information gathering, Tealeaves feature recognition, the tealeaves knowledge data base that information independence is searched for and system is detailed, improve the practicality of tealeaves application software And interface alternation, realize that user can be made instantly available the tealeaves characteristic information of accurately certain or certain class tealeaves when in use.
Brief description of the drawings
The embodiment of the present invention is described in further detail below in conjunction with the accompanying drawings.
Fig. 1 is the module diagram of the tealeaves identifying system of the invention based on image recognition technology;
Fig. 2 is the implementing procedure figure of the tealeaves recognition methods of the invention based on image recognition technology.
Embodiment
With reference to specific embodiment, the present invention is described further, but protection scope of the present invention is not limited in This.
Embodiment 1, the tealeaves identifying system based on image recognition technology, as shown in figure 1, including mobile terminal, this implementation Mobile terminal selects the mobile phone with camera device in example, such as apple iphone7 mobile phones:Mobile terminal is provided with image capture module 1st, feature recognition module 2, image processor module 3, database 4 and display module 5;
Database 4 includes the sample image database 41 and tealeaves knowledge data base 42 that signal is connected;Image processor module 3 Including the name search module 31 and classified inquiry module 32 being connected with the signal of tealeaves knowledge data base 42;Image capture module 1st, feature recognition module 2 is connected with sample image database 41 successively signal;Image capture module 1, feature recognition module 2, name Claim search module 31 and classified inquiry module 32 be connected with the signal of display module 5 it is (clean and tidy for drawing, therefore eliminate display The line of module 5 and specific module).
Image capture module 1 is used to obtain tealeaves image to be identified, and sends it to feature recognition module 2;This implementation Image capture module 1 is the camera device of apple iphone7 mobile phones in example.
Feature recognition module 2 is used for the tealeaves image procossing to be identified acquired in image capture module 1 and analysis, and carries Take tealeaves image feature vector to be identified;The image recognition model that feature recognition module 2 uses in the present embodiment is convolutional Neural Network model;The tealeaves image to be identified gathered by image capture module 1 by feature recognition module 2 image preprocessing, Feature extraction and feature learning training, can realize automatic identification process of the computer to tealeaves.
Sample image database 41 includes tealeaves title, and the sample tealeaves view data being stored under tealeaves title Collection, the characteristic vector that sample tealeaves image data set comprises at least corresponding sample tealeaves image (preserve the sample image spy of built in advance Levy in vectorial storehouse);The characteristic vector of the tealeaves image to be identified of extraction is utilized sample image database 41 by feature recognition module 2 Match with the characteristic vector of sample tealeaves image, so as to realize that the image according to tealeaves to be identified is known to tealeaves to be identified Not.
Tealeaves knowledge data base 42 includes tealeaves characteristic information;Title of the tealeaves characteristic information including such corresponding tealeaves, Classification, represent producing region, processing technology, organoleptic quality feature, and dry tea, millet paste and tea residue figure.Name search module 31 will be defeated Enter or the title of tealeaves that selects called in tealeaves knowledge data base 42 corresponding to tealeaves characteristic information;Inquire module by classification The 32 tealeaves classifications that will be inputted or select are matched with tealeaves characteristic information;The specific matching way of above-mentioned 3 kinds of identification methods It is as follows:
Sample image database 41 is connected with the signal of feature recognition module 2, and sample image database 41 is used for stored samples Tealeaves image information (being stored in sample image storehouse) and the characteristic vector of sample tealeaves image (preserve sample image characteristic vector In storehouse);After feature recognition module 2 extracts the characteristic vector of tealeaves image to be identified, sample in sample image database 41 is utilized The characteristic vector of tealeaves image is matched, and according to weight (in images to be recognized characteristic vector and sample image database 41 The sample tealeaves image feature vector similarity-rough set to match) calculate the matching degree of sample tealeaves and tealeaves to be identified, it is special Tealeaves title is arranged and shown by display module 5 by sign identification module 2 according to the order of matching degree from high to low;
Tealeaves knowledge data base 42 is connected with name search module 31 and the classified inquiry signal of module 32, for storing tealeaves Characteristic information;
Classified inquiry module 32 is used to inputting or selecting tealeaves classification, and the classification of tealeaves to be identified is utilized into tealeaves knowledge number Scan for and match with tealeaves classification in tealeaves characteristic information according to storehouse 42, and the tealeaves title to match is passed through into display module 5 displays, so that user can carry out the study of system using tealeaves knowledge data base 42 to tealeaves.
Name search module 31 is used to inputting or selecting tealeaves title, and the title of tealeaves to be identified is utilized into tealeaves knowledge number Scan for and match with tealeaves title in tealeaves characteristic information according to storehouse 42, name search module 31 title is matched such Tealeaves characteristic information is sent to showing on display module 5, so that user can be entered using tealeaves knowledge data base 42 to tealeaves Row targetedly knowledge acquisition.
Display module 5 is used for tealeaves title acquired after display is matched by above-mentioned tealeaves image and classification, and Such acquired tealeaves characteristic information after above-mentioned tealeaves name-matches success;Display module 5 is additionally operable to display image collection mould The tealeaves image information to be identified that block 1 is gathered, and the tealeaves that name search module 31 and classified inquiry module 32 are inputted Title and classification information.Display module 5 is the display device of apple iphone7 mobile phones in the present embodiment.
The tealeaves recognition methods carried out by said system, as shown in Figure 1-2, user can input to be identified to mobile terminal Tealeaves information is inquired about, and tealeaves information to be identified includes the tealeaves image of the collection of image capture module 1, by image processor module The tealeaves title that name search module 31 is inputted or selected in 3, and it is defeated in module 32 by being inquired by classification in image processor module 3 The tealeaves classification for entering or selecting.Mobile terminal selects the mobile phone with camera device in the present embodiment, such as apple iphone7 hands Machine.
The realization of the above needs number of the structure comprising sample image database 41 and tealeaves knowledge data base 42 in advance According to storehouse 4, and for the sample image construction feature identification module 2 in sample image database 41, so as to obtain final identification Model, feature recognition module 2 uses convolutional neural networks model in the present embodiment.
Sample image database 41 includes tealeaves title, and the sample tealeaves view data being stored under tealeaves title Collection, sample tealeaves image data set comprise at least sample image characteristic vector;Sample tealeaves image data set bag in the present embodiment Include the sample image storehouse containing sample tealeaves image information and the sample image containing characteristic vector corresponding to sample tealeaves image Characteristic vector storehouse.
The structure in sample image storehouse:Choose six big teas it is all kinds of in represent tea sample (including oolong tea, white tea, green tea, red Tea, yellow tea and black tea), Tea Samples every kind of first fully mix, and therefrom randomly select 10g or so, evenly laid out on blank sheet of paper, IMAQ is carried out using photographing function of mobile phone, collection environment meets《Tealeaves organoleptic evaluation method》National standard (GB/ T23776-2009 the environmental condition of defined in), shooting height are 15cm or so directly over tealeaves, each Tea Samples collection Image >=200.The sample tealeaves view data that above-mentioned gathered sample image is stored under corresponding tealeaves title is concentrated.
Sample image characteristic vector storehouse is built:To each sample tealeaves image in the sample image storehouse that collects by making an uproar Point filter carries out noise abatement pretreatment, and pretreated image repeatedly carries out extraction image local feature, power by convolution filter Value shares and convolutional layer pond, so as to realize extraction characteristics of image, links all characteristics of image using full articulamentum afterwards and obtains Image feature vector.Image feature vector obtained by above-mentioned is stored in the sample tealeaves image data set under corresponding tealeaves title In.
The structure of tealeaves knowledge data base 42:For the Tea Samples of above-mentioned IMAQ, organoleptic feature Information Number is carried out According to collection.Tealeaves sensory review comments tea teacher by 5 with national authentication qualification, according to《Tealeaves organoleptic evaluation method》Country's mark Defined evaluates method to the profile of Tea Samples, soup look, fragrance, flavour, tea residue " five in accurate (GB/T23776-2009) The factor " carries out typical quality feature description respectively, and carries out IMAQ to dry tea, millet paste and tea residue, so as to obtain by corresponding The title of such tealeaves, classification, producing region, processing technology, organoleptic quality feature are represented, and dry tea, millet paste and tea residue figure are formed Tealeaves characteristic information.
The structure of feature recognition module 2:Figure is used for the dry tea picture gathered in above-mentioned sample image database 41 As structure, model training and the optimization of recognizer progress image recognition model, model is identified.The present invention is by having increased income Image recognition model (such as convolutional neural networks model) construction feature identification module 2, specially first to collecting All tealeaves images carry out noise filter and carry out noise abatement pretreatment in database, and pretreated image is more by convolution filter It is secondary to carry out extraction image local feature (local receptive fields), weights shared (shared weights) and convolution Layer pond (pooling), so as to extract characteristics of image, link all characteristics of image using full articulamentum afterwards and obtain characteristics of image Vector, image feature vector are stored in sample image database 41.Opened using such as Caffe, TensorFlow, Theano Depth learning tool bag builds convolutional neural networks model;Using the calculation such as batch stochastic gradient descent algorithm (MSGD) Method, the image data set stored with image data base 41 trains the model, and carries out algorithm optimization, obtains final identification Model.
Inquired about by inputting tealeaves image to mobile terminal:
1.1) tealeaves image to be identified, is gathered by image capture module 1;
Image capture module 1 is the camera device of apple iphone7 mobile phones in the present embodiment, and user passes through to tea to be identified Leaf is taken pictures, and carries out the collection of instant tealeaves figure, or selection band identifies the photo of tealeaves from photograph album, and by above-mentioned tealeaves Photo is transmitted to feature recognition module 2 and is identified.
2.1), by the tealeaves image to be identified obtained by step 1.1) send to feature recognition module 2 carry out image procossing and Analysis, extract the characteristic parameter of tealeaves image to be identified;The characteristic parameter of the tealeaves image to be identified of gained is utilized into sample graph As database 41 with the characteristic vector of sample tealeaves image is scanned for and matched, the tealeaves title to be matched;
Features described above identification module 2 carries out the structure of image recognition model, model training and excellent using image recognition algorithm Change, be identified model.Feature recognition module 2 is using the image recognition model increased income in the prior art, tool in the present embodiment Body uses convolutional neural networks model;Convolutional neural networks model comprises the following steps that to image procossing and analysis:
2.1.1) pre-process:Noise filter is passed through to the tealeaves image to be identified received by convolutional neural networks model Carry out noise abatement pretreatment;
2.1.2 image features) are extracted, the image features include the global information of whole image;
Image features are obtained by multilayer convolution algorithm, the tealeaves figure to be identified of gained after specially above-mentioned pretreatment As repeatedly carrying out that extraction image local feature, weights are shared by convolution filter and convolutional layer pond etc. operates, so as to realize Characteristics of image is extracted, linking all characteristics of image using full articulamentum afterwards obtains image feature vector.
2.1.3) by step 2.1.2) in using the image feature vector of multilayer convolution algorithm extraction by having built and excellent Prediction is trained (for conventional skill in this area based on sample image database 41 in the convolutional neural networks identification model changed Art, therefore be not described in detail), according to the tealeaves image feature vector to be identified of input with matching in sample image database 41 The ratio between image feature vector similarity (i.e. weight) calculate the matching degree of tealeaves to be identified.
3.1), convolutional neural networks model is by step 2.1.3) in match with the characteristic parameter of tealeaves image to be identified Tealeaves title is arranged according to the order of matching degree from high to low and shown by display module 5;When user selects tealeaves title, name Claim search module 31 to recall corresponding tealeaves characteristic information in tealeaves knowledge data base 42 and show it by display module 5.
Display module 5 is the display device of apple iphone7 mobile phones in the present embodiment, and the feature of tealeaves image to be identified is joined The tealeaves title corresponding to sample tealeaves image that number matches, according to sample image and the characteristic parameter of tealeaves image to be identified Matching degree order from high to low be shown on display module 5;When user selects to be checked after the tealeaves title, title is searched Rope module 31 recalls the characteristic information of such tealeaves corresponding with the tealeaves title from tealeaves knowledge data base 42, and passes through display Module 5 is shown;Tealeaves characteristic information includes, tealeaves title, classification, the place of production, processing technology, organoleptic quality feature (profile, soup Color, fragrance, flavour, the description of tea residue feature) and the description such as dry tea, millet paste and tea residue image;User not only can be by checking this The characteristic information of class tealeaves determines whether tealeaves to be identified, additionally it is possible to according to the study of the characteristic information of tealeaves and decorrelation tea Leaf information, meet consumer's demand different with fan.
Inquired about by inputting tealeaves classification to mobile terminal, realize the function of custom system study tealeaves knowledge, Specifically comprise the following steps:
1.2), user inputs or selected tealeaves classification to be identified by inquiring module 32 by classification;
2.2), inquire by classification module 32 by the tealeaves classification to be identified in step 1.2) using tealeaves knowledge data base 42 with Tealeaves classification is scanned for and matched in tealeaves characteristic information, is inquired module 32 by classification and is being shown the tealeaves title to match Shown in module 5;By the tealeaves that tealeaves knowledge data base 42 is included being carried out respectively according to tealeaves classification in the present embodiment Sort out, therefore can realize according to the function that tealeaves classification is checked step by step on the interface of selection sort enquiry module 32.
3.2), the tealeaves title of the same tealeaves classification of step 1.2) is arranged in order and exported to aobvious on display module 5 Show, when selecting tealeaves title, name search module 31 recalls corresponding tealeaves characteristic information in tealeaves knowledge data base 42 And it is shown by display module 5.
By being inputted to mobile terminal or selecting tealeaves title to be inquired about, realize user and be directed to inquiry learning tealeaves knowledge Function, specifically comprise the following steps:
1.3), user is realized by name search module 31 inputs tealeaves title to be identified, or directly selects above-mentioned steps 3.1) the tealeaves title being shown in step 3.2) on display module 5 is inquired about.
2.3), tealeaves title to be identified in step 1.3) is utilized tealeaves knowledge data base 42 and tea by name search module 31 Tealeaves title is scanned for and matched in leaf characteristic information, the tealeaves title matched;
3.3), name search module 31 is by corresponding to tealeaves title to be identified matches in step 2.3) tealeaves title Such tealeaves characteristic information (tealeaves characteristic information includes tealeaves title) is shown by display module 5.
The specific workflow of embodiment 2, tealeaves identifying system based on image recognition technology;
As shown by the arrows in Figure 2, the specific workflow of the tealeaves identifying system based on image recognition technology of the invention is such as Under:
(1), user inputs tealeaves information to be identified to mobile terminal;Including following 3 kinds of situations:
(1.1) image querying:User is opened in mobile terminal and known based on image when running into the tealeaves of oneself None- identified The tealeaves identifying system of other technology, display module 5 will show " IMAQ " and " tealeaves data bank " two function keys;
" IMAQ " function key is selected, image capture module 1 is started working, and now shows ' taking pictures ' in display module 5 ' photograph album ' function key, user can select ' taking pictures ' to carry out the collection of instant tealeaves image, or select ' photograph album ' by existing Tealeaves image is selected in photo library;
(1.2) inquire by classification:" tealeaves data bank " function key is selected, automatic search module 3 is started working, and now shows mould There is search dialogue in the top of block 5, show below the dialog box tealeaves category list (including oolong tea, white tea, green tea, black tea, Yellow tea and black tea).User can input tealeaves classification by dialog box and be inquired about, and the tealeaves in tealeaves category list also may be selected Classification is inquired about, and now the classified inquiry module 32 of automatic search module 3 realizes classified inquiry work.
(1.3) name query:User inputs tealeaves title directly in dialog box, clicks on the order of ' search ' function key and searches automatically The name search module 31 of rope module realizes that name query works.
(2), the tealeaves information to be identified inputted in step (1) scanned in database 4 after treatment and Matching;Including following 3 kinds of situations;
Image capture module 1 obtains the success of tealeaves image and shown after in display module 5 in step (1.1), user Ke Gen Tealeaves image is adjusted according to prompting, image capture module 1 passes tealeaves image after clicking on the ACK button on display module 5 Transport to feature recognition module 2, through feature recognition module 2 by the image features extracted and with sample image database 41 Carry out information matches.Now display the tealeaves title that the match is successful and Corresponding matching degree on display module 5, the tealeaves title according to Matching degree is arranged in order from top to bottom.
Tealeaves classification selected in step (1.2) is carried out by inquiring module 32 by classification in tealeaves knowledge data base 42 Information matches, the tealeaves title of corresponding tealeaves classification is now shown on display module 5;
The tealeaves title that step (1.3) is inputted carries out letter by name search module 31 in tealeaves knowledge data base 42 Breath matching, such tealeaves characteristic information of corresponding tealeaves is now shown on display module 5;
The tealeaves title that user may be selected to be shown on display module 5 in step (1.1) or step (1.2) is inquired about; After user selects tealeaves title, such tealeaves characteristic information as shown corresponding tealeaves in step (1.3) on display module 5;
User can click on above-mentioned display module 5 successively and match tealeaves title, and now display module 5 will show such tealeaves Corresponding characteristic information describes interface, facilitates user to understand the tealeaves in detail;Find that matching result has deviation in user When, i.e., when tealeaves to be identified and shown matching degree highest tealeaves information are inconsistent, user can return to tealeaves name list In other tealeaves are checked, so as to be compared in detail and finally be determined.
Finally, it is also necessary to it is noted that listed above is only several specific embodiments of the invention.Obviously, this hair It is bright to be not limited to above example, there can also be many deformations.One of ordinary skill in the art can be from present disclosure All deformations for directly exporting or associating, are considered as protection scope of the present invention.

Claims (10)

1. the tealeaves identifying system based on image recognition technology, including mobile terminal;It is characterized in that:
The mobile terminal includes image capture module (1), feature recognition module (2), image processor module (3), database (4) With display module (5);Described image acquisition module (1), feature recognition module (2) are connected with database (4) successively signal;It is described Image capture module (1), feature recognition module (2) and image processor module (3) are connected with display module (5) signal, wherein Image processor module (3) is connected with database (4) signal;
Described image acquisition module (1) is used to obtain tealeaves image to be identified, and tealeaves image to be identified is sent to feature and known Other module (2);
The feature recognition module (2) be used for by the tealeaves image to be identified acquired in image capture module (1) carry out processing and Analysis, and extract the characteristic vector of the tealeaves image to be identified;The feature recognition module (2) uses image recognition model;
The image processor module (3) is used for the title or classification for inputting tealeaves to be identified, and is scanned for using database (4) And matching;
The database (4) is used to store tealeaves sample image data collection and tealeaves characteristic information;The feature recognition module (2) By the tealeaves image feature vector to be identified of extraction using database (4) tealeaves sample image data concentrate scan for and Match somebody with somebody, feature recognition module (2) by tealeaves title corresponding to the sample tealeaves image to match according to matching degree from high to low Order arrangement is shown by display module (5);
The image processor module (3) will input or the tealeaves classification of selection utilizes database (4) and tea in tealeaves characteristic information Leaf classification is matched, and the tealeaves title to match corresponding to tealeaves classification is exported to display module (5);
The image processor module (3) will input or the tealeaves title of selection utilizes database (4) and tea in tealeaves characteristic information Leaf title is matched, and the tealeaves characteristic information corresponding to the tealeaves title to match is exported to display module (5);
The display module (5) is used for the tealeaves image information to be identified of display image acquisition module (1) collection, and shows by spy Levy the tealeaves title that identification module (2) is sent;The display module (5) is additionally operable to show the tea of image processor module (3) input Leaf title or classification information, and show the tealeaves title sent by image processor module (3) or tealeaves characteristic information.
2. the tealeaves identifying system according to claim 1 based on image recognition technology, it is characterised in that:
Described image identification model is convolutional neural networks model;
The database (4) includes the sample image database (41) and tealeaves knowledge data base (42) that signal is connected;
The image processor module (3) include the name search module (31) that is connected with tealeaves knowledge data base (42) signal and Inquire module (32) by classification;
The sample image database (41) is connected with feature recognition module (2) signal, and sample image database (41) is used to store up Tealeaves title is deposited, and the sample tealeaves image data set being stored under tealeaves title, the sample tealeaves image data set is extremely Few characteristic vector for including corresponding sample tealeaves image;The feature recognition module (2) is by the tealeaves image to be identified of extraction Characteristic vector is matched in sample image database (41);
The tealeaves knowledge data base (42) is connected with name search module (31) and classified inquiry module (32) signal respectively, tea Leaf knowledge data base (42) is used to store tealeaves characteristic information, and tealeaves characteristic information comprises at least tealeaves title and tealeaves classification; The tealeaves classification of classified inquiry module (32) input or selection and tealeaves classification progress in tealeaves knowledge data base (42) Match somebody with somebody, classified inquiry module (32) is shown the successful tealeaves title of tealeaves categorical match by display module (5);The name The tealeaves title of search module (31) input or selection is claimed to match with the tealeaves title in tealeaves knowledge data base (42), title Search module (31) enters the tealeaves characteristic information corresponding to the successful tealeaves title of tealeaves categorical match by display module (5) Row display.
3. the tealeaves recognition methods that system as claimed in claim 1 or 2 is carried out, it is characterised in that follow the steps below successively:
1), the feature recognition module (2) or image processor module (3) are sharp after treatment by the tealeaves information to be identified of input Scanned for and matched with database (4);
2), feature recognition module (2) or image processor module (3) pass through the tealeaves title searched in step 1) and match to obtain Display module (5) is shown.
4. the tealeaves recognition methods according to claim 3 based on image recognition technology, it is characterised in that:
Described tealeaves information to be identified is included by the tealeaves image of image capture module (1) collection and by image processor module (3) the tealeaves title and tealeaves classification for inputting or selecting.
5. the tealeaves recognition methods according to claim 4 based on image recognition technology, it is characterised in that:
When the tealeaves information to be identified is tealeaves image, follow the steps below successively:
1.1) tealeaves image to be identified, is gathered by image capture module (1);
2.1), the tealeaves image to be identified described in step 1.1) is sent to feature recognition module (2) by image capture module (1) Row image procossing and analysis, feature recognition module (2) extract the characteristic parameter of the tealeaves image to be identified;Feature recognition module (2) characteristic parameter of the tealeaves image to be identified of gained is carried out using the characteristic vector of database (4) and sample tealeaves image Search and matching, the tealeaves title and matching degree to be matched;
3.1), the tealeaves name that feature recognition module (2) will match in step 2.1) with the characteristic parameter of tealeaves image to be identified Claim according to matching degree being arranged sequentially on display module (5) and show from high to low;When user selects tealeaves title, image processor Module (3) recalls such tealeaves characteristic information of corresponding tealeaves title from database and shown by display module (5).
6. the tealeaves recognition methods according to claim 5 based on image recognition technology, it is characterised in that:
The feature recognition module (2) uses convolutional neural networks model;
The concrete operation step of the step 2.1) is as follows:
2.1.1) pre-process:Feature recognition module (2) carries out noise abatement to the tealeaves image to be identified received by noise filter Pretreatment;
2.1.2) extraction image features:Described image characteristic parameter is obtained by multilayer convolution algorithm, comprising entirely waiting to know The global information of other tealeaves image;Concretely comprise the following steps:
The tealeaves image to be identified of gained after above-mentioned pretreatment is carried out extraction figure by feature recognition module (2) by convolution filter As local feature, weights are shared and convolutional layer pondization extracts characteristics of image at least twice, link institute using full articulamentum afterwards There is characteristics of image to obtain image feature vector;
2.1.3) by step 2.1.2) in using the characteristic vector of multilayer convolution algorithm extraction pass through the feature recognition mould that has built Simultaneously prediction, last feature recognition module (2) root is identified based on the tealeaves sample image data collection in database (4) in block (2) The image feature vector phase to match is concentrated according to the image feature vector and the tealeaves sample image data in database (4) of input The matching degree of tealeaves to be identified is calculated like degree.
7. the tealeaves recognition methods according to claim 4 based on image recognition technology, it is characterised in that:
When tealeaves information to be identified is tealeaves classification, follow the steps below successively:
1.2) tealeaves classification to be identified, is inputted or selected by the classified inquiry module (32) in image processor module (3);
2.2), inquire module (32) by classification and the tealeaves classification to be identified that step 1.2) inputs or selects is utilized into database (4) Scan for and match with tealeaves classification in tealeaves characteristic information, obtain the tealeaves title that classification matches;
3.2) module (32), is inquired by classification by all tealeaves titles to match with tealeaves classification to be identified obtained by step 2.2) It is arranged in order, is shown by display module (5);When user selects tealeaves title, the name search mould in image processor module (3) Block (31) recalls such tealeaves characteristic information of corresponding tealeaves title from database (4) and shown by display module (5).
8. according to any described tealeaves recognition methods based on image recognition technology of claim 3-7, it is characterised in that:
Tealeaves information to be identified is tealeaves title;Follow the steps below successively:
1.3) tealeaves title to be identified, is inputted by the name search module (31) in image processor module (3), or directly selected The tealeaves title to be identified shown on display module (5);
2.3), name search module (31) is special with tealeaves using database (4) by the tealeaves title to be identified described in step 1.3) Tealeaves title is scanned for and matched in reference breath, the tealeaves title matched;
3.3), name search module (31) is right by the tealeaves title institute to match with tealeaves title to be identified obtained by step 2.2) Tealeaves characteristic information is answered to be shown by display module (5).
9. the tealeaves recognition methods according to claim 8 based on image recognition technology, it is characterised in that:
The database (4) includes the sample image database (41) and tealeaves knowledge data base (42) that signal is connected;
The sample image database (41) includes tealeaves title, and the sample tealeaves view data being stored under tealeaves title Collection, the sample tealeaves image data set comprise at least the characteristic vector of corresponding sample tealeaves image;
The step 2.1.3) in after feature recognition module (2) extracts the characteristic vector of tealeaves image to be identified, feature recognition Module (2) using sample image database (41) by the feature of the characteristic vector of tealeaves image to be identified and sample tealeaves image to Amount is scanned for and matched;After the completion of matching feature recognition module (2) according to the tealeaves image feature vector to be identified of input with The matching of the Similarity Measure tealeaves to be identified of the sample tealeaves image feature vector to match in sample image database (41) Degree, feature recognition module (2) are arranged the order of corresponding tealeaves title from high to low according to matching degree, and by aobvious Show that module (5) is shown;
After classified inquiry module (32) inputs or selected the classification of tealeaves to be identified in the step 2.2), module is inquired by classification (32) tealeaves classification to be identified is scanned for and matched using the classification of tealeaves knowledge data base (42) tealeaves characteristic information, obtained To the tealeaves title to match;The tealeaves title is arranged in order and shown by display module (5);
After name search module (31) inputs or selected the title of tealeaves to be identified in the step 2.3), name search module (31) tealeaves title to be identified is scanned for and matched using the title of tealeaves knowledge data base (42) tealeaves characteristic information, obtained To the tealeaves title to match, and pass through tealeaves characteristic information corresponding to display module (5) display.
10. the tealeaves recognition methods according to claim 9 based on image recognition technology, it is characterised in that:
Title of the tealeaves characteristic information including tealeaves, classification, represent producing region, processing technology, organoleptic quality feature, Yi Jigan Tea, millet paste and tea residue figure.
CN201710821319.4A 2017-09-13 2017-09-13 Tealeaves recognition methods and its system based on image recognition technology Pending CN107633038A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710821319.4A CN107633038A (en) 2017-09-13 2017-09-13 Tealeaves recognition methods and its system based on image recognition technology

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710821319.4A CN107633038A (en) 2017-09-13 2017-09-13 Tealeaves recognition methods and its system based on image recognition technology

Publications (1)

Publication Number Publication Date
CN107633038A true CN107633038A (en) 2018-01-26

Family

ID=61101125

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710821319.4A Pending CN107633038A (en) 2017-09-13 2017-09-13 Tealeaves recognition methods and its system based on image recognition technology

Country Status (1)

Country Link
CN (1) CN107633038A (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109117829A (en) * 2018-09-10 2019-01-01 安徽信息工程学院 Local tea variety identifying system based on tensorflow
CN109446985A (en) * 2018-10-28 2019-03-08 贵州师范学院 Multi-angle plants identification method based on vector neural network
CN109738438A (en) * 2018-12-29 2019-05-10 扬州大学 A kind of Wheat Leaves Senescence degree method for fast measuring
CN110244009A (en) * 2019-06-18 2019-09-17 六盘水市食品药品检验检测所 A kind of quick identification detection system and method for honey quality
CN110532441A (en) * 2019-08-23 2019-12-03 广州医科大学 A kind of electronic component wisdom management method and its system
CN111414971A (en) * 2020-03-27 2020-07-14 南京工业大学 Finished product tea type and grade identification method based on convolutional neural network
CN111568195A (en) * 2020-02-29 2020-08-25 佛山市云米电器科技有限公司 Brewed beverage identification method, device and computer-readable storage medium
CN111949815A (en) * 2020-08-17 2020-11-17 西南石油大学 Artificial intelligent corrosion material selection system based on image recognition technology and working method
CN112149630A (en) * 2020-10-20 2020-12-29 云南省林业和草原科学院 Wood microscopic identification system

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102521564A (en) * 2011-11-22 2012-06-27 常熟市董浜镇华进电器厂 Method for identifying tea leaves based on colors and shapes
CN103902996A (en) * 2014-03-31 2014-07-02 合肥晶奇电子科技有限公司 Mobile phone APP designing method for recognizing diversification plants
EP3038013A1 (en) * 2014-12-23 2016-06-29 Ricoh Company, Ltd. Distinguishing between stock keeping units using hough voting methodology
CN106096563A (en) * 2016-06-17 2016-11-09 深圳市易特科信息技术有限公司 Plant automatic recognition system and method
CN106599925A (en) * 2016-12-19 2017-04-26 广东技术师范学院 Plant leaf identification system and method based on deep learning

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102521564A (en) * 2011-11-22 2012-06-27 常熟市董浜镇华进电器厂 Method for identifying tea leaves based on colors and shapes
CN103902996A (en) * 2014-03-31 2014-07-02 合肥晶奇电子科技有限公司 Mobile phone APP designing method for recognizing diversification plants
EP3038013A1 (en) * 2014-12-23 2016-06-29 Ricoh Company, Ltd. Distinguishing between stock keeping units using hough voting methodology
CN106096563A (en) * 2016-06-17 2016-11-09 深圳市易特科信息技术有限公司 Plant automatic recognition system and method
CN106599925A (en) * 2016-12-19 2017-04-26 广东技术师范学院 Plant leaf identification system and method based on deep learning

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109117829A (en) * 2018-09-10 2019-01-01 安徽信息工程学院 Local tea variety identifying system based on tensorflow
CN109446985A (en) * 2018-10-28 2019-03-08 贵州师范学院 Multi-angle plants identification method based on vector neural network
CN109446985B (en) * 2018-10-28 2021-06-04 贵州师范学院 Multi-angle plant identification method based on vector neural network
CN109738438A (en) * 2018-12-29 2019-05-10 扬州大学 A kind of Wheat Leaves Senescence degree method for fast measuring
CN110244009A (en) * 2019-06-18 2019-09-17 六盘水市食品药品检验检测所 A kind of quick identification detection system and method for honey quality
CN110532441A (en) * 2019-08-23 2019-12-03 广州医科大学 A kind of electronic component wisdom management method and its system
CN111568195A (en) * 2020-02-29 2020-08-25 佛山市云米电器科技有限公司 Brewed beverage identification method, device and computer-readable storage medium
CN111414971A (en) * 2020-03-27 2020-07-14 南京工业大学 Finished product tea type and grade identification method based on convolutional neural network
CN111414971B (en) * 2020-03-27 2023-05-02 南京工业大学 Convolutional neural network-based finished tea variety and grade identification method
CN111949815A (en) * 2020-08-17 2020-11-17 西南石油大学 Artificial intelligent corrosion material selection system based on image recognition technology and working method
CN112149630A (en) * 2020-10-20 2020-12-29 云南省林业和草原科学院 Wood microscopic identification system

Similar Documents

Publication Publication Date Title
CN107633038A (en) Tealeaves recognition methods and its system based on image recognition technology
CN104992142B (en) A kind of pedestrian recognition method being combined based on deep learning and attribute study
CN108564049A (en) A kind of fast face detection recognition method based on deep learning
CN107122375A (en) The recognition methods of image subject based on characteristics of image
CN106844614A (en) A kind of floor plan functional area system for rapidly identifying
CN106599925A (en) Plant leaf identification system and method based on deep learning
CN109325395A (en) The recognition methods of image, convolutional neural networks model training method and device
CN109829467A (en) Image labeling method, electronic device and non-transient computer-readable storage medium
CN106227851A (en) Based on the image search method searched for by depth of seam division that degree of depth convolutional neural networks is end-to-end
CN109543602A (en) A kind of recognition methods again of the pedestrian based on multi-view image feature decomposition
CN106203490A (en) Based on attribute study and the image ONLINE RECOGNITION of interaction feedback, search method under a kind of Android platform
CN104915643A (en) Deep-learning-based pedestrian re-identification method
CN113076927B (en) Finger vein identification method and system based on multi-source domain migration
CN109614508A (en) A kind of image of clothing searching method based on deep learning
CN106844739B (en) Remote sensing image change information retrieval method based on neural network collaborative training
CN102682309A (en) Face feature registering method and device based on template learning
CN107169106A (en) Video retrieval method, device, storage medium and processor
CN107590491A (en) A kind of image processing method and device
CN108647702A (en) A kind of extensive food materials image classification method based on transfer learning
CN110348494A (en) A kind of human motion recognition method based on binary channels residual error neural network
CN112668486A (en) Method, device and carrier for identifying facial expressions of pre-activated residual depth separable convolutional network
CN113379504A (en) Commodity information search recommendation method, system and computer storage medium
Sosa et al. An algorithm for detection of nutritional deficiencies from digital images of coffee leaves based on descriptors and neural networks
CN110287940A (en) A kind of palm grain identification method and system based on artificial intelligence
CN116597287A (en) Remote sensing image landslide recognition method based on deep learning method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20180126

RJ01 Rejection of invention patent application after publication