WO2019127400A1 - 一种谷物识别方法、装置和计算机存储介质 - Google Patents

一种谷物识别方法、装置和计算机存储介质 Download PDF

Info

Publication number
WO2019127400A1
WO2019127400A1 PCT/CN2017/119932 CN2017119932W WO2019127400A1 WO 2019127400 A1 WO2019127400 A1 WO 2019127400A1 CN 2017119932 W CN2017119932 W CN 2017119932W WO 2019127400 A1 WO2019127400 A1 WO 2019127400A1
Authority
WO
WIPO (PCT)
Prior art keywords
image data
grain
data
identified
obtaining
Prior art date
Application number
PCT/CN2017/119932
Other languages
English (en)
French (fr)
Inventor
朱林楠
周均扬
龙永文
周宗旭
陈必东
肖群虎
Original Assignee
美的集团股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 美的集团股份有限公司 filed Critical 美的集团股份有限公司
Priority to JP2020533125A priority Critical patent/JP7073607B2/ja
Priority to EP17936231.4A priority patent/EP3715834B1/en
Priority to PCT/CN2017/119932 priority patent/WO2019127400A1/zh
Priority to CN201780092478.3A priority patent/CN111033240A/zh
Priority to KR1020207017704A priority patent/KR102333500B1/ko
Publication of WO2019127400A1 publication Critical patent/WO2019127400A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N2021/8466—Investigation of vegetal material, e.g. leaves, plants, fruits
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/469—Contour-based spatial representations, e.g. vector-coding
    • G06V10/476—Contour-based spatial representations, e.g. vector-coding using statistical shape modelling, e.g. point distribution models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/68—Food, e.g. fruit or vegetables

Definitions

  • the present invention relates to image recognition technology, and in particular to a grain identification method, apparatus and computer storage medium.
  • the rice cooker only needs to select the cooking mode, and the cooking can be automatically performed according to the cooking time corresponding to the selected cooking mode.
  • the mode interaction mode of the rice cooker only includes the rice-free selection mode and the manual selection of the rice type interaction mode. If the user does not understand the rice species, only the rice-free seed selection mode is available for selection. If a rice cooker capable of automatically identifying the rice species is proposed, and then the cooking mode is determined based on the rice type, this will provide a more convenient interaction mode for the user and enhance the user experience.
  • embodiments of the present invention provide a grain identification method, apparatus, and computer storage medium.
  • Embodiments of the present invention provide a grain identification method, the method comprising:
  • Information of the grain to be identified is determined based on the first recognition result and the second recognition result.
  • the method further includes:
  • the second image information includes second image data and corresponding tag data
  • Learning training is performed based on the data enhanced image data and corresponding tag data to obtain a recognition model.
  • the obtaining the recognition model comprises: obtaining a first recognition model
  • the obtaining the recognition model includes: obtaining a second recognition model.
  • the method before performing the feature enhancement processing on the second image data, the method further includes:
  • the performing feature enhancement processing on the second image data comprises: performing feature enhancement processing on the second image subdata.
  • the performing feature enhancement processing on the second image data to obtain feature enhanced image data includes:
  • the performing data enhancement processing on the feature enhancement image data to obtain data enhancement image data includes:
  • Performing inversion and/or rotation of the feature enhanced image data obtaining inverted image data and/or rotated image data corresponding to the feature enhanced image data, and generating a data enhanced image based on the inverted image data and/or the rotated image data data.
  • the result and the second identification result determine information of the grain to be identified, including:
  • the embodiment of the invention further provides a grain identification device, the device comprising:
  • a memory storing a computer program capable of running on a processor
  • the processor configured to: when the computer program is executed, perform: obtaining first image data including a grain to be identified; obtaining a first recognition result based on the first image data and the first recognition model, based on the first Obtaining a second recognition result, the first recognition result characterizing a type to which the grain to be identified belongs; the second recognition result characterizing a variety to which the grain to be identified belongs; based on the first The recognition result and the second recognition result determine information of the grain to be identified.
  • the processor is further configured to: when the computer program is executed, perform: obtaining a plurality of second image information; the second image information includes second image data and corresponding tag data; Performing feature enhancement processing on the second image data to obtain feature enhanced image data; performing data enhancement processing on the feature enhanced image data to obtain data enhanced image data; and performing learning training based on the data enhanced image data and corresponding tag data, Obtain a recognition model.
  • the processor further configured to, when the computer program is executed, perform: obtaining a first recognition model when the tag data characterizes a type to which the grain belongs; and when the tag data characterizes that the grain belongs When the variety is obtained, the second recognition model is obtained.
  • the processor is further configured to: when the computer program is executed, perform: identifying a brightness of the second image data, and cutting the second image data based on the brightness to obtain a second Image sub-data; the difference in brightness of the second image sub-data satisfies a preset condition; performing feature enhancement processing on the second image sub-data to obtain feature-enhanced image data.
  • the processor configured to run the computer program, performs: converting the second image data into a grayscale image, performing contrast enhancement processing on the grayscale image, and obtaining a contrast enhanced image Data; obtaining feature enhanced image data based on the contrast enhanced image data.
  • the processor configured to execute the computer program, performs: inverting and/or rotating the feature enhanced image data to obtain inverted image data corresponding to the feature enhanced image data and And/or rotating the image data to generate data enhancement image data based on the inverted image data and/or the rotated image data.
  • the processor when configured to run the computer program, performs: obtaining a first recognition result based on the first image data and the first recognition model; obtaining the first recognition result a confidence level of the category to which the cereal to be identified belongs; when the confidence level of the category to which the cereal to be identified belongs reaches the first preset condition, obtaining a second recognition result based on the first image data and the second recognition model; Determining the confidence of the variety to be identified in the second identification result; determining the information of the grain to be identified as the grain to be identified when the confidence of the variety to be identified reaches the second predetermined condition When the confidence level of the variety to be identified does not reach the second predetermined condition, the information determining the grain to be identified is the type of the grain to be identified.
  • the embodiment of the invention further provides a computer storage medium on which computer instructions are stored, wherein the instructions are executed by the processor to implement the steps of the grain identification method according to the embodiment of the invention.
  • a grain identification method, apparatus and computer storage medium provided by an embodiment of the present invention, the method comprising: obtaining first image data including a grain to be identified; obtaining a first recognition result based on the first image data and a first recognition model, Obtaining a second recognition result based on the first image data and the second recognition model, the first recognition result characterizing a type to which the grain to be identified belongs; the second recognition result characterizing a variety to which the grain to be identified belongs; Information of the grain to be identified is determined based on the first recognition result and the second recognition result.
  • the technical solution of the embodiment of the invention can automatically identify the grain by means of image recognition, and provide technical support for the cooking device to automatically set the cooking mode automatically based on the type of grain and the variety, without the user's human eye to identify the type and variety of the grain, especially to the unclear
  • the cooking process of the user of the type and variety of the grain provides convenience and greatly enhances the user experience.
  • FIG. 1 is a schematic flow chart of a method for identifying a grain according to an embodiment of the present invention
  • FIGS. 2a and 2b are schematic views showing the types of cereals in the cereal identification method according to an embodiment of the present invention.
  • FIG. 3 is a schematic flow chart of the training of the recognition model in the grain identification method according to the embodiment of the present invention.
  • FIG. 4 is another schematic flowchart of the training of the recognition model in the grain identification method according to the embodiment of the present invention.
  • 5a and 5b are respectively a first application schematic diagram of the recognition model training in the grain identification method according to the embodiment of the present invention.
  • FIG. 6 is a schematic diagram of a second application of the recognition model training in the grain identification method according to an embodiment of the present invention.
  • 7a to 7h are respectively a third application schematic diagram of the recognition model training in the grain identification method according to the embodiment of the present invention.
  • FIG. 8 is a schematic diagram of an application flow of a grain identification method according to an embodiment of the present invention.
  • Fig. 9 is a schematic view showing the structure of a grain identifying device according to an embodiment of the present invention.
  • Embodiments of the present invention provide a grain identification method.
  • 1 is a schematic flow chart of a grain identification method according to an embodiment of the present invention; as shown in FIG. 1, the method includes:
  • Step 101 Obtain first image data including a grain to be identified.
  • Step 102 Obtain a first recognition result based on the first image data and the first recognition model, and obtain a second recognition result based on the first image data and the second recognition model, the first recognition result characterizing the to-be-identified The type to which the grain belongs; the second recognition result characterizes the variety to which the grain to be identified belongs.
  • Step 103 Determine information of the grain to be identified based on the first recognition result and the second recognition result.
  • the grain identification method of the embodiment of the invention can be applied to the device; as a first embodiment, the device can be a kitchen device, and the kitchen device is provided with an image acquisition component (such as a camera), and the image data is collected by the image acquisition component, and the image is collected. The image data is analyzed and identified to determine the variety of the grain to be identified; as a second embodiment, the device may be a kitchen device, the kitchen device does not have an image capture function, and the kitchen device can communicate with another device having an image capture component.
  • an image acquisition component such as a camera
  • the kitchen device obtaining image data collected by the another device through a communication link with the another device;
  • the device may be an electronic device,
  • the electronic device may be a mobile device, such as a mobile phone, a tablet computer, etc., collecting image data through the electronic device, analyzing and identifying the collected image data, determining the variety of the grain to be identified, and further determining the cooking parameter based on the variety of the grain to be identified, Cooking parameters are sent to the kitchen equipment.
  • the first image data includes a grain to be identified, and the grain to be identified is, for example, rice.
  • the kitchen equipment may be a kitchen heating device such as a rice cooker or an electric pressure cooker.
  • the device has a cooking (eg cooking) function, ie for heating the grain contained in the device.
  • a cooking eg cooking
  • rice is used as an example, and rice varieties are diverse.
  • 2a and 2b are respectively schematic diagrams of grain types in the grain identification method according to an embodiment of the present invention; as shown in Fig. 2a, the rice can be divided into glutinous rice and glutinous rice, for example, glutinous rice and panjin rice belong to glutinous rice, and the rice and the silk are covered.
  • Miao rice belongs to glutinous rice, and it is highly similar even in different types of rice.
  • the four rice varieties are all glutinous rice, and the different varieties of glutinous rice are highly similar.
  • the rice of the same variety and different brands are not completely identical.
  • broken rice/broken rice is easy to cause misjudgment.
  • glutinous rice is relatively long, but if a large amount of glutinous rice is broken rice, the shape is closer to glutinous rice. Based on this, it is difficult for the user to identify the type of grain through the shape of the grain.
  • the apparatus of the embodiments of the present invention identifies the type of grain by image acquisition for the grain. In the following examples of the present invention, the description was made by taking rice as a rice as an example.
  • the device obtains the recognition model in advance through the learning training method.
  • FIG. 3 is a schematic flowchart of the recognition model training in the grain identification method according to the embodiment of the present invention.
  • the grain identification method further includes:
  • Step 201 Obtain a plurality of second image information; the second image information includes second image data and corresponding tag data.
  • Step 202 Perform feature enhancement processing on the second image data to obtain feature enhancement image data.
  • Step 203 Perform data enhancement processing on the feature enhancement image data to obtain data enhancement image data.
  • Step 204 Perform learning training based on the data enhanced image data and corresponding tag data to obtain a recognition model.
  • a plurality of second image information for identifying model training is obtained, the second image information includes second image data and corresponding tag data, and the tag data represents a second image data included in the corresponding image data.
  • the type of grain includes a first recognition model and a second recognition model; wherein the first recognition model is for outputting a recognition result of a grain type, and then for training the second of the first recognition model
  • the label information included in the image information is grain type information; the second recognition model is used to output the recognition result of the grain variety, and the label data included in the second image information for training the second recognition model is the grain type information.
  • the grain variety can be used as a sub-category of the grain type.
  • the grain type may be glutinous rice or glutinous rice, and the grain variety may be basmati rice under the glutinous rice type.
  • the method may further include:
  • Step 202a Identify the brightness of the second image data, and perform cropping on the second image data based on the brightness to obtain second image subdata; the brightness difference of the second image subdata satisfies a preset condition.
  • Step 202b Perform feature enhancement processing on the second image subdata to obtain feature enhancement image data.
  • FIG. 5a This embodiment can be specifically referred to FIG. 5a and FIG. 5b.
  • the left side illumination in the second image data is relatively strong, and it is not suitable to use the complete second image data as the training data.
  • the method of stretching and cropping the entire image data into a positive direction may cause the image to be deformed.
  • by identifying the brightness of the second image data, based on the pixels between the pixels The second image data is cropped by the difference in brightness, and the second image sub-data in which the difference in luminance in the second image data satisfies the preset condition is obtained. As shown in FIG.
  • the left side illumination in the second image data is strong, the left side area with a large brightness can be cropped to obtain a right square area with a small difference in luminance.
  • the image data used by the training recognition model can be unified into a region size, and the second image data can be cropped based on the set size of the region, so that the brightness difference of the second image subdata obtained after the cropping is minimized.
  • the brightness difference of the second image sub-data satisfies the preset condition, which may be: when the set area size is satisfied, the brightness difference of the second image sub-data is the smallest.
  • the obtained second image data since the color space of the image data of the rice is special, the obtained second image data has almost no color information, and even if the second image data is converted into a gray image, the morphological characteristics of the rice are not obvious. The classification effect is not good. Based on this, the second image data is enhanced in the embodiment of the present invention, and the contrast of the second image data is mainly enhanced.
  • performing feature enhancement processing on the second image data to obtain feature enhanced image data includes: converting the second image data into a grayscale image, Contrast enhancement processing is performed on the grayscale image to obtain contrast enhanced image data; and feature enhanced image data is obtained based on the contrast enhanced image data.
  • the obtained second image data is usually color data, and then the red, green and blue (RGB) three-channel color data corresponding to the second image data is first converted into a grayscale image, and the grayscale image is further subjected to a contrast enhancement algorithm.
  • the contrast represents measurement of different brightness levels between the brightest pixel point and the darkest pixel point in the image data.
  • the contrast enhancement algorithm includes, but is not limited to, at least one of the following algorithms: a linear transformation algorithm, an exponential change algorithm, a logarithmic variation algorithm, a histogram algorithm, etc., thereby enhancing contrast of image data, especially when useful data of image data
  • the contrast is quite close to the situation.
  • FIG. 6 is a schematic diagram of a second application diagram of the recognition model training in the grain identification method according to an embodiment of the present invention; as shown in FIG. 6, the contrast between different rice grains is more obvious by enhancing the contrast of the image data. It can reflect the degree of light transmission of different rice grains.
  • the data enhancement processing is performed on the feature enhanced image data corresponding to the second image data, and the data enhancement image data is obtained, including: flipping the feature enhanced image data corresponding to the second image data. And/or rotating, obtaining inverted image data and/or rotated image data corresponding to the feature enhanced image data, and generating data enhanced image data based on the inverted image data and/or the rotated image data.
  • the feature enhancement image data is rotated, and the angle of rotation may be a first preset angle, and the first preset angle is one of the following angles: 90 degrees, 180 degrees, 270 degrees; After the flipping, the flipped feature enhanced image data is further rotated, the angle of rotation may be a second preset angle, and the second preset angle is one of the following angles: 90 degrees, 180 degrees, 270 degrees.
  • the feature enhancement image data may be rotated according to the first preset angle, and the feature enhancement image data and the rotated feature enhancement image data are used as model training. Training set image.
  • the inverted feature enhanced image data is rotated according to the second preset angle, and the feature is enhanced after the image data is inverted.
  • the feature enhancement image data and the inverted and rotated feature enhancement image data are used as model training training set images.
  • the feature enhancement image data, the rotated feature enhancement image data, the inverted feature enhancement image data, and the inverted and rotated features may be combined with the first embodiment and the second embodiment described above.
  • the image data is enhanced as a training set image for model training.
  • FIG. 7a to 7h are respectively a third application schematic diagram of the recognition model training in the grain identification method according to the embodiment of the present invention.
  • the feature enhancement image data, and FIGS. 7b to 7d respectively rotate the FIG. 7a Rotating feature-enhanced image data obtained after 90 degrees, 180 degrees, and 270 degrees;
  • FIG. 7e is feature-enhanced image data in which FIG. 7a is flipped left and right;
  • FIGS. 7f to 7h are respectively rotated 90 degrees, 180 degrees, and 270 in FIG. 7e.
  • the post-flip-rotated feature obtained after the degree enhances the image data. This expands an image data into 8 image data, greatly increasing the quality of the training set without increasing the amount of data collected. It also takes into account the effects of different environments that may be faced in a real-world environment, such as lighting effects. In order to better deal with the influence of illumination and improve the generalization ability of the model, the image data of the training set is processed to different degrees.
  • the learning training is performed based on the data enhanced image data and the corresponding tag data, and the recognition model is obtained.
  • the tag data characterizes the type to which the grain belongs
  • learning training is performed based on the data enhancement image data and the corresponding tag data to obtain a first recognition model.
  • the tag data represents a variety to which the grain belongs
  • learning training is performed based on the data enhanced image data and the corresponding tag data to obtain a second recognition model.
  • the data enhancement image data and the corresponding tag data are learned and trained by using a convolutional neural network structure, and the data enhancement image data is specifically used as an input of a convolutional neural network model, and the tag data is used as a convolution.
  • the output of the neural network model is trained by the stochastic gradient descent method to obtain the recognition model.
  • the tag data representing the type to which the grain belongs is used as the output of the convolutional neural network model, and the learning training is performed by the stochastic gradient descent method to obtain the first recognition model.
  • the tag data representing the variety to which the grain belongs is used as the output of the convolutional neural network model, and the learning is trained by the stochastic gradient descent method to obtain the second recognition model.
  • the convolutional neural network structure may adopt one of the following network structures: AlexNet, VGGNet, GoogleNet, and ResNet.
  • the convolutional neural network structure is used for grain identification, which can accurately identify varieties of high similarity and fine grain size, and improve the accuracy of recognition.
  • an image data is preprocessed.
  • the preprocessing manner of the first image data includes: performing feature enhancement processing on the first image data to obtain a feature enhancement image corresponding to the first image data. data.
  • the preprocessing manner of the first image data includes: identifying a brightness of the first image data, and cutting the first image data based on the brightness to obtain a first image subdata.
  • the difference in brightness of the first image sub-data satisfies a preset condition; performing feature enhancement processing on the first image sub-data to obtain feature-enhanced image data corresponding to the first image data.
  • the obtaining the first recognition result based on the first image data and the first recognition model comprises: obtaining a first recognition result based on the feature enhanced image data corresponding to the first image data and the first recognition model.
  • the method for performing the cropping and the feature enhancement processing on the first image data may refer to the cropping of the second image data and the feature enhancement processing manner, and details are not described herein again.
  • the feature enhancement image data corresponding to the first image data is input into the first recognition model to obtain a first recognition result; the first recognition result may include a type to which the rice included in the first image data belongs.
  • the feature enhanced image data corresponding to the first image data is input into the second recognition model to obtain a second recognition result; and the second recognition result may include a variety to which the rice included in the first image data belongs.
  • the output recognition result may include a label and a corresponding probability; wherein, when the first recognition result is output, the first recognition result includes a type label and a corresponding probability; when the second recognition result is output, the The second recognition result includes the breed label and the corresponding probability.
  • the first recognition result of the output may be glutinous rice, and the corresponding probability of 89%, glutinous rice, and the corresponding probability of 11%, then the type of rice may be determined to be glutinous rice based on the first recognition result.
  • the device may include a first identification model for type identification of rice and a second recognition model for species identification of rice.
  • the first recognition result is obtained based on the first image data and the first recognition model
  • the second recognition result is obtained based on the first image data and the second recognition model, based on the first Determining the information of the grain to be identified by the recognition result and the second recognition result, comprising: obtaining a first recognition result based on the first image data and the first recognition model; obtaining the to-be-identified in the first recognition result a confidence level of the category to which the cereal belongs; when the confidence level of the category to which the cereal to be identified belongs reaches the first predetermined condition, obtaining a second recognition result based on the first image data and the second recognition model; a confidence level of the variety to be identified in the second identification result; when the confidence level of the variety to be identified reaches the second predetermined condition, determining the information of the grain to be identified is
  • the first image data is input to the first recognition model to identify the rice type, the type of the rice is determined based on the type included in the obtained first recognition result, and the corresponding probability, and the rice is determined to be Confidence of the type; when it is determined that the confidence that the rice belongs to the type satisfies the preset condition, the first image data is input to the second recognition model for identification of the rice variety, based on the varieties included in the obtained second recognition result and The corresponding probability determines the variety to which the rice belongs, and determines the confidence that the rice belongs to the variety; when it is determined that the confidence that the rice belongs to the variety meets the preset condition, the variety of the rice is output; and the confidence that the rice belongs to the variety is not determined When the preset condition is met, the type of rice is output.
  • the type of rice may be determined according to the comparison result of the probability corresponding to the obtained type; but it is likely that the grain is not rice but the form is similar to rice, so the type of rice obtained may be present, but the type corresponds to The probability is not high. Based on this, if the probability that the type of the rice belongs is lower than a preset threshold, it can be determined that the confidence that the rice belongs to the type is low, and the recognition result of the deployed rice is output.
  • the rice variety is identified to obtain the variety and the corresponding probability; Further, the variety to which the rice belongs can be determined based on the probability comparison. Among them, if the probability of rice belonging to the variety is low, it can be determined that the rice has a low degree of confidence, and the rice type is directly output; if the probability that the rice belongs to the variety is high, it can be determined that the rice belongs to the variety. If the degree is high, the rice variety will be directly exported.
  • the device when the device supports identification of a plurality of cereals (eg, rice, corn, soybeans, etc.), the device may include a first identification model for each type of grain for type identification and for the variety A second recognition model identified.
  • the first recognition result is obtained based on the first image data and the first recognition model
  • the second recognition result is obtained based on the first image data and the second recognition model, based on the first Determining the information of the grain to be identified by the recognition result and the second recognition result, comprising: obtaining a first recognition result based on the first image data and the first recognition model; obtaining the to-be-identified in the first recognition result a confidence level of the first category of the first grain to which the grain belongs; when the confidence level of the first category of the first grain to which the grain to be identified belongs reaches a first predetermined condition, based on the first image data and the first Obtaining a second recognition result corresponding to the second identification model of the grain, obtaining a confidence level of the first variety of
  • the embodiment of the invention determines the information of the grain to be identified by using the result fusion of the two recognition models, and can improve the robustness of the recognition result.
  • This embodiment corresponds to a scenario in which the device is capable of recognizing a plurality of cereals. It can be understood that the device includes two identification models corresponding to the plurality of grains respectively. The device may initially identify the grain to be identified in the first image data as the first grain based on the image recognition manner; and further determine the information of the first grain by using the first recognition model and the second recognition model corresponding to the first grain. The specific determination of the information of the first grain can be determined by referring to the above-mentioned method of determining the grain as rice, and details are not described herein again.
  • the method further comprises selecting an operating mode based on the information of the grain to be identified, and heating the grain to be identified based on the operating mode.
  • the device may select an operation mode based on information such as the type, variety, and the like of the grain to be identified, the operation mode having matching heating parameters; and then the device heats the grain to be identified based on the heating parameter corresponding to the operation mode.
  • the technical solution of the embodiment of the invention can automatically identify the grain by means of image recognition, and provide technical support for the cooking device to automatically set the cooking mode automatically based on the type of grain and the variety, without the user's human eye to identify the type and variety of the grain, especially to the unclear
  • the cooking process of the user of the type and variety of the grain provides convenience and greatly enhances the user experience.
  • Embodiments of the present invention also provide a grain identification device.
  • Figure 9 is a block diagram showing the structure of a grain identifying apparatus according to an embodiment of the present invention.
  • the apparatus includes: a memory 32 storing a computer program executable on the processor 31; the processor 31, When the computer program is run, performing: obtaining first image data including a grain to be identified; obtaining a first recognition result based on the first image data and the first recognition model, based on the first image data and the second recognition The model obtains a second recognition result, the first recognition result characterizing a type to which the grain to be identified belongs; the second recognition result characterizing a variety to which the grain to be identified belongs; based on the first recognition result and the first The second recognition result determines information of the grain to be identified.
  • the processor 31 is further configured to: when the computer program is executed, obtain: obtain a plurality of second image information; the second image information includes second image data and corresponding tag data; Performing feature enhancement processing on the second image data to obtain feature enhanced image data; performing data enhancement processing on the feature enhanced image data to obtain data enhanced image data; and performing learning training based on the data enhanced image data and corresponding tag data , get the recognition model.
  • the processor 31 is further configured to: when the computer program is executed, perform: when the tag data represents a type to which the grain belongs, obtain a first recognition model; and when the tag data represents a grain When the variety is obtained, the second recognition model is obtained.
  • the processor 31 is further configured to: when the computer program is executed, perform: identifying a brightness of the second image data, and cutting the second image data based on the brightness to obtain a first Two image sub-data; the brightness difference of the second image sub-data satisfies a preset condition; performing feature enhancement processing on the second image sub-data to obtain feature-enhanced image data.
  • the processor 31 is configured to: when the computer program is executed, convert the second image data into a grayscale image, perform contrast enhancement processing on the grayscale image, and obtain contrast enhancement. Image data; obtaining feature enhanced image data based on the contrast enhanced image data.
  • the processor 31 is configured to: when the computer program is executed, perform: flipping and/or rotating the feature enhanced image data to obtain inverted image data corresponding to the feature enhanced image data. And/or rotating the image data, generating data enhancement image data based on the inverted image data and/or the rotated image data.
  • the processor 31 is configured to: when the computer program is executed, obtain a first recognition result based on the first image data and the first recognition model; and obtain the first recognition result Determining the confidence level of the category to which the cereal belongs; when the confidence level of the category to which the cereal to be identified belongs reaches the first preset condition, obtaining the second recognition result based on the first image data and the second recognition model; a confidence level of the variety to be identified in the second identification result; determining the information of the grain to be identified as the to-be-identified when the confidence level of the item to be identified belongs to the second preset condition
  • the grain belongs to the variety; when the confidence level of the variety to which the grain to be identified belongs does not reach the second preset condition, the information determining the grain to be identified is the type of the grain to be identified.
  • the grain identification device provided in the above embodiment is only exemplified by the division of each of the above-mentioned program modules when performing grain identification. In actual application, the above-mentioned process allocation can be completed by different program modules as needed. The internal structure of the device is divided into different program modules to complete all or part of the processing described above.
  • the grain identification device provided by the above embodiment is the same as the embodiment of the grain identification method, and the specific implementation process is described in detail in the method embodiment, and details are not described herein again.
  • the apparatus also includes a bus system 33, the various components of which are coupled together by a bus system 33. It will be appreciated that the bus system 33 is used to implement connection communication between these components.
  • the bus system 33 includes a power bus, a control bus, and a status signal bus in addition to the data bus. However, for clarity of description, various buses are labeled as the bus system 33 in FIG.
  • memory 32 can be either volatile memory or non-volatile memory, and can include both volatile and nonvolatile memory.
  • the non-volatile memory may be a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), or an Erasable Programmable Read (EPROM). Only Memory), Electrically Erasable Programmable Read-Only Memory (EEPROM), Ferromagnetic Random Access Memory (FRAM), Flash Memory, Magnetic Surface Memory , CD-ROM, or Compact Disc Read-Only Memory (CD-ROM); the magnetic surface memory can be a disk storage or a tape storage.
  • the volatile memory can be a random access memory (RAM) that acts as an external cache.
  • RAM Random Access Memory
  • SRAM Static Random Access Memory
  • SSRAM Synchronous Static Random Access Memory
  • SSRAM Dynamic Random Access
  • DRAM Dynamic Random Access Memory
  • SDRAM Synchronous Dynamic Random Access Memory
  • DDRSDRAM Double Data Rate Synchronous Dynamic Random Access Memory
  • ESDRAM enhancement Enhanced Synchronous Dynamic Random Access Memory
  • SLDRAM Synchronous Dynamic Random Access Memory
  • DRRAM Direct Memory Bus Random Access Memory
  • Processor 31 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method may be completed by an integrated logic circuit of hardware in the processor 31 or an instruction in a form of software.
  • the processor 31 described above may be a general purpose processor, a digital signal processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, or the like.
  • DSP digital signal processor
  • the processor 31 can implement or perform the various methods, steps, and logic blocks disclosed in the embodiments of the present invention.
  • a general purpose processor can be a microprocessor or any conventional processor or the like.
  • the steps of the method disclosed in the embodiment of the present invention may be directly implemented as a hardware decoding processor, or may be performed by a combination of hardware and software modules in the decoding processor.
  • the software module can reside in a storage medium located in memory 32, and processor 31 reads the information in memory 32 and, in conjunction with its hardware, performs the steps of the foregoing method.
  • the grain identification device may be configured by one or more Application Specific Integrated Circuits (ASICs), DSPs, Programmable Logic Devices (PLDs), and Complex Programmable Logic Devices (CPLDs). , Complex Programmable Logic Device), Field-Programmable Gate Array (FPGA), General Purpose Processor, Controller, Micro Controller Unit (MCU), Microprocessor, or other electronics Element implementation for performing the aforementioned method.
  • ASICs Application Specific Integrated Circuits
  • DSPs Programmable Logic Devices
  • PLDs Programmable Logic Devices
  • CPLDs Complex Programmable Logic Device
  • FPGA Field-Programmable Gate Array
  • MCU Micro Controller Unit
  • Microprocessor or other electronics Element implementation for performing the aforementioned method.
  • embodiments of the present invention also provide a computer readable storage medium, such as a memory 32 including a computer program executable by processor 31 of a grain identification device to perform the steps of the foregoing method .
  • the computer readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories, such as Mobile phones, computers, tablet devices, personal digital assistants, etc.
  • Embodiments of the present invention provide a computer storage medium having stored thereon computer instructions, which are executed by a processor to: obtain first image data including a grain to be identified; and based on the first image data and first identification
  • the model obtains a first recognition result, and obtains a second recognition result based on the first image data and the second recognition model, the first recognition result characterizing a type to which the grain to be identified belongs; the second recognition result characterizing the The variety to which the grain to be identified belongs; determining the information of the grain to be identified based on the first recognition result and the second recognition result.
  • the instruction is executed by the processor to: obtain a plurality of second image information; the second image information includes second image data and corresponding tag data; and feature enhancement of the second image data Processing, obtaining feature enhanced image data; performing data enhancement processing on the feature enhanced image data to obtain data enhanced image data; performing learning training based on the data enhanced image data and corresponding tag data to obtain a recognition model.
  • the instructions are executed by the processor to: obtain a first recognition model when the tag data characterizes the type to which the grain belongs; and obtain a second recognition model when the tag data characterizes the species to which the grain belongs .
  • the brightness of the second image data is recognized, the second image data is cropped based on the brightness, and the second image subdata is obtained; The difference in brightness of the image sub-data satisfies a preset condition; performing feature enhancement processing on the second image sub-data to obtain feature-enhanced image data.
  • the instruction is executed by the processor: converting the second image data into a grayscale image, performing contrast enhancement processing on the grayscale image, obtaining contrast enhanced image data; and enhancing the contrast based on the contrast
  • the image data obtains feature enhancement image data.
  • the instruction is executed by the processor to: flip and/or rotate the feature enhanced image data to obtain inverted image data and/or rotated image data corresponding to the feature enhanced image data, based on The flip image data and/or the rotated image data generate data enhancement image data.
  • the first recognition result is obtained based on the first image data and the first recognition model; and the confidence of the category of the cereal to be identified in the first recognition result is obtained.
  • the disclosed apparatus and method may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the unit is only a logical function division.
  • there may be another division manner such as: multiple units or components may be combined, or Can be integrated into another system, or some features can be ignored or not executed.
  • the coupling, or direct coupling, or communication connection of the components shown or discussed may be indirect coupling or communication connection through some interfaces, devices or units, and may be electrical, mechanical or other forms. of.
  • the units described above as separate components may or may not be physically separated, and the components displayed as the unit may or may not be physical units, that is, may be located in one place or distributed to multiple network units; Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may be separately used as one unit, or two or more units may be integrated into one unit;
  • the unit can be implemented in the form of hardware or in the form of hardware plus software functional units.
  • the foregoing program may be stored in a computer readable storage medium, and the program is executed when executed.
  • the foregoing steps include the steps of the foregoing method embodiments; and the foregoing storage medium includes: a removable storage device, a ROM, a RAM, a magnetic disk, or an optical disk, and the like, which can store program codes.
  • the above-described integrated unit of the present invention may be stored in a computer readable storage medium if it is implemented in the form of a software function module and sold or used as a standalone product.
  • the technical solution of the embodiments of the present invention may be embodied in the form of a software product in essence or in the form of a software product stored in a storage medium, including a plurality of instructions.
  • a computer device (which may be a personal computer, server, or network device, etc.) is caused to perform all or part of the methods described in various embodiments of the present invention.
  • the foregoing storage medium includes various media that can store program codes, such as a mobile storage device, a ROM, a RAM, a magnetic disk, or an optical disk.

Abstract

一种谷物识别方法、装置和计算机存储介质。所述方法包括：获得包括待识别谷物的第一图像数据（101）；基于所述第一图像数据和第一识别模型获得第一识别结果，基于所述第一图像数据和第二识别模型获得第二识别结果（102），所述第一识别结果表征所述待识别谷物属于的类型；所述第二识别结果表征所述待识别谷物属于的品种；基于所述第一识别结果和所述第二识别结果确定所述待识别谷物的信息（103）。

Description

一种谷物识别方法、装置和计算机存储介质 技术领域
本发明涉及图像识别技术，具体涉及一种谷物识别方法、装置和计算机存储介质。
背景技术
智能家电的出现给人们的日常生活带来的极大的便利。如电饭煲，用户仅需要选择烹饪模式，便可根据所选择的烹饪模式对应的烹饪时间自动进行烹饪。然而，对于米种的选择方式，电饭煲的模式交互方式仅包括无米种选择模式和手动选择米种交互方式，若用户并不了解米种的情况下，只有无米种选择模式可供选择。如果提出一种能够自动识别米种的电饭煲，进而基于米种确定烹饪模式，这样会给用户提供更加便利的交互方式，提升用户的体验。然而，现有技术中，目前尚无有效解决方案。
发明内容
为解决现有存在的技术问题，本发明实施例提供一种谷物识别方法、装置和计算机存储介质。
为达到上述目的，本发明实施例的技术方案是这样实现的：
本发明实施例提供了一种谷物识别方法，所述方法包括：
获得包括待识别谷物的第一图像数据；
基于所述第一图像数据和第一识别模型获得第一识别结果，基于所述第一图像数据和第二识别模型获得第二识别结果，所述第一识别结果表征所述待识别谷物属于的类型；所述第二识别结果表征所述待识别谷物属于的品种；
基于所述第一识别结果和所述第二识别结果确定所述待识别谷物的信息。
在一实施例中，所述方法还包括：
获得多个第二图像信息；所述第二图像信息包括第二图像数据和对应的标签数据；
对所述第二图像数据进行特征增强处理，获得特征增强图像数据；
对所述特征增强图像数据进行数据增强处理，获得数据增强图像数据；
基于所述数据增强图像数据和对应的标签数据进行学习训练，获得识别模型。
在一实施例中，当所述标签数据表征谷物属于的类型时，所述获得识别模型，包括：获得第一识别模型；
当所述标签数据表征谷物属于的品种时，所述获得识别模型，包括：获得第二识别模型。
在一实施例中，所述对所述第二图像数据进行特征增强处理之前，所述方法还包括：
识别所述第二图像数据的亮度，基于所述亮度对所述第二图像数据进行裁剪，获得第二图像子数据；所述第二图像子数据的亮度差异满足预设条件；
相应的，所述对所述第二图像数据进行特征增强处理，包括：对所述第二图像子数据进行特征增强处理。
在一实施例中，所述对所述第二图像数据进行特征增强处理，获得特征增强图像数据，包括：
将所述第二图像数据转换为灰度图像，对所述灰度图像进行对比度增强处理，获得对比度增强图像数据；基于所述对比度增强图像数据获得特征增强图像数据。
在一实施例中，所述对所述特征增强图像数据进行数据增强处理，获得数据增强图像数据，包括：
对所述特征增强图像数据进行翻转和/或旋转，获得与所述特征增强图像数据对应的翻转图像数据和/或旋转图像数据，基于所述翻转图像数据和/或旋转图像数据生成数据增强图像数据。
在一实施例中，所述基于所述第一图像数据和第一识别模型获得第一识别结果，基于所述第一图像数据和第二识别模型获得第二识别结果，基于所述第一识别结果和所述第二识别结果确定所述待识别谷物的信息，包括：
基于所述第一图像数据和第一识别模型获得第一识别结果；获得所述第一识别结果中所述待识别谷物所属类别的置信度；当所述待识别谷物所属类别的置信度达到第一预设条件时，基于所述第一图像数据和所述第二识别模型获得第二识别结果；获得所述第二识别结果中所述待识别谷物所属品种的置信度；当所述待识别谷物所属品种的置信度达到第二预设条件时，确定所述待识别谷物的信息为所述待识别谷物所属品种；当所述待识别谷物所属品种的置信度未达到第二预设条件时，确定所述待识别谷物的信息为所述待识别谷物所属类型。
本发明实施例还提供了一种谷物识别装置，所述装置包括：
存储有能够在处理器上运行的计算机程序的存储器；
所述处理器，配置为运行所述计算机程序时，执行：获得包括待识别谷物的第一图像数据；基于所述第一图像数据和第一识别模型获得第一识别结果，基于所述第一图像数据和第二识别模型获得第二识别结果，所述第一识别结果表征所述待识别谷物属于的类型；所述第二识别结果表征所述待识别谷物属于的品种；基于所述第一识别结果和所述第二识别结果确定所述待识别谷物的信息。
在一实施例中，所述处理器，还配置为运行所述计算机程序时，执行：获得多个第二图像信息；所述第二图像信息包括第二图像数据和对应的标签数据；对所述第二图像数据进行特征增强处理，获得特征增强图像数据；对所述特征增强图像数据进行数据增强处理，获得数据增强图像数据；基于所述数据增强图像数据和对应的标签数据进行学习训练，获得识别模型。
在一实施例中，所述处理器，还配置为运行所述计算机程序时，执行：当所述标签数据表征谷物属于的类型时，获得第一识别模型；当所述标签数据表征谷物属于的品种时，获得第二识别模型。
在一实施例中，所述处理器，还配置为运行所述计算机程序时，执行：识别所述第二图像数据的亮度，基于所述亮度对所述第二图像数据进行裁剪，获得第二图像子数据；所述第二图像子数据的亮度差异满足预设条件；对所述第二图像子数据进行特征增强处理，获得特征增强图像数据。
在一实施例中，所述处理器，配置为运行所述计算机程序时，执行：将所述第二图像数据转换为灰度图像，对所述灰度图像进行对比度增强处理，获得对比度增强图像数据；基于所述对比度增强图像数据获得特征增强图像数据。
在一实施例中，所述处理器，配置为运行所述计算机程序时，执行：对所述特征增强图像数据进行翻转和/或旋转，获得与所述特征增强图像数据对应的翻转图像数据和/或旋转图像数据，基于所述翻转图像数据和/或旋转图像数据生成数据增强图像数据。
在一实施例中，所述处理器，配置为运行所述计算机程序时，执行：基于所述第一图像数据和第一识别模型获得第一识别结果；获得所述第一识别结果中所述待识别谷物所属类别的置信度；当所述待识别谷物所属类别的置信度达到第一预设条件时，基于所述第一图像数据和所述第二识别模型获得第二识别结果；获得所述第二识别结果中所述待识别谷物所属品 种的置信度；当所述待识别谷物所属品种的置信度达到第二预设条件时，确定所述待识别谷物的信息为所述待识别谷物所属品种；当所述待识别谷物所属品种的置信度未达到第二预设条件时，确定所述待识别谷物的信息为所述待识别谷物所属类型。
本发明实施例还提供了一种计算机存储介质，其上存储有计算机指令，其中，该指令被处理器执行时实现本发明实施例所述谷物识别方法的步骤。
本发明实施例提供的谷物识别方法、装置和计算机存储介质，所述方法包括：获得包括待识别谷物的第一图像数据；基于所述第一图像数据和第一识别模型获得第一识别结果，基于所述第一图像数据和第二识别模型获得第二识别结果，所述第一识别结果表征所述待识别谷物属于的类型；所述第二识别结果表征所述待识别谷物属于的品种；基于所述第一识别结果和所述第二识别结果确定所述待识别谷物的信息。采用本发明实施例的技术方案，通过图像识别方式可自动识别谷物，给烹饪设备自动基于谷物类型、品种自动设置烹饪模式提供了技术支持，无需用户人眼识别谷物类型、品种，尤其给不清楚谷物类型、品种的用户的烹饪过程提供了便利，大大提升了用户的体验。
附图说明
图1为本发明实施例的谷物识别方法的流程示意图；
图2a和图2b分别为本发明实施例的谷物识别方法中的谷物类型示意图；
图3为本发明实施例的谷物识别方法中的识别模型训练的一种流程示意图；
图4为本发明实施例的谷物识别方法中的识别模型训练的另一种流程示意图；
图5a和图5b分别为本发明实施例的谷物识别方法中的识别模型训练 的第一种应用示意图；
图6为本发明实施例为本发明实施例的谷物识别方法中的识别模型训练的第二种应用示意图；
图7a至图7h分别为本发明实施例的谷物识别方法中的识别模型训练的第三种应用示意图；
图8为本发明实施例的谷物识别方法的一种应用流程示意图；
图9为本发明实施例的谷物识别装置的组成结构示意图。
具体实施方式
下面结合附图及具体实施例对本发明作进一步详细的说明。
本发明实施例提供了一种谷物识别方法。图1为本发明实施例的谷物识别方法的流程示意图；如图1所示，所述方法包括：
步骤101：获得包括待识别谷物的第一图像数据。
步骤102：基于所述第一图像数据和第一识别模型获得第一识别结果，基于所述第一图像数据和第二识别模型获得第二识别结果，所述第一识别结果表征所述待识别谷物属于的类型；所述第二识别结果表征所述待识别谷物属于的品种。
步骤103：基于所述第一识别结果和所述第二识别结果确定所述待识别谷物的信息。
本发明实施例的谷物识别方法可应用于设备中；作为第一种实施方式，该设备可以是厨房设备，厨房设备设置有图像采集组件(如摄像头)，通过图像采集组件采集图像数据，对采集的图像数据进行分析识别，确定待识别谷物所属品种；作为第二种实施方式，设备可以是厨房设备，该厨房设备不具有图像采集功能，厨房设备可与具有图像采集组件的另一设备通信，通过另一设备的图像采集组件采集图像数据，厨房设备通过与所述另一设备的通信链路获得所述另一设备采集的图像数据；作为第三种实施方式， 设备可以是电子设备，该电子设备可以是移动设备，例如手机、平板电脑等设备，通过电子设备采集图像数据，对采集的图像数据进行分析识别，确定待识别谷物所属品种，进一步基于待识别谷物的品种确定烹饪参数，将烹饪参数发送给厨房设备。其中，所述第一图像数据中包括待识别的谷物，所述待识别的谷物例如大米。实际应用中，厨房设备可以是电饭煲、电压力锅等厨房加热设备。
作为一种实施方式，设备具有烹饪(例如煮饭)功能，即用于对容置于设备中的谷物进行加热。实际应用中，以谷物为大米为例，大米的品种有多样。图2a和图2b分别为本发明实施例的谷物识别方法中的谷物类型示意图；如图2a所示，大米可分为粳米和籼米，例如，五常大米和盘锦大米属于粳米，遮放大米和丝苗米属于籼米，即使处于不同类型的大米外形也高度相似。如图2b所示，四种大米均为籼米，不同品种的籼米高度相似。第二方面，由于加工工艺的不同，可能会出现同一品种不同品牌的大米外形不完全一致的情况。第三方面，碎米/断米容易引起误判，比如籼米比较长，但是如果大量的籼米是断米的话，形态上就和粳米比较接近了。基于此，用户很难通过谷物的外形识别谷物的种类。本发明实施例的设备通过针对谷物的图像采集对谷物的种类进行识别。在本发明以下各实施例中，均以谷物为大米为例进行说明。
本发明实施例中，设备预先通过学习训练方法获得识别模型，则在一实施例中，图3为本发明实施例的谷物识别方法中的识别模型训练的一种流程示意图；如图3所示，所述谷物识别方法还包括：
步骤201：获得多个第二图像信息；所述第二图像信息包括第二图像数据和对应的标签数据。
步骤202：对所述第二图像数据进行特征增强处理，获得特征增强图像数据。
步骤203：对所述特征增强图像数据进行数据增强处理，获得数据增强图像数据。
步骤204：基于所述数据增强图像数据和对应的标签数据进行学习训练，获得识别模型。
本实施例中，获得多个用于识别模型训练的第二图像信息，所述第二图像信息包括第二图像数据和对应的标签数据，所述标签数据表征对应的第二图像数据中包括的谷物的种类。这里，本实施例中的识别模型包括第一识别模型和第二识别模型；其中，所述第一识别模型用于输出谷物类型的识别结果，则用于训练所述第一识别模型的第二图像信息包括的标签数据为谷物类型信息；所述第二识别模型用于输出谷物品种的识别结果，则用于训练所述第二识别模型的第二图像信息包括的标签数据为谷物品种信息。其中，谷物品种可作为谷物类型下的子类，作为一种示例，谷物类型可以是粳米或籼米，谷物品种可以是籼米类型下的泰国香米等。
实际应用中，考虑到图像采集组件采集图像数据时光源对图像数据的影响，如图4所示，在对第二图像数据进行特征增强处理之前，所述方法还可以包括：
步骤202a：识别所述第二图像数据的亮度，基于所述亮度对所述第二图像数据进行裁剪，获得第二图像子数据；所述第二图像子数据的亮度差异满足预设条件。
则对第二图像数据进行特征增强处理具体包括：
步骤202b：对所述第二图像子数据进行特征增强处理，获得特征增强图像数据。
本实施例具体可参照图5a和图5b所示。如图5a所示，第二图像数据中的左侧光照较强烈，不适合将完整的第二图像数据作为训练数据。区别于其他的物体识别任务将整张图像数据拉伸裁剪为正方向的方法会导致图 像发生形变，则本发明实施例中，通过识别所述第二图像数据的亮度，基于像素点之间的亮度差异对第二图像数据进行裁剪，获得第二图像数据中的亮度差异满足预设条件的第二图像子数据。如图5b所示，由于第二图像数据中的左侧光照较强烈，可裁剪去除亮度较大的左侧区域，获得亮度差异较小的右侧正方形区域。实际应用中，训练识别模型采用的图像数据可统一区域大小，则可基于设定的该区域大小对第二图像数据进行裁剪，使得裁剪后获得的第二图像子数据的亮度差异最小。则本实施例中，所述第二图像子数据的亮度差异满足预设条件具体可以为：在满足设定的区域大小的情况下，所述第二图像子数据的亮度差异最小。
本发明实施例中，由于大米的图像数据的色彩空间较为特殊，使得获得的第二图像数据几乎没有彩色信息，即使将第二图像数据转换为灰度图像后，大米的形态特征也不是很明显，分类效果不佳，基于此，本发明实施例中对第二图像数据进行特征增强，主要是将第二图像数据的对比度进行增强处理。
基于此，本发明实施例中，作为一种实施方式，所述对所述第二图像数据进行特征增强处理，获得特征增强图像数据，包括：将所述第二图像数据转换为灰度图像，对所述灰度图像进行对比度增强处理，获得对比度增强图像数据；基于所述对比度增强图像数据获得特征增强图像数据。
这里，获得的第二图像数据通常为彩色数据，则先将第二图像数据对应的红绿蓝(RGB)三通道的彩色数据转换为灰度图像，进一步将所述灰度图像采用对比度增强算法进行处理，将获得的对比度增强图像数据作为所述第二图像数据对应的特征增强图像数据；其中，对比度表示图像数据中的最亮的像素点和最暗的像素点之间不同亮度层级的测量，差异范围越大表示对比度越大，差异范围越小表示对比度越小。其中，所述对比度增强算法包括但不限于以下算法的至少之一：线性变换算法、指数变化算法、 对数变化算法、直方图算法等，从而加强图像数据的对比度，尤其当图像数据的有用数据的对比度相当接近的情况。图6为本发明实施例为本发明实施例的谷物识别方法中的识别模型训练的第二种应用示意图；如图6所示，通过增强图像数据的对比度，使不同米粒之间的区别更加明显，可以反映出不同米粒的透光程度。
考虑到米粒过于相似，非常容易出现过拟合的问题，因此本发明实施例中需要通过翻转和/或旋转的方式扩充训练集。则本实施例中，所述对所述第二图像数据对应的特征增强图像数据进行数据增强处理，获得数据增强图像数据，包括：对所述第二图像数据对应的特征增强图像数据进行翻转和/或旋转，获得与所述特征增强图像数据对应的翻转图像数据和/或旋转图像数据，基于所述翻转图像数据和/或旋转图像数据生成数据增强图像数据。其中，对特征增强图像数据进行旋转，旋转的角度可为第一预设角度，所述第一预设角度为以下角度的其中之一：90度、180度、270度；对特征增强图像数据进行翻转，翻转后的特征增强图像数据进一步进行旋转，旋转的角度可为第二预设角度，所述第二预设角度为以下角度的其中之一：90度、180度、270度。可以理解，作为第一种实施方式，本发明实施例中可将特征增强图像数据按照所述第一预设角度进行旋转，将所述特征增强图像数据和旋转后的特征增强图像数据作为模型训练的训练集图像。作为第二种实施方式，本发明实施例中还可将特征增强图像数据翻转后，翻转后的特征增强图像数据按照所述第二预设角度进行旋转，将所述特征增强图像数据、翻转后的特征增强图像数据以及翻转后且旋转的特征增强图像数据作为模型训练的训练集图像。作为第三种实施方式，可结合上述第一种实施方式和第二种实施方式，将特征增强图像数据，旋转后的特征增强图像数据、翻转后的特征增强图像数据以及翻转后且旋转的特征增强图像数据作为模型训练的训练集图像。
图7a至图7h分别为本发明实施例的谷物识别方法中的识别模型训练的第三种应用示意图；如图7a所示，为特征增强图像数据，图7b至图7d分别为将图7a旋转90度、180度、270度后获得的旋转的特征增强图像数据；图7e为将图7a左右翻转的特征增强图像数据；图7f至图7h分别为将图7e旋转90度、180度、270度后获得的翻转后旋转的特征增强图像数据。这样可将一个图像数据扩充为8个图像数据，在不增加数据采集量的情况下，大幅度的增加高质量的训练集。同时也考虑到在真实使用的环境下，可能会面对的不同环境的影响，比如光照影响。为了更好的处理光照的影响，提高模型的泛化能力，将训练集的图像数据进行了不同程度的光照处理。
本发明实施例中，基于所述数据增强图像数据和对应的标签数据进行学习训练，获得识别模型。作为一种实施方式，当所述标签数据表征谷物属于的类型时，基于所述数据增强图像数据和对应的标签数据进行学习训练，获得第一识别模型。作为另一种实施方式，当所述标签数据表征谷物属于的品种时，基于所述数据增强图像数据和对应的标签数据进行学习训练，获得第二识别模型。
本实施例中，采用卷积神经网络结构对所述数据增强图像数据和对应的标签数据进行学习训练，具体将所述数据增强图像数据作为卷积神经网络模型的输入，将标签数据作为卷积神经网络模型的输出，通过随机梯度下降方法进行学习训练，获得识别模型。当所述标签数据表征谷物属于的类型时，则将表征谷物属于的类型的标签数据作为卷积神经网络模型的输出，通过随机梯度下降方法进行学习训练，获得第一识别模型。当所述标签数据表征谷物属于的品种时，则将表征谷物属于的品种的标签数据作为卷积神经网络模型的输出，通过随机梯度下降方法进行学习训练，获得第二识别模型。
其中，所述卷积神经网络结构可采用以下网络结构的其中之一：AlexNet、VGGNet、GoogleNet和ResNet。采用卷积神经网络结构用于谷物的识别，可以准确识别出高相似度、细颗粒度的谷物的品种，提升了识别的准确度。
本发明实施例中，在基于所述第一图像数据和第一识别模型获得第一识别结果之前，以及基于所述第一图像数据和第二识别模型获得第二识别结果之前，对所述第一图像数据进行预处理，作为一种实施方式，对所述第一图像数据进行预处理方式包括：对所述第一图像数据进行特征增强处理，获得所述第一图像数据对应的特征增强图像数据。作为另一种实施方式，对所述第一图像数据进行预处理方式包括：识别所述第一图像数据的亮度，基于所述亮度对所述第一图像数据进行裁剪，获得第一图像子数据；所述第一图像子数据的亮度差异满足预设条件；对所述第一图像子数据进行特征增强处理，获得所述第一图像数据对应的特征增强图像数据。相应的，所述基于所述第一图像数据和第一识别模型获得第一识别结果，包括：基于所述第一图像数据对应的特征增强图像数据和第一识别模型获得第一识别结果。其中，对所述第一图像数据进行裁剪以及特征增强处理方式可参照第二图像数据的裁剪以及特征增强处理方式，这里不再赘述。
可以理解，将所述第一图像数据对应的特征增强图像数据输入第一识别模型，获得第一识别结果；所述第一识别结果可包括所述第一图像数据包括的大米属于的类型。相应的，将所述第一图像数据对应的特征增强图像数据输入第二识别模型，获得第二识别结果；所述第二识别结果可包括所述第一图像数据包括的大米属于的品种。
实际应用中，输出的识别结果可包括标签以及对应的概率；其中，当输出第一识别结果时，所述第一识别结果包括类型标签以及对应的概率；当输出第二识别结果时，所述第二识别结果包括品种标签以及对应的概率。 当谷物为大米时，可以理解，输出的第一识别结果可以是粳米，以及对应的概率89％，籼米，以及对应的概率11％，则可基于第一识别结果确定大米的类型为粳米。
作为一种实施方式，在谷物为单一类别(例如大米)时，设备中可包括用于对大米进行类型识别的第一识别模型和用于对大米进行品种识别的第二识别模型。则本发明实施例中，所述基于所述第一图像数据和第一识别模型获得第一识别结果，基于所述第一图像数据和第二识别模型获得第二识别结果，基于所述第一识别结果和所述第二识别结果确定所述待识别谷物的信息，包括：基于所述第一图像数据和第一识别模型获得第一识别结果；获得所述第一识别结果中所述待识别谷物所属类别的置信度；当所述待识别谷物所属类别的置信度达到第一预设条件时，基于所述第一图像数据和所述第二识别模型获得第二识别结果；获得所述第二识别结果中所述待识别谷物所属品种的置信度；当所述待识别谷物所属品种的置信度达到第二预设条件时，确定所述待识别谷物的信息为所述待识别谷物所属品种；当所述待识别谷物所属品种的置信度未达到第二预设条件时，确定所述待识别谷物的信息为所述待识别谷物所属类型。
具体可参照图8所示，输入第一图像数据至第一识别模型以进行大米类型的识别，基于获得的第一识别结果中包括的类型以及对应的概率确定大米属于的类型，以及确定大米属于该类型的置信度；在确定大米属于该类型的置信度满足预设条件时，输入第一图像数据至第二识别模型以进行大米品种的识别，基于获得的第二识别结果中包括的品种以及对应的概率确定大米属于的品种，以及确定大米属于该品种的置信度；在确定大米属于该品种的置信度满足预设条件时，输出该大米的品种；在确定大米属于该品种的置信度不满足预设条件时，输出大米的类型。
实际应用中，可依据获得的类型对应的概率的比较结果确定大米属于 的类型；但很可能谷物并不是大米、但形态与大米相近的情况，因此可能出现获得的大米的类型、但该类型对应的概率不高的情况。基于此，若出现该大米属于的类型的概率低于预设阈值时，可确定大米属于该类型的置信度较低，则输出部署大米的识别结果。相反的，若确定大米属于该类型，即确定大米属于该类型的置信度达到预设阈值时，则输入第一图像数据至第二识别模型，进行大米品种的识别，获得品种以及对应的概率；进一步可基于概率比较的方式确定大米属于的品种。其中，若大米属于该品种的概率较低，则可确定大米属于该品种的置信度较低，则直接输出大米类型；若大米属于该品种的概率较高，则可确定大米属于该品种的置信度较高，则直接输出大米品种。
作为另一种实施方式，当设备支持对多种谷物(例如大米、粟米、黄豆等)识别时，所述设备中可包括每种谷物对应的用于类型识别的第一识别模型和用于品种识别的第二识别模型。则本发明实施例中，所述基于所述第一图像数据和第一识别模型获得第一识别结果，基于所述第一图像数据和第二识别模型获得第二识别结果，基于所述第一识别结果和所述第二识别结果确定所述待识别谷物的信息，包括：基于所述第一图像数据和第一识别模型获得第一识别结果；获得所述第一识别结果中所述待识别谷物所属第一谷物的第一类别的置信度；当所述待识别谷物所属第一谷物的第一类别的置信度达到第一预设条件时，基于所述第一图像数据和所述第一谷物对应的第二识别模型获得第二识别结果，获得所述第二识别结果中所述待识别谷物所属第一谷物的第一品种的置信度；当所述待识别谷物所属第一谷物的第一品种的置信度达到第二预设条件时，确定所述待识别谷物的信息为所述待识别谷物所属的第一谷物的第一品种；当所述待识别谷物所属第一谷物的第一品种的置信度未达到第二预设条件时，确定所述待识别谷物的信息为所述待识别谷物所属的第一谷物的第一类型。
本发明实施例采用两个识别模型的结果融合的方式确定待识别谷物的信息，能够提升识别结果的鲁棒性。
本实施方式对应于设备能够识别多种谷物的场景。可以理解，设备中包括多种谷物分别对应的两个识别模型。设备可基于图像识别方式初步识别出第一图像数据中的待识别谷物为第一谷物；进一步通过第一谷物对应的第一识别模型和第二识别模型确定第一谷物的信息。第一谷物的信息的具体确定方式可参照上述谷物为大米的确定方式，这里不再赘述。
在一实施例中，所述方法还包括：基于所述待识别谷物的信息选择操作模式，基于所述操作模式对所述待识别谷物进行加热。实际应用中，设备可基于待识别谷物的类型、品种等信息选择操作模式，所述操作模式具有相匹配的加热参数；则设备基于该操作模式对应的加热参数加热待识别谷物。
采用本发明实施例的技术方案，通过图像识别方式可自动识别谷物，给烹饪设备自动基于谷物类型、品种自动设置烹饪模式提供了技术支持，无需用户人眼识别谷物类型、品种，尤其给不清楚谷物类型、品种的用户的烹饪过程提供了便利，大大提升了用户的体验。
本发明实施例还提供了一种谷物识别装置。图9为本发明实施例的谷物识别装置的组成结构示意图，如图9所示，所述装置包括：存储有能够在处理器31上运行的计算机程序的存储器32；所述处理器31，用于运行所述计算机程序时，执行：获得包括待识别谷物的第一图像数据；基于所述第一图像数据和第一识别模型获得第一识别结果，基于所述第一图像数据和第二识别模型获得第二识别结果，所述第一识别结果表征所述待识别谷物属于的类型；所述第二识别结果表征所述待识别谷物属于的品种；基于所述第一识别结果和所述第二识别结果确定所述待识别谷物的信息。
在一实施例中，所述处理器31，还用于运行所述计算机程序时，执行： 获得多个第二图像信息；所述第二图像信息包括第二图像数据和对应的标签数据；对所述第二图像数据进行特征增强处理，获得特征增强图像数据；对所述特征增强图像数据进行数据增强处理，获得数据增强图像数据；基于所述数据增强图像数据和对应的标签数据进行学习训练，获得识别模型。
在一实施例中，所述处理器31，还用于运行所述计算机程序时，执行：当所述标签数据表征谷物属于的类型时，获得第一识别模型；当所述标签数据表征谷物属于的品种时，获得第二识别模型。
在一实施例中，所述处理器31，还用于运行所述计算机程序时，执行：识别所述第二图像数据的亮度，基于所述亮度对所述第二图像数据进行裁剪，获得第二图像子数据；所述第二图像子数据的亮度差异满足预设条件；对所述第二图像子数据进行特征增强处理，获得特征增强图像数据。
在一实施例中，所述处理器31，用于运行所述计算机程序时，执行：将所述第二图像数据转换为灰度图像，对所述灰度图像进行对比度增强处理，获得对比度增强图像数据；基于所述对比度增强图像数据获得特征增强图像数据。
在一实施例中，所述处理器31，用于运行所述计算机程序时，执行：对所述特征增强图像数据进行翻转和/或旋转，获得与所述特征增强图像数据对应的翻转图像数据和/或旋转图像数据，基于所述翻转图像数据和/或旋转图像数据生成数据增强图像数据。
在一实施例中，所述处理器31，用于运行所述计算机程序时，执行：基于所述第一图像数据和第一识别模型获得第一识别结果；获得所述第一识别结果中所述待识别谷物所属类别的置信度；当所述待识别谷物所属类别的置信度达到第一预设条件时，基于所述第一图像数据和所述第二识别模型获得第二识别结果；获得所述第二识别结果中所述待识别谷物所属品种的置信度；当所述待识别谷物所属品种的置信度达到第二预设条件时， 确定所述待识别谷物的信息为所述待识别谷物所属品种；当所述待识别谷物所属品种的置信度未达到第二预设条件时，确定所述待识别谷物的信息为所述待识别谷物所属类型。
需要说明的是：上述实施例提供的谷物识别装置在进行谷物识别时，仅以上述各程序模块的划分进行举例说明，实际应用中，可以根据需要而将上述处理分配由不同的程序模块完成，即将装置的内部结构划分成不同的程序模块，以完成以上描述的全部或者部分处理。另外，上述实施例提供的谷物识别装置与谷物识别方法实施例属于同一构思，其具体实现过程详见方法实施例，这里不再赘述。
可以理解，装置还包括总线系统33，装置中的各个组件通过总线系统33耦合在一起。可理解，总线系统33用于实现这些组件之间的连接通信。总线系统33除包括数据总线之外，还包括电源总线、控制总线和状态信号总线。但是为了清楚说明起见，在图9中将各种总线都标为总线系统33。
可以理解，存储器32可以是易失性存储器或非易失性存储器，也可包括易失性和非易失性存储器两者。其中，非易失性存储器可以是只读存储器(ROM，Read Only Memory)、可编程只读存储器(PROM，Programmable Read-Only Memory)、可擦除可编程只读存储器(EPROM，Erasable Programmable Read-Only Memory)、电可擦除可编程只读存储器(EEPROM，Electrically Erasable Programmable Read-Only Memory)、磁性随机存取存储器(FRAM，ferromagnetic random access memory)、快闪存储器(Flash Memory)、磁表面存储器、光盘、或只读光盘(CD-ROM，Compact Disc Read-Only Memory)；磁表面存储器可以是磁盘存储器或磁带存储器。易失性存储器可以是随机存取存储器(RAM，Random Access Memory)，其用作外部高速缓存。通过示例性但不是限制性说明，许多形式的RAM可用，例如静态随机存取存储器(SRAM，Static Random Access Memory)、同步静 态随机存取存储器(SSRAM，Synchronous Static Random Access Memory)、动态随机存取存储器(DRAM，Dynamic Random Access Memory)、同步动态随机存取存储器(SDRAM，Synchronous Dynamic Random Access Memory)、双倍数据速率同步动态随机存取存储器(DDRSDRAM，Double Data Rate Synchronous Dynamic Random Access Memory)、增强型同步动态随机存取存储器(ESDRAM，Enhanced Synchronous Dynamic Random Access Memory)、同步连接动态随机存取存储器(SLDRAM，SyncLink Dynamic Random Access Memory)、直接内存总线随机存取存储器(DRRAM，Direct Rambus Random Access Memory)。本发明实施例描述的存储器32旨在包括但不限于这些和任意其它适合类型的存储器。
上述本发明实施例揭示的方法可以应用于处理器31中，或者由处理器31实现。处理器31可能是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法的各步骤可以通过处理器31中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器31可以是通用处理器、数字信号处理器(DSP，Digital Signal Processor)，或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。处理器31可以实现或者执行本发明实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者任何常规的处理器等。结合本发明实施例所公开的方法的步骤，可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于存储介质中，该存储介质位于存储器32，处理器31读取存储器32中的信息，结合其硬件完成前述方法的步骤。
在示例性实施例中，谷物识别装置可以被一个或多个应用专用集成电路(ASIC，Application Specific Integrated Circuit)、DSP、可编程逻辑器件(PLD，Programmable Logic Device)、复杂可编程逻辑器件(CPLD，Complex  Programmable Logic Device)、现场可编程门阵列(FPGA，Field-Programmable Gate Array)、通用处理器、控制器、微控制器(MCU，Micro Controller Unit)、微处理器(Microprocessor)、或其他电子元件实现，用于执行前述方法。
在示例性实施例中，本发明实施例还提供了一种计算机可读存储介质，例如包括计算机程序的存储器32，上述计算机程序可由谷物识别装置的处理器31执行，以完成前述方法所述步骤。计算机可读存储介质可以是FRAM、ROM、PROM、EPROM、EEPROM、Flash Memory、磁表面存储器、光盘、或CD-ROM等存储器；也可以是包括上述存储器之一或任意组合的各种设备，如移动电话、计算机、平板设备、个人数字助理等。
本发明实施例提供了一种计算机存储介质，其上存储有计算机指令，该指令被处理器执行时实现：获得包括待识别谷物的第一图像数据；基于所述第一图像数据和第一识别模型获得第一识别结果，基于所述第一图像数据和第二识别模型获得第二识别结果，所述第一识别结果表征所述待识别谷物属于的类型；所述第二识别结果表征所述待识别谷物属于的品种；基于所述第一识别结果和所述第二识别结果确定所述待识别谷物的信息。
在一实施例中，该指令被处理器执行时实现：获得多个第二图像信息；所述第二图像信息包括第二图像数据和对应的标签数据；对所述第二图像数据进行特征增强处理，获得特征增强图像数据；对所述特征增强图像数据进行数据增强处理，获得数据增强图像数据；基于所述数据增强图像数据和对应的标签数据进行学习训练，获得识别模型。
在一实施例中，该指令被处理器执行时实现：当所述标签数据表征谷物属于的类型时，获得第一识别模型；当所述标签数据表征谷物属于的品种时，获得第二识别模型。
在一实施例中，该指令被处理器执行时实现：识别所述第二图像数据的亮度，基于所述亮度对所述第二图像数据进行裁剪，获得第二图像子数 据；所述第二图像子数据的亮度差异满足预设条件；对所述第二图像子数据进行特征增强处理，获得特征增强图像数据。
在一实施例中，该指令被处理器执行时实现：将所述第二图像数据转换为灰度图像，对所述灰度图像进行对比度增强处理，获得对比度增强图像数据；基于所述对比度增强图像数据获得特征增强图像数据。
在一实施例中，该指令被处理器执行时实现：对所述特征增强图像数据进行翻转和/或旋转，获得与所述特征增强图像数据对应的翻转图像数据和/或旋转图像数据，基于所述翻转图像数据和/或旋转图像数据生成数据增强图像数据。
在一实施例中，该指令被处理器执行时实现：基于所述第一图像数据和第一识别模型获得第一识别结果；获得所述第一识别结果中所述待识别谷物所属类别的置信度；当所述待识别谷物所属类别的置信度达到第一预设条件时，基于所述第一图像数据和所述第二识别模型获得第二识别结果；获得所述第二识别结果中所述待识别谷物所属品种的置信度；当所述待识别谷物所属品种的置信度达到第二预设条件时，确定所述待识别谷物的信息为所述待识别谷物所属品种；当所述待识别谷物所属品种的置信度未达到第二预设条件时，确定所述待识别谷物的信息为所述待识别谷物所属类型。
在本申请所提供的几个实施例中，应该理解到，所揭露的设备和方法，可以通过其它的方式实现。以上所描述的设备实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，如：多个单元或组件可以结合，或可以集成到另一个系统，或一些特征可以忽略，或不执行。另外，所显示或讨论的各组成部分相互之间的耦合、或直接耦合、或通信连接可以是通过一些接口，设备或单元的间接耦合或通信连接，可以是电性的、机械的或其它形式的。
上述作为分离部件说明的单元可以是、或也可以不是物理上分开的，作为单元显示的部件可以是、或也可以不是物理单元，即可以位于一个地方，也可以分布到多个网络单元上；可以根据实际的需要选择其中的部分或全部单元来实现本实施例方案的目的。
另外，在本发明各实施例中的各功能单元可以全部集成在一个处理单元中，也可以是各单元分别单独作为一个单元，也可以两个或两个以上单元集成在一个单元中；上述集成的单元既可以采用硬件的形式实现，也可以采用硬件加软件功能单元的形式实现。
本领域普通技术人员可以理解：实现上述方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成，前述的程序可以存储于一计算机可读取存储介质中，该程序在执行时，执行包括上述方法实施例的步骤；而前述的存储介质包括：移动存储设备、ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。
或者，本发明上述集成的单元如果以软件功能模块的形式实现并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明实施例的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机、服务器、或者网络设备等)执行本发明各个实施例所述方法的全部或部分。而前述的存储介质包括：移动存储设备、ROM、RAM、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本发明的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应以所述权利要求的保护范围为准。

Claims (15)

  1. 一种谷物识别方法，所述方法包括：
    获得包括待识别谷物的第一图像数据；
    基于所述第一图像数据和第一识别模型获得第一识别结果，基于所述第一图像数据和第二识别模型获得第二识别结果，所述第一识别结果表征所述待识别谷物属于的类型；所述第二识别结果表征所述待识别谷物属于的品种；
    基于所述第一识别结果和所述第二识别结果确定所述待识别谷物的信息。
  2. 根据权利要求1所述的方法，其中，所述方法还包括：
    获得多个第二图像信息；所述第二图像信息包括第二图像数据和对应的标签数据；
    对所述第二图像数据进行特征增强处理，获得特征增强图像数据；
    对所述特征增强图像数据进行数据增强处理，获得数据增强图像数据；
    基于所述数据增强图像数据和对应的标签数据进行学习训练，获得识别模型。
  3. 根据权利要求2所述的方法，其中，当所述标签数据表征谷物属于的类型时，所述获得识别模型，包括：获得第一识别模型；
    当所述标签数据表征谷物属于的品种时，所述获得识别模型，包括：获得第二识别模型。
  4. 根据权利要求2所述的方法，其中，所述对所述第二图像数据进行特征增强处理之前，所述方法还包括：
    识别所述第二图像数据的亮度，基于所述亮度对所述第二图像数据进行裁剪，获得第二图像子数据；所述第二图像子数据的亮度差异满足预设条件；
    相应的，所述对所述第二图像数据进行特征增强处理，包括：对所述第二图像子数据进行特征增强处理。
  5. 根据权利要求2至4任一项所述的方法，其中，所述对所述第二图像数据进行特征增强处理，获得特征增强图像数据，包括：
    将所述第二图像数据转换为灰度图像，对所述灰度图像进行对比度增强处理，获得对比度增强图像数据；基于所述对比度增强图像数据获得特征增强图像数据。
  6. 根据权利要求2至4任一项所述的方法，其中，所述对所述特征增强图像数据进行数据增强处理，获得数据增强图像数据，包括：
    对所述特征增强图像数据进行翻转和/或旋转，获得与所述特征增强图像数据对应的翻转图像数据和/或旋转图像数据，基于所述翻转图像数据和/或旋转图像数据生成数据增强图像数据。
  7. 根据权利要求1所述的方法，其中，所述基于所述第一图像数据和第一识别模型获得第一识别结果，基于所述第一图像数据和第二识别模型获得第二识别结果，基于所述第一识别结果和所述第二识别结果确定所述待识别谷物的信息，包括：
    基于所述第一图像数据和第一识别模型获得第一识别结果；获得所述第一识别结果中所述待识别谷物所属类别的置信度；当所述待识别谷物所属类别的置信度达到第一预设条件时，基于所述第一图像数据和所述第二识别模型获得第二识别结果；获得所述第二识别结果中所述待识别谷物所属品种的置信度；当所述待识别谷物所属品种的置信度达到第二预设条件时，确定所述待识别谷物的信息为所述待识别谷物所属品种；当所述待识别谷物所属品种的置信度未达到第二预设条件时，确定所述待识别谷物的信息为所述待识别谷物所属类型。
  8. 一种谷物识别装置，所述装置包括：
    存储有能够在处理器上运行的计算机程序的存储器；
    所述处理器，配置为运行所述计算机程序时，执行：获得包括待识别谷物的第一图像数据；基于所述第一图像数据和第一识别模型获得第一识别结果，基于所述第一图像数据和第二识别模型获得第二识别结果，所述第一识别结果表征所述待识别谷物属于的类型；所述第二识别结果表征所述待识别谷物属于的品种；基于所述第一识别结果和所述第二识别结果确定所述待识别谷物的信息。
  9. 根据权利要求8所述的装置，其中，所述处理器，还配置为运行所述计算机程序时，执行：获得多个第二图像信息；所述第二图像信息包括第二图像数据和对应的标签数据；对所述第二图像数据进行特征增强处理，获得特征增强图像数据；对所述特征增强图像数据进行数据增强处理，获得数据增强图像数据；基于所述数据增强图像数据和对应的标签数据进行学习训练，获得识别模型。
  10. 根据权利要求9所述的装置，其中，所述处理器，还配置为运行所述计算机程序时，执行：当所述标签数据表征谷物属于的类型时，获得第一识别模型；当所述标签数据表征谷物属于的品种时，获得第二识别模型。
  11. 根据权利要求9所述的装置，其中，所述处理器，还配置为运行所述计算机程序时，执行：识别所述第二图像数据的亮度，基于所述亮度对所述第二图像数据进行裁剪，获得第二图像子数据；所述第二图像子数据的亮度差异满足预设条件；对所述第二图像子数据进行特征增强处理，获得特征增强图像数据。
  12. 根据权利要求9至11任一项所述的装置，其中，所述处理器，配置为运行所述计算机程序时，执行：将所述第二图像数据转换为灰度图像，对所述灰度图像进行对比度增强处理，获得对比度增强图像数据；基于所 述对比度增强图像数据获得特征增强图像数据。
  13. 根据权利要求9至11任一项所述的装置，其中，所述处理器，配置为运行所述计算机程序时，执行：对所述特征增强图像数据进行翻转和/或旋转，获得与所述特征增强图像数据对应的翻转图像数据和/或旋转图像数据，基于所述翻转图像数据和/或旋转图像数据生成数据增强图像数据。
  14. 根据权利要求8所述的装置，其中，所述处理器，配置为运行所述计算机程序时，执行：基于所述第一图像数据和第一识别模型获得第一识别结果；获得所述第一识别结果中所述待识别谷物所属类别的置信度；当所述待识别谷物所属类别的置信度达到第一预设条件时，基于所述第一图像数据和所述第二识别模型获得第二识别结果；获得所述第二识别结果中所述待识别谷物所属品种的置信度；当所述待识别谷物所属品种的置信度达到第二预设条件时，确定所述待识别谷物的信息为所述待识别谷物所属品种；当所述待识别谷物所属品种的置信度未达到第二预设条件时，确定所述待识别谷物的信息为所述待识别谷物所属类型。
  15. 一种计算机存储介质，其上存储有计算机指令，其中，该指令被处理器执行时实现权利要求1至7任一项所述方法的步骤。
PCT/CN2017/119932 2017-12-29 2017-12-29 一种谷物识别方法、装置和计算机存储介质 WO2019127400A1 (zh)

Priority Applications (5)

Application Number Priority Date Filing Date Title
JP2020533125A JP7073607B2 (ja) 2017-12-29 2017-12-29 穀物の識別方法、装置およびコンピュータ記憶媒体
EP17936231.4A EP3715834B1 (en) 2017-12-29 2017-12-29 Grain identification method and device, and computer storage medium
PCT/CN2017/119932 WO2019127400A1 (zh) 2017-12-29 2017-12-29 一种谷物识别方法、装置和计算机存储介质
CN201780092478.3A CN111033240A (zh) 2017-12-29 2017-12-29 一种谷物识别方法、装置和计算机存储介质
KR1020207017704A KR102333500B1 (ko) 2017-12-29 2017-12-29 곡물 인식 방법, 장치 및 컴퓨터 판독 가능 저장 매체

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/119932 WO2019127400A1 (zh) 2017-12-29 2017-12-29 一种谷物识别方法、装置和计算机存储介质

Publications (1)

Publication Number Publication Date
WO2019127400A1 true WO2019127400A1 (zh) 2019-07-04

Family

ID=67062898

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/119932 WO2019127400A1 (zh) 2017-12-29 2017-12-29 一种谷物识别方法、装置和计算机存储介质

Country Status (5)

Country Link
EP (1) EP3715834B1 (zh)
JP (1) JP7073607B2 (zh)
KR (1) KR102333500B1 (zh)
CN (1) CN111033240A (zh)
WO (1) WO2019127400A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111310815A (zh) * 2020-02-07 2020-06-19 北京字节跳动网络技术有限公司 图像识别方法、装置、电子设备及存储介质
CN116343201A (zh) * 2023-05-29 2023-06-27 安徽高哲信息技术有限公司 谷粒类别识别方法、装置及计算机设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002312762A (ja) * 2001-04-12 2002-10-25 Seirei Ind Co Ltd ニューラルネットワークを利用した穀粒選別装置
CN101701916A (zh) * 2009-12-01 2010-05-05 中国农业大学 一种玉米品种快速鉴定、鉴别方法
CN101788497A (zh) * 2009-12-30 2010-07-28 深圳先进技术研究院 一种基于图像识别技术的嵌入式豆类分类系统
CN102279976A (zh) * 2011-09-22 2011-12-14 河南工业大学 不同糙米籽粒识别的bp神经网络构建及识别方法
CN103268492A (zh) * 2013-04-19 2013-08-28 北京农业信息技术研究中心 一种玉米籽粒类型识别方法

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6148099A (en) * 1997-07-03 2000-11-14 Neopath, Inc. Method and apparatus for incremental concurrent learning in automatic semiconductor wafer and liquid crystal display defect classification
KR101402764B1 (ko) * 2012-06-21 2014-06-03 박지명 취사시 물의 양을 자동 조절하는 전기밥솥
JP6435740B2 (ja) * 2014-09-22 2018-12-12 日本電気株式会社 データ処理システム、データ処理方法およびデータ処理プログラム
CN106326794A (zh) * 2015-07-07 2017-01-11 北京奈思膳品科技有限公司 一种烹饪系统、终端、服务器及烹饪方法
CN105547915A (zh) * 2016-01-11 2016-05-04 北京广度漫想科技有限公司 基于密度模型的硬壳类食品的新鲜度测定系统及其方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002312762A (ja) * 2001-04-12 2002-10-25 Seirei Ind Co Ltd ニューラルネットワークを利用した穀粒選別装置
CN101701916A (zh) * 2009-12-01 2010-05-05 中国农业大学 一种玉米品种快速鉴定、鉴别方法
CN101788497A (zh) * 2009-12-30 2010-07-28 深圳先进技术研究院 一种基于图像识别技术的嵌入式豆类分类系统
CN102279976A (zh) * 2011-09-22 2011-12-14 河南工业大学 不同糙米籽粒识别的bp神经网络构建及识别方法
CN103268492A (zh) * 2013-04-19 2013-08-28 北京农业信息技术研究中心 一种玉米籽粒类型识别方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3715834A4 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111310815A (zh) * 2020-02-07 2020-06-19 北京字节跳动网络技术有限公司 图像识别方法、装置、电子设备及存储介质
CN116343201A (zh) * 2023-05-29 2023-06-27 安徽高哲信息技术有限公司 谷粒类别识别方法、装置及计算机设备
CN116343201B (zh) * 2023-05-29 2023-09-19 安徽高哲信息技术有限公司 谷粒类别识别方法、装置及计算机设备

Also Published As

Publication number Publication date
KR102333500B1 (ko) 2021-12-01
EP3715834A4 (en) 2021-01-13
EP3715834A1 (en) 2020-09-30
EP3715834B1 (en) 2024-04-03
JP7073607B2 (ja) 2022-05-24
CN111033240A (zh) 2020-04-17
KR20200087243A (ko) 2020-07-20
JP2021507389A (ja) 2021-02-22

Similar Documents

Publication Publication Date Title
CN108108767B (zh) 一种谷物识别方法、装置和计算机存储介质
TWI746674B (zh) 識別圖像中物件的類型預測方法、裝置及電子設備
US20180330522A1 (en) System and method for assessing wound
TW201933176A (zh) 實物識別方法及驗證方法
AU2021201933B2 (en) Hierarchical multiclass exposure defects classification in images
TW201923707A (zh) 圖像處理方法和處理設備
CN112052831B (zh) 人脸检测的方法、装置和计算机存储介质
CN107810505A (zh) 实时图像捕获参数的机器学习
KR20190099914A (ko) 전자 장치, 이의 영상 처리 방법 및 컴퓨터 판독가능 기록 매체
CN110689056A (zh) 一种分类方法及装置、设备和存储介质
KR102453207B1 (ko) 곡물 평가 방법, 장치 및 저장 매체
WO2019127400A1 (zh) 一种谷物识别方法、装置和计算机存储介质
Lee et al. Mt-diet: Automated smartphone based diet assessment with infrared images
US11335128B2 (en) Methods and systems for evaluating a face recognition system using a face mountable device
CN114743224B (zh) 一种基于计算机视觉的畜牧业家畜体温监控方法与系统
US11562561B2 (en) Object verification/recognition with limited input
JP7362924B2 (ja) データ増強基盤空間分析モデル学習装置及び方法
US10474893B2 (en) Electronic device, iris recognition method and computer-readable medium
US20140254864A1 (en) System and method for gesture detection through local product map
CN113657245B (zh) 一种用于人脸活体检测的方法、设备、介质及程序产品
CN115623317B (zh) 一种对焦的方法、设备及存储介质
CN114202553A (zh) 目标信息识别、边缘图生成方法、装置、存储介质及设备
Condominas Mobile phone based imaging system for selected tele-healthcare applications
Condominas Guàrdia Mobile phone based imaging system for selected tele-healthcare applications

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17936231

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020533125

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 20207017704

Country of ref document: KR

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2017936231

Country of ref document: EP

Effective date: 20200625