WO2022262586A1

WO2022262586A1 - Method for plant identification, computer system and computer-readable storage medium

Info

Publication number: WO2022262586A1
Application number: PCT/CN2022/096706
Authority: WO
Inventors: 徐青松; 李青
Original assignee: 杭州睿胜软件有限公司
Priority date: 2021-06-15
Filing date: 2022-06-01
Publication date: 2022-12-22
Also published as: CN113298180A

Abstract

A method for plant identification, a computer system, and a computer-readable storage medium, the method comprising: receiving an image, the image comprising at least a part of a plant; according to the image, identifying at least one item among the part, growing site and growth cycle of the plant in the image and the classification of the plant in the image on the basis of a trained neural network model; and according to an operation request of a user, performing a requested operation on the identified classification of the plant and at least one item.

Description

Method, computer system, and computer-readable storage medium for plant identification

technical field

The present disclosure relates to the field of computer technology, and in particular to a method for identifying plants, a computer system, and a computer-readable storage medium.

Background technique

In the field of computer technology there are applications for identifying plants. These applications usually receive images from users (including static images, dynamic images, and videos, etc.), and identify the classification of plants to be identified in the images based on the plant recognition model established by artificial intelligence technology. For example, the recognition result, that is, the classification of the plant may be the species of the plant (Species) or the like. The image from the user usually includes at least a part of the plant to be identified, for example, the image may include stems, leaves, and flowers of the plant to be identified.

Contents of the invention

An object of the present disclosure is to provide a method and computer system for plant identification.

According to a first aspect of the present disclosure, there is provided a method for plant identification, comprising: receiving an image, wherein the image includes at least one part of a plant; according to the image, identifying the plant based on a trained neural network model At least one of the parts, growth sites, and growth cycles of the plants in the image, and the classification of the plants in the image; and according to the user's operation request, perform classification on the identified plant and the at least one item The requested action.

According to a first aspect of the present disclosure, there is provided a method for plant identification, comprising: receiving an image, wherein the image includes at least one part of a plant; according to the image, identifying the plant based on a trained neural network model The classification of the plants in the image, the parts of the plants in the image, the growing places of the plants in the image, the growth cycle of the plants in the image, and the image quality of the image; and according to the user's operation request , select one or more identified items among the identified items to perform the requested operation.

According to a third aspect of the present disclosure, there is provided a method for plant identification, including: identifying the classification of plants based on a trained neural network model according to images, and identifying the growth place, growth cycle and the image of the plants At least two of the parts of the plant in the image, wherein the image includes at least one part of the plant; according to the identified classification of the plant, and the identified growth place, growth cycle and the identified plant The at least two of the parts of the plant in the image determine a maintenance program for the plant; and output the maintenance program for the plant.

According to a fourth aspect of the present disclosure, there is provided a method for plant identification, including: receiving an image, wherein the image includes at least one part of the plant; according to the image, identifying the plant in the image The part and the classification of the plant; according to the recognized part of the plant in the image, determine the output classification level; and output the classification of the corresponding classification level of the plant according to the determined classification level.

According to a fifth aspect of the present disclosure, there is provided a computer system for plant identification, comprising: one or more processors; and one or more memories configured to store a series of computer executable instructions and computer-accessible data associated with the series of computer-executable instructions, which, when executed by the one or more processors, cause the The computer system performs any of the methods described above.

According to a sixth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium, the non-transitory computer-readable storage medium stores a series of computer-executable instructions, when the series of computer-executable instructions When executed by one or more computer systems, the instructions cause the one or more computer systems to perform any of the methods described above.

Other features of the present disclosure and advantages thereof will become apparent through the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings.

Description of drawings

The accompanying drawings, which constitute a part of this specification, illustrate the embodiments of the disclosure and together with the description serve to explain the principles of the disclosure.

The present disclosure can be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

Fig. 1 is a flowchart schematically illustrating at least part of a method for plant identification according to an embodiment of the present disclosure.

Fig. 2 is a flowchart schematically illustrating at least part of a method for plant identification according to another embodiment of the present disclosure.

Fig. 3 is a structural diagram schematically showing at least a part of a computer system for plant identification according to an embodiment of the present disclosure.

FIG. 4 is a structural diagram schematically showing at least a part of a computer system for plant identification according to another embodiment of the present disclosure.

Fig. 5 is a flowchart schematically illustrating at least part of a method for plant identification according to yet another embodiment of the present disclosure.

Fig. 6 is a flowchart schematically illustrating at least part of a method for plant identification according to yet another embodiment of the present disclosure.

Note that in the embodiments described below, the same reference numerals may be used in common between different drawings to denote the same parts or parts having the same functions, and repeated descriptions thereof will be omitted. In this specification, similar reference numerals and letters are used to refer to similar items, therefore, once an item is defined in one figure, it does not require further discussion in subsequent figures.

detailed description

Various exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. It should be noted that relative arrangements of components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise. In the following description, numerous details are set forth in order to better explain the disclosure, however it is understood that the disclosure may be practiced without these details.

The following description of at least one exemplary embodiment is merely illustrative in nature and in no way intended as any limitation of the disclosure, its application or uses. In all examples shown and discussed herein, any specific values should be construed as illustrative only, and not as limiting.

Techniques, methods and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, such techniques, methods and devices should be considered part of the description.

Fig. 1 is a flowchart schematically illustrating at least part of a method 100 for plant identification according to an embodiment of the present disclosure. The method 100 includes: identifying the classification of the plant according to the image, and identifying at least two of the growth location of the plant, the growth cycle, and the part of the plant in the image, wherein the image includes at least one part of the plant (step 110); The classification of the plants, and at least two of the identified plant growth location, growth cycle, and plant parts in the image determine a plant maintenance program (step 120); and output a plant maintenance program (step 130). Plant maintenance programs can include, for example, watering, spraying, changing water, adding water, fertilizing, pruning, weeding, turning pots, changing pots, sunshine, shading, adjusting temperature, adjusting humidity, winter protection, and pest control. At least one execution plan. The inventors of the present application have found that the maintenance scheme is not only related to the type of plant, but also related to the growth place, growth cycle, and part of the plant, so the method according to the embodiment of the present disclosure can At least two of the growth location, growth cycle and plant parts in the image are used to determine a personalized maintenance plan for the plants in the image.

The user can input the image including at least one part of the plant to be identified into the application program capable of identifying the plant to identify the classification of the plant. It should be noted that, the image mentioned herein includes at least one part of the plant, which means including one or more parts of the plant, wherein each part may be the whole or part of the part. The image can be previously stored by the user, captured in real time, or downloaded from the Internet. Imagery may include any form of visual representation, such as still images, moving images, and video. Images can be taken with devices including cameras, such as mobile phones, tablet computers, and the like.

An application capable of implementing the method 100 may receive the image from the user, and perform plant recognition based on the image in step 110 . Any known method of image-based plant identification may be included. For example, the identified plants in the image can be identified by means of a computing device and a pre-trained (or called "trained") plant identification model to obtain a recognition result, that is, a plant classification. A plant recognition model can be established based on a neural network (such as a deep convolutional neural network (CNN) or a deep residual network (Resnet), etc.). For example, for each plant classification, obtain a certain number of image samples marked with the classification name of the plant, that is, the training sample set, and use these image samples to train the neural network until the output accuracy of the neural network meets the requirements. Before image-based plant recognition, images can also be preprocessed. Preprocessing may include normalization, brightness adjustment, or noise reduction, among others. Noise reduction processing can highlight the description of the feature parts in the image, making the features more distinct.

At step 110 , the method 100 further identifies at least two items of plant growth location, growth cycle, and plant parts in the image according to the image. Recognition can be based on a trained neural network model.

The maintenance regimen can be related to where the plants are grown. The habitat of the plant can be identified based on the trained habitat classification model. A plurality of classifications can be established for plant growth locations according to the relationship between the growth location of the plants and the maintenance regimen of the plants. A certain number of image samples can be obtained for each classification of growing places, and the classification names of the growing places of plants in each image sample can be marked on these image samples, so as to create a training sample set. Use the training sample set to train the neural network until the output accuracy of the neural network meets the requirements.

Some maintenance programs differentiate between potted plants and non-potted plants, for example, the tasks of repotting and repotting are only for potted plants. Some care regimens differentiate between cut and non-cut flowers, for example, tasks for cut flowers may include changing water while tasks for non-cut flowers may include watering. Thus, in one example, multiple classifications of where plants grow may include potted, non-potted, and cut flowers. For example, you can label those that can be seen in flower pots or grow indoors as "potted plants", those that are placed in vases as "fresh-cut flowers", and those that are not potted or fresh-cut flowers as "non-potted plants" , label no plants, fake plants, specimens, or unidentifiable images as "Other".

The maintenance regimen can be related to the growth cycle of the plant. The growth cycle of the plant can be identified based on the trained growth cycle classification model. Multiple classifications can be established for growth cycles of plants based on the relationship between the growth cycle of the plant and the maintenance regimen of the plant. A certain number of image samples may be obtained for the classification of each growth cycle, and the classification name of the growth cycle of the plants in each image sample is marked on these image samples, so as to create a training sample set. Use the training sample set to train the neural network until the output accuracy of the neural network meets the requirements.

The maintenance program may be different for plants in different growth cycles. For example, the same plant may require different watering frequency and amount in different growth cycles. For example, the potting task for potted plants can only be for plants whose growth cycle is leaf stage, flowering stage or fruit stage. In one example, the multiple classifications of growth cycles of plants may include emerging seedlings, young seedlings, leaf stage, flowering stage, fruiting stage, leaf falling stage, and dormancy stage. For example, an image that only includes two cotyledons can be labeled as "emergent seedling", an image with multiple leaves but not fully grown can be labeled "small seedling", and an image before flowering can be labeled "leaf stage". , label those that are difficult to distinguish into other growth cycles as "Other".

The maintenance regimen can be related to the part of the plant. Plant parts may be identified based on the trained part classification model. Multiple classifications can be established for plant parts based on the relationship between the plant part and the plant's care regimen. A certain number of image samples may be obtained for the classification of each part, and the classification names of the plant parts in each image sample are marked on these image samples, thereby creating a training sample set. Use the training sample set to train the neural network until the output accuracy of the neural network meets the requirements.

The location of the plants in the image can affect the maintenance regimen output to the user. For example, if it is found that the part of the plant is the leaves, and some are yellowing, then the user can be reminded to water more. For example, if there are spots on the leaves, it may be some kind of disease and insect pest, and the user can be reminded to prevent and control the corresponding disease and insect pest. In one example, multiple classifications of parts of plants may include trunks, buds, seeds, buds, fruits, seedlings, leaves, flowers, stems, and roots. In addition, in other examples, a corresponding classification can also be established to include that the plant body in the image is too far away to distinguish its detailed features, and the plant body in the image is too close so that the plant in the image is too partial without complete organs, etc.

At step 120 , the method 100 determines a plant maintenance plan according to the identified plant classification, and at least two of the plant's growth location, growth cycle, and plant parts in the image. At step 130, the method 100 outputs a maintenance regimen for the plant.

In an example, a maintenance scheme lookup table as shown in Table 1 may be pre-established, and a plant maintenance scheme may be determined based on the maintenance scheme lookup table. The value in each cell in Table 1 indicates the frequency at which the maintenance task should be performed in the maintenance plan, that is, the interval at which the task is repeated, and the unit is the number of days. For example, the maintenance plan for plant classification 1 in the case of growth cycle 1 and growth location 1 is to perform a corresponding maintenance task (for example, pruning) every 28 days. No value in the unit can indicate that the corresponding plant classification does not need to perform the task in the corresponding growth cycle, and a unit with a value of -1 can indicate that the task of the corresponding plant classification only needs to be performed once in the corresponding growth cycle. The frequency at which the maintenance task should be performed can be determined according to the classification and growth cycle of the plant according to the maintenance scheme lookup table shown in Table 1.

Table 1 Maintenance program lookup table

In some embodiments, the maintenance program includes a maintenance program related to pest control, ie, an implementation program for pest control. In this embodiment, the method 100 needs to identify the types of plant diseases and insect pests (herein also referred to as "diagnostic information of diseases and insect pests") according to the image, so as to recommend to the user a personalized maintenance plan related to disease and insect pest control. The trained pest diagnosis model can be used to identify the diagnostic information of plant diseases and insect pests based on images. The diagnostic information may include pest information or no pest detection information. The pest diagnosis model can be a neural network model, specifically a convolutional neural network model or a residual network model.

A large number of images can be included in the training sample set of the disease and pest diagnosis model, and each image is correspondingly marked with diagnostic information, such as the information of the diseases and insect pests suffered by the plants in this image, or corresponding to healthy plants Information about undetected pests and diseases. The image is input into the pest diagnosis model to generate the output diagnostic information, and then according to the comparison result between the output diagnostic information and the labeled diagnostic information, the relevant parameters in the pest diagnosis model can be adjusted, that is, the pest diagnosis model is trained. The training ends when the output accuracy of the pest diagnosis model meets the requirements, so as to obtain the trained pest diagnosis model.

After identifying the diagnostic information of plant diseases and insect pests, according to the classification of the identified plants and the diagnostic information of plant diseases and insect pests, and according to at least two of the plant's growth location, growth cycle and plant parts in the image, in the established database A maintenance scheme related to pest control is extracted, and the maintenance scheme is output. When extracting the relevant maintenance plan in the database, it can be retrieved according to the classification of the plant and the diagnosis information of diseases and insect pests, and the retrieved maintenance plan can be compared according to at least two of the plant's growth place, growth cycle and plant parts in the image Make appropriate adjustments. When a large amount of data is pre-stored in the database, it can cover the classification of most plants and the maintenance scheme of the diagnosis information of diseases and insect pests, so as to provide users with corresponding maintenance schemes.

In some embodiments, the output classification level may be determined according to the recognized plant parts in the image. In some cases, if the parts of the plants in the image are trunks, buds, seeds, flower buds, fruits or seedlings, it will be more difficult to obtain more accurate species information (that is, information at the taxonomic level). In these cases, if the results of the identified species are directly output, it is likely to be wrong, which will mislead or confuse the user. At this time, if the Genus information is identified, it is generally more accurate. And if the plant parts in the image are characteristic parts such as leaves, flowers, stems, roots, etc., the identified species information is usually reliable. Therefore, in response to the part of the plant in the image being one of trunk, bud, seed, flower bud, fruit, and seedling, the classification level of the output is determined to be a genus; and in response to the part of the plant in the image being leaf, flower, One of stem and root, determine the output classification level as species.

The identification results provided by the plant identification model usually include one or more classifications of the identified plants. One or more classifications are ranked from high to low by confidence (how close the classification is to the true classification). In one embodiment, the classification level of one or more classifications included in the recognition result provided by the plant recognition model is species. According to the corresponding relationship between species and genus, it can be known that the classification level of each recognition result is the classification of genus. In one embodiment, the classification level of one or more classifications included in the recognition result provided by the plant recognition model is species and genus.

In some embodiments, the various identifications mentioned above are all performed according to the subject in the image. The subject in the image may refer to the entity occupying the largest area in the image, or the entity located substantially in the middle of the image, or the entity not located at the corner of the image. In some embodiments, in response to the subject in the image being unclear (the subject cannot be distinguished, such as a distant view of a large tree), the subject is a non-plant (the subject can be distinguished, but the subject is not a plant), and the subject is a distant view of a whole plant (The subject can be distinguished, but the subject is too far away to recognize details, such as the shape of the blade, etc.), no recognition is performed, that is, step 110 of the above-mentioned method 100 is not executed, and a prompt message asking the user to re-input the image is output. In this way, invalid identifications can be avoided. Judging whether the subject of the image is unclear, the subject is a non-plant, or the subject is a distant view of a whole plant can be performed through a trained neural network model. A certain number of image samples marked with the classification can be prepared for each of the classifications of the classifications such as the subject is not clear, the subject is non-plant, and the subject is the whole plant, and these image samples are used to train the neural network until The output accuracy of the neural network meets the requirements.

In some embodiments, the image quality of the imagery may be identified based on a trained quality classification model. In response to the image quality being clear, identification is performed, that is, step 110 of the above-mentioned method 100 is executed. If the image quality is not clear, no recognition is performed, that is, step 110 of the above method 100 is not executed, and a message prompting the user to re-input the image is output. Among them, the classification of unclear image quality can be classified in more detail, such as unclear caused by light, unclear caused by focal length, etc., so that more specific prompt information can be output to the user, such as prompting the user to fill in light and then reshoot etc. Quality classification models can be trained based on image sample sets. The image sample set includes a certain number of image samples marked with the classification prepared for each classification. These image samples are used to train the neural network until the output accuracy of the neural network meets the requirements, so as to obtain the trained quality classification model.

Fig. 2 is a flowchart schematically illustrating at least a part of a method 200 for plant identification according to another embodiment of the present disclosure. The method 200 includes: receiving an image, wherein the image includes at least one part of a plant (step 210); identifying the part of the plant in the image and the classification of the plant according to the image (step 220); , determine the classification level of the output (step 230); and output the classification of the corresponding classification level of the plant according to the determined classification level (step 240).

In step 210, the application capable of implementing the method 200 may receive an image from a user including at least one part of the plant to be identified. It should be noted that, the image mentioned herein includes at least one part of the plant, which means including one or more parts of the plant, wherein each part may be the whole or part of the part. The image can be previously stored by the user, captured in real time, or downloaded from the Internet. Imagery may include any form of visual representation, such as still images, moving images, and video. Images can be taken with devices including cameras, such as mobile phones, tablet computers, and the like.

In step 220 , the method 200 identifies plant parts and plant classifications in the image according to the image. The trained plant recognition model described above can be used to identify the classification of the plant in the image, and the trained part classification model described above can be used to identify the part of the plant in the image.

In step 230 , the method 200 determines an output classification level according to the recognized plant parts in the image. In some cases, if the parts of the plants in the image are trunks, buds, seeds, flower buds, fruits or seedlings, it will be more difficult to obtain more accurate species information (that is, information at the taxonomic level). In these cases, if the results of the identified species are directly output, it is likely to be wrong, which will mislead or confuse the user. At this time, if the identified information is generally more accurate. However, if the plant parts in the image are characteristic parts such as leaves, flowers, stems, roots, etc., the identified species information is usually reliable. Therefore, in response to the part of the plant in the image being one of trunk, bud, seed, flower bud, fruit, and seedling, the classification level of the output is determined to be a genus; and in response to the part of the plant in the image being leaf, flower, One of stem and root, determine the output classification level as species.

In some embodiments, the above identifications are all performed according to the subject in the image. The subject in the image may refer to the entity occupying the largest area in the image, or the entity located substantially in the middle of the image, or the entity not located at the corner of the image. In some embodiments, in response to the subject in the image being unclear (the subject cannot be distinguished, such as a distant view of a large tree), the subject is a non-plant (the subject can be distinguished, but the subject is not a plant), and the subject is a distant view of a whole plant (The subject can be distinguished, but the subject is too far away to recognize details, such as the shape of the blade, etc.), no recognition is performed, that is, step 220 of the above method 200 is not executed, and information prompting the user to re-input the image is output. In this way, invalid identifications can be avoided. Judging whether the subject of the image is unclear, the subject is a non-plant, or the subject is a distant view of a whole plant can also be performed through a trained neural network model. A certain number of image samples marked with the classification can be prepared for each of the classifications of the classifications such as the subject is not clear, the subject is non-plant, and the subject is the whole plant, and these image samples are used to train the neural network until The output accuracy of the neural network meets the requirements.

In some embodiments, the image quality of the imagery may be identified based on a trained quality classification model. In response to the image quality being clear, identification is performed, that is, step 220 of the above-mentioned method 200 is performed. If the image quality is not clear, no recognition is performed, that is, step 220 of the above method 200 is not executed, and a message prompting the user to re-input the image is output. Among them, the classification of unclear image quality can be classified in more detail, such as unclear caused by light, unclear caused by focal length, etc., so that more specific prompt information can be output to the user, such as prompting the user to fill in light and then reshoot etc. Quality classification models can be trained based on image sample sets. The image sample set includes a certain number of image samples marked with the classification prepared for each classification. These image samples are used to train the neural network until the output accuracy of the neural network meets the requirements, so as to obtain the trained quality classification model.

In some embodiments, the user's needs may not only be plant maintenance schemes or plant classifications, and at this time the

methods

100 and 200 according to the embodiments of the present invention are difficult to meet the needs. Fig. 5 is a flowchart schematically illustrating at least a part of a method 500 for plant identification according to yet another embodiment of the present disclosure. The method 500 includes: receiving an image, wherein the image includes at least one part of a plant (step 510); according to the image, identifying at least one of the part of the plant in the image, the place of growth and the growth cycle, and the image based on a trained neural network model. Classification of the plants in (step 520); and according to the user's operation request, perform the requested operation on the classification of the identified plants and at least one of the above items (step 530).

In step 510 , an application capable of implementing method 500 may receive an image from a user including at least one part of a plant to be identified. It should be noted that, the image mentioned herein includes at least one part of the plant, which means including one or more parts of the plant, wherein each part may be the whole or part of the part. The image can be previously stored by the user, captured in real time, or downloaded from the Internet. Imagery may include any form of visual representation, such as still images, moving images, and video. Images can be taken with devices including cameras, such as mobile phones, tablet computers, and the like.

In step 520 , the method 500 recognizes at least one of plant parts, growth locations and growth cycles in the images, and classification of the plants in the images based on the images and the trained neural network model. For example, the trained plant recognition model described above can be used to identify the classification of plants in the image, and the trained part classification model, growth place classification model and growth cycle classification model described above can be used to separately identify the classification of plants in the image. The parts, growth places and growth cycles of the plants.

In step 530, the method 500 performs the requested operation on at least one of the identified plant parts, growth locations, and growth cycles and plant classifications according to the user's operation request. In one embodiment, the user's operation request is a request for the identified information, for example, requesting to output the identified information, then the application program capable of implementing the method 500 may output (eg, through the interface of the application program) the identified plant information. At least one of the plant's part, growth place and growth cycle, and information on the classification of the plant.

In one embodiment, the user's operation request is a request for a maintenance scheme for plants, and the application program capable of implementing method 500 can call the maintenance scheme determination module of the application program, and according to the identified plant location, growth place and growth cycle at least one of them and the classification of the plants to determine the plant maintenance scheme, and output the determined plant maintenance scheme to the user (for example, through the interface of the application program). The maintenance program determination module can pre-establish the maintenance program lookup table shown in Table 1 as described in the method 100, and determine the plant maintenance program based on the maintenance program lookup table. In one embodiment, the user's request for a plant maintenance plan may be a request for a disease and pest control plan. The application program capable of implementing the method 500 can use the trained pest diagnosis model to identify the types of plant diseases and insect pests according to the image (also referred to as "diagnostic information of diseases and insect pests"), and can use the identified plant classification and pest diagnosis information, and According to at least one of the plant's growth location, growth cycle, and plant parts in the image, the maintenance plan related to pest control is extracted from the established database, thereby recommending a personalized maintenance plan related to pest control to the user.

Fig. 6 is a flowchart schematically illustrating at least part of a method 600 for plant identification according to yet another embodiment of the present disclosure. The method 600 includes: receiving an image, wherein the image includes at least one part of a plant (step 610); according to the image, based on the trained neural network model, identifying the classification of the plant in the image, the part of the plant in the image, and the location of the plant in the image. The growing place, the growth cycle of the plants in the image, and the image quality of the image (step 620); and according to the user's operation request, select one or more identified contents to perform the requested operation (step 630). Different from method 500, method 600 identifies the classification, location, growth place, growth cycle and image quality of the plants in the received images, and then selects one of them according to needs (need to be determined according to the user's operation request) or multiple items to be used in subsequent steps. In one embodiment, each item of content identified in step 620 is stored in association with the corresponding image, for example, may be stored as each tag of the image for subsequent use. In this way, after the user's request is subsequently received, it is not necessary to re-identify the image to obtain the required information, but to directly extract the required information from the tag information of the picture.

In step 610, an application capable of implementing method 600 may receive an image from a user including at least one part of a plant to be identified. It should be noted that, the image mentioned herein includes at least one part of the plant, which means including one or more parts of the plant, wherein each part may be the whole or part of the part. The image can be previously stored by the user, captured in real time, or downloaded from the Internet. Imagery may include any form of visual representation, such as still images, moving images, and video. Images can be taken with devices including cameras, such as mobile phones, tablet computers, and the like.

In step 620 , the method 600 can identify information such as classification, location, growth place, growth cycle, and image quality of plants in the image based on the trained neural network model based on the image. For example, the trained plant recognition models described above can be used to identify the classification of plants in imagery, using the trained parts classification models, growth location classification models, growth cycle classification models, and quality classification models described above To identify the part, growth place, growth cycle and image quality of the plants in the image respectively. As mentioned above, in the identification step, the method 600 identifies information such as classification, location, growth place, growth cycle, and image quality of the plants in the received images for selective use in subsequent steps.

In step 630, the method 600 may select one or more contents among all the contents identified in step 620 to perform the requested operation according to the user's operation request. In one embodiment, the user's operation request is a request for identified information, for example, a request to output information about one or more items of identified content. Then the application program capable of implementing method 600 may select one or more contents from all the contents identified in step 620 according to the user's request, and output (for example, through the interface of the application program) the selected one or more contents Information.

In one embodiment, the user's operation request is a request for a plant maintenance plan. Then the application program capable of implementing the method 600 can select at least one of the part, growth location, and growth cycle of the plant in the image identified in step 620, as well as the classification of the plant in the image, and call the maintenance program of the application program to determine The module is used to determine the maintenance program of the plant, and output the determined plant maintenance program to the user (for example, through the interface of the application program). The maintenance program determination module can pre-establish the maintenance program lookup table shown in Table 1 as described in the method 100, and determine the plant maintenance program based on the maintenance program lookup table. In one embodiment, the user's request for a plant maintenance plan may be a request for a disease and pest control plan. The application program capable of implementing the method 600 can use the trained pest diagnosis model to identify the types of plant diseases and insect pests according to the image (also referred to as "diagnostic information of diseases and insect pests"), and can use the identified plant classification and pest diagnosis information, and According to at least one of the growth location, growth cycle and part of the plant, the maintenance plan related to pest control is extracted from the established database, so as to recommend a personalized maintenance plan related to pest control to the user.

FIG. 3 is a structural diagram schematically showing at least a part of a computer system 300 for plant identification according to an embodiment of the present disclosure. Those skilled in the art will appreciate that system 300 is merely an example and should not be considered as limiting the scope of the disclosure or the features described herein. In this example, system 300 may include one or more storage devices 310 , one or more user devices 320 , and one or more computing devices (computer systems) 330 , which may be communicatively connected to each other via a network or bus 340 . One or more storage devices 310 provide storage services for one or more user devices 320 , and one or more computing devices 330 . Although one or more storage devices 310 are shown in system 300 as a separate block from one or more user devices 320 and one or more computing devices 330, it should be understood that one or more storage devices 310 May actually be stored on any of the

other entities

320, 330 included in the system 300. Each of the one or more user devices 320 and the one or more computing devices 330 may be located at different nodes of the network or bus 340 and be capable of communicating directly or indirectly with other nodes of the network or bus 340 . Those skilled in the art can understand that the system 300 may also include other devices not shown in FIG. 3 , where each different device is located at a different node of the network or bus 340 .

One or more storage devices 310 may be configured to store any data mentioned above, including but not limited to: images input from users, image samples, neural network models, recognition results, application files and other data. One or more computing devices 330 may be configured to perform one or more of the above-mentioned methods according to the embodiments, and/or one or more steps in the one or more methods according to the embodiments. One or more user devices 320 may be configured to provide services to the user, for example, receiving images from the user, outputting maintenance plans for plants, outputting plant classifications, and outputting information prompting users to re-input images, and the like. One or more user equipments 320 may also be configured to execute one or more of the above methods according to the embodiments, and/or one or more steps in the one or more methods according to the embodiments.

Network or bus 340 may be any wired or wireless network, and may include cables. Network or bus 340 may be part of the Internet, the World Wide Web, a specific intranet, a wide area network or a local area network. Network or bus 340 may utilize standard communication protocols such as Ethernet, WiFi, and HTTP, protocols proprietary to one or more companies, and various combinations of the foregoing. The network or bus 340 may also include, but is not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a peripheral component interconnect (PCI) bus.

Each of the one or more user equipment 320 and the one or more computing devices 330 may be configured similarly to the system 400 shown in FIG. And instruction 421 and data 422. Each of the one or more user devices 320 and the one or more computing devices 330 may be a personal computing device intended for use by a user or a business computing device for use by an All components used in conjunction, such as the central processing unit (CPU), memory for storing data and instructions (e.g., RAM and internal hard drives), such as displays (e.g., monitors with screens, touch screens, projectors, televisions, or operable other devices to display information), mouse, keyboard, touch screen, microphone, speakers, and/or one or more I/O devices such as network interface devices.

One or more user devices 320 may also include one or more cameras for capturing still images or recording video streams, and all components for connecting these elements to each other. While one or more user devices 320 may each comprise a full-sized personal computing device, they may alternatively comprise a mobile computing device capable of wirelessly exchanging data with a server over a network, such as the Internet. One or more user devices 320 may be, for example, a mobile phone, or a device such as a PDA with wireless support, a tablet PC, or a netbook capable of obtaining information via the Internet. In another example, one or more user devices 320 may be a wearable computing system.

FIG. 4 is a structural diagram schematically showing at least a part of a computer system 400 for plant identification according to an embodiment of the present disclosure. System 400 includes one or more processors 410, one or more memories 420, and other components (not shown) typically found in a computer or the like. Each of the one or more memories 420 can store content that can be accessed by the one or more processors 410, including instructions 421 that can be executed by the one or more processors 410, and that can be executed by the one or more processors 410. Data 422 retrieved, manipulated or stored.

Instructions 421 may be any set of instructions to be executed directly by one or more processors 410, such as machine code, or indirectly, such as a script. The terms "instruction", "application", "process", "step" and "program" are used interchangeably herein. Instructions 421 may be stored in object code format for direct processing by one or more processors 410, or in any other computer language, including scripts or collections of stand-alone source code modules interpreted on demand or compiled ahead of time. Instructions 421 may include instructions that cause, for example, one or more processors 410 to function as various neural networks herein. The function, method and routine of instruction 421 are explained in more detail elsewhere herein.

The one or more memories 420 may be any temporary or non-transitory computer-readable storage media capable of storing content accessible by the one or more processors 410, such as hard drives, memory cards, ROM, RAM, DVDs, CDs, USB memory, writable memory and read-only memory, etc. One or more of the one or more memories 420 may comprise a distributed storage system where instructions 421 and/or data 422 may be stored on multiple different storage devices which may be physically located at the same or different geographic locations. One or more of the one or more memories 420 may be connected to the one or more first devices 410 via a network, and/or may be directly connected to or incorporated in any of the one or more processors 410 .

One or more processors 410 may retrieve, store or modify data 422 according to instructions 421 . The data 422 stored in the one or more memories 420 may include at least a portion of one or more of the items stored in the one or more storage devices 310 described above. For example, while the subject matter described herein is not limited to any particular data structure, data 422 could also be stored in computer registers (not shown), as tables or XML documents with many different fields and records stored in relational type database. Data 422 may be formatted in any computing device readable format, such as, but not limited to, binary values, ASCII, or Unicode. Additionally, data 422 may include any information sufficient to identify the relevant information, such as numbers, descriptive text, proprietary code, pointers, references to data stored in other storage, such as at other network locations, or used by functions to compute relevant information. data information.

The one or more processors 410 may be any conventional processor, such as a commercially available central processing unit (CPU), graphics processing unit (GPU), or the like. Alternatively, one or more processors 410 may also be a dedicated component, such as an application specific integrated circuit (ASIC) or other hardware-based processor. Although not required, one or more processors 410 may include specialized hardware components to more quickly or efficiently perform certain computational processes, such as image processing of imagery and the like.

Although one or more processors 410 and one or more memories 420 are schematically shown in the same box in FIG. 4 , system 400 may actually include multiple Multiple processors or memory within a physical enclosure. For example, one of the one or more memories 420 may be a hard drive or other storage medium located in a different housing than that of each of the one or more computing devices (not shown) described above . Accordingly, references to a processor, computer, computing device or memory shall be understood to include references to a collection of processors, computers, computing devices or memory which may or may not operate in parallel.

The word "A or B" in the specification and claims includes "A and B" and "A or B", and does not exclusively include only "A" or only "B", unless specifically stated otherwise.

In the present disclosure, reference to "one embodiment" or "some embodiments" means that a feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment, at least some embodiments of the present disclosure. Thus, appearances of the phrase "in one embodiment" and "in some embodiments" in various places in this disclosure are not necessarily referring to the same embodiment or embodiments. Furthermore, features, structures or characteristics may be combined in any suitable combination and/or subcombination in one or more embodiments.

As used herein, the word "exemplary" means "serving as an example, instance, or illustration" rather than as a "model" to be exactly reproduced. Any implementation described illustratively herein is not necessarily to be construed as preferred or advantageous over other implementations. Furthermore, the disclosure is not to be bound by any expressed or implied theory presented in the preceding technical field, background, brief summary or detailed description.

In addition, certain terms may also be used in the following description for reference purposes only, and thus are not intended to be limiting. For example, the words "first," "second," and other such numerical terms referring to structures or elements do not imply a sequence or order unless clearly indicated by the context. It should also be understood that when the word "comprises/comprises" is used herein, it indicates the presence of indicated features, integers, steps, operations, units and/or components, but does not exclude the presence or addition of one or more other features, whole, steps, operations, units and/or components and/or combinations thereof.

In this disclosure, the terms "component" and "system" are intended to refer to a computer-related entity, either hardware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to being, a process, object, executable, thread of execution, and/or program running on a processor. By way of example, both an application running on a server and the server may be a component. One or more components can reside within a process and/or thread of execution and a component can be localized on one computer and/or distributed between two or more computers.

In addition, implementations of the present disclosure may also include the following examples:

1. A method for plant identification comprising:

receiving an image, wherein the image includes at least one part of a plant;

According to the image, identifying at least one of the location, growth site, and growth cycle of the plant in the image and the classification of the plant in the image based on the trained neural network model; and

According to the user's operation request, the requested operation is performed on the identified plant classification and the at least one item.

2. The method of 1, wherein the operation request comprises a request for identified information, the method further comprising:

The identified classification of the plant and the at least one item of information are output.

3. The method according to 1, wherein the operation request includes a request for a plant maintenance program, and the method also includes:

determining a maintenance regimen for the plant based on the identified classification of the plant and the at least one item; and

A maintenance regimen for the plant is output.

4. The method according to 3, wherein the maintenance plan includes watering, spraying water, changing water, adding water, fertilizing, pruning, weeding, turning pots, changing pots, sunshine, shading, adjusting temperature, adjusting humidity, winter protection , and an implementation plan for at least one of pest control.

5. The method according to 1, wherein said at least one item includes a part of a plant in said image, said method further comprising:

Determining a taxonomic level of the taxonomy of the plant based on the identified parts of the plant in the image; and

The result of the identified classification of the plant is adjusted according to the determined classification level.

6. The method according to 5, further comprising:

Determining that the taxonomic level is a genus in response to the part of the plant in the image being one of trunk, bud, seed, bud, fruit, and seedling; and

In response to the part of the plant in the image being one of leaf, flower, stem, and root, the classification level is determined to be a species.

7. The method according to 1, wherein, before the identification, further comprising:

determining an image quality of the image based on a trained quality classification model;

in response to the image quality being clear, performing the identifying; and

In response to the image quality being unclear, a prompt message to re-input the image is output.

8. The method according to 1, wherein the identification is performed based on the subject in the image, the method further comprising:

In response to the subject in the image being unclear, the subject being a non-plant, and the subject being a distant view of a whole plant, the recognition is not performed, and a prompt message for re-inputting the image is output.

9. The method according to 1, wherein, the growth place of the plant is identified based on a trained growth place classification model, and the growth place classification model is passed by multiple classifications under each classification of the growth place. Annotated samples are trained, wherein the multiple classifications of the growing places include potted plants, non-potted plants, and fresh-cut flowers.

10. The method according to 1, wherein the growth cycle of the plant is identified based on a trained growth cycle classification model passed by multiple subclasses under each of a plurality of classifications of growth cycles The marked samples are trained, wherein the multiple classifications of the growth cycle include emerging seedlings, young seedlings, leaf stage, flowering stage, fruit stage, leaf falling stage, and dormant stage.

11. The method of 1, wherein the parts of the plant in the image are identified based on a trained part classification model annotated by a plurality of each of the plurality of classifications of parts The samples of are trained, wherein the multiple classifications of the parts include trunk, bud, seed, bud, fruit, seedling, leaf, flower, stem, and root.

12. A method for plant identification comprising:

receiving an image, wherein the image includes at least one part of a plant;

According to the image, identify the classification of the plant in the image, the part of the plant in the image, the growth place of the plant in the image, and the growth cycle of the plant in the image based on the trained neural network model , and the image quality of said image; and

According to the user's operation request, one or more identified contents are selected from among the identified contents to perform the requested operation.

13. The method of 12, wherein the operation request comprises a request for identified information, the method further comprising:

Output information about one or more items selected.

14. The method according to 12, wherein the operation request comprises a request for a plant maintenance program, the method further comprising:

Selecting at least one of the part of the plant in the image, the growth place of the plant in the image, and the growth cycle of the plant in the image, and the classification of the plant in the image;

According to the selected classification of the plant and the at least one item, determine the maintenance program for the plant; and

A maintenance regimen for the plant is output.

15. The method according to 12, further comprising storing the identified items in association with the image.

16. A method for plant identification comprising:

Identifying the classification of the plant based on the trained neural network model according to the image, and identifying at least two of the growth location of the plant, the growth cycle, and the part of the plant in the image, wherein the image includes at least one of the plant a part;

According to the identified classification of the plant, and at least two of the identified growth location, growth cycle, and plant part in the image, determine a maintenance plan for the plant; and

A maintenance regimen for the plant is output.

17. A method for plant identification comprising:

receiving an image, wherein the image includes at least one part of the plant;

According to the image, identifying the parts of the plant in the image and the classification of the plant;

determining an output classification level based on the identified parts of the plant in the image; and

A classification of the corresponding classification level of the plant is output according to the determined classification level.

18. The method according to 17, further comprising:

In response to the part of the plant in the image being one of trunk, bud, seed, bud, fruit, and seedling, determining that the output classification level is genus; and

In response to the part of the plant in the image being one of leaf, flower, stem, and root, it is determined that the output classification level is species.

19. The method of 17, wherein the identifying is based on subjects in the images.

20. The method according to 19, further comprising:

In response to the subject in the image being unclear, the subject being a non-plant, and the subject being a distant view of a whole plant, the recognition is not performed, and information prompting the user to re-input the image is output.

21. A computer system for plant identification comprising:

one or more processors; and

one or more memories configured to store a series of computer-executable instructions and computer-accessible data associated with the series of computer-executable instructions,

Wherein, when the series of computer-executable instructions are executed by the one or more processors, the computer system is made to perform the method described in any one of 1-20.

22. A non-transitory computer-readable storage medium storing a series of computer-executable instructions on the non-transitory computer-readable storage medium, when the series of computer-executable instructions are executed by one or more computer systems When executed, the one or more computer systems are caused to perform the method described in any one of 1-20.

Those skilled in the art will appreciate that the boundaries between the above-described operations are merely illustrative. Multiple operations may be combined into a single operation, a single operation may be distributed among additional operations, and operations may be performed with at least partial overlap in time. Also, alternative embodiments may include multiple instances of a particular operation, and the order of operations may be altered in other various embodiments. However, other modifications, changes and substitutions are also possible. Accordingly, the specification and drawings are to be regarded as illustrative rather than restrictive.

Although some specific embodiments of the present disclosure have been described in detail through examples, those skilled in the art should understand that the above examples are for illustration only, rather than limiting the scope of the present disclosure. The various embodiments disclosed herein can be combined arbitrarily without departing from the spirit and scope of the present disclosure. Those skilled in the art will also appreciate that various modifications may be made to the embodiments without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

A method for plant identification, comprising:

receiving an image, wherein the image includes at least one part of a plant;

According to the image, identifying at least one of the location, growth site, and growth cycle of the plant in the image and the classification of the plant in the image based on the trained neural network model; and

According to the user's operation request, the requested operation is performed on the identified plant classification and the at least one item.
The method according to claim 1, wherein the operation request comprises a request for identified information, the method further comprising:

The identified classification of the plant and the at least one item of information are output.
The method according to claim 1, wherein the operation request comprises a request for a plant maintenance program, and the method further comprises:

determining a maintenance regimen for the plant based on the identified classification of the plant and the at least one item; and

A maintenance regimen for the plant is output.
The method according to claim 3, wherein the maintenance program includes watering, spraying water, changing water, adding water, fertilizing, pruning, weeding, turning pots, changing pots, sunshine, shading, adjusting temperature, and adjusting humidity , overwintering protection, and an implementation plan for at least one of pest control.
The method according to claim 1, wherein said at least one item includes parts of plants in said image, said method further comprising:

Determining a taxonomic level of the taxonomy of the plant based on the identified parts of the plant in the image; and

The result of the identified classification of the plant is adjusted according to the determined classification level.
The method according to claim 5, further comprising:

Determining that the taxonomic level is a genus in response to the part of the plant in the image being one of trunk, bud, seed, bud, fruit, and seedling; and

In response to the part of the plant in the image being one of leaf, flower, stem, and root, the classification level is determined to be a species.
The method according to claim 1, further comprising:

determining an image quality of the image based on a trained quality classification model;

in response to the image quality being clear, performing the identifying; and

In response to the image quality being unclear, a prompt message to re-input the image is output.
The method according to claim 1, wherein the identification is performed according to the subject in the image, the method further comprising:

In response to the subject in the image being unclear, the subject being a non-plant, and the subject being a distant view of a whole plant, the recognition is not performed, and a prompt message for re-inputting the image is output.
The method according to claim 1, wherein the growth place of the plant is identified based on a trained growth place classification model, and the growth place classification model is based on multiple classifications under each classification of a plurality of growth place classifications. labeled samples are trained, wherein the multiple classifications of the growing places include potted plants, non-potted plants, and fresh-cut flowers.
The method according to claim 1, wherein the growth cycle of the plant is identified based on a trained growth cycle classification model, and the growth cycle classification model passes multiple classifications under each of a plurality of classifications of the growth cycle. The labeled samples are trained, wherein the multiple classifications of the growth cycle include emerging seedlings, young seedlings, leaf stage, flowering stage, fruit stage, leaf falling stage, and dormancy stage.
The method according to claim 1, wherein the part of the plant in the image is identified based on a trained part classification model, the part classification model passing multiple Annotated samples are trained, wherein multiple classifications of parts include trunk, bud, seed, bud, fruit, seedling, leaf, flower, stem, and root.
A method for plant identification, comprising:

receiving an image, wherein the image includes at least one part of a plant;

According to the image, identify the classification of the plant in the image, the part of the plant in the image, the growth place of the plant in the image, and the growth cycle of the plant in the image based on the trained neural network model , and the image quality of said image; and

According to the user's operation request, one or more identified contents are selected from among the identified contents to perform the requested operation.
The method of claim 12, wherein the operation request comprises a request for identified information, the method further comprising:

Output information about one or more items selected.
The method according to claim 12, wherein the operation request comprises a request for a plant maintenance program, and the method further comprises:

Selecting at least one of the part of the plant in the image, the growth place of the plant in the image, and the growth cycle of the plant in the image, and the classification of the plant in the image;

According to the selected classification of the plant and the at least one item, determine the maintenance program for the plant; and

A maintenance regimen for the plant is output.
The method according to claim 12, further comprising storing the identified items in association with the image.
A method for plant identification, comprising:

Identifying the classification of the plant based on the trained neural network model according to the image, and identifying at least two of the growth location of the plant, the growth cycle and the part of the plant in the image, wherein the image includes at least one of the plant a part;

According to the identified classification of the plant, and at least two of the identified growth location, growth cycle, and plant part in the image, determine a maintenance plan for the plant; and

A maintenance regimen for the plant is output.
A method for plant identification, comprising:

receiving an image, wherein the image includes at least one part of the plant;

According to the image, identifying the parts of the plant in the image and the classification of the plant;

determining an output classification level based on the identified parts of the plant in the image; and

A classification of the corresponding classification level of the plant is output according to the determined classification level.
The method according to claim 17, further comprising:

In response to the part of the plant in the image being one of trunk, bud, seed, bud, fruit, and seedling, determining that the output classification level is genus; and

In response to the part of the plant in the image being one of leaf, flower, stem, and root, it is determined that the output classification level is species.
The method of claim 17, wherein said identifying is performed based on a subject in said image.
The method according to claim 19, further comprising:

In response to the subject in the image being unclear, the subject being a non-plant, and the subject being a distant view of a whole plant, the recognition is not performed, and information prompting the user to re-input the image is output.
A computer system for plant identification, characterized in that it comprises:

one or more processors; and

one or more memories configured to store a series of computer-executable instructions and computer-accessible data associated with the series of computer-executable instructions,

Wherein, when the series of computer-executable instructions are executed by the one or more processors, the computer system is caused to perform the method according to any one of claims 1-20.
A non-transitory computer-readable storage medium, wherein a series of computer-executable instructions are stored on the non-transitory computer-readable storage medium, and when the series of computer-executable instructions are executed by one or more When the computer system is executed, the one or more computer systems are made to perform the method according to any one of claims 1-20.