WO2021223607A1

WO2021223607A1 - Method and system for evaluating state of plant, and computer readable storage medium

Info

Publication number: WO2021223607A1
Application number: PCT/CN2021/089485
Authority: WO
Inventors: 徐青松; 李青
Original assignee: 杭州睿琪软件有限公司
Priority date: 2020-05-07
Filing date: 2021-04-25
Publication date: 2021-11-11
Also published as: CN113627216B; CN113627216A

Abstract

Provided are a method and a system for evaluating the state of a plant, and a computer readable storage medium. Said method comprises: receiving a plant image, and recognizing a plant in the plant image so as to obtain the category of the plant; recognizing parts of the plant contained in the plant image; determining the growth state of the parts; and comprehensively evaluating the maintenance state of the plant according to the growth state of the parts. By means of artificial intelligence, the present invention can easily, quickly and accurately evaluate the maintenance state of a plant.

Description

Plant state assessment method, system and computer readable storage medium

Technical field

The present invention relates to the field of artificial intelligence technology, in particular to a plant state assessment method, system and computer readable storage medium.

Background technique

With the improvement of living standards, people have higher and higher requirements for the living environment. Indoors and outdoors will always adjust the living environment by planting green plants. Green plants can also relieve visual fatigue and increase the interest of life in the office. The existing green plants need to be carefully managed and cultivated, and especially the asparagus, aloe, and succulent plants have high requirements for their growth environment, and often die from drought and waterlogging. Only the green plants can grow tenaciously and their growth conditions are simple. , It can survive with regular hydration. Even if there is a lack of water, it can grow roots on the side of the cane to absorb moisture from the air. It has a tenacious life and can absorb harmful indoor gases. It is widely cultivated in homes, offices and other places.

However, when people plant green plants, the maintenance methods are different, and the maintenance state of green plants is different. Therefore, people usually want to know whether their green plants are in good maintenance state.

Summary of the invention

The purpose of the present invention is to provide a plant state assessment method, system and computer-readable storage medium to easily, quickly and accurately assess the conservation state of plants. The specific technical solutions are as follows:

In order to achieve the above objective, the present invention provides a plant state assessment method, which includes:

Receiving a plant image, and identifying plants in the plant image to obtain the type of the plant;

Identifying the plant parts contained in the plant image;

Determining the growth state of each part of the plant;

According to the growth state of each part of the plant, the conservation state of the plant is comprehensively evaluated.

Optionally, the identifying the part of the plant included in the plant image includes:

The plant part recognition model established by pre-training is used to recognize plants in the plant image to obtain the contained plant parts, and the plant part recognition model is a neural network model.

Optionally, determining the growth state of each part of the plant includes:

The plant state recognition model established by pre-training is used to separately recognize each part to obtain the growth state of each part, and the plant state recognition model is a neural network model.

Optionally, when it is determined that the growth state of a part is abnormal, the method further includes:

Calling a pre-trained pest identification model to identify the plant image, to determine whether the plant has a pest, and obtain at least one candidate pest information if there is a pest;

The at least one candidate pest information is screened in combination with the species of the plant to determine the pest information of the plant.

Optionally, the growth status is divided into multiple levels according to the degree of quality from high to low. When it is determined that the growth status of a part is not higher than a preset level, the possible reasons for the low growth status of the part and the countermeasures are output method.

Optionally, determining the growth state of each part of the plant includes:

The growth state of each part of the plant is determined in combination with the type of the plant.

Optionally, when identifying plants in the plant image to obtain the type of the plant, identifying the current growth cycle of the plant;

The determining the growth state of each part of the plant includes:

Combining with the current growth cycle of the plant, the growth state of each part of the plant is determined.

Optionally, if the plant state recognition model cannot recognize the growth state of a part, the determining the growth state of each part of the plant includes:

Collect maintenance information of the plant from the user, and determine the growth state of the part according to the maintenance information.

Optionally, the conservation information of the plant is collected from the user in the form of a questionnaire, and the topic in the questionnaire is determined according to the type of the plant.

Optionally, according to the recognition result of each part of the plant by the plant state recognition model, it is determined whether the conservation state of the plant is excellent, and if so, the plant image is recommended for sharing.

Optionally, a pre-trained aesthetic recognition model is used to identify plants in the plant image to obtain the degree of beauty of the plant, and the aesthetic recognition model is a neural network model.

Optionally, the degree of beauty is divided into multiple levels from high to low, and if the plant has a good conservation state and a high degree of beauty, then the plant image is recommended for sharing.

Based on the same inventive concept, the present invention also provides a plant state assessment system. The system includes a processor and a memory. The memory stores instructions. When the instructions are executed by the processor, the plant state assessment method is implemented. The method includes: receiving a plant image, identifying plants in the plant image to obtain the type of the plant; identifying the plant parts contained in the plant image; determining the growth state of each part; The growth state of the part, the comprehensive evaluation of the conservation state of the plant.

Based on the same inventive concept, the present invention also provides a computer-readable storage medium with instructions stored on the computer-readable storage medium. When the instructions are executed, the steps of the plant state assessment method are implemented, and the method includes : Receiving plant images, identifying plants in the plant images to obtain the types of plants; identifying the plant parts contained in the plant images; determining the growth status of each part; comprehensively assessing the growth status of each part The conservation status of the plant.

Compared with the prior art, the plant state assessment method, system and computer-readable storage medium provided by the present invention have the following advantages:

The present invention receives the plant image uploaded by the user, recognizes the plant in the plant image to obtain the type of the plant, and recognizes the plant parts contained in the plant image, and then determines the growth state of each part, thereby According to the growth state of each part, the maintenance state of the plant is comprehensively evaluated. The invention objectively determines the growth state of each part of the plant and comprehensively evaluates it through artificial intelligence, so that the conservation state of the plant can be easily, quickly and accurately evaluated.

Description of the drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

Fig. 1 is a schematic diagram of a network environment of a plant state assessment system provided by an embodiment of the present invention;

Fig. 2 is a schematic flow chart of a method for assessing the state of plants according to an embodiment of the present invention;

Fig. 3 is a schematic structural diagram of a plant state assessment system provided by an embodiment of the present invention.

Detailed ways

The method, system, and computer-readable storage medium for plant state evaluation proposed by the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. According to the following description, the advantages and features of the present invention will be clearer. It should be noted that the drawings all adopt a very simplified form and all use imprecise proportions, which are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention. It should be noted that the structure, ratio, size, etc. shown in the accompanying drawings in this specification are only used to match the content disclosed in the specification for the understanding and reading of those who are familiar with this technology, and are not intended to limit the implementation of the present invention. Conditions, so it has no technical significance. Any structural modification, proportional relationship change, or size adjustment should still fall within the scope of the present invention without affecting the effects and objectives that can be achieved by the present invention. The disclosed technical content can cover the range.

The inventor of the present application has conducted in-depth research on methods and systems for plant state assessment. Fig. 1 shows a schematic diagram of a network environment 100 of a plant state assessment system according to an embodiment of the present invention.

The network environment 100 of the plant state assessment system may include a mobile device 102, a remote server 103, a training device 104, and a database 105, which are wired or wirelessly coupled to each other through the network 106. The network 106 may be embodied as a wide area network (such as a mobile phone network, a public switched telephone network, a satellite network, the Internet, etc.), a local area network (such as Wi-Fi, Wi-Max, ZigBeeTM, BluetoothTM, etc.) and/or other forms of networking functions.

The mobile device 102 may include a mobile phone, a tablet computer, a laptop computer, a personal digital assistant, and/or other computing devices configured to capture, store, and/or transmit images such as digital photos. Therefore, the mobile device 102 may include an image capture device such as a digital camera and/or may be configured to receive images from other devices. The mobile device 102 may include a display. The display may be configured to provide one or more user interfaces to the user 101, the user interface may include multiple interface elements, the user 101 may interact with the interface elements, and so on. For example, the user 101 can use the mobile device 102 to photograph a certain plant and upload or store the image. The mobile device 102 can output to the user the detailed introduction of the plant’s category information, conservation status, etc., or can push the user the reasons for the poor conservation status and the countermeasures, as well as the excellent conservation of plants, and remind the user of the possible existence of the plant. Pest information, etc.

The remote server 103 can be configured to analyze images received from the mobile device 102 via the network 106 to determine the type of plant, and to identify the plant's parts and detailed information such as the growth status of each part, and then comprehensively evaluate the conservation of the plant state. The remote server 103 may also be configured to create and train the plant species recognition model, plant part recognition model, plant state recognition model, and aesthetic recognition model of this embodiment. The specific training process of the plant species recognition model, the plant part recognition model, the plant state recognition model, and the beauty recognition model will be described below in conjunction with specific embodiments. In other embodiments, the remote server 103 may only be used to train each of the above-mentioned recognition models, and then deploy each of the trained recognition models directly on the client mobile device 102, and the client mobile device 102 can compare the acquired plant images For subsequent identification processing, it is also possible to update each identification model in the client mobile device 102 via the network 106 when necessary.

The training device 104 may be coupled to the network 106 to facilitate the training of the plant species recognition model, the plant part recognition model, the plant state recognition model, and the aesthetic recognition model. The training device 104 may have multiple CPUs and/or GPUs to assist in training the plant species recognition model, the plant part recognition model, the plant state recognition model, and the aesthetic recognition model.

The database 105 may be coupled to the network 106 and provide data required by the remote server 103 for relevant calculations. For example, the database 105 may include a sample library storing a large number of images of different types of plants, and a sample library of images of multiple varieties of plants under the same category. In one embodiment, taking S. sylvestris as an example, the sample library may include a large number of image samples of different varieties of S. sylvestris in different locations, different seasons, weather at different times, and different shooting angles. In one embodiment, the selected plant photos taken by the user can also be stored in a sample library corresponding to the plant species. At the same time, the location information, seasonal information, and time information of the plant can also be recorded in the database. One or more corresponding menstrual cycle information and morphological information among weather information and shooting angle information. The database can be implemented by various database technologies known in the art. The remote server 103 can access the database 105 to perform related operations as needed.

It should be understood that the network environment 100 herein is only an example. Those skilled in the art can add more devices or delete some devices as needed, and can modify the functions and configurations of some devices.

The following describes a plant state assessment method provided by an embodiment of the present invention with reference to FIG. 2. As shown in FIG. 2, the method for assessing the state of plants provided by an embodiment of the present invention includes the following steps:

Step S101, receiving a plant image, and identifying plants in the plant image to obtain the type of the plant.

As mentioned earlier, the received plant image may be previously stored by the user or captured by the user in real time. For example, the plant image may be previously stored in the mobile device 102 by the user or captured in real time by the user using an external camera connected to the mobile device 102 or a built-in camera of the mobile device 102. In an embodiment, the user can also obtain the plant image in real time via the network.

In one embodiment, a plant species recognition model established by pre-training may be used to identify plants in the plant image to obtain the plant species. The training step of the plant species recognition model may include: step a, obtaining a training sample set, each sample in the training sample set is labeled with the type of plant; step b, obtaining a test sample set, each of the test sample set The sample is also marked with the types of plants, where the test sample set is different from the training sample set; step c, training the plant species recognition model based on the training sample set; step d, based on the test The sample set tests the plant species recognition model; step e, when the test result indicates that the recognition accuracy rate of the plant species recognition model is less than the preset accuracy rate, increase the number of samples in the training sample set for re-training And step f, when the test result indicates that the recognition accuracy of the plant species recognition model is greater than or equal to the preset accuracy, the training is completed.

For example, a certain number of image samples labeled with corresponding information are acquired for each plant type, and the number of image samples prepared for each plant type may be equal or different. The corresponding information annotated for each image sample may include the type of plant in the image sample, each part of the plant, the conservation state of each part, and so on. The image samples obtained for each plant species can include as much as possible the different shooting angles, different lighting conditions, different weather (for example, the same plant may have different forms in sunny days and rainy days), different months or seasons (for example, the same The shape of plants may be different in different months or seasons), different times (for example, the shape of the same plant may be different in the morning and night of each day), different growth environments (for example, the shape of the same plant may be different indoors and outdoors), and different geographies An image of a location (for example, the same plant may grow differently in different geographic locations). In these cases, the corresponding information labeled for each image sample may also include information such as the shooting angle, illumination, weather, season, time, growth environment, or geographic location of the image sample.

The image samples that have undergone the above-mentioned annotation processing can be divided into a training sample set for training the plant species recognition model and a test sample set for testing the training result. Generally, the number of samples in the training sample set is significantly greater than the number of samples in the test sample set. For example, the number of samples in the test sample set can account for 5% to 20% of the total number of image samples, and the corresponding training sample set The number of samples can account for 80% to 95% of the total number of image samples. Those skilled in the art should understand that the number of samples in the training sample set and the test sample set can be adjusted as needed.

The training sample set can be used to train the plant species recognition model, and the test sample set can be used to test the recognition accuracy of the trained plant species recognition model. If the recognition accuracy does not meet the requirements, increase the number of image samples in the training sample set, and use the updated training sample set to retrain the plant species recognition model until the recognition accuracy of the trained plant species recognition model meets the requirements . If the recognition accuracy meets the requirements, the training ends. In one embodiment, it can be judged whether the training can end based on whether the recognition accuracy rate is less than the preset accuracy rate. In this way, the trained plant species recognition model whose output accuracy meets the requirements can be used for plant species recognition.

The plant species recognition model is a neural network model, such as a deep convolutional neural network (CNN) or a deep residual network (Resnet). Among them, the deep convolutional neural network is a deep feedforward neural network, which uses the convolution kernel to scan the plant image, extracts the features to be recognized in the plant image, and then recognizes the features to be recognized in the plant. In addition, in the process of recognizing the plant image, the original plant image can be directly input to the deep convolutional neural network model without preprocessing the plant image. Compared with other recognition models, the deep convolutional neural network model has higher recognition accuracy and recognition efficiency. Compared with the deep convolutional neural network model, the deep residual network model adds an identity mapping layer, which can avoid the accuracy saturation or even saturation caused by the convolutional neural network as the network depth (the number of layers in the network) increases. The phenomenon of decline. The identity mapping function of the identity mapping layer in the residual network model needs to satisfy: the sum of the input of the identity mapping function and the residual network model is equal to the output of the residual network model. After the introduction of identity mapping, the residual network model changes the output more obviously, so it can greatly improve the recognition accuracy and recognition efficiency of plant physiological period recognition, and then improve the recognition accuracy and recognition efficiency of plants.

Step S102: Identify the plant parts included in the plant image.

The parts of the plant may include roots, stems, leaves, flowers, etc., and the plant image may include one or more parts of the plant, which is not limited in this embodiment. For example, when the user collects the plant image, he can take a panoramic picture of the plant. In this way, the plant image may include all parts of the plant, or only a partial picture of the plant, for example, only Shooting parts such as leaves, flowers, etc., so that the plant image only includes the partial parts of the plant. When the plant image contains multiple parts, all parts are identified and subsequent processing is performed on each part.

Specifically, a plant part recognition model established by pre-training may be used to recognize plants in the plant image, so as to obtain the contained plant parts. The training step of the plant part recognition model may include: step a, obtaining a training sample set, each sample in the training sample set is labeled with various parts of the plant; step b, obtaining a test sample set, the test sample set Each sample is also marked with various parts of the plant, wherein the test sample set is different from the training sample set; step c, training the plant part recognition model based on the training sample set; step d, based on the training sample set; The test sample set tests the plant part recognition model; step e, when the test result indicates that the recognition accuracy rate of the plant part recognition model is less than a preset accuracy rate, increase the number of samples in the training sample set to perform Training again; and step f, when the test result indicates that the recognition accuracy rate of the plant part recognition model is greater than or equal to the preset accuracy rate, the training is completed. The plant part recognition model is a neural network model, such as a deep convolutional neural network (CNN) or a deep residual network (Resnet).

During training, each plant species acquires a certain number of image samples labeled with corresponding information, and the number of image samples prepared for each plant species can be equal or unequal. The corresponding information annotated for each image sample may include various parts of the plant in the image sample, and so on. Among them, the training sample set and the test sample set can be the same sample set as the above-mentioned plant species identification model, or different sample sets can be used, which is not limited in this embodiment.

Step S103: Determine the growth state of each part.

After the various parts of the plant are identified, each part can be marked out so as to determine the growth state of each part. Preferably, each part can be sliced to obtain pictures of each part, and then the growth state can be determined according to the pictures of each part. For example, for the leaf parts of plants, when abnormal conditions such as yellow leaves, dead leaves, leaf spots, wormholes, etc. appear, it means that the growth status of the part is not good, and the maintenance method needs to be adjusted.

Specifically, the growth status can be divided into multiple levels according to the degree of quality from high to low, for example, it can be divided into four levels: excellent, good, fair, and poor. The growth status can also be scored and divided into multiple levels according to the score segment. grade. You can intuitively evaluate the growth status of a part by setting the level. Preferably, when it is determined that the growth state of a part is not higher than the preset level, the possible reasons for the low growth state of the part and the countermeasures are output. The preset level can be, for example, good. When it is recognized that the growth state of a part is not higher than good, that is, the growth state is average or poor, the possible reasons and countermeasures for the low growth state of the part are output at this time, so as to remind the user The conservation status of the plant is insufficient, and the conservation method needs to be adjusted or strengthened, and specific suggestions are given.

For example, when there are yellow leaves on the leaf part of a plant, it can be determined that the growth state of the plant is poor, and at this time, the possible reasons for the yellow leaves on the leaves and the countermeasures are output to the user. For example, first check whether there is rot at the roots, and the rot at the roots is likely to be caused by stagnant water, so the countermeasures can be to dry the roots or use soil fungicides for treatment. If the roots rot due to excessive watering, the watering needs to be improved. Method; Secondly, if the roots are not rotted, and the symptoms of yellowing leaves are green veins and yellowing between the veins, or spot-like yellowing, this may be yellowing due to lack of trace elements. The countermeasure is to supplement trace elements or nitrogen fertilizer; again; If the roots do not rot, but the leaves do not have the above symptoms, it may be that the roots are broken, because the roots are damaged by underground pests and the plant can't keep up with the moisture and turn yellow, or it may be because the leaf pests bite the petioles or stems. If the leaves turn yellow, the solution is to pour or spray pesticides. Finally, to determine whether it is caused by lack of water, the solution is to solve the problem of soil compaction by loosening the soil and supplementing water.

It should be noted that, for some special plants, such as Tiger Piran, the leaves themselves have markings, which is a normal state. Therefore, when determining the growth status of each part, it needs to be determined in conjunction with the type of plant. For example, when the plant is determined to be Tiger Piran, if it is recognized that the leaf of the plant has markings, it is normal for the plant to have markings on the leaves. Therefore, it should not be considered that the growth state of the leaves of the plant is not good. It can be seen that for special plants, the growth status should be further combined with plant species to avoid misidentification and improve the accuracy of identification. Specifically, these special plants can be counted and recorded in the database, and the special normal state they have can be recorded at the same time. For example, the tiger Piran can be recorded in the database and the leaf markings are recorded in the normal state. In this way, when the type of plant is identified, a search can be performed in the database to determine whether it belongs to a special plant, and if it belongs, the special normal state of the plant is obtained, and the growth of its corresponding part is assisted in confirming the growth in step S103 Whether the status is abnormal.

In addition, because the growth state of a plant is also related to its growth cycle, for example, when the plant is in the dead leaf period, it is normal for the plant's leaf parts to have yellow leaves and dead leaves. Therefore, when determining the growth state of each part It also needs to be combined with the current growth cycle of the plant to determine. Specifically, when the plant in the plant image is identified in step S101 to obtain the type of the plant, the current growth cycle of the plant is also identified at the same time, and then in step S103, the current growth cycle of the plant can be combined to determine each The growth status of the site. For example, when it is recognized that the current growth cycle of the plant is the dead leaf stage, even if it is recognized that yellow leaves and dead leaves are present in the leaf parts of the plant, it will not be considered that the leaf part of the plant is in a poor growth state. It can be seen that combining the growth cycle of plants to further determine the growth state can avoid misidentification and improve the accuracy of identification.

Specifically, the plant state recognition model established by pre-training can be used to identify each part separately to obtain the growth state of each part. The training step of the plant state recognition model may include: step a, obtaining a training sample set, each sample in the training sample set is labeled with various parts of the plant and the corresponding growth state; step b, obtaining a test sample set, so Each sample in the test sample set is also marked with various parts of the plant and the corresponding growth state, wherein the test sample set is different from the training sample set; step c, the plant state is compared based on the training sample set. The recognition model is trained; step d, the plant state recognition model is tested based on the test sample set; step e, when the test result indicates that the recognition accuracy rate of the plant state recognition model is less than a preset accuracy rate, Increase the number of samples in the training sample set for re-training; and step f, when the test result indicates that the recognition accuracy of the plant state recognition model is greater than or equal to the preset accuracy, the training is completed. The plant state recognition model is a neural network model, such as a deep convolutional neural network (CNN) or a deep residual network (Resnet).

During training, each plant species acquires a certain number of image samples labeled with corresponding information, and the number of image samples prepared for each plant species can be equal or unequal. The corresponding information annotated for each image sample may include various parts of the plant in the image sample and the corresponding growth state. The training sample set and the test sample set may use the same sample set as the above-mentioned plant species recognition model or plant part recognition model, or different sample sets, which is not limited in this embodiment.

It should be noted that when labeling the growth status of each part of the plant in the image sample, the growth status can be divided into multiple levels according to the degree of quality from high to low, such as excellent, good, and poor. However, it is necessary to distinguish between normal growth state and abnormal growth state when labeling special plants. As mentioned earlier, Tiger Piran is a special plant, and the markings on its leaf parts are normal growth conditions. Therefore, when the image sample is Tiger Piran, the markings on the leaf parts are normal, so when labeling the growth state, it cannot be labeled as poor, and it needs to be labeled according to the actual growth status. In this way, when the trained plant state recognition model is recognizing Tiger Piran, it can accurately determine whether its leaf part is a dead leaf or a normal leaf patch according to the plant species information, thereby avoiding misrecognition and improving the recognition accuracy.

In practical applications, for different plants, a large number of well-maintained sample images will be obtained during training. When the trained plant state recognition model is used to identify a certain plant image, if the plant image is similar to a certain well-maintained sample image to reach the desired level Set a threshold (for example, 80%) to determine the plant image as excellent, and score it according to the similarity of the feature value. For example, if the similarity of the feature value is 98%, then the plant image is regarded as an excellent conservation plant image , And scored 98 points. In the same way, the same processing is performed for images of plants that are well maintained, average, or poor. In this way, the growth and maintenance of plants can be accurately characterized by scoring. At the same time, when the plant image features extracted by the plant state recognition model for each part of the plant are close to the standard of excellent conservation, the conservation state of the plant can be judged to be excellent, so that the plant can be marked as excellent conservation Plants, and then recommend and share the plant images.

Further, if the plant state recognition model cannot accurately identify the growth state of a part, the maintenance information of the plant can also be collected from the user, and the growth state of the part can be determined according to the maintenance information. The maintenance information includes, for example, watering frequency, hydroponic or soil culture, indoor or outdoor, selection of approximate leaf state pictures, etc., and may also include other information that can reflect the growth state of plants. According to the maintenance information, it can be judged whether the growth environment and the maintenance method of the plant are appropriate, thereby helping to infer the growth state of the plant from the side. Specifically, the conservation information of the plant can be collected from the user in the form of a questionnaire, and the questions in the questionnaire will pop up in sequence on the interface of the mobile device, and the user can select options or fill in information. Since the maintenance methods of different types of plants are different, the questions in the questionnaire are preferably determined according to the types of plants. In practical applications, you can set up a set of questions for each plant species in advance and form a questionnaire. When the plant state recognition model fails to identify the growth state, the corresponding questionnaire is called according to the current plant species and presented to the user for the user to provide maintenance. information.

In addition, a pre-trained aesthetic recognition model can also be used to identify plants in the plant image, so as to obtain the degree of beauty of the plants. The degree of aesthetic feeling can be divided into multiple levels from high to low, such as beautiful, good, fair, etc., or can be divided into multiple levels according to score segments. If the plant is well maintained and has a high degree of beauty, the plant image can also be recommended for sharing. For example, a two-dimensional scoring method is used to comprehensively evaluate the conservation status and aesthetics of the plant, that is, to set weights for the conservation status and aesthetics respectively, and after obtaining the scores for the conservation and aesthetics of the plant, use the weights to calculate For the comprehensive score of the plant, a high plant score indicates a good conservation state and a high degree of beauty. Therefore, the plant with a higher score is recommended for sharing.

The aesthetic recognition model is a neural network model, such as a deep convolutional neural network (CNN) or a deep residual network (Resnet). The training step of the beauty recognition model may include: step a, obtaining a training sample set, each sample in the training sample set is labeled with the beauty degree of the plant; step b, obtaining a test sample set, each of the test sample set The sample is also marked with the degree of beauty of the plant, wherein the test sample set is different from the training sample set; step c, training the beauty recognition model based on the training sample set; step d, based on the test The sample set tests the beauty recognition model; step e, when the test result indicates that the recognition accuracy of the beauty recognition model is less than the preset accuracy, increase the number of samples in the training sample set for retraining; and Step f: When the test result indicates that the recognition accuracy of the aesthetic recognition model is greater than or equal to the preset accuracy, the training is completed.

During training, each plant species acquires a certain number of image samples labeled with corresponding information, and the number of image samples prepared for each plant species can be equal or unequal. The corresponding information annotated for each image sample may include the degree of beauty of the plants in the image sample, and the like. Among them, the training sample set and the test sample set may use the same sample set as the above-mentioned plant species recognition model, plant part recognition model, or plant state recognition model, or different sample sets, which are not limited in this embodiment.

It should be noted that when labeling the beauty of the plants in the image sample, the beauty can be divided into multiple levels according to the degree of quality from high to low, such as beautiful, good, and fair. However, because people's judgments on the beauty of plants are very subjective, different people may have different judgments on the beauty of the same plant. Therefore, when labeling the beauty of the plants in the image sample, multiple people can separately label, and then comprehensively judge the beauty of the plants based on the labeling results of multiple people. For example, an image sample is annotated by three persons, and when at least two have the same annotation result, the annotation result is used as the final annotation result of the image sample. In this way, the beauty labeling results of each image sample are more accurate, and the recognition accuracy of the trained beauty recognition model is also higher. When the trained beauty recognition model is used for plant beauty recognition, misrecognition can be avoided and the recognition accuracy can be improved. Rate.

In practical applications, for different plants, a large number of sample images with excellent aesthetics will be obtained during training. When the trained aesthetic recognition model is used to identify a plant image, if the plant image is similar to a sample image with excellent aesthetics, it will reach the expected level. Set a threshold (for example, 80%) to determine the plant image as excellent, and score it according to the similarity of the feature value. For example, if the similarity of the feature value is 98%, the plant image is regarded as a plant image with excellent aesthetics. , And scored 98 points. In the same way, the same processing is applied to plant images with good, average, or poor aesthetics. In this way, the degree of beauty of plants can be accurately characterized by the way of scoring. At the same time, when the plant image features extracted from the plant by the aesthetic recognition model are close to the standard of excellent aesthetics, the plant can be identified as a plant with excellent aesthetics, and then the plant image can be recommended for sharing.

Further, when it is determined that the growth state of a part is abnormal, for example, there is a wormhole, a pest identification model established by pre-training can also be called to identify the plant image, so as to determine whether the plant has pests, and whether there are pests or diseases. In the case of obtaining at least one candidate disease and insect pest information; at the same time, in combination with the type of the plant, the at least one candidate disease and insect pest information is screened to determine the disease and insect pest information of the plant. As mentioned above, to determine that a certain part of the growth state is abnormal, it can be judged by the plant state recognition model. When the plant state recognition model is used to recognize the growth state of the part, and the recognized state level is lower than the preset level, it is determined The growth state of this part is abnormal. For example, the preset level can be set to fair. When the recognized state level is fair or poor, it is determined that the growth state of the part is abnormal. At this time, it is necessary to further identify whether the abnormal growth state of the part is caused by plant diseases and insect pests.

Specifically, the pest identification model can identify whether there are pests and diseases in the plant image, and if so, it can also output at least one possible pest and disease as candidate pest information. Then, the candidate pest information can be screened in combination with the identified plant species, and the pests matching the plant species can be screened out, so as to avoid outputting inaccurate pest information to the user. For example, the disease and insect pest identification model recognizes a certain plant image and recognizes four kinds of pest information A, B, C, and D. If only the pest information A will appear on that kind of plant, the pest information can be determined A is the pests and diseases of the plant. If both of the pest information A and B may appear on the plant of this type, then it can be comprehensively judged whether the plant’s pest information is actually A or B based on the occurrence probability of these two pests and the recognition accuracy of the model. Preferably, if the plant disease and insect pest information is determined, the user can also output specific disease and insect pest information and countermeasures to remind the user that the plant has disease and insect pests and give specific suggestions to eliminate the plant diseases and insect pests. In addition, if the pest identification model fails to identify specific pest information, it can also guide users to upload additional pictures for pest identification, or guide users to use expert identification to manually diagnose pest information.

The pest identification model is a neural network model, for example, a deep convolutional neural network (CNN) or a deep residual network (Resnet). The training steps may include: step a, obtaining a training sample set, each sample in the training sample set is labeled with plant diseases and insect pests and disease and pest information; step b, obtaining a test sample set, each sample in the test sample set It is also marked whether there are plant diseases and insect pests and information on plant diseases and insect pests, wherein the test sample set is different from the training sample set; step c, training the pest identification model based on the training sample set; step d, based on the training sample set; The test sample set tests the disease and insect pest identification model; step e, when the test result indicates that the recognition accuracy rate of the disease and insect pest model is less than a preset accuracy rate, increase the number of samples in the training sample set for retraining; and Step f: When the test result indicates that the recognition accuracy rate of the pest identification model is greater than or equal to the preset accuracy rate, the training is completed.

During training, each plant species acquires a certain number of image samples labeled with corresponding information, and the number of image samples prepared for each plant species can be equal or unequal. The corresponding information annotated for each image sample may include whether the plants in the image sample have diseases and insect pests, and information on the diseases and insect pests. Among them, the training sample set and the test sample set can be the same sample set as the above-mentioned plant species recognition model, plant part recognition model, plant state recognition model, or aesthetic recognition model, or different sample sets can be used, which is not the case in this embodiment. Make a limit.

S104: Comprehensively evaluate the conservation status of the plant according to the growth status of each part.

After obtaining the growth status of each part in step S103, the maintenance status of the plant can be comprehensively evaluated. For example, if the growth status of each part is excellent, it can be determined that the maintenance status of the plant is excellent, if the growth status is average or poor If there are more parts, it can be determined that the conservation status of the plant is average or poor. In addition, it is preferable to use a scoring method to evaluate the conservation status of plants. For example, when identifying the growth status of each part as mentioned above, each part is scored, and then the scores of each part are added up. If the total score exceeds a certain threshold, the plant is considered to be in good conservation status or Excellent, on the contrary, if the total score is lower than a certain threshold, it is considered that the conservation status of the plant is average or poor.

In addition, when assessing the conservation status of plants, it can also be combined with pest information and aesthetics for comprehensive assessment. That is, when the growth status of each part of the plant is good or excellent, and there is no pest information, and the aesthetics is high, it is considered The conservation status of the plant is good or excellent.

To sum up, the plant state assessment method of the present invention, upon receiving a plant image uploaded by a user, recognizes the plant in the plant image to obtain the type of the plant, and recognizes the status of the plant contained in the plant image Parts, and then determine the growth status of each part, so as to comprehensively evaluate the conservation status of the plant according to the growth status of each part. The invention objectively determines the growth state of each part of the plant and comprehensively evaluates it through artificial intelligence, so that the conservation state of the plant can be easily, quickly and accurately evaluated.

Based on the same inventive concept, the present invention also provides a plant state assessment system. As shown in FIG. 3, the plant state assessment system 200 may include a processor 210 and a memory 220. The memory 220 stores instructions. When the instructions are executed by the processor 210, the steps in the plant state assessment method described above can be implemented. .

The processor 210 may perform various actions and processing according to instructions stored in the memory 220. Specifically, the processor 210 may be an integrated circuit chip with signal processing capability. The above-mentioned processor may be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, or discrete hardware components. Various methods, steps, and logical block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc., and may be an X86 architecture or an ARM architecture.

The memory 220 stores executable instructions, which are executed by the processor 210 in the above-mentioned plant state assessment method. The memory 220 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read only memory (ROM), programmable read only memory (PROM), erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM), which acts as an external cache. By way of exemplary but not restrictive description, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic Random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous connection dynamic random access memory (SLDRAM), and direct memory bus random access memory (DR RAM). It should be noted that the memory of the method described herein is intended to include, but is not limited to, these and any other suitable types of memory.

Based on the same inventive concept, the present invention also provides a computer-readable storage medium with instructions stored on the computer-readable storage medium. When the instructions are executed, the steps in the above-described plant state assessment method can be implemented.

Similarly, the computer-readable storage medium in the embodiment of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. It should be noted that the computer-readable storage media described herein are intended to include, but are not limited to, these and any other suitable types of memory.

It should be noted that the flowcharts and block diagrams in the drawings illustrate the possible implementation architecture, functions, and operations of the system, method, and computer program product according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, program segment, or part of code, and the module, program segment, or part of code contains one or more logic for implementing the specified Executable instructions for the function. It should also be noted that, in some alternative implementations, the functions marked in the block may also occur in a different order from the order marked in the drawings. For example, two blocks shown one after another can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and/or flowchart, and the combination of the blocks in the block diagram and/or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations Or it can be realized by a combination of dedicated hardware and computer instructions.

Generally speaking, the various exemplary embodiments of the present invention may be implemented in hardware or special purpose circuits, software, firmware, logic, or any combination thereof. Certain aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. When various aspects of the embodiments of the present invention are illustrated or described as block diagrams, flowcharts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein can be regarded as non-limiting Examples are implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

It should be noted that the various embodiments in this specification are described in a related manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. . In particular, as for the system and the computer-readable storage medium, since they are basically similar to the method embodiments, the description is relatively simple, and for related parts, please refer to the part of the description of the method embodiments.

In this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such existence between these entities or operations. The actual relationship or order. Moreover, the terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes those that are not explicitly listed Other elements of, or also include elements inherent to this process, method, article or equipment. If there are no more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other identical elements in the process, method, article, or equipment that includes the element.

The above description is only a description of the preferred embodiments of the present invention and does not limit the scope of the present invention in any way. Any changes or modifications made by a person of ordinary skill in the field of the present invention based on the above disclosure shall fall within the protection scope of the claims.

Claims

A method for assessing the state of plants, characterized in that it comprises:

Receiving a plant image, and identifying plants in the plant image to obtain the type of the plant;

Identifying the plant parts contained in the plant image;

Determining the growth state of each part of the plant;

According to the growth state of each part of the plant, the conservation state of the plant is comprehensively evaluated.
The method for evaluating the state of a plant according to claim 1, wherein said recognizing the part of said plant contained in said plant image comprises:

The plant part recognition model established by pre-training is used to recognize plants in the plant image to obtain the contained plant parts, and the plant part recognition model is a neural network model.
The method for evaluating plant state according to claim 1, wherein determining the growth state of each part of the plant comprises:

The plant state recognition model established by pre-training is used to separately recognize each part of the plant to obtain the growth state of each part of the plant, and the plant state recognition model is a neural network model.
The method for evaluating plant state according to claim 1, wherein when it is determined that the growth state of a part is abnormal, the method further comprises:

Calling a pre-trained pest identification model to identify the plant image, to determine whether the plant has a pest, and obtain at least one candidate pest information if there is a pest;

The at least one candidate pest information is screened in combination with the species of the plant to determine the pest information of the plant.
The method for evaluating plant state according to claim 1, wherein the growth state is divided into a plurality of levels according to the degree of quality from high to low, and when it is determined that the growth state of a part is not higher than a preset level, The possible reasons and countermeasures for the low growth status of this part are output.
The method for evaluating plant state according to claim 1, wherein determining the growth state of each part of the plant comprises:

The growth state of each part of the plant is determined in combination with the type of the plant.
4. The method for evaluating plant state according to claim 1, wherein when identifying plants in the plant image to obtain the type of the plants, identifying the current growth cycle of the plants;

The determining the growth state of each part of the plant includes:

Combining with the current growth cycle of the plant, the growth state of each part of the plant is determined.
3. The plant state assessment method of claim 3, wherein if the plant state recognition model cannot identify the growth state of a part, the determining the growth state of each part of the plant comprises:

Collect maintenance information of the plant from the user, and determine the growth state of the part according to the maintenance information.
8. The plant state assessment method according to claim 8, wherein the conservation information of the plant is collected from the user in the form of a questionnaire, and the questions in the questionnaire are determined according to the type of the plant.
The plant state assessment method according to claim 3, characterized in that, according to the recognition result of each part of the plant by the plant state recognition model, it is judged whether the conservation state of the plant is excellent, and if it is, the The plant image is recommended for sharing.
The plant state assessment method according to claim 1, wherein the beauty recognition model established by pre-training is used to recognize plants in the plant image to obtain the beauty degree of the plant, and the beauty recognition model is a neural network Model.
The method for evaluating the state of plants according to claim 11, wherein the degree of beauty is divided into multiple levels from high to low, and if the state of conservation of the plant is good and the degree of beauty is high, the plant image is Recommend to share.
A plant state assessment system, characterized in that the system comprises a processor and a memory, and instructions are stored on the memory, and when the instructions are executed by the processor, the system can implement any one of claims 1 to 12 The steps of the method for assessing the state of plants described in the item.
A computer-readable storage medium, characterized in that instructions are stored on the computer-readable storage medium, and when the instructions are executed, the plant state assessment method according to any one of claims 1 to 12 is realized A step of.