TWI762152B - Method and device for determining plant height, computer device and medium - Google Patents

Abstract

The present application relates to an image analysis technology, and the present application provides a method and a device for determining a plant height, a computer device and a medium. The method obtains a color image and a depth image of a plant to be tested, maps the color image with the depth image to obtain a target image, uses a pre-trained mobilenet-ssd network to detect the color image, and obtains a detection box including the plant to be tested, extracts a target outline of the plant to be tested from the detection box, determines a depth of pixel points in the target outline according to the target image, and de-noises all depths to obtain a target depth, and determines a height of the plant to be tested according to the target depth. The present application can improve a highly determined accuracy of the plant to be tested.

Description

Plant height determination method, device, computer device and medium

The present application relates to the technical field of image analysis, and in particular, to a method, device, computer device and medium for determining plant height.

At present, by analyzing the daily growth of plants, it is helpful to determine the best way of planting plants, so as to improve the yield and quality of plants, thereby reducing planting costs and bringing good news to growers. The existing method is to detect the bounding box with the plant to be detected through a deep learning algorithm, and then determine the height of the plant to be detected according to the depth value in the bounding box. However, since the bounding box is usually a quadrilateral, and the plant to be detected is not quadrilateral. , resulting in some irrelevant information in the detected bounding box, such as leaves, weeds, etc., resulting in highly inaccurate calculation of the plants to be detected.

In view of the above, it is necessary to provide a method, device, computer device and medium for determining plant height, which can improve the accuracy of plant height.

A method for determining plant height, comprising: acquiring a color image and a depth image of a plant to be detected; performing mapping processing on the color image and the depth image to obtain a target image; The trained mobilenet-ssd network detects the color image, and obtains a detection frame with the plant to be detected; Extract the target contour of the plant to be detected from the detection frame; determine the depth values of all pixels in the target contour according to the target image; perform denoising processing on all the depth values to obtain the target depth value , and determine the height of the plant to be detected according to the target depth value.

According to an optional embodiment of the present application, the acquiring a color image and a depth image of the plant to be detected includes one or a combination of the following: using a first lens of a camera to photograph the plant to be detected, and obtaining the color image image, and use the second lens of the camera device to photograph the plant to be detected to obtain the depth image; and/or determine the label of the plant to be detected, and obtain a label with the label from the first configuration library The image of the color is used as the color image, and the image with the label is obtained from the second configuration library as the depth image.

According to an optional embodiment of the present application, performing denoising processing on all the depth values to obtain a target depth value, and determining the height of the plant to be detected according to the target depth value includes: from all the depth values Acquire a depth value equal to a preset value, and determine the acquired depth value as a zero value; delete the zero value from all the depth values, and determine the remaining depth value as the target depth value; determine the The number of the target depth values; Calculate the sum of the target depth values; Divide the sum by the number to obtain the distance between the plant to be detected and the camera device; Determine the camera height at the location of the camera device ; Subtract the height of the camera and the distance to obtain the height of the plant to be detected.

According to an optional embodiment of the present application, the color image and the depth image are processed by The mapping process to obtain the target image includes: acquiring all the depth pixels on the depth image; mapping all the depth pixels to a preset depth coordinate system to obtain the depth coordinates of all the depth pixels; according to all the depth coordinates and the preset world coordinate system to determine the world coordinates of all the depth pixels; determine the positions of all the depth pixels on the color image according to all the world coordinates, and determine the color of the positions on the color image pixel; each depth pixel is fused with each color pixel to obtain the target image.

According to an optional embodiment of the present application, the detecting the color image by using the pre-trained mobilenet-ssd network, and obtaining the detection frame with the plant to be detected includes: obtaining the information in the mobilenet-ssd network. Depth convolution kernel and point convolution kernel; use the depth convolution kernel to extract the features of the color image to obtain a first feature map; use the point convolution kernel to process the first feature map to obtain the first feature map Two feature maps; the detection frame is determined according to the second feature map.

According to an optional embodiment of the present application, the extracting the target contour of the plant to be detected from the detection frame includes: deleting a background image in the detection frame to obtain a grayscale image; detecting the grayscale image Like the outline of the plant to be detected as described above, the target outline is obtained.

According to an optional embodiment of the present application, the determining the depth values of all pixels in the target contour according to the target image includes: determining the target position of each pixel on the target image; The depth value at the target position is obtained on the image as the depth value of each pixel.

A plant height determination device, the plant height determination device comprising: an acquisition unit for acquiring a color image and a depth image of a plant to be detected; The mapping unit is used to perform mapping processing on the color image and the depth image to obtain a target image; the detection unit is used to detect the color image by using the pre-trained mobilenet-ssd network to obtain a There is a detection frame of the plant to be detected; an extraction unit is used to extract the target outline of the plant to be detected from the detection frame; a determination unit is used to determine all pixels in the target outline according to the target image The depth value of the point; the determining unit is further configured to perform denoising processing on all the depth values to obtain a target depth value, and determine the height of the plant to be detected according to the target depth value.

A computer device comprising: a storage for storing at least one instruction; and a processor for executing the instructions stored in the storage to implement the method for determining plant height.

A computer-readable storage medium having at least one instruction stored in the computer-readable storage medium, the at least one instruction being executed by a processor in a computer device to implement the plant height determination method.

It can be seen from the above technical solutions that the present application can obtain the target image containing the depth value by mapping the color image and the depth image, and then the pre-trained mobilenet-ssd network can quickly The detection frame is detected, the detection efficiency is improved, the target contour of the plant to be detected is extracted from the detection frame, and the target contour other than irrelevant information can be extracted, and the target contour is determined according to the target image. The depth values of all the pixels in , can accurately and quickly determine the depth values of all the pixels, and then obtain the target depth value by denoising all the depth values, which can again ensure that the target depth value does not contain The depth value of irrelevant information, and the height of the plant to be detected can be accurately determined according to the target depth value.

S10~S15: Steps

11: Plant height determination device

110: Get Unit

111: Mapping unit

112: Detection unit

113: Extraction unit

114: Determine unit

115: Fusion Unit

116: Division Units

117: Training Unit

118: Adjustment unit

119: Computing Unit

120: Enhancement unit

121: Generation unit

122: encryption unit

123: sending unit

1: Computer device

12: Storage

13: Processor

2: Camera device

20: First Shot

21: Second Shot

FIG. 1 is an application environment diagram of a preferred embodiment of the plant height determination method of the present application.

Fig. 2 is a flow chart of a preferred embodiment of the method for determining plant height of the present application.

FIG. 3 is a functional module diagram of a preferred embodiment of the plant height determination device of the present application.

FIG. 4 is a schematic structural diagram of a computer device for implementing a preferred embodiment of the method for determining plant height in the present application.

In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be described in detail below with reference to the accompanying drawings and specific embodiments.

As shown in FIG. 1 , it is an application environment diagram of a preferred embodiment of the plant height determination method of the present application. The camera device 2 communicates with the computer device 1 , and the camera device 2 includes a first lens 20 and a second lens 21 . Color images can be captured through the first lens 20 , and depth images can be captured through the second lens 21 .

As shown in FIG. 2 , it is a flow chart of a preferred embodiment of the method for determining plant height of the present application. According to different requirements, the order of the steps in this flowchart can be changed, and some steps can be omitted.

The plant height determination method is applied to one or more computer devices 1, and the computer device 1 is a device that can automatically perform numerical calculation and/or information processing according to pre-set or stored instructions, and its hardware includes but Not limited to microprocessors, Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), Digital Signal Processors (DSPs), embedded devices, etc. .

The computer device 1 can be any electronic product that can interact with a user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, and an interactive network television. (Internet Protocol Television, IPTV), smart wearable devices, etc.

The computer device 1 may also include network equipment and/or user equipment. Wherein, the network device includes, but is not limited to, a single network server, a server group formed by multiple network servers, or a cloud formed by a large number of hosts or network servers based on cloud computing (Cloud Computing).

The network where the computer device 1 is located includes, but is not limited to, the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (Virtual Private Network, VPN), and the like.

Step S10, acquiring a color image and a depth image of the plant to be detected.

In at least one embodiment of the present application, the color image refers to an RGB three-channel color image, and the depth image refers to an image in which the distance from the image collector to each point in the scene is taken as the pixel value .

In at least one embodiment of the present application, the color image and the depth image may be acquired from a camera, or may be acquired from a configuration library.

Further, both the color image and the depth image include the plant to be detected.

In at least one embodiment of the present application, obtaining the color image and the depth image of the plant to be detected by the computer device includes one or a combination of the following methods:

(1) The computer device uses the first lens of the camera to photograph the plant to be detected to obtain the color image, and uses the second lens of the camera to photograph the plant to be detected to obtain the depth image.

Wherein, the camera device includes dual lenses, which are the first lens and the second lens respectively. Further, the camera device may be a camera, and the camera device may be installed directly above the detected plant for photographing.

Further, the plant to be detected can be any plant that needs to be analyzed for daily growth, such as roses, sunflowers, rice and the like.

Specifically, when it is detected that the plant to be detected is directly in front of the camera device, the computer device turns on the camera device, and uses the first lens to photograph the plant to be detected, obtaining the color image; further, the computer device uses the second lens to photograph the plant to be detected to obtain the depth image.

Through the above-mentioned embodiments, a color image and a depth image including the plant to be detected can be quickly acquired.

(2) The computer device determines the label of the plant to be detected, and further, the computer device obtains an image with the label from the first configuration library as the color image, and further, the The computer device obtains the image with the label from the second configuration library as the depth image.

Wherein, the first configuration library stores the mapping relationship between a plurality of color images and the label, and the second configuration library stores the mapping relationship between a plurality of depth images and the label.

Further, the label may be the serial number of the plant to be detected, for example, the label may be 0001.

The color image can be accurately obtained through the mapping relationship between the label and the color image, and the depth image can be accurately obtained through the mapping relationship between the label and the depth image.

Step S11, performing mapping processing on the color image and the depth image to obtain a target image.

In at least one embodiment of the present application, the target image refers to an image generated by fusing pixels of the color image and pixels of the depth image.

In at least one embodiment of the present application, the computer device performs mapping processing on the color image and the depth image, and obtaining the target image includes: acquiring, by the computer device, all depths on the depth image pixel, further, the computer device maps all the depth pixels to a preset depth coordinate system to obtain the depth coordinates of all the depth pixels, further, the computer device according to all the depth coordinates and the preset world The coordinate system determines the world coordinates of all the depth pixels, and further, the computer device determines the positions of all the depth pixels on the color image according to all the world coordinates, and determines that the positions are in the color image. For the color pixels on the color image, further, the computer device fuses each depth pixel with each color pixel to obtain the target image.

Wherein, the preset depth coordinate system and the preset world coordinate system can be obtained from an open source system, or can be arbitrarily set by a user according to an application scenario, which is not limited in this application.

Through the above embodiments, a target image containing depth values can be generated, so as to subsequently determine the height of the plant to be detected.

Step S12, using the pre-trained mobilenet-ssd network to detect the color image to obtain a detection frame with the plant to be detected.

In at least one embodiment of the present application, the frame to be detected is obtained by performing feature extraction on the color image by using a convolution check in the mobilenet-ssd network.

In at least one embodiment of the present application, before using a pre-trained mobilenet-ssd network to detect the color image and obtain a detection frame with the plant to be detected, the method further includes: the computer The device determines the field to which the plant to be detected belongs, and further, the computer device acquires multiple color training images in the field, and further, the computer device fuses the acquired multiple color training images, Obtaining a data set, the computer device divides the data set into a training set and a verification set, further, the computer device uses the color training images in the training set to train an open-source convolutional network to obtain a learner, Further, the computer device adjusts the learner by using the color training images in the verification set to obtain the mobilenet-ssd network.

In at least one embodiment of the present application, before dividing the data set into a training set and a validation set, the method further includes: the computer device calculates the number of color training images in the data set, when the data set is When the number is less than the preset number, the computer device uses a data enhancement algorithm to increase the number of color training images in the data set.

Through the above implementation, it can be avoided that the generalization ability of the mobilenet-ssd network obtained by training is poor due to the insufficient number of color training images.

In at least one embodiment of the present application, the computer device dividing the data set into a training set and a verification set includes: the computer device randomly dividing the data set into at least one data package according to a preset ratio, and dividing the data set into at least one data package by the computer device. Any one of the at least one data package is determined as the validation set, and the rest of the data packages are determined as the training set, and the above steps are repeated until all the data packages are sequentially used as the validation set.

The preset ratio can be set by yourself, which is not limited in this application.

The data set is divided by the above-mentioned embodiment, so that each color training image in the data set participates in training and verification, thereby improving the fitness of training the mobilenet-ssd network.

In at least one embodiment of the present application, the computer device adjusts the learner by using the color training images in the validation set, and obtaining the mobilenet-ssd network includes: the computer device adopts a hyperparameter grid search The method determines the optimal hyperparameter point from the verification set, and further, the computer device adjusts the learner through the optimal hyperparameter point to obtain the mobilenet-ssd network.

Specifically, the computer device splits the verification set according to a fixed step size to obtain a target subset, traverses the parameters of the endpoints at both ends of the target subset, and verifies all the endpoints through the parameters of the endpoints at both ends. The learner obtains the learning rate of each parameter, determines the parameter with the best learning rate as the first hyperparameter point, and in the neighborhood of the first hyperparameter point, reduces the step size and continues to traverse until The step size is a preset step size, that is, the obtained hyperparameter point is the optimal hyperparameter point. Further, the computer device adjusts the learner according to the optimal hyperparameter point, and obtains the mobilenet-ssd network.

Wherein, the present application does not limit the preset step size.

Through the above embodiments, the mobilenet-ssd network can be more suitable for the detection of color images in the field.

In at least one embodiment of the present application, the computer device detects the color image by using a pre-trained mobilenet-ssd network, and obtaining a detection frame with the plant to be detected includes: The computer device obtains the depth convolution kernel and the point convolution kernel in the mobilenet-ssd network, and further, the computer device uses the depth convolution kernel to extract the features of the color image to obtain the first Further, the computer device processes the first feature map by using the point convolution kernel to obtain the detection frame.

Wherein, the depth convolution kernel may be a matrix of 16*16*128, and further, the point convolution kernel may be a matrix of 1*1*16.

The detection frame can be quickly detected through the pre-trained mobilenet-ssd network, which improves the detection efficiency.

Step S13, extracting the target contour of the plant to be detected from the detection frame.

In at least one embodiment of the present application, the target contour refers to the contour after removing irrelevant information in the detection frame, and the shape of the target contour is determined according to the shape of the plant to be detected.

In at least one embodiment of the present application, the computer device extracting the target contour of the plant to be detected from the detection frame includes: the computer device deletes the background image in the detection frame to obtain a grayscale image Like, further, the computer device detects the outline of the plant to be detected on the grayscale image to obtain the target outline.

By deleting the background image, interference caused by the background image can be avoided, thereby improving the accuracy of extracting the target contour.

Step S14: Determine the depth values of all pixels in the target contour according to the target image.

In at least one embodiment of the present application, the depth value refers to the height of the pixel corresponding to the feature point on the plant to be detected from the camera device.

In at least one embodiment of the present application, the computer device determining the depth values of all pixels in the target contour according to the target image includes: The computer device determines the target position of each pixel on the target image, and further, the computer device obtains the depth value on the target position from the target image, as the value of each pixel. depth value.

By determining the depth values of all pixels in the target contour according to the target image, the depth values of all pixels can be accurately and quickly determined.

In step S15, denoising is performed on all the depth values to obtain a target depth value, and the height of the plant to be detected is determined according to the target depth value.

In at least one embodiment of the present application, the computer device performs denoising processing on all the depth values to obtain a target depth value, and determines the height of the plant to be detected according to the target depth value, comprising: the computer The device obtains a depth value equal to a preset value from all the depth values, and determines the obtained depth value as a zero value, further, the computer device deletes the zero value from all the depth values, and determining the remaining depth values as the target depth value, further, the computer device determining the number of the target depth values, the computer device calculating a sum of the target depth values and dividing the sum by The number is obtained from the distance between the plant to be detected and the camera device. Further, the computer device determines the camera height of the location where the camera device is located, and the computer device compares the camera height with the distance. Subtraction is performed to obtain the height of the plant to be detected.

By denoising all the depth values to obtain the target depth value, it can be ensured again that there is no depth value containing irrelevant information in the target depth value, and the plant to be detected can be accurately determined according to the target depth value. the height of.

In at least one embodiment of the present application, after determining the height of the plant to be detected according to the target depth value, the method for determining the plant height further includes: when the height is less than a preset height, the computer device Generate alarm information according to the height, further, the computer device encrypts the alarm information using a symmetric encryption algorithm to obtain ciphertext, and further, the computer device determines the ciphertext according to the plant to be detected. alert level, the computer device determines an alarm mode according to the alarm level, and further, the computer device sends the ciphertext in the alarm mode.

Wherein, the preset height can be set according to the expected generation rate of the plants to be detected, and the application does not limit the value of the preset height.

Further, the alarm levels include: level one, level two, and the like.

Further, the alarm method includes: the alarm sound of the speaker, the mail method, the telephone method, and the like.

Through the above embodiment, alarm information can be issued when the height is less than the preset height. In addition, by encrypting the alarm information, the alarm information can be prevented from being tampered with, and the security of the alarm information can be improved. At the same time, the alarm is determined according to the alarm level. In this way, the alarm information can be sent in an appropriate alarm mode, so that the sending of the alarm information is more user-friendly.

It can be seen from the above technical solutions that the present application can obtain the target image containing the depth value by mapping the color image and the depth image, and then the pre-trained mobilenet-ssd network can quickly The detection frame is detected, the detection efficiency is improved, the target contour of the plant to be detected is extracted from the detection frame, and the target contour other than irrelevant information can be extracted, and the target contour is determined according to the target image. The depth values of all the pixels in , can accurately and quickly determine the depth values of all the pixels, and then obtain the target depth value by denoising all the depth values, which can again ensure that the target depth value does not contain The depth value of irrelevant information, and the height of the plant to be detected can be accurately determined according to the target depth value.

As shown in FIG. 3 , it is a functional module diagram of a preferred embodiment of the plant height determination device of the present application. The plant height determination device 11 includes an acquisition unit 110, a mapping unit 111, a detection unit 112, an extraction unit 113, a determination unit 114, a fusion unit 115, a division unit 116, a training unit 117, an adjustment unit 118, a calculation unit 119, and an enhancement unit 120 , a generating unit 121 , an encryption unit 122 and a sending unit 123 . The module/unit referred to in this application refers to a series of computer program segments that can be executed by the processor 13 and can perform fixed functions, and are stored in the storage 12 . In this embodiment, the functions of each module/unit will be described in detail in subsequent embodiments.

The acquiring unit 110 acquires a color image and a depth image of the plant to be detected.

In at least one embodiment of the present application, the color image refers to an RGB three-channel color image, and the depth image refers to an image in which the distance from the image collector to each point in the scene is taken as the pixel value .

In at least one embodiment of the present application, the color image and the depth image may be acquired from a camera, or may be acquired from a configuration library.

Further, both the color image and the depth image include the plant to be detected.

In at least one embodiment of the present application, the acquiring unit 110 acquires the color image and the depth image of the plant to be detected, including one or a combination of the following methods:

(1) The acquiring unit 110 uses the first lens of the camera to photograph the plant to be detected to obtain the color image, and uses the second lens of the camera to photograph the plant to be detected to obtain the described depth image.

Wherein, the camera device includes dual lenses, which are the first lens and the second lens respectively. Further, the camera device may be a camera, and the camera device may be installed directly above the detected plant for photographing.

Further, the plant to be detected can be any plant that needs to be analyzed for daily growth, such as roses, sunflowers, rice and the like.

Specifically, when it is detected that the plant to be detected is directly in front of the camera, the acquisition unit 110 turns on the camera, and uses the first lens to photograph the plant to be detected to obtain the color image; further, the computer device uses the second lens to photograph the plant to be detected to obtain the depth image.

Through the above-mentioned embodiments, a color image and a depth image including the plant to be detected can be quickly acquired.

(2) The acquisition unit 110 determines the label of the plant to be detected, and further, the acquisition unit 110 acquires an image with the label from the first configuration library as the color image, Further, the obtaining unit 110 obtains the image with the label from the second configuration library as the depth image.

Wherein, the first configuration library stores the mapping relationship between a plurality of color images and the label, and the second configuration library stores the mapping relationship between a plurality of depth images and the label.

Further, the label may be the serial number of the plant to be detected, for example, the label may be 0001.

The color image can be accurately obtained through the mapping relationship between the label and the color image, and the depth image can be accurately obtained through the mapping relationship between the label and the depth image.

The mapping unit 111 performs mapping processing on the color image and the depth image to obtain a target image.

In at least one embodiment of the present application, the target image refers to an image generated by fusing pixels of the color image and pixels of the depth image.

In at least one embodiment of the present application, the mapping unit 111 performs mapping processing on the color image and the depth image, and obtaining the target image includes: the mapping unit 111 obtains the depth image on the depth image. All depth pixels, further, the mapping unit 111 maps all the depth pixels to a preset depth coordinate system to obtain the depth coordinates of all the depth pixels, and further, the mapping unit 111 according to all the depth coordinates and the preset world coordinate system to determine the world coordinates of all the depth pixels, further, the mapping unit 111 determines the positions of all the depth pixels on the color image according to all the world coordinates, and determines that the positions are in For the color pixels on the color image, further, the computer device fuses each depth pixel with each color pixel to obtain the target image.

Wherein, the preset depth coordinate system and the preset world coordinate system can be obtained from an open source system, or can be arbitrarily set by a user according to an application scenario, which is not limited in this application.

Through the above embodiments, a target image containing depth values can be generated, so as to subsequently determine the height of the plant to be detected.

The detection unit 112 detects the color image by using the pre-trained mobilenet-ssd network to obtain a detection frame with the plant to be detected.

In at least one embodiment of the present application, the frame to be detected is obtained by performing feature extraction on the color image by using a convolution check in the mobilenet-ssd network.

In at least one embodiment of the present application, before using the pre-trained mobilenet-ssd network to detect the color image to obtain a detection frame with the plant to be detected, the determining unit 114 determines the plant to be detected belong to the field, further, the acquisition unit 110 acquires a plurality of color training images in the field, and further, the fusion unit 115 fuses the acquired color training images to obtain a data set, and the division unit 116 The data set is divided into a training set and a verification set, further, the training unit 117 uses the color training images in the training set to train an open-source convolutional network to obtain a learner, and further, the adjustment unit 118 uses the Color training images in the validation set adjust the learner to obtain the mobilenet-ssd network.

In at least one embodiment of the present application, before dividing the data set into a training set and a validation set, the computing unit 119 calculates the number of color training images in the data set, and when the number is less than a preset number, The augmentation unit 120 utilizes a data augmentation algorithm to augment the number of color training images in the data set.

Through the above implementation, it can be avoided that the generalization ability of the mobilenet-ssd network obtained by training is poor due to the insufficient number of color training images.

In at least one embodiment of the present application, the dividing unit 116 dividing the data set into a training set and a validation set includes: the dividing unit 116 randomly dividing the data set into at least one data package according to a preset ratio , any one of the at least one data package is determined as the verification set, and the rest of the data packages are determined as the training set, and the above steps are repeated until all the data packages are used as the verification set in turn .

The preset ratio can be set by yourself, which is not limited in this application.

The data set is divided by the above-mentioned embodiment, so that each color training image in the data set participates in training and verification, thereby improving the fitness of training the mobilenet-ssd network.

In at least one embodiment of the present application, the adjustment unit 118 adjusts the learner by using the color training images in the validation set, and obtaining the mobilenet-ssd network includes: the adjustment unit 118 adopts a hyperparameter network The lattice search method determines the optimal hyperparameter point from the verification set. Further, the adjustment unit 118 adjusts the learner through the optimal hyperparameter point to obtain the mobilenet-ssd network.

Specifically, the adjustment unit 118 splits the verification set according to a fixed step size to obtain a target subset, traverses the parameters of the endpoints at both ends of the target subset, and verifies the parameters of the endpoints at the two ends. The learner obtains the learning rate of each parameter, determines the parameter with the best learning rate as the first hyperparameter point, and in the neighborhood of the first hyperparameter point, reduces the step size and continues to traverse, Until the step size is the preset step size, that is, the obtained hyperparameter point is the optimal hyperparameter point. Further, the adjustment unit 118 adjusts the learner according to the optimal hyperparameter point, and obtains The mobilenet-ssd network.

Wherein, the present application does not limit the preset step size.

Through the above embodiments, the mobilenet-ssd network can be more suitable for the detection of color images in the field.

In at least one embodiment of the present application, the detection unit 112 uses a pre-trained mobilenet-ssd network to detect the color image, and obtains a detection frame with the plants to be detected, including: the detection unit 112 Obtain the depth convolution kernel and the point convolution kernel in the mobilenet-ssd network, further, the detection unit 112 uses the depth convolution kernel to extract the features of the color image to obtain a first feature map, Further, the detection unit 112 uses the point convolution kernel to process the first feature map to obtain the detection frame.

Wherein, the depth convolution kernel may be a matrix of 16*16*128, and further, the point convolution kernel may be a matrix of 1*1*16.

The detection frame can be quickly detected through the pre-trained mobilenet-ssd network, which improves the detection efficiency.

The extraction unit 113 extracts the target contour of the plant to be detected from the detection frame.

In at least one embodiment of the present application, the target contour refers to the contour after removing irrelevant information in the detection frame, and the shape of the target contour is determined according to the shape of the plant to be detected.

In at least one embodiment of the present application, the extraction unit 113 extracting the target contour of the plant to be detected from the detection frame includes: the extraction unit 113 deletes the background image in the detection frame to obtain grayscale and further, the extraction unit 113 detects the contour of the plant to be detected on the grayscale image to obtain the target contour.

By deleting the background image, interference caused by the background image can be avoided, thereby improving the accuracy of extracting the target contour.

The determining unit 114 determines the depth values of all pixels in the target contour according to the target image.

In at least one embodiment of the present application, the depth value refers to the height of the pixel corresponding to the feature point on the plant to be detected from the camera device.

In at least one embodiment of the present application, the determining unit 114 determining, according to the target image, the depth values of all pixels in the target contour includes: the determining unit 114 determining that each pixel is in the target image The target position on the image, further, the determining unit 114 obtains the depth value on the target position from the target image as the depth value of each pixel point.

By determining the depth values of all pixels in the target contour according to the target image, the depth values of all pixels can be accurately and quickly determined.

The determining unit 114 performs denoising processing on all the depth values to obtain a target depth value, and determines the height of the plant to be detected according to the target depth value.

In at least one embodiment of the present application, the determining unit 114 performs denoising processing on all the depth values to obtain a target depth value, and determining the height of the plant to be detected according to the target depth value includes: the The determining unit 114 acquires a depth value equal to a preset value from all the depth values, and determines the acquired depth value as a zero value, further, the determining unit 114 deletes the zero value from all the depth values value, and determine the remaining depth values as the target depth value, further, the determining unit 114 determines the number of the target depth values, the determining unit 114 calculates the sum of the target depth values, and sets the The sum is divided by the number to obtain the distance between the plant to be detected and the camera. The camera height and the distance are subtracted to obtain the height of the plant to be detected.

By denoising all the depth values to obtain the target depth value, it can be ensured again that there is no depth value containing irrelevant information in the target depth value, and the plant to be detected can be accurately determined according to the target depth value. the height of.

In at least one embodiment of the present application, after the height of the plant to be detected is determined according to the target depth value, when the height is less than a preset height, the generating unit 121 generates alarm information according to the height, and further , the encryption unit 122 uses a symmetric encryption algorithm to encrypt the alarm information to obtain a ciphertext. Further, the determining unit 114 determines the alarm level of the ciphertext according to the plant to be detected, and the determining unit 114 determines the alarm level of the ciphertext according to the The alarm level determines an alarm mode, and further, the sending unit 123 sends the ciphertext in the alarm mode.

Wherein, the preset height can be set according to the expected generation rate of the plants to be detected, and the application does not limit the value of the preset height.

Further, the alarm levels include: level one, level two, and the like.

Further, the alarm method includes: the alarm sound of the speaker, the mail method, the telephone method, and the like.

Through the above embodiment, when the height is less than the preset height, alarm information can be issued. In addition, by encrypting the alarm information, the alarm information can be prevented from being tampered with, and the security of the alarm information can be improved. At the same time, the alarm is determined according to the alarm level. In this way, the alarm information can be sent in an appropriate alarm mode, so that the sending of the alarm information is more user-friendly.

It can be seen from the above technical solutions that the present application can obtain the target image containing the depth value by mapping the color image and the depth image, and then the pre-trained mobilenet-ssd network can quickly The detection frame is detected, the detection efficiency is improved, the target contour of the plant to be detected is extracted from the detection frame, and the target contour other than irrelevant information can be extracted, and the target contour is determined according to the target image. The depth values of all the pixels in , can accurately and quickly determine the depth values of all the pixels, and then obtain the target depth value by denoising all the depth values, which can again ensure that the target depth value does not contain The depth value of irrelevant information, and the height of the plant to be detected can be accurately determined according to the target depth value.

As shown in FIG. 4 , it is a schematic structural diagram of a computer device according to a preferred embodiment of the method for determining plant height of the present application.

In one embodiment of the present application, the computer device 1 includes, but is not limited to, a storage 12 , a processor 13 , and a computer program stored in the storage 12 and running on the processor 13 , such as the plant height determination program.

Those skilled in the art can understand that the schematic diagram is only an example of the computer device 1, and does not constitute a limitation on the computer device 1. It may include more or less components than the one shown, or combine some components, or different Components, such as the computer device 1, may also include input and output devices, network access devices, bus bars, and the like.

The processor 13 may be a Central Processing Unit (CPU), other general-purpose processors, a Digital Signal Processor (DSP), or an Application Specific Integrated Circuit (ASIC). , Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the The processor can also be any conventional processor, etc. The processor 13 is the computing core and control center of the computer device 1, and uses various interfaces and lines to connect various parts of the entire computer device 1, and execute the computer device. 1 operating system and installed various applications, code, etc.

The processor 13 executes the operating system of the computer device 1 and various installed applications. The processor 13 executes the application program to implement the steps in each of the above embodiments of the plant height determination method, such as the steps shown in FIG. 2 .

Exemplarily, the computer program can be divided into one or more modules/units, and the one or more modules/units are stored in the storage 12 and executed by the processor 13, to complete this application. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in the computer device 1 . For example, the computer program can be divided into an acquisition unit 110, a mapping unit 111, a detection unit 112, an extraction unit 113, a determination unit 114, a fusion unit 115, a division unit 116, a training unit 117, an adjustment unit 118, a calculation unit 119, Enhancement unit 120 , generation unit 121 , encryption unit 122 and transmission unit 123 .

The storage 12 can be used to store the computer programs and/or modules. The processor 13 executes or executes the computer programs and/or modules stored in the storage 12, and calls the computer programs and/or modules stored in the storage 12. 12 to realize various functions of the computer device 1 . The storage 12 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; storage data The area can store data etc. created according to the use of the computer device. In addition, the storage 12 may include non-volatile storage such as hard disk, storage, plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, flash memory A memory card (Flash Card), at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device.

The storage 12 may be an external storage and/or an internal storage of the computer device 1 . Further, the storage 12 may be a storage in physical form, such as a storage bar, a TF card (Trans-flash Card), and the like.

If the modules/units integrated in the computer device 1 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the present application realizes all or part of the processes in the methods of the above embodiments, and can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned method embodiments can be implemented.

Wherein, the computer program includes computer program code, and the computer program code may be in the form of original code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, pen drive, removable hard disk, magnetic disk, optical disk, computer storage, read-only storage (ROM, Read-only storage) Only Memory).

Referring to FIG. 2 , the storage 12 in the computer device 1 stores a plurality of instructions to implement a method for determining plant height, and the processor 13 can execute the plurality of instructions to achieve: acquiring a color map of the plant to be detected image and depth image; perform mapping processing on the color image and the depth image to obtain the target image; use the pre-trained mobilenet-ssd network to detect the color image, and obtain the target image with the Detecting the detection frame of the plant; extracting the target contour of the plant to be detected from the detection frame; determining the depth values of all pixels in the target contour according to the target image; denoising all the depth values process to obtain a target depth value, and determine the height of the plant to be detected according to the target depth value.

Specifically, for the specific implementation method of the above-mentioned instruction by the processor 13, reference may be made to the description of the relevant steps in the embodiment corresponding to FIG. 2, and details are not described herein.

In the several embodiments provided in this application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the device embodiments described above are only illustrative. For example, the division of the modules is only a logical function division, and other division methods may be used in actual implementation.

The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical units, that is, they may be located in one place. square, or you can distribute the image to multiple network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional module in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware, or can be implemented in the form of hardware plus software function modules.

Accordingly, the embodiments are to be regarded in all respects as illustrative and not restrictive, and the scope of this application is defined by the appended claims rather than the foregoing description, and is therefore intended to fall within the scope of the claims. All changes within the meaning and scope of the equivalents of , are included in this application. Any associated icon indicia in a claim should not be considered to limit the claim to which it relates.

Furthermore, it is clear that the word "comprising" does not exclude other units or steps and the singular does not exclude the plural. A plurality of units or devices stated in this application may also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to denote names and do not denote any particular order.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit them. Although the present application has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present application can be Modifications or equivalent substitutions can be made without departing from the spirit and scope of the technical solutions of the present application.

S10~S15: Steps

Claims (9)

A plant height determination method, applied to a computer device, wherein the plant height determination method comprises: acquiring a color image and a depth image of a plant to be detected; performing mapping processing on the color image and the depth image, Obtain a target image; use a pre-trained mobilenet-ssd network to detect the color image to obtain a detection frame with the plant to be detected; extract the target contour of the plant to be detected from the detection frame; Determine the depth values of all pixels in the target contour according to the target image; perform denoising on all the depth values to obtain a target depth value, and determine the height of the plant to be detected according to the target depth value , including: obtaining a depth value equal to a preset value from all the depth values, and determining the obtained depth value as a zero value; deleting the zero value from all the depth values, and using the remaining depth values determine the target depth value; determine the number of the target depth values; calculate the sum of the target depth values; divide the sum by the number to obtain the distance between the plant to be detected and the camera device; The imaging height of the location where the imaging device is located; and the height of the plant to be detected is obtained by subtracting the imaging height and the distance. The method for determining plant height according to claim 1, wherein the acquiring a color image and a depth image of the plant to be detected includes one or a combination of the following: using a first lens of the camera to shoot the to-be-detected plant. Detecting a plant, obtaining the color image, and using the second lens of the camera to photograph the plant to be detected to obtain the depth image; and/or determining the label of the plant to be detected, from the first configuration The image with the label is obtained from the library as the color image, and the image with the label is obtained from the second configuration library as the depth image. The method for determining plant height according to claim 1, wherein the mapping of the color image and the depth image to obtain the target image includes: Acquire all depth pixels on the depth image; map all depth pixels to a preset depth coordinate system to obtain the depth coordinates of all depth pixels; determine the depth coordinates according to all depth coordinates and a preset world coordinate system The world coordinates of all depth pixels; determine the positions of all depth pixels on the color image according to all the world coordinates, and determine the color pixels of the positions on the color image; associate each depth pixel with each depth pixel The color pixels are fused to obtain the target image. The method for determining plant height according to claim 1, wherein the detecting the color image by using a pre-trained mobilenet-ssd network to obtain a detection frame with the plant to be detected comprises: obtaining the mobilenet - the depth convolution kernel and point convolution kernel in the ssd network; use the depth convolution kernel to extract the features of the color image to obtain the first feature map; use the point convolution kernel to check the first feature The image is processed to obtain the detection frame. The method for determining plant height according to claim 1, wherein the extracting the target contour of the plant to be detected from the detection frame comprises: deleting a background image in the detection frame to obtain a grayscale image; The contour of the plant to be detected on the grayscale image is detected to obtain the target contour. The method for determining plant height according to claim 1, wherein the determining the depth values of all pixels in the target outline according to the target image comprises: determining the target of each pixel on the target image position; obtain the depth value at the target position from the target image as the depth value of each pixel. A plant height determination device, running on a computer device, wherein the plant height determination device comprises: an acquisition unit for acquiring a color image and a depth image of a plant to be detected; The mapping unit is used to perform mapping processing on the color image and the depth image to obtain a target image; the detection unit is used to detect the color image by using the pre-trained mobilenet-ssd network to obtain a There is a detection frame of the plant to be detected; an extraction unit is used to extract the target outline of the plant to be detected from the detection frame; a determination unit is used to determine all pixels in the target outline according to the target image The depth value of the point; the determining unit is further configured to perform denoising processing on all the depth values to obtain a target depth value, and determine the height of the plant to be detected according to the target depth value, including: from the Acquire a depth value equal to a preset value from all the depth values, and determine the acquired depth value as a zero value; delete the zero value from all the depth values, and determine the remaining depth value as the target depth determine the number of the target depth values; calculate the sum of the target depth values; divide the sum by the number to obtain the distance between the plant to be detected and the camera device; determine the location of the camera device Camera height; subtract the camera height from the distance to obtain the height of the plant to be detected. A computer device, wherein the computer device comprises: a storage for storing at least one instruction; and a processor for acquiring the instructions stored in the storage to achieve the plant height according to any one of claim 1 to 6 Determine the method. A computer-readable storage medium, wherein: the computer-readable storage medium stores at least one instruction, and the at least one instruction is acquired by a processor in a computer device to implement any one of claim 1 to 6. method for determining plant height.

Images