CN110599507B - A kind of tomato identification and positioning method and system - Google Patents

Abstract

The embodiment of the invention provides a tomato identification and positioning method and a tomato identification and positioning system, wherein the tomato identification and positioning method comprises the following steps: acquiring an image to be detected containing tomatoes, converting the image to be detected into an HSV image, after masking the HSV image, integrating the HSV image into an RGB image and masking the RGB image to obtain a tomato image with a background removed; cutting the red tomato image with the background removed through a watershed algorithm to obtain an image-segmented tomato image; performing edge detection on the tomato image after image segmentation, and determining the coordinate position of the tomato in the tomato image after image segmentation; and acquiring point cloud data of the tomatoes according to the coordinate positions of the tomatoes in the tomato images after the image segmentation, and determining the actual positions of the tomatoes according to the point cloud data of the tomatoes. The method provided by the invention eliminates the influence of uneven light, the occlusion of the branches and the leaves to the fruits, the shadow caused by the mutual occlusion of the fruits and the like on the fruit identification.

Description

A kind of tomato identification and positioning method and system

technical field

Embodiments of the present invention relate to the technical field of image recognition, and in particular, to a method and system for tomato recognition and positioning.

Background technique

Tomato is one of the most common vegetables cultivated in people's life, but manual picking is labor-intensive and low-efficiency. The development of automatic tomato harvesting equipment can greatly reduce the picking cost and effectively improve the production status. Due to the complex background of tomato growing environment and overlapping fruit clusters, the ability to accurately obtain fruit location and picking information has become a major bottleneck restricting tomato robot picking.

In the prior art, a lot of related researches and applications have been done on tomato identification and localization. The commonly used method is to use binocular stereo vision technology for localization. Specifically, two CCD cameras are used to collect images. The tomato position information is restored in the middle, so as to use the machine vision to control the movement and positioning of the manipulator, so as to achieve the purpose of picking tomato fruits. Another tomato identification method is to use the active detection method of line structured light vision, specifically the target area segmentation method based on the (2R-G-B) color difference model, which highlights the difference in color characteristics between the ripe fruit and the background under different sunlight conditions. The distance mapping relationship establishes a depth image, based on which regional segmentation and stereo measurement of overlapping fruits are performed. In addition, Qiu Ruicheng et al. in the document "Measurement Method of Corn Stem Thickness Based on RGB-D Camera", took corn at the small bell mouth stage as the observation object, and used the RGB-D camera to obtain the color image and depth image of field corn. processing, can quickly obtain crop phenotypic parameters.

In the prior art, the positioning and identification method of tomato is too complicated, the camera equipment used is expensive, and the active detection method of line structured light vision is used, which cannot effectively identify the red ripe fruit, and in terms of image processing, the fruit and the background cannot be effectively segmented at the same time. , Fruit adhesion part.

SUMMARY OF THE INVENTION

Embodiments of the present invention provide a tomato identification and positioning method and system, which are used to solve the problem that the prior art positioning and identification method for tomato is too complicated, the camera equipment used is expensive, and the line structured light vision active detection method is adopted, which cannot effectively identify red color. Ripe fruit, in terms of image processing, cannot effectively segment the fruit, the background, and the adhering part of the fruit at the same time.

According to a first aspect of the present invention, a tomato identification and positioning method is provided, comprising:

Obtain the to-be-detected image containing tomatoes, convert the to-be-detected image into an HSV image, and after masking the HSV image, integrate the HSV image into an RGB image and perform masking processing to obtain a background-removed image. tomato image;

The red tomato image after the background removal is cut through a watershed algorithm to obtain a tomato image after image segmentation;

Perform edge detection on the tomato image after the image segmentation, and determine the coordinate position of the tomato in the tomato image after the image segmentation;

According to the coordinate position of the tomato in the tomato image after the image segmentation, the point cloud data of the tomato is acquired, and the actual position of the tomato is determined according to the point cloud data of the tomato.

Wherein, said acquiring the to-be-detected image containing tomatoes, converting the to-be-detected image into an HSV image, and after performing mask processing on the HSV image, integrating the HSV image into an RGB image and performing mask processing to obtain The tomato image after removing the background, including:

Convert the to-be-detected image into an HSV image and convert it into an HSV image, and separate out the H channel histogram, the S channel histogram and the V channel histogram; The image is masked on a preset threshold, and the H channel histogram, S channel histogram and V channel histogram after mask processing are integrated into an RGB image; the RGB image is closed and opened by morphological technology. After the operation, perform R, G, B masking processing, and then integrate to obtain the tomato image after removing the background.

Wherein, the red tomato image after the background removal is cut through a watershed algorithm to obtain a tomato image after image segmentation, specifically including:

Converting the background-removed tomato image into a grayscale image, calculating the gradient magnitude of the grayscale image according to the Sobel operator, and obtaining the gradient image of the grayscale image; cutting the gradient image by a watershed algorithm , to obtain the tomato image after image segmentation.

Wherein, the step of cutting the gradient image through the watershed algorithm to obtain the tomato image after the image segmentation further includes:

The morphological opening and closing reconstruction operation and watershed calculation are performed on the tomato image after the image segmentation, to obtain the tomato image calculated by the secondary watershed algorithm, which is used as a new tomato image after image segmentation.

Wherein, after the step of obtaining the image-segmented tomato image, the method further includes: obtaining the number of the tomatoes in the image-segmented tomato image.

Wherein, performing edge detection on the image segmented tomato image to determine the coordinate position of the tomato in the segmented tomato image specifically includes:

The number of the tomatoes is used as the circle detection input parameter of Hough transform, and edge detection is performed on the segmented tomato image to obtain the foreground point of the segmented tomato image; the segmented tomato image is composed of The x-y coordinate system is converted into the a-b coordinate system, and according to the coordinates of the foreground point, the coordinates of the center point and the radius of the tomato corresponding to the tomato in the segmented tomato image are obtained; according to the coordinates of the center point and the radius of the tomato Radius, to obtain the coordinate position of the tomato in the tomato image after the image is segmented.

Wherein, acquiring the point cloud data of the tomato according to the coordinate position of the tomato in the tomato image after the image segmentation, and determining the actual position of the tomato according to the point cloud data of the tomato, specifically includes: :

Obtain the depth image corresponding to the to-be-detected image through a depth camera; obtain the point cloud data of the tomato through the coordinate position of the tomato in the image segmented tomato image and the depth image; through the tomato point cloud data to determine the actual location of the tomato.

According to a second aspect of the present invention, a tomato identification and positioning system is provided, comprising:

The fruit extraction module is used to obtain the to-be-detected image containing tomatoes, convert the to-be-detected image into an HSV image, and after masking the HSV image, integrate the HSV image into an RGB image and perform masking processing , get the tomato image after removing the background;

The fruit separation module is used for cutting the red tomato image after removing the background by the watershed algorithm to obtain the tomato image after image segmentation;

a coordinate determination module, configured to perform edge detection on the tomato image after the image segmentation, and determine the coordinate position of the tomato in the tomato image after the image segmentation;

The positioning module is configured to obtain the point cloud data of the tomato according to the coordinate position of the tomato in the tomato image after the image segmentation, and determine the actual position of the tomato according to the point cloud data of the tomato.

According to a third aspect of the present invention, there is provided a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the steps of the method provided in the above-mentioned first aspect.

According to the fourth aspect of the present invention, a tomato identification and positioning device is provided, comprising:

at least one processor; and at least one memory connected to the processor, wherein:

The memory stores program instructions executable by the processor, and the processor invokes the program instructions to perform the steps of the method provided in the first aspect above.

The tomato identification and positioning method and device provided by the embodiments of the present invention realize the spatial positioning of the tomato based on the depth camera and the original RGB image, lay a foundation for the fast and accurate picking of the tomato fruit, and eliminate uneven light, branches and leaves. The occlusion of the fruit and the shadows caused by the mutual occlusion between the fruits have an impact on fruit recognition, realizing the recognition of ripe red tomatoes, which reduces the cost of tomato recognition compared to the traditional binocular camera.

Description of drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.

1 is a flowchart of a method for identifying and locating tomato according to an embodiment of the present invention;

2 is a schematic flowchart of a red tomato fruit extraction process based on HSV in a tomato identification and positioning method provided in an embodiment of the present invention;

3 is a schematic diagram of a tomato fruit separation process in a tomato identification and positioning method provided by an embodiment of the present invention;

4 is a schematic diagram of the calculation flow of the fruit sphere center and radius in a tomato identification and positioning method provided by an embodiment of the present invention;

5 is a schematic flowchart of the actual location of tomato in a tomato identification and positioning method provided by an embodiment of the present invention;

6 is a schematic structural diagram of a tomato identification and positioning system according to an embodiment of the present invention;

FIG. 7 is a schematic structural diagram of a tomato identification and positioning device according to an embodiment of the present invention.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Referring to FIG. 1, FIG. 1 is a flowchart of a method for identifying and locating tomato according to an embodiment of the present invention, and the provided method includes:

S1, acquire an image to be detected containing tomatoes, convert the to-be-detected image into an HSV image, and after masking the HSV image, integrate the HSV image into an RGB image and perform masking processing to obtain a background-removed image Post tomato image.

Specifically, after acquiring the to-be-detected image containing tomatoes, the to-be-detected image is converted into an HSV image, and mask processing is performed on the converted HSV image. The specific steps are to separate the H channel, the S channel and the V channel from the HSV image. channel, the H channel and S channel are used as the original color information to calculate the color information saliency of the image, and the V channel is used as the original brightness information to calculate the brightness information saliency, through the weighted color information saliency and luminance information saliency to obtain a comprehensive saliency map. The selected thresholds are applied on H, S, and V for mask processing, and then the processed images are reintegrated into RGB images and masked, so that the background-removed tomato image can be obtained.

S2, the red tomato image after the background removal is cut through a watershed algorithm to obtain a tomato image after image segmentation.

Specifically, during the ripening process of tomatoes, fruits often touch together, and many different image processing algorithms cannot detect two touched tomatoes. In this embodiment, a watershed segmentation method based on graphic morphology is used to perform a watershed algorithm to perform watershed segmentation on the image, retain important contour information, correctly separate the connected fruits, realize image segmentation, and obtain a tomato image after image segmentation.

S3: Perform edge detection on the tomato image after the image segmentation, and determine the coordinate position of the tomato in the tomato image after the image segmentation.

Specifically, after the tomatoes in the tomato image are segmented, it is further necessary to obtain the center point and radius data of each tomato, so that the mechanical claw can grasp the tomato according to these values. Therefore, in this step , the Hough transform is used to detect the center and radius of the tomato in the image. The specific principle is to first perform edge detection on the tomato image after image segmentation to obtain the boundary points in the image, that is, the foreground points, and the foreground points are the tomato's Contour position, you can determine the center position of the tomato in the image and the radius of the tomato according to the coordinate position of the foreground point in the image, and then obtain the coordinate position of each tomato in the image.

S4: Acquire point cloud data of the tomato according to the coordinate position of the tomato in the tomato image after the image segmentation, and determine the actual position of the tomato according to the point cloud data of the tomato.

Specifically, after obtaining the coordinate position of each tomato in the image, since the depth information of the tomato in the image cannot be determined, it is necessary to confirm the depth information of the tomato in the actual position through the depth camera. Obtaining the point cloud data of the tomato in the image Since the point cloud data contains the depth information of the image, that is, the actual distance between the camera and the fruit, the three-dimensional model of the tomato can be reconstructed according to the obtained point cloud data to provide positioning information for tomato picking.

Through this method, based on the depth camera and the original RGB image, the spatial positioning of the tomato is realized, which lays the foundation for the fast and accurate picking of the tomato fruit, and eliminates uneven light, the occlusion of the fruit by branches and leaves, and the interference between the fruits. The impact of shadows caused by mutual occlusion on fruit recognition enables the recognition of ripe red tomatoes, which reduces the cost of tomato recognition compared to traditional binocular cameras.

On the basis of the above embodiment, the to-be-detected image containing tomatoes is acquired, the to-be-detected image is converted into an HSV image, and the HSV image is integrated into an RGB image after mask processing is performed on the HSV image And perform masking processing to obtain the tomato image after removing the background, including:

Converting the to-be-detected image into an HSV image is converted into an HSV image, from which an H channel histogram, an S channel histogram and a V channel histogram are separated;

The H channel histogram, the S channel histogram and the V channel histogram are respectively subjected to mask processing on a preset threshold, and the H channel histogram, the S channel histogram and the V channel histogram after the mask processing are integrated is an RGB image;

After the RGB image is closed and opened by morphological technology, R, G, B masking processing is performed, and then the tomato image after background removal is obtained by integration.

Specifically, in the background removal step of the image to be detected, the specific steps are shown in FIG. 2 , the HSV color space is composed of three components: hue (Hue), saturation (Saturation) and brightness (Value). Hue (H) represents different colors, and the value range is 0 to 360°; Saturation (S) represents the depth of the color, usually 0% to 100%; Brightness (V) represents the lightness and darkness of the color, and the value range is 0% (black) to 100% (white). First, the image to be detected is converted into an HSV image, and the H channel, S channel and V channel are separated from it, and the H channel and S channel are used as the original color information to calculate the color information saliency of the image, and the V channel is used as the original color information. The original luminance information and thus the luminance information saliency are calculated, and the comprehensive saliency map is obtained by weighting the color information saliency and the luminance information saliency. In this embodiment, the histogram is obtained first, and on this basis, the thresholds of the H, S, and V components are selected, and the region of interest is defined as 0 to 1. In the hue histogram, 0 is red, in order to identify red tomatoes, select 0.07 as the high hue H threshold, and use the Otsu algorithm to detect red; similarly, in the saturation S histogram, select 0.35 as the saturation threshold; in the brightness In the V histogram, 0.4 is selected as the brightness threshold. The selected thresholds are then applied on H, S, V, respectively, masked, and then integrated into an RGB image. At this time, only the red tomato remains in the image, and then the morphological technology is used to close and open, and then R, G, B masking processing is performed, and after integration, the red tomato image with the background removed is finally obtained.

On the basis of the above-mentioned embodiment, the red tomato image after the background removal is cut through a watershed algorithm to obtain a tomato image after image segmentation, which specifically includes:

The tomato image after the described background removal is converted into a grayscale image, the gradient magnitude of the grayscale image is calculated according to the Sobel operator, and the gradient image of the grayscale image is obtained;

The gradient image is cut through a watershed algorithm to obtain a segmented tomato image.

Specifically, during the ripening process of tomatoes, fruits often touch together, and many different image processing algorithms cannot detect two touched tomatoes. In this embodiment, the watershed algorithm is used to perform the watershed algorithm calculation on the tomato image with the background removed. In the specific implementation, firstly, the gradient image is filtered by the morphological opening and closing reconstruction operation to eliminate the local extrema caused by irregular grayscale disturbance and noise in the image; then the image is divided into watershed to retain important contour information; Finally, the similarity measurement is carried out by combining the normalized product correlation gray matching algorithm, and the target image area is separated from the natural background of the image by guiding the region merging based on the shape template.

On the basis of the above-mentioned embodiment, the step of cutting the gradient image through the watershed algorithm to obtain the tomato image after the image segmentation further includes:

The morphological opening and closing reconstruction operation and watershed calculation are performed on the tomato image after the image segmentation, to obtain the tomato image calculated by the secondary watershed algorithm, which is used as a new tomato image after image segmentation.

Wherein, after the step of obtaining the image-segmented tomato image, the method further includes: obtaining the number of the tomatoes in the image-segmented tomato image.

Specifically, referring to FIG. 3 , in the step of fruit separation, two watershed algorithms can be used to improve the accuracy of fruit segmentation. In the specific implementation, the red tomato image after background removal is converted into a grayscale image. , the Sobel operator is used to calculate the gradient magnitude, the horizontal and vertical edges are detected, the first watershed algorithm is performed on the gradient image, and then the morphological opening and closing reconstruction operation is used to remove the blobs and make the image smooth. Foreground, modify the regional extreme value of the image to realize the detection of the largest area. In order to distinguish the background and foreground of the image, an improved thresholding method is proposed, and the background pixels are marked as black. After using the Euclidean matrix for distance transformation, the second watershed algorithm is used to obtain the ridge line. On the basis of the first watershed, the connected fruits are correctly separated to achieve image segmentation, and the number of fruits after segmentation N is obtained as the next step. Input parameters for one-step Hough transform-based fruit localization.

On the basis of the above-mentioned embodiment, the performing edge detection on the image segmented tomato image to determine the coordinate position of the tomato in the segmented tomato image specifically includes:

Using the number of the tomatoes as the circle detection input parameter of the Hough transform, performing edge detection on the segmented tomato image to obtain the foreground point of the segmented tomato image;

Convert the segmented tomato image from the x-y coordinate system to the a-b coordinate system, and obtain the coordinates of the center point of the tomato and the radius of the tomato in the segmented tomato image according to the coordinates of the foreground point;

According to the coordinates of the center point of the circle and the radius of the tomato, the coordinate position of the tomato in the tomato image after the image segmentation is obtained.

Specifically, in the tomato coordinate calculation process, as shown in FIG. 4 , the number N of fruits obtained by separating the red tomato fruits is used as the circle detection input parameter of the Hough transform. The principle of applying the Hough transform to detect the center and radius of the circle is as follows: first, perform edge detection on the input image, and obtain the boundary point, that is, the foreground point. If there is a circle in the image, then its outline must belong to the foreground point. Represent the general equation of a circle in another way, and perform coordinate transformation. Converting from the x-y coordinate system to the a-b coordinate system, then a point on the circle boundary in the x-y coordinate system corresponds to the a-b coordinate system, which is a circle. There are many points on a circular boundary in the x-y coordinate system, and there will be many circles in the a-b coordinate system. Since these points in the original image are all on the same circle, a and b must also satisfy the equations of all circles in the a-b coordinate system after conversion. Intuitively, the circles corresponding to these many points will intersect at a point, then this intersection point may be the center of the circle (a, b). Count the number of circles at the local intersection points, and take each local maximum value to obtain the center coordinates (a, b) of the corresponding circle in the original image. Once a circle is detected under a certain r, the value of r is determined accordingly, and finally the coordinates (a, b, r) of the tomato in the image are obtained.

On the basis of the above embodiment, the point cloud data of the tomato is acquired according to the coordinate position of the tomato in the tomato image after the image segmentation, and the tomato is determined according to the point cloud data of the tomato actual location, including:

Obtain a depth image corresponding to the to-be-detected image through a depth camera;

Obtain the point cloud data of the tomato through the coordinate position of the tomato in the tomato image after the image segmentation and the depth image;

The actual position of the tomato is determined through the point cloud data of the tomato.

Specifically, the steps of obtaining the actual position of the tomato are shown in Figure 5. The Kinect2.0 camera produced by Microsoft Corporation is used to collect the tomato plant information, and the color RGB image of the tomato plant can be collected as the image to be detected. The depth information of the tomato plant; the RGB image is matched with the depth information, and the point cloud data of the corresponding red tomato is obtained according to the (a, b, r) coordinates of the red tomato image obtained by color image processing. Since the point cloud data contains the depth information of the image, that is, the actual distance between the RGB-D camera and the fruit, the tomato 3D model can be reconstructed according to the obtained point cloud data to provide positioning information for tomato picking.

To sum up, the embodiment of the present invention eliminates the uneven light, the occlusion of the fruit by the branches and leaves, and the shadow caused by the mutual occlusion between the fruits through the tomato identification and positioning method based on the RGB-D camera. The impact of recognition can realize the recognition of ripe red tomato, which is of certain significance for the protection of the fruit during the picking process. At the same time, based on the cheap depth camera relative to the binocular camera, the red tomato fruit with complex background and mutual adhesion has a certain significance. Realize rapid identification and lay the foundation for fast and accurate picking of tomato fruits.

Referring to FIG. 6 , FIG. 6 is a schematic structural diagram of a tomato identification and positioning system according to an embodiment of the present invention. The provided system includes: a fruit extraction module 61 , a fruit separation module 62 , a coordinate determination module 63 and a positioning module 64 .

Among them, the fruit extraction module 61 is used to obtain the to-be-detected image containing tomatoes, convert the to-be-detected image into an HSV image, and after masking the HSV image, integrate the HSV image into an RGB image and perform a Masking process to obtain the tomato image after removing the background.

Specifically, after acquiring the to-be-detected image containing tomatoes, the to-be-detected image is converted into an HSV image, and mask processing is performed on the converted HSV image. The specific steps are to separate the H channel, the S channel and the V channel from the HSV image. channel, the H channel and S channel are used as the original color information to calculate the color information saliency of the image, and the V channel is used as the original brightness information to calculate the brightness information saliency, through the weighted color information saliency and luminance information saliency to obtain a comprehensive saliency map. The selected thresholds are applied on H, S, and V for mask processing, and then the processed images are reintegrated into RGB images and masked, so that the background-removed tomato image can be obtained.

The fruit separation module 62 is used for cutting the red tomato image after removing the background through a watershed algorithm to obtain a tomato image after image segmentation.

Specifically, during the ripening process of tomatoes, fruits often touch together, and many different image processing algorithms cannot detect two touched tomatoes. In this embodiment, a watershed segmentation method based on graphic morphology is used to perform a watershed algorithm to perform watershed segmentation on the image, retain important contour information, correctly separate the connected fruits, realize image segmentation, and obtain a tomato image after image segmentation.

The coordinate determination module 63 is used to perform edge detection on the tomato image after the image segmentation, and determine the coordinate position of the tomato in the tomato image after the image segmentation;

Specifically, after the tomatoes in the tomato image are segmented, it is further necessary to obtain the center point and radius data of each tomato, so that the mechanical claw can grasp the tomato according to these values. Therefore, in this step , the Hough transform is used to detect the center and radius of the tomato in the image. The specific principle is to first perform edge detection on the tomato image after image segmentation to obtain the boundary points in the image, that is, the foreground points, and the foreground points are the tomato's Contour position, you can determine the center position of the tomato in the image and the radius of the tomato according to the coordinate position of the foreground point in the image, and then obtain the coordinate position of each tomato in the image.

The positioning module 64 is configured to acquire point cloud data of the tomato according to the coordinate position of the tomato in the tomato image after the image segmentation, and determine the actual position of the tomato according to the point cloud data of the tomato.

Specifically, after obtaining the coordinate position of each tomato in the image, since the depth information of the tomato in the image cannot be determined, it is necessary to confirm the depth information of the tomato in the actual position through the depth camera. Obtaining the point cloud data of the tomato in the image Since the point cloud data contains the depth information of the image, that is, the actual distance between the camera and the fruit, the three-dimensional model of the tomato can be reconstructed according to the obtained point cloud data to provide positioning information for tomato picking.

Through this system, based on the depth camera and the original RGB image, the spatial positioning of the tomato is realized, which lays the foundation for the fast and accurate picking of tomato fruits, and eliminates uneven light, branches and leaves, and the blocking of the fruit. The impact of shadows caused by mutual occlusion on fruit recognition enables the recognition of ripe red tomatoes, which reduces the cost of tomato recognition compared to traditional binocular cameras.

FIG. 7 illustrates a schematic structural diagram of a tomato identification and positioning device. As shown in FIG. 7 , the server may include: a processor (processor) 710, a memory (memory) 730 and a bus 740, wherein the processor 710 and the memory 730 pass through The bus 740 performs communication with each other. The processor 710 can call the logic instructions in the memory 730 to perform the following method: acquiring the to-be-detected image containing tomato, converting the to-be-detected image into an HSV image, and after performing mask processing on the HSV image, converting the detected image into an HSV image. The HSV image is integrated into an RGB image and masked to obtain a background-removed tomato image; the background-removed red tomato image is cut through a watershed algorithm to obtain a tomato image after image segmentation; Perform edge detection on the tomato image after the image segmentation to determine the coordinate position of the tomato in the tomato image after the image segmentation; obtain the point cloud data of the tomato according to the coordinate position of the tomato in the tomato image after the image segmentation , and determine the actual position of the tomato according to the point cloud data of the tomato.

This embodiment also provides a computer program product, where the computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program includes program instructions, and when the program instructions are executed by a computer, The computer can execute the methods provided by the above method embodiments, for example, including: acquiring an image to be detected including tomatoes, converting the to-be-detected image into an HSV image, and after performing mask processing on the HSV image, converting the image to be detected The HSV image is integrated into an RGB image and masked to obtain a tomato image after background removal; the red tomato image after background removal is cut through a watershed algorithm to obtain a tomato image after image segmentation; Perform edge detection on the tomato image of the tomato image to determine the coordinate position of the tomato in the tomato image after the image segmentation; obtain the point cloud data of the tomato according to the coordinate position of the tomato in the tomato image after the image segmentation, According to the point cloud data of the tomato, the actual position of the tomato is determined.

This embodiment provides a non-transitory computer-readable storage medium, where the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions cause the computer to execute the methods provided by the foregoing method embodiments, for example, including : Obtain an image to be detected containing tomatoes, convert the to-be-detected image into an HSV image, and after masking the HSV image, integrate the HSV image into an RGB image and perform masking processing to obtain a background-removed image. The red tomato image after the background removal is cut through the watershed algorithm to obtain the tomato image after image segmentation; the edge detection is performed on the tomato image after the image segmentation, and it is determined that the tomato image is after the image segmentation. According to the coordinate position of the tomato in the tomato image after the image segmentation, the point cloud data of the tomato is obtained, and the actual point cloud data of the tomato is determined according to the point cloud data of the tomato. Location.

The device embodiments described above are only illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in One place, or it can be distributed over multiple network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.

From the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and certainly can also be implemented by hardware. Based on this understanding, the above-mentioned technical solutions can be embodied in the form of software products in essence or the parts that make contributions to the prior art, and the computer software products can be stored in computer-readable storage media, such as ROM/RAM, magnetic A disc, an optical disc, etc., includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform the methods described in various embodiments or some parts of the embodiments.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that it can still be The technical solutions described in the foregoing embodiments are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (4)

1. A tomato identification and positioning method is characterized by comprising the following steps:

acquiring an image to be detected containing tomatoes, converting the image to be detected into an HSV image, after masking the HSV image, integrating the HSV image into an RGB image and masking the RGB image to obtain a red tomato image with a background removed;

cutting the red tomato image with the background removed through a watershed algorithm to obtain an image-segmented tomato image, and acquiring the number of tomatoes in the image-segmented tomato image;

performing edge detection on the tomato image after the image segmentation, and determining the coordinate position of the red tomato in the tomato image after the image segmentation, specifically comprising: taking the number of the tomatoes as circle detection input parameters of Hough transform, and carrying out edge detection on the segmented tomato image to obtain foreground points of the segmented tomato image;

converting the segmented tomato image into an a-b coordinate system from an x-y coordinate system, and obtaining coordinates of a point of a circle center corresponding to the tomato in the segmented tomato image and the radius of the tomato according to the coordinates of the foreground point;

obtaining the coordinate position of the tomato in the tomato image after the image segmentation according to the coordinate of the point of the circle center and the radius of the tomato;

acquiring point cloud data of the red tomatoes according to the coordinate positions of the red tomatoes in the tomato images after the image segmentation, determining the actual positions of the red tomatoes according to the point cloud data of the red tomatoes,

the method comprises the steps of obtaining an image to be detected containing tomatoes, converting the image to be detected into an HSV image, integrating the HSV image into an RGB image after masking the HSV image, and masking the RGB image to obtain a red tomato image with a background removed, and specifically comprises the following steps:

converting the image to be detected into an HSV image, and separating an H channel histogram, an S channel histogram and a V channel histogram from the HSV image;

respectively carrying out mask processing on the H channel histogram, the S channel histogram and the V channel histogram on a preset threshold value, and integrating the H channel histogram, the S channel histogram and the V channel histogram which are subjected to the mask processing into an RGB image;

carrying out R, G, B masking treatment after the RGB image is closed and opened by a morphological technology, and further integrating to obtain a red tomato image with the background removed;

the method comprises the steps of obtaining point cloud data of the red tomatoes according to coordinate positions of the red tomatoes in the tomato images after image segmentation, and determining actual positions of the red tomatoes according to the point cloud data of the red tomatoes, and specifically comprises the following steps:

acquiring a depth image corresponding to the image to be detected through a depth camera;

obtaining point cloud data of the red tomatoes according to the coordinate positions of the red tomatoes in the tomato images after the image segmentation and the depth images;

determining the actual position of the red tomato through the point cloud data of the red tomato;

the background-removed red tomato image is cut by a watershed algorithm to obtain an image-segmented tomato image, and the method specifically comprises the following steps:

converting the red tomato image after the background is removed into a gray image, and calculating the gradient amplitude of the gray image according to a Sobel operator to obtain a gradient image of the gray image;

cutting the gradient image through a watershed algorithm to obtain an image of the tomato after image segmentation;

the step of cutting the gradient image through the watershed algorithm to obtain the tomato image after image segmentation further comprises:

and performing morphological opening and closing reconstruction operation and watershed calculation on the tomato image after the image segmentation to obtain a tomato image calculated by a secondary watershed algorithm, wherein the tomato image is used as a new tomato image after the image segmentation.

2. A tomato identification and positioning system, comprising:

the fruit extraction module is used for acquiring an image to be detected containing tomatoes, converting the image to be detected into an HSV image, after masking the HSV image, integrating the HSV image into an RGB image and masking the RGB image to obtain a red tomato image with a background removed;

the fruit separation module is used for cutting the red tomato image with the background removed through a watershed algorithm to obtain an image-segmented tomato image and acquiring the number of tomatoes in the image-segmented tomato image;

the coordinate determination module is configured to perform edge detection on the tomato image after the image segmentation, and determine a coordinate position of a red tomato in the tomato image after the image segmentation, and specifically includes: taking the number of the tomatoes as circle detection input parameters of Hough transform, and carrying out edge detection on the segmented tomato image to obtain foreground points of the segmented tomato image;

converting the segmented tomato image into an a-b coordinate system from an x-y coordinate system, and obtaining coordinates of a point of a circle center corresponding to the tomato in the segmented tomato image and the radius of the tomato according to the coordinates of the foreground point;

obtaining the coordinate position of the tomato in the tomato image after the image segmentation according to the coordinate of the point of the circle center and the radius of the tomato;

a positioning module for obtaining the point cloud data of the red tomato according to the coordinate position of the red tomato in the tomato image after the image segmentation, determining the actual position of the red tomato according to the point cloud data of the red tomato,

wherein, the fruit extraction module specifically comprises:

converting the image to be detected into an HSV image, and separating an H channel histogram, an S channel histogram and a V channel histogram from the HSV image;

respectively carrying out mask processing on the H channel histogram, the S channel histogram and the V channel histogram on a preset threshold value, and integrating the H channel histogram, the S channel histogram and the V channel histogram which are subjected to the mask processing into an RGB image;

carrying out R, G, B masking treatment after the RGB image is closed and opened by a morphological technology, and further integrating to obtain a red tomato image with the background removed;

the method comprises the steps of obtaining point cloud data of the red tomatoes according to coordinate positions of the red tomatoes in the tomato images after image segmentation, and determining actual positions of the red tomatoes according to the point cloud data of the red tomatoes, and specifically comprises the following steps:

acquiring a depth image corresponding to the image to be detected through a depth camera;

obtaining point cloud data of the red tomatoes according to the coordinate positions of the red tomatoes in the tomato images after the image segmentation and the depth images;

determining the actual position of the red tomato through the point cloud data of the red tomato;

the background-removed red tomato image is cut by a watershed algorithm to obtain an image-segmented tomato image, and the method specifically comprises the following steps:

converting the red tomato image after the background is removed into a gray image, and calculating the gradient amplitude of the gray image according to a Sobel operator to obtain a gradient image of the gray image;

cutting the gradient image through a watershed algorithm to obtain an image of the tomato after image segmentation; the step of cutting the gradient image through the watershed algorithm to obtain the tomato image after image segmentation further comprises:

and performing morphological opening and closing reconstruction operation and watershed calculation on the tomato image after the image segmentation to obtain a tomato image calculated by a secondary watershed algorithm, wherein the tomato image is used as a new tomato image after the image segmentation.

3. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method as claimed in claim 1.

4. Tomato identification and positioning equipment is characterized by comprising:

at least one processor; and at least one memory coupled to the processor, wherein:

the memory stores program instructions executable by the processor, the processor invoking the program instructions to perform the method of claim 1.

Images