CN110599507B - A kind of tomato identification and positioning method and system - Google Patents

A kind of tomato identification and positioning method and system Download PDF

Info

Publication number
CN110599507B
CN110599507B CN201810608651.7A CN201810608651A CN110599507B CN 110599507 B CN110599507 B CN 110599507B CN 201810608651 A CN201810608651 A CN 201810608651A CN 110599507 B CN110599507 B CN 110599507B
Authority
CN
China
Prior art keywords
image
tomato
red
tomatoes
segmentation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810608651.7A
Other languages
Chinese (zh)
Other versions
CN110599507A (en
Inventor
李寒
王玉竹
马克
张漫
高阳
孙建桐
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Agricultural University
Original Assignee
China Agricultural University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Agricultural University filed Critical China Agricultural University
Priority to CN201810608651.7A priority Critical patent/CN110599507B/en
Publication of CN110599507A publication Critical patent/CN110599507A/en
Application granted granted Critical
Publication of CN110599507B publication Critical patent/CN110599507B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/40Image enhancement or restoration using histogram techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/50Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/13Edge detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/194Segmentation; Edge detection involving foreground-background segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/50Depth or shape recovery
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/70Determining position or orientation of objects or cameras
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/90Determination of colour characteristics
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30181Earth observation
    • G06T2207/30188Vegetation; Agriculture

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The embodiment of the invention provides a tomato identification and positioning method and a tomato identification and positioning system, wherein the tomato identification and positioning method comprises the following steps: acquiring an image to be detected containing tomatoes, converting the image to be detected into an HSV image, after masking the HSV image, integrating the HSV image into an RGB image and masking the RGB image to obtain a tomato image with a background removed; cutting the red tomato image with the background removed through a watershed algorithm to obtain an image-segmented tomato image; performing edge detection on the tomato image after image segmentation, and determining the coordinate position of the tomato in the tomato image after image segmentation; and acquiring point cloud data of the tomatoes according to the coordinate positions of the tomatoes in the tomato images after the image segmentation, and determining the actual positions of the tomatoes according to the point cloud data of the tomatoes. The method provided by the invention eliminates the influence of uneven light, the occlusion of the branches and the leaves to the fruits, the shadow caused by the mutual occlusion of the fruits and the like on the fruit identification.

Description

一种番茄识别定位方法及系统A kind of tomato identification and positioning method and system

技术领域technical field

本发明实施例涉及图像识别技术领域,尤其涉及一种番茄识别定位方法及系统。Embodiments of the present invention relate to the technical field of image recognition, and in particular, to a method and system for tomato recognition and positioning.

背景技术Background technique

番茄是人们生活中栽培最为普遍的蔬菜之一,但人工采摘劳动强度大、效率低,开发番茄自动采收设备可以极大降低采摘成本,有效改善生产现状。由于番茄生长环境背景复杂、果实簇生重叠,能否准确获取果实位置和采摘信息,成为制约番茄机器人采摘的一大瓶颈。Tomato is one of the most common vegetables cultivated in people's life, but manual picking is labor-intensive and low-efficiency. The development of automatic tomato harvesting equipment can greatly reduce the picking cost and effectively improve the production status. Due to the complex background of tomato growing environment and overlapping fruit clusters, the ability to accurately obtain fruit location and picking information has become a major bottleneck restricting tomato robot picking.

在现有技术中,对番茄识别与定位方面做了很多相关研究与应用,其中常用的方法为使用双目立体视觉技术进行定位,具体为使用的是两个CCD摄影机采集图像,利用视觉从图像中恢复番茄位置信息,从而达到利用机械视觉控制机械手的运动和定位,去达到采摘番茄果实的目的。另一种番茄识别方法为采用线结构光视觉主动探测手段,具体为基于(2R-G-B)色差模型的目标区域分割方法,凸显不同太阳光照条件下成熟果实与背景的色彩特征差异,根据灰度距离映射关系建立深度图像,据此进行重叠果实的区域分割和立体测量。另外,仇瑞承等在文献《基于RGB-D相机的玉米茎粗测量方法》中,以小喇叭口期玉米为观测对象,利用RGB-D相机获取田间玉米的彩色图像和深度图像,通过对图像的处理,可快速获取作物表型参数。In the prior art, a lot of related researches and applications have been done on tomato identification and localization. The commonly used method is to use binocular stereo vision technology for localization. Specifically, two CCD cameras are used to collect images. The tomato position information is restored in the middle, so as to use the machine vision to control the movement and positioning of the manipulator, so as to achieve the purpose of picking tomato fruits. Another tomato identification method is to use the active detection method of line structured light vision, specifically the target area segmentation method based on the (2R-G-B) color difference model, which highlights the difference in color characteristics between the ripe fruit and the background under different sunlight conditions. The distance mapping relationship establishes a depth image, based on which regional segmentation and stereo measurement of overlapping fruits are performed. In addition, Qiu Ruicheng et al. in the document "Measurement Method of Corn Stem Thickness Based on RGB-D Camera", took corn at the small bell mouth stage as the observation object, and used the RGB-D camera to obtain the color image and depth image of field corn. processing, can quickly obtain crop phenotypic parameters.

在现有技术中,对番茄的定位识别方法过于复杂,使用的相机设备昂贵,同时采用线结构光视觉主动探测手段,不能有效识别红色成熟果实,在图像处理方面,不能同时有效分割果实与背景、果实粘连部分。In the prior art, the positioning and identification method of tomato is too complicated, the camera equipment used is expensive, and the active detection method of line structured light vision is used, which cannot effectively identify the red ripe fruit, and in terms of image processing, the fruit and the background cannot be effectively segmented at the same time. , Fruit adhesion part.

发明内容SUMMARY OF THE INVENTION

本发明实施例提供一种番茄识别定位方法及系统,用以解决现有技术中对番茄的定位识别方法过于复杂,使用的相机设备昂贵,同时采用线结构光视觉主动探测手段,不能有效识别红色成熟果实,在图像处理方面,不能同时有效分割果实与背景、果实粘连部分的问题。Embodiments of the present invention provide a tomato identification and positioning method and system, which are used to solve the problem that the prior art positioning and identification method for tomato is too complicated, the camera equipment used is expensive, and the line structured light vision active detection method is adopted, which cannot effectively identify red color. Ripe fruit, in terms of image processing, cannot effectively segment the fruit, the background, and the adhering part of the fruit at the same time.

根据本发明的第一方面,提供一种番茄识别定位方法,包括:According to a first aspect of the present invention, a tomato identification and positioning method is provided, comprising:

获取包含番茄的待检测图像,将所述待检测图像转化为HSV图像,在对所述HSV图像进行掩模处理后,将所述HSV图像整合为RGB图像并进行掩蔽处理,获得去除背景后的番茄图像;Obtain the to-be-detected image containing tomatoes, convert the to-be-detected image into an HSV image, and after masking the HSV image, integrate the HSV image into an RGB image and perform masking processing to obtain a background-removed image. tomato image;

将所述去除背景后的红色番茄图像,通过分水岭算法进行切割,获得图像分割后的番茄图像;The red tomato image after the background removal is cut through a watershed algorithm to obtain a tomato image after image segmentation;

对所述图像分割后的番茄图像进行边缘检测,确定番茄在所述图像分割后的番茄图像中的坐标位置;Perform edge detection on the tomato image after the image segmentation, and determine the coordinate position of the tomato in the tomato image after the image segmentation;

根据所述番茄在所述图像分割后的番茄图像中的坐标位置,获取所述番茄的点云数据,根据所述番茄的点云数据,确定所述番茄的实际位置。According to the coordinate position of the tomato in the tomato image after the image segmentation, the point cloud data of the tomato is acquired, and the actual position of the tomato is determined according to the point cloud data of the tomato.

其中,所述获取包含番茄的待检测图像,将所述待检测图像转化为HSV图像,在对所述HSV图像进行掩模处理后,将所述HSV图像整合为RGB图像并进行掩蔽处理,获得去除背景后的番茄图像,具体包括:Wherein, said acquiring the to-be-detected image containing tomatoes, converting the to-be-detected image into an HSV image, and after performing mask processing on the HSV image, integrating the HSV image into an RGB image and performing mask processing to obtain The tomato image after removing the background, including:

将所述待检测图像转化为HSV图像转换为HSV图像,从中分离出H通道直方图、S通道直方图和V通道直方图;分别对所述H通道直方图、S通道直方图和V通道直方图在预设阈值上进行掩模处理,将进行掩模处理后的H通道直方图、S通道直方图和V通道直方图整合为RGB图像;对所述RGB图像通过形态学技术进行闭、开操作后,进行R、G、B掩蔽处理,进而整合获得去除背景后的番茄图像。Convert the to-be-detected image into an HSV image and convert it into an HSV image, and separate out the H channel histogram, the S channel histogram and the V channel histogram; The image is masked on a preset threshold, and the H channel histogram, S channel histogram and V channel histogram after mask processing are integrated into an RGB image; the RGB image is closed and opened by morphological technology. After the operation, perform R, G, B masking processing, and then integrate to obtain the tomato image after removing the background.

其中,所述将所述去除背景后的红色番茄图像,通过分水岭算法进行切割,获得图像分割后的番茄图像,具体包括:Wherein, the red tomato image after the background removal is cut through a watershed algorithm to obtain a tomato image after image segmentation, specifically including:

将所述去除背景后的番茄图像转换为灰度图像,根据Sobel算子计算所述灰度图像的梯度幅值,获得所述灰度图像的梯度图像;通过分水岭算法对所述梯度图像进行切割,获得图像分割后的番茄图像。Converting the background-removed tomato image into a grayscale image, calculating the gradient magnitude of the grayscale image according to the Sobel operator, and obtaining the gradient image of the grayscale image; cutting the gradient image by a watershed algorithm , to obtain the tomato image after image segmentation.

其中,所述通过分水岭算法对所述梯度图像进行切割,获得图像分割后的番茄图像的步骤还包括:Wherein, the step of cutting the gradient image through the watershed algorithm to obtain the tomato image after the image segmentation further includes:

对所述图像分割后的番茄图像进行形态学开闭重建运算和分水岭计算,获得二次分水岭算法计算的番茄图像,作为新的图像分割后的番茄图像。The morphological opening and closing reconstruction operation and watershed calculation are performed on the tomato image after the image segmentation, to obtain the tomato image calculated by the secondary watershed algorithm, which is used as a new tomato image after image segmentation.

其中,所述获得图像分割后的番茄图像的步骤之后,还包括:获取所述图像分割后的番茄图像中,所述番茄的个数。Wherein, after the step of obtaining the image-segmented tomato image, the method further includes: obtaining the number of the tomatoes in the image-segmented tomato image.

其中,所述对所述图像分割后的番茄图像进行边缘检测,确定番茄在所述图像分割后的番茄图像中的坐标位置,具体包括:Wherein, performing edge detection on the image segmented tomato image to determine the coordinate position of the tomato in the segmented tomato image specifically includes:

将所述番茄的个数作为霍夫变换的圆检测输入参数,对所述分割后的番茄图像进行边缘检测,获得所述分割后的番茄图像的前景点;将所述分割后的番茄图像由x-y坐标系转换为a-b坐标系,根据所述前景点的坐标,获得所述分割后的番茄图像中番茄对应的圆心的点的坐标及番茄的半径;根据所述圆心的点的坐标及番茄的半径,获得所述番茄在所述图像分割后的番茄图像中的坐标位置。The number of the tomatoes is used as the circle detection input parameter of Hough transform, and edge detection is performed on the segmented tomato image to obtain the foreground point of the segmented tomato image; the segmented tomato image is composed of The x-y coordinate system is converted into the a-b coordinate system, and according to the coordinates of the foreground point, the coordinates of the center point and the radius of the tomato corresponding to the tomato in the segmented tomato image are obtained; according to the coordinates of the center point and the radius of the tomato Radius, to obtain the coordinate position of the tomato in the tomato image after the image is segmented.

其中,所述根据所述番茄在所述图像分割后的番茄图像中的坐标位置,获取所述番茄的点云数据,根据所述番茄的点云数据,确定所述番茄的实际位置,具体包括:Wherein, acquiring the point cloud data of the tomato according to the coordinate position of the tomato in the tomato image after the image segmentation, and determining the actual position of the tomato according to the point cloud data of the tomato, specifically includes: :

通过深度相机获取所述待检测图像对应的深度图像;通过所述番茄在所述图像分割后的番茄图像中的坐标位置和所述深度图像,获得所述番茄的点云数据;通过所述番茄的点云数据,确定所述番茄的实际位置。Obtain the depth image corresponding to the to-be-detected image through a depth camera; obtain the point cloud data of the tomato through the coordinate position of the tomato in the image segmented tomato image and the depth image; through the tomato point cloud data to determine the actual location of the tomato.

根据本发明的第二方面,提供一种番茄识别定位系统,包括:According to a second aspect of the present invention, a tomato identification and positioning system is provided, comprising:

果实提取模块,用于获取包含番茄的待检测图像,将所述待检测图像转化为HSV图像,在对所述HSV图像进行掩模处理后,将所述HSV图像整合为RGB图像并进行掩蔽处理,获得去除背景后的番茄图像;The fruit extraction module is used to obtain the to-be-detected image containing tomatoes, convert the to-be-detected image into an HSV image, and after masking the HSV image, integrate the HSV image into an RGB image and perform masking processing , get the tomato image after removing the background;

果实分离模块,用于将所述去除背景后的红色番茄图像,通过分水岭算法进行切割,获得图像分割后的番茄图像;The fruit separation module is used for cutting the red tomato image after removing the background by the watershed algorithm to obtain the tomato image after image segmentation;

坐标确定模块,用于对所述图像分割后的番茄图像进行边缘检测,确定番茄在所述图像分割后的番茄图像中的坐标位置;a coordinate determination module, configured to perform edge detection on the tomato image after the image segmentation, and determine the coordinate position of the tomato in the tomato image after the image segmentation;

定位模块,用于根据所述番茄在所述图像分割后的番茄图像中的坐标位置,获取所述番茄的点云数据,根据所述番茄的点云数据,确定所述番茄的实际位置。The positioning module is configured to obtain the point cloud data of the tomato according to the coordinate position of the tomato in the tomato image after the image segmentation, and determine the actual position of the tomato according to the point cloud data of the tomato.

根据本发明的第三方面,提供一种计算机可读存储介质,其上存储有计算机程序,该程序被处理器执行时实现如上述第一方面所提供的方法的步骤。According to a third aspect of the present invention, there is provided a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the steps of the method provided in the above-mentioned first aspect.

根据本发明的第四方,提供一种番茄识别定位设备,包括:According to the fourth aspect of the present invention, a tomato identification and positioning device is provided, comprising:

至少一个处理器;以及与所述处理器连接的至少一个存储器,其中:at least one processor; and at least one memory connected to the processor, wherein:

所述存储器存储有可被所述处理器执行的程序指令,所述处理器调用所述程序指令能够执行如上述第一方面所提供的方法的步骤。The memory stores program instructions executable by the processor, and the processor invokes the program instructions to perform the steps of the method provided in the first aspect above.

本发明实施例提供的番茄识别定位方法及设备,基于深度相机和原始RGB图像,实现了番茄在空间上的定位,为番茄果实快速准确采摘奠定基础,消除了光线不均、枝干、叶片对果实的遮挡、以及果实之间的相互遮挡造成的阴影等对果实识别的影响,实现了对成熟红色番茄的识别,相对于传统的双目相机,降低了番茄识别的成本。The tomato identification and positioning method and device provided by the embodiments of the present invention realize the spatial positioning of the tomato based on the depth camera and the original RGB image, lay a foundation for the fast and accurate picking of the tomato fruit, and eliminate uneven light, branches and leaves. The occlusion of the fruit and the shadows caused by the mutual occlusion between the fruits have an impact on fruit recognition, realizing the recognition of ripe red tomatoes, which reduces the cost of tomato recognition compared to the traditional binocular camera.

附图说明Description of drawings

为了更清楚地说明本发明实施例或现有技术中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍,显而易见地,下面描述中的附图是本发明的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.

图1为本发明实施例提供的一种番茄识别定位方法的流程图;1 is a flowchart of a method for identifying and locating tomato according to an embodiment of the present invention;

图2为本发明实施例提供的一种番茄识别定位方法中,基于HSV的红色番茄果实提取流程示意图;2 is a schematic flowchart of a red tomato fruit extraction process based on HSV in a tomato identification and positioning method provided in an embodiment of the present invention;

图3为本发明实施例提供的一种番茄识别定位方法中,番茄果实分离流程示意图;3 is a schematic diagram of a tomato fruit separation process in a tomato identification and positioning method provided by an embodiment of the present invention;

图4为本发明实施例提供的一种番茄识别定位方法中,果实球心和半径计算流程示意图;4 is a schematic diagram of the calculation flow of the fruit sphere center and radius in a tomato identification and positioning method provided by an embodiment of the present invention;

图5为本发明实施例提供的一种番茄识别定位方法中,番茄实际位置定位的流程示意图;5 is a schematic flowchart of the actual location of tomato in a tomato identification and positioning method provided by an embodiment of the present invention;

图6为本发明实施例提供的一种番茄识别定位系统的结构示意图;6 is a schematic structural diagram of a tomato identification and positioning system according to an embodiment of the present invention;

图7为本发明实施例提供的一种番茄识别定位设备的结构示意图。FIG. 7 is a schematic structural diagram of a tomato identification and positioning device according to an embodiment of the present invention.

具体实施方式Detailed ways

为使本发明实施例的目的、技术方案和优点更加清楚,下面将结合本发明实施例中的附图,对本发明实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例是本发明一部分实施例,而不是全部的实施例。基于本发明中的实施例,本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例,都属于本发明保护的范围。In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

参考图1,图1为本发明实施例提供的一种番茄识别定位方法的流程图,所提供的方法包括:Referring to FIG. 1, FIG. 1 is a flowchart of a method for identifying and locating tomato according to an embodiment of the present invention, and the provided method includes:

S1,获取包含番茄的待检测图像,将所述待检测图像转化为HSV图像,在对所述HSV图像进行掩模处理后,将所述HSV图像整合为RGB图像并进行掩蔽处理,获得去除背景后的番茄图像。S1, acquire an image to be detected containing tomatoes, convert the to-be-detected image into an HSV image, and after masking the HSV image, integrate the HSV image into an RGB image and perform masking processing to obtain a background-removed image Post tomato image.

具体的,在获取了包含番茄的待检测图像后,将待检测图像转化为HSV图像,对转化后的HSV图像进行掩模处理,具体步骤为从HSV图像中分离出H通道、S通道和V通道,将其中的H通道和S通道作为原始的颜色信息来计算图像的颜色信息显著性,把其中的V通道作为原始的亮度信息并由此计算亮度信息显著性,通过加权的颜色信息显著性和亮度信息显著性得到综合显著图。分别在H、S、V上应用选定的阈值进行掩模处理,随后将处理后的图像重新整合为RGB图像,并进行掩蔽处理,从而可以获得去除背景后的番茄图像。Specifically, after acquiring the to-be-detected image containing tomatoes, the to-be-detected image is converted into an HSV image, and mask processing is performed on the converted HSV image. The specific steps are to separate the H channel, the S channel and the V channel from the HSV image. channel, the H channel and S channel are used as the original color information to calculate the color information saliency of the image, and the V channel is used as the original brightness information to calculate the brightness information saliency, through the weighted color information saliency and luminance information saliency to obtain a comprehensive saliency map. The selected thresholds are applied on H, S, and V for mask processing, and then the processed images are reintegrated into RGB images and masked, so that the background-removed tomato image can be obtained.

S2,将所述去除背景后的红色番茄图像,通过分水岭算法进行切割,获得图像分割后的番茄图像。S2, the red tomato image after the background removal is cut through a watershed algorithm to obtain a tomato image after image segmentation.

具体的,番茄在成熟过程中,经常有果实触碰在一起,许多不同的图像处理算法无法检测到两个触碰的番茄。本实施例中,采用基于图形形态的分水岭分割方法,进行分水岭算法,对图像进行分水岭分割,保留重要的轮廓信息,正确分离连接的果实,实现图像分割,获得进行了图像分割后的番茄图像。Specifically, during the ripening process of tomatoes, fruits often touch together, and many different image processing algorithms cannot detect two touched tomatoes. In this embodiment, a watershed segmentation method based on graphic morphology is used to perform a watershed algorithm to perform watershed segmentation on the image, retain important contour information, correctly separate the connected fruits, realize image segmentation, and obtain a tomato image after image segmentation.

S3,对所述图像分割后的番茄图像进行边缘检测,确定番茄在所述图像分割后的番茄图像中的坐标位置。S3: Perform edge detection on the tomato image after the image segmentation, and determine the coordinate position of the tomato in the tomato image after the image segmentation.

具体的,在将番茄图像中的番茄进行了分割以后,进一步的还需要获得每一个番茄的中心点和半径数据,以供机械爪能够根据这些数值来进行番茄的抓取,因此,在本步骤中,通过霍夫变换来检测图像中番茄的圆心和半径,具体原理是首先对图像分割后的番茄图像进行边缘检测,获得图像中的边界点,即前景点,则其中前景点即为番茄的轮廓位置,则可以根据前景点的在图中的坐标位置,确定番茄在图中的圆心位置和番茄的半径大小,进而获得每个番茄在图像中的坐标位置。Specifically, after the tomatoes in the tomato image are segmented, it is further necessary to obtain the center point and radius data of each tomato, so that the mechanical claw can grasp the tomato according to these values. Therefore, in this step , the Hough transform is used to detect the center and radius of the tomato in the image. The specific principle is to first perform edge detection on the tomato image after image segmentation to obtain the boundary points in the image, that is, the foreground points, and the foreground points are the tomato's Contour position, you can determine the center position of the tomato in the image and the radius of the tomato according to the coordinate position of the foreground point in the image, and then obtain the coordinate position of each tomato in the image.

S4,根据所述番茄在所述图像分割后的番茄图像中的坐标位置,获取所述番茄的点云数据,根据所述番茄的点云数据,确定所述番茄的实际位置。S4: Acquire point cloud data of the tomato according to the coordinate position of the tomato in the tomato image after the image segmentation, and determine the actual position of the tomato according to the point cloud data of the tomato.

具体的,在获得了每个番茄在图像中的坐标位置后,由于还不能确定番茄在图像中的深度信息,因此需要通过深度相机确认番茄在实际位置的深度信息,具体实施中,通过深度相机获得番茄在图像中的点云数据由于点云数据包含图像的深度信息,即相机与果实实际相隔距离,所以可根据获取的点云数据重建番茄三维模型,为番茄采摘提供定位信息。Specifically, after obtaining the coordinate position of each tomato in the image, since the depth information of the tomato in the image cannot be determined, it is necessary to confirm the depth information of the tomato in the actual position through the depth camera. Obtaining the point cloud data of the tomato in the image Since the point cloud data contains the depth information of the image, that is, the actual distance between the camera and the fruit, the three-dimensional model of the tomato can be reconstructed according to the obtained point cloud data to provide positioning information for tomato picking.

通过此方法,基于深度相机和原始RGB图像,实现了番茄在空间上的定位,为番茄果实快速准确采摘奠定基础,消除了光线不均、枝干、叶片对果实的遮挡、以及果实之间的相互遮挡造成的阴影等对果实识别的影响,实现了对成熟红色番茄的识别,相对于传统的双目相机,降低了番茄识别的成本。Through this method, based on the depth camera and the original RGB image, the spatial positioning of the tomato is realized, which lays the foundation for the fast and accurate picking of the tomato fruit, and eliminates uneven light, the occlusion of the fruit by branches and leaves, and the interference between the fruits. The impact of shadows caused by mutual occlusion on fruit recognition enables the recognition of ripe red tomatoes, which reduces the cost of tomato recognition compared to traditional binocular cameras.

在上述实施例的基础上,所述获取包含番茄的待检测图像,将所述待检测图像转化为HSV图像,在对所述HSV图像进行掩模处理后,将所述HSV图像整合为RGB图像并进行掩蔽处理,获得去除背景后的番茄图像,具体包括:On the basis of the above embodiment, the to-be-detected image containing tomatoes is acquired, the to-be-detected image is converted into an HSV image, and the HSV image is integrated into an RGB image after mask processing is performed on the HSV image And perform masking processing to obtain the tomato image after removing the background, including:

将所述待检测图像转化为HSV图像转换为HSV图像,从中分离出H通道直方图、S通道直方图和V通道直方图;Converting the to-be-detected image into an HSV image is converted into an HSV image, from which an H channel histogram, an S channel histogram and a V channel histogram are separated;

分别对所述H通道直方图、S通道直方图和V通道直方图在预设阈值上进行掩模处理,将进行掩模处理后的H通道直方图、S通道直方图和V通道直方图整合为RGB图像;The H channel histogram, the S channel histogram and the V channel histogram are respectively subjected to mask processing on a preset threshold, and the H channel histogram, the S channel histogram and the V channel histogram after the mask processing are integrated is an RGB image;

对所述RGB图像通过形态学技术进行闭、开操作后,进行R、G、B掩蔽处理,进而整合获得去除背景后的番茄图像。After the RGB image is closed and opened by morphological technology, R, G, B masking processing is performed, and then the tomato image after background removal is obtained by integration.

具体的,在对待检测图像的背景消除步骤中,具体步骤如图2所示,HSV色彩空间是由色调(Hue)、饱和度(Saturation)和亮度(Value)三个分量组成。色调(H)表示不同的颜色,取值范围为0~360°;饱和度(S)表示颜色的深浅,通常取0%~100%;亮度(V)表示颜色的明暗程度,取值范围为0%(黑)到100%(白)。首先将待检测图像转换为HSV图像,从中分离出H通道、S通道和V通道,将其中的H通道和S通道作为原始的颜色信息来计算图像的颜色信息显著性,把其中的V通道作为原始的亮度信息并由此计算亮度信息显著性,通过加权的颜色信息显著性和亮度信息显著性得到综合显著图。本实施例中,先得到直方图,在此基础上,选择H、S和V分量的阈值,定义感兴趣的区域为0到1。在色相直方图中,0值为红色,为识别红色番茄,选取0.07为高色调H阈值,并使用Otsu算法检测红色;同样,在饱和度S直方图中,选取0.35为饱和度阈值;在亮度V直方图中,选取0.4为亮度阈值。然后分别在H、S、V上分别应用所选阈值,进行掩模处理,然后整合为RGB图像。此时图像中只剩下红色的番茄,之后运用形态学技术进行闭、开操作,再进行R、G、B掩蔽处理,将其整合后,最终得到去除背景后的红色番茄图像。Specifically, in the background removal step of the image to be detected, the specific steps are shown in FIG. 2 , the HSV color space is composed of three components: hue (Hue), saturation (Saturation) and brightness (Value). Hue (H) represents different colors, and the value range is 0 to 360°; Saturation (S) represents the depth of the color, usually 0% to 100%; Brightness (V) represents the lightness and darkness of the color, and the value range is 0% (black) to 100% (white). First, the image to be detected is converted into an HSV image, and the H channel, S channel and V channel are separated from it, and the H channel and S channel are used as the original color information to calculate the color information saliency of the image, and the V channel is used as the original color information. The original luminance information and thus the luminance information saliency are calculated, and the comprehensive saliency map is obtained by weighting the color information saliency and the luminance information saliency. In this embodiment, the histogram is obtained first, and on this basis, the thresholds of the H, S, and V components are selected, and the region of interest is defined as 0 to 1. In the hue histogram, 0 is red, in order to identify red tomatoes, select 0.07 as the high hue H threshold, and use the Otsu algorithm to detect red; similarly, in the saturation S histogram, select 0.35 as the saturation threshold; in the brightness In the V histogram, 0.4 is selected as the brightness threshold. The selected thresholds are then applied on H, S, V, respectively, masked, and then integrated into an RGB image. At this time, only the red tomato remains in the image, and then the morphological technology is used to close and open, and then R, G, B masking processing is performed, and after integration, the red tomato image with the background removed is finally obtained.

在上述实施例的基础上,所述将所述去除背景后的红色番茄图像,通过分水岭算法进行切割,获得图像分割后的番茄图像,具体包括:On the basis of the above-mentioned embodiment, the red tomato image after the background removal is cut through a watershed algorithm to obtain a tomato image after image segmentation, which specifically includes:

将所述去除背景后的番茄图像转换为灰度图像,根据Sobel算子计算所述灰度图像的梯度幅值,获得所述灰度图像的梯度图像;The tomato image after the described background removal is converted into a grayscale image, the gradient magnitude of the grayscale image is calculated according to the Sobel operator, and the gradient image of the grayscale image is obtained;

通过分水岭算法对所述梯度图像进行切割,获得图像分割后的番茄图像。The gradient image is cut through a watershed algorithm to obtain a segmented tomato image.

具体的,番茄在成熟过程中,经常有果实触碰在一起,许多不同的图像处理算法无法检测到两个触碰的番茄。本实施例中,采用分水岭算法对去除了背景的番茄图像进行分水岭算法计算。具体实施中,首先利用形态学的开闭重建运算对梯度图像进行滤波处理,消除图像中由非规则灰度扰动和噪声引起的局部极值;然后对图像进行分水岭分割,保留重要的轮廓信息;最后结合归一化积相关灰度匹配的算法进行相似性度量,通过基于图形的形态模板指导区域合并,将目标图形区域从图像自然背景中分离出来。Specifically, during the ripening process of tomatoes, fruits often touch together, and many different image processing algorithms cannot detect two touched tomatoes. In this embodiment, the watershed algorithm is used to perform the watershed algorithm calculation on the tomato image with the background removed. In the specific implementation, firstly, the gradient image is filtered by the morphological opening and closing reconstruction operation to eliminate the local extrema caused by irregular grayscale disturbance and noise in the image; then the image is divided into watershed to retain important contour information; Finally, the similarity measurement is carried out by combining the normalized product correlation gray matching algorithm, and the target image area is separated from the natural background of the image by guiding the region merging based on the shape template.

在上述实施例的基础上,所述通过分水岭算法对所述梯度图像进行切割,获得图像分割后的番茄图像的步骤还包括:On the basis of the above-mentioned embodiment, the step of cutting the gradient image through the watershed algorithm to obtain the tomato image after the image segmentation further includes:

对所述图像分割后的番茄图像进行形态学开闭重建运算和分水岭计算,获得二次分水岭算法计算的番茄图像,作为新的图像分割后的番茄图像。The morphological opening and closing reconstruction operation and watershed calculation are performed on the tomato image after the image segmentation, to obtain the tomato image calculated by the secondary watershed algorithm, which is used as a new tomato image after image segmentation.

其中,所述获得图像分割后的番茄图像的步骤之后,还包括:获取所述图像分割后的番茄图像中,所述番茄的个数。Wherein, after the step of obtaining the image-segmented tomato image, the method further includes: obtaining the number of the tomatoes in the image-segmented tomato image.

具体的,参考图3,在进行果实分离的步骤中,可以采用两次分水岭算法,来提升果实分割的准确性,具体实施中,对去除背景后的红色番茄图像,将图像转换为灰度图像,应用Sobel算子计算梯度幅值,检测水平边缘和垂直边缘,在梯度图像上进行第一次分水岭算法,然后采用形态学的开闭重建运算,移除斑点,使得图像平滑,为了得到良好的前景,对图像的区域极值进行修改,实现最大区域的检测。为了区分图像的背景和前景,提出改进的阈值法,背景像素被标记为黑色。运用欧几里德矩阵进行距离变换后,运用第二次分水岭算法获取山脊线,在第一次分水岭基础上,正确分离连接的水果,实现图像分割,并获取分割后果实个数N,作为下一步基于霍夫变换的果实定位的输入参数。Specifically, referring to FIG. 3 , in the step of fruit separation, two watershed algorithms can be used to improve the accuracy of fruit segmentation. In the specific implementation, the red tomato image after background removal is converted into a grayscale image. , the Sobel operator is used to calculate the gradient magnitude, the horizontal and vertical edges are detected, the first watershed algorithm is performed on the gradient image, and then the morphological opening and closing reconstruction operation is used to remove the blobs and make the image smooth. Foreground, modify the regional extreme value of the image to realize the detection of the largest area. In order to distinguish the background and foreground of the image, an improved thresholding method is proposed, and the background pixels are marked as black. After using the Euclidean matrix for distance transformation, the second watershed algorithm is used to obtain the ridge line. On the basis of the first watershed, the connected fruits are correctly separated to achieve image segmentation, and the number of fruits after segmentation N is obtained as the next step. Input parameters for one-step Hough transform-based fruit localization.

在上述实施例的基础上,所述对所述图像分割后的番茄图像进行边缘检测,确定番茄在所述图像分割后的番茄图像中的坐标位置,具体包括:On the basis of the above-mentioned embodiment, the performing edge detection on the image segmented tomato image to determine the coordinate position of the tomato in the segmented tomato image specifically includes:

将所述番茄的个数作为霍夫变换的圆检测输入参数,对所述分割后的番茄图像进行边缘检测,获得所述分割后的番茄图像的前景点;Using the number of the tomatoes as the circle detection input parameter of the Hough transform, performing edge detection on the segmented tomato image to obtain the foreground point of the segmented tomato image;

将所述分割后的番茄图像由x-y坐标系转换为a-b坐标系,根据所述前景点的坐标,获得所述分割后的番茄图像中番茄对应的圆心的点的坐标及番茄的半径;Convert the segmented tomato image from the x-y coordinate system to the a-b coordinate system, and obtain the coordinates of the center point of the tomato and the radius of the tomato in the segmented tomato image according to the coordinates of the foreground point;

根据所述圆心的点的坐标及番茄的半径,获得所述番茄在所述图像分割后的番茄图像中的坐标位置。According to the coordinates of the center point of the circle and the radius of the tomato, the coordinate position of the tomato in the tomato image after the image segmentation is obtained.

具体的,在番茄坐标计算过程中,如图4所示,将红色番茄果实分离获取的果实个数N作为霍夫变换的圆检测输入参数。应用霍夫变换检测圆心和半径的原理是:首先对输入图像进行边缘检测,获取边界点,即前景点。假如图像中存在圆形,那么其轮廓必定属于前景点。将圆形的一般性方程换一种方式表示,进行坐标变换。由x-y坐标系转换到a-b坐标系,那么x-y坐标系中圆形边界上的一点对应到a-b坐标系中即为一个圆。那x-y坐标系中一个圆形边界上有很多个点,对应到a-b坐标系中就会有很多个圆。由于原图像中这些点都在同一个圆形上,那么转换后a,b必定也满足a-b坐标系下的所有圆形的方程式。直观表现为这许多点对应的圆都会相交于一个点,那么这个交点就可能是圆心(a,b)。统计局部交点处圆的个数,取每一个局部最大值,就可以获得原图像中对应的圆形的圆心坐标(a,b)。一旦在某一个r下面检测到圆,那么r的值也就随之确定,最终获得番茄在图像中的坐标(a,b,r)。Specifically, in the tomato coordinate calculation process, as shown in FIG. 4 , the number N of fruits obtained by separating the red tomato fruits is used as the circle detection input parameter of the Hough transform. The principle of applying the Hough transform to detect the center and radius of the circle is as follows: first, perform edge detection on the input image, and obtain the boundary point, that is, the foreground point. If there is a circle in the image, then its outline must belong to the foreground point. Represent the general equation of a circle in another way, and perform coordinate transformation. Converting from the x-y coordinate system to the a-b coordinate system, then a point on the circle boundary in the x-y coordinate system corresponds to the a-b coordinate system, which is a circle. There are many points on a circular boundary in the x-y coordinate system, and there will be many circles in the a-b coordinate system. Since these points in the original image are all on the same circle, a and b must also satisfy the equations of all circles in the a-b coordinate system after conversion. Intuitively, the circles corresponding to these many points will intersect at a point, then this intersection point may be the center of the circle (a, b). Count the number of circles at the local intersection points, and take each local maximum value to obtain the center coordinates (a, b) of the corresponding circle in the original image. Once a circle is detected under a certain r, the value of r is determined accordingly, and finally the coordinates (a, b, r) of the tomato in the image are obtained.

在上述实施例的基础上,所述根据所述番茄在所述图像分割后的番茄图像中的坐标位置,获取所述番茄的点云数据,根据所述番茄的点云数据,确定所述番茄的实际位置,具体包括:On the basis of the above embodiment, the point cloud data of the tomato is acquired according to the coordinate position of the tomato in the tomato image after the image segmentation, and the tomato is determined according to the point cloud data of the tomato actual location, including:

通过深度相机获取所述待检测图像对应的深度图像;Obtain a depth image corresponding to the to-be-detected image through a depth camera;

通过所述番茄在所述图像分割后的番茄图像中的坐标位置和所述深度图像,获得所述番茄的点云数据;Obtain the point cloud data of the tomato through the coordinate position of the tomato in the tomato image after the image segmentation and the depth image;

通过所述番茄的点云数据,确定所述番茄的实际位置。The actual position of the tomato is determined through the point cloud data of the tomato.

具体的,获得番茄的实际位置的步骤聚参照图5所示,选用微软公司生产的Kinect2.0相机采集番茄植株信息,可以在采集到番茄植株的彩色RGB图像作为待检测图像,同时可以采集到番茄植株的深度信息;将RGB图像与深度信息进行匹配,根据彩色图像处理获得的红色番茄图像(a,b,r)坐标,获取对应的红色番茄的点云数据。由于点云数据包含图像的深度信息,即RGB-D相机与果实实际相隔距离,所以可根据获取的点云数据重建番茄三维模型,为番茄采摘提供定位信息。Specifically, the steps of obtaining the actual position of the tomato are shown in Figure 5. The Kinect2.0 camera produced by Microsoft Corporation is used to collect the tomato plant information, and the color RGB image of the tomato plant can be collected as the image to be detected. The depth information of the tomato plant; the RGB image is matched with the depth information, and the point cloud data of the corresponding red tomato is obtained according to the (a, b, r) coordinates of the red tomato image obtained by color image processing. Since the point cloud data contains the depth information of the image, that is, the actual distance between the RGB-D camera and the fruit, the tomato 3D model can be reconstructed according to the obtained point cloud data to provide positioning information for tomato picking.

综上所述,本发明实施例通过基于RGB-D相机的番茄识别与定位方法,消除了光线不均、枝干、叶片对果实的遮挡、以及果实之间的相互遮挡造成的阴影等对果实识别的影响,实现了对成熟红色番茄的识别,对采摘过程中对果实的保护等具有一定的意义,同时基于相对双目相机价格便宜的深度相机,对具有复杂背景、相互粘连的红色番茄果实实现快速识别,为番茄果实快速准确采摘奠定基础。To sum up, the embodiment of the present invention eliminates the uneven light, the occlusion of the fruit by the branches and leaves, and the shadow caused by the mutual occlusion between the fruits through the tomato identification and positioning method based on the RGB-D camera. The impact of recognition can realize the recognition of ripe red tomato, which is of certain significance for the protection of the fruit during the picking process. At the same time, based on the cheap depth camera relative to the binocular camera, the red tomato fruit with complex background and mutual adhesion has a certain significance. Realize rapid identification and lay the foundation for fast and accurate picking of tomato fruits.

参考图6,图6为本发明实施例提供的一种番茄识别定位系统的结构示意图,所提供的系统包括:果实提取模块61,果实分离模块62,坐标确定模块63和定位模块64。Referring to FIG. 6 , FIG. 6 is a schematic structural diagram of a tomato identification and positioning system according to an embodiment of the present invention. The provided system includes: a fruit extraction module 61 , a fruit separation module 62 , a coordinate determination module 63 and a positioning module 64 .

其中,果实提取模块61用于获取包含番茄的待检测图像,将所述待检测图像转化为HSV图像,在对所述HSV图像进行掩模处理后,将所述HSV图像整合为RGB图像并进行掩蔽处理,获得去除背景后的番茄图像。Among them, the fruit extraction module 61 is used to obtain the to-be-detected image containing tomatoes, convert the to-be-detected image into an HSV image, and after masking the HSV image, integrate the HSV image into an RGB image and perform a Masking process to obtain the tomato image after removing the background.

具体的,在获取了包含番茄的待检测图像后,将待检测图像转化为HSV图像,对转化后的HSV图像进行掩模处理,具体步骤为从HSV图像中分离出H通道、S通道和V通道,将其中的H通道和S通道作为原始的颜色信息来计算图像的颜色信息显著性,把其中的V通道作为原始的亮度信息并由此计算亮度信息显著性,通过加权的颜色信息显著性和亮度信息显著性得到综合显著图。分别在H、S、V上应用选定的阈值进行掩模处理,随后将处理后的图像重新整合为RGB图像,并进行掩蔽处理,从而可以获得去除背景后的番茄图像。Specifically, after acquiring the to-be-detected image containing tomatoes, the to-be-detected image is converted into an HSV image, and mask processing is performed on the converted HSV image. The specific steps are to separate the H channel, the S channel and the V channel from the HSV image. channel, the H channel and S channel are used as the original color information to calculate the color information saliency of the image, and the V channel is used as the original brightness information to calculate the brightness information saliency, through the weighted color information saliency and luminance information saliency to obtain a comprehensive saliency map. The selected thresholds are applied on H, S, and V for mask processing, and then the processed images are reintegrated into RGB images and masked, so that the background-removed tomato image can be obtained.

果实分离模块62用于将所述去除背景后的红色番茄图像,通过分水岭算法进行切割,获得图像分割后的番茄图像。The fruit separation module 62 is used for cutting the red tomato image after removing the background through a watershed algorithm to obtain a tomato image after image segmentation.

具体的,番茄在成熟过程中,经常有果实触碰在一起,许多不同的图像处理算法无法检测到两个触碰的番茄。本实施例中,采用基于图形形态的分水岭分割方法,进行分水岭算法,对图像进行分水岭分割,保留重要的轮廓信息,正确分离连接的果实,实现图像分割,获得进行了图像分割后的番茄图像。Specifically, during the ripening process of tomatoes, fruits often touch together, and many different image processing algorithms cannot detect two touched tomatoes. In this embodiment, a watershed segmentation method based on graphic morphology is used to perform a watershed algorithm to perform watershed segmentation on the image, retain important contour information, correctly separate the connected fruits, realize image segmentation, and obtain a tomato image after image segmentation.

坐标确定模块63用于对所述图像分割后的番茄图像进行边缘检测,确定番茄在所述图像分割后的番茄图像中的坐标位置;The coordinate determination module 63 is used to perform edge detection on the tomato image after the image segmentation, and determine the coordinate position of the tomato in the tomato image after the image segmentation;

具体的,在将番茄图像中的番茄进行了分割以后,进一步的还需要获得每一个番茄的中心点和半径数据,以供机械爪能够根据这些数值来进行番茄的抓取,因此,在本步骤中,通过霍夫变换来检测图像中番茄的圆心和半径,具体原理是首先对图像分割后的番茄图像进行边缘检测,获得图像中的边界点,即前景点,则其中前景点即为番茄的轮廓位置,则可以根据前景点的在图中的坐标位置,确定番茄在图中的圆心位置和番茄的半径大小,进而获得每个番茄在图像中的坐标位置。Specifically, after the tomatoes in the tomato image are segmented, it is further necessary to obtain the center point and radius data of each tomato, so that the mechanical claw can grasp the tomato according to these values. Therefore, in this step , the Hough transform is used to detect the center and radius of the tomato in the image. The specific principle is to first perform edge detection on the tomato image after image segmentation to obtain the boundary points in the image, that is, the foreground points, and the foreground points are the tomato's Contour position, you can determine the center position of the tomato in the image and the radius of the tomato according to the coordinate position of the foreground point in the image, and then obtain the coordinate position of each tomato in the image.

定位模块64用于根据所述番茄在所述图像分割后的番茄图像中的坐标位置,获取所述番茄的点云数据,根据所述番茄的点云数据,确定所述番茄的实际位置。The positioning module 64 is configured to acquire point cloud data of the tomato according to the coordinate position of the tomato in the tomato image after the image segmentation, and determine the actual position of the tomato according to the point cloud data of the tomato.

具体的,在获得了每个番茄在图像中的坐标位置后,由于还不能确定番茄在图像中的深度信息,因此需要通过深度相机确认番茄在实际位置的深度信息,具体实施中,通过深度相机获得番茄在图像中的点云数据由于点云数据包含图像的深度信息,即相机与果实实际相隔距离,所以可根据获取的点云数据重建番茄三维模型,为番茄采摘提供定位信息。Specifically, after obtaining the coordinate position of each tomato in the image, since the depth information of the tomato in the image cannot be determined, it is necessary to confirm the depth information of the tomato in the actual position through the depth camera. Obtaining the point cloud data of the tomato in the image Since the point cloud data contains the depth information of the image, that is, the actual distance between the camera and the fruit, the three-dimensional model of the tomato can be reconstructed according to the obtained point cloud data to provide positioning information for tomato picking.

通过此系统,基于深度相机和原始RGB图像,实现了番茄在空间上的定位,为番茄果实快速准确采摘奠定基础,消除了光线不均、枝干、叶片对果实的遮挡、以及果实之间的相互遮挡造成的阴影等对果实识别的影响,实现了对成熟红色番茄的识别,相对于传统的双目相机,降低了番茄识别的成本。Through this system, based on the depth camera and the original RGB image, the spatial positioning of the tomato is realized, which lays the foundation for the fast and accurate picking of tomato fruits, and eliminates uneven light, branches and leaves, and the blocking of the fruit. The impact of shadows caused by mutual occlusion on fruit recognition enables the recognition of ripe red tomatoes, which reduces the cost of tomato recognition compared to traditional binocular cameras.

图7示例了一种番茄识别定位设备的结构示意图,如图7所示,该服务器可以包括:处理器(processor)710、存储器(memory)730和总线740,其中,处理器710,存储器730通过总线740完成相互间的通信。处理器710可以调用存储器730中的逻辑指令,以执行如下方法:获取包含番茄的待检测图像,将所述待检测图像转化为HSV图像,在对所述HSV图像进行掩模处理后,将所述HSV图像整合为RGB图像并进行掩蔽处理,获得去除背景后的番茄图像;将所述去除背景后的红色番茄图像,通过分水岭算法进行切割,获得图像分割后的番茄图像;对所述图像分割后的番茄图像进行边缘检测,确定番茄在所述图像分割后的番茄图像中的坐标位置;根据所述番茄在所述图像分割后的番茄图像中的坐标位置,获取所述番茄的点云数据,根据所述番茄的点云数据,确定所述番茄的实际位置。FIG. 7 illustrates a schematic structural diagram of a tomato identification and positioning device. As shown in FIG. 7 , the server may include: a processor (processor) 710, a memory (memory) 730 and a bus 740, wherein the processor 710 and the memory 730 pass through The bus 740 performs communication with each other. The processor 710 can call the logic instructions in the memory 730 to perform the following method: acquiring the to-be-detected image containing tomato, converting the to-be-detected image into an HSV image, and after performing mask processing on the HSV image, converting the detected image into an HSV image. The HSV image is integrated into an RGB image and masked to obtain a background-removed tomato image; the background-removed red tomato image is cut through a watershed algorithm to obtain a tomato image after image segmentation; Perform edge detection on the tomato image after the image segmentation to determine the coordinate position of the tomato in the tomato image after the image segmentation; obtain the point cloud data of the tomato according to the coordinate position of the tomato in the tomato image after the image segmentation , and determine the actual position of the tomato according to the point cloud data of the tomato.

本实施例还提供一种计算机程序产品,所述计算机程序产品包括存储在非暂态计算机可读存储介质上的计算机程序,所述计算机程序包括程序指令,当所述程序指令被计算机执行时,计算机能够执行上述各方法实施例所提供的方法,例如包括:获取包含番茄的待检测图像,将所述待检测图像转化为HSV图像,在对所述HSV图像进行掩模处理后,将所述HSV图像整合为RGB图像并进行掩蔽处理,获得去除背景后的番茄图像;将所述去除背景后的红色番茄图像,通过分水岭算法进行切割,获得图像分割后的番茄图像;对所述图像分割后的番茄图像进行边缘检测,确定番茄在所述图像分割后的番茄图像中的坐标位置;根据所述番茄在所述图像分割后的番茄图像中的坐标位置,获取所述番茄的点云数据,根据所述番茄的点云数据,确定所述番茄的实际位置。This embodiment also provides a computer program product, where the computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program includes program instructions, and when the program instructions are executed by a computer, The computer can execute the methods provided by the above method embodiments, for example, including: acquiring an image to be detected including tomatoes, converting the to-be-detected image into an HSV image, and after performing mask processing on the HSV image, converting the image to be detected The HSV image is integrated into an RGB image and masked to obtain a tomato image after background removal; the red tomato image after background removal is cut through a watershed algorithm to obtain a tomato image after image segmentation; Perform edge detection on the tomato image of the tomato image to determine the coordinate position of the tomato in the tomato image after the image segmentation; obtain the point cloud data of the tomato according to the coordinate position of the tomato in the tomato image after the image segmentation, According to the point cloud data of the tomato, the actual position of the tomato is determined.

本实施例提供一种非暂态计算机可读存储介质,所述非暂态计算机可读存储介质存储计算机指令,所述计算机指令使所述计算机执行上述各方法实施例所提供的方法,例如包括:获取包含番茄的待检测图像,将所述待检测图像转化为HSV图像,在对所述HSV图像进行掩模处理后,将所述HSV图像整合为RGB图像并进行掩蔽处理,获得去除背景后的番茄图像;将所述去除背景后的红色番茄图像,通过分水岭算法进行切割,获得图像分割后的番茄图像;对所述图像分割后的番茄图像进行边缘检测,确定番茄在所述图像分割后的番茄图像中的坐标位置;根据所述番茄在所述图像分割后的番茄图像中的坐标位置,获取所述番茄的点云数据,根据所述番茄的点云数据,确定所述番茄的实际位置。This embodiment provides a non-transitory computer-readable storage medium, where the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions cause the computer to execute the methods provided by the foregoing method embodiments, for example, including : Obtain an image to be detected containing tomatoes, convert the to-be-detected image into an HSV image, and after masking the HSV image, integrate the HSV image into an RGB image and perform masking processing to obtain a background-removed image. The red tomato image after the background removal is cut through the watershed algorithm to obtain the tomato image after image segmentation; the edge detection is performed on the tomato image after the image segmentation, and it is determined that the tomato image is after the image segmentation. According to the coordinate position of the tomato in the tomato image after the image segmentation, the point cloud data of the tomato is obtained, and the actual point cloud data of the tomato is determined according to the point cloud data of the tomato. Location.

以上所描述的装置实施例仅仅是示意性的,其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是或者也可以不是物理单元,即可以位于一个地方,或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性的劳动的情况下,即可以理解并实施。The device embodiments described above are only illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in One place, or it can be distributed over multiple network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.

通过以上的实施方式的描述,本领域的技术人员可以清楚地了解到各实施方式可借助软件加必需的通用硬件平台的方式来实现,当然也可以通过硬件。基于这样的理解,上述技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来,该计算机软件产品可以存储在计算机可读存储介质中,如ROM/RAM、磁碟、光盘等,包括若干指令用以使得一台计算机设备(可以是个人计算机,服务器,或者网络设备等)执行各个实施例或者实施例的某些部分所述的方法。From the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and certainly can also be implemented by hardware. Based on this understanding, the above-mentioned technical solutions can be embodied in the form of software products in essence or the parts that make contributions to the prior art, and the computer software products can be stored in computer-readable storage media, such as ROM/RAM, magnetic A disc, an optical disc, etc., includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform the methods described in various embodiments or some parts of the embodiments.

最后应说明的是:以上实施例仅用以说明本发明的技术方案,而非对其限制;尽管参照前述实施例对本发明进行了详细的说明,本领域的普通技术人员应当理解:其依然可以对前述各实施例所记载的技术方案进行修改,或者对其中部分技术特征进行等同替换;而这些修改或者替换,并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围。Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that it can still be The technical solutions described in the foregoing embodiments are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (4)

1. A tomato identification and positioning method is characterized by comprising the following steps:
acquiring an image to be detected containing tomatoes, converting the image to be detected into an HSV image, after masking the HSV image, integrating the HSV image into an RGB image and masking the RGB image to obtain a red tomato image with a background removed;
cutting the red tomato image with the background removed through a watershed algorithm to obtain an image-segmented tomato image, and acquiring the number of tomatoes in the image-segmented tomato image;
performing edge detection on the tomato image after the image segmentation, and determining the coordinate position of the red tomato in the tomato image after the image segmentation, specifically comprising: taking the number of the tomatoes as circle detection input parameters of Hough transform, and carrying out edge detection on the segmented tomato image to obtain foreground points of the segmented tomato image;
converting the segmented tomato image into an a-b coordinate system from an x-y coordinate system, and obtaining coordinates of a point of a circle center corresponding to the tomato in the segmented tomato image and the radius of the tomato according to the coordinates of the foreground point;
obtaining the coordinate position of the tomato in the tomato image after the image segmentation according to the coordinate of the point of the circle center and the radius of the tomato;
acquiring point cloud data of the red tomatoes according to the coordinate positions of the red tomatoes in the tomato images after the image segmentation, determining the actual positions of the red tomatoes according to the point cloud data of the red tomatoes,
the method comprises the steps of obtaining an image to be detected containing tomatoes, converting the image to be detected into an HSV image, integrating the HSV image into an RGB image after masking the HSV image, and masking the RGB image to obtain a red tomato image with a background removed, and specifically comprises the following steps:
converting the image to be detected into an HSV image, and separating an H channel histogram, an S channel histogram and a V channel histogram from the HSV image;
respectively carrying out mask processing on the H channel histogram, the S channel histogram and the V channel histogram on a preset threshold value, and integrating the H channel histogram, the S channel histogram and the V channel histogram which are subjected to the mask processing into an RGB image;
carrying out R, G, B masking treatment after the RGB image is closed and opened by a morphological technology, and further integrating to obtain a red tomato image with the background removed;
the method comprises the steps of obtaining point cloud data of the red tomatoes according to coordinate positions of the red tomatoes in the tomato images after image segmentation, and determining actual positions of the red tomatoes according to the point cloud data of the red tomatoes, and specifically comprises the following steps:
acquiring a depth image corresponding to the image to be detected through a depth camera;
obtaining point cloud data of the red tomatoes according to the coordinate positions of the red tomatoes in the tomato images after the image segmentation and the depth images;
determining the actual position of the red tomato through the point cloud data of the red tomato;
the background-removed red tomato image is cut by a watershed algorithm to obtain an image-segmented tomato image, and the method specifically comprises the following steps:
converting the red tomato image after the background is removed into a gray image, and calculating the gradient amplitude of the gray image according to a Sobel operator to obtain a gradient image of the gray image;
cutting the gradient image through a watershed algorithm to obtain an image of the tomato after image segmentation;
the step of cutting the gradient image through the watershed algorithm to obtain the tomato image after image segmentation further comprises:
and performing morphological opening and closing reconstruction operation and watershed calculation on the tomato image after the image segmentation to obtain a tomato image calculated by a secondary watershed algorithm, wherein the tomato image is used as a new tomato image after the image segmentation.
2. A tomato identification and positioning system, comprising:
the fruit extraction module is used for acquiring an image to be detected containing tomatoes, converting the image to be detected into an HSV image, after masking the HSV image, integrating the HSV image into an RGB image and masking the RGB image to obtain a red tomato image with a background removed;
the fruit separation module is used for cutting the red tomato image with the background removed through a watershed algorithm to obtain an image-segmented tomato image and acquiring the number of tomatoes in the image-segmented tomato image;
the coordinate determination module is configured to perform edge detection on the tomato image after the image segmentation, and determine a coordinate position of a red tomato in the tomato image after the image segmentation, and specifically includes: taking the number of the tomatoes as circle detection input parameters of Hough transform, and carrying out edge detection on the segmented tomato image to obtain foreground points of the segmented tomato image;
converting the segmented tomato image into an a-b coordinate system from an x-y coordinate system, and obtaining coordinates of a point of a circle center corresponding to the tomato in the segmented tomato image and the radius of the tomato according to the coordinates of the foreground point;
obtaining the coordinate position of the tomato in the tomato image after the image segmentation according to the coordinate of the point of the circle center and the radius of the tomato;
a positioning module for obtaining the point cloud data of the red tomato according to the coordinate position of the red tomato in the tomato image after the image segmentation, determining the actual position of the red tomato according to the point cloud data of the red tomato,
wherein, the fruit extraction module specifically comprises:
converting the image to be detected into an HSV image, and separating an H channel histogram, an S channel histogram and a V channel histogram from the HSV image;
respectively carrying out mask processing on the H channel histogram, the S channel histogram and the V channel histogram on a preset threshold value, and integrating the H channel histogram, the S channel histogram and the V channel histogram which are subjected to the mask processing into an RGB image;
carrying out R, G, B masking treatment after the RGB image is closed and opened by a morphological technology, and further integrating to obtain a red tomato image with the background removed;
the method comprises the steps of obtaining point cloud data of the red tomatoes according to coordinate positions of the red tomatoes in the tomato images after image segmentation, and determining actual positions of the red tomatoes according to the point cloud data of the red tomatoes, and specifically comprises the following steps:
acquiring a depth image corresponding to the image to be detected through a depth camera;
obtaining point cloud data of the red tomatoes according to the coordinate positions of the red tomatoes in the tomato images after the image segmentation and the depth images;
determining the actual position of the red tomato through the point cloud data of the red tomato;
the background-removed red tomato image is cut by a watershed algorithm to obtain an image-segmented tomato image, and the method specifically comprises the following steps:
converting the red tomato image after the background is removed into a gray image, and calculating the gradient amplitude of the gray image according to a Sobel operator to obtain a gradient image of the gray image;
cutting the gradient image through a watershed algorithm to obtain an image of the tomato after image segmentation; the step of cutting the gradient image through the watershed algorithm to obtain the tomato image after image segmentation further comprises:
and performing morphological opening and closing reconstruction operation and watershed calculation on the tomato image after the image segmentation to obtain a tomato image calculated by a secondary watershed algorithm, wherein the tomato image is used as a new tomato image after the image segmentation.
3. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method as claimed in claim 1.
4. Tomato identification and positioning equipment is characterized by comprising:
at least one processor; and at least one memory coupled to the processor, wherein:
the memory stores program instructions executable by the processor, the processor invoking the program instructions to perform the method of claim 1.
CN201810608651.7A 2018-06-13 2018-06-13 A kind of tomato identification and positioning method and system Active CN110599507B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810608651.7A CN110599507B (en) 2018-06-13 2018-06-13 A kind of tomato identification and positioning method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810608651.7A CN110599507B (en) 2018-06-13 2018-06-13 A kind of tomato identification and positioning method and system

Publications (2)

Publication Number Publication Date
CN110599507A CN110599507A (en) 2019-12-20
CN110599507B true CN110599507B (en) 2022-04-22

Family

ID=68849635

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810608651.7A Active CN110599507B (en) 2018-06-13 2018-06-13 A kind of tomato identification and positioning method and system

Country Status (1)

Country Link
CN (1) CN110599507B (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112021016A (en) * 2020-09-07 2020-12-04 江苏农林职业技术学院 Flower and fruit thinning system for strawberries
CN112132153B (en) * 2020-09-22 2022-09-09 山东大学 Tomato fruit identification method and system based on clustering and morphological processing
CN112529922A (en) * 2020-11-18 2021-03-19 南京农业大学 Method for accurately separating RGB image backgrounds of multicolor blades in open environment
CN112580646A (en) * 2020-12-08 2021-03-30 北京农业智能装备技术研究中心 Tomato fruit maturity dividing method and picking robot
CN114260895B (en) * 2021-12-22 2023-08-22 江苏大学 A method and system for determining the direction of grabbing and avoiding obstacles by the mechanical arm of a picking robot
CN114581661B (en) * 2022-01-25 2024-12-06 山东锋士信息技术有限公司 Cotton boll detection method and system in complex natural scenes
CN116778474A (en) * 2023-06-02 2023-09-19 河南农业大学 Intelligent phenotype analyzer for tomato fruits

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101106716A (en) * 2007-08-21 2008-01-16 北京大学软件与微电子学院 A shed image division processing method
CN102968782A (en) * 2012-09-12 2013-03-13 苏州大学 Automatic digging method for remarkable objects of color images
CN104700404A (en) * 2015-03-02 2015-06-10 中国农业大学 Fruit location identification method
CN106067181A (en) * 2016-05-30 2016-11-02 湖南生物机电职业技术学院 Citrusfruit color multi-fractal describes and stage division
CN107392963A (en) * 2017-06-28 2017-11-24 北京航空航天大学 A kind of imitative hawkeye moving target localization method for soft autonomous air refuelling

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101106716A (en) * 2007-08-21 2008-01-16 北京大学软件与微电子学院 A shed image division processing method
CN102968782A (en) * 2012-09-12 2013-03-13 苏州大学 Automatic digging method for remarkable objects of color images
CN104700404A (en) * 2015-03-02 2015-06-10 中国农业大学 Fruit location identification method
CN106067181A (en) * 2016-05-30 2016-11-02 湖南生物机电职业技术学院 Citrusfruit color multi-fractal describes and stage division
CN107392963A (en) * 2017-06-28 2017-11-24 北京航空航天大学 A kind of imitative hawkeye moving target localization method for soft autonomous air refuelling

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
图像算法:分水岭分割程序实现;第一江山临;《百度经验》;20171228;第1-8页 *
温室绿熟番茄机器视觉检测方法;李寒等;《农业工程学报》;20170228;第33卷;第328-334页 *

Also Published As

Publication number Publication date
CN110599507A (en) 2019-12-20

Similar Documents

Publication Publication Date Title
CN110599507B (en) A kind of tomato identification and positioning method and system
Luo et al. A vision methodology for harvesting robot to detect cutting points on peduncles of double overlapping grape clusters in a vineyard
Wu et al. Using color and 3D geometry features to segment fruit point cloud and improve fruit recognition accuracy
Malik et al. Mature tomato fruit detection algorithm based on improved HSV and watershed algorithm
Zhao et al. On-tree fruit recognition using texture properties and color data
Bakar et al. Rice leaf blast disease detection using multi-level colour image thresholding
Raut et al. Plant disease detection in image processing using MATLAB
Jidong et al. Recognition of apple fruit in natural environment
Aquino et al. Grapevine flower estimation by applying artificial vision techniques on images with uncontrolled scene and multi-model analysis
Liu et al. Detection of citrus fruit and tree trunks in natural environments using a multi-elliptical boundary model
CN112990103B (en) String mining secondary positioning method based on machine vision
Li et al. An improved binocular localization method for apple based on fruit detection using deep learning
WO2014031086A1 (en) Methods and systems for automatic location of optic structures in an image of an eye, and for automatic retina cup-to-disc ratio computation
CN102760228B (en) Specimen-based automatic lepidoptera insect species identification method
Gutzeit et al. Automatic segmentation of wood logs by combining detection and segmentation
Devi et al. Image processing system for automatic segmentation and yield prediction of fruits using open CV
Galsgaard et al. Circular hough transform and local circularity measure for weight estimation of a graph-cut based wood stack measurement
Revathi et al. Homogenous segmentation based edge detection techniques for proficient identification of the cotton leaf spot diseases
Pandit et al. Literature review on object counting using image processing techniques
KR20210126485A (en) Matching method, apparatus, electronic device, computer readable storage medium, and computer program
CN105023272A (en) Crop leaf insect pest detection method and system
Raut et al. Review on leaf disease detection using image processing techniques
KR20210028966A (en) Method and apparatus for disease classification of plant leafs
Changhui et al. Overlapped fruit recognition for citrus harvesting robot in natural scenes
Kshirsagar et al. Plant disease detection in image processing using MATLAB

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant