WO2024119824A1 - 基于生物资产阴影区盘点计数的图像识别方法及系统 - Google Patents

基于生物资产阴影区盘点计数的图像识别方法及系统 Download PDF

Info

Publication number
WO2024119824A1
WO2024119824A1 PCT/CN2023/106856 CN2023106856W WO2024119824A1 WO 2024119824 A1 WO2024119824 A1 WO 2024119824A1 CN 2023106856 W CN2023106856 W CN 2023106856W WO 2024119824 A1 WO2024119824 A1 WO 2024119824A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
learning
shadow
module
analyzed
Prior art date
Application number
PCT/CN2023/106856
Other languages
English (en)
French (fr)
Inventor
钱超
李烨星
郭濮瑞
Original Assignee
上海万向区块链股份公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海万向区块链股份公司 filed Critical 上海万向区块链股份公司
Publication of WO2024119824A1 publication Critical patent/WO2024119824A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G06V10/12—Details of acquisition arrangements; Constructional details thereof
    • G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
    • G06V10/141—Control of illumination
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/98—Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
    • Y02A40/70—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in livestock or poultry

Definitions

  • the present invention relates to the field of artificial intelligence technology, and in particular, to an image recognition method and system based on inventory counting of biological asset shadow areas, and in particular, preferably to an image recognition method and edge-side device process based on inventory counting of biological asset shadow areas.
  • the Chinese invention patent document with publication number CN112330107A discloses a biological asset management system and method for aquaculture based on blockchain.
  • the system includes: blockchain platform, procurement module, breeding module, transportation module, slaughtering module, sales module, insurance module, and financing module; the method includes: purchasing breeding objects and raw materials, and saving the raw material information on the blockchain platform; breeding the breeding objects, and saving the breeding data and market data on the blockchain platform; transporting the breeding objects, and saving the vehicle information and transportation information on the blockchain platform; slaughtering the breeding objects after quarantine registration, and saving the quarantine data and slaughtering data on the blockchain platform; selling the breeding objects, and querying the full cycle information through the label on the package; providing insurance services for breeders; and providing financing services for breeders, slaughtering enterprises, and sales companies.
  • an embodiment of the present invention aims to provide an image recognition method and system based on inventory counting of shadow areas of biological assets.
  • an image recognition method based on inventory counting of biological asset shadow areas is provided.
  • the steps include:
  • Image segmentation learning steps collect images to be analyzed of the farming scene, perform segmentation learning on the images to be analyzed, and determine the context of the images to be analyzed;
  • Image training and learning step performing image training and learning from the image to be analyzed with the determined context, and determining whether there is a shadow, and obtaining a first result if there is no shadow;
  • Edge device supplementary training step If it is determined that there is a shadow in the image training learning step, fill light photography is performed to obtain the image to be analyzed after the supplementary photography, and image segmentation learning and training learning are performed to obtain a second result.
  • the image segmentation learning step includes the following steps:
  • Step S1 Deploy cameras for farming scenarios in large-scale livestock farming
  • Step S2 Collect the image to be analyzed through the camera and transmit the image to be analyzed to the processing server;
  • Step S3 The processing server uses a multi-task convolutional neural network to segment the image to be analyzed, and uses the color spots of the segmented image as input to the convolutional neural network, which marks the pixels;
  • Step S4 the pixel-marked image is divided into a*b rectangular blocks, the RGB values of all pixels in the n corner rectangular blocks are extracted, the number of pixels with the same RGB value among the RGB values of all pixels in the n corner rectangular blocks is obtained by counting, and the RGB value corresponding to the predetermined number is determined as a threshold, where a represents the long side and b represents the wide side;
  • Step S5 Calculate the RGB mean of the remaining a*b-n rectangular blocks, retain the RGB means that are greater than or equal to the threshold, and obtain C rectangular blocks corresponding to the C means.
  • the RGB values of the pixels of the C rectangular blocks are formed into input data, which is passed to the convolution layer to classify each pixel and determine the context of the image to be analyzed.
  • the image training learning step includes the following steps:
  • Step S6 Each pixel data is converted from two dimensions to one dimension, and the covariance matrix of the converted one-dimensional channel is obtained, and the eigenvector p and eigenvalue ⁇ of the covariance matrix are obtained, and the conversion is performed according to the formula to complete the color random jitter judgment;
  • Step S7 color gamut adjustment, generating a shadow judgment model after color judgment
  • Step S8 If the shadow judgment model determines that there is no shadow, the sliding window determines whether there is a living creature, and the network randomly captures the border from the original image, performs positive and negative sample learning, enters the output network, and obtains the first result.
  • the edge device supplementary training step includes the following steps:
  • Step S9 providing an edge device
  • Step S10 Perform semantic analysis through the processor in the edge device. For step S7, if the shadow judgment model determines that there is a shadow, fill light is used to take pictures, and then continue with steps S2 to S6.
  • the sliding window determines whether there is a living creature.
  • the network randomly captures the border from the picture after fill light, performs positive and negative sample learning, enters the output network, and obtains the second result.
  • the image recognition method based on inventory counting of biological asset shadow area according to claim 4 is characterized in that In other words, the edge device supplementary training step further includes the following steps:
  • Step S11 Compare the first result and the second result, perform convolutional network learning, and then calculate the average value within T time;
  • Step S12 Determine the sum of the number of organisms in a*b based on the average value.
  • an image recognition system based on inventory counting of biological asset shadow areas comprising the following modules:
  • Image segmentation learning module collects images to be analyzed in farming scenes, performs segmentation learning on the images to be analyzed, and determines the context of the images to be analyzed;
  • Image training and learning module performs image training and learning from the image to be analyzed with the determined context, and determines whether there is a shadow. If there is no shadow, a first result is obtained;
  • Edge device supplementary training module If it is determined that there is a shadow in the image training learning module, fill light photography is performed to obtain the image to be analyzed after the supplementary photography, and image segmentation learning and training learning are performed to obtain the second result.
  • the image segmentation learning module includes the following modules:
  • Module M1 Deploy cameras for farming scenarios in large-scale livestock farming
  • Module M2 collects the image to be analyzed through the camera and transmits the image to be analyzed to the processing server;
  • Module M3 The processing server uses a multi-task convolutional neural network to segment the image to be analyzed, and uses the color spots of the segmented image as input to the convolutional neural network, which marks the pixels;
  • Module M4 the pixel-marked image is divided into a*b rectangular blocks, the RGB values of all pixels in the n corner rectangular blocks are extracted, the number of pixels with the same RGB value among the RGB values of all pixels in the n corner rectangular blocks is obtained by counting, and the RGB value corresponding to the predetermined number is determined as the threshold, where a represents the long side and b represents the wide side;
  • Module M5 Calculate the RGB mean of the remaining a*b-n rectangular blocks, retain the RGB means that are greater than or equal to the threshold, and obtain C rectangular blocks corresponding to the C means.
  • the RGB values of the pixels of the C rectangular blocks are formed into input data, which is passed to the convolution layer to classify each pixel and determine the context of the image to be analyzed.
  • the image training learning module includes the following modules:
  • Each pixel data is converted from two dimensions to one dimension, the covariance matrix of the converted one-dimensional channel is calculated, the eigenvector p and eigenvalue ⁇ of the covariance matrix are calculated, and the conversion is performed according to the formula to complete the color random jitter judgment;
  • Module M7 Color gamut adjustment, generating a shadow judgment model after color judgment
  • Module M8 If the shadow judgment model determines that there is no shadow, the sliding window determines whether there is a living creature. The network randomly extracts the border from the original image, performs positive and negative sample learning, enters the output network, and obtains the first result.
  • the edge device supplementary training module includes the following modules:
  • Module M9 provides edge devices
  • Module M10 Perform semantic analysis through the processor in the edge device. For module M7, if the shadow judgment model determines that there is a shadow, fill light is used to take pictures, and then continue with modules M2 to M6. The sliding window determines whether there is a living creature. The network randomly captures the border from the picture after fill light, performs positive and negative sample learning, enters the output network, and obtains the second result.
  • the edge device supplementary training module also includes the following modules:
  • Module M11 Compare the first result and the second result, perform convolutional network learning, and then calculate the average value within T time;
  • Module M12 Determine the sum of the number of organisms in a*b based on the average value.
  • Figure 1 is a flowchart of shadow-based beef cattle counting image learning
  • Figure 2 is a diagram of the Resnet (residual network) network structure.
  • the embodiment of the present invention discloses an image recognition method based on inventory counting of biological asset shadow areas and an edge device process, as shown in FIG1 and FIG2 , including the following steps:
  • the image segmentation learning process is the image segmentation learning step: collect the images to be analyzed of the farming scene, perform segmentation learning on the images to be analyzed, and determine the context of the images to be analyzed.
  • the specific steps include the following:
  • Step 1 Deploy cameras for farming scenarios in large-scale livestock farming.
  • Step 2 Collect the image to be analyzed through the camera and transmit the image to the processing server.
  • Step 3 The processing server uses MTCNN (Multi-task convolutional neural network) to segment the image and input the image patch (spot, patch) to the convolutional neural network Refine-net (a multi-path reinforcement network), which marks the pixels.
  • MTCNN Multi-task convolutional neural network
  • CNN convolutional neural network
  • Step 4 Divide the image into a*b rectangular blocks, and extract the RGB values of all pixels in the four corner rectangular blocks.
  • the number of pixels with the same RGB value among the RGB values of all pixels in the four corner rectangular blocks is obtained by counting, and the RGB value corresponding to the maximum number is determined as the threshold.
  • Step 5 Calculate the RGB mean of the remaining a*b-4 rectangular blocks (obtained by comparing with the threshold), retain all means greater than or equal to the first threshold, and obtain C rectangular blocks corresponding to the C means.
  • the RGB values of the pixels of the C rectangular blocks form the input data Refine-net, which is passed to the convolutional layer to classify each pixel (pixels are classified through softmax) to determine the context of the image, including the location of the target.
  • the image training learning process is the image training learning step: image training learning is performed from the image to be analyzed with a determined context, and whether there is a shadow is determined, and if there is no shadow, the first result is obtained. Specifically, the following steps are included:
  • Step 6 The pixel data of each channel is first converted from two dimensions to one dimension (for example, 256*256*3 is converted to 65536*3), and then the covariance matrix (3*3) of the three channels of the image (65536*3) is calculated, and then the eigenvector p and eigenvalue ⁇ of the covariance matrix are calculated, and finally the conversion is performed according to the formula to complete the color random jitter judgment.
  • Step 7 Color gamut adjustment, after the above color judgment, a shadow judgment model is generated.
  • Step 8 Resnet pre-processing is color gamut adjustment, and then sliding window is used to determine whether there are beef cattle inside.
  • the network randomly extracts borders from the above data, performs positive and negative sample learning, and enters the Output Network.
  • Edge device supplementary training step If it is determined that there is a shadow in the image training learning step, fill light photography is performed to obtain the image to be analyzed after the supplementary photography, and image segmentation learning and training learning are performed to obtain the second result.
  • the steps include:
  • Step 9 Provide an edge device, wherein the terminal device includes: a processor, a memory, a communication unit, a camera, a fill light, and a bus (data acquisition protocol of the Internet of Things).
  • the terminal device includes: a processor, a memory, a communication unit, a camera, a fill light, and a bus (data acquisition protocol of the Internet of Things).
  • Step 10 The edge device processor performs a first semantic analysis, performs fill light photography after determining the shadow in step 7, and continues with steps 2 to 6.
  • Step 11 Compare the results of step 8 with those of step 10, and perform more convolutional network learning (subsequently, the mean (average value) within time T is used to enhance robustness).
  • Step S12 Sum the number of organisms in a*b.
  • the RPN layer extracts areas that may be targets; in Figure 2, conv means convolution and pool means pooling.
  • the present invention also provides an image recognition system based on inventory counting of biological disability shadow areas.
  • the image recognition system based on inventory counting of biological disability shadow areas can be realized by executing the process steps of the image recognition method based on inventory counting of biological disability shadow areas, that is, those skilled in the art can understand the image recognition method based on inventory counting of biological disability shadow areas as a preferred implementation of the image recognition system based on inventory counting of biological disability shadow areas.
  • the embodiment of the present invention also discloses an image recognition system based on biological disability shadow area inventory counting, comprising the following modules:
  • Image segmentation learning module collects images to be analyzed in farming scenes, performs segmentation learning on the images to be analyzed, and determines the context of the images to be analyzed.
  • Image training and learning module image training and learning are performed from the image to be analyzed with a determined context, and whether there is a shadow is determined. If there is no shadow, the first result is obtained.
  • Edge device supplementary training module If it is determined that there is a shadow in the image training learning module, fill light photography is performed to obtain the image to be analyzed after the supplementary photography, and image segmentation learning and training learning are performed to obtain the second result.
  • the image segmentation learning module includes the following modules:
  • Module M1 Deploy cameras for farming scenarios in large-scale livestock farming
  • Module M2 collects the image to be analyzed through the camera and transmits the image to be analyzed to the processing server;
  • Module M3 The processing server uses a multi-task convolutional neural network to segment the image to be analyzed, and uses the color spots of the segmented image as input to the convolutional neural network, which marks the pixels;
  • Module M4 the pixel-marked image is divided into a*b rectangular blocks, the RGB values of all pixels in the n corner rectangular blocks are extracted, the number of pixels with the same RGB value among the RGB values of all pixels in the n corner rectangular blocks is obtained by counting, and the RGB value corresponding to the predetermined number is determined as the threshold, where a represents the long side and b represents the wide side;
  • Module M5 Calculate the RGB mean of the remaining a*b-n rectangular blocks, retain the RGB means that are greater than or equal to the threshold, and obtain C rectangular blocks corresponding to the C means.
  • the RGB values of the pixels of the C rectangular blocks are formed into input data, which is passed to the convolution layer to classify each pixel and determine the context of the image to be analyzed.
  • the image training learning module includes the following modules:
  • Each pixel data is converted from two dimensions to one dimension, the covariance matrix of the converted one-dimensional channel is calculated, the eigenvector p and eigenvalue ⁇ of the covariance matrix are calculated, and the conversion is performed according to the formula to complete the color random jitter judgment;
  • Module M7 Color gamut adjustment, generating a shadow judgment model after color judgment
  • Module M8 If the shadow judgment model determines that there is no shadow, the sliding window determines whether there is a living creature. The network randomly extracts the border from the original image, performs positive and negative sample learning, enters the output network, and obtains the first result.
  • the edge device supplementary training module includes the following modules:
  • Module M9 provides edge devices
  • Module M10 Perform semantic analysis through the processor in the edge device. For module M7, if the shadow judgment model determines that there is a shadow, fill light is used to take pictures, and then continue with modules M2 to M6. The sliding window determines whether there is a living creature. The network randomly captures the border from the picture after fill light, performs positive and negative sample learning, enters the output network, and obtains the second result.
  • Module M11 Compare the first result and the second result, perform convolutional network learning, and then calculate the average value within T time;
  • Module M12 Determine the sum of the number of organisms in a*b based on the average value.
  • the present application relates to the field of artificial intelligence technology, and in particular to an image recognition method and device for on-site AI in the vertical field of biological assets; a deep learning method and an edge-side AI processor device and process for effectively counting biological assets with macroscopic signs, mainly represented by beef cattle, in complex scenarios such as shadow areas; in the method, the principle of light compensation is used for the above-mentioned scenarios, and the pixels in the A*B large grid area are averaged for the edge-side processor to achieve the accuracy of the counting operation; in terms of the process, a general processor is provided to perform light compensation on the RGB values of the pixels and compare the thresholds.
  • the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.
  • the present invention has the following beneficial effects:
  • the present invention adopts an image AI (Artificial Intelligence) recognition algorithm to solve the counting interference caused by scenes such as shadows, based on convolutional network neural learning of image segmentation and edge-side terminal assistance, to solve the counting deviation problem of large biological assets in complex scenes (shadows).
  • image AI Artificial Intelligence

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Image Analysis (AREA)

Abstract

本发明提供了一种基于生物资产阴影区盘点计数的图像识别方法及系统，包括如下步骤：图像分割学习步骤：采集养殖场景的待分析图像，并对待分析图像进行分割学习，确定待分析图像的上下文；图像训练学习步骤：从确定上下文的待分析图像中进行图像训练学习，并判断是否有阴影，若无阴影则得到第一结果；边缘装置补充训练步骤：在图像训练学习步骤中若判断有阴影，则进行补光拍照，得到补充拍照后的待分析图片，进行图像分割学习和训练学习，得到第二结果。本发明通过采用图像AI识别算法，就阴影等场景产生的计数干扰，基于图像分割的卷积网络神经学习和边缘侧终端辅助，解决大型生物资产在复杂场景(阴影)等计数盘点的偏差问题。

Description

基于生物资产阴影区盘点计数的图像识别方法及系统 技术领域
本发明涉及人工智能技术领域，具体地，涉及一种基于生物资产阴影区盘点计数的图像识别方法及系统，尤其是，优选的涉及一种基于生物资产阴影区盘点计数的图像识别方法和边缘侧装置流程。
背景技术
随着对农业金融发展的促进和扶持，物流、冷链、金融等领域对于农业和畜牧业的支持力度也在加大。经济价值比较高的生物比如牛、羊、马等除了消费属性外也会带上金融属性，如何通过技术手段高效可信地把生物资产数字化，成了农业金融发展的一个问题。养殖产业一直以来都面临着“活体资产”价值认定难、抵押难、监管难所带来的融资难问题，这在很大程度上制约了养殖户，养殖企业的规模化发展。
公开号为CN112330107A的中国发明专利文献公开了一种基于区块链的养殖业生物资产管理系统及方法。系统包括：区块链平台、采购模块、养殖模块、运输模块、屠宰模块、销售模块、保险模块、融资模块；方法包括：采购养殖对象以及原料，将原料信息保存在区块链平台上；对养殖对象进行养殖，将养殖数据以及出栏数据保存在区块链平台上；将养殖对象进行运输，并将车辆信息、运输信息保存在区块链平台上；将养殖对象经过检疫登记后进行屠宰，并将检疫数据、屠宰数据保存在区块链平台上；将养殖对象进行销售，通过包装上的标签，查询全周期信息；为养殖户提供保险业务；为养殖户、屠宰企业、销售公司提供融资业务。
针对上述中的相关技术，发明人认为阴影等场景会产生计数干扰，在大型生物资产在复杂场景(阴影)等计数盘点会产生偏差。
发明内容
针对现有技术中的缺陷，本发明的实施例的目的是提供一种基于生物资产阴影区盘点计数的图像识别方法及系统。
根据本发明的实施例提供的一种基于生物资产阴影区盘点计数的图像识别方法， 包括如下步骤：
图像分割学习步骤：采集养殖场景的待分析图像，并对待分析图像进行分割学习，确定待分析图像的上下文；
图像训练学习步骤：从确定上下文的待分析图像中进行图像训练学习，并判断是否有阴影，若无阴影则得到第一结果；
边缘装置补充训练步骤：在图像训练学习步骤中若判断有阴影，则进行补光拍照，得到补充拍照后的待分析图片，进行图像分割学习和训练学习，得到第二结果。
进一步地，所述图像分割学习步骤包括如下步骤：
步骤S1：在畜牧规模化养殖部署面向养殖场景的摄像头；
步骤S2：通过摄像头采集待分析图片，将待分析图片传输给处理服务器；
步骤S3：处理服务器将待分析图片进行使用多任务卷积神经网络进行图像分割，将分割后图像的色斑作为输入，输入给卷积神经网络，卷积神经网络对像素进行标记；
步骤S4：像素标记后的图片划分成a*b个矩形块，提取n个边角矩形块中的所有像素点的RGB值，统计得到n个边角矩形块中的所有像素点的RGB值中相同RGB值的像素点的数量，将预定数量对应的RGB值确定为阈值，其中a表示长边，b表示宽边；
步骤S5：计算剩余a*b-n个矩形块的RGB均值，将RGB均值中大于等于阈值的进行保留，得到C个均值对应的C个矩形块，将C个矩形块的像素点的RGB值形成输入数据，传给卷积层对每个像素进行分类，确定待分析图像的上下文。
进一步地，所述图像训练学习步骤包括如下步骤：
步骤S6：每个像素数据由二维转成一维，对转成一维的通道求出协方差矩阵，求出协方差矩阵的特征向量p和特征值λ，按照公式进行转换，完成颜色随机抖动判断；
步骤S7：色域调整，颜色判断后生成阴影判断模型；
步骤S8：若阴影判断模型判断没有阴影，滑窗判断是否有生物，网络从原图片随机截取边框，进行正负样本学习，进入输出网络，得到第一结果。
进一步地，所述边缘装置补充训练步骤包括如下步骤：
步骤S9：提供边缘装置；
步骤S10：通过边缘装置中的处理器进行语义分析，对于步骤S7若阴影判断模型判断有阴影后通过补光灯进行补光拍照，继续步骤S2～步骤S6，滑窗判断是否有生物，网络从补光后的图片随机截取边框，进行正负样本学习，进入输出网络，得到第二结果。
5、根据权利要求4所述的基于生物资产阴影区盘点计数的图像识别方法，其特征 在于，所述边缘装置补充训练步骤还包括如下步骤：
步骤S11：对第一结果和第二结果进行比对，进行卷积网络学习，后续采用T时刻内求平均值；
步骤S12：根据平均值确定a*b内的生物数量求和。
根据本发明提供的一种基于生物资产阴影区盘点计数的图像识别系统，包括如下模块：
图像分割学习模块：采集养殖场景的待分析图像，并对待分析图像进行分割学习，确定待分析图像的上下文；
图像训练学习模块：从确定上下文的待分析图像中进行图像训练学习，并判断是否有阴影，若无阴影则得到第一结果；
边缘装置补充训练模块：在图像训练学习模块中若判断有阴影，则进行补光拍照，得到补充拍照后的待分析图片，进行图像分割学习和训练学习，得到第二结果。
优选的，所述图像分割学习模块包括如下模块：
模块M1：在畜牧规模化养殖部署面向养殖场景的摄像头；
模块M2：通过摄像头采集待分析图片，将待分析图片传输给处理服务器；
模块M3：处理服务器将待分析图片进行使用多任务卷积神经网络进行图像分割，将分割后图像的色斑作为输入，输入给卷积神经网络，卷积神经网络对像素进行标记；
模块M4：像素标记后的图片划分成a*b个矩形块，提取n个边角矩形块中的所有像素点的RGB值，统计得到n个边角矩形块中的所有像素点的RGB值中相同RGB值的像素点的数量，将预定数量对应的RGB值确定为阈值，其中a表示长边，b表示宽边；
模块M5：计算剩余a*b-n个矩形块的RGB均值，将RGB均值中大于等于阈值的进行保留，得到C个均值对应的C个矩形块，将C个矩形块的像素点的RGB值形成输入数据，传给卷积层对每个像素进行分类，确定待分析图像的上下文。
进一步地，所述图像训练学习模块包括如下模块：
模块M6：每个像素数据由二维转成一维，对转成一维的通道求出协方差矩阵，求出协方差矩阵的特征向量p和特征值λ，按照公式进行转换，完成颜色随机抖动判断；
模块M7：色域调整，颜色判断后生成阴影判断模型；
模块M8：若阴影判断模型判断没有阴影，滑窗判断是否有生物，网络从原图片随机截取边框，进行正负样本学习，进入输出网络，得到第一结果。
进一步地，所述边缘装置补充训练模块包括如下模块：
模块M9：提供边缘装置；
模块M10：通过边缘装置中的处理器进行语义分析，对于模块M7若阴影判断模型判断有阴影后通过补光灯进行补光拍照，继续模块M2～模块M6，滑窗判断是否有生物，网络从补光后的图片随机截取边框，进行正负样本学习，进入输出网络，得到第二结果。
进一步地，所述边缘装置补充训练模块还包括如下模块：
模块M11：对第一结果和第二结果进行比对，进行卷积网络学习，后续采用T时刻内求平均值；
模块M12：根据平均值确定a*b内的生物数量求和。
附图说明
通过阅读参照以下附图对非限制性实施例所作的详细描述，本发明的其它特征、目的和优点将会变得更明显：
图1为基于阴影的肉牛计数图像学习流程图；
图2为Resnet(残差网络)网络结构图。
具体实施方式
下面结合具体实施例对本发明进行详细说明。以下实施例将有助于本领域的技术人员进一步理解本发明，但不以任何形式限制本发明。应当指出的是，对本领域的普通技术人员来说，在不脱离本发明构思的前提下，还可以做出若干变化和改进。这些都属于本发明的保护范围。
本发明实施例公开了一种基于生物资产阴影区盘点计数的图像识别方法和边缘侧装置流程，如图1和图2所示，包括如下步骤：
图像分割学习过程即图像分割学习步骤：采集养殖场景的待分析图像，并对待分析图像进行分割学习，确定待分析图像的上下文。具体包括如下步骤：
步骤1：在畜牧规模化养殖部署面向养殖场景的摄像头。
步骤2：通过摄像头采集待分析图片，将图片传输给处理服务器。
步骤3：处理服务器将图片进行使用MTCNN(Multi-task convolutional neural network，多任务卷积神经网络)进行图像分割，将图像的patch(色斑、补丁)作为输入，输入给卷积神经网络Refine-net(一种多路径强化网络)，卷积神经网络对像素进行标记。CNN表示卷积神经网络。
步骤4：图片划分成a*b个矩形块，提取四个边角矩形块中的所有像素点的RGB值， 统计得到四个边角矩形块中的所有像素点的RGB值中相同RGB值的像素点的数量，将数量最大值对应的RGB值确定为阈值。
步骤5：计算剩余a*b-4个矩形块的RGB均值(与阈值比较所得)，将所有均值中大于等于第一阈值的进行保留，得到C个均值对应的C个矩形块，将C个矩形块的像素点的RGB值形成输入数据Refine-net，传给卷积层对每个像素进行分类(通过softmax对像素进行分类)，以确定图像的上下文，包括目标的位置。
图像训练学习过程即图像训练学习步骤：从确定上下文的待分析图像中进行图像训练学习，并判断是否有阴影，若无阴影则得到第一结果。具体包括如下步骤：
步骤6：每个通道的像素数据先由二维转成一维(例如256*256*3，转成65536*3)，再对该图片(65536*3)三个通道求出协方差矩阵(3*3)，再求出协方差矩阵的特征向量p和特征值λ，最后按照公式进行转换，完成颜色随机抖动判断。
步骤7：色域调整，上述颜色判断后生成阴影判断模型。
步骤8：Resnet前处理是色域调整，然后滑窗判断里面有没有肉牛，网络从上述数据中随机截取边框，进行正负样本学习，进入Output Network(输出网络)。
边缘装置补充训练步骤：在图像训练学习步骤中若判断有阴影，则进行补光拍照，得到补充拍照后的待分析图片，进行图像分割学习和训练学习，得到第二结果。具体包括如下步骤：
步骤9：提供一种边缘装置，所述终端装置包括：处理器、存储器、通信单元、摄像头、补光灯和总线(物联网的数据采集协议)。
步骤10：边缘装置处理器进行第一语义的分析，对于步骤7判断阴影后进行补光拍照，继续步骤2～6步骤。
步骤11：步骤8结果和步骤10结果进行比对，进行更多卷积网络学习(后续采用在T时刻内求mean(平均值)来增强鲁棒性)。
步骤S12：a*b内的生物数量求和。
图1中，RPN层提取可能为目标的区域；图2中，conv表示卷积，pool表示池化。
本发明还提供一种基于生物致残阴影区盘点计数的图像识别系统，所述基于生物致残阴影区盘点计数的图像识别系统可以通过执行所述基于生物致残阴影区盘点计数的图像识别方法的流程步骤予以实现，即本领域技术人员可以将所述基于生物致残阴影区盘点计数的图像识别方法理解为所述基于生物致残阴影区盘点计数的图像识别系统的优选实施方式。
本发明实施例还公开了一种基于生物致残阴影区盘点计数的图像识别系统，包括如下模块：
图像分割学习模块：采集养殖场景的待分析图像，并对待分析图像进行分割学习，确定待分析图像的上下文。
图像训练学习模块：从确定上下文的待分析图像中进行图像训练学习，并判断是否有阴影，若无阴影则得到第一结果。
边缘装置补充训练模块：在图像训练学习模块中若判断有阴影，则进行补光拍照，得到补充拍照后的待分析图片，进行图像分割学习和训练学习，得到第二结果。
图像分割学习模块包括如下模块：
模块M1：在畜牧规模化养殖部署面向养殖场景的摄像头；
模块M2：通过摄像头采集待分析图片，将待分析图片传输给处理服务器；
模块M3：处理服务器将待分析图片进行使用多任务卷积神经网络进行图像分割，将分割后图像的色斑作为输入，输入给卷积神经网络，卷积神经网络对像素进行标记；
模块M4：像素标记后的图片划分成a*b个矩形块，提取n个边角矩形块中的所有像素点的RGB值，统计得到n个边角矩形块中的所有像素点的RGB值中相同RGB值的像素点的数量，将预定数量对应的RGB值确定为阈值，其中a表示长边，b表示宽边；
模块M5：计算剩余a*b-n个矩形块的RGB均值，将RGB均值中大于等于阈值的进行保留，得到C个均值对应的C个矩形块，将C个矩形块的像素点的RGB值形成输入数据，传给卷积层对每个像素进行分类，确定待分析图像的上下文。
图像训练学习模块包括如下模块：
模块M6：每个像素数据由二维转成一维，对转成一维的通道求出协方差矩阵，求出协方差矩阵的特征向量p和特征值λ，按照公式进行转换，完成颜色随机抖动判断；
模块M7：色域调整，颜色判断后生成阴影判断模型；
模块M8：若阴影判断模型判断没有阴影，滑窗判断是否有生物，网络从原图片随机截取边框，进行正负样本学习，进入输出网络，得到第一结果。
边缘装置补充训练模块包括如下模块：
模块M9：提供边缘装置；
模块M10：通过边缘装置中的处理器进行语义分析，对于模块M7若阴影判断模型判断有阴影后通过补光灯进行补光拍照，继续模块M2～模块M6，滑窗判断是否有生物，网络从补光后的图片随机截取边框，进行正负样本学习，进入输出网络，得到第二结果。
模块M11：对第一结果和第二结果进行比对，进行卷积网络学习，后续采用T时刻内求平均值；
模块M12：根据平均值确定a*b内的生物数量求和。
本申请涉及人工智能技术领域，尤其涉及一种在生物资产垂直领域进行现场ai的图像识别方法及装置；主要以肉牛为代表的大体征生物资产，在面临阴影区等复合场景下如何有效计数的深度学习方法及边缘侧AI处理器装置和流程；方法中，将对上述场景利用光补偿的原理，针对边缘侧处理器进行A*B大网格区域像素进行均值补光，从而实现计数运算的准确度；流程上，提供一种通用处理器，针对像素点的RGB值进行光补偿并进行阈值的比较。
本领域技术人员知道，除了以纯计算机可读程序代码方式实现本发明提供的系统及其各个装置、模块、单元以外，完全可以通过将方法步骤进行逻辑编程来使得本发明提供的系统及其各个装置、模块、单元以逻辑门、开关、专用集成电路、可编程逻辑控制器以及嵌入式微控制器等的形式来实现相同功能。所以，本发明提供的系统及其各项装置、模块、单元可以被认为是一种硬件部件，而对其内包括的用于实现各种功能的装置、模块、单元也可以视为硬件部件内的结构；也可以将用于实现各种功能的装置、模块、单元视为既可以是实现方法的软件模块又可以是硬件部件内的结构。
与现有技术相比，本发明具有如下的有益效果：
1、本发明通过采用图像AI(Artificial Intelligence，人工智能)识别算法，就阴影等场景产生的计数干扰，基于图像分割的卷积网络神经学习和边缘侧终端辅助，解决大型生物资产在复杂场景(阴影)等计数盘点的偏差问题。
以上对本发明的具体实施例进行了描述。需要理解的是，本发明并不局限于上述特定实施方式，本领域技术人员可以在权利要求的范围内做出各种变化或修改，这并不影响本发明的实质内容。在不冲突的情况下，本申请的实施例和实施例中的特征可以任意相互组合。

Claims (10)

  1. 一种基于生物资产阴影区盘点计数的图像识别方法，其特征在于，包括如下步骤：
    图像分割学习步骤：采集养殖场景的待分析图像，并对待分析图像进行分割学习，确定待分析图像的上下文；
    图像训练学习步骤：从确定上下文的待分析图像中进行图像训练学习，并判断是否有阴影，若无阴影则得到第一结果；
    边缘装置补充训练步骤：在图像训练学习步骤中若判断有阴影，则进行补光拍照，得到补充拍照后的待分析图片，进行图像分割学习和训练学习，得到第二结果。
  2. 根据权利要求1所述的基于生物资产阴影区盘点计数的图像识别方法，其特征在于，所述图像分割学习步骤包括如下步骤：
    步骤S1：在畜牧规模化养殖部署面向养殖场景的摄像头；
    步骤S2：通过摄像头采集待分析图片，将待分析图片传输给处理服务器；
    步骤S3：处理服务器将待分析图片进行使用多任务卷积神经网络进行图像分割，将分割后图像的色斑作为输入，输入给卷积神经网络，卷积神经网络对像素进行标记；
    步骤S4：像素标记后的图片划分成a*b个矩形块，提取n个边角矩形块中的所有像素点的RGB值，统计得到n个边角矩形块中的所有像素点的RGB值中相同RGB值的像素点的数量，将预定数量对应的RGB值确定为阈值，其中a表示长边，b表示宽边；
    步骤S5：计算剩余a*b-n个矩形块的RGB均值，将RGB均值中大于等于阈值的进行保留，得到C个均值对应的C个矩形块，将C个矩形块的像素点的RGB值形成输入数据，传给卷积层对每个像素进行分类，确定待分析图像的上下文。
  3. 根据权利要求2所述的基于生物资产阴影区盘点计数的图像识别方法，其特征在于，所述图像训练学习步骤包括如下步骤：
    步骤S6：每个像素数据由二维转成一维，对转成一维的通道求出协方差矩阵，求出协方差矩阵的特征向量p和特征值λ，按照公式进行转换，完成颜色随机抖动判断；
    步骤S7：色域调整，颜色判断后生成阴影判断模型；
    步骤S8：若阴影判断模型判断没有阴影，滑窗判断是否有生物，网络从原图片随机截取边框，进行正负样本学习，进入输出网络，得到第一结果。
  4. 根据权利要求3所述的基于生物资产阴影区盘点计数的图像识别方法，其特征 在于，所述边缘装置补充训练步骤包括如下步骤：
    步骤S9：提供边缘装置；
    步骤S10：通过边缘装置中的处理器进行语义分析，对于步骤S7若阴影判断模型判断有阴影后通过补光灯进行补光拍照，继续步骤S2～步骤S6，滑窗判断是否有生物，网络从补光后的图片随机截取边框，进行正负样本学习，进入输出网络，得到第二结果。
  5. 根据权利要求4所述的基于生物资产阴影区盘点计数的图像识别方法，其特征在于，所述边缘装置补充训练步骤还包括如下步骤：
    步骤S11：对第一结果和第二结果进行比对，进行卷积网络学习，后续采用T时刻内求平均值；
    步骤S12：根据平均值确定a*b内的生物数量求和。
  6. 一种基于生物资产阴影区盘点计数的图像识别系统，其特征在于，包括如下模块：
    图像分割学习模块：采集养殖场景的待分析图像，并对待分析图像进行分割学习，确定待分析图像的上下文；
    图像训练学习模块：从确定上下文的待分析图像中进行图像训练学习，并判断是否有阴影，若无阴影则得到第一结果；
    边缘装置补充训练模块：在图像训练学习模块中若判断有阴影，则进行补光拍照，得到补充拍照后的待分析图片，进行图像分割学习和训练学习，得到第二结果。
  7. 根据权利要求6所述的基于生物资产阴影区盘点计数的图像识别系统，其特征在于，所述图像分割学习模块包括如下模块：
    模块M1：在畜牧规模化养殖部署面向养殖场景的摄像头；
    模块M2：通过摄像头采集待分析图片，将待分析图片传输给处理服务器；
    模块M3：处理服务器将待分析图片进行使用多任务卷积神经网络进行图像分割，将分割后图像的色斑作为输入，输入给卷积神经网络，卷积神经网络对像素进行标记；
    模块M4：像素标记后的图片划分成a*b个矩形块，提取n个边角矩形块中的所有像素点的RGB值，统计得到n个边角矩形块中的所有像素点的RGB值中相同RGB值的像素点的数量，将预定数量对应的RGB值确定为阈值，其中a表示长边，b表示宽边；
    模块M5：计算剩余a*b-n个矩形块的RGB均值，将RGB均值中大于等于阈值的进行保留，得到C个均值对应的C个矩形块，将C个矩形块的像素点的RGB值形成输入数据，传给卷积层对每个像素进行分类，确定待分析图像的上下文。
  8. 根据权利要求7所述的基于生物资产阴影区盘点计数的图像识别系统，其特征在于，所述图像训练学习模块包括如下模块：
    模块M6：每个像素数据由二维转成一维，对转成一维的通道求出协方差矩阵，求出协方差矩阵的特征向量p和特征值λ，按照公式进行转换，完成颜色随机抖动判断；
    模块M7：色域调整，颜色判断后生成阴影判断模型；
    模块M8：若阴影判断模型判断没有阴影，滑窗判断是否有生物，网络从原图片随机截取边框，进行正负样本学习，进入输出网络，得到第一结果。
  9. 根据权利要求8所述的基于生物资产阴影区盘点计数的图像识别系统，其特征在于，所述边缘装置补充训练模块包括如下模块：
    模块M9：提供边缘装置；
    模块M10：通过边缘装置中的处理器进行语义分析，对于模块M7若阴影判断模型判断有阴影后通过补光灯进行补光拍照，继续模块M2～模块M6，滑窗判断是否有生物，网络从补光后的图片随机截取边框，进行正负样本学习，进入输出网络，得到第二结果。
  10. 根据权利要求9所述的基于生物资产阴影区盘点计数的图像识别系统，其特征在于，所述边缘装置补充训练模块还包括如下模块：
    模块M11：对第一结果和第二结果进行比对，进行卷积网络学习，后续采用T时刻内求平均值；
    模块M12：根据平均值确定a*b内的生物数量求和。
PCT/CN2023/106856 2022-12-09 2023-07-12 基于生物资产阴影区盘点计数的图像识别方法及系统 WO2024119824A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211587300.5 2022-12-09
CN202211587300.5A CN115761610A (zh) 2022-12-09 2022-12-09 基于生物资产阴影区盘点计数的图像识别方法及系统

Publications (1)

Publication Number Publication Date
WO2024119824A1 true WO2024119824A1 (zh) 2024-06-13

Family

ID=85345286

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/106856 WO2024119824A1 (zh) 2022-12-09 2023-07-12 基于生物资产阴影区盘点计数的图像识别方法及系统

Country Status (2)

Country Link
CN (1) CN115761610A (zh)
WO (1) WO2024119824A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115761610A (zh) * 2022-12-09 2023-03-07 上海万向区块链股份公司 基于生物资产阴影区盘点计数的图像识别方法及系统

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150199579A1 (en) * 2014-01-16 2015-07-16 GM Global Technology Operations LLC Cooperative vision-range sensors shade removal and illumination field correction
CN112243582A (zh) * 2019-08-30 2021-01-19 深圳市大疆创新科技有限公司 补光控制方法、装置、系统和存储介质
CN114998132A (zh) * 2022-05-30 2022-09-02 中国电子科技集团公司第二十研究所 一种通过双网络挖掘互补特征的弱监督阴影检测方法
CN115294535A (zh) * 2022-08-08 2022-11-04 河南垂天科技有限公司 一种禁停区域违章停车监测方法
CN115330703A (zh) * 2022-08-03 2022-11-11 南京信息工程大学 基于上下文信息融合的遥感影像云与云阴影检测方法
CN115410251A (zh) * 2022-08-22 2022-11-29 深圳市维度创新实业有限公司 一种基于人脸识别算法的摄像头系统
CN115761610A (zh) * 2022-12-09 2023-03-07 上海万向区块链股份公司 基于生物资产阴影区盘点计数的图像识别方法及系统

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150199579A1 (en) * 2014-01-16 2015-07-16 GM Global Technology Operations LLC Cooperative vision-range sensors shade removal and illumination field correction
CN112243582A (zh) * 2019-08-30 2021-01-19 深圳市大疆创新科技有限公司 补光控制方法、装置、系统和存储介质
CN114998132A (zh) * 2022-05-30 2022-09-02 中国电子科技集团公司第二十研究所 一种通过双网络挖掘互补特征的弱监督阴影检测方法
CN115330703A (zh) * 2022-08-03 2022-11-11 南京信息工程大学 基于上下文信息融合的遥感影像云与云阴影检测方法
CN115294535A (zh) * 2022-08-08 2022-11-04 河南垂天科技有限公司 一种禁停区域违章停车监测方法
CN115410251A (zh) * 2022-08-22 2022-11-29 深圳市维度创新实业有限公司 一种基于人脸识别算法的摄像头系统
CN115761610A (zh) * 2022-12-09 2023-03-07 上海万向区块链股份公司 基于生物资产阴影区盘点计数的图像识别方法及系统

Also Published As

Publication number Publication date
CN115761610A (zh) 2023-03-07

Similar Documents

Publication Publication Date Title
US11429818B2 (en) Method, system and device for multi-label object detection based on an object detection network
US10410353B2 (en) Multi-label semantic boundary detection system
Sarwar et al. Detecting and counting sheep with a convolutional neural network
Li et al. Deep cascaded convolutional models for cattle pose estimation
US11328525B2 (en) Method for calculating deviation relations of a population
Zhu et al. Automatic recognition of lactating sow postures by refined two-stream RGB-D faster R-CNN
CN110909618B (zh) 一种宠物身份的识别方法及装置
WO2024119824A1 (zh) 基于生物资产阴影区盘点计数的图像识别方法及系统
US10331968B2 (en) One shot color calibrated metric learning for object re-identification
WO2021017283A1 (zh) 基于离线型的在线跟踪方法、装置、计算机设备及存储介质
CN111753697B (zh) 智能宠物管理系统及其管理方法
US20240087368A1 (en) Companion animal life management system and method therefor
CN113221864A (zh) 多区域深度特征融合的病鸡视觉识别模型构建及应用方法
US20240087352A1 (en) System for identifying companion animal and method therefor
Xie et al. Online multiple instance gradient feature selection for robust visual tracking
Tang et al. Salient object detection of dairy goats in farm image based on background and foreground priors
Yang et al. Fusion of RetinaFace and improved FaceNet for individual cow identification in natural scenes
Liu et al. Where's your head at? Detecting the orientation and position of pigs with rotated bounding boxes
Bello et al. Mask YOLOv7-based drone vision system for automated cattle detection and counting
Cai et al. Single shot multibox detector for honeybee detection
Bello et al. Features-based individual cattle instance identification method using hybrid deep learning models for sustainable livestock management
Manivannan et al. Dog Breed Classification using Inception-ResNet-V2
CN113221704A (zh) 基于深度学习的动物姿态识别方法、系统及存储介质
Warhade et al. Attention module incorporated transfer learning empowered deep learning-based models for classification of phenotypically similar tropical cattle breeds (Bos indicus)
Altiparmak Development of a vision-based feral vertebrate identifier using fuzzy type II