TWI752493B - Poultry image recognition dwell time analysis system and method - Google Patents

Abstract

A method for analyzing residence time of poultry image recognition, which is suitable for analyzing a plurality of images of poultry to be analyzed, and is executed by a system. Analyze the bird detection results of the bird images, each bird detection result includes at least one bird area position; (B) for each bird image to be analyzed, according to the corresponding bird image to be analyzed. The bird detection result, and the previous bird detection result corresponding to the previous bird image to be analyzed, calculate the overlap rate of each bird area position and its corresponding bird area position; and ( C) According to the bird image waiting to be analyzed, the detection results of the birds and the overlap ratio corresponding to the bird image waiting to be analyzed, calculate the dwell time of the bird waiting to be analyzed.

Description

Poultry image recognition dwell time analysis system and method

The present invention relates to an analysis method for image recognition, in particular to a system and method for analyzing residence time of bird image recognition.

When birds are sick, they often have obvious clinical symptoms such as crouching, lethargy, diarrhea, and feather wrinkling. These symptoms will reduce the activity of birds and stay in place. In order to observe whether the birds have abnormal behaviors due to disease, in the feeding management of traditional poultry farms, managers often enter the on-site poultry house to observe the feeding and behavior of the birds. However, most birds are naturally sensitive and easily frightened, and once frightened, it may be difficult to calm down for several days. Under human interference, it may cause urgent situations of birds, which in turn affect the quality of life of birds or the correctness of abnormal behavior judgments.

At present, poultry farms have gradually begun to introduce photographic equipment to assist in breeding management. As long as the management personnel are in the office, they can monitor the staying status of the birds in the on-site poultry house to analyze whether the birds have decreased activity and risk of disease, and reduce the management personnel. The frequency of entry to the on-site poultry house to reduce disturbance to the birds and the risk of accidental introduction of pathogens.

However, the existing management method still needs to rely on manual 24-hour monitoring of the images of the photographic equipment to grasp the staying status of the birds, which is very labor-intensive.

Therefore, the purpose of the present invention is to provide a bird image recognition dwell time analysis method for automatically calculating the bird dwell time according to the bird image.

Therefore, the residence time analysis method for bird image recognition according to the present invention is suitable for analyzing a plurality of images of birds to be analyzed that are continuously photographed. Only the image recognition dwell time analysis system is performed, and the method includes a step (A), a step (B), and a step (C).

In the step (A), the bird image recognition dwell time analysis system uses a bird detection model for detecting bird regions to detect the bird regions waiting to be analyzed for the bird images to generate A plurality of bird detection results respectively corresponding to the bird images waiting to be analyzed, and each bird detection result includes at least one bird area location.

In this step (B), for each bird image to be analyzed, the bird image recognition dwell time analysis system according to the bird image to be analyzed corresponds to the bird detection result corresponding to at least one bird area position , and at least one bird region position of a previous bird detection result corresponding to a bird image to be analyzed before the previous shooting time, and calculate each bird region position and its corresponding bird The overlap ratio of the region locations.

In this step (C), the poultry image recognition residence time analysis system is based on The bird image waiting to be analyzed, the detection results of the birds and the overlap ratio corresponding to the bird image waiting to be analyzed, the dwell time of the bird waiting to be analyzed is calculated.

Another object of the present invention is to provide a bird image recognition dwell time analysis system that automatically calculates the bird dwell time according to the bird image.

Therefore, the poultry image recognition residence time analysis system of the present invention includes a storage device, an image capture device, and an analysis server.

The storage device is connected to a communication network.

The image capturing device is connected to the storage device via the communication network, and is used for continuously capturing a poultry farm to generate and transmit a plurality of images of the birds to be analyzed to the storage device through the communication network, each of which is to be analyzed. The bird image includes at least one bird and a shooting time.

The analysis server is connected to the storage device via the communication network, and uses a bird detection model for detecting the bird region to detect the bird region waiting to be analyzed for the bird image, so as to generate a plurality of corresponding bird regions respectively. Waiting for the bird detection result of the analyzed bird image, each bird detection result includes at least one bird area position, and for each bird image to be analyzed, according to the bird image corresponding to the bird image to be analyzed Only the position of at least one bird area of the detection result and the position of at least one bird area of the previous bird detection result corresponding to the image of the bird to be analyzed before the previous shooting time was calculated, and each The overlap ratio between the position of the poultry area and the position of the corresponding poultry area, and then according to the pending analysis poultry image, the poultry detection results and the weight corresponding to the pending analysis poultry image The overlap rate is calculated to calculate the dwell time of the bird waiting to analyze the bird image.

The effect of the present invention is: the analysis server uses the bird detection model to detect the bird area of the bird image to be analyzed, and according to the image of the bird to be analyzed and the previous bird to be analyzed image, calculate the overlap rate of each bird area position and its corresponding bird area position, and then according to the pending analysis bird image, the poultry detection results and the corresponding poultry image to be analyzed Calculate the dwell time of the bird waiting to analyze the bird image.

11: Storage device

12: Image capture device

13: Train the server

14: Analysis Server

100: Communication Network

2: Training program

201~205: Steps

3: Analysis procedure

301~312: Steps

41: Poultry area location

42: Poultry area location

43: Intersection area

44: Union area

Other features and effects of the present invention will be clearly presented in the embodiments with reference to the drawings, wherein: Fig. 1 is a block diagram illustrating an embodiment of the dwell time analysis system for bird image recognition according to the present invention; Fig. 2 Fig. 3 is a flow chart illustrating a training procedure of an embodiment of the dwell time analysis method for bird image recognition according to the present invention; an analysis procedure; and FIG. 4 is a schematic diagram illustrating a calculation method of an overlap ratio of two regional positions of birds.

Before the present invention is described in detail, it should be noted that in the following description, similar elements are designated by the same reference numerals.

Referring to FIG. 1 , an embodiment of a system for analyzing residence time of poultry image recognition according to the present invention includes a storage device 11 , an image capturing device 12 , a training server 13 , and an analysis server 14 .

The storage device 11 stores a plurality of training bird images and is connected to a communication network 100 , and each training bird image is marked with at least one bird area. It should be noted that, in this embodiment, the communication network 100 is, for example, the Internet, and each poultry area is marked with a rectangle, but not limited thereto. In other embodiments, the poultry area One can be marked in shapes such as circle, ellipse, and polygon.

The image capturing device 12 is connected to the storage device 11 via the communication network 100 and is used to continuously capture a poultry farm to generate and transmit a plurality of images of the birds to be analyzed to the storage device 11 via the communication network 100 , each bird image to be analyzed includes at least one bird and a shooting time. It should be noted that, in this embodiment, the image capturing device 12 has a camera, and an embedded system (Embedded System) electrically connected to the camera, and the embedded system controls the camera to capture the poultry farm.

It is worth noting that, in this embodiment, the images of the training birds and the images of the birds to be analyzed are all taken from an overhead angle to avoid the phenomenon of overlapping birds in the same image, but this is not the case. limited.

The training server 13 is connected to the storage device 11 via the communication network 100 .

The analysis server 14 is connected to the storage device 11 and the training server 13 via the communication network 100 .

It should be noted that, in this embodiment, since the training server 13 and the analysis server 14 need to process high-resolution images, they both have a Graphics Processing Unit (GPU).

An embodiment of the method for analyzing dwell time for bird image recognition according to the present invention includes a training program 2 and an analysis program 3. The following describes in detail how the dwell time analysis for bird image recognition implements the embodiment.

Referring to FIG. 2 , the training program 2 of this embodiment includes steps 201 to 205 .

In step 201 , the training server 13 obtains the training bird image from the storage device 11 via the communication network 100 .

In step 202, the training server 13 groups the training bird images into a training dataset and a validation dataset.

In step 203, the training server 13 uses a machine learning algorithm to establish a training model according to all the training bird images in the training set. In this embodiment, the machine learning algorithm is, for example, a Convolutional Neural Network (CNN) algorithm, but it is not limit.

In step 204, the training server 13 adjusts the training model according to all training bird images in the validation set to obtain a bird detection model for detecting bird areas.

In step 205 , the training server 13 transmits the bird detection model to the analysis server 14 via the communication network 100 .

Referring to FIG. 3 , the analysis program 3 of this embodiment includes steps 301 to 312 . It should be noted that, in this embodiment, the image capturing device 12, for example, generates and transmits N images of birds to be analyzed to the storage device 11 via the communication network 100, where N>1.

In step 301, initially, that is, when starting to execute the analysis program 3, the analysis server 14 sets i=1 to indicate that the current image corresponds to the first bird image to be analyzed.

1。值得注意的是，每一禽隻區域位置例如包括在待分析禽隻圖像定義的一個二維坐標系的一最小x軸座標、一最小y座標、一最大x軸座標，及一最大y軸座標，其中根據該最小x軸座標、該最小y座標、該最大x軸座標，及該最大y軸座標可在待分析 禽隻圖像中構成一定界框(bounding box)。 In step 302, the analysis server 14 uses the bird detection model to detect the bird area of the i-th bird image to be analyzed, so as to generate a pair of birds corresponding to the i-th bird image to be analyzed Only the detection result, the bird detection result includes K _i poultry area locations, where K _i

1. It is worth noting that the position of each bird area includes, for example, a minimum x-axis coordinate, a minimum y-coordinate, a maximum x-axis coordinate, and a maximum y-axis in a two-dimensional coordinate system defined by the image of the bird to be analyzed. Coordinates, wherein a certain bounding box can be formed in the image of the bird to be analyzed according to the minimum x-axis coordinate, the minimum y-axis coordinate, the maximum x-axis coordinate, and the maximum y-axis coordinate.

In step 303, the analysis server 14 determines whether i is greater than 1. When the analysis server 14 determines that i is not greater than 1, the process proceeds to step 304; and when the analysis server 14 determines that i is greater than 1, the process proceeds to step 305.

In step 304 , the analysis server 14 sets i as i+1 for the (i+1)th bird image to be analyzed, and repeats step 302 .

In step 305, the analysis server 14 sets j=1 to indicate the position of the j-th bird area currently corresponding to the i-th bird image to be analyzed.

In step 306, the analysis server 14 calculates the position of the j-th bird area and its position in the i-1-th bird area according to the position of the j-th bird area and the i-1-th bird image to be analyzed An overlap ratio of the position of the bird region corresponding to the bird image to be analyzed.

It should be noted that, referring to FIG. 4 , in this embodiment, the analysis server 14 is based on the position 41 of the j-th bird area and the bird corresponding to the i-1-th bird image to be analyzed. The intersection area 43 and the union area 44 of the area position 42 are calculated, and the overlap ratio is the jth poultry area position 41 and the poultry area position 42 corresponding to the i-1th bird image to be analyzed. The intersection area 43 of , divided by the union area 44 of the j-th bird region position 41 and the bird region position 42 corresponding to the i-1th bird image to be analyzed, but not limited thereto.

In step 307, the analysis server 14 determines whether the overlap ratio is greater than a threshold value. When the analysis server 14 determines that the overlap ratio is not greater than the threshold value, The process goes to step 308 ; and when the analysis server 14 determines that the overlap ratio is greater than the threshold value, the process goes to step 309 . It should be noted that, in this embodiment, the threshold value is, for example, 0.9, but not limited thereto.

In step 308, the analysis server 14 generates a bird stop event data including a stop start time, the j-th bird area position, and a cumulative time, wherein the stop start time is the image of the bird to be analyzed The shooting time of the image, and the initial value of the accumulated time is 0.

In step 309, the analysis server 14 obtains the bird stay event data including the position of the jth bird area at the position of the bird area corresponding to the i-1th bird image to be analyzed. The stop start time of the bird stop event data and the shooting time of the i-th bird image to be analyzed, update the accumulated time of the bird stop event data, and store the j-th bird area location to the bird Stop event data. It is worth noting that the accumulated time is the stay time of the bird, and the accumulated time is the time taken from the shooting time of the i-th bird image to be analyzed minus the stay start time of the bird stay event data.

It should be noted that, for the convenience of management, in other embodiments, for each bird area location, the analysis server 14 may also generate a unique identifier (Unique Identifier, UID) for the corresponding bird area location, And in step 308, the position of the j-th poultry area in the poultry stay event data is replaced with the unique identification code corresponding to the position of the j-th poultry area, and in step 309, the unique identification code can be quickly Quickly obtain the bird stay event data including the unique identification code, and store the unique identification code corresponding to the jth bird area position to the bird stay event data.

After step 308, 309, 310, the analysis server 14 determines that the j-th position is the region of the poultry K _i th poultry region position, that is, whether j = K _i. When the analysis server 14 determines that the position of the j-th region is not the first poultry K _i th poultry area location, the flow proceeds to step 311; and when the analysis server 14 determines that the position of the j-th region poultry When it is the K _i th bird area position, the process proceeds to step 312 .

In step 311 , the analysis server 14 sets j to j+1 for the (j+1)th bird area position, and repeats step 306 .

In step 312, the analysis server 14 determines whether the i-th bird image to be analyzed is the N-th bird image to be analyzed, that is, determines whether i=N. When the analysis server 14 determines that the i-th bird image to be analyzed is not the N-th bird image to be analyzed, step 304 is repeated; and when the analysis server 14 determines that the i-th bird image to be analyzed is not the bird image to be analyzed If only the image is the Nth bird image to be analyzed, the analysis procedure 3 is ended.

To sum up, in the system and method for analyzing dwell time of bird image recognition according to the present invention, the training server 13 establishes the bird detection model according to the training bird images, and then uses the analysis server 14 to use the bird detection model. The bird detection model detects the area of the bird to be analyzed in the bird image, and calculates the i-th bird image according to the i-th bird image to be analyzed and the i-1th bird image to be analyzed. The j-th bird region of the bird image to be analyzed Set the overlap ratio with the position of the bird area corresponding to the i-1 th bird image to be analyzed, and then according to the image of the bird to be analyzed, the detection results of the birds and the image of the bird to be analyzed According to the corresponding overlap rate, the dwell time of the bird waiting to be analyzed for the bird image is calculated, so the object of the present invention can indeed be achieved.

However, the above are only examples of the present invention, and should not limit the scope of implementation of the present invention. Any simple equivalent changes and modifications made according to the scope of the patent application of the present invention and the contents of the patent specification are still included in the scope of the present invention. within the scope of the invention patent.

3: Analysis procedure

301~312: Steps

Claims (14)

A method for analyzing residence time of poultry image recognition, which is suitable for analyzing a plurality of consecutively shot images of poultry to be analyzed. The identification dwell time analysis system is executed, and the method includes the following steps: (A) using a bird detection model for detecting bird regions to detect the bird region waiting to be analyzed for the bird image, so as to generate a plurality of Corresponding to the bird detection results of the bird images waiting to be analyzed, each bird detection result includes at least one bird area position; (B) for each bird image to be analyzed, according to the bird to be analyzed image The position of at least one bird region of the bird detection result corresponding to the image, and the at least one bird region of a previous bird detection result corresponding to a bird image to be analyzed before the previous shooting time position, calculating the overlap ratio between the position of each bird region in the bird image to be analyzed and the position of the bird region corresponding to the previous bird image to be analyzed; and (C) according to the waiting analysis The bird images, the bird detection results and the overlap ratios corresponding to the bird images waiting to be analyzed are used to calculate the dwell time of the birds waiting to be analyzed. The method for analyzing residence time of bird image recognition according to claim 1, wherein step (C) includes the following sub-steps: (C-1) for each overlap ratio corresponding to each bird image to be analyzed, Determine whether the overlap ratio is greater than a threshold value; (C-2) for each overlap ratio corresponding to each bird image to be analyzed, when it is determined that the overlap ratio is not greater than the threshold value, generate an image including a The stay start time, the position of the bird area corresponding to the overlap rate in the at least one bird area position, and the poultry stay event data of a cumulative time, wherein the stay start time is the shooting time of the image of the bird to be analyzed; and (C-3) for each overlap ratio corresponding to each bird image to be analyzed, when it is determined that the overlap ratio is greater than the threshold value, obtain an image including the overlap ratio corresponding to the at least one bird area position. The bird stay event data at the location of the poultry area, and according to the stay start time of the bird stay event data and the shooting time of the image of the bird to be analyzed, update the cumulative time of the bird stay event data, and at least A bird area position corresponding to the overlap rate in a bird area position is stored in the bird stop event data. The method for analyzing residence time of poultry image recognition according to claim 2, wherein, in step (C-1), the threshold value is 0.9. The method for analyzing residence time of poultry image recognition according to claim 1, wherein, in step (B), each overlap ratio is the intersection area of the position of the poultry area and its corresponding position of the poultry area divided by the poultry area The combined area of the area location and its corresponding poultry area location. According to the method for analyzing residence time of bird image recognition according to claim 1, the system for analyzing residence time of bird image recognition stores a plurality of training bird images, and each training bird image is marked with at least one bird region, before step (A), it also includes the following steps: (D) according to the training bird images, using a machine learning algorithm to establish the bird detection model. The method for analyzing residence time of poultry image recognition according to claim 5, wherein step (D) comprises the following sub-steps: (D-1) Group the training bird images into a training set and a validation set; (D-2) According to all the training bird images in the training set, use the machine learning algorithm to establish a training model ; and (D-3) adjusting the training model according to all training bird images in the validation set to obtain the bird detection model. The method for analyzing residence time of poultry image recognition according to claim 5, wherein, in step (D), the machine learning algorithm is a convolutional neural network road algorithm. A system for analyzing residence time of poultry image recognition, comprising: a storage device connected to a communication network; an image capture device connected to the storage device via the communication network, and used for continuously photographing a poultry farm to generate and transmit a plurality of images of the poultry to be analyzed to the storage device through the communication network, each image of the poultry to be analyzed includes at least one bird and a shooting time; an analysis server is connected to the storage device through the communication network , using a bird detection model for detecting the bird area, to detect the bird area waiting for the analysis of the bird image, so as to generate a plurality of bird detection results corresponding to the bird image waiting to be analyzed , each bird detection result includes at least one bird region position, and for each bird image to be analyzed, according to the at least one bird region position of the bird detection result corresponding to the bird image to be analyzed, and a position of at least one bird region of a previous bird detection result corresponding to a bird image to be analyzed before the previous shooting time, to calculate each bird located in the bird image to be analyzed The location of the area and its corresponding the overlap ratio of the position of the bird region in the previous bird image to be analyzed, and then the overlap ratio corresponding to the bird image to be analyzed, the bird detection results and the bird image to be analyzed, The dwell time of the bird waiting to analyze the bird image is calculated. The poultry image recognition residence time analysis system according to claim 8, wherein, for each overlap ratio corresponding to each bird image to be analyzed, the analysis server determines whether the overlap ratio is greater than a threshold value, When it is determined that the overlap rate is not greater than the threshold value, the analysis server generates a bird area including a stop start time, a bird area position corresponding to the overlap rate in the at least one bird area position, and an accumulated time. stay event data, the stay start time is the shooting time of the image of the bird to be analyzed; when it is determined that the overlap rate is greater than the threshold value, the analysis server obtains the location of the at least one bird area corresponding to the overlap rate The bird stop event data at the location of the bird area, and according to the stop start time of the bird stop event data and the shooting time of the image of the bird to be analyzed, update the accumulated time of the bird stop event data, and the The bird area position corresponding to the overlap ratio in at least one bird area position is stored in the bird stop event data. The poultry image recognition residence time analysis system according to claim 9, wherein the threshold value is 0.9. The bird image recognition residence time analysis system according to claim 8, wherein each overlap ratio is the intersection area of the bird area position and its corresponding bird area position divided by the bird area position and its corresponding bird area Only the union area of the region location. The poultry image recognition residence time analysis system according to claim 8, further Including: a training server, connected to the storage device and the analysis server through the communication network, storing a plurality of training bird images, each training bird image is marked with at least one bird area, and according to the training bird images, using a machine learning algorithm to build the bird detection model, and sending the bird detection model to the analysis server via the communication network. The bird image recognition dwell time analysis system of claim 12, wherein the training server groups the training bird images into a training set and a validation set, and analyzes all training birds according to the training set. image, use the machine learning algorithm to establish a training model, and then adjust the training model according to all training bird images in the validation set to obtain the bird detection model. The system for analyzing residence time of poultry image recognition according to claim 12, wherein the machine learning algorithm is a convolutional neural network road algorithm.

Images