KR101910843B1 - Method for depth based pig segmentation using concave-points and superpixel information - Google Patents

Abstract

The present invention relates to a method to distinguish pigs adjacent to each other from an image photographed by a closed cable television (CCTV) terminal installed in a pigpen. According to the present invention, the method comprises: a step (1) of photographing pigs inside a pig room by a monitoring CCTV terminal to acquire a digitized depth image; a step (2) of acquiring a region of interest (ROI); a step (3) of fixing two neighboring points with a predetermined interval in an ROI image to generate time-series data; a step (4) of extracting a concave point with a point having a minimum value from the time-series data; a step (5) of using the concave point to define start and end points; a step (6) of using the start and end points to fix a boundary extraction area; a step (7) of using a superpixel found from the depth image within the boundary extraction area to fix points in contact with each boundary line as a node so as to generate a graph; and a step (8) of separating and distinguishing the pigs adjacent to each other with respect to the shortest path on the graph.

Description

METHOD FOR DEPTH BASED PIG SEGMENTATION USING CONCAVE-POINTS AND SUPERPIXEL INFORMATION BACKGROUND OF THE INVENTION 1. Field of the Invention < RTI ID = 0.0 >

The present invention relates to a method for distinguishing pigs close to each other in an image captured by a CCTV terminal installed in a pig house, more specifically, using a concave point and super-pixel information to determine a boundary line between adjacent pigs, Based proximity pig discrimination method using a concave dot and super pixel information.

Smart farms are being introduced as the introduction of ICT is accelerating in agriculture and livestock industries. Smart Farm is a farm where automated management is performed by using Internet of Things (IoT) technology. The farm where smart farm is implemented analyzes the temperature and humidity of the crop cultivation facility, and according to the analysis result, .

Smart farms are applied not only to crop cultivation facilities but also to pig farms. Smart pig farms, which are generally applied to the Internet technology of things, are called Smart pig farms. In such a smart puzzle company, it is possible to check real-time situation of a pig company using object internet technology, and to control temperature and humidity. In particular, smart pens are developing systems that can monitor and analyze the condition of individual pigs, which are built on costly facility control techniques. Therefore, individual pig status analysis system based on facility control technology is being introduced mainly in developed countries where large investment is made in pork production environment, and domestic pork producers can not introduce such a system because of cost problem.

On the other hand, apart from facility control technology, studies on technology for analyzing the status of individual pigs using CCTV terminal are under way in Smart Pig. Using CCTV technology for analyzing individual pigs' condition, it is possible to check the situation in the pig house in real time and to check how pigs react according to the temperature, humidity, It is possible to understand disease and development status of individual pigs.

CCTV technology for individual pig status analysis can significantly improve the productivity of pork at a very low cost compared to existing facility control based pig managment technology and is especially suitable for domestic pig farming environment. However, the CCTV technology for analyzing the individual pig condition may cause the monitoring CCTV terminal price to rise when the RGB camera is used. Also, when the pigs are close to each other in the piggy bank without illumination at night, .

On the other hand, the Superpixel algorithm used in the image processing field is a technology for grouping semantic regions by capturing redundancy of images. These superpixel algorithms are mainly used for image depth estimation, object segmentation, and object localization. In the super pixel algorithm, SLIC is used to extract the superpixel boundary, and SLIC samples the center of the initial cluster at the interval set in the grid interval. Then, the cluster is searched from the edge of the cluster to improve the cluster, and the center of the cluster is set again. This process can be repeated for a predetermined number of repetitions to calculate super pixels.

Meanwhile, Korean Patent No. 10-1072881 " remote management system of agriculture and livestock facility " has been proposed as a prior art related to smart farm.

The present invention has been proposed in order to solve the above-mentioned problems of the previously proposed methods. By using the depth camera of the IR system in the monitoring CCTV, the depth image information The present invention aims to provide a depth-based proximity pig classification method using a concave point and superpixel information, which can distinguish individual pigs using superpixel information near a concave point.

Further, the present invention provides a depth-based proximity pig classification method using a concave point and super pixel information that can minimize the price of a monitoring CCTV terminal using an RGB camera by using an IR type depth camera for monitoring CCTV The purpose of that is to do.

According to an aspect of the present invention, there is provided a depth-based proximity pig classification method using a concave point and super pixel information,

Monitoring using IR-based depth camera A pig identification method for distinguishing pigs in a piggy bank using depth image information obtained from a CCTV terminal,

(1) acquiring a binned depth image by photographing a pig in a money bag at the monitoring CCTV terminal;

(2) acquiring the region of interest in the depth image obtained in the step (1) as an area of interest in a pig close to each other when the pig is adjacent to the depth image obtained in the step (1);

(3) searching for an outline of the region of interest on the ROI image obtained in the step (2), designating two adjacent points having a predetermined interval before and after the search point on the outline, and generating time series data;

(4) extracting a point having a minimum value from the time series data generated in the step (3) as a concave point;

(5) defining a start point and an end point using the concave point extracted in the step (4);

(6) designating a boundary extraction region using the starting point and the ending point defined in the step (5);

(7) generating a graph by designating points where the respective boundary lines meet, using super pixels obtained from the depth image within the boundary extracting area specified in step (6); And

(8) a step of obtaining a shortest path on the node from the starting point to the end point in the graph generated in the step (7), and separating and separating the pigs close to each other on the basis of the shortest path, .

Preferably, the step (3)

Time-series data can be generated by calculating the width of the triangle formed by the search point and the two adjacent points.

Preferably, the step (3)

The outline of the area of interest can be searched counterclockwise.

Preferably, the step (3)

The predetermined interval may be an interval of 5-20% of the entire length of the outline of the area of interest.

Advantageously, said step (5)

If the number of the concave points extracted in the step (4) is 2, the two concave points are defined as the starting point and the end point, respectively. If the number of the concave points extracted in the step (4) is 1, A straight line that divides the angle between the concave point and the two adjacent points that are matched with the concave point is divided into half and a point at which the straight line and the outline of the RO meet meet can be defined as an end point.

Advantageously, said step (6)

A rectangle having a width corresponding to the two adjacent point intervals matching the starting point can be designated as the boundary extraction area.

Advantageously, said step (7)

A super pixel can be obtained using SLIC in the boundary extracting area specified in step (6).

Preferably, the step (1)

The histogram smoothing technique can be applied to obtain the binarized image with the corrected depth value.

According to the depth-based proximity pig classification method using the concave point and super-pixel information proposed in the present invention, by using an IR type depth camera for the monitoring CCTV, the depth obtained by the depth camera even in the pig- It is possible to extract the concave points using only the image information and distinguish the individual pigs using the super pixel information near the concave points.

In addition, according to the present invention, by using an IR depth camera for the monitoring CCTV, the price of the monitoring CCTV terminal using the RGB camera can be minimized.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram showing each step of a depth-based proximity pig classification method using a concave point and superpixel information according to an embodiment of the present invention; FIG.
FIG. 2 is a view illustrating a camera installed in a money pouch in a pig house in a depth-based proximity pig classification method using a concave point and super pixel information according to an embodiment of the present invention.
FIG. 3 is a view showing an interest region image of a frame in which two pigs are not distinguished from each other due to the occurrence of pigs close to each other in the depth-based proximity pig classification method using the concave point and super pixel information according to an embodiment of the present invention.
FIG. 4 is a view showing a state in which time-series data is generated from the outline of a region of interest of a pig close to each other and a concave point is extracted in a depth-based proximity pig classification method using a concave point and super pixel information according to an embodiment of the present invention .
5 is a view showing a state in which a boundary extraction region is designated when one concave point extracted in the depth-based proximity pig classification method using the concave point and super pixel information according to an embodiment of the present invention is used.
6 is a diagram illustrating a superpixel and a path graph obtained by SLIC in a depth-based proximity pig classification method using a concave point and superpixel information according to an embodiment of the present invention.
FIG. 7 is a view showing a region of interest separated into two pigs by a shortest path in a depth-based proximity pig classification method using a concave point and superpixel information according to an embodiment of the present invention; FIG.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. In the following detailed description of the preferred embodiments of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. The same or similar reference numerals are used throughout the drawings for portions having similar functions and functions.

In addition, in the entire specification, when a part is referred to as being 'connected' to another part, it may be referred to as 'indirectly connected' not only with 'directly connected' . Also, to "include" an element means that it may include other elements, rather than excluding other elements, unless specifically stated otherwise.

FIG. 1 is a diagram illustrating respective steps of a depth-based proximity pig classification method using a concave point and superpixel information according to an embodiment of the present invention. 1, a depth-based proximity pig classification method using a concave point and super pixel information according to an exemplary embodiment of the present invention includes acquiring a binned depth image by photographing a pig in a money bag in a monitoring CCTV terminal A step S200 of setting an area in which the object to be separated in the pig close to each other is set as an area of interest and a region of interest image is obtained when a pig close to each other is generated in the depth image obtained in the step S100, The time series data is generated by searching for an outline of a region of interest in the interest region image obtained in S200 by designating two adjacent points having a predetermined interval before and after the search point on the outline (S300). In the step S300, Extracting a point having a minimum value as a concave point (S400), and determining a starting point and an end point using the concave point extracted in S400 A step S600 of designating a boundary extracting area using the starting point and the ending point defined in S500, a step S600 of using the super pixel obtained from the depth image in the boundary extracting area designated in S600, A shortest path from the start point to the end point is obtained in the graph generated in step S700, and the pigs separated from each other on the basis of the shortest path are separated (S800) . ≪ / RTI > Each step of the present invention can be performed in the control unit included in the CCTV terminal according to the embodiment, or can be performed in a separate server. Hereinafter, each step of the depth-based proximity pig classification method using the concave point and super pixel information according to an embodiment of the present invention will be described in detail.

In step S100, the monitoring CCTV terminal can acquire a binned depth image by photographing a pig in a money bag. Here, the monitoring CCTV terminal can use the IR depth camera.

2 is a view showing a camera installed in a money pouch in a piggy bank in a depth-based proximity pig classification method using a concave point and super pixel information according to an embodiment of the present invention. According to an embodiment of the present invention, the depth camera is installed at a height of 3 m from the money room in the pig, as shown in FIG. 2, and acquires the binarized depth image by photographing the pig in the pig pool. At this time, in step S100, a binarized image obtained by correcting the depth value can be obtained by applying the histogram smoothing technique to improve the accuracy of the depth value in the depth image.

In step S200, if a pig close to each other is generated in the depth image obtained in step S100 and a distinction is required, a part that requires object separation in a pig close to each other can be set as an area of interest and a region of interest image can be obtained.

3 is a view showing a region of interest of a frame in which two pigs are separated from each other due to the occurrence of pigs close to each other in the depth-based proximity pig classification method using the concave point and the super pixel information according to an embodiment of the present invention . In step S200, when pigs close to each other are generated, the pigs are in close proximity to each other as shown in FIG. 3, and the pigs are connected to each other in the binarized image. In this case, in the binarized image, two pigs can be identified as one pig, not each pig. Therefore, object separation is necessary to distinguish pigs that are close to each other.

In step S300, the contour of the area of interest is searched for in the interest area image acquired in step S200, and two adjacent points having a predetermined interval are specified before and after the search point on the outline, thereby generating time series data. In this case, the time series data can be generated by calculating the width of the triple shape composed of the search point and two adjacent points.

FIG. 4 is a view showing a state in which time-series data is generated from the outline of a region of interest of a pig close to each other and a concave point is extracted in a depth-based proximity pig classification method using a concave point and super pixel information according to an embodiment of the present invention to be. 4A, in step S300, an arbitrary point on the outline of the area of interest is set as a search point, and two adjacent points having a predetermined interval before and after the search point are specified. Here, the predetermined interval may be an interval of 5-20% of the total length of the outline of the area of interest. But may be, but is not limited to, 10% of the total length of the outline of the area of interest. On the other hand, in step S300, the outline of the region of interest can be searched clockwise or counterclockwise. In step S300, as shown in FIG. 4B, time series data may be generated by calculating the widths of the three-dimensional shape including the search point and the two adjacent points along the outline of the ROI.

In step S400, a point having a minimum value in the time series data generated in step S300 can be extracted as a concave point. These concave points are used as a reference point for separating pigs close to each other.

In step S500, the start point and the end point can be defined using the concave point extracted in step S400. More specifically, when the number of concave points extracted in step S400 is two, two concave points can be defined as a starting point and an ending point, respectively. Alternatively, if the number of concave points extracted in step S400 is 1, a straight line dividing the angle formed by two adjacent points matching the concave and concave points with the concave point as the starting point is obtained, and the straight line and the outline Can be defined as an end point.

In step S600, the boundary extraction area can be designated using the starting point and the ending point defined in step S500.

FIG. 5 is a view illustrating a method of specifying a boundary extraction region when there is one concave point extracted in the depth-based proximity pig classification method using the concave point and super-pixel information according to an embodiment of the present invention. As shown in Fig. 5, if the number of concave points extracted in step S400 is 1, the concave point can be defined as the starting point in step S500. In step S500, a straight line that divides the angle formed by two adjacent points that are matched with the concave and the concave points is obtained, and a point where the straight line and the outline of the ROI meet each other can be defined as an end point. At this time, the starting point and the ending point are located on the same straight line. In step S600, a rectangle having a width corresponding to two adjacent point intervals matching the starting point can be designated as the boundary extraction area, and the long side of the rectangle is parallel to the straight line. On the other hand, in step S600, when there are two concave points, the two concave points can be designated as the boundary extraction area as the start point and the end point, respectively.

In step S700, a super pixel obtained from the depth image in the boundary extracting area designated in step S600 can be used to designate the points where the boundary lines meet to form a graph. At this time, in step S700, a super pixel can be obtained using SLIC in the boundary extracting area specified in step S600. The superpixel algorithm used in the present invention and the SLIC for obtaining a superpixel may be understood by those skilled in the art.

FIG. 6 is a diagram illustrating a superpixel and a path graph obtained by SLIC in a depth-based proximity pig classification method using a concave point and superpixel information according to an embodiment of the present invention. As shown in FIG. 6A, in step S700, the points where the respective boundary lines meet within the boundary extracting area are set as nodes by using the super pixels obtained using SLIC. In step S700, as shown in (b) of FIG. 6, a graph can be generated by connecting each node along a boundary line.

In step S800, the shortest path on the node from the start point to the end point may be obtained in the graph generated in step S700, and the pigs close to each other may be separated based on the shortest path.

FIG. 7 is a view showing a state in which a region of interest is divided into two pigs by a shortest path in a depth-based proximity pig classification method using a concave point and super pixel information according to an embodiment of the present invention. As shown in FIG. 7, in step S800, the shortest path on the node is obtained from the starting point to the end point, and the shortest path is obtained by connecting the boundary lines connecting the nodes. In step S800, the pigs close to each other can be separated based on the shortest path connecting the boundary lines on the node.

As described above, according to the depth-based proximity pig classification method using the concave point and super-pixel information proposed in the present invention, by using the IR-type depth camera for the monitoring CCTV, the depth of the pig- It is possible to extract the concave points using only the depth image information obtained from the camera and to distinguish the individual pigs using the super pixel information near the concave points. In addition, according to the present invention, by using an IR depth camera for the monitoring CCTV, the price of the monitoring CCTV terminal using the RGB camera can be minimized.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

S100: monitoring CCTV terminal to capture the binned depth image of the pig in the money bag
S200: If a pig close to each other is generated in the depth image acquired in S100 and separation is required, a region requiring an object separation in a pig close to each other is set as an area of interest and a region of interest is acquired
S300: generating the time series data by searching the outline of the region of interest in the interest region image acquired in S200, designating two adjacent points having a predetermined interval before and after the search point on the outline,
S400: extracting a point having a minimum value in the time series data generated in S300 as a concave point
S500: Define a starting point and an end point using the concave point extracted in S400
S600: Step of designating boundary extraction region using starting point and ending point defined in S500
S700: generating a graph by designating points where the respective boundary lines meet by using the super pixels obtained from the depth image within the boundary extraction region designated in S600
S800: The shortest path from the start point to the end point is obtained from the graph generated in S700, and the pigs separated from each other based on the shortest path are separated and classified

Claims (8)

Monitoring using IR-based depth camera A pig identification method for distinguishing pigs in a piggy bank using depth image information obtained from a CCTV terminal,
(1) acquiring a binned depth image by photographing a pig in a money bag at the monitoring CCTV terminal;
(2) acquiring the region of interest in the depth image obtained in the step (1) as an area of interest in a pig close to each other when the pig is adjacent to the depth image obtained in the step (1);
(3) searching for an outline of the region of interest on the ROI image obtained in the step (2), designating two adjacent points having a predetermined interval before and after the search point on the outline, and generating time series data;
(4) extracting a point having a minimum value from the time series data generated in the step (3) as a concave point;
(5) defining a start point and an end point using the concave point extracted in the step (4);
(6) designating a boundary extraction region using the starting point and the ending point defined in the step (5);
(7) generating a graph by designating points where the respective boundary lines meet, using super pixels obtained from the depth image within the boundary extracting area specified in step (6); And
(8) calculating a shortest path on the node from the start point to the end point in the graph generated in the step (7), and separating and distinguishing the pigs close to each other based on the shortest path,
The step (3)
Wherein the time series data is generated by calculating a width of a triangle composed of the search point and the two adjacent points, and generating the time series data by using the concave point and the super pixel information.
delete 2. The method of claim 1, wherein step (3)
Wherein the contour of the region of interest is searched in a counterclockwise direction.
2. The method of claim 1, wherein step (3)
Wherein the predetermined interval is an interval of 5-20% of the entire length of the contour of the area of interest.
2. The method of claim 1, wherein step (5)
If the number of the concave points extracted in the step (4) is 2, the two concave points are defined as the starting point and the end point, respectively. If the number of the concave points extracted in the step (4) is 1, Wherein a straight line dividing the angle formed by the two adjacent points that are matched with the concave point and the concave point is obtained and the point where the straight line and the outline of the area of interest meet each other is defined as an end point Depth based proximity pig identification method using point and super pixel information.
2. The method of claim 1, wherein step (6)
And a rectangle having a width corresponding to the two adjacent point intervals matching with the starting point is designated as a boundary extraction region by using the concave point and the super pixel information.
The method of claim 1, wherein step (7)
Wherein a super pixel is obtained using a slick (SLIC) in a boundary extraction region designated in the step (6).
2. The method of claim 1, wherein the step (1)
And a binarized image obtained by correcting the depth value is obtained by applying a histogram smoothing technique. The depth-based proximity pig classification method using the concave point and the super pixel information.

Images