KR102434210B1 - Smart farm growth management system and method for various objects by panoramic image - Google Patents

Abstract

A smart farm growth management system and method of various objects by panoramic images are presented. According to an embodiment, a method for managing the growth of various entities in a smart farm by using a panoramic image includes: taking a video in a smart farm using a camera; generating a panoramic image representing an entire image by using the captured video; designating a part of a crop to be managed by a user in the panoramic image as a region of interest (ROI); analyzing characteristic information of a portion designated as the region of interest (ROI); extracting a portion matching the region of interest (ROI) from the panoramic image by using the feature information; accumulating feature information on the extracted part; And it may be made including the step of confirming the change during the growth period through the analysis of the accumulated data.

Description

SMART FARM GROWTH MANAGEMENT SYSTEM AND METHOD FOR VARIOUS OBJECTS BY PANORRAMIC IMAGE

The following embodiments relate to a smart farm growth management system and method of various objects by a panoramic image, and more specifically, in the process of generating a panoramic image through a video and checking the growth status through analysis of characteristic information for an area of interest it's about

In the smart farm, many technologies are applied to obtain information on growth information during growth as well as agricultural products produced as technologies such as IoT (Internet of Things) and data-based management are applied. Various technologies are used to obtain information on these agricultural products. In particular, a lot of information is inherent in the image of the camera and can be stored. However, in order to obtain information from an image, image processing technology is required. Although general image processing technology has been widely distributed, as interest in smart farms deepens, more stable technologies for processing and sharing information are being distributed, and appropriate image processing technology is needed.

In the smart farm, information on agricultural products can be obtained and stored by repeatedly shooting every hour and every day using a camera in a certain area or a fixed location. A large amount of information is obtained, and it takes a lot of time to process the information for this purpose, and the latest advanced technologies are used. A method of obtaining a desired area or object from several photos or videos using features has been obtained based on computer vision technology and image processing technology. Its features include color, texture, and edge, and in particular, in the case of an edge, methods for obtaining a region and an object by combining it with color have been proposed. However, it is still difficult to process images collected from cameras in a natural environment. Recently, as advanced forms such as RCNN networks using deep learning have been proposed, they are applied to the field of extracting edges from images and obtaining the contours of objects, but methods for object detection are still not available. continues to be studied.

As such, the existing method cannot manage a large area by locating a camera at a specific point for the recognition of crops, and the recognition methods for automation are performed only under a specific environment and require pre-accumulated databases.

Korean Patent Application Laid-Open No. 10-2020-0045219 describes a technology related to a smart farm to which a remote control system using a smart control board for simultaneous communication of camera images and data is applied.

Korean Patent Publication No. 10-2020-0045219

The embodiments describe a smart farm growth management system and method of various entities by a panoramic image, and more specifically, provide a technology for generating a panoramic image through an image and confirming the growth status through analysis of characteristic information on an area of interest. .

In the embodiments, a video is periodically taken using a camera moving in a certain path, a panoramic image is constructed using the captured video, and then a database for the efficiency of crop cultivation management of a smart farm is created using panoramic image processing. A smart farm growth management system and method for various objects by panoramic images that can be built and extracted data for each part of interest, and accumulated the state by the size and color of the extracted area, so that the entire growth process can be efficiently managed. is to provide

According to an embodiment, a method for managing the growth of various entities in a smart farm by using a panoramic image includes: taking a video in a smart farm using a camera; generating a panoramic image representing an entire image by using the captured video; designating a part of a crop to be managed by a user in the panoramic image as a region of interest (ROI); analyzing characteristic information of a portion designated as the region of interest (ROI); extracting a portion matching the region of interest (ROI) from the panoramic image by using the feature information; accumulating feature information on the extracted part; And it may be made including the step of confirming the change during the growth period through the analysis of the accumulated data.

In the step of generating a panoramic image representing the entire image by using the moving image, the image is divided in frame units in the moving image, and matching parts of the image divided based on the feature points obtained through the SIFT operation are connected to create a panoramic image. can be completed.

The analyzing of the characteristic information of the portion designated as the region of interest (ROI) may include analyzing characteristic information of at least any one of color, texture, and shape in the selected region of interest (ROI).

In the step of extracting the portion matching the region of interest (ROI) from the panoramic image using the feature information, the feature coincides with the region of interest (ROI) in the entire panoramic image as the region of interest is selected from the panoramic image. A portion having information may be selected, and the selected portion may be registered as a new region of interest (ROI).

The step of extracting a portion matching the region of interest (ROI) from the panoramic image by using the feature information may include selecting at least any one or more of feature information among color, texture, and shape from the selected region of interest (ROI). A portion matching the region of interest (ROI) may be extracted from the panoramic image.

The step of extracting a portion that matches the region of interest (ROI) from the panoramic image by using the feature information includes selecting feature information about a color from the selected region of interest (ROI) and selecting the feature information for the color in the panoramic image. ROI), and the matching parts can be obtained and displayed through the analysis result in the RGB color space and the analysis result in the HSV space.

The step of extracting a portion that matches the region of interest (ROI) from the panoramic image using the feature information includes selecting feature information about a texture in the selected region of interest (ROI) and selecting the feature information on the texture from the panoramic image. ROI), and by analyzing the texture characteristics obtained in the region of interest (ROI) using Color-LBP, the portions matching the analysis result may be obtained and displayed.

The step of extracting a portion matching the region of interest (ROI) from the panoramic image by using the feature information includes selecting feature information about a shape in the selected region of interest (ROI) and selecting the feature information on the shape from the panoramic image. ROI) is extracted, an outline is found in the region of interest (ROI), and then a portion matching the shape of the region of interest (ROI) can be found using TCDS for shape matching.

The method may further include managing the region of interest (ROI), wherein the managing of the region of interest (ROI) may include adding or deleting the region of interest (ROI).

The step of accumulating the feature information on the extracted portion includes continuously accumulating feature information or values for the location of the extracted portions coincident with the region of interest (ROI) even if the panoramic image continuously changes according to the growth period. can do it

In the step of accumulating the feature information on the extracted part, it is possible to build a database for the growth period by storing at least any one or more feature information of color, texture, and shape for the region of interest (ROI) in the panoramic image. have.

A smart farm growth management system of various entities by a panoramic image according to another embodiment includes: a panoramic image generator for generating a panoramic image representing an entire image by using a video captured in a smart farm; an ROI selection unit that analyzes characteristic information of a portion designated as a region of interest (ROI) in response to designating a portion of a crop that a user wants to manage in the panoramic image as a region of interest (ROI); an ROI matching unit that extracts a portion matching the region of interest (ROI) from the panoramic image by using the feature information; a data storage unit for accumulating the extracted feature information on the part; and a data analysis unit for confirming changes during the growth period through analysis of accumulated data.

The ROI selector may analyze characteristic information of at least any one of a color, a texture, and a shape in the selected region of interest (ROI).

The ROI matching unit selects a portion having characteristic information matching the region of interest (ROI) in the entire panoramic image as the region of interest is selected in the panoramic image, and registers the selected portion as a new region of interest (ROI). can do.

It further includes an ROI manager for managing the region of interest (ROI), wherein the ROI manager may add or delete the region of interest (ROI).

According to embodiments, a video is periodically shot using a camera moving in a certain path, a panoramic image is constructed using the captured video, and then a panoramic image processing is used to improve the efficiency of crop cultivation management in a smart farm. A smart farm growth management system for various objects by panoramic images that builds a database, extracts data for each part of interest, and accumulates the state by size and color of the extracted area to efficiently manage the entire growth process; and method can be provided.

According to embodiments, by using a panoramic image, overlapping or repetitive information on the same area can be identified, information processing is easy, and a lot of information required when image management for a large area is required through video recording. It is possible to provide a smart farm growth management system and method of various objects by panoramic images, which eliminates the need to install a number of fixed cameras and significantly reduces the cost of ownership required for management.

1 is a view for explaining a smart farm growth management method of various entities by a panoramic image according to an embodiment.
2 is a view for explaining a smart farm growth management system of various entities by a panoramic image according to an embodiment.
3 is a flowchart illustrating a smart farm growth management method of various entities using a panoramic image according to an embodiment.
4 is a diagram illustrating a smart farm growth management system of various entities using a panoramic image according to an embodiment.
5 is a diagram for explaining a method of extracting a panoramic image from a moving picture according to an exemplary embodiment.
6 is a diagram illustrating an example of an analysis result of ROI characteristic information according to an embodiment.
7 is a diagram illustrating an example of a screen of a smart farm growth management system of various objects by a panoramic image according to an embodiment.
8 is a diagram for describing a method of obtaining a color outline according to an exemplary embodiment.
9 is a view for explaining patterning for a partial outline according to an exemplary embodiment.
10 is a diagram for describing partial matching according to an embodiment.
11 is a diagram illustrating a comparison of outlines according to colors according to an exemplary embodiment.

Hereinafter, embodiments will be described with reference to the accompanying drawings. However, the described embodiments may be modified in various other forms, and the scope of the present invention is not limited by the embodiments described below. In addition, various embodiments are provided in order to more completely explain the present invention to those of ordinary skill in the art. The shapes and sizes of elements in the drawings may be exaggerated for clearer description.

The following embodiments relate to a smart farm growth management system and method of various objects by a panoramic image, and periodically shoot a video using a camera moving in a certain path in a smart farm, and record the captured video. After constructing a panoramic image using panoramic image processing, a database is built for the efficiency of crop cultivation management in smart farms using panoramic image processing, and data is extracted for each part of interest, and the state by size, color, etc. of the extracted area is accumulated. Thus, the entire growth process can be efficiently managed.

1 is a view for explaining a smart farm growth management method of various entities by a panoramic image according to an embodiment.

Referring to FIG. 1 , a video 110 may be acquired by placing a camera at a predetermined position in a region to be analyzed for growth and moving the camera to a part that needs to be managed. When a panoramic image 120 is generated through the acquired video 110 and a part of a crop that the user wants to manage is designated as a region of interest (ROI), the characteristic information of the part designated as the region of interest (ROI) is displayed. The location may be memorized by finding a portion similar to the region of interest (ROI) in the panoramic image 120 that is the overall picture using the acquired feature information. As the image flows according to the primary, characteristic information such as color, texture, and shape of the region of interest (ROI) and the photographed location may be continuously accumulated, and information during the growth period may be accumulated.

Here, for the convenience of data collection for smart farm growth management, it can be taken as a video 110 . In the case of video, management is convenient, but it is difficult for humans to visually check or automate, and it is difficult to duplicate recorded videos or to understand repetitive information about the same area. Accordingly, the embodiments may use the panoramic image 120 that can grasp information with one glance.

As such, embodiments may have the advantages of the moving image 110 and the panoramic image 120 by generating the panoramic image 120 after shooting with the moving image 110 for smart farm growth management. In other words, the embodiments eliminate the need to install a large number of fixed cameras, which was required when image management for a large area is required through video recording, and can significantly reduce the cost of ownership required for management. In addition, if there is a certain path, you can always get a video in the same environment. In the embodiments, overlapping or repetitive information on the same area can be identified by using the panoramic image 120 , and information processing is easy.

2 is a view for explaining a smart farm growth management system of various entities by a panoramic image according to an embodiment.

Referring to FIG. 2 , the smart farm growth management system 200 of various entities using a panoramic image according to an embodiment includes a panoramic image generator (not shown), an ROI selector 210 , an ROI matching unit 220 , It may include a data storage unit 240 and a data analysis unit 250 . According to an embodiment, the smart farm growth management system of various entities using a panoramic image may further include an ROI management unit 230 .

The panoramic image generator may generate a panoramic image representing the entire image by using the video captured in the smart farm.

More specifically, the panoramic image generator may convert a video acquired through a camera into a panoramic image, and the panoramic image conversion may be performed through an image divided by frames in the video. In this case, the panoramic image can be completed by connecting the matching parts of the divided images based on the feature points obtained through the SIFT operation.

The ROI selector 210 may analyze characteristic information of a portion designated as a region of interest (ROI) by designating a portion of a crop to be managed by the user as a region of interest (ROI) in the panoramic image. The ROI selector 210 may analyze characteristic information of at least any one of a color, a texture, and a shape in the selected region of interest (ROI).

The ROI matching unit 220 may extract a portion matching the region of interest (ROI) from the panoramic image by using the feature information. That is, when a region of interest (ROI) for the region of interest is specified, the ROI matching unit 220 may extract color, texture, and shape information for the region of interest (ROI) and extract a matching portion from the panoramic image.

The ROI matching unit 220 selects a portion having characteristic information matching the region of interest (ROI) in the entire panoramic image as the region of interest is selected in the panoramic image, and registers the selected portion as a new region of interest (ROI). can In this way, as the information on the region of interest (ROI) is directly compared to the panoramic image, the prior information of the crop, which was previously required, is not required, and the information about the region of interest (ROI) is matched through color, texture, and shape, which are characteristic information of the region of interest (ROI). You can get the parts at the same time, so you can extract the part you want as a whole.

Also, the ROI manager 230 manages the ROI, and may add or delete the ROI. As described above, the ROI management unit 230 may improve the usefulness of the management system by performing editing such as correction and deletion of the region of interest (ROI) during the growth period.

The data storage unit 240 may accumulate feature information on the extracted part.

And, the data analysis unit 250 may check the change during the growth period through the analysis of the accumulated data. By accumulating and storing information on the region of interest (ROI) during the growth period and analyzing it, it is possible to confirm the change of the crop designated by the user. That is, growth information can be managed by accumulating data during growth. By continuously obtaining information on parts that match the region of interest (ROI) from the panoramic image, you can check how the color, texture, and shape information of each part changes, and statistical data and parts with large changes and parts with small changes can be identified. can be checked

According to the embodiments, through the smart farm growth management system of various objects by panoramic images, the cost of ownership according to the number of cameras required by the existing methods to manage a large area, a specific environment and points, and a database must be accumulated. It can solve the problem of the recognition method.

3 is a flowchart illustrating a smart farm growth management method of various entities using a panoramic image according to an embodiment.

Referring to FIG. 3 , in the smart farm growth management method of various objects by a panoramic image according to an embodiment, a step 310 of shooting a video using a camera in the smart farm, the entire image using the captured video Step 320 of generating a panoramic image representing a step 320 of designating the part of the crop that the user wants to manage in the panoramic image as a Region of Interest (ROI), the step 330 , the characteristic of the portion designated as the region of interest (ROI) The step of analyzing the information (340), the step of extracting a portion matching the region of interest (ROI) from the panoramic image using the characteristic information (350), the step of accumulating the characteristic information on the extracted portion (360), and A step 370 of confirming the change during the growth period through the analysis of the accumulated data may be included.

The method may further include managing a region of interest (ROI).

Hereinafter, each step of the smart farm growth management method of various entities by a panoramic image according to an embodiment will be described.

The smart farm growth management method of various entities using a panoramic image according to an embodiment may be described in more detail with an example of the smart farm growth management system of various entities using a panoramic image according to the embodiment described with reference to FIG. 2 . The smart farm growth management system of various entities using a panoramic image according to an embodiment may include a panoramic image generating unit, an ROI selecting unit, an ROI matching unit, a data storage unit, and a data analyzing unit. According to an embodiment, the smart farm growth management system of various objects by the panoramic image may further include an ROI management unit.

In step 310 , a video may be captured using a camera in the smart farm.

In operation 320 , the panoramic image generator may generate a panoramic image representing the entire image by using the captured video. The panoramic image generator may divide the image in frame units in the moving picture, and may complete the panoramic image by connecting matching parts of the divided images based on the feature points obtained through the SIFT operation.

In step 330 , the ROI selector may designate a part of a crop to be managed by the user in the panoramic image as a region of interest (ROI).

In operation 340 , the ROI selector may analyze characteristic information of a portion designated as a region of interest (ROI). The ROI selector may analyze characteristic information of at least any one of color, texture, and shape in the selected region of interest (ROI).

In operation 350 , the ROI matching unit may extract a portion matching the region of interest (ROI) from the panoramic image by using the feature information. As the ROI is selected from the panoramic image, the ROI matching unit may select a portion having characteristic information matching the ROI from the entire panoramic image, and may register the selected portion as a new ROI.

The ROI matching unit may extract a portion matching the ROI from the panoramic image by selecting at least one or more characteristic information of color, texture, and shape from the selected region of interest (ROI).

As an example, the ROI matching unit selects color feature information from the selected region of interest (ROI), extracts a portion that matches the region of interest (ROI) from the panoramic image, and analyzes the analysis result in the RGB color space and the HSV space. Through the result, matching parts can be obtained and displayed.

As another example, the ROI matching unit selects feature information about the texture from the selected region of interest (ROI), extracts a portion that matches the region of interest (ROI) from the panoramic image, and acquires it from the region of interest (ROI) using Color-LBP By analyzing the texture characteristics, it is possible to obtain and display parts that match the analysis result.

As another example, the ROI matching unit selects feature information about the shape from the selected region of interest (ROI), extracts a portion that matches the region of interest (ROI) from the panoramic image, finds the contour in the region of interest (ROI), and then matches the shape TCDS for shape matching can be used to find parts that match the shape of the region of interest (ROI).

In addition, color, texture, and shape can be combined if necessary to find a part that matches the region of interest (ROI).

Meanwhile, the method may further include managing a region of interest (ROI). The ROI manager can add or delete a region of interest (ROI), and it is possible to increase the usefulness of the management system by performing editing such as correction and deletion of the region of interest (ROI) during the growth period.

In operation 360 , the data storage unit may accumulate feature information on the extracted part. The data storage unit may continuously accumulate feature information or values for positions corresponding to the extracted region of interest (ROI) even if the panoramic image continuously changes according to the growth period.

In addition, the data storage unit may build a database during the growth period by storing characteristic information of at least any one of a color, a texture, and a shape of a region of interest (ROI) in the panoramic image.

In step 370 , the data analyzer may check the change during the growth period through the analysis of the accumulated data.

4 is a diagram illustrating a smart farm growth management system of various entities using a panoramic image according to an embodiment.

Referring to FIG. 4 , the smart farm growth management system 420 of various entities using a panoramic image according to an embodiment may efficiently manage the growth by receiving a video 410 captured by a camera. The smart farm growth management system 420 of various entities using a panoramic image according to an embodiment may include an analysis unit 430 and a management unit 440 . Here, the analysis unit 430 may include the ROI selection unit and the ROI matching unit described with reference to FIG. 2 , and the management unit 440 may include the ROI management unit, the data storage unit, and the data analysis unit described with reference to FIG. 2 .

The analysis unit 430 may include an image analysis area 431 , a color, texture, and shape collection area 432 , and a matching area 433 according to feature values. In the image analysis area 431 , the characteristic information of the ROI selected from the panoramic image can be analyzed, and the color, texture, and shape information can be collected by analyzing the characteristic information of the ROI in the color, texture and shape collection area 432 . . Also, it is possible to find a region having the same characteristic information as the characteristic information of the analyzed ROI in the matching region 433 according to the characteristic value.

The manager 440 may include an ROI management area 441 , a change checking area 442 during the growth period, and a color, texture, and shape checking area 443 between ROIs. The parts matching the extracted ROI can continuously accumulate features or values for the location even if the panoramic image continuously changes according to the growth period. A database during the growing season can be built by storing color, texture and shape characteristics. The value may be stored through the ROI management area 441 for each time and date. The ROI management area 441 facilitates database construction by storing position values of matching parts for each ROI series.

The stored database can be checked through the ROI management area 441 and the change confirmation area 442 during the growth period. The ROI management area 441 allows for comparison between ROI series and between matching portions of the ROI. Comparison between ROI series can confirm the difference in feature values at a specified date and time, and compare the difference during the growth period by selecting color, texture, and shape.

5 is a diagram for explaining a method of extracting a panoramic image from a moving picture according to an exemplary embodiment.

Referring to FIG. 5 , a moving picture acquired through a camera may be converted into a panoramic image, and the panoramic image conversion may be performed through an image divided into frames in a moving image. In this case, the panoramic image can be completed by connecting the matching parts of the divided images based on the feature points obtained through the SIFT operation.

A user can select a desired region of interest (ROI) from the panoramic image. When a region of interest is selected in one image, a region having the same characteristics is selected in the entire panoramic image, and the selected region is registered as a new region of interest. In this case, it is possible to add or delete ROI.

6 is a diagram illustrating an example of an analysis result of ROI characteristic information according to an embodiment.

Referring to FIG. 6 , a part matching the ROI can be found by selecting three types of color, texture, and shape of characteristic information from the selected ROI. As shown in (a) of FIG. 6 , the color may be displayed by obtaining matching parts through the analysis result in the RGB color space and the analysis result in the HSV space. As shown in (b) of FIG. 6 , in the case of texture, the texture characteristics obtained from the ROI part are analyzed using Color-LBP, and parts consistent with the analysis result can be obtained and displayed. As shown in (c) of FIG. 6 , in the case of a shape, it is possible to find a part that matches the shape of the ROI by using TCDs for shape matching after finding a contour in the ROI. In addition, color, texture, and shape can be combined if necessary to find a part that matches the ROI.

7 is a diagram illustrating an example of a screen of a smart farm growth management system of various entities by a panoramic image according to an embodiment.

Referring to FIG. 7 , the comparison between parts matching the ROI can confirm the overall change in color, texture, and shape during the growth period, and, if desired, the change trend during a specific period can be confirmed together with the panoramic image. The storage of color, texture, and shape values can be confirmed and made through the standards each has.

As an example, the color part of the feature values may be expressed based on the maximum ratio value of each color space, and may be expressed based on the average value of the color space. In addition, it is possible to collect images to visually check the color change for the ROI.

As another example, in the texture part of the feature value, the change during the growth period can be confirmed through graphs and drawings according to each numerical value. Also, if desired, you can check the change in the ratio value of each numerical value representing the texture in the entire ROI.

As another example, for the shape part of the feature values, the change of the contour part shown in the ROI can be checked as a picture, and the shape change can be confirmed with a graph through the numerical value. In addition, the relative size change during the growth period can be confirmed for the contours shown in the ROI.

When one or more ROIs are selected, the difference in feature values between the ROIs can be checked. It is possible to check each value according to the date of the growth period, and it can also be confirmed between the matching parts in the ROI and the panoramic image.

Hereinafter, an example of a method of finding an object using a partial color contour in a panoramic image will be described. In the general natural environment, the object of the image appears as several types of partial contours rather than as one distinct contour. Below, we propose a method of matching the partial contours shown in these images according to the partial contours and patterns of the object. Partial contours that may vary depending on color were considered, and the TCDs method was used for partial matching, and the combination and frequency of parts were used for pattern matching. Therefore, we show how to find a combined object through color and shape matching from partial contours.

When a similar color appears due to natural light or an external environment, the outline divided based on the color value does not appear clearly, and it is difficult to accurately obtain an edge feature for one region of interest in the image. To this end, methods have been proposed to obtain edge features of a region of interest by applying pre-processing techniques using various filters to an image, dividing them according to a large color space, and processing each. The contour matching technique is a method of designating consecutive points having the same color or intensity of an edge obtained from an image as the contour and matching using the difference between several contours. This contour matching field belongs to shape matching and is used to find and classify characteristic information about the shape of each object.

Existing object detection uses an edge detector to obtain the contour of a product from an image and performs contour matching to obtain shape features. Partial contour detection is a subject that is continuously studied in the field of contour detection to obtain a part that partially matches an obscured object. Partial contour matching aims to identify the matching part of two partial contours or to find the best match between a part of a closed contour and an open contour. TCDs are a method for partial shape matching proposed in 'A novel method for 2D nonrigid partial shape matching (2018)' and extract shape features using triangle-centroid-distance as a method for shape matching using triangle-centroid-distance. can do.

8 is a diagram for describing a method of obtaining a color outline according to an exemplary embodiment.

Referring to FIG. 8 , the image is separated according to color and the result of applying Sobel is shown. The image can be divided into R, G, and B spaces to obtain contours according to the color of the image according to R, G, and B, respectively. By applying a Sobel filter to this one-dimensional image, it is possible to obtain as many partial contours as possible.

9 is a view for explaining patterning for a partial outline according to an exemplary embodiment.

Referring to FIG. 9 , shape features of an object may be patterned to specify an object from partial contours obtained from an image. In order to determine a reference object, an area is defined, and the contour obtained within the area is divided into four, and values can be obtained through TCDs. These values become the pattern values for the shape characteristics of the object. Four classifications of patterns can be classified according to the starting point from 128 to a quarter of the sample points.

10 is a diagram for describing partial matching according to an embodiment.

Referring to FIG. 10 , it shows a partial point when the values of Local TCDs are changed due to the length ratio. As a pre-step of object detection, it is possible to match the values of each pattern and parts of the image. For partial matching, it is possible to compare the TCDs values of the contour of the image and the TCDs values of the pattern. When the local TCDs values are compared with the image contour, the length ratio is obtained through the length value of the local TCDs and the length value of the image contour, divided into 5 steps and multiplied by each ratio to consider the case where the part can be changed by the sample point.

11 is a diagram illustrating a comparison of outlines according to colors according to an exemplary embodiment.

As shown in FIG. 11 , it can be seen that the process of finding an outline after processing based on the main color that each object can have in order to detect an object in a natural environment helps to improve performance. Therefore, it can be confirmed through experiments that the outlines of fruits and crops in the smart farm environment are expressed differently through the main colors.

According to embodiments, there is provided a method for object detection using a partial contour in a natural environment. It is possible to find the partial contours of the R, G, and B spaces by considering the contours that can vary depending on the light and environment, and to express the local characteristics of agricultural products in the smart farm environment from the partial contours to Local TCDs. By defining Local TCDs, it is possible to associate the shape characteristics of an object with the location of the part. Labeling based on defined Local TCDs can find a part having object shape characteristics locally among matching parts as a result of partial matching.

In the above, when it is mentioned that a component is "connected" or "connected" to another component, it may be directly connected or connected to the other component, but other components may exist in the middle. It should be understood that there is On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle.

The terms used herein are used only to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present specification, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It is to be understood that it does not preclude the possibility of the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

In addition, terms such as “…unit” and “…module” described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software or a combination of hardware and software.

In addition, the components of the embodiment described with reference to each drawing are not limitedly applied only to the embodiment, and may be implemented to be included in other embodiments within the scope of maintaining the technical spirit of the present invention, and also Even if the description is omitted, it is natural that a plurality of embodiments may be re-implemented as a single integrated embodiment.

In addition, in the description with reference to the accompanying drawings, the same components regardless of the reference numerals are given the same or related reference numerals, and the overlapping description thereof will be omitted. In describing the present invention, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted.

As described above, although the embodiments have been described with reference to the limited embodiments and drawings, various modifications and variations are possible by those skilled in the art from the above description. For example, the described techniques are performed in a different order than the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (15)

In the smart farm growth management method using the smart farm growth management system,
capturing a video by the panoramic image generator of the smart farm growth management system using a camera in the smart farm;
generating a panoramic image representing an entire image by using the video captured by the panoramic image generator;
designating, by an ROI selector, a portion of a crop to be managed by a user in the panoramic image as a Region of Interest (ROI);
analyzing, by the ROI selector, characteristic information of a portion designated as the region of interest (ROI);
extracting, by an ROI matching unit, a portion matching the region of interest (ROI) from the panoramic image by using the feature information;
accumulating, by a data storage unit, feature information about the extracted part; and
A step of confirming the change during the growth period through the analysis of the accumulated data by the data analysis unit
including,
The step of generating a panoramic image representing the entire image by using the video includes:
Divides the image in frame units in the video, and completes the panoramic image by connecting the matching parts of the divided image based on the feature points obtained through the SIFT operation,
The step of extracting a portion that matches the region of interest (ROI) from the panoramic image by using the feature information includes:
As a region of interest is selected in the panoramic image, a portion having characteristic information matching the region of interest (ROI) is selected from the entire panoramic image, and the selected portion is registered as a new region of interest (ROI), but the selected region of interest Selecting at least any one or more characteristic information of color, texture, and shape from the region (ROI) and extracting a portion that matches the region of interest (ROI) from the panoramic image
characterized in that, a smart farm growth management method. delete According to claim 1,
The step of analyzing the characteristic information of the portion designated as the region of interest (ROI) comprises:
Analyze the characteristic information of at least any one of color, texture, and shape in the selected region of interest (ROI)
characterized in that, a smart farm growth management method. delete delete According to claim 1,
The step of extracting a portion that matches the region of interest (ROI) from the panoramic image by using the feature information includes:
Select the color feature information from the selected region of interest (ROI) to extract the part that matches the region of interest (ROI) from the panoramic image, and through the analysis result in the RGB color space and the analysis result in the HSV space Obtaining and displaying matching parts
characterized in that, a smart farm growth management method. According to claim 1,
The step of extracting a portion that matches the region of interest (ROI) from the panoramic image by using the feature information includes:
A texture obtained from the region of interest (ROI) using Color-LBP is extracted from the panoramic image by selecting feature information about a texture in the selected region of interest (ROI). Obtaining and displaying parts that are consistent with the analysis result by analyzing the characteristics
characterized in that, a smart farm growth management method. According to claim 1,
The step of extracting a portion that matches the region of interest (ROI) from the panoramic image by using the feature information includes:
A portion matching the ROI is extracted from the panoramic image by selecting feature information on a shape from the selected region of interest (ROI), and an outline is found in the region of interest (ROI) and then shape matching ) to find a part that matches the shape of the region of interest (ROI)
characterized in that, a smart farm growth management method. According to claim 1,
ROI manager managing the region of interest (ROI)
further comprising,
Managing the region of interest (ROI) includes:
adding or deleting the region of interest (ROI)
characterized in that, a smart farm growth management method. According to claim 1,
The step of accumulating the feature information on the extracted part is,
Continuously accumulating feature information or values about the location of the extracted portions coincident with the region of interest (ROI) even if the panoramic image is continuously changed according to the growth period.
characterized in that, a smart farm growth management method. According to claim 1,
The step of accumulating the feature information on the extracted part is,
To build a database during the growth period by storing at least any one or more characteristic information of color, texture, and shape of the region of interest (ROI) in the panoramic image
characterized in that, a smart farm growth management method. a panoramic image generating unit for generating a panoramic image representing the entire image by using the video captured in the smart farm;
an ROI selection unit that analyzes characteristic information of a portion designated as a region of interest (ROI) in response to designating a portion of a crop that a user wants to manage in the panoramic image as a region of interest (ROI);
an ROI matching unit that extracts a portion matching the region of interest (ROI) from the panoramic image by using the feature information;
a data storage unit for accumulating the extracted feature information on the part; and
Data analysis unit to check changes during the growth period through analysis of accumulated data
including,
The panoramic image generating unit,
Divides the image in frame units in the video, and completes the panoramic image by connecting the matching parts of the divided image based on the feature points obtained through the SIFT operation,
The ROI matching unit,
As a region of interest is selected in the panoramic image, a portion having characteristic information matching the region of interest (ROI) is selected from the entire panoramic image, and the selected portion is registered as a new region of interest (ROI), but the selected region of interest Extracting a portion matching the region of interest (ROI) from the panoramic image by selecting at least one or more characteristic information of color, texture, and shape in the region (ROI)
characterized in that, a smart farm growth management system. 13. The method of claim 12,
The ROI selection unit,
Analyze the characteristic information of at least any one of color, texture, and shape in the selected region of interest (ROI)
characterized in that, a smart farm growth management system. delete 13. The method of claim 12,
ROI management unit for managing the region of interest (ROI)
further comprising,
The ROI management unit,
adding or deleting the region of interest (ROI)
characterized in that, a smart farm growth management system.

Images