KR20230038941A - Crop monitering apparatus - Google Patents

Abstract

Provided is a crop monitoring device that monitors the growth status of crops by moving between beds that are installed in multiple rows at a predetermined height and provide growth space for crops. The crop monitoring device comprises: a moving unit movably installed on at least one of the ground between the beds and an upper part of the bed to move along the bed; a photographing unit installed on the moving unit to provide a first image and a second image while photographing crops; at least one lifting unit installed on the moving unit to support the photographing unit so as to be able to lift the photographing unit and adjust the height of the photographing unit; a memory for storing photographing status information of the photographing unit; and a control unit for determining the photographing status information of the photographing unit based on the first image, controlling the moving unit, the photographing unit, and the lifting unit to photograph a target object among parts of crops based on the photographing status information, and generating diagnostic information based on the second image photographing the target object. Accordingly, a crop monitoring device that can accurately diagnose the growth status of crops can be provided.

Description

Crop monitoring device {CROP MONITERING APPARATUS}

Embodiments disclosed herein relate to a crop monitoring device, and more particularly, to a crop monitoring device capable of monitoring the growth state of a crop by photographing the crop while moving between beds of the crop.

In recent years, information and communication technology has been grafted onto agricultural production/processing/distribution/consumption to remotely and automatically manage crop growth environments and increase production efficiency. Smart farm technology is being developed.

Specifically, smart farm is related to a farm that can remotely and automatically maintain and manage the growth environment of crops or livestock appropriately by grafting information communication technology to existing farming technologies such as vinyl houses and barns. refers to technology By using smart farms, it is possible to improve the productivity and quality of agricultural products by creating an optimal growth environment based on data on crop growth and environmental information.

Here, it is important to observe the growth point of the crop in order to accurately diagnose the growth state of the crop.

However, since the height of the medium constituting the growing space of crops is not uniform, each photographed image is different from each other, and the position of the growing point changes as the crop grows, so the camera for photographing the crops has to be installed individually according to the height of the growing point. There is a romance.

As a related prior art for the implementation of smart farms, there is an autonomous driving unmanned pest control vehicle spraying device disclosed in Korean Patent Laid-open Publication No. 10-2020-0140953. There is a problem in that only the configuration for spraying the chemical solution is disclosed, so that it cannot be photographed according to the height of the growing point of the crop.

Therefore, a technique for solving the above problems has been required.

On the other hand, the above-mentioned background art is technical information that the inventor possessed for derivation of the present invention or acquired in the process of derivation of the present invention, and cannot necessarily be said to be known art disclosed to the general public prior to filing the present invention. .

Embodiments disclosed herein are intended to provide a crop monitoring device capable of accurately diagnosing the growth state of crops by photographing crops at a height corresponding to the growing point of the crop.

Specifically, the embodiments disclosed herein present a crop monitoring device capable of photographing crops while elevating the photographing unit to a height corresponding to the growing point of the crop through the elevation of the lifting unit supporting the photographing unit. There is a purpose.

In addition, the embodiments disclosed in this specification are aimed at presenting a crop monitoring device capable of photographing the growing point of a crop while moving the ground between beds or the top of the bed in which the crop grows.

In addition, the embodiments disclosed herein are aimed at presenting a crop monitoring device capable of photographing the growing point of a crop while moving in a longitudinal direction of the bed as well as a direction transverse to the bed when moving the top of the bed.

As a technical means for achieving the above-described technical problem, according to an embodiment of the crop monitoring device, while moving between beds installed at a predetermined height while forming a plurality of rows to provide a growth space for crops, the growth state of crops is monitored. A crop monitoring device for monitoring, comprising: a moving unit movably installed on at least one of a ground between the beds and an upper portion of the beds and moving along the beds; a photographing unit installed in the moving unit to provide a first image and a second image while photographing crops; At least one elevating unit installed in the moving unit to adjust the height of the recording unit while supporting the recording unit so as to be able to move up and down; a memory for storing photographing state information of the photographing unit; and determining photographing state information of the photographing unit based on the first image, and controlling the moving unit, the photographing unit, and the elevating unit to photograph a target object among parts of a crop based on the photographing state information; A control unit generating diagnostic information based on the second image of the target object may be included.

According to any one of the above-mentioned problem solving means, it is possible to present a crop monitoring device capable of accurately diagnosing the growth state of a crop by photographing the crop at a height corresponding to the growth point of the crop.

According to any one of the above-described problem solving means, it is possible to present a crop monitoring device capable of photographing crops while elevating the photographing unit to a height corresponding to the growing point of the crop through the elevation of the lifting unit supporting the photographing unit. there is.

According to any one of the above-described problem solving means, it is possible to present a crop monitoring device capable of photographing the growing point of a crop while moving the upper part of the bed or the ground between the beds in which the crop grows.

According to any one of the above-described problem solving means, when moving the top of the bed, it is possible to present a crop monitoring device capable of photographing the growing point of the crop while moving in the longitudinal direction of the bed as well as in the direction crossing the bed.

Effects obtainable from the disclosed embodiments are not limited to those mentioned above, and other effects not mentioned are clear to those skilled in the art from the description below to which the disclosed embodiments belong. will be understandable.

1 is a configuration diagram showing a state in which a crop monitoring device according to an embodiment is installed on the ground.
Figure 2 is a configuration diagram showing a state in which the crop monitoring device according to an embodiment is installed on top of the bed.
Figure 3 is a block diagram showing the configuration of a crop monitoring device according to an embodiment.
Figure 4 is a configuration diagram showing a state in which the movable rail of the crop monitoring device according to an embodiment is installed on top of the bed.
5 is a flowchart illustrating a crop monitoring method by a crop monitoring device according to an embodiment.

Hereinafter, various embodiments will be described in detail with reference to the accompanying drawings. Embodiments described below may be modified and implemented in various different forms. In order to more clearly describe the characteristics of the embodiments, detailed descriptions of matters widely known to those skilled in the art to which the following embodiments belong are omitted. And, in the drawings, parts irrelevant to the description of the embodiments are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a component is said to be “connected” to another component, this includes not only the case of being “directly connected” but also the case of being “connected with another component intervening therebetween”. In addition, when a certain component "includes" a certain component, this means that other components may be further included without excluding other components unless otherwise specified.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

However, prior to explaining this, the meanings of the terms used below are first defined.

'Crops' refers not only to plants grown on farms, but also to forest plants and greenhouse plants, and may be, for example, food crops, craft crops, fodder crops, fertilizer crops, and horticultural crops.

The 'target object' refers to an object for growth diagnosis or pest detection, and may be fruits, fruits, flowers, leaves, etc. of crops on a farm.

1 is a configuration diagram showing a state in which a crop monitoring device according to an embodiment is installed on the ground, Figure 2 is a configuration diagram showing a state in which the crop monitoring device according to an embodiment is installed on an upper portion of a bed, and FIG. It is a block diagram showing the configuration of the crop monitoring device according to the embodiment. In addition, Figure 4 is a configuration diagram showing a state in which the moving rail of the crop monitoring device according to an embodiment is installed on top of the bed, Figure 5 is a flow chart showing a crop monitoring method by the crop monitoring device according to an embodiment.

As shown in FIG. 1, the crop monitoring device 10 according to an embodiment is a device for observing the growth state of crops by photographing them while moving between beds 1 providing a growth space for crops. .

Here, the bed 1 may be installed at a predetermined height in a configuration in which nutrients are provided for the growth of crops and crops not shown are planted and grown, and may be continuous in the longitudinal direction while forming a plurality of rows.

Specifically, the crop monitoring device 10 according to an embodiment includes a moving unit 100, a photographing unit 200, a lifting unit 300, a memory 400 and a control unit as shown in FIGS. 1 to 3 ( 500) may be configured.

The moving unit 100 is a component that provides an installation space for components described later, and moves the ground between the beds 1 as shown in FIG. 1 or moves the bed 1 as shown in FIG. It is a component that moves along the longitudinal direction of the bed 1 while moving the top.

As shown in FIGS. 1 and 2 , the moving unit 100 may include a robot 110 and a moving rail 120 .

The robot 110 moves the photographing unit 200 and the lifting unit 300 to be described later while traveling on the ground between the beds 1 or the upper part of the bed 1 through autonomous driving or remote control. ) can be moved along the length direction of

The robot 110 may drive while operating under the control of a control unit 400 to be described later.

Here, the robot 110 may move through rotation of a plurality of wheels operated by the control unit 500 or travel through rotation of an endless track, but is not limited thereto, and all configurations capable of traveling by remote control are can be applied

The movement rail 120 is a component for providing a movement path of the robot 110 described above.

These movable rails 120 may be installed along the longitudinal direction of the ground between the beds 1 or installed on top of the beds 1.

Here, the moving rail 120 may be omitted when the robot 110 has a configuration in which the robot 110 moves along the ground between the beds 1.

Meanwhile, when the robot 110 is configured to move the top of the bed 1, the moving rail 120 may be installed between the beds 1 along the longitudinal direction, as shown in FIG.

Accordingly, the robot 110 is installed in a suspended state on the movable rail 120 installed above the bed 1 and can move along the longitudinal direction of the movable rail 120 .

The photographing unit 200 is a component for photographing crops of the bed 1 while moving with the robot 110 in a state installed on the robot 110 .

The photographing unit 200 may be installed in the robot 110 via the elevating unit 300 to be described later, and may perform photographing while being adjusted to a predetermined height.

Specifically, according to the embodiment, the photographing unit 200 is composed of a PTZ (Pan Tilt Zoom) camera so that the rotation, angle, and magnification of the lens can be remotely controlled by the controller 500, and the image of the subject is received. After receiving an optical signal, converting it into an electrical signal, and generating an image signal from the converted signal, an image may be generated.

Accordingly, the photographing unit 200 generates images by photographing the crops of the beds 1 disposed on both sides of the robot 110 respectively by performing shooting while rotating between the two rows of beds 1 through rotation. can do.

In addition, the recording unit 200 is equipped with an auto-focusing function can automatically focus on the subject. According to the embodiment, the photographing unit 200 may be implemented as an EO (Electro-Optical) sensor (electro-optical sensor) or an IR (infrared) sensor (infrared ray sensor), and accordingly within the field of view (FOV). A subject can be photographed and a photographed image to be analyzed can be created. In addition, as an image of a scene to be photographed within the angle of view, the photographing unit 200 may transmit a temporary image representing an object within the angle of view to the controller 500 .

The lifting unit 300 is installed on the robot 110 constituting the moving unit 100 and is a component supporting the shooting unit 200 so as to be able to move up and down, while operating under the control of the controller 500, the shooting unit ( 200) is a component for adjusting the height.

That is, the elevating unit 300 is a component for elevating the photographing unit 200 to a height corresponding to a target object, such as a growth point of crops planted on the bed 1.

As shown in FIG. 1, the lifting unit 300 may include a telescopic rod 310 and a length adjusting member 320.

The telescopic rod 310 is a component whose length expands and contracts while connecting the robot 110 constituting the moving unit 100 and the photographing unit 200.

Specifically, the telescopic rod 310 is composed of a rod of a telescopic structure divided into a plurality of rods, the lower end is fixed to the robot 110, the recording unit 200 can be installed on the upper end, and the length is increased through the withdrawal of the unit rods. It is possible to elevate the photographing unit 200 while the length is reduced through the introduction of the increased or unit rods.

The length adjusting member 320 is a component for adjusting the recording unit 200 to a predetermined height by adjusting the length of the telescopic rod 310 .

The length adjusting member 320 may adjust the length of the telescopic rod 310 to a predetermined length while operating based on the photographing state information of the photographing unit 200 determined by the controller 500 to be described later.

Specifically, the length adjusting member 320 may include a rack gear 321 and a pinion gear 322 as shown in FIG.

The rack gear 321 may be formed on a portion of the telescopic rod 310 in a sawtooth shape along the longitudinal direction.

The pinion gear 322 is a component for linear motion of the rack gear 321, and is a component for moving the rack gear 321 forward and backward while forward or reverse rotation while being engaged with the rack gear 321.

The pinion gear 322 may be connected to a drive motor (not shown) that rotates under the control of the control unit 500, and the telescopic rod 310 moves forward while the rack gear 321 moves forward through forward rotation by the control unit 500. The length of the telescopic rod 310 may be reduced while extending the length of the rack or moving the rack gear 321 backward in the direction of the robot 110 through reverse rotation.

On the other hand, the length adjusting member 320 is composed of a hydraulic pump or a pneumatic pump and operates under the control of the control unit 500 to supply hydraulic pressure or pneumatic pressure so that the length of the telescopic rod 310 can be expanded or contracted.

The memory 400 is a component that stores photographing state information of the photographing unit 200 determined by the controller 500 to be described later.

In addition, various types of data such as files, applications, and programs may be installed and stored in the memory 400 .

Meanwhile, the aforementioned photographing unit 200 may provide a first image and a second image to the controller 500 while photographing crops under the control of the controller 500 .

Hereinafter, for convenience of description, an image corresponding to a target object occupying an area of a predetermined size or more within the entire area is referred to as a 'second image', and other images are referred to as a 'first image'.

The control unit 500 determines the photographing state information of the photographing unit 200 based on the first image and, based on this, moves the moving unit 100 and the photographing unit 200 to photograph a target object such as a growing point among parts of a crop. ) and a component that controls the lifting unit 300, and also a component that generates and provides diagnostic information on crops based on the second image of the target object.

The control unit 500 controls the overall operation of the crop monitoring device 10 and may include a processor such as a CPU or GPU. Also, a program stored in the memory 400 may be executed, a file stored in the memory 400 may be read, or a new file may be stored in the memory 400 .

Specifically, the controller 500 may identify one or more target candidate objects in the first image.

For example, if the image generated by the photographing unit 200 is a photograph of the entire crop, the control unit 500 may have a plurality of growing points of the crop, which is a target object, in the photographed image. Therefore, the controller 500 can identify one or more target candidate objects estimated as target objects in the acquired image.

To this end, the control unit 500 uses artificial intelligence implemented with an object identification model advantageous for target object detection, such as a YOU ONLY LOOK ONCE (YOLO) model, a REGION BASED CONVOLUTIONAL NETWORK (R-CNN) model, or a SINGLE SHOT MULTIBOX DETECTOR (SSD) model. A neural network may be trained, and a target candidate object in the first image may be identified based on the learned artificial neural network.

For example, when the growing point of a crop is a target object, the control unit 500 inputs a first image obtained by photographing the crop to the learned artificial neural network to identify the growing point in the first image and places the identified location in a bounding box. By processing with , the target candidate object can be extracted.

Accordingly, the controller 500 may determine the location coordinates of target candidate objects and the number of target candidate objects in the first image.

For example, if the target object candidate area in the first image is divided into bounding boxes, the control unit 500 can extract the center point of the bounding box or one vertex of the bounding box as location coordinates, and also counts the number of bounding boxes. Thus, the number of target candidate objects can be determined.

According to an embodiment, if there are a plurality of target candidate objects, the controller 500 may select one of the plurality of target candidate objects and determine the selected target candidate object as the target object in order to acquire the second image.

And, if diagnostic information about the selected target candidate object is generated, the controller 500 may select another target candidate object identified in the first image as the target object, and the controller 500 may select all target candidate objects in the first image. The above process may be repeated until diagnostic information is generated.

According to another embodiment, if there is one target candidate object, the control unit 500 may determine the one target candidate object as the target object.

The controller 500 may determine the photographing state information based on the current photographing state information, the current distance information, and the positional coordinates of the target object in the first image.

To this end, the control unit 500 may determine the location coordinates of the target object.

According to the embodiment, the control unit 500 calculates the coordinates of the center point of the bounding box surrounding the target candidate object in order to identify the target object in the first image, and determines the coordinates of the center point as the location coordinates of the target object, or The coordinates of one vertex of can be determined as the location coordinates of the target object.

According to another embodiment, the control unit 500 may determine the coordinates of a position having a predetermined pixel value among target objects as the position coordinates of the target object. For example, the brightest pixel value in an area determined as the target object The location of the target object can be determined by the location coordinates of the target object.

In addition, the controller 500 may obtain current photographing state information. At this time, the current photographing state information may be acquired from the memory 400 updating the most recent photographing state information.

Here, 'capturing state information' may include at least one of a pan angle, a tilt angle, and a zoom magnification. Also, the photographing state information may further include a focusing value. For example, the control unit 500 may move the photographing unit 200 up, down, left, or right according to the photographing state information, adjust the zoom magnification of the lens, or adjust the focusing so that a far/near subject may be appropriately photographed. there is.

Also, the photographing state information may additionally include information about the environment at the time of photographing, and may include, for example, information such as illuminance, temperature, humidity, and wind speed.

Since the center coordinates of the first image are the pan and tilt angles of the photographing device looking at the subject, the controller 500 may obtain the corresponding pan and tilt angles as current photographing state information.

In addition, the control unit 500 may obtain current distance information, which may be obtained from the distance measurement unit 130 according to an embodiment. Therefore, for example, the distance to the subject corresponding to the center coordinates of the first image may be obtained by measuring the distance measuring unit 130 .

As one embodiment, the distance measuring unit 130 may measure the distance from the photographing unit 200 to the target object based on the principle of time of flight (TOF). For example, the distance measurement unit 130 may calculate the distance by emitting a laser having excellent directivity to a target object to be measured and then measuring the time the laser is reflected from the target object and returns.

This distance measurement unit 130 may be implemented as, for example, a laser distance finder or a depth camera. However, the method of measuring the distance to the target object by the distance measurement unit 130 is not limited to the above-described example.

Therefore, according to the embodiment, the control unit 500 calculates the coordinate difference between the center coordinates of the first image and the location coordinates of the target object, and calculates the pan angle and tilt angle for capturing the location coordinates based on the current distance information can do. Also, photographing state information may be generated by reflecting the calculated pan angle and tilt angle to the current photographing state information.

Accordingly, the control unit 500 can control the moving unit 100 and the lifting unit 300 to operate, and accordingly, the target object can be positioned within the angle of view of the photographing unit 200 .

In addition, when controlling the lifting unit 300, the control unit 500 may control the lifting unit 300 based on the bed 1 while identifying the bed 1 as a target object in the first image.

For example, the controller 500 may control the lifting unit 300 based on the boundary of the bed 1 identified as the target object, and set the height of the recording unit 200 to the center of the boundary of the bed 1. It is possible to control the lifting unit 300 to position.

In addition, the control unit 500 may update the photographing state information so that the photographing unit 200 more precisely photographs the target object after the moving unit 100 moves based on the photographing state information.

Depending on the embodiment, the control unit 500 may determine a focusing value so that the photographing unit 200 can photograph the target object.

For example, the controller 500 may calculate a focusing value for focusing based on the distance to the target object, and for this, the distance measurement unit 130 may operate the distance from the photographing unit 200 to the target object can be measured.

Also, for example, the controller 500 may operate an auto-focusing function provided in the photographing unit 200 and store the focusing value determined accordingly in the memory 400 .

According to another embodiment, the control unit 500 may set a zoom magnification so that an image area related to a target object occupies a predetermined ratio or more within the entire image area in an image within an angle of view seen through the photographing unit 200 .

To this end, for example, the controller 500 may determine a zoom magnification such that an image area related to the target object occupies a predetermined ratio or more within the entire image area by analyzing a pixel ratio. That is, the control unit 500 may adjust the zoom magnification so that the number of pixels corresponding to the target object occupies a predetermined number or more among the total number of pixels of the image.

To this end, the controller 500 may count pixels of an area inferred as a target object within the subject image within the field of view. For example, pixels having a color corresponding to the target object may be determined as pixels in an area estimated to be the target object, and the controller 500 may count pixels in the area estimated to be the target object.

Also, for example, the controller 500 may calculate the area occupied by the target object and adjust the zoom magnification so that the area of the target object occupies a predetermined ratio or more in the area of the image. Therefore, when the target object is a tangerine, the ratio of the tangerine image to the entire image can be calculated by measuring the horizontal ratio of the tangerine, which is a circular fruit, and calculating the area occupied by the tangerine in the image by calculating the area of the circle. there is.

Alternatively, for example, the controller 500 may adjust the zoom magnification so that the image area of the target object occupies a predetermined ratio or more within the entire image area by identifying the boundary of the target object and calculating the size of the target object.

Alternatively, for example, the controller 500 may identify the boundary of the target object and adjust the zoom magnification so that the distance between the boundary of the target object and the boundary of the image is within a predetermined distance.

When the focusing value and the zoom magnification are calculated as described above, the controller 500 may update the photographing state information. That is, the controller 500 may control the photographing unit 200 to photograph a subject according to the calculated focusing value and zoom magnification.

Accordingly, the controller 500 may acquire the second image including the target object.

Further, the controller 500 may generate at least one of growth information and pest information of the target object as diagnosis information by analyzing the second image.

In relation to this, the diagnostic information may also include information about the positional coordinates of the target object in the space where the moving unit 100 or the photographing unit 200 is located, and for example, when generating the diagnostic information, the moving unit 100 ) Or the GPS information of the recording unit 200 or the mobile unit 100 or the recording unit 200 may include information about the section where it is located. In relation to this, the section information may be, for example, information for identifying the growth facility of the crop or the location of the specific bed 1, and specifically, a crop in which one or more fruits (or flowers, leaves, or stems) are present. It may be identification information or identification information of the bed 1 in which crops are located.

The controller 500 may identify the target object in the second image by separating the target object and the background in order to generate diagnosis information.

For example, the controller 500 may separate the target object located within the edge and the background located outside the edge by identifying the edge of the target object based on the deep learning model.

The controller 500 may generate growth information by analyzing the second image.

At this time, the growth information is information about how much the target object has grown to determine the harvestable time when the target object is, for example, a fruit, and may include, for example, information such as variety, size, and coloration, and also size and coloration. Based on , information on which period of the growth period is also included.

According to the embodiment, the controller 500 may measure the size of the target object.

The controller 500 may calculate the horizontal and vertical lengths of the actual target object based on the horizontal and vertical lengths of the target object within the edge of the second image and the distance to the target object. The size of the target object can be determined based on the calculated horizontal and vertical lengths.

According to the embodiment, the controller 500 may measure the degree of coloring according to the type and color of the target object.

For example, the controller 500 may measure the coloration degree by measuring the degree of red color when the variety of the target object is strawberry.

The controller 500 may generate the size and coloration measured as described above as growth information.

Also, the controller 500 may determine the growth stage of the target object based on the variety, size, and coloration of the target object. Therefore, for example, if the target object is sufficiently grown and is to be harvested, the harvest time may be added to the growth information.

On the other hand, the control unit 500 may analyze the second image to generate pest information of the target object.

To this end, the controller 500 may train a deep learning model to output pest areas by inputting an image including a possible appearance of a target object for each species, pest damage, and germ (or virus). The deep learning model at this time may be a CONVOLUTIONAL NEURAL NETWORK (CNN) or a RECURRENT NEURAL NETWORK (RNN). In addition, a second image may be input to the learned deep learning model, and at this time, a pest area may be detected and displayed in an area corresponding to the target object by inputting the second image in which the edge is identified.

In addition, the controller 500 may input an image including a shape that a target object for each breed and each pest may have, and train a deep learning model to identify pests. The deep learning model at this time may be a CONVOLUTIONAL NEURAL NETWORK (CNN) or a RECURRENT NEURAL NETWORK (RNN). In addition, the second image may be input to the learned deep learning model, and at this time, the location of pests may be detected and displayed, or the number or type of pests may be detected and displayed.

In addition, the control unit 500 may verify diagnostic information.

In the case of crops, when pests exist or have diseases, they are often found in a group rather than a single individual, so when a pest is detected as diagnostic information for one target object, the control unit 500 detects another target object The diagnostic information can be verified by comparing the diagnostic information about the .

For example, if diagnostic information indicating that a pest has been found for one target object is generated, the control unit 500 determines that the generated diagnostic information is incorrectly generated if no pest information is included in the diagnostic information for another target object. After determining, the analysis of the second image may be performed again.

At this time, another target object may be located within a predetermined range of the target object to be verified. For example, another target object may be a target object captured in the same image as the target object to be verified. Alternatively, for example, another target object may be a target object located within a predetermined range based on location information included in the diagnosis information.

Meanwhile, various types of data such as files, applications, and programs may be installed and stored in the memory 400 .

At this time, the controller 500 may access and use the data stored in the memory 400 or may store new data in the memory 400 . Also, the controller 500 may execute a program installed in the memory 400 . For example, the memory 400 may be installed with a program for performing a photographing method by controlling the moving unit 100, the photographing unit 200, and the elevating unit 300, or may store photographing state information. . The memory 400 may also store initial photographing state information in order to return the pan angle or tilt angle of the photographing device to the origin, and may also store updated photographing state information when the photographing state information is updated.

Meanwhile, the controller 500 may perform wired/wireless communication with other devices or networks through the communication unit 550 .

For example, the communication unit 550 may communicate with an input/output device (not shown) or an electronic terminal (not shown) to transmit/receive various types of information for photographing crops. To this end, the communication unit 550 may include a communication module supporting at least one of various wired/wireless communication methods. For example, the communication module may be implemented in the form of a chipset.

For example, wireless communication supported by the communication unit 550 includes, for example, Wi-Fi (Wireless Fidelity), Wi-Fi Direct, Bluetooth, UWB (Ultra Wide Band) or NFC (Near Field Communication), It may be 2G to 5G communication technology or more communication technology. In addition, wired communication supported by the communication unit 550 may be, for example, USB, Ethernet, or HDMI (High Definition Multimedia Interface).

According to the embodiment, the communication unit 550 may transmit diagnostic information generated by the control unit 500 to an external device. At this time, the external device is, for example, an input/output device (not shown), or, for example, a fruit collector, a sprinkler in a greenhouse, an air conditioner, a heater, a lighting device, a door opener, a fertilizer dispenser, a pesticide sprayer, etc. Or, for example, it may be a marking device for displaying fruits with pests.

According to another embodiment, the communication unit 550 may transmit diagnosis information generated by the control unit 500 to the server. The server may be implemented as a computer capable of communicating with the controller 500 through a network or implemented as a cloud computing server. In addition, the server may include a storage device capable of storing data or may store data through a third server. Also, the server may communicate with the administrator's terminal.

In addition, the crop monitoring device 10 according to an embodiment may additionally include an input/output unit (not shown).

The input/output unit (not shown) may include an input unit for receiving an input from a manager and an output unit for displaying information such as a job performance result or a state of the photographing device 100 . For example, the input/output unit (not shown) may include an operation panel that receives an input from a manager who operates a farm, and a display panel that displays a screen.

Specifically, the input unit may include devices capable of receiving various types of inputs, such as a keyboard, a physical button, a touch screen, or a microphone. Also, the output unit may include a display panel or a speaker. However, it is not limited thereto, and the input/output unit (not shown) may include a configuration supporting various inputs and outputs.

According to the embodiment, the input unit may receive a manipulation for shooting from a manager. In addition, the output unit may output and provide diagnostic information so that a manager can check the diagnostic information, or output an alarm or warning sound corresponding to the diagnostic information.

This input/output unit (not shown) may be implemented as an input/output device (not shown), which is a separate device, to communicate with the crop monitoring device 10 through the communication unit 550 and may be located outside the crop monitoring device 10.

On the other hand, when the robot 110 is configured to move the top of the bed 1, the above-described moving rail 120 includes a longitudinal rail 121 and a transverse rail 122 as shown in FIG. can be configured.

The longitudinal rail 121 is installed on the upper part of the bed 1 and extends between the beds 1 to provide a movement path of the robot 110 in the longitudinal direction of the bed 1, that is, in the longitudinal direction.

The transverse rail 122 is installed on the top of the bed 1 and extends perpendicularly to the longitudinal rail 121, so that the moving path of the robot 110 is changed in the transverse direction of the bed 1, that is, the bed 1 It can be provided in a transversal direction.

Accordingly, the photographing unit 200 may perform photographing of the crops of the bed 1 disposed on both sides of the longitudinal rail 121 while moving along the longitudinal rail 121 together with the robot 110, While moving along the transverse rail 122, it is possible to move to the adjacent longitudinal rail 121 to perform photographing of crops.

Here, the recording unit 200 may collide with the bed 1 in the process of moving because it must cross the bed 1 when moving along the transverse rail 122 .

At this time, the above-described distance measuring unit 130 may measure the distance between the recording unit 200 and the bed 1 and apply the distance to the control unit 500, and the control unit 500 may measure the distance from the distance measuring unit 130 Based on the data, the lifting unit 300 may be operated to raise the photographing unit 200 .

Accordingly, the recording unit 200 can be moved while achieving a predetermined interval without colliding with the bed (1) in the process of moving the transverse rail (122).

On the other hand, at least one of the longitudinal rail 121 and the transverse rail 122 may be extended while forming an arc shape having a predetermined curvature.

That is, the longitudinal rail 121 or the transverse rail 122 does not extend in a straight line but in an arc shape so that the robot 110 can go through the plurality of beds 1 at various angles in the process of moving. there is.

On the other hand, Figure 5 is a flow chart for explaining a photographing method by the crop monitoring device 10 according to an embodiment. The photographing method shown in FIG. 5 includes steps of time-sequential processing in the crop monitoring device 10 described with reference to FIGS. 1 to 4 . Therefore, even if the content is omitted below, the information described above with respect to the crop monitoring device 10 shown in FIGS. 1 to 4 can also be used in the photographing method according to the embodiment shown in FIG. 5 .

As shown in FIG. 5 , the crop monitoring device 10 may obtain a first image (S100).

At this time, the control unit 500 controls the moving unit 100 to move the recording unit 200 along the ground between the bed 1 or the top of the bed 1 together with the moving unit 100, and then the recording unit A first image may be acquired while photographing the crops planted on the bed 1 by controlling 100 .

Having acquired the first image, the control unit 500 can identify a target candidate object in the first image (S200).

To this end, the controller 500 may identify a target candidate object by learning an artificial neural network implemented as a YOLO model or an R-CNN model and inputting a first image to the learned artificial neural network.

In addition, the control unit 500 may select one of the target candidate objects identified as described above and determine it as the target object (S300).

In this case, the control unit 500 may sequentially select target candidate objects in the x-axis direction based on the lowermost left vertex of the first image based on the center point of the bounding box surrounding each target candidate object. Accordingly, the control unit 500 may first sequentially select target candidate objects, determine them as target objects, and sequentially generate diagnostic information about the target candidate objects.

To capture the target object determined in this way, the control unit 500 may adjust the height of the recording unit 200 to a height corresponding to the target object (S400).

At this time. The control unit 500 controls the elevation unit 300 to elevate the photographing unit 200 while elevating the photographing unit 200 to the coordinates corresponding to the center point of the bounding box of the target object so that the photographing unit 2000 corresponds to the target object. can be moved to any height.

In addition, the control unit 500 may move the height of the photographing unit 200 to be positioned at the center of the boundary of the bed 1 while controlling the lifting unit 300 by identifying the boundary of the bed 1 as a target object.

Also, the control unit 500 may determine photographing state information of the photographing unit 200 (S500).

The control unit 500 may determine the photographing state information based on the current photographing state information of the photographing unit 200, the current distance information, and the positional coordinates of the target object in the first image.

According to an embodiment, the controller 500 may generate photographing state information by calculating a pan angle and a tilt angle based on current photographing state information, current distance information, and positional coordinates of a target object in the first image. there is.

Meanwhile, the controller 500 may update the photographing state information when the target object is photographed in a small size due to a low zoom factor or when the target object is photographed in a low resolution due to inconsistent focusing values (S600).

According to the embodiment, the control unit 500 may set a zoom factor so that an image area related to a target object occupies a predetermined ratio or more within the entire image area in an image within an angle of view seen through the lens of the photographing unit 200 .

The controller 500 may obtain a second image by tilting or focusing the photographing unit 200 based on the photographing state information or by photographing a target object by adjusting a zoom magnification (S700).

At this time, the control unit 500 selects a target object within a photographed image or a focused image.

If the ratio occupied by the identified area in the entire image is not greater than a predetermined ratio, the zoom magnification of the photographing unit 200 may be adjusted so that the area identified as the target object occupies a predetermined ratio or more in the entire image.

When the second image is obtained by photographing the target object in this way, the controller 500 may generate diagnostic information based on the second image (S800).

In this case, the controller 500 may separate the target object located within the edge and the background located outside the edge by identifying the edge of the target object in the second image based on the deep learning model.

Further, the control unit 500 may generate diagnostic information including at least one of growth information and pest information by analyzing the image of the target object.

For example, the control unit 500 may generate pest information from an image, and may generate diagnostic information including related information by identifying a pest area, which is a location where pests burrow. For example, by analyzing the number of pest areas, the number of pests, and pest or pest areas, diagnosis information including information on pests or information on pathogens can be generated.

Steps S300 to S800 described above may be repeated until there is no target candidate object for which diagnostic information is not generated among the target candidate objects identified in the first image (S900).

When diagnostic information on the first image is generated by the above-described photographing method, the crop monitoring device 10 may obtain the first image by photographing another subject.

At this time, if it is determined that there is an area where the obtained first image overlaps with the previously analyzed first image, the controller 500 updates the photographing state information to obtain the first image by photographing at a different pan angle or tilt angle. can do. Alternatively, based on the location information and the pan and tilt angles of the photographing unit 200 from which the first image was obtained, it may be determined that the obtained first image is a photograph of the same subject at the same position as the previously analyzed first image. Accordingly, the controller 500 may acquire the first image by updating the photographing state information and photographing at a different pan angle or tilt angle.

The photographing method described above may be implemented in the form of a computer-readable medium storing instructions and data executable by a computer. In this case, instructions and data may be stored in the form of program codes, and when executed by a processor, a predetermined program module may be generated to perform a predetermined operation. Also, computer-readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, a computer-readable medium may be a computer recording medium, which is a volatile and non-volatile memory implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. It can include both volatile, removable and non-removable media. For example, the computer recording medium may be a magnetic storage medium such as HDD and SSD, an optical recording medium such as CD, DVD, and Blu-ray disc, or a memory included in a server accessible through a network.

The photographing method described above may be implemented as a computer program (or computer program product) including instructions executable by a computer. A computer program includes programmable machine instructions processed by a processor and may be implemented in a high-level programming language, object-oriented programming language, assembly language, or machine language. . Also, the computer program may be recorded on a tangible computer-readable recording medium (eg, a memory, a hard disk, a magnetic/optical medium, or a solid-state drive (SSD)).

The photographing method described above may be implemented by executing the above-described computer program by a computing device. A computing device may include at least some of a processor, a memory, a storage device, a high-speed interface connected to the memory and a high-speed expansion port, and a low-speed interface connected to a low-speed bus and a storage device. Each of these components are connected to each other using various buses and may be mounted on a common motherboard or mounted in any other suitable manner.

Here, the processor may process commands within the computing device, for example, to display graphic information for providing a GUI (Graphic User Interface) on an external input/output device, such as a display connected to a high-speed interface. Examples include instructions stored in memory or storage devices. As another example, multiple processors and/or multiple buses may be used along with multiple memories and memory types as appropriate. Also, the processor may be implemented as a chipset comprising chips including a plurality of independent analog and/or digital processors.

Memory also stores information within the computing device. In one example, the memory may consist of a volatile memory unit or a collection thereof. As another example, the memory may be composed of a non-volatile memory unit or a collection thereof. Memory may also be another form of computer readable medium, such as, for example, a magnetic or optical disk.

Also, the storage device may provide a large amount of storage space to the computing device. A storage device may be a computer-readable medium or a component that includes such a medium, and may include, for example, devices in a storage area network (SAN) or other components, such as a floppy disk device, a hard disk device, an optical disk device, or a tape device, flash memory, or other semiconductor memory device or device array of the like.

The term '~unit' used in the above embodiments means software or a hardware component such as a field programmable gate array (FPGA) or ASIC, and '~unit' performs certain roles. However, '~ part' is not limited to software or hardware. '~bu' may be configured to be in an addressable storage medium and may be configured to reproduce one or more processors. Therefore, as an example, '~unit' refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, and procedures. , subroutines, segments of program patent code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables.

Functions provided within components and '~units' may be combined into smaller numbers of components and '~units' or separated from additional components and '~units'.

In addition, components and '~units' may be implemented to play one or more CPUs in a device or a secure multimedia card. The above-described embodiments are for illustrative purposes, and those skilled in the art to which the above-described embodiments belong can easily transform into other specific forms without changing the technical spirit or essential features of the above-described embodiments. You will understand. Therefore, it should be understood that the above-described embodiments are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope to be protected through this specification is indicated by the following claims rather than the detailed description above, and should be construed to include all changes or modifications derived from the meaning and scope of the claims and equivalent concepts thereof. .

10: crop monitoring device
100: mobile unit
110: robot
120: moving rail
121: longitudinal rail
122: transverse rail
130: distance measuring unit
200: shooting unit
300: lifting unit
310: telescopic rod
320: length adjustment member
321: rack gear
322: pinion gear
400: memory
500: control unit
550: Ministry of Communication

Claims (12)

In the crop monitoring device for monitoring the growth state of crops while moving between beds installed at a predetermined height while forming a plurality of rows to provide a growth space for crops,
a moving unit movably installed on at least one of the upper part of the bed and the ground between the beds to move along the bed;
a photographing unit installed in the moving unit to provide a first image and a second image while photographing crops;
At least one elevating unit installed in the moving unit to adjust the height of the recording unit while supporting the recording unit so as to be able to move up and down;
a memory for storing photographing state information of the photographing unit; and
determining photographing state information of the photographing unit based on the first image, and controlling the moving unit, the photographing unit, and the elevating unit to photograph a target object among parts of a crop based on the photographing state information; Crop monitoring device comprising a control unit for generating diagnostic information based on the second image of the target object.
According to claim 1,
The mobile unit,
A robot disposed on one of the upper part of the bed and the ground between the beds while providing an installation space for the shooting unit and the lifting unit and moving the shooting unit and the lifting unit while moving under the control of the control unit. crop monitoring device.
According to claim 2,
The mobile unit,
Crop monitoring device further comprising a moving rail installed on one of the upper part of the bed and the ground between the beds to provide a moving path of the robot.
According to claim 3,
The moving rail,
Longitudinal rails respectively extending along the gap between the beds while being installed on the top of the bed; and
Crop monitoring device comprising a transverse rail installed on top of the bed while forming a state orthogonal to the longitudinal rail to allow the robot to move in a direction crossing the bed.
According to claim 4,
At least one of the longitudinal rail and the transverse rail,
Extending while forming an arc shape having a predetermined curvature, crop monitoring device.
According to claim 4,
The mobile unit,
It is operated by the control unit in a state installed on the robot, and when the robot moves to the transverse rail, detects the distance between the recording unit and the bed and provides it to the control unit while maintaining the distance between the recording unit and the bed. Crop monitoring device further comprising a measuring unit.
According to claim 1,
The lifting unit,
A telescopic rod having a predetermined length and having a telescopic structure so as to be stretchable and connecting the moving unit and the recording unit; and
Crop monitoring device comprising a length adjusting member for adjusting the length of the telescopic rod to a length corresponding to the photographing state information while operating under the control of the control unit.
According to claim 7,
The length adjusting member,
A toothed rack gear formed along a longitudinal direction of a portion of the telescopic rod; and
A crop monitoring device comprising a pinion gear for linearly moving the rack gear while being engaged with the rack gear and rotating forward or reverse by the control unit.
According to claim 1,
The control unit,
Identifying the bed in the first image as the target object, and controlling the height of the photographing unit while controlling the lifting unit based on the boundary of the bed, crop monitoring device.
According to claim 1,
The control unit,
Crop monitoring device for identifying the target object based on the first image, and determining the photographing state information of the photographing unit based on the position of the target object in the first image.
According to claim 1,
The control unit,
and updating the photographing state information so that an image corresponding to the target object occupies an area of a predetermined size or more within an entire area of the second image.
According to claim 1,
The control unit,
A crop monitoring device for generating diagnostic information on at least one of growth information and pest information on the target object.

Images