KR20220109990A - Apparatus and method for photographing - Google Patents

Abstract

A photographing device and a photographing method are presented, and according to an embodiment, the photographing device includes: a memory for storing photographing state information of the photographing device; and a control unit for determining the photographing state information of the photographing device based on a first image, controlling the photographing device to photograph a target object based on the photographing state information, and generating diagnosis information based on a second image obtained by photographing the target object.

Description

Filming device and method

The embodiments disclosed herein relate to a photographing apparatus and a photographing method, and more particularly, to an apparatus and a method for performing photographing for the implementation of a smart farm.

Recently, interest in implementing a smart farm is increasing.

Smart farm is an agricultural system that not only increases efficiency but also convenience of production by enabling automatic management to properly maintain the growth environment of crops, livestock and aquatic products.

For example, in order to implement a smart farm for growing crops, it is necessary to diagnose the growth of crops or automatically detect pests and pests. There is this.

In addition, for example, in order to implement a smart farm for raising livestock or aquatic products, it is necessary to automatically detect the growth diagnosis or disease of livestock (or aquatic products), but in reality, it is difficult to diagnose growth or detect diseases. There is a problem in that it is difficult to build the system.

In relation to this, a prior art document, Republic of Korea Patent Registration No. 10-1681891, discloses a smart farm operation system that can measure the soil, air, and water environment inside the plastic house and manage it in a server, but the cultivation environment of crops There is a limitation in not diagnosing the growth of crops itself or diagnosing diseases and pests because it only analyzes the crops.

Therefore, there is a need for a technique for solving the above-mentioned problems.

On the other hand, the above-mentioned background art is technical information that the inventor possessed for the derivation of the present invention or acquired in the process of derivation of the present invention, and it cannot be said that it is necessarily a known technique disclosed to the general public before the filing of the present invention. .

Embodiments disclosed in this specification are for the purpose of providing a photographing apparatus and a photographing method.

In addition, embodiments disclosed in the present specification, it is an object of the present invention to present a photographing apparatus and method for diagnosing the growth of crops or detecting pests.

In addition, the embodiments disclosed in the present specification aim to provide a photographing apparatus and method capable of accurately performing the growth of crops or detection of pests and pests.

In addition, the embodiments disclosed herein aim to provide a photographing apparatus and method capable of performing growth diagnosis of crops or detection of pests with minimal cost and resources.

In addition, embodiments disclosed in the present specification, it is an object of the present invention to provide a photographing apparatus and method capable of generating diagnostic information on livestock and marine products with minimal cost and resources.

As a technical means for achieving the above-described technical problem, according to an embodiment described in the present specification, in the photographing apparatus, a memory for storing photographing state information of the photographing apparatus, and a first image of the photographing apparatus are and a controller configured to determine photographing state information, control the photographing apparatus to photograph a target object based on the photographing state information, and generate diagnostic information based on a second image obtained by photographing the target object.

In addition, as a technical means for achieving the above-described technical problem, according to an embodiment described in the present specification, a method of photographing by a photographing apparatus, comprising: obtaining a first image; determining photographing state information, controlling the photographing apparatus to photograph a target object based on the photographing state information, and generating diagnostic information based on a second image photographing the target object can

According to one of the above-described problem solving means, an object of the present invention is to provide a photographing apparatus and a photographing method.

According to one of the above-described problem solving means, it is possible to present a photographing apparatus and an imaging method for diagnosing the growth of crops or detecting pests and diseases. Accordingly, it is possible to automatically control irrigation/regulation/control/feeding according to the diagnosed growth information and pest information.

According to one of the above-described problem solving means, it is possible to provide a photographing apparatus and an imaging method capable of accurately performing the growth of crops or detection of pests.

According to one of the above-described problem solving means, it is possible to provide an imaging apparatus and an imaging method capable of performing the growth diagnosis of crops or the detection of pests with minimal cost and resources. Accordingly, it is possible to accurately diagnose the growth of crops or detect pests and diseases without having a high-resolution camera.

According to one of the above-described problem solving means, it is possible to present a photographing apparatus and a photographing method capable of generating diagnostic information on livestock and aquatic products with minimal cost and resources.

Effects obtainable in the disclosed embodiments are not limited to the above-mentioned effects, and other effects not mentioned are clear to those of ordinary skill in the art to which the embodiments disclosed from the description below belong. will be able to be understood

1 is a configuration diagram showing the configuration of a photographing apparatus according to an embodiment disclosed in the present specification.
2 is a block diagram illustrating a configuration of a photographing apparatus according to an exemplary embodiment disclosed in the present specification.
3 to 4 are exemplary views illustrating a photographing system in which a photographing apparatus according to an embodiment disclosed in the present specification is installed.
5 is a flowchart illustrating a photographing method according to an exemplary embodiment.
6 to 9 are exemplary views for explaining a photographing method according to an embodiment disclosed in the present specification.

Hereinafter, various embodiments will be described in detail with reference to the accompanying drawings. The embodiments described below may be modified and implemented in various different forms. In order to more clearly describe the characteristics of the embodiments, detailed descriptions of matters widely known to those of ordinary skill in the art to which the following embodiments belong are omitted. And, in the drawings, parts not related to the description of the embodiments are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a component is said to be "connected" with another component, it includes not only a case of 'directly connected' but also a case of 'connected with another component interposed therebetween'. In addition, when a component "includes" a component, it means that other components may be further included, rather than excluding other components, unless otherwise stated.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

However, before explaining this, the meaning of the terms used below is first defined.

'Target object' means a target for growth diagnosis or pest detection, and may be fruits, berries, flowers, leaves, etc. on farms can

The term 'crop' refers not only to plants grown on farms, but also to forest plants or plants in greenhouses, and may include, for example, food crops, craft crops, fodder crops, fertilizer crops, and horticultural crops.

In addition, terms that require explanation will be separately explained below.

1 is a block diagram showing the configuration of a photographing apparatus according to an embodiment disclosed in the present specification, and FIG. 2 is a block diagram illustrating the configuration of a photographing apparatus according to an embodiment disclosed in the present specification.

The photographing apparatus 100 provides diagnostic information about the target object by photographing the target object.

The photographing apparatus 100 may include a lens unit 210 , a distance measurement unit 220 , a control unit 230 , a rotation driving unit 240 , a holding unit 250 , a memory 260 , and a communication unit 270 . .

The lens unit 210 may receive an image of a subject, receive an optical signal, convert it into an electrical signal, and then generate an image by generating the converted signal as an image signal.

The lens unit 100 is equipped with a zoom (ZOOM) function to enable long-distance/short-distance shooting.

In addition, since the lens unit 100 is equipped with an auto-focusing function, it is possible to automatically focus on a subject.

According to an embodiment, the lens unit 210 may be implemented as an EO (Electro-Optical) sensor (electro-optical sensor) or an IR (infrared) sensor (infrared sensor), and accordingly, within a field of view (FOV). A target can be photographed and a photographed image to be analyzed can be created. Also, the lens unit 210 may transmit a temporary image representing an object within the angle of view to the controller 230 as an image related to a scene to be photographed within the angle of view.

In relation to this, hereinafter, both the photographed image and the temporary image are referred to as 'images'.

Meanwhile, the distance measuring unit 220 may measure a distance between the target object and the photographing device.

According to an embodiment, the distance measuring unit 220 may measure the distance from the photographing device to the target object based on the TOF (TIME OF FLIGHT) principle. For example, the distance measuring unit 220 may calculate the distance by emitting a laser having excellent directivity to the target object to be measured distance, and measuring the time when the laser is reflected from the target object and returned.

The distance measurement unit 220 may be implemented as, for example, a laser rangefinder or a depth camera.

However, the method by which the distance measuring unit 220 measures the distance to the target object is not limited to the above-described example.

On the other hand, the controller 230 controls the overall operation of the photographing apparatus 100 and may include a processor such as a CPU or GPU. The controller 230 may control other components included in the photographing apparatus 100 to perform an operation according to the photographing method disclosed herein. For example, the controller 230 may execute a program stored in the memory 260 to be described later, read a file stored in the memory 260 , or store a new file in the memory 260 .

According to an embodiment, the controller 230 may acquire an image from the lens unit 210 .

In relation to this, for convenience of explanation, an image in which an image corresponding to a target object occupies an area of a predetermined size or more within the entire area is referred to as a 'second image', and other images are referred to as a 'first image'.

The controller 230 may identify one or more target candidate objects from the first image.

For example, if the image generated by the lens unit 210 is a photograph of the entire apple tree, a plurality of apples, which are target objects, may exist in the photographed image. Accordingly, the controller 230 may identify one or more target candidate objects estimated to be target objects in the acquired image.

To this end, the control unit 230 is an artificial object identification model implemented as an object identification model advantageous for detecting a target object, such as a YOU ONLY LOOK ONCE (YOLO) model, a REGION BASED CONVOLUTIONAL NEURAL NETWORK (R-CNN) model, or a SINGLE SHOT MULTIBOX DETECTOR (SSD) model. A neural network may be trained, and a target candidate object in the first image may be identified based on the learned artificial neural network.

For example, if an apple in an apple tree in an orchard is a target object, the controller 230 inputs the first image of the apple tree to the trained artificial neural network to identify the apple in the first image and identify the identified position. By treating as a bounding box, the target candidate object can be extracted.

Or, for example, if the pig is the target object in the pig room, the control unit 230 inputs the first image photographed in the pig room into the learned artificial neural network to identify the pig in the first image, and set the identified position in the bounding box. By processing with , the target candidate object can be extracted.

Accordingly, the controller 230 may determine the position coordinates of the target candidate object and the number of target candidate objects in the first image.

For example, if the target object candidate region in the first image is divided into a bounding box, the control unit 230 may extract a center point of the bounding box or a vertex of the bounding box as a position coordinate, and also counting the number of bounding boxes Thus, the number of target candidate objects can be determined.

According to an embodiment, if there are a plurality of target candidate objects, the controller 230 may select one of the plurality of target candidate objects to obtain the second image and determine the selected target candidate object as the target object.

And, if diagnostic information on the selected target candidate object is generated, the controller 230 may select another target candidate object identified in the first image as the target object, The above process may be repeated until diagnostic information is generated.

According to another embodiment, if there is only one target candidate object, the controller 230 may determine the target candidate object as the target object.

The controller 230 may determine the photographing state information based on the current photographing state information, the current distance information, and the location coordinates of the target object in the first image.

To this end, the controller 230 may determine the location coordinates of the target object.

According to the embodiment, the control unit 230 calculates the coordinates of the center point of the bounding box surrounding the target candidate object for identification of the target object in the first image and determines the coordinates of the center point as the location coordinates of the target object, or the bounding box It is possible to determine the coordinates of one vertex of the target object as the position coordinates of the target object.

According to another embodiment, the controller 230 may determine the coordinates of a position having a predetermined pixel value among the target objects as the position coordinates of the target object. The position it has can be determined as the position coordinates of the target object.

Also, the control unit 230 may obtain current photographing state information. In this case, the current photographing state information may be acquired from the memory 260 that is updating the latest photographing state information.

In this case, the 'photographing state information' includes at least one of a pan angle, a tilt angle, and a zoom magnification. In addition, the photographing state information may further include a focusing value. For example, according to the photographing state information, the control unit 230 moves the photographing apparatus 100 up, down, left and right, adjusts the zoom magnification of the lens unit 210, or adjusts the focusing to cause the lens unit 210 to make the long-distance/short-distance It is possible to properly photograph the subject of

In addition, the photographing state information may further include information about the environment at the time of photographing, for example, information such as illuminance, temperature, humidity, and wind speed.

Since the central coordinates of the first image are the pan angle and the tilt angle of the photographing apparatus looking at the subject, the controller 230 may obtain the corresponding pan angle and the tilt angle as current photographing state information.

Also, the control unit 230 may obtain current distance information, and according to an embodiment, may obtain it from the distance measurement unit 220 . Therefore, for example, the distance to the subject corresponding to the center coordinate of the first image may be obtained by measuring the distance measuring unit 220 .

Therefore, according to the embodiment, the control unit 230 calculates the coordinate difference between the position coordinates of the target object from the central coordinates of the first image, and calculates the pan angle and the tilt angle for photographing the position coordinates based on the current distance information. can do. In addition, by reflecting the calculated pan angle and tilt angle in the current photographing state information, it is possible to generate photographing state information.

Accordingly, the controller 230 may control the rotation driving unit 240 to move the photographing apparatus 100, and accordingly, the target object may be located within the angle of view.

In addition, the controller 230 may update the photographing state information so that the lens unit 210 captures the target object more precisely after the photographing apparatus 100 moves based on the photographing state information.

According to an embodiment, a focusing value may be determined so that the lens unit 210 may photograph the target object.

For example, the controller 230 may calculate a focusing value for focusing based on the distance to the target object, and for this purpose, the controller 230 may cause the distance measuring unit 220 to measure the distance from the photographing device to the target object. can do.

Also, for example, the control unit 230 may operate an auto-focusing function provided in the lens unit 210 and store the determined focusing value in the memory 260 .

According to another exemplary embodiment, in the image within the angle of view viewed through the lens unit 210 , the controller 230 may determine the zoom magnification so that the image area of the target object occupies a predetermined ratio or more within the entire image area.

To this end, for example, the controller 230 may determine the zoom magnification so that the image area of the target object occupies a predetermined ratio or more within the entire image area by analyzing the pixel ratio. That is, the controller 230 may adjust the zoom magnification so that the number of pixels corresponding to the target object occupies a predetermined number or more among the total number of pixels of the image.

To this end, the controller 230 may count pixels in an area estimated to be a target object in the subject image within the angle of view. For example, a pixel of a color corresponding to the target object may be determined as a pixel of an area estimated to be the target object, and the controller 230 may count pixels of the area estimated to be the target object.

Also, for example, the controller 230 may calculate the area occupied by the target object and adjust the zoom magnification so that the area of the target object occupies a predetermined ratio or more in the area of the image. Therefore, when the target object is a mandarin orange, it is possible to calculate the ratio of the orange image to the total image by measuring the horizontal ratio of the orange, the fruit of the circle, and calculating the area occupied by the orange in the image by calculating the area of the circle. have.

Alternatively, for example, the controller 230 may adjust the zoom magnification so that the image area of the target object occupies a predetermined ratio or more within the entire image area by identifying the boundary of the target object and calculating the size of the target object.

Alternatively, for example, the controller 230 may identify the boundary of the target object and adjust the zoom magnification so that the distance between the boundary of the target object and the boundary of the image is within a predetermined distance.

When the focusing value and the zoom magnification are calculated as described above, the controller 230 may update the photographing state information. That is, the controller 230 may control the lens unit 210 to photograph the subject according to the calculated focusing value and zoom magnification.

Accordingly, the controller 230 may acquire the second image including the target object.

The controller 230 may generate at least one of growth information and pest information of the target object as diagnostic information by analyzing the second image.

In relation to this, the diagnostic information may also include information on the position coordinates of the target object in the space in which the imaging device 100 is located, for example, gps information or imaging of the imaging device 100 when generating diagnostic information. It may include information about the compartment in which the device 100 is located, and the like. In this regard, the section information may be, for example, information for identifying a pig farm, a pig house, a barn, a cage, a tank, and the like, or may be information for identifying a single or more growth measurement crop. For example, as section information, it may be pig farm identification information in which one or more pigs are present as target objects, or, for example, crop identification information or crops in which one or more fruits (or flowers, or leaves or stems) exist. It may be identification information of the compartment in which it is located.

To generate the diagnostic information, the controller 230 may identify the target object in the second image by separating the target object and the background.

For example, the controller 230 may separate the target object located within the edge and the background located outside the edge by identifying the edge of the target object based on the deep learning model.

The controller 230 may analyze the second image to generate growth information.

In this case, the growth information is information about how much the target object has grown to determine the harvestable time when it is a fruit, for example, and may include, for example, information such as variety, size, and coloration, and also size and coloration. It may also include information on which time of the growth period is based on the.

According to an embodiment, the controller 230 may measure the size of the target object.

The controller 230 may calculate the actual horizontal and vertical lengths of the target object based on the horizontal and vertical lengths of the target object within the edge in the second image and the distance to the target object. The size of the target object can be determined based on the calculated horizontal and vertical lengths.

According to an embodiment, the controller 230 may measure the coloration degree according to the type of the target object and the color of the target object.

For example, when the variety of the target object is strawberry, the controller 230 may measure the degree of coloration by measuring the degree of red.

The controller 230 may generate the size and coloration measured as described above as growth information.

Also, the controller 230 may determine the growth stage of the target object based on the type, size, and coloration of the target object. Therefore, for example, if the target object is sufficiently grown and it is time to harvest, the harvest time may be added to the growth information.

On the other hand, the controller 230 may analyze the second image to generate pest information of the target object.

To this end, the controller 230 may train the deep learning model to output the pest area by inputting an image including the appearance of the target object for each breed, pest damage, and germ (or virus). The deep learning model at this time may be a CONVOLUTIONAL NEURAL NETWORK (CNN), a RECURRENT NEURAL NETWORK (RNN), or the like. In addition, a second image may be input to the learned deep learning model, and at this time, a second image with an identified edge may be input to detect and display the pest area in the area corresponding to the target object.

In addition, the controller 230 may train the deep learning model to identify the pest by inputting an image including the appearance of the target object for each breed and each pest. The deep learning model at this time may be a CONVOLUTIONAL NEURAL NETWORK (CNN), a RECURRENT NEURAL NETWORK (RNN), or the like. And the second image may be input to the learned deep learning model, and at this time, the position of the pest may be detected and displayed, or the number or type of the pest may be detected and displayed.

Additionally, if the target object is an animal, the controller 230 may generate diagnostic information to include fever information.

To this end, when the second image is obtained, the controller 230 generates body temperature information corresponding to a second image obtained from a body temperature sensor (not shown) to be described later as diagnostic information, or a target included in the second image. If an animal, which is an object, is not at normal body temperature, information indicating that it has a disease state may be generated as diagnostic information.

In addition, the control unit 230 may verify the diagnosis information.

In the case of crops, when pests exist or have diseases, they are usually found not only in one individual but in a cluster, so when a pest is detected as diagnostic information for one target object, the controller 230 controls the other target object. The diagnostic information can be verified by comparing the diagnostic information about

For example, if diagnostic information indicating that a pest has been found for one target object is generated, the control unit 230 determines that the generated diagnostic information is erroneously generated if information on pests is not included in the diagnostic information on another target object. It may be determined and the analysis of the second image may be performed again.

In this case, the other target object may be located within a predetermined range from the target object to be verified. For example, the other target object may be a target object that was captured in the same image as the target object to be verified. Alternatively, for example, the other target object may be a target object located within a predetermined range based on location information included in the diagnosis information.

Meanwhile, if the controller 230 analyzes the acquired image and determines that there is a foreign substance or moisture exists in the lens, the controller 230 may cause the holding unit 250 to remove the foreign substance or moisture.

In this case, the controller 230 may determine that there is a foreign material on the surface of the lens unit 210 based on at least one of the acquired first image and the second image. For example, the same target object at a predetermined time interval. By comparing two images acquired by photographing in chronological order, if there is a change in the later time image compared to the previous time image, it can be determined that there is a foreign substance.

On the other hand, the rotation driving unit 240 may rotate the photographing apparatus 100 left and right or vertically.

Also, the rotation driving unit 240 may return to the origin when the operation of the photographing apparatus 100 starts or the operation of the photographing apparatus 100 ends according to the initial photographing state information.

The rotation driving unit 240 may be implemented as a motor, and may move the entire photographing apparatus 100 according to a pan angle and a tilt angle under the control of the controller 230 .

In addition, the rotation driving unit 240 may rotate the entire photographing apparatus 100 by being positioned at the lowermost or uppermost end of the photographing apparatus 100 , for example.

Meanwhile, the holding unit 250 may maintain the environment of the photographing apparatus 100 so that the lens unit 210 of the photographing apparatus 100 may photograph a subject.

According to an embodiment, the holding unit 250 may remove foreign substances existing on the lens unit 210 .

In order for the lens unit 210 to photograph a subject, foreign substances should not exist on the surface of the lens unit 210, and since the photographing device 100 is often exposed to the outside such as a barn or an orchard, foreign substances such as soil or water droplets may exist.

For example, the holding unit 250 may be implemented as a wiper 251 and move left and right to wipe off raindrops or dew located on the glass surface constituting the lens unit 210 . The holding unit 250 may be positioned at the lower end of the lens unit 210 to remove foreign substances adhering to the surface of the lens unit 210 .

According to another embodiment, the holding unit 250 may also remove moisture from the lens unit 210 .

For example, the holding unit 250 may be implemented as a hot wire to remove moisture from the surface of the lens unit 210 . The holding unit 250 may be included in the lens unit 210 to remove moisture from the lens unit 210 .

Meanwhile, various types of data such as files, applications, and programs may be installed and stored in the memory 260 . The controller 230 may access and use data stored in the memory 260 , or may store new data in the memory 260 . Also, the controller 230 may execute a program installed in the memory 260 . For example, the memory 260 may be installed with a program for performing a photographing method, or may store photographing state information. The memory 260 may also store initial photographing state information to return the pan angle or tilt angle of the photographing apparatus to the origin, and may also store the updated photographing state information when the photographing state information is updated.

Meanwhile, the communication unit 270 may perform wired/wireless communication with other devices or networks. For example, the communication unit 270 may communicate with an input/output device (not shown) or an electronic terminal (not shown) to transmit/receive various types of information for performing a photographing method.

To this end, the communication unit 270 may include a communication module supporting at least one of various wired and wireless communication methods. For example, the communication module may be implemented in the form of a chipset.

For example, the wireless communication supported by the communication unit 270 is, for example, Wi-Fi (Wireless Fidelity), Wi-Fi Direct, Bluetooth (Bluetooth), UWB (Ultra Wide Band) or NFC (Near Field Communication), It may be 2G to 5G communication technology or higher communication technology. In addition, the wired communication supported by the communication unit 110 may be, for example, USB, Ethernet, or High Definition Multimedia Interface (HDMI).

According to an embodiment, the communication unit 270 may transmit the diagnostic information generated by the photographing apparatus 100 to an external device. In this case, the external device is, for example, an input/output device (not shown) to be described later, or, for example, a fruit collector, a feed feeder that provides feed, or a sprinkler, air conditioner, It may be a heater, a lighting device, a door opener, a fertilizer injector, a pesticide sprayer, or the like, or, for example, a marking device for displaying fruits or animals in which pests or pests exist.

According to another embodiment, the communication unit 270 may transmit the diagnostic information generated by the photographing apparatus 100 to the server. The server may be implemented as a computer capable of communicating with the imaging device 100 through a network or may be implemented as a cloud computing server. In addition, the server may include a storage device capable of storing data or may store data through a third server. In addition, the server may communicate with the terminal of the manager.

Meanwhile, the photographing apparatus 100 may further include a body temperature sensing unit (not shown).

When the photographing apparatus 100 is located in a livestock barn or a farm, it may further include a body temperature sensing unit (not shown) to measure the body temperature of an animal, which is a target object.

Such a body temperature sensing unit (not shown) may be implemented as, for example, a thermal imaging camera or an infrared temperature sensor, and may be located in the photographing apparatus 100 so as to have the same axis as the axis of the lens unit 210 . .

In addition, the photographing apparatus 100 may further include an input/output unit (not shown).

The input/output unit (not shown) may include an input unit for receiving an input from a manager and an output unit for displaying information such as a result of a job or the state of the photographing apparatus 100 . For example, the input/output unit (not shown) may include an operation panel that receives an input from a manager who operates a farm, a livestock bar, or a farm, and a display panel that displays a screen.

Specifically, the input unit may include devices capable of receiving various types of inputs, such as a keyboard, a physical button, a touch screen, or a microphone. In addition, the output unit may include a display panel or a speaker. However, the present invention is not limited thereto, and the input/output unit (not shown) may include a configuration supporting various input/output.

According to an embodiment, the input unit may receive a manipulation for performing photographing from a manager. In addition, the output unit may output and provide diagnostic information so that an administrator can check the diagnostic information, or output an alarm or a warning sound corresponding to the diagnostic information.

The input/output unit (not shown) may be implemented as an input/output device (not shown), which is a separate device, to communicate with the photographing apparatus 100 through the communication unit 170 , and may be located outside the photographing apparatus 100 .

The photographing apparatus 100 according to the above description may be fixedly installed in a place where an object to be photographed exists or may be installed in a movable apparatus.

3 to 4 are exemplary views showing a photographing system in which the photographing apparatus 100 according to an embodiment disclosed in the present specification is installed.

As shown in FIG. 3 , the photographing apparatus 100 may be installed in a movable photographing system 300 .

The imaging system 300 may include an environmental sensor 310 , a body 320 , a pillar 330 , an arm 340 , and a driving device 350 . And the imaging system 300 may additionally include a marking device (not shown).

The environmental sensor 310 is a sensor for detecting an environment in which photographing is performed, and may include, for example, a temperature sensor, a humidity sensor, a carbon dioxide measurement sensor, and the like. It is possible to measure the environment of the livestock.

The body 320 may be formed in a polyhedral shape, and a device necessary for driving or controlling the imaging system 300 may be mounted thereon. For example, the main body 320 is implemented in the form of any one of wheels or caterpillars and includes a driving device 350 for moving the imaging system 300 and an electric motor providing driving force to the driving device 350 . can do. Also, for example, the main body 300 may be equipped with a battery (not shown) for providing power to the electric motor and a computing device for controlling the imaging system 300 .

In addition, a pillar 330 , an arm 340 , and a driving device 350 , which will be described later, may be coupled to the body 320 to the outside of the body 320 .

For example, the body 320 may have a rectangular parallelepiped shape, the pillar 330 and the arm 340 may be provided on the upper surface, and the driving device 350 may be provided on the lower surface.

The pillar 330 is formed in a rod shape having a certain length, and one end may be coupled to an area of the upper surface of the body 320 , for example, the pillar 330 is a pillar while being disposed in a vertical direction with the ground. One end of the 330 is coupled to the upper surface of the main body 320, and may be coupled to be rotatable by 180 degrees to 270 degrees left and right in the longitudinal direction as an axis.

In addition, the column 330 may have a variable length, so that the height of the arm 340 coupled to the other end of the column 330 may be changed. For example, the pillar 330 may be implemented in the form of a telescopic rail, and by changing the length, the height from the ground to the arm 340 coupled to the other end of the pillar 330 may be changed.

The arm 340 is formed in a rod shape having a certain length and is disposed parallel to the ground so that one end of the arm 340 can be coupled to the post 330 , and the imaging device 100 is installed at the other end of the arm 340 . can be combined. For example, the arm 340 is implemented in the form of a telescopic rail so that the length can be changed, and accordingly, the separation distance between the photographing device 100 provided at the other end of the arm 340 and the pillar 330 is increased or decreased. can

In addition, the marking device (not shown) is provided at one position of the arm 340 to which the photographing device 100 is coupled, and is a device capable of applying a physical external force to add external features to fruit or livestock, for example, It may be a launch device that fires paint bullets or a spray device capable of spraying a liquid mixed with paint or paint.

Meanwhile, as shown in FIG. 4 , the photographing apparatus 100 may be installed in a movable photographing system 400 .

The imaging system 400 may be implemented as a drone, and the drone is an unmanned aerial vehicle that can be controlled remotely or can fly autonomously, and according to the flight method, a propeller is installed on a fixed wing and a rotating shaft with wings attached to the aircraft horizontally to generate lift. It may include a rotor blade, a tilt rotor capable of tilting the rotor blade, and the like. The imaging system 400 may also include a marking device (not shown) installed in the imaging system 300 of FIG. 3 .

The photographing apparatus 100 may be installed at the lower end of the photographing system 400 so as to secure a visible distance when the photographing system 400 is flying.

According to an embodiment, when the photographing device 100 takes a picture of a subject on a farm, the photographing device 100 may be installed in a fixed state on the ground so that crops can be seen well, or the above-described imaging systems 300 and 400 ) can be installed.

According to another embodiment, when the photographing device 100 takes a picture of a subject in a barn, it may be installed in a fixed state on the ceiling or wall, or may be installed movably on a rail installed on the ceiling, wall, or floor, or It may be installed in the above-described imaging systems 300 and 400 .

Meanwhile, FIG. 5 is a flowchart illustrating a photographing method according to an exemplary embodiment. The photographing method illustrated in FIG. 5 includes the steps of time-series processing in the photographing apparatus 100 described with reference to FIGS. 1 to 4 . Therefore, even if omitted below, the content described above with respect to the photographing apparatus 100 illustrated in FIGS. 1 to 4 may also be used in the photographing method according to the embodiment illustrated in FIG. 5 .

6 to 9 are exemplary views for explaining the photographing method disclosed in the present specification.

As shown in FIG. 5 , the photographing apparatus 100 may acquire a first image ( S510 ).

In relation to this, FIG. 6 shows a tangerine farm in which the photographing device 100 is located, and FIG. 7 shows an image of a subject in the tangerine farm of FIG. 6 .

The photographing apparatus 100 located in the tangerine farm of FIG. 6 may photograph a subject located within the angle of view 600 . Accordingly, the first image 700 as shown in FIG. 7 may be acquired.

The photographing apparatus 100 that has obtained the first image may identify the target candidate object in the first image (S520).

As shown in FIG. 7 , the photographing apparatus 100 may identify target candidate objects 710 , 720 , 730 , 740 , and 750 estimated to be oranges, and for this purpose, artificial intelligence implemented as a YOLO model or an R-CNN model. A target candidate object can be identified by training the neural network and inputting the first image to the learned artificial neural network.

As described above, one of the identified target candidate objects may be selected to determine the target object ( S530 ).

For example, the photographing apparatus 100 may sequentially select target candidate objects in the x-axis direction based on the center point of the bounding box surrounding each target candidate object and the lowermost left vertex of the first image. Candidate objects 710, 720, 730, 740, and 750 may be sequentially selected in order. Therefore, the photographing apparatus 100 first selects the target candidate object 710 from among the target candidate objects 710, 720, 730, 740, and 750, determines it as a target object, and generates diagnostic information about the target candidate object 710, and After that, diagnosis information may be generated for each target object in the order of the following target objects 720 .

In order to photograph the determined target object, the photographing apparatus 100 may determine photographing state information ( S540 ).

The photographing apparatus 100 may determine the photographing state information based on the current photographing state information, the current distance information, and the location coordinates of the target object in the first image.

According to an embodiment, the photographing apparatus 100 may generate photographing state information by calculating a pan angle and a tilt angle based on the current photographing state information, the current distance information, and the position coordinates of the target object in the first image. can

On the other hand, when the target object is photographed in a small size due to a low zoom magnification, or when the target object is photographed at a low resolution because the focusing value does not match, the photographing state information may be updated (S550).

According to an exemplary embodiment, in an image within an angle of view viewed through a lens, a zoom magnification may be determined such that an image area of the target object occupies a predetermined ratio or more within the entire image area.

Based on the photographing state information, the photographing apparatus 100 may acquire a second image by photographing the target object by moving, focusing, or adjusting a zoom magnification (S560).

8 and 9 are exemplary views for explaining a process in which the photographing apparatus 100 acquires a second image.

As shown in FIG. 8 , the photographing apparatus 100 performs the image capturing method shown in FIG. 9 if the area 810 identified as a mandarin orange in the photographed image or the focused image does not occupy a predetermined ratio or more in the entire image 800 . As described above, the photographing apparatus 100 may adjust the zoom magnification so that the area 910 identified as an orange occupies a predetermined ratio or more in the entire image 900 .

By updating the photographing state information in this way, the photographing apparatus 100 may acquire the second image as shown in FIG. 9 .

When the second image is obtained by photographing the target object as described above, the photographing apparatus 100 may generate diagnostic information based on the second image (S570).

The photographing apparatus 100 may separate the target object located within the edge and the background located outside the edge by identifying the edge of the target object in the second image based on the deep learning model.

In addition, the imaging apparatus 100 may generate diagnostic information including at least one of growth information and pest information by analyzing the image of the target object.

In the image of a tangerine as shown in FIG. 9 , the photographing apparatus 100 may generate diagnostic information by measuring the size of the tangerine 910 and the coloration of the tangerine.

Alternatively, the imaging device 100 may generate pest information of the tangerine 910 , as shown in FIG. 9 , by identifying the pest area 920 , which is a location where the pest has been devoured, diagnostic information including related information is provided. can create For example, by analyzing the number of pest areas 920 , the number of pests if the pest is in a tangerine, and information on pests or pests, diagnostic information including information about pests or germs may be generated.

Steps S530 to S570 described above may be repeated until there is no target candidate object for which diagnostic information has not been generated among the target candidate objects identified in the first image (S580).

When diagnostic information about the first image is generated by the photographing method as described above, the photographing apparatus 100 may photograph another subject to obtain the first image.

At this time, if it is determined that there is an area in which the acquired first image overlaps with the previously analyzed first image, the photographing apparatus 100 updates the photographing state information and captures the first image by photographing it at a different pan angle or tilt angle. can be obtained Alternatively, based on the position information of the photographing apparatus 100 from which the first image was obtained, and the pan and tilt angles, it may be determined that the obtained first image was photographed at the same location as the previously analyzed first image and the same object Accordingly, the photographing apparatus 100 may acquire the first image by updating the photographing state information and photographing it at a different pan angle or tilt angle.

The photographing method described above may also be implemented in the form of a computer-readable medium for storing instructions and data executable by a computer. In this case, the instructions and data may be stored in the form of program codes, and when executed by the processor, a predetermined program module may be generated to perform a predetermined operation. In addition, computer-readable media can be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. In addition, the computer-readable medium may be a computer recording medium, which is a volatile and non-volatile and non-volatile embodied in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. It may include both volatile, removable and non-removable media. For example, the computer recording medium may be a magnetic storage medium such as HDD and SSD, an optical recording medium such as CD, DVD, and Blu-ray disc, or a memory included in a server accessible through a network.

The photographing method described above may be implemented as a computer program (or computer program product) including instructions executable by a computer. The computer program includes programmable machine instructions processed by a processor, and may be implemented in a high-level programming language, an object-oriented programming language, an assembly language, or a machine language. . In addition, the computer program may be recorded in a tangible computer-readable recording medium (eg, a memory, a hard disk, a magnetic/optical medium, or a solid-state drive (SSD), etc.).

The photographing method described above may be implemented by executing the computer program as described above by a computing device. The computing device may include at least a portion of a processor, a memory, a storage device, a high-speed interface connected to the memory and the high-speed expansion port, and a low-speed interface connected to the low-speed bus and the storage device. Each of these components is connected to each other using various buses, and may be mounted on a common motherboard or mounted in any other suitable manner.

Here, the processor may process a command within the computing device, such as for displaying graphic information for providing a Graphical User Interface (GUI) on an external input or output device, such as a display connected to a high-speed interface, for example. For example, instructions stored in memory or a storage device. In other embodiments, multiple processors and/or multiple buses may be used with multiple memories and types of memory as appropriate. In addition, the processor may be implemented as a chipset formed by chips including a plurality of independent analog and/or digital processors.

Memory also stores information within the computing device. As an example, the memory may be configured as a volatile memory unit or a set thereof. As another example, the memory may be configured as a non-volatile memory unit or a set thereof. The memory may also be another form of computer readable medium such as, for example, a magnetic or optical disk.

In addition, the storage device may provide a large-capacity storage space to the computing device. A storage device may be a computer-readable medium or a component comprising such a medium, and may include, for example, devices or other components within a storage area network (SAN), a floppy disk device, a hard disk device, an optical disk device, or a tape device, a flash memory, or other semiconductor memory device or device array similar thereto.

The term '~ unit' used in the above embodiments means software or hardware components such as field programmable gate array (FPGA) or ASIC, and '~ unit' performs certain roles. However, '-part' is not limited to software or hardware. '~unit' may be configured to reside on an addressable storage medium or may be configured to refresh one or more processors. Thus, as an example, '~' denotes components such as software components, object-oriented software components, class components, and task components, and processes, functions, properties, and procedures. , subroutines, segments of program patent code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables.

Functions provided in components and '~ units' may be combined into a smaller number of components and '~ units' or separated from additional components and '~ units'.

In addition, components and '~ units' may be implemented to play one or more CPUs in a device or secure multimedia card. The above-described embodiments are for illustration, and those of ordinary skill in the art to which the above-described embodiments pertain can easily transform into other specific forms without changing the technical idea or essential features of the above-described embodiments. You will understand. Therefore, it should be understood that the above-described embodiments are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and likewise components described as distributed may also be implemented in a combined form.

The scope to be protected through this specification is indicated by the claims described below rather than the above detailed description, and should be construed to include all changes or modifications derived from the meaning and scope of the claims and their equivalents. .

100: shooting device
210: lens unit 220: distance measurement unit
230: control unit 240: rotation drive unit
250: holding unit 260: memory
270: communication department

Claims (14)

In the photographing device,
a memory for storing photographing state information of the photographing apparatus; and
Determines photographing state information of the photographing apparatus based on a first image, controls the photographing apparatus to photograph a target object based on the photographing state information, and provides diagnostic information based on a second image photographing the target object A photographing device comprising a control unit for generating According to claim 1,
the control unit
and identifying a target object based on the first image, and determining photographing state information of the photographing apparatus based on a location of the target object in the first image. According to claim 1,
Further comprising a distance measuring unit for calculating the distance between the target object and the photographing device,
The control unit is
and updating the photographing state information based on the distance. According to claim 1,
The control unit is
and updating the photographing state information so that an image corresponding to a target object occupies an area of a predetermined size or more within the entire area of the second image. According to claim 1,
The control unit is
An imaging device for generating diagnostic information on at least one of growth information and pest information on the target object. According to claim 1,
Further comprising a wiper for removing foreign substances on the lens unit,
The control unit is
and controlling the wiper to operate based on at least one of the first image and the second image. According to claim 1,
The first image is a photographing image of a crop, a photographing device. According to claim 1,
The control unit is
and verifying the diagnostic information by comparing the diagnostic information with diagnostic information about another target object located within a predetermined range with respect to the target object. As a method of photographing by a photographing device,
acquiring a first image;
determining photographing state information of the photographing apparatus based on a first image;
controlling the photographing apparatus to photograph a target object based on the photographing state information; and
and generating diagnostic information based on a second image of the target object. 10. The method of claim 9,
The step of determining the photographing state information includes:
and identifying a target object based on the first image, and determining photographing state information of the photographing apparatus based on a location of the target object in the first image. 10. The method of claim 9,
The step of determining the photographing state information includes:
The method further comprising calculating a distance between the target object and the photographing device and updating the photographing state information based on the distance. 10. The method of claim 9,
The step of determining the photographing state information,
and updating the photographing state information so that an image corresponding to a target object occupies an area of a predetermined size or more within the entire area of the second image. 10. The method of claim 9,
The step of determining the photographing state information includes:
and updating the photographing state information by calculating a distance between the target object and the photographing device and calculating a focusing value for focusing based on the distance. 10. The method of claim 9,
The step of generating the diagnostic information includes:
A photographing method of generating diagnostic information on at least one of growth information and pest information on the target object.

Images