KR20200061821A - Apparatus and method for detecting plant in real time - Google Patents

Abstract

The present invention relates to an apparatus for detecting plant in real-time which detects and notifies a specific plant designated from a moving image to a user, and stores an image and a location in which a corresponding plant is detected so that the user may easily find the specific plant from a field, and a method thereof. The method for detecting plant in real-time comprises: a step of selecting a specific pant to be detected; a step of acquiring a moving image by an image photographing unit; a step of dividing the acquired moving image to detect the specific plant from the divided image frame; a step of storing the image frame in which the specific plant is recognized when the specific plant is recognized; a step of outputting a recognition signal indicating that the specific plant is recognized; and a step of displaying the image frame with the recognized specific plant. In an environment having difficulty in being able to communicate or not with a remote server of mountains, valleys, swamps, or far distance, the method searches a specific plant to be searched by a user from an environment in which multiple types of plants are distributed to notify the searched specific plant to the user.

Description

Apparatus and method for detecting plant in real time}

The present invention relates to a plant recognition apparatus and method, in particular, by searching for a specific plant designated from a video that is photographed in real time, and notifying the mobile phone, and storing the image and location of the plant, the specific plant the mobile phone is looking for in the field It relates to a real-time plant discovery device and method to make it easy to find.

Recently, companies that are building vast databases such as Google and Naver are providing plant recognition and search services such as flowers and trees.

In the conventional plant recognition and search service, when a user provides a plant picture taken by photographing a specific plant with a mobile terminal, the system provides the same or similar plant information in contrast to the plant picture and various plant pictures in the database.

However, since conventional service applications (hereinafter referred to as “apps”) that recognize various plants must always be connected to wired/wireless data communication, there are limitations in recognition and search in places where communication failure occurs, such as deep mountains or valleys, and the recognition rate. This is inaccurate up to 50% level, and in the case of the application with the highest recognition rate, it is a way to listen to the opinions of other experts after uploading the plant photo, so in the process of uploading the plant photo and receiving feedback from other users including experts. There is a problem that it takes time.

Looking at the prior art related to plant recognition, it is a technology that is intended to provide plant information to users without any knowledge, or to increase the recognition rate using characteristic parameters of plant leaves or flowers. This can be useful if you want to know the information of plants you don't know, but if you want to find the plants you want in deep mountains, valleys, or swamps, you have to take pictures of many plants around you and wait for an answer, so it is not useful at all.

KR10-2012-0135965 A (2012.12.18) KR10-1498455 B1 (2015.02.26) KR10-1373415 B1 (2014.03.05) JP10-2002-14203242 A (2002.07.19)

FROBA, Bernhard; ERNST, Andreas. Face detection with the modified census transform. In: Automatic Face and Gesture Recognition, 2004. Proceedings. Sixth IEEE International Conference on. IEEE, 2004. p. 91-96.

Accordingly, the present inventor puts emphasis on solving the need to find a specific plant desired for research and academic purposes or for the purpose of gathering by comprehensively considering all the above-mentioned matters, so that the user can easily find the plant he or she is looking for in the field. As a result of researching constantly with his efforts, the present invention was invented.

Therefore, the technical problem and object to be solved by the present invention is not a method of recognizing a DB with a picture of a specific plant, but a plant recognition device and method of finding and retrieving a plant in a picture or video in real time. , In an environment in which a variety of plants are distributed, if a user designates a specific plant to be searched for and shoots the surrounding landscape as a video, the specified plant is found in real time from a video that is shot in real time, and alerts and location information to the user in real time It is to provide a real-time plant discovery device and method that allows the user to easily find a specific plant at a desired time by providing a.

In addition, the present invention is to provide a real-time plant discovery apparatus and method that can find a specific plant that a user wants to find in real-time even in an environment in which communication with a remote server, such as a deep mountain, valley, swamp, or remote location is impossible or difficult. .

In addition, the present invention not only finds and notifies a user of a specific plant designated from a video recorded in real time, but also stores an image and a location where the plant is found, so that the user can find the plant at any time through the stored image and location information. To provide a real-time plant discovery device and method.

Here, the technical problems and objects to be solved by the present invention are not limited to the technical problems and objectives mentioned above, and other technical problems and objectives not mentioned will be clearly understood by those skilled in the art from the following description.

Real-time plant discovery method according to the present invention for achieving the above object, selecting a specific plant to search; Obtaining a video through the image capturing unit; Dividing the obtained video into an image frame and searching for the specific plant from the divided image frame; If the specific plant is recognized, storing an image frame in which the specific plant is recognized; Outputting a recognition signal informing that the specific plant has been recognized; And displaying the image frame in which the specific plant is recognized; in the environment in which a variety of plants are distributed, a specific plant to be searched by the user can be searched in real time from a captured video to inform the user.

In addition, in the real-time plant discovery method according to the present invention, in the step of displaying the image frame, the user can make it easier by displaying the location information of the specific plant recognized together with the image frame in which the specific plant is recognized. You can quickly find a specific plant.

In addition, in the real-time plant discovery method according to the present invention, the recognition rate selection step, the recognition distance selection step, and the recognition signal indicating that the specific plant is recognized before the step of obtaining the video through the image capturing unit Accordingly, by performing at least one of the alarm selection steps to be used, the user can determine a recognition speed and an alarm method suitable for the user.

In addition, the real-time plant discovery apparatus according to the present invention for achieving the above object, a specific plant DB for storing the data of the specific plant to be searched; A video photographing unit that acquires a video when a specific plant to be searched is selected; A search unit for dividing the video acquired by the image photographing unit into an image frame and comparing the divided image frame and the specific plant DB to search for the specific plant; A recognition result DB for storing an image frame in which the specific plant is recognized when a specific plant is searched from the search unit; Alarm means for outputting an alarm according to a recognition signal indicating that the specific plant has been searched from the search unit; And a display means for displaying the image frame in which the specific plant is recognized. It can search in real time from the video filmed in the environment where they are distributed and inform the user.

In addition, in the real-time plant discovery apparatus according to the present invention, the search unit includes an input unit for selecting a specific plant to be searched by a user, a shooting video receiving unit for receiving a video captured from the video shooting unit, and the specific plant DB. Search for the specific plant by dividing the video received by the plant data receiving unit receiving the specific plant data into the video frame, and comparing the divided video frame and the specific plant data received by the plant data receiving unit. An image frame search unit to be used, an identification frame transmission unit to transmit the image frame recognized by the specific plant to the recognition result DB when the specific plant is recognized by the image frame search unit, and the specific plant from the image frame search unit And a recognition notification unit for applying the recognition signal indicating that the search has been made to the alarm means, a display unit for displaying the image frame on which the specific plant has been recognized, on the display means, and a control unit for overall controlling the components of the search unit It is characterized by.

In addition, in the real-time plant discovery apparatus according to the present invention, the input unit allows the user to further select at least one of a recognition rate, a recognition distance, and an alarm of a specific plant to be searched by the user, thereby allowing a user to recognize a recognition speed and an alarm method suitable for the user. Can decide.

As described above, the present invention has the advantage of being able to find in real time a specific plant that is desired to find even in a difficult environment where communication with a remote server such as a deep mountain or valley, swamp or remote location is impossible.

In addition, the present invention provides an advantage in that a user can easily find a specific plant in real time at a desired time by providing a search, analysis, notification, and location information in real time in a environment in which a variety of plants are distributed. have.

As a result, researchers who want to find the habitat of a specific plant for research, and experts who collect specific plants such as wild ginseng and camphor ginseng, as well as the general tool for collecting wild herbs, can increase the efficiency of work. have.

In addition, the present invention not only informs the user by finding a specific plant designated from a video shot in real time, but also stores the image and location where the plant is found, so that the user can find the plant at any time through the stored image and location information. Therefore, in places where it is difficult to access, there is an advantage that a user can find a desired plant by using a drone equipped with a photographing function.

In addition, there is an effect that can be prevented in advance to cause human life accidents that cause damage to life by erroneously contacting or ingesting plants that are toxic or harmful to the human body in the field.

In addition, the present invention is not only possible to be installed in the form of an application on a smart terminal, which is an essential portable product, but also can be implemented as a separate miniaturized device, so that hikers visiting mountains and mountains can be easily carried to ensure safe walking from toxic plants. It has an effect.

1 is a conceptual diagram of a real-time plant discovery apparatus and method according to the present invention,
2 is a control flow diagram of a real-time plant discovery method performed in a real-time plant discovery apparatus according to the present invention,
3 is an internal block diagram of a search unit of a real-time plant discovery apparatus according to the present invention.

Hereinafter, an embodiment of a real-time plant discovery apparatus and method according to the present invention will be described in detail with reference to the drawings. The embodiments described below are provided to fully inform the person of ordinary skill in the scope of the invention, and the present invention is not limited to the embodiments disclosed below and may be implemented in various forms.

Identical elements in the figures indicate the same reference numerals wherever possible. In the following description, specific specific matters are shown, which are provided to help a more comprehensive understanding of the present invention, and are not intended to limit the present invention to specific embodiments, and all modifications included in the spirit and scope of the present invention. , It should be understood to include equivalents or substitutes. And in the description of the present invention, when it is determined that a detailed description of related known functions or configurations may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted.

Here, the attached drawings are exaggerated or simplified for a convenience and clarity of description and understanding of the structure and operation of the technology, and it is revealed that each component does not exactly match the actual size.

Terms such as first and second may be used to describe various components, but the components are not limited by the terms. The terms are used only for the purpose of distinguishing one component from other components. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component.

When an element is said to be "connected" or "connected" to another component, it is understood that other components may be directly connected to or connected to the other component, but there may be other components in between. Should be. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that no other component exists in the middle.

Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, terms such as “include” or “have” are intended to indicate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, one or more other features. It should be understood that the existence or addition possibilities of fields or numbers, steps, operations, components, parts or combinations thereof are not excluded in advance.

In addition, when a part includes a certain component, this means that other components may be further included, not excluded from other components, unless otherwise stated. Means a unit or module type that processes at least one function or operation, and may be implemented by hardware or software or a combination of hardware and software.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person skilled in the art to which the present invention pertains. Terms, such as those defined in the dictionary used in general, are defined in consideration of the functions in the present invention, which should be interpreted as concepts consistent with the technical spirit of the present invention and meanings commonly or commonly recognized in the art. And, unless expressly defined in the present application, is not to be interpreted in an ideal or excessively formal sense.

Combinations of each block of the block diagram and each step of the flowchart may be performed by computer program instructions. These computer program instructions may be mounted on a processor of a general purpose computer, special purpose computer, or other programmable data processing equipment, so that instructions executed through a processor of a computer or other programmable data processing equipment may be used in each block or flowchart of the block diagram. In each step, means are created to perform the functions described.

Each block or each step can represent a module, segment, or portion of code that includes one or more executable instructions for executing the specified logical function(s). It should also be noted that in some alternative embodiments it is possible that the functions mentioned in blocks or steps occur out of order. For example, two blocks or steps shown in succession may in fact be executed substantially simultaneously, or it is also possible that the blocks or steps are sometimes performed in reverse order depending on the corresponding function.

The real-time plant discovery device and method according to the present invention drive an application for a smart terminal, thereby real-time video shooting by a camera and other optical devices associated with the application, or real-time video shooting through a camera mounted on a separately-produced discovery device, smart terminal Or, in connection with the DB of the specific plant stored in the memory built in the discovery device, the specific plant recognition, the video frame and the position and time of the specific plant recognized in the captured video are stored, and an alarm means to the user that the specific plant has been searched This is a technology to notify in the form of alarm sound, voice, vibration, etc., and to display the location of a specific plant searched through the display means of a smart terminal or discovery device. By providing the location in real time and automatically, it is useful for the identification of academically valuable plants, and can prevent accidents due to ignorant contact with toxic or harmful plants and accidents due to incorrect collection and ingestion.

1 is a conceptual diagram of a real-time plant discovery apparatus and method according to the present invention. The real-time plant discovery device according to the present invention may be a smart terminal equipped with an application for a smart terminal driven on a smart terminal such as a smart phone or a smart pad, and a real-time plant discovery device (100, hereinafter) in which a real-time plant discovery function is separately implemented , “Discovery device”).

Referring to Figure 1, the real-time plant discovery apparatus and method according to the present invention, first, after specifying a specific plant to be searched, the image pickup unit 120 mounted on the discovery device 100 photographs the location to search for a specific plant When the photographed video 10 is sent to the search unit 110, the search unit 110 performs an algorithm for recognizing a specific plant for each image frame. The recognition algorithm recognizes a specific plant by comparing it with the specific plant DB 130 for each plant detected in the image frame. The specific plant DB 130 is a set of related data including various photos of a specific plant that has been previously downloaded and stored in the discovery device 100, and the specific plant data to be compared with data extracted from the image frame at the request of the search unit 110 Since it has to be provided, it is built in the memory of the discovery device 100.

As a result of performing the recognition algorithm in the search unit 110, when a specific plant is recognized, the image frame 20 in which the specific plant is recognized is constructed as the recognition result DB 140 in the memory of the discovery apparatus 100. The image frame 20 in which the specific plant is recognized is stored in correspondence with the GPS location information and time information in which the specific plant is recognized, and must be provided according to a user's request in real time or after a certain time has passed, and thus stored in the recognition result DB 140 The search unit 110 reports to the user that a specific plant has been recognized through the alarm means 150.

The user using the real-time plant discovery apparatus and method according to the present invention searches for a specific plant to be searched while moving in the mountains while driving the discovery apparatus 100, for example. At this time, the discovery apparatus 100 according to the present invention may assist the user to find a specific plant that the user has not seen or has not seen.

On the other hand, since the image capturing unit 120 finds a specific plant in a region surrounding the user centering on the user, it is effective for the user to access, and thus it is preferable to be implemented as a camera capable of rotating 360 degrees or rotating a predetermined angle.

The search unit 110 is a processor on which the plant recognition algorithm is driven, and techniques such as Haar-like Feature, adaboost, Histogram of Oriented Gradients (HOG), Local Binary Pattern (LBP), Modified Census Transform (MCT), alone or It can be used in plural, and OpenCV can be used as a representative algorithm.

The specific plant DB 130 stores various data on one or more specific plants so that a specific plant can be found even in a situation in which wired/wireless data communication is difficult. When the recognition algorithm is performed on the specific plant according to the user's selection, Data related to a specific plant requested by the search unit 110 is provided.

The recognition result DB 140 is a DB in which the recognized specific plant is recorded, including the image frame where the specific plant was photographed, and the location and time of the captured plant are recorded. The user accesses the location where the specific plant is found even in real time or even after a certain time. Make it possible.

The screens 20 and 30 provided by the recognition result DB 140 are provided through the display means 160, and the location of the specific plant may be displayed in the image frame 20 of the specific plant recognized by the user, The location information image 30 may be provided by displaying the location provided by the GPS on a map or the like. Accordingly, it is possible to find a specific plant desired in a location that cannot be approached by using a drone equipped with a real-time plant discovery device according to the present invention or a device equipped with a real-time plant discovery method according to the present invention.

2 is a control flowchart of a real-time plant discovery method performed in the real-time plant discovery apparatus according to the present invention, and FIG. 3 is an internal block configuration diagram of a search unit of the real-time plant discovery apparatus according to the present invention. Hereinafter, a real-time plant discovery apparatus and method according to the present invention will be described in detail with reference to FIGS. 1 to 3.

The real-time plant discovery method performed in the real-time plant discovery apparatus according to the present invention first performs steps 201 to 207 through the input unit 112 of the search unit 110 before finding a specific plant by taking a video. It is preferred to perform. On the other hand, except for step 201 in step 201 to step 207 of selecting a specific plant to be searched, steps 203 to 207 may be automatically applied when selecting a specific plant after setting it only once for the first time. .

The control unit 111 of the search unit 110 selects a specific plant to be searched according to the input signal generated by the input unit 112 according to the user's selection (201). Subsequently, the control unit 111 selects the recognition rate of the specific plant selected in step 201 according to the input signal generated according to the user's selection which level of recognition rate is desired (203). Since the selection of the recognition rate is related to the recognition speed according to the computation amount of the recognition algorithm, it is desirable to determine the level required by the user according to the recognition level of the user's specific plant.

Next, the controller 111 selects the recognition distance of the specific plant according to the input signal generated by the input unit 112 according to the user's selection within a certain distance from the discovery device 100 to the desired distance. (205). Since the selection of the recognition distance is related to the recognition rate, there may be limitations according to the selection of the recognition rate in step 203, and like the recognition rate, the recognition speed according to the computation amount of the recognition algorithm is determined, and according to the recognition level of the user's specific plant It is desirable that the user decides to the required level. Steps 203 and 205 may be performed by changing the order.

The final step of the initial setting process is a selection step 207 for a method of notifying a user of a specific plant discovery when a specific plant is found. When the user selects a specific notification sound, voice, music, or vibration, or selects at least one of various alarm means 150 providing each of them, the control unit 111 according to the input signal generated by the input unit 112 according to the user's selection ) Controls the recognition alarm unit 117 so that when a specific plant is found, the corresponding alarm means 150 is activated so that the user can immediately recognize it.

When the setting process of steps 201 to 207 is completed, the control unit 111 causes the shooting video receiving unit 113 to receive the video 10 from the video shooting unit 120, and the image frame search unit 114. The video frame is retrieved from the video that is controlled and photographed (step 209). At this time, the control unit 111 allows the plant data receiving unit 115 to provide data for a specific plant from the specific plant DB 130 to the image frame search unit 114 to find a specific plant.

Subsequently, if a specific plant is recognized in step 211, the control unit 111 controls the recognition frame transmitter 116 in step 215 to store the corresponding image frame in which the specific plant is recognized in the recognition result DB 140. On the other hand, if a specific plant is not recognized as a result of the video frame search in step 211, the process proceeds to step 213 to delete the recognized frame that has been searched and searches for a specific plant from the new video frame.

After storing the specific plant recognition frame in step 215 or at the same time, the control unit 111 sends the recognition notification unit 117 the recognition signal to the alarm unit 150 to notify the user that the specific plant has been found. 150) The notification signal is output to allow the user to recognize that a specific plant has been found (step 217).

At this time, when the specific plant recognition frame is stored in step 215, the controller 111 refers to the GPS location information, and causes the recognition frame transmitter 116 to recognize the location information and time information of the specific plant recognized in the recognition frame. It may be stored in the result DB 140.

Accordingly, in step 219, if the user wants to immediately confirm the specific plant recognition frame in response to the notification signal, the controller 111 proceeds to step 223 and causes the display unit 118 to find the specific plant through the display means 160. The provided image frame 20 is provided so that the user can visually confirm whether it is a specific plant to be searched for. At this time, the display unit 118 provides the location information image 30 together with the image frame 20 to the display means 160 to help the user find the actual location of the specific plant photographed.

On the other hand, if the user does not immediately receive the recognition frame in step 219, at this time, the control unit 111 refers to the location information of the GPS to allow the recognition frame transmitter 116 to store the corresponding location information and time information with the recognized frame. It can be stored in association (step 221), and the image search in step 209 is restarted.

If it is determined in step 223 that the user is looking for a specific plant, the controller 111 inquires whether the search has been completed in step 225, ends the search according to the user's input, or restarts the video search in step 209.

As described above, in a real-time plant discovery apparatus and method according to the present invention, in a environment in which a variety of plants are distributed, when a user designates a specific plant to be searched for and shoots the surrounding landscape as a video, from the video recorded in real time By searching for a specific specific plant in real time and providing notification and location information to the user in real time, the user can easily find a specific plant at a desired moment.

In addition, the real-time plant discovery apparatus and method according to the present invention can find a specific plant that a user wants to find in real-time even in environments where it is impossible or difficult to communicate with remote servers such as deep mountains, valleys, swamps and remote areas.

In addition, the real-time plant discovery apparatus and method according to the present invention not only finds and notifies a user of a specific plant designated from a video recorded in real time, but also stores the image and location where the plant is found, so that the user can view the stored image and location information. Through this, you can always find the plant.

On the other hand, in the detailed description of the present invention, it has been specifically described with reference to preferred embodiments referenced by the accompanying drawings, but of course, various modifications are possible without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined not only by the scope of claims to be described later, but also by the scope and equivalents of the claims.

100: discovery device 110: search unit
111: control unit 112: input unit
113: recording video receiving unit 114: video frame search unit
115: plant data receiver 116: recognition frame transmitter
117: recognition notification unit 118: display unit
120: imaging unit 130: specific plant DB
140: recognition result DB 150: alarm means
160: display means

Claims (6)

Selecting a specific plant to be searched;
Obtaining a video through the image capturing unit;
Dividing the obtained video into an image frame and searching for the specific plant from the divided image frame;
If the specific plant is recognized, storing an image frame in which the specific plant is recognized;
Outputting a recognition signal informing that the specific plant has been recognized; And
And displaying the image frame in which the specific plant is recognized.
The method of claim 1, wherein the step of displaying the image frame,
The real-time plant discovery device, characterized in that further display the location information of the specific plant is recognized together with the video frame in which the specific plant is recognized.
According to claim 1, Before the step of obtaining a video through the image capture unit,
A real-time plant discovery method characterized by performing at least one of the recognition rate selection step of the specific plant to be searched, the selection step of the recognition distance and the alarm selection step to be used according to the recognition signal indicating that the specific plant is recognized.
A specific plant DB that stores data of a specific plant to be searched;
A video photographing unit that acquires a video when a specific plant to be searched is selected;
A search unit for dividing the video acquired by the image photographing unit into an image frame, and comparing the divided image frame and the specific plant DB to search for the specific plant;
A recognition result DB for storing an image frame in which the specific plant is recognized when a specific plant is searched from the search unit;
Alarm means for outputting an alarm according to a recognition signal indicating that the specific plant has been searched from the search unit; And
And display means for displaying the image frame in which the specific plant is recognized.
The method of claim 4, wherein the search unit,
Input unit for selecting a specific plant to be searched from the user,
A shooting video receiving unit that receives a video taken from the video shooting unit;
Plant data receiving unit for receiving the specific plant data from the specific plant DB,
A video frame search unit for dividing the video received by the photographed video receiving unit into an image frame, and comparing the divided video frame and the specific plant data received by the plant data receiving unit to search for the specific plant;
When the specific plant is recognized from the image frame search unit, a recognition frame transmitter for transmitting the image frame in which the specific plant is recognized to the recognition result DB,
A recognition notification unit for applying a recognition signal indicating that the specific plant has been searched from the image frame search unit to the alarm means;
A display unit that displays the image frame on which the specific plant is recognized on the display means;
Real-time plant discovery device comprising a control unit for controlling the overall components of the search unit.
The method of claim 5, wherein the input unit,
A real-time plant discovery device characterized in that at least one of a recognition rate, a recognition distance, and an alarm of a specific plant to be searched from the user is further selected.

Images