KR20240080180A - Device and operating method thereof for learning and inferring for each variety of kalanchoe - Google Patents

Abstract

According to the present disclosure, based on image data, an artificial intelligence model is trained to determine the variety of the Kalanchoe, and based on the image data, the canopy of the Kalanchoe, the number of flowers of the Kalanchoe, the flowering rate of the Kalanchoe, and the The artificial intelligence model is trained to predict and determine a plurality of quality factors, including the ratio of the Kalanchoe's peduncle and the amount of damage to the Kalanchoe due to pests, and the new image data received from the image sensor is used to learn the new image data. An electronic device and method of operating the same are provided for predicting and determining the plurality of quality factors by inputting them into an artificial intelligence model.

Description

Electronic device and operating method for learning and predicting quality factors for each variety of Kalanchoe {DEVICE AND OPERATING METHOD THEREOF FOR LEARNING AND INFERRING FOR EACH VARIETY OF KALANCHOE}

Embodiments of the present disclosure relate to artificial intelligence, and more specifically, to an electronic device and method of operating the same for learning and predicting quality factors for each variety of Kalanchoe.

The size of the plant market is gradually increasing. It is necessary to provide high-quality plants to consumers by inspecting the quality of plants when shipping and delivering them to consumers. If a consumer receives a low-quality plant, the consumer's satisfaction with purchasing the plant may decrease or be disappointed, which may reduce the consumer's willingness to purchase. It is very difficult for people to determine the quality of plants, especially Kalanchoe, and as people handle Kalanchoe during quality inspection and cause a lot of damage to Kalanchoe, the quality of Kalanchoe may deteriorate. Accordingly, there is a need for technology to determine the quality of plants, including Kalanchoe, in a non-contact manner without contacting them.

Republic of Korea Patent Publication No. 10-2019-0108275 (2021.02.05)

In the past, as people handled kalanchoes and caused a lot of damage to them, the quality of the kalanchoes deteriorated, and the shape of the kalanchoes was limited, making it difficult for people to check them with the naked eye.

Embodiments of the present disclosure are intended to solve various problems including the problems described above, and include an electronic device and operating method for learning and predicting quality factors for each variety of Kalanchoe in a non-contact manner without contacting plants, including Kalanchoe. We would like to provide. However, these tasks are illustrative and do not limit the scope of the present disclosure.

According to one aspect of the present disclosure, an image sensor for photographing a pollen containing a kalanchoe and generating image data including images of the kalanchoe and the pollen; A memory that stores one or more instructions; And a processor that executes the one or more instructions stored in the memory, wherein the processor trains an artificial intelligence model to determine the variety of the Kalanchoe based on the image data received from the image sensor, and the image Based on the data, predict and determine a plurality of quality factors including the canopy of the kalanchoe, the number of flowers of the kalanchoe, the flowering rate of the kalanchoe, the ratio of peduncles of the kalanchoe, and the amount of damage to the kalanchoe due to pests. An electronic device is provided that trains the artificial intelligence model and inputs new image data received from the image sensor into the trained artificial intelligence model to predict and determine the plurality of quality factors.

According to this embodiment, the processor receives first image data including a front image in which the front of the Kalanchoe is photographed from the image sensor, detects the front image from the first image data, and detects the front image The width and length of the Kalanchoe excluding the pollen can be measured, and the artificial intelligence model that determines the canopy according to the variety can be trained based on learning data including the variety and the canopy.

According to this embodiment, the processor receives a plurality of first image data each including a front view of the kalanchoe photographed while the kalanchoe is rotated, and in the front image included in each of the plurality of first image data The width and the length may be measured, and the canopy including the maximum width and maximum length among the plurality of measured widths and lengths may be set as the learning data.

According to this embodiment, the processor receives a plurality of second image data including a top-view image in which the Kalanchoe is photographed from a top-view perspective from the image sensor, and receives from the plurality of second image data Detect a plurality of top-view images according to the flowering state of the flower, and predict the number of flowers of the Kalanchoe and the flowering rate of the Kalanchoe according to the variety based on learning data including each top-view image according to the flowering state. And the artificial intelligence model that makes the decision can be trained.

According to this embodiment, the processor may detect a first top-view image according to non-blooming, a second top-view image according to partial flowering, and a third top-view image according to full flowering from the plurality of second image data. .

According to this embodiment, the processor receives third image data including a top view image in which the Kalanchoe is photographed from a top view viewpoint from the image sensor, and selects the flower of the Kalanchoe in the top view image of the third image data. Distinguish between the image representing the leaf and the image representing the leaf of the Kalanchoe, measure the area of the flower in the image representing the flower of the Kalanchoe, and based on learning data including the area, the flower size of the Kalanchoe according to the variety. The artificial intelligence model that predicts and determines the ratio can be trained.

According to this embodiment, the processor sets an outermost circumscribed circle centered on the flowerpot and an inscribed circle in contact with a flower located closest to the center in the third image data, and sets the circumscribed circle and the inscribed circle to 4, respectively. By dividing the flower into equal parts, eight regions can be set, the partial area of the flower included in each of the eight regions can be measured, and the area of the flower can be calculated based on the partial area.

According to this embodiment, the processor receives fourth image data from the image sensor including a top view image in which the kalanchoe is photographed from a top view point, and the kalanchoe is damaged by the pest in the top view image of the fourth image data. Select an image representing a first area damaged by the pest and an image representing a second area not damaged by the pest, and based on learning data including the image representing the first area, the pest according to the variety The artificial intelligence model that determines the amount of damage to the Kalanchoe due to can be trained.

According to this embodiment, the processor may further select images representing the pests from the top-view image and include the images representing the pests in the learning data.

According to another aspect of the present disclosure, photographing a pollen containing kalanchoe to generate image data containing images of the kalanchoe and the pollen; Based on the image data, training an artificial intelligence model to determine the variety of the Kalanchoe; Based on the image data, predict a plurality of quality factors including the canopy of the kalanchoe, the number of flowers of the kalanchoe, the flowering rate of the kalanchoe, the ratio of peduncles of the kalanchoe, and the amount of damage to the kalanchoe due to pests. And training the artificial intelligence model to make a decision; and inputting new image data into the trained artificial intelligence model to predict and determine the plurality of quality factors.

Other aspects, features and advantages other than those described above will become apparent from the detailed description, claims and drawings for carrying out the invention below.

Additionally, these general and specific aspects may be practiced using any system, method, computer program, or combination of any system, method, or computer program.

According to an exemplary embodiment of the present disclosure made as described above, an electronic device for learning and predicting quality factors for each variety of Kalanchoe that does not damage or harm the plant and does not deteriorate the quality of the delivered plant, and the same The operation method can be implemented. Of course, the scope of the present disclosure is not limited by this effect.

Figure 1 is a block diagram schematically showing an electronic device according to an exemplary embodiment of the present disclosure.
Figure 2 is a flowchart for explaining a method of operating an electronic device according to an exemplary embodiment of the present disclosure.
Figure 3 is a flowchart conceptually explaining the operation of an electronic device according to an exemplary embodiment of the present disclosure.
Figure 4 is a flowchart illustrating the steps of learning an artificial intelligence model for determining a canopy according to an exemplary embodiment of the present disclosure.
Figure 5 is a diagram showing a method for sensing the width and length of a kalanchoe according to an exemplary embodiment of the present disclosure.
Figure 6 is a flowchart illustrating the steps of learning an artificial intelligence model for predicting and determining the number of Kalanchoe flowers and/or the flowering rate according to an exemplary embodiment of the present disclosure.
Figure 7 is a diagram illustrating a method for detecting images at each stage of flowering according to an exemplary embodiment of the present disclosure.
Figure 8 is a flowchart illustrating the steps of learning an artificial intelligence model for predicting and determining the flower diameter ratio of Kalanchoe according to an exemplary embodiment of the present disclosure.
9 and 10 are diagrams schematically showing areas divided into concentric circles according to an exemplary embodiment of the present disclosure.
Figure 11 is a flowchart illustrating the steps of learning an artificial intelligence model for determining the amount of damage to Kalanchoe caused by pests according to an exemplary embodiment of the present disclosure.
Figure 12 is a diagram visually illustrating a process for detecting pest damage according to an exemplary embodiment of the present disclosure.

Since the present disclosure can be modified and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. The effects and features of the present disclosure and methods for achieving them will become clear by referring to the embodiments described in detail below along with the drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various forms.

In the following embodiments, terms such as first and second are used not in a limiting sense but for the purpose of distinguishing one component from another component.

In the following examples, singular terms include plural terms unless the context clearly dictates otherwise.

In the following embodiments, terms such as include or have mean that the features or components described in the specification exist, and do not exclude in advance the possibility of adding one or more other features or components.

In the following embodiments, when a part of a layer, area, component, etc. is said to be on or on another part, not only when it is directly on top of the other part, but also when other areas, components, etc. are interposed between them. Also includes.

In the drawings, the sizes of components may be exaggerated or reduced for convenience of explanation. For example, the size and thickness of each component shown in the drawings are shown arbitrarily for convenience of explanation, so the present disclosure is not necessarily limited to what is shown.

If an embodiment can be implemented differently, a specific operation sequence may be performed differently from the described sequence. For example, two steps described in succession may be performed substantially simultaneously, or may be performed in an order opposite to that described.

In this specification, “A and/or B” refers to A, B, or A and B. And, “At least one of A and B is A, B, or A and B.

In the following embodiments, when layers, areas, components, etc. are said to be connected, when the layers, areas, and components are directly connected, or/and other layers, areas, and components are in the middle of the layers, areas, and components. This also includes cases where they are interposed and indirectly connected. For example, when layers, regions, components, etc. are said to be electrically connected in this specification, when the layers, regions, components, etc. are directly electrically connected, and/or other layers, regions, components, etc. are interposed between them. indicates a case of indirect electrical connection.

The x-axis, y-axis, and z-axis are not limited to the three axes in the Cartesian coordinate system and can be interpreted in a broad sense including these. For example, the x-axis, y-axis, and z-axis may be orthogonal to each other, but may also refer to different directions that are not orthogonal to each other.

The advantages and features of the present disclosure and methods for achieving them will become clear by referring to the embodiments described in detail below along with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms. The present embodiments are merely provided to ensure that the disclosure is complete and to provide a general understanding of the technical field to which the present disclosure pertains. It is provided to fully inform those skilled in the art of the scope of the present disclosure, and the present disclosure is defined only by the scope of the claims.

The terminology used in this disclosure is for describing embodiments and is not intended to limit the disclosure. In this disclosure, singular forms may also include plural forms unless specifically stated otherwise in the context. As used in the disclosure, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other elements in addition to the mentioned elements. Like reference numerals refer to like elements throughout the disclosure, and “and/or” can include each and every combination of one or more of the referenced elements. Although “first”, “second”, etc. are used to describe various components, these components are of course not limited by these terms. These terms are merely used to distinguish one component from another. Therefore, it goes without saying that the first component mentioned below may also be the second component within the technical spirit of the present disclosure.

The word “exemplary” is used in this disclosure to mean “used as an example or illustration.” Any embodiment described as “exemplary” in this disclosure should not necessarily be construed as preferred or as having an advantage over other embodiments.

Embodiments of the present disclosure may be described in terms of a function or a block that performs a function. Blocks that may be referred to as 'units' or 'modules' of the present disclosure include logic gates, integrated circuits, microprocessors, microcontrollers, memories, passive electronic components, active electronic components, optical components, and hardwired circuits. It is physically implemented by analog or digital circuits, etc., and can optionally be driven by firmware and software. Additionally, the term “unit” used in the disclosure refers to a hardware element such as software, FPGA, or ASIC, and the “unit” may perform certain roles. However, “wealth” is not limited to software or hardware. The “copy” may be configured to reside on an addressable storage medium and may be configured to run on one or more processors. Thus, as an example, “part” refers to elements such as software elements, object-oriented software elements, class elements, and task elements, processes, functions, properties, procedures, subroutines, and programs. May include segments of code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. The functionality provided within elements and “parts” may be combined into smaller numbers of elements and “parts” or may be further separated into additional elements and “parts”.

Embodiments of the present disclosure may be implemented using at least one software program running on at least one hardware device and may perform network management functions to control elements.

Spatially relative terms such as “below”, “beneath”, “lower”, “above”, “upper”, etc. are used as a single term as shown in the drawing. It can be used to easily describe the correlation between a component and other components. Spatially relative terms may be understood as terms that include different directions of components during use or operation in addition to the directions shown in the drawings. For example, if you flip a component shown in a drawing, a component described as “below” or “beneath” another component will be placed “above” the other component. You can lose. Accordingly, the illustrative term “down” may include both downward and upward directions. Components can also be oriented in other directions, so spatially relative terms can be interpreted according to orientation.

Unless otherwise defined, all terms (including technical and scientific terms) used in this disclosure may be used with meanings that can be commonly understood by those skilled in the art to which this disclosure pertains. Additionally, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless clearly specifically defined.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings. When describing with reference to the drawings, identical or corresponding components will be assigned the same reference numerals and redundant description thereof will be omitted. .

Terms in this disclosure are defined as follows. Differentiation: Plant growing in a pot, Plant height: Length from the root and stem border to the tip of the leaf (vertical plant length), Plant width: Horizontal length of the plant, Canopy: Shape of the above-ground part of the plant (plant height and plant width), Flowering: The phenomenon of flowering , non-blooming: a phenomenon in which flowers have not yet bloomed, flowering rate: the rate at which flowers have bloomed, peduncle: the peduncle, or the part of the peduncle connected to the peduncle when one flower is blooming on the peduncle, forgery: a phenomenon in which the plant wilts and dries out, insect damage: Damage caused by insects, supplies: unmarketable plants or unsold plants, sorting: the task of organizing items according to size or grade

FIG. 1 is a block diagram schematically showing an electronic device 100 according to an exemplary embodiment of the present disclosure.

Referring to FIG. 1, the electronic device 100 can communicate with a user terminal and execute a program of program data. In this specification, 'electronic device for learning and predicting quality factors for each variety of Kalanchoe (hereinafter referred to as device according to the present disclosure)' includes various devices that can perform computational processing and provide results to the user. For example, the device according to the present disclosure is a computing device, and may include all of a computer, a server device, and a portable terminal, or may take the form of any one. Here, the computer may include, for example, a laptop equipped with a web browser, a desktop, a laptop, a tablet PC, a slate PC, etc. A server device is a server that processes information by communicating with external devices and may include an application server, computing server, database server, file server, game server, mail server, proxy server, and web server. Portable terminals are, for example, wireless communication devices that ensure portability and mobility, such as PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handy-phone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminal, smart phone (Smart Phone) All types of handheld wireless communication devices such as phones and wearables such as watches, rings, bracelets, anklets, necklaces, glasses, contact lenses, or head-mounted devices (HMDs). May include devices.

The image sensor 110 can photograph plants. In example embodiments, the image sensor 110 may image kalanchoe and pollen containing kalanchoe. In example embodiments, the image sensor 110 may generate image data including images of the kalanchoe and the pollen by photographing a pollen containing the kalanchoe. The image sensor 110 can photograph the kalanchoe from various perspectives, such as the front, side, top, and back of the kalanchoe. In example embodiments, the image sensor 110 may be implemented as a camera, but the present disclosure is not limited to the example embodiments. In the present disclosure, an image taken of the upper surface of a kalanchoe is referred to as a 'top-view image'.

The memory 120 can store data for an algorithm for controlling the operation of components within the device or a program that reproduces the algorithm, and at least one device that performs the above-described operation using the data stored in the memory 120. It may be implemented with the processor 130. Here, the memory 120 and the processor 130 may each be implemented as separate chips. Additionally, the memory 120 and processor 130 may be implemented as a single chip.

The memory 120 can store data supporting various functions of the device and a program for the operation of the processor 130, can store input/output data, and can store a plurality of application programs running on the device. program or application), data for operation of the device, commands, and one or more instructions can be stored. At least some of these applications may be downloaded from an external server via wireless communication.

The memory 120 includes a flash memory type, a hard disk type, a solid state disk type, an SDD type (Silicon Disk Drive type), and a multimedia card micro type. micro type), card-type memory (e.g. SD or XD memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), EEPROM (electrically erasable) It may include at least one type of storage medium among programmable read-only memory (PROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, and optical disk. Additionally, the memory 120 is separate from the device, but may be a database connected by wire or wirelessly.

The processor 130 may execute one or more instructions stored in the memory 120. The processor 130 may train an artificial intelligence model to determine the variety of Kalanchoe based on image data received from the image sensor 110. Additionally, the processor 130 can train an artificial intelligence model to predict and determine a plurality of quality factors based on image data through an artificial neural network processing unit. In example embodiments, the plurality of quality factors may include the canopy of the kalanchoe, the number of flowers of the kalanchoe, the flowering rate of the kalanchoe, the proportion of peduncles of the kalanchoe, and the amount of damage to the kalanchoe due to pests. The processor 130 may input new image data received from the image sensor 110 into a trained artificial intelligence model to predict and determine a plurality of quality factors.

Artificial intelligence models can be implemented through an artificial neural network processing unit. For example, the artificial neural network processing unit can implement an artificial intelligence model by calling functions, libraries, parameter values, hyperparameters, etc. stored in the memory 120.

In example embodiments, the processor 130 may appropriately preprocess images of image data as input to an artificial intelligence model.

Functions related to artificial intelligence according to the present disclosure are operated through the processor 130 and memory 120. The processor 130 may be comprised of one or multiple processors. At this time, one or more processors may be a general-purpose processor such as a CPU, AP, or DSP (Digital Signal Processor), a graphics-specific processor such as a GPU or VPU (Vision Processing Unit), or an artificial intelligence-specific processor such as an NPU. One or more processors control input data to be processed according to predefined operation rules or artificial intelligence models stored in the memory 120. Alternatively, when one or more processors are dedicated artificial intelligence processors, the artificial intelligence dedicated processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

Predefined operation rules or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that the basic artificial intelligence model is learned using a large number of learning data by a learning algorithm, thereby creating a predefined operation rule or artificial intelligence model set to perform the desired characteristics (or purpose). It means burden. This learning may be performed on the device itself that performs the artificial intelligence according to the present disclosure, or may be performed through a separate server and/or system. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the examples described above.

An artificial intelligence model may be composed of multiple neural network layers. Each of the plurality of neural network layers has a plurality of weight values, and neural network calculation is performed through calculation between the calculation result of the previous layer and the plurality of weights. Multiple weights of multiple neural network layers can be optimized by the learning results of the artificial intelligence model. For example, a plurality of weights may be updated so that loss or cost values obtained from the artificial intelligence model are reduced or minimized during the learning process. Artificial neural networks may include deep neural networks (DNN), such as Convolutional Neural Network (CNN), Deep Neural Network (DNN), Recurrent Neural Network (RNN), Restricted Boltzmann Machine (RBM), Deep Belief Network (DBN), Bidirectional Recurrent Deep Neural Network (BRDNN), or Deep Q-Networks, etc., but are not limited to the examples described above.

According to an exemplary embodiment of the present disclosure, the processor 130 may implement artificial intelligence. Artificial intelligence refers to a machine learning method based on an artificial neural network that allows machines to learn by imitating human biological neurons. Methodology of artificial intelligence includes supervised learning, in which the answer (output data) to the problem (input data) is determined by providing input data and output data together as training data according to the learning method, and only input data is provided without output data. In unsupervised learning, in which the solution (output data) to the problem (input data) is not determined, and a reward is given from the external environment whenever an action is taken in the current state, , It can be divided into reinforcement learning, which conducts learning in the direction of maximizing these rewards. In addition, artificial intelligence methodologies can be divided according to the architecture, which is the structure of the learning model. The architecture of widely used deep learning technology is convolutional neural network (CNN) and recurrent neural network (RNN). , Transformer, generative adversarial networks (GAN), etc.

The devices and systems may include artificial intelligence models. An artificial intelligence model may be a single artificial intelligence model or may be implemented as multiple artificial intelligence models. Artificial intelligence models may be composed of neural networks (or artificial neural networks) and may include statistical learning algorithms that mimic biological neurons in machine learning and cognitive science. A neural network can refer to an overall model in which artificial neurons (nodes), which form a network through the combination of synapses, change the strength of the synapse connection through learning and have problem-solving capabilities. Neurons in a neural network can contain combinations of weights or biases. A neural network may include one or more layers consisting of one or more neurons or nodes. By way of example, a device may include an input layer, a hidden layer, and an output layer. The neural network that makes up the device can infer the result (output) to be predicted from arbitrary input (input) by changing the weight of neurons through learning.

The processor 130 creates a neural network, trains or learns a neural network, performs an operation based on received input data, and generates an information signal based on the performance result. Alternatively, neural network models can be CNN (Convolution Neural Network) such as GoogleNet, AlexNet, VGG Network, R-CNN (Region with Convolution Neural Network), and RPN (Region Proposal Network). ), Recurrent Neural Network (RNN), Stacking-based deep Neural Network (S-DNN), State-Space Dynamic Neural Network (S-SDNN), Deconvolution Network, Deep Belief Network (DBN), Restrcted Boltzman Machine (RBM), The processor 130 may include various types of models such as Fully Convolutional Network, LSTM (Long Short-Term Memory) Network, and Classification Network, but is not limited thereto. The processor 130 is one for performing operations according to neural network models. For example, a neural network may include a deep neural network.

Neural networks include CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), perceptron, multilayer perceptron, FF (Feed Forward), RBF (Radial Basis Network), DFF (Deep Feed Forward), and LSTM. (Long Short Term Memory), GRU (Gated Recurrent Unit), AE (Auto Encoder), VAE (Variational Auto Encoder), DAE (Denoising Auto Encoder), SAE (Sparse Auto Encoder), MC (Markov Chain), HN (Hopfield) Network), BM (Boltzmann Machine), RBM (Restricted Boltzmann Machine), DBN (Depp Belief Network), DCN (Deep Convolutional Network), DN (Deconvolutional Network), DCIGN (Deep Convolutional Inverse Graphics Network), GAN (Generative Adversarial Network) ), Liquid State Machine (LSM), Extreme Learning Machine (ELM), Echo State Network (ESN), Deep Residual Network (DRN), Differential Neural Computer (DNC), Neural Turning Machine (NTM), Capsule Network (CN), Those skilled in the art will understand that it may include any neural network, including, but not limited to, KN (Kohonen Network) and AN (Attention Network).

According to an exemplary embodiment of the present disclosure, the processor 130 is configured to operate a Convolution Neural Network (CNN), Region with Convolution Neural Network (R-CNN), Region Proposal Network (RPN), and RNN (such as GoogleNet, AlexNet, VGG Network, etc.). Recurrent Neural Network), S-DNN (Stacking-based deep Neural Network), S-SDNN (State-Space Dynamic Neural Network), Deconvolution Network, DBN (Deep Belief Network), RBM (Restrcted Boltzman Machine), Fully Convolutional Network, LSTM (Long Short-Term Memory) Network, Classification Network, Generative Modeling, eXplainable AI, Continual AI, Representation Learning, AI for Material Design, BERT for natural language processing, SP-BERT, MRC/QA, Text Analysis, Dialog System, Various artificial intelligence structures and algorithms such as GPT-3, GPT-4, Visual Analytics for vision processing, Visual Understanding, Video Synthesis, Anomaly Detection, Prediction, Time-Series Forecasting, Optimization, Recommendation, and Data Creation for ResNet data intelligence. It can be used, but is not limited to this.

Although not shown, the electronic device 100 may further include a communication unit. The communication unit may perform communication with at least one user terminal. At this time, the communication unit includes GSM (global system for mobile communication), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), UMTS (universal mobile telecommunications system), in addition to Wi-Fi module and WiBro (Wireless broadband) module. ), TDMA (Time Division Multiple Access), LTE (Long Term Evolution), 4G, 5G, 6G, etc. may include a wireless communication module that supports various wireless communication methods.

According to the above-described exemplary embodiments, damage and damage to plants, including Kalanchoe, can be prevented by identifying quality factors for each variety of Kalanchoe through an artificial intelligence model in a non-contact manner without directly contacting or destroying Kalanchoe. It can prevent the quality of plants from deteriorating and has the effect of improving the marketability of plants.

FIG. 2 is a flowchart illustrating a method of operating the electronic device 100 according to an exemplary embodiment of the present disclosure.

Referring to FIG. 2, a step (S100) of generating image data including images of kalanchoe and pollen is performed. In an exemplary embodiment, the image sensor 110 may generate image data including images of the kalanchoe and the pollen by photographing a pollen included in the kalanchoe. Additionally, the image sensor 110 may provide image data to the processor 130. In the present disclosure, the image data is exemplified as a two-dimensional RGB image, but is not limited to this and various imaging techniques such as image images taken based on heat, spectral images based on spectral spectra, and depth images including depth values can be used.

A step (S120) of learning an artificial intelligence model to determine the variety of Kalanchoe based on image data is performed. In an example embodiment, the processor 130 may train an artificial intelligence model to determine the variety of Kalanchoe based on image data. It is important to establish a standard for understanding the quality of Kalanchoe for each variety and to establish guidelines for each variety. Accordingly, the electronic device 100 collects frontal images of Kalanchoe, detects images of Kalanchoe, detects feature objects such as flower color and shape for each Kalanchoe variety, and identifies and determines the variety through an artificial intelligence model. You can. In an exemplary embodiment, the processor 130 extracts the flowerpot image included in the box by boxing the entire image of the flowerpot, detects features such as color and size of the flower in the flowerpot image, and detects the detected features. A dataset containing fields and labeled Kalanchoe varieties can be trained on an artificial intelligence model that determines Kalanchoe varieties.

A step (S130) of training an artificial intelligence model to predict and determine a plurality of quality factors of Kalanchoe based on image data is performed. In an exemplary embodiment, the processor 130 determines, based on the image data, the canopy of the kalanchoe, the number of flowers of the kalanchoe, the flowering rate of the kalanchoe, and the peduncle of the kalanchoe (the state in which the flowers are arranged on the flower axis, or flowers). Artificial intelligence models can be trained to determine multiple quality factors, including the proportion of kalanchoe plants (running shapes) and the amount of damage to kalanchoes due to pests.

In the present disclosure, the number of flowers is the sum of the number of flowered flowers and the number of unflowered flowers, and the flowering rate is the number of flowered flowers divided by the number of flowers and then multiplied by 100 (percentage).

A step (S140) of predicting and determining a plurality of quality factors is performed by inputting new image data into a trained artificial intelligence model. In an exemplary embodiment, the image sensor 110 may generate new image data and provide it to the processor 130, and the processor 130 may generate new image data received from the image sensor 110, By inputting it into an artificial intelligence model, multiple quality factors can be predicted and determined. In an exemplary embodiment, the prediction and determination results of quality factors predicted and determined by the artificial intelligence model may be output as scores. The scores will be explained in more detail through FIG. 9.

Figure 3 is a flowchart conceptually explaining the operation of an electronic device according to an exemplary embodiment of the present disclosure.

Referring to FIG. 3, a sensing device such as the image sensor 110 included in the electronic device 100 can sense kalanchoe. Kalanchoe is a succulent plant of the rose order Sedum family. Its scientific name is am.

According to an exemplary embodiment, the electronic device 100 can detect the flower color, flower shape, and leaf shape of Kalanchoe, and identify and determine the variety of Kalanchoe through calculations of the processor 130.

Figure 4 is a flowchart illustrating the steps of learning an artificial intelligence model for determining a canopy according to an exemplary embodiment of the present disclosure.

Referring to Figure 4, consumers do not prefer the length of the kalanchoe to be too long or too short. Unlike cut flowers, whose length can be adjusted by cutting, Kalanchoe is a crop whose length cannot be artificially adjusted during distribution, so the canopy, such as the length and width when Kalanchoe is shipped, can be an important quality factor for Kalanchoe. The electronic device 100 can measure the width and length of the Kalanchoe flower by detecting only the portion corresponding to the Kalanchoe flower in the image.

In an exemplary embodiment, receiving first image data including a frontal image in which the front of the kalanchoe is photographed (S210) is performed. In an exemplary embodiment, the processor 130 may receive first image data including a front image in which the front of the kalanchoe is photographed from the image sensor 110.

A step (S220) of detecting a frontal image from the first image data is performed. In an example embodiment, processor 130 may detect a frontal image from the first image data. At this time, the front image may be an image captured by the image sensor 110 of one side of the flowerpot including kalanchoe.

A detection step (S230) is performed to measure the width and length of the kalanchoe in the frontal image. In an example embodiment, the processor 130 may measure the width and length of the kalanchoe excluding the pollen in the front image.

Based on the learning data including the variety and canopy, a step (S240) of learning an artificial intelligence model that determines the canopy according to the variety is performed. In an exemplary embodiment, the processor 130 may train an artificial intelligence model that determines the canopy according to the variety, based on learning data including the variety and the canopy. According to this, when modeling an artificial intelligence model based on the value measured by the image sensor 110 spaced a certain distance away from the flower pot, there is an advantage of being able to obtain a value that is almost the same as the actual value.

According to an exemplary embodiment of the present disclosure, the image sensor 110 can not only fix a flower pot containing kalanchoe, but also rotate the entire flower pot and measure the appearance of the flower pot from all angles. Therefore, since the appearance at every angle is imaged, the maximum length, maximum width, etc. of the kalanchoe can be accurately measured.

According to an exemplary embodiment, the image sensor 110 may sense a plurality of image data and provide the information to the processor 130, and the processor 130 may use a function or algorithm to extract the maximum value from the sensed image data. The length and width of Kalanchoe, which has the maximum value among multiple pieces of image data, can be extracted by recalling from memory or through an artificial neural network model trained exclusively for image processing. In an exemplary embodiment, the processor 130 plots the length graph and the width graph for the length and width extracted from each of the plurality of image data, and plots the length graph and the width graph to calculate the maximum value. By applying first-order differentiation, the maximum length and maximum width can be calculated.

In an exemplary embodiment, the processor 130 may receive a plurality of first image data each including a front view of the kalanchoe photographed while the kalanchoe is rotating. Additionally, the processor 130 may measure the width and length of the front image included in each of the plurality of first image data. Additionally, the processor 130 may set the canopy including the maximum width and maximum length among the plurality of measured widths and lengths as learning data. According to this, there is an advantage in that the values can be read from each shooting viewpoint, and the maximum length and maximum width among a plurality of values can be measured more accurately.

In an exemplary embodiment, the electronic device 100 collects a frontal image, detects an image of the Kalanchoe, measures the width and length of the plant excluding the pollen, and determines the Kalanchoe's image based on the variety and characteristics through an artificial intelligence model. Quality can be evaluated and scored. The scores will be explained in more detail through FIG. 9.

According to the above-described exemplary embodiments, by photographing Kalanchoe in a non-contact manner and determining the quality of Kalanchoe without contacting or destroying plants containing Kalanchoe, favorable conditions, convenience, and marketability can be provided to farmers and farmers. There is a possible effect.

Figure 5 is a diagram showing a method for sensing the width and length of a kalanchoe according to an exemplary embodiment of the present disclosure.

Referring to FIG. 5 together with FIG. 4 , the electronic device 100 can sense a pot containing Kalanchoe and sense the width and height of the plant. For example, the horizontal length (plant width) of Kalanchoe may be 191.84 mm (millimeters), and the vertical length (plant height) may be 147.62 mm. The values from which multiple Kalanchoes are sensed can be plotted on a graph and a trend line of plant width can be fitted.

Figure 6 is a flowchart illustrating the steps of learning an artificial intelligence model for predicting and determining the number of Kalanchoe flowers and/or the flowering rate according to an exemplary embodiment of the present disclosure.

Referring to Figure 6, in general, more than approximately 100 flowers may be hung on Kalanchoe in one pot. The number of flowers (or number of flowers, amount of flowers) and flowering rate can be important quality factors in evaluating the quality of Kalanchoe. However, because the number of flowers is relatively large, images taken in a non-contact manner may be necessary to accurately determine these quality factors.

In an exemplary embodiment, a step S310 in which Kalanchoe receives a plurality of second image data including a top-view image is performed. In an exemplary embodiment, the processor 130 may receive a plurality of second image data including a top-view image of the kalanchoe captured from a top-view perspective from the image sensor 110. A step (S320) of detecting a plurality of top view images according to the flowering state of the flower is performed from the plurality of second image data. In an exemplary embodiment, the processor 130 may detect a plurality of top view images according to the flowering state of a flower from the plurality of second image data.

Based on learning data including each top-view image according to the flowering state, a step (S330) of learning an artificial intelligence model that predicts and determines the number of Kalanchoe flowers and flowering rate according to the variety is performed. In an exemplary embodiment, the processor 130 learns an artificial intelligence model that predicts and determines the number of Kalanchoe flowers and the flowering rate of Kalanchoe according to the variety, based on learning data including each top view image according to the flowering state. You can do it.

Meanwhile, when flowers bloom (that is, when flowers bloom), they bloom in various forms. Especially when flowers are shipped, they are still blooming, so not all flowers included in Kalanchoe may be blooming. In other words, the flowering states of all flowers included in Kalanchoe may be different. Therefore, it is necessary to individually train images of each stage of Kalanchoe flower blooming into an artificial intelligence model. In an exemplary embodiment, the processor 130 may detect a first top-view image according to non-blooming, a second top-view image according to partial flowering, and a third top-view image according to full flowering from the plurality of second image data. there is. In addition, the processor 130 can train the artificial intelligence model by using the first to third top view images as individual learning datasets. In an exemplary embodiment of partial flowering, partial flowering may be, but is not limited to, half of the total number of flowers being flowered.

In an exemplary embodiment, the processor 130 distinguishes and detects the first to third top-view images by performing a square boxing process for unflowered, half-flowered, and/or bloomed flowers in the plurality of second image data. , can be generated, and preprocessed. In an example embodiment, processor 130 may process blurred portions that are loopholes in the image.

Petals that are out of focus or outside the region of interest are generally treated as exceptions or interpreted as noise, but according to an exemplary embodiment of the present disclosure, objects such as petals that look like blur or noise are also labeled and sensed, so that they are used in the object recognition model. Accuracy can be further improved.

In an exemplary embodiment, the electronic device 100 collects images from a top view perspective, detects images of kalanchoes, selects objects by classifying them as unflowered, half-flowered, and fully bloomed, and displays the entire object through an artificial intelligence model. The number of flowers and flowering rate can be judged and scored. According to an exemplary embodiment of the present disclosure, flowers bloom in various forms, and when flowers are shipped, they are often not in full bloom, so image sensing is performed by individually learning flowering images for each stage of flower unfolding. accuracy can be improved. According to an exemplary embodiment of the present disclosure, the Kalanchoe flower pattern can be broadly divided into three stages in terms of shape change at each flowering stage, and each stage is a non-flowering state, a half-flowering (1/2) state, and a fully flowering state. It can be classified by status. As a result of performing image learning according to the present disclosure, it was confirmed that classification of the three flowering states improved the performance of the learning model.

According to the above-described exemplary embodiments, there is an effect of providing advantageous conditions, convenience, and marketability to farms and farmers by photographing kalanchoe in a non-contact manner and predicting and determining the quality of kalanchoe.

Figure 7 is a diagram illustrating a method for detecting images at each stage of flowering according to an exemplary embodiment of the present disclosure.

Referring to FIG. 7 together with FIG. 6 , the electronic device 100 may receive an image of Kalanchoe and perform object recognition for each flowering state. According to an exemplary embodiment, the electronic device 100 can detect petals from a kalanchoe petal image as a result of learning by classifying the flowering stage into non-flowering, partial flowering, and fully flowering states. For example, the electronic device 100 can create a bounding box and detect Kalanchoe petals one by one.

According to an exemplary embodiment, the electronic device 100 can label and sense objects such as flower petals that are out of focus or outside the region of interest, or that appear like blur or noise, further improving the accuracy of the object recognition model. You can do it.

Figure 8 is a flowchart illustrating the steps of learning an artificial intelligence model for predicting and determining the flower diameter ratio of Kalanchoe according to an exemplary embodiment of the present disclosure. 9 and 10 are diagrams schematically showing areas divided into concentric circles according to an exemplary embodiment of the present disclosure.

Referring to Figure 8, there may be multiple stems on which flowers hang, that is, a Kalanchoe contained in one flower pot. Additionally, because multiple flowers hang on a single stem, a relatively thin stem may not be able to support the weight of multiple flowers and may droop. When peduncles of equal proportions are located symmetrically, there may be relatively few cases of stems drooping or falling over. Therefore, the proportion of flowers when initially shipped can be included as an important quality factor.

In an exemplary embodiment, step S410 of Kalanchoe receiving third image data including a top-view image is performed. In an exemplary embodiment, the processor 130 may receive third image data from the image sensor 110 including a top view image of the kalanchoe captured from a top view perspective. A step (S420) of distinguishing images representing Kalanchoe flowers and leaves from the top view image of the third image data is performed. In an exemplary embodiment, the processor 130 may distinguish between a first image representing a Kalanchoe flower and a second image representing a Kalanchoe leaf in a top view image of the third image data. A step (S430) of measuring the area of a flower in an image representing a Kalanchoe flower is performed. In an example embodiment, processor 130 may measure the area of a flower in a first image representing a Kalanchoe flower. Based on the learning data including the area, a step (S440) of learning an artificial intelligence model that predicts and determines the flowering ratio of Kalanchoe according to the variety is performed. In an exemplary embodiment, the processor 130 may train an artificial intelligence model that predicts and determines the flower scape ratio of Kalanchoe according to the variety, based on learning data including the area.

In an exemplary embodiment, in order to distinguish between flowers and leaves in a top-view image and accurately detect and determine the area of a flower, it is necessary to determine the location as well as the amount of flowers. Accordingly, by dividing the inscribed circle (IC) and circumscribed circle (OC) set in the top view image into four parts and dividing the inscribed circle (IC) and circumscribed circle (OC) into a total of 8 areas, a more accurate ratio of the flower landscape can be measured. .

In an exemplary embodiment, the processor 130 may set an outermost circumscribed circle (OC) centered on the flowerpot and an inscribed circle (IC) in contact with the flower located closest to the center in the third image data, and the circumscribed circle ( By dividing the OC) and the inscribed circle (IC) into 4 each, 8 areas can be set, the partial area of the flower included in each of the 8 areas can be measured, and the area of the flower can be calculated based on the partial area. there is.

In an exemplary embodiment of the present disclosure, the reference point in the top view image is the center of the flower pot rather than the circumscribed circle (OC), and the center of the flower pot may be fixed. Therefore, judgment accuracy can be further improved compared to simply using the center of the circumscribed circle (OC) as a reference.

In an exemplary embodiment, the electronic device 100 collects images from a top view perspective, detects images of kalanchoes, detects flower landscapes, measures the flower landscape ratio, and determines and scores the flower landscape ratio through an artificial intelligence model. You can.

According to the above-described exemplary embodiments, there is an effect of providing advantageous conditions, convenience, and marketability to farms and farmers by photographing kalanchoe in a non-contact manner and predicting and determining the quality of kalanchoe.

Figure 11 is a flowchart illustrating the steps of learning an artificial intelligence model for determining the amount of damage to Kalanchoe caused by pests according to an exemplary embodiment of the present disclosure.

Referring to Figure 11, Kalanchoe can generally suffer more damage from pests than damage from diseases such as fungal diseases, bacterial diseases, etc. Because pests are relatively small, it can generally be difficult for humans to see them. Since damage caused by pests should not be delivered as supplies, it is very difficult for people to directly determine and select whether there is damage due to pests. Accordingly, the area damaged by pests and/or pests may correspond to an important quality factor of Kalanchoe, and it is necessary to determine the area damaged by pests and/or pests from the top view image.

In an exemplary embodiment, receiving fourth image data including a top view image of the kalanchoe (S510) is performed. In an exemplary embodiment, the processor 130 may receive fourth image data from the image sensor 110 including a top view image of the kalanchoe captured from a top view perspective.

A step (S520) of selecting images representing each of the first and second areas from the top view image of the fourth image data is performed. In an exemplary embodiment, the processor 130 selects a third image representing a first area damaged by pests and a fourth image representing a second area not damaged by pests in a top view image of the fourth image data. can be selected.

Based on learning data including an image representing the first area, a step (S540) of learning an artificial intelligence model that determines the amount of damage to Kalanchoe due to pests according to the variety is performed. In an exemplary embodiment, the processor 130 may train an artificial intelligence model that determines the amount of damage to Kalanchoe due to pests according to breed, based on learning data including a third image representing the first area.

In an exemplary embodiment, a step (S530) of selecting an image representing a pest from the top view image of the fourth image data may be further included. In an exemplary embodiment, the processor 130 may further select a fifth image representing a pest from the top view image and include the fifth image representing a pest in the learning data. In an exemplary embodiment for S540, the processor 130 determines the amount of damage to Kalanchoe due to pests according to the variety, based on learning data including a third image representing the first area and a fifth image representing the pests. You can learn an artificial intelligence model that does this.

In an exemplary embodiment, the electronic device 100 can collect images from a top view perspective, detect images of Kalanchoe, select pests and objects damaged by pests, and determine and score fixtures through an artificial intelligence model. there is.

Figure 12 is a diagram visually illustrating a process for detecting pest damage according to an exemplary embodiment of the present disclosure.

Referring to FIG. 12, the electronic device 100 may receive Kalanchoe image data and extract an object including an area damaged by pests. In an exemplary embodiment, the electronic device 100 may calculate the size and number of pest damage by processing the pest damage area as a bounding box.

100: electronic device
110: image sensor
120: memory
130: processor

Claims (10)

An image sensor that photographs a pollen containing kalanchoe and generates image data including images of the kalanchoe and the pollen;
A memory that stores one or more instructions; and
a processor executing the one or more instructions stored in the memory;
The processor,
Based on the image data received from the image sensor, an artificial intelligence model is trained to determine the variety of the Kalanchoe,
Based on the image data, predict a plurality of quality factors including the canopy of the kalanchoe, the number of flowers of the kalanchoe, the flowering rate of the kalanchoe, the ratio of peduncles of the kalanchoe, and the amount of damage to the kalanchoe due to pests. And training the artificial intelligence model to make a decision,
An electronic device, characterized in that the new image data received from the image sensor is input to the trained artificial intelligence model to predict and determine the plurality of quality factors. According to claim 1,
The processor,
Receiving first image data including a frontal image in which the front of the Kalanchoe is photographed from the image sensor,
Detecting the frontal image from the first image data,
Measure the width and length of the Kalanchoe excluding the flower pot in the front image,
An electronic device, characterized in that training the artificial intelligence model to determine the canopy according to the variety, based on learning data including the variety and the canopy. According to clause 2,
The processor,
Receiving a plurality of first image data each including a front view of the kalanchoe taken as the kalanchoe rotates,
Measure the width and the length in the front image included in each of the plurality of first image data,
An electronic device, characterized in that setting the canopy including the maximum width and maximum length among the plurality of measured widths and lengths as the learning data. According to claim 1,
The processor,
The Kalanchoe receives a plurality of second image data including a top-view image taken from a top-view perspective from the image sensor,
Detecting a plurality of top view images according to the flowering state of the flower from the plurality of second image data,
Based on learning data including each top view image according to the flowering state, the number of flowers of the Kalanchoe according to the variety and the flowering rate of the Kalanchoe are characterized by training the artificial intelligence model to predict and determine , electronic devices. According to clause 4,
The processor,
An electronic device, characterized in that detecting a first top-view image according to non-blooming, a second top-view image according to partial flowering, and a third top-view image according to complete flowering from the plurality of second image data. According to claim 1,
The processor,
The Kalanchoe receives third image data including a top view image taken from a top view viewpoint from the image sensor,
Distinguish between an image representing the flower of the Kalanchoe and an image representing the leaf of the Kalanchoe in the top view image of the third image data,
Measure the area of the flower in the image representing the Kalanchoe flower,
Characterized in that, based on learning data including the area, training the artificial intelligence model to predict and determine the ratio of flower shoots of the Kalanchoe according to the variety. According to clause 6,
The processor,
In the third image data, an outermost circumscribed circle centered on the flower pot and an inscribed circle in contact with the flower located closest to the center are set,
The circumscribed circle and the inscribed circle are each divided into four parts to establish eight regions,
Measure the partial area of the flower included in each of the eight regions,
An electronic device, characterized in that calculating the area of the flower based on the partial area. According to claim 1,
The processor,
The Kalanchoe receives fourth image data including a top view image taken from a top view viewpoint from the image sensor,
Selecting an image representing a first area damaged by the pest and an image representing a second area not damaged by the pest from the top view image of the fourth image data,
Characterized in that, based on learning data including the image representing the first area, training the artificial intelligence model to determine the amount of damage to the Kalanchoe caused by the pest according to the variety. According to clause 8,
The processor,
Further select images representing the pests from the top view image,
An electronic device, characterized in that an image representing the pest is included in the learning data. Photographing a flowerpot containing kalanchoe and generating image data including images of the kalanchoe and the flowerpot;
Based on the image data, training an artificial intelligence model to determine the variety of the Kalanchoe;
Based on the image data, predict a plurality of quality factors including the canopy of the kalanchoe, the number of flowers of the kalanchoe, the flowering rate of the kalanchoe, the ratio of peduncles of the kalanchoe, and the amount of damage to the kalanchoe due to pests. And training the artificial intelligence model to make a decision; and
A method of operating an electronic device, comprising inputting new image data into the trained artificial intelligence model to predict and determine the plurality of quality factors.