WO2019235776A1

WO2019235776A1 - Device and method for determining abnormal object

Info

Publication number: WO2019235776A1
Application number: PCT/KR2019/006506
Authority: WO
Inventors: 신제용; 김일봉; 류승훈
Original assignee: 엘지이노텍 주식회사
Priority date: 2018-06-07
Filing date: 2019-05-30
Publication date: 2019-12-12
Also published as: KR102615986B1; KR20190143518A

Abstract

According to an embodiment of the present invention, provided is an abnormal object determination device comprising: an environmental parameter information collection unit for collecting environmental parameter information from at least one sensor; an abnormal object data detection unit for detecting abnormal object data by using data of an image including a plurality of objects; a determination unit for determining, by using the environmental parameter information and the abnormal object data, a degree of suitability of abnormal object detection; and a control unit for adjusting environmental parameters, on the basis of the degree of suitability of abnormal object detection.

Description

An object determination method and method

The present invention relates to an apparatus and method for determining abnormalities.

Livestock raised in groups within a small kennel are very vulnerable to the spread of communicable disease. For example, legal epidemics such as foot-and-mouth disease and bird flu are spread through the air, so once they occur, the social costs of protection and prevention of infection are very high, and social anxiety about food is spreading rapidly. . If abnormal signs are found in the kennel, it is important to isolate diseased livestock as soon as possible to prevent the spread of disease.

For rapid and accurate disease detection in the kennel, a suitable environment for disease detection must be maintained. However, there is no systematic method for managing a kennel environment in order to determine the adequacy of detection of abnormal individuals, and the environment control in the kennel is only performed to determine the growth environment of the livestock.

An object of the present invention is to provide an apparatus and method for determining abnormalities for determining the suitability of an environment for detecting poultry diseases in a kennel.

According to one embodiment of the invention, the environmental parameter information collecting unit for collecting environmental parameter information from at least one sensor; An abnormal object data detector for detecting abnormal object data using image data including a plurality of objects; A determination unit to determine an ideal object detection suitability using the environment parameter information and the abnormal object data; And a controller for adjusting the environment parameter according to the abnormality object detection suitability.

The abnormal object data detector may include a first feature extractor that extracts a probability of existence of the plurality of objects in the image data, and a second feature extractor that extracts motion of the plurality of objects in the image data.

The determination unit may determine that the abnormality object detection suitability is good when the motion value is equal to or greater than a preset first movement reference value.

The determination unit may determine that the abnormality object detection suitability is not good when each environmental measurement value is out of each measurement reference range using the environment parameter information.

The determination unit may determine that the abnormality object detection suitability is not good when the motion value is less than the second preset movement reference value.

The determination unit may determine that the abnormality object detection suitability is not good when the range of a predetermined area abnormality of the entire area of the image data is indicated as a suspect area by using the abnormal object data.

The determination unit learns a correlation between the motion data and the abnormal object detection fitness using the motion data detected from the image data as an input layer and calculates the abnormal object detection fitness according to the motion value indicated in the motion data. It may include a neural network trained to be an output layer.

The determination unit learns a correlation between the abnormal object data and the abnormal object detection suitability using the abnormal object data detected from the image data as an input layer, and is calculated according to the abnormal object detection region shown in the abnormal object data. It may include a neural network that has been trained so that the anomaly detection suitability becomes an output layer.

The determination unit learns a correlation between the environmental parameter information and the abnormality object detection suitability using the environmental parameter information as an input layer, and outputs the abnormality object detection suitability calculated according to an environmental measurement value indicated in the environmental parameter information. It can include a neural network learned to be.

The control unit may output a command for controlling the environmental parameters in the kennel in accordance with the suitability of detecting abnormal objects.

According to one embodiment of the invention, the environmental parameter information collecting unit for collecting environmental parameter information from at least one sensor; An abnormal object data detector for detecting abnormal object data using image data including a plurality of objects; And a determination unit to determine an ideal object detection suitability using the environment parameter information and the abnormal object data, wherein the determination unit provides an abnormal object determination apparatus that derives an optimal environment parameter for the abnormal object detection suitability.

The determination unit may determine that the abnormality object detection suitability is not good when the range of the predetermined area abnormality in the entire area of the image data is indicated as a disease suspect area by using the abnormal object data.

The controller may further include a controller configured to output a command for controlling the environment parameter in the kennel according to the optimum environment parameter.

According to one embodiment of the invention, the step of collecting environmental parameter information from at least one sensor; Detecting abnormal object data using image data including a plurality of objects; Determining an anomaly detection accuracy using the environment parameter information and the anomaly entity data; And adjusting the environment parameter according to the suitability for detecting the abnormal object.

According to one embodiment of the invention, the step of collecting environmental parameter information from at least one sensor; Detecting abnormal object data using image data including a plurality of objects; And determining an ideal object detection suitability using the environment parameter information and the abnormal object data, and deriving an optimal environment parameter for the abnormal object detection suitability.

Apparatus and method for determining abnormalities of the inventors can determine the suitability of the environment for the detection of poultry diseases in the kennel.

In addition, the environment inside the kennel can be controlled to be suitable for detecting poultry diseases.

In addition, it is possible to detect abnormal objects having a high probability of disease on the image data photographed inside the kennel.

In addition, by detecting with a 24-hour camera can determine the suitability of the poultry raising environment in the kennel.

In addition, it is possible to identify abnormal behavior of poultry without the help of specialists.

In addition, it is possible to determine the suitability of the poultry farming environment through the deep learning camera system.

In addition, the manager can observe the situation of poultry in real time through a PC or smartphone.

In addition, the administrator can control the environment inside the kennel through the alarm, and can improve productivity through the steady management.

1 is a block diagram of an anomaly detection system according to an embodiment of the present invention.

2 is a block diagram of a learning system for detecting abnormal objects according to an embodiment of the present invention.

3 is a conceptual diagram of an apparatus for determining an anomaly according to an embodiment of the present invention.

4 is a block diagram of an apparatus for determining an anomaly according to an embodiment of the present invention.

5 is a flowchart illustrating an abnormal object detection operation of the apparatus for determining an abnormal object according to an embodiment of the present invention.

6 is a view for explaining the principle of the object density prediction network.

7 is a diagram illustrating an example of displaying abnormal entity data on a manager terminal.

8 is a diagram for explaining an example in which abnormal entity data is displayed for each block on a manager terminal.

9 is a block diagram of a controller included in the apparatus for determining an anomaly according to an embodiment of the present invention.

FIG. 10 is a diagram illustrating an abnormal object detection algorithm of the apparatus for determining an abnormal object according to an embodiment of the present invention.

FIG. 11 is a diagram illustrating a method of detecting an abnormal object by the apparatus for determining an abnormal object according to an embodiment of the present invention, using the result of re-learning by the learning server.

12 is a flowchart illustrating a method for determining an anomaly according to an embodiment of the present invention.

13 is a block diagram of an apparatus for determining an anomaly according to another embodiment of the present invention.

14 is a flowchart illustrating a method for determining an anomaly according to another embodiment of the present invention.

As the invention allows for various changes and numerous embodiments, particular embodiments will be illustrated and described in the drawings. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

Terms including ordinal numbers, such as second and first, may be used to describe various components, but the components are not limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the second component may be referred to as the first component, and similarly, the first component may also be referred to as the second component. The term and / or includes a combination of a plurality of related items or any item of a plurality of related items.

When a component is said to be "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that another component may be present in the middle. Should be. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that there is no other component in between.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present disclosure does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art and shall not be construed in ideal or excessively formal meanings unless expressly defined in this application. Do not.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments will be described in detail with reference to the accompanying drawings, and the same or corresponding components will be given the same reference numerals regardless of the reference numerals, and redundant description thereof will be omitted.

1 is a block diagram of an abnormal object detection system according to an embodiment of the present invention, FIG. 2 is a block diagram of a learning system for detecting an abnormal object according to an embodiment of the present invention, and FIG. 3 is according to an embodiment of the present invention. It is a conceptual diagram of the abnormality object discrimination apparatus.

1 to 3, an abnormal object detection system 1000 according to an exemplary embodiment of the present invention includes an abnormal object determining apparatus 100, an administrator terminal 200, an air conditioning apparatus 300, and a learning server 400. do.

The learning system 2000 for detecting an abnormal object according to an exemplary embodiment of the present invention includes a plurality of abnormal object determining apparatus 100 and a learning server 400. Here, the plurality of abnormal object determination apparatus 100 may be a plurality of abnormal object determination apparatuses installed in one kennel, or may be a plurality of abnormal object determination apparatuses installed in the plurality kennels.

The abnormality object determining apparatus 100 may detect an environment in the kennel 10 and transmit it to at least one of the manager terminal 200 and the air conditioning apparatus 300. Here, the kennel 10 means a livestock breeding barn. The livestock may be not only poultry such as chickens and ducks, but also various kinds of animals bred in groups such as cattle and pigs.

The abnormal object determination apparatus 100 extracts abnormal object data in the kennel 10. Here, the abnormal object data may include at least one of information on whether there is an abnormal object, information on a spatial area in which the abnormal object exists, and information on a time domain in which the abnormal object exists. In this case, the abnormal subject may refer to an individual who is not in a normal state due to a disease, pregnancy, or the like.

As illustrated in FIG. 3A, the abnormality object determining apparatus 100 may be arranged for each breeding ground 10. The object determining apparatus 100 may include a plurality of photographing units 111, and the plurality of photographing units 111 may be disposed at various places in the kennel 10. For example, the plurality of photographing units 111 may be disposed at the upper and side portions of the kennel 10. The abnormal object determination apparatus 100 may extract the abnormal object information by collecting a plurality of image data acquired by the plurality of photographing units 111.

Alternatively, as illustrated in FIG. 3B, a plurality of abnormality object determining apparatuses 100 may be disposed in one kennel 10. The plurality of abnormal object determination apparatus 100 may be disposed at various places in the kennel 10, and each abnormal object determination apparatus 100 uses the individual image data acquired by each photographing unit 111 to determine the abnormal object. You can also extract data. According to an exemplary embodiment of the present invention, the abnormality object determining apparatus 100 may communicate with the manager terminal 200 and the air conditioning apparatus 300 by wire or wirelessly. Here, although the abnormality object determination apparatus 100 is illustrated as communicating with the manager terminal 200 and the air conditioning apparatus 300, respectively, but is not limited thereto, and the abnormality object determination apparatus 100 may communicate with the manager terminal 200. In addition, the manager terminal 200 may communicate with the air conditioning apparatus 300.

The manager terminal 200 may be a personal computer (PC), a tablet PC, a mobile terminal, or the like, and may be mixed with a management server. When the abnormality object determining apparatus 100 transmits at least one of the environment and the abnormal object data in the breeding ground 10 to the manager terminal 200, the manager is connected to the breeding ground 10 through a screen output to the manager terminal 200. At least one of the environment and anomalous individual data within can be recognized. For example, when the abnormality object determining apparatus 100 captures an abnormal situation in the kennel 10 and transmits it to the manager terminal 200, the manager is connected to the kennel 10 through a screen output to the manager terminal 200. It can be recognized that an abnormal situation has occurred in the first place and can respond to the initial situation. Here, the abnormal situation may be, for example, the generation of diseased livestock, pregnancy of the livestock, growth of the livestock, the humidity in the kennel 10, temperature, more than the concentration of a specific molecule and the like.

The air conditioning apparatus 300 is a device for controlling the temperature of the kennel 10. When the abnormality object determination apparatus 100 captures a temperature abnormality in the kennel 10 and transmits it to the manager terminal 200, the manager generates a temperature abnormality in the kennel 10 through a screen output to the manager terminal 200. It can be recognized that, by controlling the air conditioning apparatus 300 can normalize the temperature in the kennel 10. Alternatively, when the abnormality object determination apparatus 100 detects a temperature abnormality in the kennel 10 and directly transmits it to the air conditioning apparatus 300, the air conditioner 300 may directly normalize the temperature in the kennel 10. For example, if the temperature in the kennel 10 is lower or higher than the temperature suitable for the livestock, the movement of the livestock tends to be slowed down. Accordingly, the abnormality object determining apparatus 100, the manager terminal 200, or the air conditioning apparatus 300 may detect a temperature abnormality in the kennel 10 and normalize the temperature in the kennel 10.

The air conditioning apparatus 300 may adjust the humidity of the kennel 10. When the humidity in the kennel 10 is abnormal, the air conditioning apparatus 300 may be controlled to normalize the humidity in the kennel 10.

The abnormality object determining apparatus 100 may transmit the training data to the remote learning server 400 and extract the abnormal object data by applying a parameter received from the training server 400 to the algorithm for detecting the abnormal object.

According to the exemplary embodiment of the present invention, the learning server 400 receives the training data from the plurality of abnormal object determination apparatus 100, and re-trains the training data to extract parameters. For example, the learning server 400 may learn the learning data using a deep learning technique, but is not limited thereto. The learning server 400 may learn the learning data using various techniques and extract parameters.

Here, the abnormality object determining apparatus 100 may be mixed with a local machine, and the learning server 400 may collect training data from a plurality of abnormal object determining apparatuses installed in a plurality of kennels.

The abnormality object determining apparatus 100 according to an exemplary embodiment of the present invention pre-processes the image data, selects the training data, and transmits only the selected training data to the training server 400. Accordingly, communication traffic between the abnormality object determining apparatus 100 and the learning server 400 can be reduced, and the amount of computation of the learning server 400 can be reduced.

1 to 4, the abnormality object determining apparatus 100 includes a photographing unit 111, a control unit 112, a communication unit 113, an environmental parameter information collecting unit 114, an abnormal object data detecting unit 115, The determination unit 116, the display unit 117, the user interface unit 118, the encoding unit 119, the database 120, the light source unit 121, the pan tilt unit 122, and the learning preprocessor 123 are included. do. However, according to the exemplary embodiment of the present disclosure, at least one of the photographing unit 111, the display unit 117, the user interface unit 118, the light source unit 121, and the pan tilt unit 122 may be omitted. Here, at least one of the control unit 111, the encoding unit 119, and the learning preprocessor 123 may be implemented by a computer processor or a chip, and the database 120 may be mixed with a memory and an environment parameter collection unit. The 114 and the communication unit 113 may be mixed with an antenna or a communication processor.

In addition, when the plurality of abnormal object determination apparatuses 100 are arranged in the kennel, the rest except for one or more abnormal object determination apparatuses may be configured to perform only some functions of the abnormal object determination apparatus. That is, one abnormal object determination device is set as a main device to collect the image data photographed by the remaining abnormal object determination device to detect an abnormal object and perform a function of communicating with a manager terminal, while the other abnormal object determination device is used. May be configured to perform only a function of transferring image data photographed through the photographing unit 111 to the main device.

The photographing unit 111 may include one or a plurality of photographing units. For example, the photographing unit 111 may include at least one upper photographing unit disposed above the kennel 10 and at least one side photographing unit disposed at the side of the kennel 10. Each of the upper photographing unit and the side photographing unit may be an IP camera that can communicate in a wired or wireless manner and transmit real-time images.

The photographing unit 111 may generate image data by photographing an image including a plurality of objects. In the embodiment of the present invention, a plurality of individuals may mean poultry farmed in a kennel. In the exemplary embodiment of the present invention, the image data photographed by the photographing unit 111 may be mixed with the original data, the original image, the captured image, and the like.

The photographing unit 111 may generate a plurality of image data using a plurality of images sequentially photographed. For example, the photographing unit 111 may generate first image data by capturing a first image including a plurality of objects, and generate second image data by capturing a second image including a plurality of objects. can do. Each of the first image and the second image may be an image continuously photographed in time, and one image data may mean a single frame. The photographing unit 111 may generate the first image data and the second image data by using the first image and the second image which are sequentially photographed.

The photographing unit 111 may be an image sensor photographing a subject using a complementary metal-oxide semiconductor (CMOS) module or a charge coupled device (CCD) module. At this time, the input image frame is provided to the CMOS module or CCD module in the photographing unit 111 through the lens, the CMOS module or CCD module converts the optical signal of the subject passing through the lens into an electrical signal (image data) and outputs it. do.

The photographing unit 111 may include a fisheye lens or a wide angle lens having a wide viewing angle. Accordingly, it is also possible for one photographing unit 111 to photograph the entire space inside the kennel.

In addition, the photographing unit 111 may be a depth camera. The photographing unit 111 may be driven by any one of various depth recognition methods, and the depth information may be included in the image photographed by the photographing unit 111. The photographing unit 111 may be, for example, a Kinect sensor. Kinect sensor is a depth camera of the structured light projection method, it is possible to obtain a three-dimensional information of the scene by projecting a pattern image defined using a projector or a laser, and obtains the image projected pattern through the camera. These Kinect sensors include infrared emitters that irradiate patterns using infrared lasers, and infrared cameras that capture infrared images. An RGB camera that functions like a typical webcam is disposed between the infrared emitter and the infrared camera. In addition to this, the Kinect sensor may further include a pan tilt unit 122 that adjusts the angle of the microphone array and the camera.

The basic principle of the Kinect sensor is that when the laser pattern irradiated from the infrared emitter is projected and reflected on the object, the distance to the surface of the object is obtained using the position and size of the pattern at the reflection point. According to this principle, the photographing unit 111 may generate the image data including the depth information for each object by irradiating the laser pattern to the space in the kennel and sensing the laser pattern reflected from the object.

The controller 111 controls the abnormal object determination apparatus 100 as a whole. For example, the controller 111 may perform an operation of the abnormality object determining apparatus 100 by executing a command stored in the user interface 118 or the database 120.

Alternatively, the controller 111 may control various operations of the abnormality object determining apparatus 100 using a command received from the manager terminal 200.

The controller 112 may control the photographing unit 111 to follow-up the specific region in which the abnormal object is located. The tracking shooting target may be set through the user interface 118 or may be set through a control command of the manager terminal. The control unit 112 controls the photographing unit 111 to track and photograph a specific area in which an abnormal object exists to generate image data so that continuous monitoring can be performed.

The controller 112 may control the pan tilt unit 122 of the abnormal object determination apparatus 100 to perform tracking shooting. The pan tilt unit 122 may control the photographing area of the photographing unit 111 by driving two motors, a pan and a tilt. The pan tilt unit 122 may adjust the directing direction of the photographing unit 111 to photograph a specific area under the control of the controller 112. In addition, the pan tilt unit 122 may adjust the directing direction of the photographing unit 111 to track a specific object under the control of the controller 111.

The controller 111 may adjust the environmental parameters in the kennel according to the suitability for detecting the abnormal object. If it is determined that the abnormality of object detection suitability is not good, the controller 111 may output a command for controlling at least one environmental parameter of temperature and humidity in the kennel to the air conditioning apparatus.

The communication unit 113 may perform data communication with at least one of the other abnormality entity determining device, the manager terminal 200, or the learning server 400. For example, the communication unit 113 may include a wireless LAN (WLAN), Wi-Fi, WiBro, WiMAX, World Interoperability for Microwave Access (Wimax), and HSDPA (High Speed Downlink). Data communication may be performed using telecommunication technologies such as Packet Access, IEEE 802.16, Long Term Evolution (LTE), and Wireless Mobile Broadband Service (WMBS).

Alternatively, the communication unit 113 may include Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), Zigbee, and Near Field Communication (NFC). In addition, as a wired communication technology, data communication may be performed using a short-range communication technology such as USB communication, Ethernet, serial communication, and optical / coaxial cable.

For example, the communication unit 113 may perform data communication with another abnormality object determining device using a short range communication technology, and perform data communication with the manager terminal 200 or the learning server 400 using a long distance communication technology. However, the present invention is not limited thereto, and various communication technologies may be used in consideration of various aspects of the kennel 10.

The communication unit 113 transmits the image data photographed by the imaging device 20 to the manager terminal 200, or transmits the abnormal object data extracted by the control unit 111 to the manager terminal 200, or the image data and the abnormality. The result of matching the individual data may be transmitted to the manager terminal 200. In addition, the communication unit 113 may transmit the image captured by the imaging apparatus 20 and a bitmap of the density of the object in the image to the learning server 400. Here, the bitmap transmitted to the learning server 400 may be a bitmap modified by the manager terminal 200.

The data transmitted through the communication unit 113 may be compressed data encoded through the encoding unit 119.

The environmental parameter information collecting unit 114 may collect environmental parameter information from at least one sensor. The sensor may include a temperature sensor, a humidity sensor, or the like disposed in the kennel.

The abnormal object data detector 115 may detect abnormal object data by using image data including a plurality of objects. The abnormal object data detection unit 115 generates motion data indicating movement and generates abnormal object data indicating a suspected area from the image data by using the image data including the object in the kennel 10. The abnormal object data indicates a suspected area on the image data. The abnormal object data indicates information about the presence or absence of an abnormal object in the image data, information about a spatial area in which the abnormal object exists in the image data, and a time domain in which the abnormal object exists in the image data. It may include at least one of the information. Here, the information about the spatial region in which the abnormal object exists in the image data may be coordinate information of the region in which the abnormal object exists or coordinate information of the abnormal object. In this case, the abnormal object data may be represented by region, block, or pixel, and the abnormal object data may further include a probability that there is an abnormal object by region, block, or pixel. At this time, the abnormal object data detection unit 115 may extract the abnormal object data by driving an algorithm previously stored in the database 120. The algorithm and parameters applied to the algorithm are stored by the learning server 400. The learned results may be extracted algorithms and parameters.

Also, the abnormal object data detector 115 receives an image photographed by the photographing unit 111, and an area in which no movement is detected for a predetermined time among objects in the input image, for example, a pixel in which no movement is detected. This masked image can be generated and output.

The determination unit 116 may determine the abnormality object detection suitability using at least one of the environmental parameter information and the abnormal object data. In the embodiment of the present invention, the suitability for detecting abnormal objects means whether the environment of a kennel for detecting an object representing an abnormal object is suitable among the individuals in the kennel, and determines the suitability of the environment for growth, development, and productivity of the individual. It is different from the concept.

The determination unit 116 may determine that the abnormality object detection suitability is good when the motion value is greater than or equal to the preset first movement reference value. The determination unit 116 may determine that the anomalous object detection suitability is good when the motion value detected on the image data using the motion data is equal to or greater than a preset first movement reference value. The determination unit 116 may determine that the abnormality object detection suitability is good when the motion value detected on the image data is greater than or equal to the preset first movement reference value regardless of the value of the environmental parameter information. In this case, the motion value may be calculated through calculation of the total amount of motion values, the average value, or the number of pixels in which motion is detected. The first movement reference value may be set based on an error probability in detecting an abnormal object that was previously performed. For example, the determination unit 116 may set the average value of the movement reference values calculated when the error probability is less than 10% when detecting the abnormal object as the first movement reference value. For example, in an embodiment, the first reference value may be set to 30% of the motion value of the normal individual.

The determination unit 116 may determine that the abnormality object detection suitability is not good when the motion value is less than the second preset movement reference value. The determination unit 116 may determine that the abnormality object detection suitability is not good when the motion value detected on the image data using the motion data is less than the preset second movement reference value. The determination unit 116 may determine that the abnormality object detection suitability is not good when the motion value detected on the image data is less than the preset second movement reference value regardless of the value of the environmental parameter information. In this case, the motion value may be calculated through calculation of the total amount of motion values displayed on the image data, the average value, or the number of pixels in which the motion is detected. The second movement reference value may be set based on an error probability in detecting an abnormal object that was previously performed. For example, the determination unit 116 may set the average value of the movement reference values calculated when the error probability is 10% or more when detecting the abnormal object as the second movement reference value. For example, in the embodiment, the second reference value may be set to 10% of the motion value of the normal individual.

The determination unit 116 may determine that the abnormality object detection suitability is not good when the abnormality range of the predetermined area is displayed as the suspect area using the abnormal object data. The determination unit 116 may determine that the abnormality detection suitability of the abnormal object is not good when the range of the predetermined area is displayed as the suspect area regardless of the value of the environmental parameter information. The determination unit 116 may determine that the activity of the individual is reduced by the influence of temperature and humidity when the abnormal object is detected in a region of a specific ratio or more. For example, in the exemplary embodiment, the preset area range may be set to 50% of the entire area.

The determination unit 116 may determine that the abnormality object detection suitability is not good when each environmental measurement value is out of each measurement reference range using the environmental parameter information. The determination unit 116 may determine that the abnormality object detection suitability is not good when each environmental measurement value is out of each measurement reference range according to the age of the individual. In an embodiment, the measurement reference range may be set to 20%.

Alternatively, the determination unit 116 may determine an ideal object detection suitability using a machine learning method. The determination unit 116 may learn the abnormal object detection result according to the motion value and the environmental parameter information as a training set, and then analyze the motion value and the environmental parameter information of the image data input thereafter to determine the abnormal object detection suitability.

The determination unit 116 may include a computer readable program. The program may be stored in a recording medium or a storage device that can be executed by a computer. A processor in a computer may read a program stored in a recording medium or a storage device, execute a program, that is, a trained model, calculate input information, and output a calculation result.

For example, the determiner 116 may be trained to determine anomaly detection accuracy based on the motion data. The input of the determination unit 116 may be operation data, and the output of the determination unit 116 may be an abnormality object detection suitability. The determination unit 116 uses the motion data detected from the image inside the kennel as an input layer, learns the correlation between the motion data and the abnormality object detection fitness, and calculates the abnormality object fitness for fitness calculated based on the motion value indicated in the motion data. It may include a neural network trained to be an output layer. This neural network is an example of a deep learning algorithm designed to indicate whether the environment of the kennel for detecting abnormal objects is appropriate. The neural network may be an algorithm for inputting operation data into a convolution network-based learning machine and then outputting data in which abnormality object suitability is distinguished by letters, numbers, and symbols. At this time, the motion data becomes an input layer of the neural network, and the neural network can learn a correlation between the motion data and the fitness for detecting an abnormal object. In addition, the output layer of the neural network may be motion data that is displayed such that the anomaly detection suitability is distinguished by letters, numbers, and symbols.

Alternatively, the determination unit 116 may be trained to determine an abnormality object detection suitability based on the abnormality object data. The input of the determination unit 116 may be abnormal object data, and the output of the determination unit 116 may be an abnormality object detection suitability. The determination unit 116 learns a correlation between the abnormal object data and the abnormal object detection suitability using the abnormal object data detected from the image inside the kennel as an input layer, and is calculated according to the abnormal object detection region shown in the abnormal object data. It may include a neural network that has been trained so that the anomaly detection suitability becomes an output layer. This neural network is an example of a deep learning algorithm designed to indicate whether the environment of the kennel for detecting abnormal objects is appropriate. The neural network may be an algorithm for inputting abnormal object data into a convolution network-based learning machine and then outputting data in which the abnormal object detection suitability is distinguished by letters, numbers, and symbols. At this time, the abnormal entity data becomes an input layer of the neural network, and the neural network may learn a correlation between the abnormal entity data and the anomaly detection accuracy. In addition, the output layer of the neural network may be motion data that is displayed such that the anomaly detection suitability is distinguished by letters, numbers, and symbols.

Alternatively, the determination unit 116 may be trained to determine an abnormality object detection suitability through the environmental parameter information. The input of the determination unit 116 may be environment parameter information, and the output of the determination unit 116 may be an abnormality object detection suitability. The determination unit 116 uses the environmental parameter information as an input layer, learns the correlation between the environmental parameter information and the abnormality object detection suitability, and outputs the abnormality object detection suitability calculated according to the environmental measurement value indicated in the environmental parameter information. It can include learned neural networks. This neural network is an example of a deep learning algorithm designed to indicate whether the environment of the kennel for detecting abnormal objects is appropriate. The neural network may be an algorithm for inputting environment parameter information into a convolution network-based learning machine and then outputting data in which the abnormality object detection suitability is distinguished by letters, numbers, and symbols. At this time, the environmental parameter information becomes an input layer of the neural network, and the neural network can learn a correlation between the environmental parameter information and the anomaly detection accuracy. In addition, the output layer of the neural network may be motion data that is displayed such that the anomaly detection suitability is distinguished by letters, numbers, and symbols.

In addition, the determination unit 116 may derive an optimal environmental parameter for anomaly detection. The determination unit 116 may derive a range of optimal environmental parameters for detecting the abnormal object by analyzing a correlation between at least one of the operation data, the abnormal object data, and the environmental parameter information and the fitness for detecting the abnormal object. At this time, the optimum environmental parameter may be a parameter for at least one of the temperature and humidity inside the kennel.

The determination unit 116 may calculate an optimal environmental parameter by analyzing a correlation with an environmental parameter corresponding to a case where the abnormality object detection suitability is good. That is, the determination unit 116 extracts the environmental measurement value included in the environmental parameter when the abnormality object detection suitability is good, and derives the optimal environmental parameter according to the correlation between the goodness of the abnormality object detection suitability and the environmental measurement value. can do.

The display unit 117 includes a liquid crystal display (LCD), a thin film transistor liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), and a flexible display. ), At least one of a 3D display and an e-ink display.

The display unit 117 may display at least one of image data and distribution data whose pixel value is adjusted through the control unit 111.

In addition, the display unit 117 may output the image data photographed by the imaging device 20 to the screen, or may output the result of detecting the image data and the abnormal object data to the screen.

In addition, the display unit 117 may output various user interfaces or graphic user interfaces on the screen.

The user interface 118 generates input data for controlling the operation of the abnormal object determination apparatus 100. The user interface unit 118 may include a keypad, a dome switch, a touch pad, a jog wheel, a jog switch, and the like. When the display unit 117 and the touch pad have a mutual layer structure and constitute a touch screen, the display unit 117 may be used as an input device in addition to the output device.

The user interface 118 may receive various commands for the operation of the abnormality object determining apparatus.

The encoding unit 119 encodes the image data photographed by the imaging device 20 or the processed image data processed through the control unit 111 into a digital signal. For example, the encoding unit 119 may encode image data according to H.264, H.265, Moving Picture Experts Group (MPEG), and Motion Joint Photographic Experts Group (M-JPEG) standards.

The database 120 may include a flash memory type, a hard disk type, a multimedia card micro type, and a card type memory (for example, SD or XD memory). , Magnetic memory, magnetic disk, optical disk, random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EPEPROM), programmable memory (PROM) Read-Only Memory) may include at least one storage medium. In addition, the abnormality object determining apparatus 100 may operate a web storage that performs a storage function of the database 120 on the Internet, or may operate in connection with the web storage.

The database 120 may store image data photographed by the imaging apparatus 20, and may store image data for a predetermined period of time.

In addition, the database 120 may store data and programs necessary for the abnormal object determination apparatus 100 to operate, and the controller 111 may store an algorithm and parameters applied thereto for extracting the abnormal object data. have.

In addition, the database 120 may store various user interfaces (UIs) or graphical user interfaces (GUIs).

The light source unit 121 may irradiate light in a direction directed by the control unit 111. For example, the light source unit 121 may include at least one laser diode LD and a light emitting diode LED. The light source unit 121 may irradiate light of various wavelength bands under the control of the controller 111.

For example, the light source unit 121 may irradiate light in the infrared wavelength band for night photographing. Alternatively, the light source unit 121 may irradiate light in the ultraviolet wavelength band for photochemotherapy of livestock in the kennel.

The learning preprocessor 123 extracts learning data to be used for learning by the learning server 400 from the image data acquired by the imaging apparatus 20. A detailed method of extracting the training data by the training preprocessor 123 will be described below.

5 is a flowchart illustrating an abnormal object determination operation of the apparatus for determining an abnormal object according to an embodiment of the present invention.

1 to 5, the abnormality object determining apparatus 100 obtains image data of an image obtained by photographing a plurality of objects together using the photographing unit 111 (S500), and the control unit 111 obtains the image. The abnormal object data is extracted from the video data (S502). Here, one image data may mean a single frame, and the photographing unit 111 may generate a plurality of image data by using images sequentially photographed. In this case, the controller 111 may extract the abnormal object data using a pre-stored algorithm and a parameter applied thereto. Here, the pre-stored algorithm is a motion among the objects using the first algorithm trained to display the area where the object is determined to be located in the image data and the second algorithm trained to detect the motion by using an optical flow. This may mean an algorithm for displaying an object without. Here, the first algorithm may use a real-time object detection method using an algorithm indicating an area in which an object exists, or the first algorithm may indicate distribution information of an object, that is, density information of an area in which an object is located. Can also be used.

6 is a diagram for describing an object density prediction network. Here, the object density prediction network is an example of a deep learning algorithm designed to display density information of an area where an object is located. The object density prediction network may be an algorithm for inputting an original image into a convolution network-based learning machine and then outputting a density image represented by a gray scale probability map. According to this, abnormal object information can be easily extracted even in a kennel environment in which objects having a similar shape, such as a chicken house, are accommodated at high density.

The object density prediction network may be learned using the original image and the density image, and may be learned by the training server of FIG. 11 to be described later.

As shown in FIG. 6, the object density prediction network may include at least one convolution network (layer).

The convolutional network may classify the feature points of the image using at least one feature map (W).

The convolutional network may improve the performance of the object density prediction network through at least one pooler and / or activator.

The object density prediction network further includes a concatenator, and the concatenator can concatenate the output results of at least one convolutional network, rearrange them, and output density (distribution) information of the object using feature points of the original image. have.

At least one feature map (W) may be trained (tuned) to output density information of an object in a training server to be described later with reference to FIG. 11.

In an embodiment of the present invention, the object density prediction network is configured such that the density image has a size (Width x Highet) equal to or similar to the original image, and the positions of each pixel (or block) correspond to each other, The value of the pixel may represent the probability that an object (eg, poultry) may exist in a corresponding pixel of the original image. For example, the higher the probability that an object exists in a pixel of the original image, the higher the value of the pixel of the density image may be.

The communication unit 113 of the abnormal object determination apparatus 100 transmits the abnormal object data acquired by the controller 111 to the manager terminal 200 (S504), and the manager terminal 200 outputs the abnormal object data (S506). ). Accordingly, the manager may recognize abnormal object data output to the manager terminal 200.

7 is a diagram illustrating an example of displaying abnormal entity data on a manager terminal. The manager terminal 200 may display an image as shown in FIG. 7C. That is, the manager terminal 200 includes an image obtained by matching the original image photographed by the imaging device 20 as illustrated in FIG. 7A and the abnormal object data extracted by the controller 111 as illustrated in FIG. 7B. Can be displayed. Here, the abnormal object data may be a chicken density estimation image, and may be displayed as a pseudo density 8 bit gray image, a probability map, or a bit map. In this case, the abnormal object data is not data indicating an abnormal probability for each object, but data representing an abnormal probability of an object, a body, or a head in a region, block, or pixel corresponding to each divided area, block, or pixel in the image data. to be.

If there is an error in the abnormal object data exposed to the manager terminal 200, the manager may transmit feedback information to the abnormal object determining apparatus 100 through the manager terminal 200 (S508). For example, when it is determined that the abnormal object determining apparatus 100 is an abnormal object even though it is not an abnormal object, or when the abnormal object determining apparatus 100 determines that the normal object is a normal object despite being an abnormal object, the manager terminal 200 ) May transmit feedback information indicating that there is an error in the abnormal object data to the abnormal object determining apparatus 100. Here, the feedback information may include at least one of a temporal domain and a spatial domain in which the error data is abnormal. The error time zone and the space zone may be selected or designated by an administrator, and may be input through a user interface unit of the manager terminal 200. For example, as illustrated in FIG. 8C, the manager terminal 200 displays

abnormal object data

802, 804, 806, and 808 for four areas of the screen 800, and the manager displays 4. If one or more of the abnormal object data (for example, 806) is determined to be an error, the administrator may select and feed back the abnormal object data 806 which is an error through the user interface. Alternatively, when the abnormal object data is not displayed despite the abnormal object in the screen output by the manager terminal 200, the manager may designate and feed back a time area and a spatial area that appear to be the abnormal object.

The learning preprocessor 123 of the object determining apparatus 100 extracts the training data using the feedback information received from the manager terminal 200 (S510) and transmits the extracted training data to the training server 400 ( S512).

Here, the training data may be part of the image data acquired in step S500. The learning preprocessor 123 of the object determining apparatus 100 may extract the image data according to the time domain and the spatial domain included in the feedback information received from the manager terminal 200 among the image data acquired in step S500. . For example, when the manager terminal 200 determines that there is an error in a part of the nth frame (eg, 806 of FIG. 8C), the training data is part of the nth frame (eg, FIG. 8). 806 of (c). Alternatively, when the manager terminal 200 determines that there is an error in the abnormal object data of the n + 3 th frame in the n th frame among the entire frames, the training data may be the n + 3 th frame in the n th frame of the entire frames. . Accordingly, the abnormality object determining apparatus 100 may extract only the image data having an error in detecting the abnormal object among all the image data acquired by the imaging apparatus 20 and transmit the image data to the learning server 400. Accordingly, communication traffic between the abnormality object determining apparatus 100 and the learning server 400 may be significantly reduced. Here, the training data may further include at least one of error information and correction information as well as extracted image data. The error information may be information indicating that a determination by the controller is wrong, and the correction information may be information indicating a direction to be corrected. Alternatively, the training data may include the extracted original image data and the abnormal object data after the error is corrected. For example, if it is determined that there is an error in some region (for example, 806 of FIG. 8C) in the n th frame to the n + 3 th frame, the training data is determined for the n th frame to the n + 3 th frame. Image data of the original and abnormal object data for the nth to n + 3th frames, and the abnormal object data for the nth to n + 3th frames may be included in some areas determined to be in error (eg, This may be a gray image after the error of 806 of FIG. 8C is corrected. The learning server 400 retrains using the training data received from the abnormal object determining apparatus 100 (S514), and accordingly transmits the update information for detecting the abnormal object to the abnormal object determining apparatus 100. (S516). Here, the update information may be a parameter applied to an algorithm for detecting an abnormal object, which may be in the form of an adjustable matrix.

Then, the abnormal object determination apparatus 100 obtains the image data of the image taken with the plurality of objects in step S500 (S518), the control unit 111 is obtained using the pre-stored algorithm and update information The abnormal object data is extracted from the image data (S520).

Accordingly, the abnormal object determination apparatus 100 may detect the abnormal object by reflecting the feedback information of the manager terminal 200, and may reduce communication traffic between the abnormal object determination apparatus 100 and the learning server 400. .

As illustrated in FIG. 2, when the learning server 400 communicates with the plurality of abnormal object determining apparatus 100, the learning server 400 may transmit the same update information to the plurality of abnormal object determining apparatus 100. have. Alternatively, since the individual object determining apparatus 100 may be in a different environment, the update information may be specific for the individual object determining apparatus 100.

According to an embodiment of the present invention, the learning server 400 also stores an algorithm such as an object density prediction network of the abnormality object determining apparatus 100, and the learning server 400 uses the same as an original image corresponding to the error data. It is desirable to relearn.

Hereinafter, a method of detecting an abnormal object by the abnormal object determining apparatus 100 according to an exemplary embodiment of the present invention will be described.

FIG. 9 is a block diagram of an abnormal object detecting unit included in the apparatus for determining an abnormal object according to an embodiment of the present invention. FIG. 10 is a diagram illustrating an algorithm for detecting an abnormal object of the apparatus for detecting an abnormal object according to an embodiment of the present invention. FIG. 11 is a diagram illustrating a method of detecting an abnormal object by the apparatus for determining an abnormal object according to an embodiment of the present invention, using the result of re-learning by the learning server.

Referring to FIG. 9, the abnormal object detector 115 may include a first feature extractor 600, a second feature extractor 602, and an abnormal object data generator 604.

The first feature extractor 600 extracts the position distribution of the plurality of objects in the image data of the image photographed by the photographing unit, and the second feature extractor 602 extracts the movement of the plurality of objects in the image data. do. The abnormal object data generator 604 may generate abnormal object data, for example, pixels, based on the position distribution extracted by the first feature extractor 600 and the movement extracted by the second feature extractor 602. Estimate the odds of a star anomaly.

In detail, the first feature extractor 600 may generate position distribution data indicating the position distribution of the plurality of objects using the image data. Here, the location distribution of the plurality of objects may mean a density distribution of objects for each location, and the location distribution data may be mixed with the density map. As described above, the first feature extractor 600 may use a real-time object detection method using a first algorithm trained to suggest an area in which the object is located in the image data, that is, an area suggestion algorithm. The first feature extractor 600 generates first position distribution data indicating a position distribution of the plurality of objects by using the first image data generated by including the plurality of objects, for example, and includes the plurality of objects. Second position distribution data representing a position distribution of a plurality of objects may be generated using the second image data generated by the method. Here, the first position distribution data and the second position distribution data may be position distribution data of image data generated in time series.

In the exemplary embodiment of the present invention, the position distribution data does not indicate individual individual positions but is data indicating a probability that an individual, a trunk, or a head may exist in a region or block corresponding to each divided region or block of the image data. The position distribution data may be a heat map expressing a probability that an object exists in each pixel in a different color.

In addition, the first feature extractor 600 may detect an animal object from the image data using the object detection classifier. At this time, the object detection classifier is trained by constructing a training DB from images of animal objects photographed by different postures or external environments of the animal object. The object detection classifier is a SVM (Support Vector Machine), a neural network, and an AdaBoost algorithm. Create a database of animal subjects through various learning algorithms, including In detail, the first feature extractor 600 detects an edge of the object corresponding to the foreground from the image data of the background in the previously-generated kennel, applies an edge of the foreground object detected from the image data, and edges of the foreground object. Animal objects can be detected by applying the object detection classifier to the area of the image data to which is applied.

For example, the first feature extractor 600 may be trained to detect an object in the captured image. The first feature extractor 600 may include a computer readable program. The program may be stored in a recording medium or a storage device that can be executed by a computer. A processor in a computer may read a program stored in a recording medium or a storage device, execute a program, that is, a trained model, calculate input information, and output a calculation result.

The input of the first feature extractor 600 may be one or a plurality of image data photographed inside the kennel, and the output of the first feature extractor 600 may be position distribution data from which an object is detected.

The first feature extractor 600 includes a first neural network trained to learn the correlation between the interior image of the kennel and the individual, using the image of the interior of the kennel as an input layer, and to output the position distribution data of the object. can do.

The first neural network is an example of a deep learning algorithm designed to display an area where an object is located on image data. The first neural network may be an algorithm for inputting image data to a convolution network-based learning machine and then outputting position distribution data in which an area where an object is located is distinguished. In this case, the image data photographed inside the kennel becomes an input layer of the first neural network, and the first neural network may learn the correlation between the kennel internal image data and the individual. The output layer of the first neural network may be position distribution data displayed to distinguish an area where an object is located on image data photographed inside the kennel.

The second feature extractor 602 may generate motion data indicating the movement of the motion object among the plurality of objects using the image data. Here, the motion data is not data indicating the movement of an individual object, but data indicating whether a motion exists in a corresponding area or block for each divided area or block of the image data, and the motion data may be mixed with the motion map. . The motion data may be data indicating whether a motion exists in a pixel corresponding to each pixel. As described above, the second feature extractor 602 may use a second algorithm trained to detect motion by using an optical flow. The second feature extractor 602 may detect movement at a specific point, a specific object, or a specific pixel on the distribution map by using single image data or a plurality of consecutive image data.

For example, the second feature extractor 602 generates first motion data indicating the movement of the motion object among the plurality of objects by using the first image data, and operates among the plurality of objects by using the second image data. Second motion data representing the movement of the object may be generated. Here, the first motion data and the second motion data may be motion data for a plurality of image data generated in time series.

The second feature extractor 602 may detect a movement of the moving object by using a dense optical flow method. The second feature extractor 602 may detect a motion of each pixel by calculating a motion vector for all pixels on the image data. In the case of the Dense Optical Flow method, since the motion vector is calculated for all pixels, the detection accuracy is improved, but the amount of computation is relatively increased. Therefore, the Dense Optical Flow method can be applied to a specific area where detection accuracy is very high, such as a kennel where an abnormal situation is suspected or a kennel with a large number of individuals.

Alternatively, the second feature extractor 602 may detect the movement of the moving object using a sparse optical flow method. The second feature extractor 602 may detect a motion by calculating a motion vector only on a part of a pixel that is easy to track a motion such as an edge in an image. The sparse optical flow method reduces the amount of computation, but only results for a limited number of pixels. Therefore, the sparse optical flow method may be applied to a kennel having a small number of individuals or to a specific area where the objects do not overlap.

Alternatively, the second feature extractor 602 may detect movement of the moving object by using block matching. The second feature extractor 602 may divide the image evenly or unequally, calculate a motion vector for the divided region, and detect motion. Block Matching reduces the amount of computation because it calculates the motion vector for each partition, but it can have a relatively low detection accuracy because it calculates the results for the motion vector for each region. Accordingly, the block matching method may be applied to a kennel with a small number of individuals or to a specific area where the objects do not overlap.

Alternatively, the second feature extractor 602 may detect the movement of the moving object by using a continuous frame difference method. The second feature extractor 602 may compare successive image frames for each pixel and calculate a value corresponding to the difference to detect motion. Since the Continuous Frame Difference method detects motion by using the difference between frames, the overall computational amount is reduced, but the detection accuracy of a large object or a duplicate object may be relatively low. In addition, the Continuous Frame Difference method can not distinguish between the background image and the moving object may have a relatively low accuracy. Therefore, the Continuous Frame Difference method may be applied to a kennel with a small number of objects or to a specific area where the objects do not overlap.

Alternatively, the second feature extractor 602 may detect a movement of the moving object by using a background subtraction method. The second feature extractor 602 detects motion by comparing successive image frames for each pixel in a state where the background image is initially learned, and calculating values corresponding to the differences. The background subtraction method is to learn the background image in advance so that the background image can be distinguished from the moving object. Therefore, a separate process of filtering the background image is required, thereby increasing the amount of computation but improving the accuracy. Therefore, the Background Subtraction method can be applied to a specific area where detection accuracy is very high, such as a kennel where an abnormal situation is suspected or a kennel with a large number of individuals. In the Background Subtraction method, the background image can be updated continuously.

Alternatively, the second feature extractor 602 may be trained to detect motion in the captured image. The second feature extractor 602 may include a computer readable program. The program may be stored in a recording medium or a storage device that can be executed by a computer. A processor in a computer may read a program stored in a recording medium or a storage device, execute a program, that is, a trained model, calculate input information, and output a calculation result.

The input of the second feature extractor 602 may be one or a plurality of image data photographed inside the kennel, and the output of the second feature extractor 602 may be motion data detected from the image data.

The second feature extractor 602 learns a correlation between the inside of the kennel image and the movement in the image using the image inside the kennel as an input layer, and the second neural trained so that the motion data according to the detected movement is the output layer. It may include a network.

The second neural network is an example of a deep learning algorithm designed to indicate an area in which motion exists in the image data. The second neural network may be an algorithm for inputting image data to a convolution network-based learning machine and then outputting data displayed to distinguish a region where a motion is detected. In this case, the image data photographed inside the kennel becomes the input layer of the second neural network, and the second neural network may learn the correlation between the kennel internal image data and the movement. In addition, the output layer of the second neural network may be motion data displayed to distinguish the area where the motion is detected from the image data photographing the interior of the kennel. The second feature extractor 602 may be configured according to an environment and an external setting in the kennel. Use appropriate methods to detect motion on the distribution map. The above motion detection method is just an example, and methods capable of displaying a region (for example, a pixel / block) in which a motion occurs in a frame may be used.

The process of generating position distribution data by the first feature extractor 600 and the process of generating motion data by the second feature extractor 602 may be performed simultaneously, in parallel, or sequentially. That is, the process of generating location distribution data by the first feature extractor 600 and the process of generating motion data by the second feature extractor 602 may be processed independently of each other.

The abnormal object data generation unit 604 may generate the abnormal object data representing the abnormal object by region, block, or pixel by comparing position distribution data and motion data of the image data by region, block, or pixel. The abnormal object data generation unit 604 may generate, for example, first abnormal object data representing the abnormal object by comparing the first position distribution data and the first motion data. The abnormal object data generation unit 604 may generate first abnormal object data representing information on an object for which motion is not detected on the first position distribution data by comparing the first position distribution data with the first motion data. . That is, the abnormal object data generation unit 604 may estimate that the object has no disease detected on the first position distribution data indicating the position of the object, and generate the first abnormal object data. That is, the first abnormal object data may refer to data obtained by determining whether an object is a disease by using an object's position distribution and motion detection information for single image data. Alternatively, the abnormal object data generation unit 604 compares the position distribution data and the motion data of the plurality of image data to calculate the cumulative number of motion detection and non-detection motion of the plurality of objects, and calculates the cumulative number of motion detection and the number of motion detection. The abnormal object data may be generated according to the cumulative number of motion non-detections. The abnormal object data generation unit 604 may generate the second abnormal object data by comparing the first abnormal object data, the second position distribution data, and the second motion data, for example. The abnormal object data generation unit 604 compares the first abnormal object data with the second position distribution data and the second motion data to calculate the cumulative number of motion detection of the plurality of objects and the cumulative number of motion non-detection of the plurality of objects, and detect the motion. The second or more entity data may be generated according to the cumulative number of times and the motion non-detection cumulative number of times. That is, the second abnormal object data may mean data obtained by determining whether an object is a disease by using position information and motion detection information of the object accumulated with respect to the plurality of image data.

The abnormal object data generation unit 604 may control the pixel display of the plurality of objects on the image data according to the cumulative number of motion detection and non-movement accumulation of the abnormal object data. The abnormal object data generation unit 604 may control pixel display of the plurality of objects on the image data, for example, according to the second abnormal object data. Here, the display of the pixel may include all concepts for distinguishing and displaying a pixel corresponding to an arbitrary point from other pixels, such as the saturation of the pixel, the intensity of the pixel, the color of the pixel, the outline of the pixel, and the mark display. In the embodiment of the present invention, the display of the pixel may be controlled by adjusting the pixel value. The pixel value may be adjusted in stages, and in the case of a pixel having a high pixel value, the pixel value may be visually emphasized than a pixel having a low pixel value. However, the present invention is not limited thereto, and a pixel having a low pixel value may be set to be displayed to be highlighted than a pixel having a high pixel value. Herein, the pixel value may mean an abnormality probability of each pixel.

In the following description, one pixel represents one object in order to identify and identify the pixel and the object in which the motion is detected. This is for convenience of description and in practice, a plurality of pixels represent one object. That is, in order to determine an abnormal situation by detecting only movement of some body regions of poultry, a method of controlling the display of pixels by detecting movement for each pixel will be used.

The abnormal object data generation unit 604 may classify the object as an abnormal object as the movement of the specific object is not detected, and classify the pixel as a normal object as the motion is detected.

Here, each of the first feature extracting unit 600, the second feature extracting unit 602, and the abnormal object data generating unit 604 is the first update information, the second update information, and the third update information from the learning server 400. , And each of the first update information, the second update information, and the third update information may be provided to each of the first feature extractor 600, the second feature extractor 602, and the abnormal object data generator 604. Can be applied. The first update information, the second update information, and the third update information may be update information extracted by the learning server 400 as a result of learning the training data transmitted by the abnormality object determining apparatus 100. Here, the training data may include a part of the image data acquired by the abnormal object determining apparatus 100 and a part of the abnormal object data in which the error is corrected, which means that the abnormal object determining apparatus 100 uses the algorithm for detecting the abnormal object. It may be obtained using the feedback information of the manager terminal 200 with respect to the abnormal object data obtained by driving. Here, each of the first update information, the second update information, and the third update information may include an adjustable matrix.

Referring to FIG. 10, first, n-th image data is obtained. Here, the image data may be, for example, RGB data having a size of W X H (S1101). Here, the n th image data may be mixed with the n th original data, the n th original image, the n th original image data, and the like.

The controller 111 of the object determining apparatus 100, for example, the first feature extractor 600 detects an object from the nth image data and generates position distribution data of the object with respect to the nth image data (S1102). ). Here, the position distribution data may be generated for each region, for each block, or for each pixel, and the controller 111 may use the first algorithm trained to suggest a region in which the object is located in the image data. Here, the first algorithm may use a real-time object detection method using an algorithm indicating an area where an object exists as described above, or the first algorithm may include distribution information of an object, that is, an area of an area where an object is located. It is also possible to use a method of representing density information. Here, the position distribution data may be a first density map.

In the following description, a method of calculating or displaying anomalous object data using a pixel-by-pixel operation of image data will be described.

The update parameter μ value is applied to the position distribution data. Alternatively, the update parameter μ value and the offset parameter ε value may be applied at the same time. μ is a very small value, for example 0.001. That is, the position distribution data is controlled to be gradually displayed on the pixels only after accumulating for a long time. The offset parameter ε is a parameter for adjusting the accumulation of the position distribution data and may have a value of 1/10 to 1/100 of μ (S1103).

The controller 111 of the abnormality object determining apparatus 100, for example, the second feature extractor 602 detects the movement of the object with respect to the nth image data by comparing the n-1th image data with the nth image data. do. In this case, the n−1th image data may be stored in the latch circuit or the buffer circuit. The controller 111 of the abnormality object determining apparatus 100, for example, the second feature extracting unit 602, generates motion data according to the detected movement (S1104). Here, the motion data may be a motion map. Here, the motion data may be generated for each region, for each block, or for each pixel, and the controller 111 may use a second algorithm trained to detect motion by using an optical flow. The update parameter κ may be applied to the operation data. κ is a parameter for adjusting the accumulation of motion data (S1105).

The control unit 111 of the abnormal object determining apparatus 100, for example, the abnormal object data generating unit 604 adds the n-th or more object data to the position distribution data of the n-th image data (S1106), and the n-th image. The abnormal object data for the n-th image data is generated by subtracting the motion data of the data (S1107). Here, the abnormal object data for the nth image data may be an nth abnormal object density map.

The apparatus 100 for determining an abnormal object according to an exemplary embodiment of the present invention repeats steps S1101 to S1107 to display the color of the detected object lightly or close to the original color, and accumulates the color of the detected object. Can be controlled to display dark or close to red.

Thereafter, the n-th or more entity data may be matched on the n-th image data, that is, the n-th original data, and the image in which the n-th or more entity data is matched on the n-th original data may be displayed on the manager terminal 200. have. That is, the region where the abnormal object is located in the n-th image may be masked using the n-th or more object density map, and the masked image may be displayed on the manager terminal 200.

The operation of the abnormality object determining apparatus 100 according to the exemplary embodiment of the present invention may apply the following Equation 1.

[Equation 1]

Pixel _t = Pixel _t -1 (1-μ) + μW _t -F _t

Μ in Equation 1 may be changed according to a setting as an update parameter.

In Equation 1, Pixel _t and Pixel _t ₋₁ are abnormal object data and may indicate concentration of a pixel as a value for indicating whether a abnormal object exists in the pixel. The higher the probability that an abnormal object exists, the darker the color can be displayed. For example, if there is no probability of anomalies, it is displayed in primary color (white). The higher the probability of anomaly is expressed, the closer it is to red. To make it happen. Therefore, Pixel _t and Pixel _t ₋₁ may be set to have a value between 0 and 1, and the closer to 0, the closer to the primary color (white), and the closer to 1, the red.

In Equation 1, Pixel _t _- ₁ is abnormal object data of a previous frame in which position distribution data and motion data are accumulated. In Equation 1, Pixel _t is abnormal object data updated by applying position distribution data and motion data of the current frame.

In Equation 1, W _t may be position distribution data of a current frame. The position distribution data may have a value between 0 and 1 as a probability that an object exists in a corresponding pixel. The update parameter μ may be applied to the position distribution data. μ is a very small value, for example 0.001. That is, the position distribution data is controlled to be gradually displayed on the pixels only after accumulating for a long time.

In Equation 1, F _t may be motion data of the current frame. The motion data is an absolute value of the motion vector and may have a value of 0 or more. Since the magnitude of the motion vector corresponds to the velocity of the object, it may have a value of 1 or more. Since a separate parameter is not reflected in the motion data, when the motion is detected in the pixel, the display of the pixel is controlled to be initialized.

Alternatively, the operation of the abnormality object determining apparatus according to the embodiment may apply the following Equation 2.

[Equation 2]

Pixel _t = Pixel _t _-1 (1-μ + ε) + μW _t -F _t

※ μ: update parameter, ε: offset parameter

In Equation 2, an offset parameter ε is added in Equation 1, and the same description as in Equation 1 is omitted. The offset parameter ε is a parameter for adjusting the accumulation of the position distribution data and may have a value of 1/10 to 1/100 of μ.

Alternatively, the operation of the abnormality object determining apparatus according to the embodiment may apply the following Equation 3 or Equation 4.

[Equation 3]

Pixel _t = Pixel _t _-1 (1-μ) + μW _t -κF _t

[Equation 4]

Pixel _t = Pixel _t _-1 (1-μ + ε) + μW _t -κF _t

※ μ: update parameter, ε: offset parameter, κ: update parameter

Equation 3 or 4 is a product of the operation data F _t multiplied by the update parameter κ and the same content as in Equation 1 and Equation 2 is omitted. constant κ is a parameter that adjusts the accumulation of motion data.

Alternatively, the operation of the abnormality object determining apparatus according to the embodiment may apply the following Equation 5.

[Equation 5]

max (1, 2, 3, or 4, 0)

Equation 5 is an equation for preventing the value of Equations 1, 2, 3 or 4 from falling below zero. For example, the size of the motion data Ft is greater than the sum of other parameter values, and when the value of Equation 1, 2, 3 or 4 becomes a negative number less than 0, it is corrected to be displayed as 0. Control method.

Here, the first update information, the second update information, and the third update information received from the learning server 400 may be applied during the processes S1101 to S1107. In order to obtain the first update information, the second update information, and the third update information, the abnormality object determining apparatus 100 according to the embodiment of the present invention may extract the training data and transmit the training data to the training server 400. In this case, the training data may include a part of the image data acquired by the abnormal object determination apparatus 100, which is feedback from the abnormal object data extracted by the abnormal object determination apparatus 100 by driving an algorithm for detecting the abnormal object. Can be obtained using the information. Here, the feedback information may include information modified by the manager terminal 200.

More specifically, referring to FIG. 11A, the abnormality object determining apparatus 100 according to an exemplary embodiment of the present invention may extract training data and transmit the training data to the training server 400, wherein the training server 400 is abnormal. The object density prediction network may be retrained, that is, updated using the training data received from the object determining apparatus 100. Here, the training data may include image data indicated as having an error and abnormal object data in which the error is corrected. For example, when it is detected that the abnormal object exists even though the abnormal object does not exist in the partial region 806 in the n th frame, the training data is the image data of the n th frame and the abnormal object data of the n th frame. The abnormality of the partial region 806 may include the object data. Here, the abnormality object data having the error corrected may be information in which the density of the object is corrected by the manager terminal 200, and the density distribution of the object for each region, block, or pixel, extracted by the abnormal object determination apparatus 100, may be determined. In the case of a first density map, a density distribution of an object for each region, block, or pixel modified by the manager terminal 200 may be referred to as a second density map. The learning server 400 compares the output image of the object density prediction network in the learning server 400 with the error-corrected image included in the training data, that is, the correct answer image, to obtain a loss, and to minimize the loss. In the trainer, variables (eg, feature maps) of the object density prediction network can be learned (corrected).

Subsequently, referring to FIG. 11B, the control unit 111 of the abnormality object determining apparatus 100 generates the position distribution data, that is, in step S1102, the object density network re-learned by the learning server 400. It is available. That is, the control unit 111 of the abnormality object determining apparatus 100 uses the learning data from the learning server 400, for example, the n-th image using update information obtained by re-learning using the n-th image and the second density map. The abnormal object density map for the subsequent predetermined image may be output.

The environmental parameter information collecting unit collects environmental parameter information from at least one sensor (S1201).

The abnormal object data detector generates position distribution data and motion data using image data including a plurality of objects (S1202).

The abnormal object data detection unit compares the position distribution data and the motion data to generate abnormal object data indicating the abnormal object (S1203).

The determination unit determines the abnormality object detection suitability using the environmental parameter information and the abnormal object data. If the motion value detected on the image data using the motion data is greater than or equal to the preset first motion reference value, the determination unit determines that the abnormality object detection suitability is good regardless of the environmental parameter information. Alternatively, when the motion value detected on the image data using the motion data is less than the preset second movement reference value, the determination unit determines that the abnormality object detection suitability is not good regardless of the environmental parameter information. Alternatively, the determination unit may determine that the abnormality object detection suitability is not good regardless of the environmental parameter information when the abnormality range of the preset area is displayed as a suspect region using the abnormal object data. Alternatively, the determination unit determines that the abnormality object detection suitability is not good when each environmental measurement value is out of each measurement reference range using the environmental parameter information (S1204).

If it is determined that the abnormality object detection suitability is good, the control unit transmits the abnormal object data to the manager terminal (S1205).

If it is determined that the abnormality of object detection suitability is not good, the controller outputs a command for adjusting environmental parameters in the kennel. If it is determined that the abnormality of object detection suitability is not good, the controller outputs a command for controlling at least one environmental parameter of temperature and humidity in the kennel to the air conditioning apparatus (S1206).

13 is a block diagram of an apparatus for determining an anomaly according to an embodiment of the present invention.

As illustrated in FIG. 13, a plurality of imaging apparatuses 1000 may be disposed in one kennel 10.

Referring to FIG. 13, the abnormal object determining apparatus 1000 may include a photographing unit 1110, a controller 1120, a communication unit 1130, an environment parameter information collecting unit 1140, an abnormal object data detecting unit 1150, and a determining unit ( 1160, a display unit 1170, a user interface unit 1180, an encoding unit 1190, a database 1200, a light source unit 1210, a pan tilt unit 1220, and a learning preprocessor 1230. However, according to an exemplary embodiment of the present disclosure, at least one of the photographing unit 1110, the display unit 1170, the user interface unit 1180, the light source unit 1210, and the pan tilt unit 1220 may be omitted. Here, at least one of the controller 1110, the encoder 1190, and the learning preprocessor 1230 may be implemented by a computer processor or a chip, and the database 1200 may be mixed with a memory and an environment parameter collector. The 1140 and the communication unit 1130 may be mixed with an antenna or a communication processor.

The apparatus for determining an abnormality according to the embodiment of FIG. 13 may perform the same function as the apparatus for determining an abnormality according to the embodiment of FIGS. 4 to 11, and description thereof will be omitted.

In the embodiment of FIG. 13, the determination unit 1110 may derive an optimal environment parameter for anomaly detection suitability.

The determination unit 1110 may derive the environmental parameter according to the age of the individual in the kennel. The determination unit 1110 may derive an environmental parameter for controlling at least one of temperature and humidity so as to correspond to the measurement reference range when each environment measurement value is out of each measurement reference range. For example, the judging unit 1110 maintains the temperature in the kennel at 32 to 35 ° C. for the first week, and then lowers the temperature by about 3 ° C. per week thereafter, so that the final temperature is 21 to 22 ° C. at 21 to 24 days of age. Optimum environmental parameters can be derived to reach. At this time, the determination unit may derive an optimal environmental parameter so that the relative humidity is maintained between 70 to 80%.

Alternatively, the determination unit 1110 may derive an optimal environmental parameter for reducing at least one environmental parameter of temperature and humidity when the motion value detected on the image data is less than the preset second movement reference value. Under high temperature and humidity conditions, poultry is less active and reduces feed intake to reduce body heat production. Therefore, the determination unit 1110 may create an environment suitable for the abnormal detection environment by increasing the movement value of the individual by deriving an optimal environmental parameter for reducing at least one of temperature and humidity in the kennel.

Alternatively, the determination unit 1110 may derive an optimal environmental parameter for reducing at least one environmental parameter of temperature and humidity when an abnormal range of a predetermined area is displayed as a suspect area using the abnormal object data. . If the range of the suspect area is beyond the preset range, it means that the individual's activity is reduced compared to the normal environment. Therefore, the determination unit 1110 may create an environment suitable for the abnormal detection environment by increasing the movement value of the individual by deriving an optimal environmental parameter for reducing at least one of temperature and humidity in the kennel.

The controller 1120 may adjust the environment parameters in the kennel according to the optimum environment parameters. The controller 1120 may output a control command to the air conditioner so that at least one environmental parameter of the temperature and humidity in the kennel can follow the optimal environmental parameter.

14 is a flowchart illustrating an abnormal object determination method according to an exemplary embodiment of the present invention.

The environmental parameter information collecting unit collects environmental parameter information from at least one sensor (S1401).

The abnormal object data detector generates position distribution data and motion data using image data including a plurality of objects (S1402).

The abnormal object data detection unit compares the position distribution data and the motion data to generate abnormal object data indicating the abnormal object (S1403).

The determination unit determines the abnormality object detection suitability using the environmental parameter information and the abnormal object data. If the motion value detected on the image data using the motion data is greater than or equal to the preset first motion reference value, the determination unit determines that the abnormality object detection suitability is good regardless of the environmental parameter information. Alternatively, when the motion value detected on the image data using the motion data is less than the preset second movement reference value, the determination unit determines that the abnormality object detection suitability is not good regardless of the environmental parameter information. Alternatively, the determination unit may determine that the abnormality object detection suitability is not good regardless of the environmental parameter information when the abnormality range of the preset area is displayed as a suspect region using the abnormal object data. Alternatively, the determination unit determines that the abnormality object detection suitability is not good when each environmental measurement value is out of each measurement reference range using the environmental parameter information (S1404).

If it is determined that the abnormality object detection suitability is good, the control unit transmits the abnormality detection data to the manager terminal (S1405).

If it is determined that the abnormality detection suitability is not good, the determination unit derives an optimal environmental parameter for the abnormality detection suitability. The determination unit derives an optimal environmental parameter for controlling at least one of temperature and humidity so as to correspond to the measurement reference range when each environment measurement value is out of each measurement reference range. Alternatively, the determination unit derives an optimal environmental parameter for reducing at least one environmental parameter of temperature and humidity when the motion value detected on the image data is less than the preset second movement reference value. Alternatively, the controller derives an optimal environmental parameter for reducing at least one environmental parameter of temperature and humidity when the range of the predetermined area or more of the entire area of the image data appears as a suspect area using the abnormal object data (S1406).

The control unit adjusts the environmental parameters in the kennel according to the optimum environmental parameters. The controller outputs a control command to the air conditioner so that at least one environmental parameter of the temperature and humidity in the kennel can follow the optimal environmental parameter (S1407).

The term '~ part' used in the present embodiment refers to software or a hardware component such as a field-programmable gate array (FPGA) or an ASIC, and '~ part' performs certain roles. However, '~' is not meant to be limited to software or hardware. '~ Portion' may be configured to be in an addressable storage medium or may be configured to play one or more processors. Thus, as an example, '~' means components such as software components, object-oriented software components, class components, and task components, and processes, functions, properties, procedures, and the like. Subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided within the components and the 'parts' may be combined into a smaller number of components and the 'parts' or further separated into additional components and the 'parts'. In addition, the components and '~' may be implemented to play one or more CPUs in the device or secure multimedia card.

Although described above with reference to a preferred embodiment of the present invention, those skilled in the art will be variously modified and changed within the scope of the invention without departing from the spirit and scope of the invention described in the claims below I can understand that you can.

Claims

An environment parameter information collector configured to collect environment parameter information from at least one sensor;

An abnormal object data detector for detecting abnormal object data using image data including a plurality of objects;

A determination unit to determine an ideal object detection suitability using the environment parameter information and the abnormal object data; And

And a controller for adjusting the environmental parameter according to the abnormality detection suitability.
The method of claim 1,

The abnormal object data detection unit

A first feature extractor configured to extract a probability of existence of the plurality of objects in the image data;

And a second feature extractor configured to extract motions of the plurality of objects in the image data.
The method of claim 2,

And the determination unit determines that the abnormality object detection suitability is good when the motion value is equal to or greater than a preset first movement reference value.
The method of claim 2,

And the determination unit determines that the abnormality object detection suitability is not good when each environmental measurement value is out of each measurement reference range using the environment parameter information.
The method of claim 2,

And the determining unit determines that the abnormality object detection suitability is not good when the motion value is less than the second preset movement reference value.
The method of claim 5,

And the determining unit determines that the abnormality object detection suitability is not good when the range of the predetermined area abnormality in the entire area of the image data is indicated as a suspect area by using the abnormal object data.
The method of claim 1,

The determination unit learns a correlation between the motion data and the abnormality object detection fitness using the motion data detected from the image data as an input layer and calculates the abnormality object detection fitness degree calculated according to the motion value indicated in the motion data. An apparatus for determining an anomaly including a neural network learned to be an output layer.
The method of claim 1,

The determining unit learns a correlation between the abnormal object data and the abnormal object detection suitability using the abnormal object data detected from the image data as an input layer, and is calculated according to the abnormal object detection region shown in the abnormal object data. An abnormal object determination apparatus including a neural network trained so that the abnormal object detection suitability becomes an output layer.
The method of claim 1,

The determination unit learns a correlation between the environmental parameter information and the abnormality object detection suitability using the environmental parameter information as an input layer, and outputs the abnormality object detection suitability calculated according to the environmental measurement value indicated in the environmental parameter information. An apparatus for determining anomaly including a neural network learned to be.
The method of claim 1,

And the controller outputs a command for controlling an environmental parameter in the kennel in accordance with the suitability for detecting the abnormal object.
An environment parameter information collector configured to collect environment parameter information from at least one sensor;

An abnormal object data detector for detecting abnormal object data using image data including a plurality of objects; And

Determination unit for determining the abnormality object detection suitability using the environment parameter information and the abnormal object data,

And the determination unit derives an optimal environment parameter for anomaly detection accuracy.
The method of claim 11,

The abnormal object data detection unit

A first feature extractor configured to extract the existence probability of the plurality of objects in the image data;

And a second feature extractor configured to extract motions of the plurality of objects in the image data.
The method of claim 12,

And the determination unit determines that the abnormality object detection suitability is good when the motion value is equal to or greater than a preset first movement reference value.
The method of claim 12,

And the determination unit determines that the abnormality object detection suitability is not good when each environmental measurement value is out of each measurement reference range by using the environmental parameter information.
The method of claim 12,

And the determination unit determines that the abnormality object detection suitability is not good when the motion value is less than the second predetermined movement reference value.
The method of claim 11,

And the determining unit determines that the abnormal object detection suitability is not good when the range of the predetermined area abnormality in the entire area of the image data is indicated as a disease suspect area by using the abnormal object data.
The method of claim 11,

The determination unit

The determination unit learns a correlation between the motion data and the abnormality object detection fitness using the motion data detected from the image data as an input layer and calculates the abnormality object detection fitness degree calculated according to the motion value indicated in the motion data. An apparatus for determining an anomaly including a neural network learned to be an output layer.
The method of claim 11,

The determining unit learns a correlation between the abnormal object data and the abnormal object detection suitability using the abnormal object data detected from the image data as an input layer, and is calculated according to the abnormal object detection region shown in the abnormal object data. An abnormal object determination apparatus including a neural network trained so that the abnormal object detection suitability becomes an output layer.
The method of claim 11,

The determination unit learns a correlation between the environmental parameter information and the abnormality object detection suitability using the environmental parameter information as an input layer, and outputs the abnormality object detection suitability calculated according to the environmental measurement value indicated in the environmental parameter information. An apparatus for determining anomaly including a neural network learned to be.
The method of claim 11,

And a controller for outputting a command for controlling an environment parameter in the kennel according to the optimum environment parameter.
Collecting environmental parameter information from at least one sensor;

Detecting abnormal object data using image data including a plurality of objects;

Determining an anomaly detection accuracy using the environment parameter information and the anomaly entity data; And

And adjusting the environment parameter according to the abnormality detection suitability.
Collecting environmental parameter information from at least one sensor;

Detecting abnormal object data using image data including a plurality of objects; And

Determining suitability for detecting an abnormal object by using the environment parameter information and the abnormal object data,

And deriving an optimal environment parameter for the abnormality detection fitness.