KR102606062B1 - Apparatus and method for detecting abnormal object and imaging device comprising the same - Google Patents

Abstract

According to an embodiment of the present invention, a first feature extraction unit for generating position distribution data indicating the position distribution of the plurality of objects using image data generated from an image including the plurality of objects; a second feature extraction unit that generates motion data indicating movement of a motion object among the plurality of objects using the generated image data; and generate abnormal object candidate data representing an abnormal object candidate by comparing the first position distribution data and the first motion data generated during the first period, and the second position distribution data and the second motion data generated during the second period. and a control unit that compares the abnormal entity candidate data and generates abnormal entity data indicating an abnormal entity, wherein the first period is a period in which feeding and watering are not performed, and the second period is at least one of feeding and watering. Provides a device for detecting abnormal objects during which one is in progress.

Description

Abnormal object detection device and method, imaging device including the same {APPARATUS AND METHOD FOR DETECTING ABNORMAL OBJECT AND IMAGING DEVICE COMPRISING THE SAME}

Embodiments of the present invention relate to an abnormal object detection device and method, and an imaging device including the same.

As consumption of livestock products increases, various ways to manage livestock efficiently are being proposed.

Conventionally, people would check the condition of each livestock, and receive medical treatment or isolate livestock that showed abnormal signs. However, when the number of livestock is large, there is a problem that a lot of manpower is needed to manage all the livestock.

In addition, if an infectious disease occurs in some livestock, it may result in the death of most of the livestock, so there is a need for a method to quickly and accurately determine whether there is an abnormality in the livestock.

In addition, due to the development of means of transportation, diseases that occur in a specific region are not limited to a specific location and are spread nationwide in a short period of time, so quick disease diagnosis is required at an early stage. In determining disease in livestock, the temperature measurement method based on fever is used as a standard for determining the presence or absence of disease. However, because uniform temperature measurements are made for many livestock, there is difficulty in determining disease according to the condition of each livestock individual. In addition, this method of determining disease based on temperature measurement has difficulty detecting individuals in the early stages of disease occurrence.

The technical problem to be achieved by the present invention is to provide an abnormal object detection device and method capable of detecting objects likely to have a disease from image data captured inside a breeding farm, and an imaging device including the same.

According to an embodiment of the present invention, a first feature extraction unit for generating position distribution data indicating the position distribution of the plurality of objects using image data generated from an image including the plurality of objects; a second feature extraction unit that generates motion data indicating movement of a motion object among the plurality of objects using the generated image data; and generate abnormal object candidate data representing an abnormal object candidate by comparing the first position distribution data and the first motion data generated during the first period, and the second position distribution data and the second motion data generated during the second period. and a control unit that compares the abnormal entity candidate data and generates abnormal entity data indicating an abnormal entity, wherein the first period is a period in which feeding and watering are not performed, and the second period is at least one of feeding and watering. Provides a device for detecting abnormal objects during which one is in progress.

The position distribution data may include data indicating the presence or absence of an object as a probability for each block in the image data, and the motion data may include data indicating the presence or absence of motion for each block in the image data.

The abnormal entity candidate data may include a set of blocks in which there is a probability of existence of the entity in the first location distribution data but no movement is detected in the first motion data.

The abnormal entity data may include a set of blocks among the abnormal entity candidates in which there is a probability of existence of the entity in the second location distribution data but no movement is detected in the second motion data.

The control unit may distinguish the first period and the second period through a pre-stored feeding and water supply time table.

The control unit may receive a feeding and watering time table from the outside and distinguish between the first period and the second period.

The control unit may distinguish between the first period and the second period using the first operation data and the second operation data.

When an abnormal entity candidate is detected using the abnormal entity candidate data, the control unit may output a driving command to at least one of the feeder and the waterer between the first period and the second period.

The control unit may use the abnormal entity candidate data to output a driving command to at least one of the feeder and waterer assigned to the area where the abnormal entity candidate was detected.

The control unit may use the abnormal entity candidate data to output a driving command to at least one of the feeder and waterer assigned to an area excluding the area where the abnormal entity candidate was detected.

The first feature extraction unit may detect at least one of the feeding area and the water supply area from the image data generated during the second period.

The control unit may generate the abnormal entity data through a change in the width of at least one of the feeding area and the water supply area.

The control unit may output a drive command to adjust the height of at least one of the feeder and the waterer.

According to an embodiment of the present invention, a photographing unit for generating image data by photographing an image including a plurality of objects; a first feature extraction unit that generates position distribution data indicating the position distribution of the plurality of objects using the image data; a second feature extraction unit that generates motion data indicating movement of a motion object among the plurality of objects using the generated image data; and generate abnormal object candidate data representing an abnormal object candidate by comparing the first position distribution data and the first motion data generated during the first period, and the second position distribution data and the second motion data generated during the second period. and a control unit that compares the abnormal entity candidate data and generates abnormal entity data indicating an abnormal entity, wherein the first period is a period in which feeding and watering are not performed, and the second period is at least one of feeding and watering. One provides an imaging device, a period of time in progress.

According to an embodiment of the present invention, generating first image data by photographing an image including a plurality of objects during a first period of time; generating first position distribution data representing the position distribution of the plurality of objects using the first image data; generating first motion data representing movement of a motion object among the plurality of objects using the first image data; generating abnormal entity candidate data representing an abnormal entity candidate by comparing the first location distribution data and the first motion data; generating second image data by photographing images including a plurality of objects during a second period; generating second position distribution data representing the position distribution of the plurality of objects using the second image data; generating second motion data representing movement of a motion object among the plurality of objects using the second image data; and generating abnormal entity data representing an abnormal entity by comparing the second position distribution data and the second motion data with the abnormal entity candidate data, wherein the first period is when feeding and watering are not in progress. It is a period, and the second period is a period in which at least one of feeding and water supply is in progress.

The present inventors' abnormal object detection device and method, and an imaging device including the same, can detect abnormal objects that are highly likely to have a disease on image data captured inside a breeding farm.

Additionally, abnormal objects and normal objects can be visually distinguished and displayed.

Additionally, abnormal objects can be tracked and monitored.

Additionally, when an abnormal object occurs, a notification or warning can be sent to the administrator.

Additionally, the amount of calculation of analysis data can be reduced and calculation speed can be improved.

In addition, it detects the probability of existence of an object or its head or body for each pixel or specific area, and compares it with the motion detection result to detect abnormal objects, thereby improving the amount of calculation, speed, and accuracy compared to the method of tracking a specific object. can be expected.

Figure 1 is a conceptual diagram of a breeding farm according to an embodiment of the present invention.
Figure 2 is a block diagram of an imaging device according to an embodiment of the present invention.
Figure 3 is a diagram for explaining an abnormal object detection method according to an existing method.
Figure 4 is a diagram for explaining the operation of an abnormal object detection device according to an embodiment of the present invention.
5 to 10 are flowcharts of a method for detecting abnormal entities according to an embodiment of the present invention.
Figure 11 is a conceptual diagram of the configuration of a learning server according to an embodiment of the present invention.
12 to 15 are diagrams for explaining the operation of an abnormal object detection device according to an embodiment of the present invention.
Figures 16 and 17 are diagrams for explaining the operation of the control unit according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

However, the technical idea of the present invention is not limited to some of the described embodiments, but may be implemented in various different forms, and as long as it is within the scope of the technical idea of the present invention, one or more of the components may be optionally used between the embodiments. It can be used by combining and replacing.

In addition, terms (including technical and scientific terms) used in the embodiments of the present invention, unless explicitly specifically defined and described, are generally understood by those skilled in the art to which the present invention pertains. It can be interpreted as meaning, and the meaning of commonly used terms, such as terms defined in a dictionary, can be interpreted by considering the contextual meaning of the related technology.

Additionally, the terms used in the embodiments of the present invention are for describing the embodiments and are not intended to limit the present invention.

In this specification, the singular may also include the plural unless specifically stated in the phrase, and when described as "at least one (or more than one) of A and B and C", it is combined with A, B, and C. It can contain one or more of all possible combinations.

Additionally, when describing the components of an embodiment of the present invention, terms such as first, second, A, B, (a), and (b) may be used.

These terms are only used to distinguish the component from other components, and are not limited to the essence, sequence, or order of the component.

And, when a component is described as being 'connected', 'coupled' or 'connected' to another component, the component is not only directly connected, coupled or connected to that other component, but also is connected to that component. It can also include cases where other components are 'connected', 'combined', or 'connected' due to another component between them.

Additionally, when described as being formed or disposed "above" or "below" each component, "above" or "below" refers not only to cases where two components are in direct contact with each other, but also to one This also includes cases where another component described above is formed or placed between two components. In addition, when expressed as "top (above) or bottom (bottom)", it may include not only the upward direction but also the downward direction based on one component.

Figure 1 is a conceptual diagram of a breeding farm according to an embodiment of the present invention, and Figure 2 is a block diagram of an imaging device according to an embodiment of the present invention.

Referring to Figures 1 and 2, first, the breeding farm 10 refers to a barn for raising livestock. Livestock may include not only poultry such as chickens and ducks, but also various types of animals raised in groups in livestock farms, such as cows and pigs.

The imaging device 100 may detect the environment within the breeding farm 10 and transmit it to at least one of the management server 200 and the air conditioning device 300. To this end, the imaging device 100 may communicate with the management server 200 and the air conditioning device 300 by wire or wirelessly. Here, the imaging device 100 is shown to communicate with the management server 200 and the air conditioning device 300, respectively, but is not limited to this, and the imaging device 100 communicates with the management server 200, and the management server (200) may communicate with the air conditioning device (300).

A plurality of imaging devices may be disposed within the breeding farm 10. The imaging device 100 may transmit image data captured by the photographing unit 111 to the management server 200. Alternatively, a separate data collection device 150 may be placed in the breeding farm 10 to collect image data from a plurality of imaging devices 100 and transmit it to the management server 200. In order to precisely analyze the condition inside the breeding farm 10, the image data of the imaging device 100 may have high-definition and high-capacity characteristics. Therefore, in order to transmit image data captured by a plurality of imaging devices 100 to the remote management server 200 in real time, a large bandwidth may be required, and accordingly, the transmission speed may decrease. Therefore, the separate data collection device 150 collects image data from the imaging device 100 in the breeding farm 10, encodes it, and then transmits it to the management server 200, thereby reducing communication bandwidth and improving transmission speed. there is. Alternatively, the imaging device 100 may perform primary analysis to analyze the state of the breeding farm 10 using image data and then transmit the processed data to the data collection device or to the management server 200. That is, the imaging device 100 can efficiently manage communication resources by selecting only the data necessary for analyzing the state of the breeding farm 10 and transmitting it to the data collection device 150 or the management server 200. Alternatively, the imaging device 100 primarily analyzes the internal state of the breeding farm 10 using image data captured in the breeding farm 10 and transmits the results to the data collection device 150 or the management server 200. Can be transmitted.

The management server 200 may be a server including a personal computer (PC), tablet PC, mobile terminal, etc. When the imaging device 100 transmits the environment in the kennel 10 to the management server 200, the manager can recognize the environment in the kennel 10 through a screen output to the management server 200. For example, when the imaging device 100 captures an abnormal situation in the breeding farm 10 and transmits it to the management server 200, the manager can detect the abnormality in the breeding farm 10 through the screen displayed on the management server 200. You can recognize that a situation has occurred and respond early to abnormal situations. Here, the abnormal situation may be, for example, the occurrence of diseased livestock, pregnancy of livestock, growth period of livestock, humidity, temperature, concentration of a specific molecule, etc. in the breeding farm 10.

If the management server 200 is a central server that receives information from a plurality of breeding farms 10, the manager may not have easy accessibility to the management server 200. In this case, the management server 200 can transmit information about an abnormal situation within the breeding farm 10 through a separate manager's mobile terminal, etc.

Additionally, the management server 200 may analyze the condition of the breeding farm using data transmitted from the imaging device 100 or the data collection device 150. For example, the management server 100 may receive primarily processed data from a plurality of imaging devices 100 or data collection devices 150 to perform a more precise analysis of the state of the breeding farm. The management server 200 may perform data communication with the communication unit 115 of the imaging device 100 or may perform data communication with a communication module (not shown) mounted on the data collection device 150. The management server 200 may receive the original image captured by the imaging device 100 and the primarily processed image data. At this time, the processed image data may refer to data in which the kennel management device 100 or the imaging device 100 performs primary analysis on the original image and encodes it.

The air conditioning device 300 is a device that controls the temperature of the breeding farm 10. When the imaging device 100 captures a temperature abnormality in the breeding farm 10 and transmits it to the management server 200, the manager reports that a temperature abnormality has occurred in the breeding farm 10 through the screen displayed on the management server 200. It can be recognized, and the temperature in the breeding farm 10 can be normalized by controlling the air conditioning device 300. Alternatively, when the imaging device 100 detects a temperature abnormality in the breeding farm 10 and directly transmits the detected temperature to the air conditioning device 300, the air conditioning device 300 may directly normalize the temperature in the breeding farm 10. For example, if the temperature in the breeding farm 10 is lower or higher than the temperature appropriate for livestock living, the livestock's movement tends to slow down. Accordingly, the imaging device 100, the management server 200, or the air conditioning device 300 can detect a temperature abnormality within the breeding farm 10 and normalize the temperature within the breeding farm 10.

The air conditioning device 300 can control the humidity of the breeding farm 10. If an abnormality in humidity occurs in the breeding farm 10, the humidity in the breeding farm 10 can be normalized by controlling the air conditioning device 300.

The imaging device 100 includes a photographing unit 111, a first feature extraction unit 112, a second feature extraction unit 113, a control unit 114, a communication unit 115, a display unit 116, and a user interface unit 117. ), an encoding unit 118, a database 119, an illumination sensor 120, a light source unit 121, and a pan/tilt unit 122. In an embodiment of the present invention, an abnormal object detection device including a first feature extraction unit 112, a second feature extraction unit 113, a control unit 114, and a communication unit 115 is included in the imaging device 100. It is shown as being possible. However, the abnormal object detection device may be manufactured as a separate module and included in the data collection device 150 or the management server 200. Alternatively, the abnormal object detection device may be implemented as an independent product in the form of a separate device. When the abnormal object detection device is applied in the form of a module or device separate from the imaging device 100, the abnormal object detection device may receive image data from the imaging device 100 through the communication unit 115. Additionally, the first feature extraction unit and the second feature extraction unit may be included in the control unit and implemented with one processor. In an embodiment of the present invention, for convenience of explanation, the functions of the first feature extraction unit, the second feature extraction unit, and the control unit will be separately described.

Hereinafter, it will be explained by taking as an example that the abnormal object detection device is included in the imaging device 100.

A plurality of imaging devices 100 may be installed in the breeding farm 10. For example, the imaging device 100 may include at least one upper imaging device disposed on the upper part of the kennel 10 and at least one side imaging device disposed on the side of the kennel 10. Each of the upper imaging device and the side imaging device may be an IP camera capable of transmitting real-time images through wired or wireless communication. In this embodiment, the case where one imaging device 100 is placed at the top of the breeding farm will be described as an example.

Additionally, in an embodiment of the present invention, a plurality of individuals may refer to poultry raised inside a breeding farm.

The photographing unit 111 may generate a plurality of image data using a plurality of images sequentially photographed. For example, the photographing unit 111 may generate first image data by photographing a first image including a plurality of blocks, and generate second image data by photographing a second image including a plurality of blocks. can do. The first image and the second image may be images taken sequentially in time, and one image data may represent a single frame. The photographing unit 111 may generate first image data and second image data using first and second images that are sequentially photographed.

The capturing unit 111 may generate image data using images captured during a first period and generate image data using images captured during a second period. At this time, the photographing unit 111 generates a plurality of image data using a plurality of images sequentially photographed during the first period, and generates a plurality of image data using a plurality of images sequentially photographed during the second period. can be created. Additionally, the photographing unit 111 may generate image data using images captured during the third period.

In an embodiment of the present invention, the first period may mean a period in which feeding and watering are not in progress, and the second period may mean a period in which at least one of feeding and watering is in progress.

In an embodiment of the present invention, image data generated using images captured during a first period are defined as first image data, and image data created using images captured during a second period are defined as second image data. . The first image data and the second image data may mean single image data or a plurality of image data.

The image sensor may be an image sensor that photographs a subject using a complementary metal-oxide semiconductor (CMOS) module or a charge coupled device (CCD) module. At this time, the input image frame is provided to the COMS module or CCD module in the photographing unit 111 through the lens, and the COMS module or CCD module converts the optical signal of the subject that has passed through the lens into an electrical signal (image data) and outputs it. do.

The photographing unit 111 may include a fisheye lens or a wide-angle lens with a wide viewing angle. Accordingly, it is possible for one imaging device 100 to photograph the entire space inside the breeding farm 10.

Additionally, the imaging unit 111 may be a depth camera. The photographing unit 111 may be driven by any one of various depth recognition methods, and an image captured through the photographing unit 111 may include depth information. The photographing unit 111 may be, for example, a Kinect sensor. The Kinect sensor is a structured light projection depth camera that can obtain three-dimensional information of a scene by projecting a defined pattern image using a projector or laser and acquiring the projected image of the pattern through the camera. This Kinect sensor includes an infrared emitter that irradiates a pattern using an infrared laser, and an infrared camera that takes infrared images, and an RGB camera that functions like a general webcam is placed between the infrared emitter and the infrared camera. In addition, the Kinect sensor may further include a pan/tilt unit 122 that adjusts the angle of the microphone array and camera.

The basic principle of the Kinect sensor is that when a laser pattern emitted from an infrared emitter is projected and reflected on an object, the distance to the object surface is calculated using the position and size of the pattern at the reflection point. According to this principle, the imaging unit 111 can generate image data including depth information for each object by irradiating a laser pattern into the space within the breeding farm and sensing the laser pattern reflected from the object.

In an embodiment, the image data may be image data composed of N x M pixels.

In an embodiment, a block may include a single pixel of image data or an area including a plurality of pixels.

In the embodiment, the position distribution data does not indicate the positions of individual plural objects, but is data indicating the probability that an object, a body of an object, or a head of an object may exist for each block. In other words, position distribution data is data that represents the probability that an object, the body of an object, or the head of an object may exist for a single pixel or a plurality of pixels including a divided area.

The location distribution data may include data indicating the presence or absence of an object as a probability for each block in the image data. In an embodiment, the location distribution data may be a heatmap that expresses the probability that an object exists in each pixel using different colors divided into levels from 0 to 255.

The first feature extraction unit 112 may generate position distribution data indicating the location distribution of the plurality of objects using image data generated from an image including the plurality of objects.

The first feature extraction unit 112 may generate at least one first location distribution data for each first image data using at least one first image data generated during the first period.

Additionally, the first feature extraction unit 112 may generate at least one second location distribution data for each second image data using at least one second image data generated during the second period.

Additionally, the first feature extraction unit 112 may detect at least one of the feeding area and the water supply area from at least one image data generated during the second period.

The first feature extraction unit 112 detects the outline of the object from the image data, compares the detected outline with the outline of the animal object pre-stored in the database 119, and selects a block within the outline that matches the pre-stored outline of the animal object. can be detected as a block with a high probability of an entity existing. At this time, the appearance of the animal entity stored in the database 119 may be that of at least one animal entity, and the first feature extraction unit 112 detects a block with a matching outline as an entity as described above. At the same time, the type of animal can be determined.

In addition, for example, the first feature extraction unit 112 extracts the feature points of a block in the image data, and if the extracted feature point matches the feature point of an animal object previously stored in the database 119 with a proximity greater than a threshold value, the first feature extraction unit 112 extracts the feature point of the block in the image data. Blocks in image data can be detected as blocks with a high probability of an object existing. At this time, the first feature extraction unit 112 extracts feature points from the images of the block and object being compared, and uses SIFT (Scale Invariant Feature Transform) or SURF (Speeded Feature Transform) that matches feature point descriptors of the two extracted objects. Up Robust Features) algorithm can be used.

Additionally, for example, the first feature extraction unit 112 may detect an animal entity based on the outline of a block in image data. More specifically, the first feature extraction unit 112 detects the outline of a block in the image data to generate an edge image, and detects the outline from the foreground image data, which is the background image of the breeding farm, previously stored in the database 119. An edge image is generated, and the probability of the existence of an object can be detected from a difference image obtained by subtracting the background edge image from the edge image. At this time, the first feature extraction unit 112 generates an edge image by detecting the outline of an object appearing in the frame as an edge using gradient information of the image data frame. Here, gradient information is a value generated from difference values between adjacent pixels among predetermined pixels in a frame and means the sum of the absolute values of the differences, and edge means a boundary line between objects using gradient information.

Additionally, the first feature extraction unit 112 may generate a background edge image by detecting the edge of an object corresponding to the background from previously photographed image data of the foreground in the breeding farm. At this time, the background edge image may be an image in which the outlines of objects in a preset area are detected as background edges, but the same is repeated a predetermined number of times by comparing a plurality of image data frames of the foreground within the previously photographed breeding farm 10. It may be an image in which the outline of an object that appears is detected as a background edge.

Additionally, the first feature extraction unit 112 may detect the probability of entity existence in image data using an entity detection classifier. At this time, the object detection classifier is learned by building a training database from images of animal objects taken with different postures or external environments. This object detection classifier uses SVM (Support Vector Machine), neural network, and AdaBoost algorithm. A DB of animal entities is created through various learning algorithms, including: Specifically, the first feature extraction unit 112 detects the edge of an object corresponding to the foreground from the image data of the background in the previously captured breeding farm, applies the edge of the foreground object detected in the image data, and extracts the edge of the foreground object. The probability of the existence of an object can be detected by applying an object detection classifier to the area of the image data to which is applied. The first feature extraction unit 112 may learn a training set for at least one of the object, the object's head, and the object's body. When image data is input, the first feature extraction unit 112 may detect the probability of being an object or not an object for each block of the image data according to a learned algorithm. Alternatively, when image data is input, the first feature extraction unit 112 may detect the head of an object or the probability that it is not the head of an object for each block of the image data according to a learned algorithm. When image data is input, the first feature extraction unit 112 may detect the body of an object or the probability that it is not the body of an object for each block of image data according to a learned algorithm. In an embodiment of the present invention, location distribution data May be location distribution data for at least one of the head portion and body portion of a plurality of objects.

The first feature extraction unit 112 may generate distribution data for the head portion by detecting the probability distribution of the presence of the object head portion in a block of image data. In an embodiment of the present invention, the head part may refer to a set of pixels that constitute the head of an object. For example, if the object is a chicken, the head part may refer to the head including the crest.

Alternatively, the first feature extraction unit 112 may detect the probability distribution of the presence of an object body part in a block of image data and generate distribution data for the body part. In an embodiment of the present invention, the body part may refer to a set of pixels that constitute the body of an object. For example, if the object is a chicken, the body part may refer to the body including wings and chest.

The first feature extraction unit 112 may compare the outline of the image data block with the appearance of the head of the poultry pre-stored in the database 119 to detect the probability distribution of the presence of the head of the poultry.

Alternatively, the first feature extraction unit 112 may compare the outline of the image data block with the outline of the body part of the poultry previously stored in the database 119 to detect the probability distribution of the presence of the body part of the poultry.

Alternatively, if the extracted feature point matches the feature point of the poultry head part previously stored in the database 119 with a proximity greater than or equal to a threshold, the first feature extraction unit 112 may detect a block with a high probability of the head part being present in the corresponding image data. You can.

Alternatively, if the extracted feature point matches the feature point of the poultry body part previously stored in the database 118 with a proximity greater than or equal to a threshold, the first feature extraction unit 112 may detect a block with a high probability of the body part being present in the image data. You can.

In addition, the first feature extraction unit 112 reduces noise in the image data captured by the photographing unit 111, and performs gamma correction, color filter array interpolation, and color matrix. ), image signal processing to improve image quality, such as color correction and color enhancement, can be performed. The first feature extraction unit 112 may also perform color processing, blur processing, edge emphasis processing, image analysis processing, image recognition processing, and image effect processing.

The second feature extraction unit 113 may generate motion data representing the movement of a motion object among a plurality of objects using the generated image data.

The motion data may include data indicating the presence or absence of motion for each block in the image data. In the embodiment, the motion data does not indicate the movement of a plurality of individual objects, but is data indicating whether movement has been detected for each block. That is, motion data is data indicating whether movement exists for a single pixel or a plurality of blocks or pixels including a divided area. Motion data is not data obtained by detecting and tracking individual objects to determine whether movement has occurred, but refers to data that directly detects the block in which movement occurred in the image data. In other words, motion data does not detect whether motion has occurred in the position distribution data, but rather detects whether motion has occurred in the image data itself. It is data that detects whether motion has occurred independently of the probability of the existence of an object in the position distribution data. it means.

Motion data according to an embodiment is compared with a threshold value to express the presence or absence of motion.

Motion data according to embodiments may represent not only the movement of an object, but also movements other than the object.

The second feature extraction unit 113 may generate at least one first motion data for each first image data using at least one first image data generated during the first period.

Additionally, the second feature extraction unit 113 may generate at least one second motion data for each second image data using at least one second image data generated during the second period.

The first motion data and the second motion data are generated using first image data and second video data, respectively, and are not generated based on position distribution data.

The second feature extraction unit 113 may detect movement in a single pixel or a plurality of blocks including a specific area on the distribution map using single image data or a plurality of consecutive image data.

In an embodiment, motion data is compared to a preset motion threshold to express the presence or absence of motion.

Motion data according to embodiments may represent not only the movement of an object, but also movements other than the object.

The second feature extraction unit 113 can detect movement for each block using the Dense Optical Flow method. The second feature extraction unit 113 can detect movement for each pixel by calculating a motion vector for all pixels in the image data. In the case of the Dense Optical Flow method, detection accuracy is improved because motion vectors are calculated for all pixels, but the amount of calculation is relatively increased. Therefore, the Dense Optical Flow method can be applied to specific areas that require very high detection accuracy, such as breeding farms where abnormal situations are suspected to have occurred or breeding farms with a very large number of individuals.

Alternatively, the second feature extraction unit 113 may detect movement for each block using the Sparse Optical Flow method. The second feature extraction unit 113 may detect motion by calculating a motion vector only for some characteristic pixels that are easy to track motion, such as edges in the image. The Sparse Optical Flow method reduces the amount of computation, but results can only be obtained for a limited number of pixels. Therefore, the Sparse Optical Flow method can be applied to breeding grounds with a small number of individuals or to specific areas where individuals do not appear overlapping.

Alternatively, the second feature extraction unit 113 may detect movement for each block using block matching. The second feature extraction unit 113 may divide the image equally or unevenly and detect motion by calculating a motion vector for the divided area. Block Matching reduces the amount of computation because it calculates motion vectors for each segmented region, but detection accuracy may be relatively low because it calculates results for motion vectors for each region. Therefore, the Block Matching method can be applied to breeding farms with a small number of individuals or to specific areas where individuals do not appear overlapping.

Alternatively, the second feature extraction unit 113 may detect movement for each block using the Continuous Frame Difference method. The second feature extraction unit 113 can detect motion by comparing consecutive image frames for each pixel and calculating a value equal to the difference. Since the Continuous Frame Difference method detects motion using the difference value between frames, the overall amount of calculation is reduced, but the detection accuracy for bulky objects or overlapping objects may be relatively low. Additionally, the Continuous Frame Difference method cannot distinguish between background images and non-moving objects, so its accuracy may be relatively low. Therefore, the Continuous Frame Difference method can be applied to breeding grounds with a small number of individuals or to specific areas where individuals do not appear overlapping.

Alternatively, the second feature extraction unit 113 may detect movement for each block using the Background Subtraction method. The second feature extraction unit 113 may detect motion by comparing successive image frames for each pixel while initially learning the background image, and calculating a value equal to the difference. Since the Background Subtraction method learns the background image in advance, it can distinguish between the background image and non-moving objects. Therefore, a separate process for filtering the background image is required, which increases the amount of computation but improves accuracy. Therefore, the Background Subtraction method can be applied to specific areas that require very high detection accuracy, such as breeding farms where abnormal situations are suspected to have occurred or breeding farms with a very large number of individuals. In the Background Subtraction method, the background image can be continuously updated.

The second feature extraction unit 113 detects movement on the distribution map using an appropriate method according to the environment in the breeding farm and external settings.

The control unit 114 generates abnormal entity candidate data representing an abnormal entity candidate by comparing the first location distribution data and the first motion data generated during the first period, and the second location distribution data and the first motion data generated during the second period. 2Anomalous entity data representing the abnormal entity can be generated by comparing the motion data with the abnormal entity candidate data.

In an embodiment of the present invention, the abnormal entity candidate data may include a set of blocks in which there is a probability of existence of an entity in the first location distribution data but no movement is detected in the first motion data. Abnormal entity candidate data refers to data in which a block in which no movement was detected among blocks with a probability of existence of an entity during a period in which feeding or water supply was not in progress was determined to be an abnormal entity candidate and information about it was recorded.

In an embodiment of the present invention, the abnormal entity data may include a set of blocks among abnormal entity candidates for which there is a probability of existence of the entity in the second location distribution data but no movement is detected in the second motion data. Abnormal object data refers to blocks in which movement was not detected among blocks with a probability of existence of an object during the period when at least one of feeding and watering is in progress, which is determined to be an abnormal object and records information about it. It means data.

The control unit 114 can distinguish between the first period and the second period through the previously stored feeding and water supply time table. The control unit 114 can divide the first period and the second period using the feeding and watering time table stored in the database 119, and divide the generated image data, location distribution data, and motion data according to the period.

Additionally, the control unit 114 may receive a feeding and watering time table from the outside and distinguish between the first period and the second period. The control unit 114 can perform data communication with the management server and the air conditioning device through the communication unit 115, and can receive feeding and watering time tables periodically or upon request.

Additionally, the control unit 114 may distinguish between the first period and the second period using the first motion data and the second motion data. During the second period of feeding and watering, snails show greater movement compared to the first period or show a behavior pattern of gathering near the feeding station or watering station. Accordingly, the control unit 114 can use the first motion data and the second motion data to detect the point in time when the total or average value of the poultry's motion vector suddenly increases and distinguish between the first period and the second period. Alternatively, the control unit 114 may detect the point in time when the movement value toward the feeding station or water station, which is a pre-designated location, suddenly increases, and distinguish between the first period and the second period.

Additionally, when an abnormal entity candidate is detected, the control unit 114 may output a driving command to at least one of the feeder and the waterer between the first period and the second period. When an abnormal entity candidate is detected and an entity suspected of being diseased is found, the control unit 114 may output an operation command to at least one of the feeder and the waterer to transition the environment from the first period to the second period.

Additionally, the control unit 114 may use the abnormal entity candidate data to output a driving command to at least one of the feeder and waterer assigned to the area where the abnormal entity candidate was detected. The control unit 114 can observe changes in movement of the abnormal object candidate by selectively operating only the feeders and waterers located near the area where the abnormal object candidate is detected. In this case, each of the feeders and waterers is structured to operate independently, and the control unit determines the location information of each feeder and waterer.

Additionally, the control unit 114 may use the abnormal entity candidate data to output a driving command to at least one of the feeder and waterer assigned to the area excluding the area where the abnormal entity candidate was detected. The control unit 114 can observe whether the abnormal entity candidate moves to a feeder or waterer located relatively far away by selectively operating only the feeders and waterers located near the area where the abnormal entity candidate is not detected.

The control unit 114 outputs a driving command to at least one of the feeders and waterers assigned to the area where the abnormal object candidate was detected when the abnormal object candidate hardly detects any movement during the first period and the probability of disease suspicion is high, If movement is detected intermittently during the first period and the probability of disease suspicion is relatively low, a driving command may be output to at least one of the feeders and waterers assigned to the area excluding the area where the abnormal entity candidate was detected. In other words, in the case of an abnormal entity candidate with a high probability of disease suspicion, movement is induced to a nearby feeder or waterer because there is little movement, and if the probability of disease suspicion is relatively low, movement is induced to a distant feeder or waterer. You can determine whether it is an object or not.

Additionally, the control unit 114 may generate abnormal entity data through a change in the width of at least one of the feeding area and the water supply area. The control unit 114 may generate abnormal entity data by identifying the extent to which the width of the feeding area and water supply area changes over time in the second position distribution data detected by the first feature extraction unit. The control unit 114 determines the feeding area and watering area near the abnormal object candidate through the positions of the feeder and waterer operating during the second period, and changes the width of the feeding area and watering area to other feeding areas and Abnormal entity data can be generated by comparing the change in width of the water supply area. At this time, the control unit 114 may determine the abnormal object candidate as an abnormal object if the amount of change in width of the corresponding feeding area and water supply area is smaller than the amount of change in width of other feeding areas and water supply areas than a preset value.

Additionally, the control unit 114 may output a drive command to adjust the height of at least one of the feeder and the waterer. The control unit 114 may adjust the height of at least one of the feeders and waterers assigned to the area where the abnormal entity candidate is detected to induce a jumping motion of the abnormal entity candidate. At this time, the control unit 114 may determine the jumping motion of the abnormal object candidate using motion data generated from image data captured by an imaging device located on the side.

Additionally, the control unit 114 may output a driving command to at least one of the feeder and the waterer using abnormal entity candidate data and voice detection data from a voice input means (not shown) disposed in the kennel. The control unit 114 compares the area where the abnormal object candidate was detected and the area where the voice detection data was detected, determines the overlapping area, and outputs a driving command to at least one of the feeder and waterer assigned to the overlapping area. . When the sound of an entity is detected in an area where an abnormal entity candidate is detected, the control unit 114 may induce the operation of the abnormal entity candidate by operating at least one of the feeder and waterer in the area.

The control unit 114 may compare the position distribution data and motion data of the image data to generate abnormal object data indicating blocks with overlapping object presence probability distributions for each block and whether or not motion is detected.

The control unit 114 compares the position distribution data and motion data for a plurality of image data to determine the cumulative number of motion detections for each block and the cumulative number of non-detection motions for each block shown in the motion data among blocks with a probability of presence of an object in the position distribution data. It is possible to calculate and generate abnormal object candidate data according to the accumulated number of motion detections and the accumulated number of motion non-detections.

The control unit 114 uses the first position distribution data and the first motion data generated during the first period to select a block in which no movement is detected among the blocks in which it is determined that there is a probability that an object exists in the first position distribution data. If it is determined to be an entity candidate, abnormal entity candidate data representing information about it can be generated.

The control unit 114 uses at least one first image data generated during the first period to detect movement among blocks in which it is determined that there is a probability of an object existing on the first position distribution data indicating the probability distribution of the existence of the object for each block. It is estimated that a diseased entity is located in an undetected block, and abnormal entity candidate data can be generated for this. Even if movement is detected in a block in which it is determined that there is no probability of an entity existing on the first position distribution data representing the probability distribution of the existence of an entity for each block, the control unit 114 determines it as foreign matter or noise and does not determine it as abnormal entity candidate data. In other words, the abnormal entity candidate data may refer to data resulting from a primary determination of whether the entity is diseased using the position information and motion detection information of the blocks accumulated for the plurality of first image data generated during the first period. .

The control unit 114 may compare the second position distribution data and the second motion data generated from the second image data for the second period with the abnormal entity candidate data to generate abnormal entity data representing the abnormal entity. For abnormal entity candidates, the control unit 114 determines that the diseased entity is located in a block in which no movement was detected among blocks in which it is determined that there is a probability of the entity being present in the second location distribution data generated during the second period. And, abnormal entity data for this can be generated. Even if it is an abnormal entity candidate, the control unit 114 does not determine that a block in which there is no probability of an entity existing or in which movement is detected in the second location distribution data generated during the second period is an abnormal entity. That is, the control unit may determine that, among blocks determined to be abnormal entity candidates in the abnormal entity candidate data, a block in which there is a probability of the existence of an entity during the second period but in which no movement is detected is an abnormal entity.

For example, the control unit 114 generates abnormal entity candidate data using first position distribution data and first motion data generated during a first period, second position distribution data generated during a second period, and second Abnormal entity data can be created using motion data and abnormal entity candidate data. Alternatively, the control unit 114 may generate abnormal entity candidate data using, for example, a plurality of first position distribution data and a plurality of first motion data generated during a first period, and a plurality of first position distribution data generated during a second period. Abnormal entity data can be generated using two-position distribution data, a plurality of second motion data, and abnormal entity candidate data.

The control unit 114 may control the display of pixels for each block on the image data according to the accumulated number of motion detections and the accumulated number of non-motion detections of at least one of the abnormal object candidate data and the abnormal object data. Here, pixel display may include all concepts for distinguishing and displaying a pixel corresponding to an arbitrary point from other pixels, such as pixel saturation, pixel brightness, pixel color, pixel outline, and mark display. In an embodiment of the present invention, the display of pixels can be controlled by adjusting pixel values. Pixel values can be adjusted in stages, and pixels with high pixel values can be displayed with greater visual emphasis than pixels with low pixel values. However, it is not limited to this, and pixels with low pixel values can be set to be displayed with greater emphasis than pixels with high pixel values.

The control unit 114 compares the plurality of first position distribution data, first motion data, and abnormal object candidate data generated during the first period to calculate the accumulated number of motion detections for each block during the first period and the accumulated number of motion non-detection data for the plurality of objects. It is possible to calculate the number of times and control the display of pixels for each block of image data according to the accumulated number of motion detections and the accumulated number of motion non-detections.

Alternatively, the control unit 114 compares the abnormal object candidate data, the plurality of second position distribution data generated during the second period, and the second motion data to calculate the cumulative number of motion detections for each block during the first and second periods and the plurality of objects. The cumulative number of non-detection motions can be calculated, and the display of pixels for each block of image data can be controlled according to the accumulated number of motion detections and the accumulated number of non-motion detections.

The control unit 114 may display pixel values of the image data differently by classifying the object as a normal object as movement of blocks in which there is a probability of the object being present is cumulatively detected, and as movement is not cumulatively detected, classifying it as an abnormal object.

For example, the control unit 114 may gradually increase the block value of the corresponding block according to the accumulated number of motion non-detection counts in the motion data among blocks in which there is a probability of the existence of an object in the position distribution data. If the number of times that movement of a block classified as an abnormal entity candidate is not detected in the abnormal entity candidate data accumulates during the second period, the control unit 114 may gradually increase the block value of the corresponding block to highlight it.

For example, the control unit 114 may gradually increase the block value of the corresponding block according to the accumulated number of consecutive motion non-detection counts in the motion data among blocks with a probability of existence of an object in the position distribution data. If the number of consecutive times in which no movement of a block classified as an abnormal entity candidate is detected in the abnormal entity candidate data is accumulated during the second period, the control unit 114 may gradually increase the block value of the corresponding block to highlight it.

For example, if the accumulated number of motion non-detection counts of motion data among blocks with a probability of existence of an object in the position distribution data exceeds the first threshold, the control unit 114 sets the block value of the corresponding block to the first set value. You can control it. The first set value may be the maximum value of the block value. If the cumulative number of motion non-detection counts for a block classified as an abnormal entity candidate in the abnormal entity candidate data exceeds the first threshold, the control unit 114 may highlight the block value of the corresponding block as much as possible.

For example, when the accumulated number of consecutive motion non-detection counts of motion data among blocks with a probability of existence of an object in the position distribution data exceeds the first threshold, the control unit 114 first sets the block value of the corresponding block. It can be controlled by value. The control unit 114 may highlight the block value of the block as much as possible when the cumulative number of consecutive motion non-detection times for a block classified as an abnormal object candidate in the abnormal entity candidate data exceeds the first threshold.

For example, the control unit 114 may gradually reduce the block value of the corresponding block according to the accumulated number of motion detections of motion data among blocks with a probability of existence of an object in the position distribution data. If the number of motion detections of a block classified as an abnormal object candidate in the abnormal entity candidate data is accumulated, the control unit 114 may gradually reduce the block value of the corresponding block and de-emphasize the display.

For example, the control unit 114 may gradually reduce the block value of the corresponding block according to the accumulated number of consecutive motion detections of motion data among blocks with a probability of existence of an object in the position distribution data. If the number of consecutive motion detections of a block classified as an abnormal object candidate in the abnormal entity candidate data is accumulated, the control unit 114 may gradually reduce the block value of the corresponding block and de-emphasize the display.

For example, the control unit 114 may control the block value of a block in which movement is detected in the motion data among blocks in which there is a probability of the existence of an object in the position distribution data as the second setting value. The second setting value may be a block value of the original image data. If movement of a block classified as an abnormal entity candidate is detected in the abnormal entity candidate data, the control unit 114 may de-emphasize the block value of the block to a minimum.

For example, the control unit 114 may exclude a block in which movement is detected in the motion data among blocks with a probability of the presence of an object in the position distribution data from the abnormal entity candidate data or the abnormal entity data. If movement of a block classified as an abnormal entity candidate is detected in the abnormal entity candidate data, the control unit 114 may exclude the block from the abnormal entity candidate data.

For example, if the accumulated number of motion detections of motion data among blocks with a probability of existence of an object in the position distribution data exceeds the second threshold, the control unit 114 controls the block value of the corresponding block to the second set value. can do. If the accumulated number of motion detections of a block classified as an abnormal entity candidate in the abnormal entity candidate data exceeds the second threshold, the control unit 114 may de-emphasize the block value of the corresponding block to a minimum.

For example, the control unit 114 may exclude blocks in which the cumulative number of motion detections of motion data exceeds the second threshold among blocks with a probability of existence of an object in the position distribution data from the abnormal object candidate data or the abnormal object data. there is. If the accumulated number of motion detections of a block classified as an abnormal entity candidate in the abnormal entity candidate data exceeds a second threshold, the control unit 114 may exclude the block from the abnormal entity candidate data.

For example, when the accumulated number of consecutive motion detections of motion data among blocks with a probability of existence of an object in the position distribution data exceeds the second threshold, the control unit 114 sets the block value of the corresponding block to the second set value. It can be controlled with . If the cumulative number of consecutive motion detections of a block classified as an abnormal entity candidate in the abnormal entity candidate data exceeds the second threshold, the control unit 114 may de-emphasize the block value of the corresponding block to a minimum.

For example, if the accumulated number of consecutive motion detections of motion data among blocks with a probability of existence of an object in the position distribution data exceeds the second threshold, the control unit 114 selects the corresponding block as abnormal object candidate data or abnormal object It can be excluded from the data. If the cumulative number of consecutive motion detections of a block classified as an abnormal entity candidate in the abnormal entity candidate data exceeds the second threshold, the control unit 114 may exclude the block from the abnormal entity candidate data.

The control unit 114 may execute various commands for operating the imaging device 100.

For example, the control unit 114 may operate the imaging device 100 by executing commands stored in the user interface unit 117 or the database 119.

Alternatively, the control unit 114 may control various operations of the imaging device 100 using commands received from the management server 200.

The control unit 114 can control the photographing unit 11 to track and photograph blocks whose cumulative number of motion detections is equal to or greater than a third set value. The third setting value may be set through the user interface unit 117 or through a control command of the management server 200. The control unit 114 controls the imaging unit 111 to track and photograph a specific area where an abnormal object is located and generate image data, thereby enabling continuous monitoring.

At this time, the control unit 114 may control the pan/tilt unit 122 of the imaging device 100 to perform tracking imaging. The pan/tilt unit 122 can control the photographing area of the photographing unit 111 by driving two motors, pan (horizontal direction) and tilt (vertical direction). The pan/tilt unit 122 can adjust the direction of the photographing unit 111 to photograph a specific area under the control of the controller 114. Additionally, the pan/tilt unit 122 can adjust the direction of the photographing unit 111 to track a specific block under the control of the control unit 114.

The communication unit 115 may perform data communication with another imaging device, the data collection device 150, or the management server 200. For example, the communication unit 115 supports wireless LAN (WLAN), Wi-Fi, Wibro (Wireless Broadband), World Interoperability for Microwave Access (Wimax), and High Speed Downlink (HSDPA). Data communication can be performed using long-distance communication technologies such as Packet Access, IEEE 802.16, Long Term Evolution (LTE), and Wireless Mobile Broadband Service (WMBS).

Alternatively, the communication unit 115 may include Bluetooth, RadioFrequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, Near Field Communication (NFC), etc. Additionally, as a wired communication technology, data communication can be performed using short-distance communication technologies such as USB communication, Ethernet, serial communication, and optical/coaxial cables.

For example, the communication unit 115 may perform data communication with another imaging device or data collection device 150 using short-range communication technology, and may perform data communication with the management server 200 using long-distance communication technology. However, the communication technology is not limited to this, and various communication technologies may be used in consideration of all matters of the breeding farm 10.

The communication unit 115 may transmit original image data captured by the photographing unit 111 to the data collection device 150 or the management server 200.

Alternatively, the communication unit 115 may transmit at least one of image data with adjusted pixel values, location distribution data, motion data, and coordinate information to at least one of the data collection device 150 or the remote management server 200. At this time, the coordinate information may mean the coordinates of a point or object suspected to be an abnormal object.

Data transmitted through the communication unit 115 may be compressed data encoded through the encoding unit 118.

The display unit 116 includes a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), and a flexible display. ), a 3D display, and an e-ink display.

The display unit 116 may display at least one of image data and distribution data whose pixel values have been adjusted through the control unit 114.

Additionally, the display unit 116 can output original image data captured by the photographing unit 111 on the screen.

Additionally, the display unit 116 can output various user interfaces or graphic user interfaces on the screen.

The user interface unit 117 generates input data for controlling the operation of the imaging device 100. The user interface unit 117 may be composed of a key pad, dome switch, touch pad, jog wheel, jog switch, etc. When the display unit 116 and the touch pad form a layered structure to form a touch screen, the display unit 116 can be used as an input device in addition to an output device.

The user interface unit 117 may receive various commands for operating the imaging device.

The encoding unit 118 digitalizes the original image data captured by the photographing unit 111 or the processed image data processed through the first feature extraction unit 112, the second feature extraction unit 113, and the control unit 114. Encode it into a signal. For example, the encoding unit 118 may encode image data according to H.264, H.265, MPEG (Moving Picture Experts Group), and M-JPEG (Motion Joint Photographic Experts Group) standards.

The database 119 is a flash memory type, hard disk type, multimedia card micro type, and card type memory (for example, SD or XD memory, etc.) ), magnetic memory, magnetic disk, optical disk, RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory: ROM), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM ( It may include at least one storage medium among Programmable Read-Only Memory. Additionally, the imaging device 100 may operate web storage that performs a storage function of the database 119 on the Internet, or may operate in relation to web storage.

The database 119 can store image data captured by the photographing unit 111 and can store image data for a certain period of time in the past.

Additionally, the database 119 may store data and programs necessary for the imaging device 100 to operate.

Additionally, the database 119 may store various user interfaces (UI) or graphic user interfaces (GUI).

The illuminance sensor 120 can detect the surrounding illuminance. The illuminance sensor may include at least one of a photoconductive cell, a photodiode, a phototransistor, a photothyristor, a photomultiplier tube, and a photoelectric tube.

Photoconductive cells use the photoconductive effect to detect surrounding illumination. The photoconduction effect is a phenomenon in which when light is irradiated to a semiconductor, the carrier density in the semiconductor increases and the conductivity increases. It is a phenomenon in which free electrons in the valence band are excited to the conduction band by the energy of light from the outside, resulting in conductivity. .

Photodiodes, phototransistors, and photothyristors use the photovoltaic effect to sense ambient illumination. The photovoltaic effect means that when light is irradiated to the boundary layer of the N-type and P-type semiconductors in a PN junction, electrons and holes are generated (electrons). It is a phenomenon in which an electromotive force is generated by moving beyond the potential barrier of the PN junction due to an electric field and thermal diffusion. It is a type of photoelectric effect when light is incident on the PN junction of a semiconductor or the junction surface of a semiconductor and a metal. This is a phenomenon in which electromotive force is generated.

Photomultiplier tubes and photoelectron tubes detect ambient illuminance using the photoelectron emission effect. The photoelectron emission effect refers to the effect in which energy greater than the work function is obtained when light hits a metal or metal oxide, and atomic electrons are emitted from the metal surface. The light-receiving surface that uses this effect is called the photofront.

The light source unit 121 may radiate light in a direction under the control of the control unit 114. For example, the light source unit 121 may include at least one laser diode (LD) or light emitting diode (LED). The light source unit 121 can irradiate light of various wavelengths under the control of the control unit 114.

For example, the light source unit 121 may irradiate light in the infrared wavelength range for night photography. Alternatively, the light source unit 121 may irradiate light in the ultraviolet wavelength range for photochemotherapy of livestock in the farm.

Figure 3 is a diagram for explaining an abnormal object detection method according to an existing method, and Figure 4 is a diagram for explaining the operation of an abnormal object detection device according to an embodiment of the present invention.

Referring to Figure 3, the existing abnormal object detection method first detects an object and then continuously tracks the detected object to observe whether it moves. The detected object is continuously tracked, and if no movement is detected for a certain period of time, it is determined to be diseased and an alarm is output. According to these existing methods, one-to-one tracking of detected objects must be continuously performed. Therefore, it is practically impossible to detect individual objects in video data captured in a farm where at least hundreds or as many as hundreds of thousands of poultry are raised, and continuous tracking of detected objects places an excessive burden on computational traffic. I can give it. In addition, because an object without movement is simply judged to be an abnormal object without considering the surrounding situation, the accuracy and reliability of the judgment are significantly reduced.

Referring to Figure 4, the abnormal object detection apparatus according to an embodiment of the present invention determines the probability of existence of an object for each block of image data and detects movement for each block independently of the probability of existence of the object. That is, the abnormal object detection device according to an embodiment of the present invention probabilistically determines whether an object exists for each block and detects whether there is movement for each block regardless of whether an object exists. Afterwards, a block in which there is a probability that an object exists and movement is detected is judged as a normal object, and a block in which there is a probability that an object exists in which movement is not detected is judged as an abnormal object. According to an embodiment of the present invention, since the probability that an object exists is determined for each block rather than detecting the object itself, errors do not occur in object detection even if the objects are clustered or overlapped. Additionally, because the object itself is not tracked, the amount of computation is significantly reduced.

Referring to Figure 4(a), the abnormal object detection device according to an embodiment of the present invention primarily determines abnormal object candidates based on the probability of existence of the object and whether or not motion is detected for each block of image data generated during the first period. . Thereafter, according to FIG. 4(b), during the second period, at least one of the feeder and the waterer operates to induce movement of the object, and the probability of existence of the object and whether or not movement is detected for each block of the image data generated during the second period are determined. Accordingly, it can be determined whether the abnormal entity candidate is an abnormal entity. The abnormal object detection device according to an embodiment of the present invention primarily detects abnormal object candidates, operates the feeder and waterer to induce the movement of the abnormal object candidates, and finally determines the abnormal object, thereby improving the accuracy of abnormal object detection. and reliability can be significantly improved.

Figure 5 is a flowchart of a method for detecting abnormal objects according to an embodiment of the present invention.

Referring to Figure 5, first, the photographing unit captures images including a plurality of objects during the first period (S501).

Next, the photographing unit generates first image data using an image including a plurality of objects. One image data may mean a single frame. The photographing unit generates a plurality of first image data using images sequentially captured during the first period (S502).

Image data according to an embodiment may be image data composed of N x M pixels.

Next, the first feature extraction unit generates first position distribution data indicating the position distribution of a plurality of objects using the first image data (S503).

Next, the second feature extraction unit uses the first image data to generate first motion data indicating the movement of the motion object among the plurality of objects (S504).

Next, the control unit compares the first position distribution data and the first motion data to generate abnormal entity candidate data representing the abnormal entity candidate (S505).

Next, the control unit controls pixel display for a plurality of blocks on the first image data according to the accumulated number of motion detections and non-detection motions of the abnormal object data (S506).

Next, the photographing unit captures images including multiple objects during the second period (S507).

Next, the imaging unit generates second image data using images including a plurality of objects during the second period (S508).

Next, the first feature extraction unit generates second position distribution data indicating the position distribution of a plurality of objects using the second image data (S509).

Next, the second feature extraction unit uses the second image data to generate second motion data indicating the movement of the motion object among the plurality of objects (S510).

Next, the control unit compares the second position distribution data and the second motion data with the abnormal entity candidate data to generate abnormal entity data representing the abnormal entity. At this time, the control unit divides the first period and the second period using a pre-stored feeding and watering time table or a feeding and watering time table received from the outside (S511).

Next, the control unit controls pixel display for a plurality of blocks on the second image data according to the accumulated number of motion detections and non-detection motions of the abnormal object data (S512).

Next, the control unit outputs an alarm regarding abnormal entity data to the outside (S513).

The steps of generating location distribution data and generating motion data according to the embodiment may be performed simultaneously, or either data generation process may be performed first.

Position distribution data generation and motion data according to the embodiment each generate position distribution data and motion data by performing separate operations in parallel, and motion data is generated using position distribution data or motion data is used. Therefore, it does not generate location distribution data.

Steps S501 to S513 are briefly shown as generating single abnormal entity candidate data and single abnormal entity data, but abnormal entity candidate data and abnormal entity data are continuously generated as image data is continuously generated through image capture. and updated to operate to control pixel display.

Figure 6 is a flowchart of a method for detecting abnormal objects according to an embodiment of the present invention.

Referring to Figure 6, first, the photographing unit captures images including a plurality of objects during the first period (S601).

Next, the photographing unit generates first image data using an image including a plurality of objects. One image data may mean a single frame. The photographing unit generates a plurality of first image data using images sequentially photographed during the first period (S602).

Image data according to an embodiment may be image data composed of N x M pixels.

Next, the first feature extraction unit generates first position distribution data representing the position distribution of a plurality of objects using the first image data (S603).

Next, the second feature extraction unit uses the first image data to generate first motion data indicating the movement of the motion object among the plurality of objects (S604).

Next, the control unit compares the first position distribution data and the first motion data to generate abnormal entity candidate data representing the abnormal entity candidate (S605).

Next, the control unit controls pixel display for a plurality of blocks on the first image data according to the accumulated number of motion detections and accumulated number of non-motion detections of the abnormal object candidate data (S606).

Next, the photographing unit captures images including multiple objects during the second period (S607).

Next, the photographing unit captures images including a plurality of objects during the second period and generates second image data (S608).

Next, the first feature extraction unit generates second position distribution data indicating the position distribution of a plurality of objects using the second image data (S609).

Next, the second feature extraction unit uses the second image data to generate second motion data indicating the movement of the motion object among the plurality of objects (S610).

Next, the control unit divides the first period and the second period using the first operation data and the second operation data (S611).

Next, the control unit compares the second position distribution data and the second motion data with the abnormal entity candidate data to generate abnormal entity data representing the abnormal entity (S612).

Next, the control unit controls pixel display for a plurality of blocks on the second image data according to the accumulated number of motion detections and non-detection motions of the abnormal object data (S613).

Next, the control unit outputs an alarm regarding abnormal entity data to the outside (S614).

The steps of generating location distribution data and generating motion data according to the embodiment may be performed simultaneously, or either data generation process may be performed first.

Position distribution data generation and motion data according to the embodiment each generate position distribution data and motion data by performing separate operations in parallel, and motion data is generated using position distribution data or motion data is used. Therefore, it does not generate location distribution data.

Steps S601 to S614 are briefly shown as generating single abnormal entity candidate data and single abnormal entity data, but abnormal entity candidate data and abnormal entity data are continuously generated as image data is continuously generated through image capture. and updated to operate to control pixel display.

Figure 7 is a flowchart of a method for detecting abnormal objects according to an embodiment of the present invention.

Referring to Figure 7, first, the photographing unit captures images including a plurality of objects during the first period (S701).

Next, the photographing unit generates first image data using an image including a plurality of objects. One image data may mean a single frame. The photographing unit generates a plurality of first image data using images sequentially photographed during the first period (S702).

Image data according to an embodiment may be image data composed of N x M pixels.

Next, the first feature extraction unit generates first position distribution data indicating the position distribution of a plurality of objects using the first image data (S703).

Next, the second feature extraction unit uses the first image data to generate first motion data indicating the movement of a motion object among the plurality of objects (S704).

Next, the control unit compares the first position distribution data and the first motion data to generate abnormal entity candidate data representing the abnormal entity candidate (S705).

Next, the control unit controls the pixel display of a plurality of blocks on the first image data according to the accumulated number of motion detections and the accumulated number of non-motion detections of the abnormal object candidate data (S706).

Next, the control unit outputs a driving command to at least one of the feeder and waterer assigned to the area where the abnormal object candidate was detected (S707).

Next, the photographing unit captures images including multiple objects during the second period (S708).

Next, the photographing unit captures images including a plurality of objects during the second period and generates second image data (S709).

Next, the first feature extraction unit generates second position distribution data indicating the position distribution of a plurality of objects using the second image data (S710).

Next, the second feature extraction unit uses the second image data to generate second motion data indicating the movement of the motion object among the plurality of objects (S711).

Next, the control unit compares the second position distribution data and the second motion data with the abnormal entity candidate data to generate abnormal entity data representing the abnormal entity (S712).

Next, the control unit controls pixel display for a plurality of blocks on the second image data according to the accumulated number of motion detections and non-detection motions of the abnormal object data (S713).

Next, the control unit outputs an alarm regarding abnormal entity data to the outside (S714).

The steps of generating location distribution data and generating motion data according to the embodiment may be performed simultaneously, or either data generation process may be performed first.

Position distribution data generation and motion data according to the embodiment each generate position distribution data and motion data by performing separate operations in parallel, and motion data is generated using position distribution data or motion data is used. Therefore, it does not generate location distribution data.

Steps S701 to S714 are briefly shown as generating single abnormal entity candidate data and single abnormal entity data, but abnormal entity candidate data and abnormal entity data are continuously generated as image data is continuously generated through image capture. and updated to operate to control pixel display.

Figure 8 is a flowchart of a method for detecting abnormal objects according to an embodiment of the present invention.

Referring to Figure 8, first, the photographing unit captures an image including a plurality of objects during the first period (S801).

Next, the photographing unit generates first image data using an image including a plurality of objects. One image data may mean a single frame. The photographing unit generates a plurality of first image data using images sequentially photographed during the first period (S802).

Image data according to an embodiment may be image data composed of N x M pixels.

Next, the first feature extraction unit generates first position distribution data indicating the position distribution of a plurality of objects using the first image data (S803).

Next, the second feature extraction unit uses the first image data to generate first motion data indicating the movement of the motion object among the plurality of objects (S804).

Next, the control unit compares the first position distribution data and the first motion data to generate abnormal entity candidate data representing the abnormal entity candidate (S805).

Next, the control unit controls pixel display for a plurality of blocks on the first image data according to the accumulated number of motion detections and accumulated number of non-motion detections of the abnormal object candidate data (S806).

Next, the control unit uses the abnormal entity candidate data to output a driving command to at least one of the feeder and waterer assigned to the area excluding the area where the abnormal entity candidate was detected. The control unit can observe whether the abnormal object candidate moves to a feeder or waterer located relatively far away by selectively operating only the feeders and waterers located around the area where the abnormal object candidate is not detected (S807).

Next, the photographing unit captures images including multiple objects during the second period (S808).

Next, the photographing unit captures images including a plurality of objects during the second period and generates second image data (S809).

Next, the first feature extraction unit generates second position distribution data indicating the position distribution of a plurality of objects using the second image data (S810).

Next, the second feature extraction unit uses the second image data to generate second motion data indicating the movement of the motion object among the plurality of objects (S811).

Next, the control unit compares the second position distribution data and the second motion data with the abnormal entity candidate data to generate abnormal entity data representing the abnormal entity (S812).

Next, the control unit controls pixel display for a plurality of blocks on the second image data according to the accumulated number of motion detections and non-detection motions of the abnormal object data (S813).

Next, the control unit outputs an alarm regarding abnormal entity data to the outside (S814).

The steps of generating location distribution data and generating motion data according to the embodiment may be performed simultaneously, or either data generation process may be performed first.

Position distribution data generation and motion data according to the embodiment each generate position distribution data and motion data by performing separate operations in parallel, and motion data is generated using position distribution data or motion data is used. Therefore, it does not generate location distribution data.

Steps S801 to S814 are briefly shown as generating single abnormal entity candidate data and single abnormal entity data, but abnormal entity candidate data and abnormal entity data are continuously generated as image data is continuously generated through image capture. and updated to operate to control pixel display.

Figure 9 is a flowchart of a method for detecting abnormal objects according to an embodiment of the present invention.

Referring to Figure 9, first, the photographing unit captures an image including a plurality of objects during the first period (S901).

Next, the photographing unit generates first image data using an image including a plurality of objects. One image data may mean a single frame. The photographing unit generates a plurality of first image data using images sequentially photographed during the first period (S902).

Image data according to an embodiment may be image data composed of N x M pixels.

Next, the first feature extraction unit generates first position distribution data indicating the position distribution of a plurality of objects using the first image data (S903).

Next, the second feature extraction unit uses the first image data to generate first motion data indicating the movement of a motion object among the plurality of objects (S904).

Next, the control unit compares the first position distribution data and the first motion data to generate abnormal entity candidate data representing the abnormal entity candidate (S905).

Next, the control unit controls pixel display for a plurality of blocks on the first image data according to the accumulated number of motion detections and accumulated number of non-motion detections of the abnormal object candidate data (S906).

Next, the control unit outputs a drive command to adjust the height of at least one of the feeder and the waterer. The control unit may increase the activity of the entity by adjusting the height of at least one of the feeders and waterers assigned to the area where the abnormal entity candidate was detected to make it difficult to consume feed and water (S907).

Next, the photographing unit captures images including a plurality of objects during the second period (S908).

Next, the photographing unit captures images including a plurality of objects during the second period and generates second image data (S909).

Next, the first feature extraction unit generates second position distribution data indicating the position distribution of a plurality of objects using the second image data (S910).

Next, the second feature extraction unit uses the second image data to generate second motion data indicating the movement of the motion object among the plurality of objects (S911).

Next, the control unit compares the second position distribution data and the second motion data with the abnormal entity candidate data to generate abnormal entity data representing the abnormal entity (S912).

Next, the control unit controls pixel display for a plurality of blocks on the second image data according to the accumulated number of motion detections and non-detection motions of the abnormal object data (S913).

Next, the control unit outputs an alarm regarding abnormal entity data to the outside (S914).

The steps of generating location distribution data and generating motion data according to the embodiment may be performed simultaneously, or either data generation process may be performed first.

Position distribution data generation and motion data according to the embodiment each generate position distribution data and motion data by performing separate operations in parallel, and motion data is generated using position distribution data or motion data is used. Therefore, it does not generate location distribution data.

Steps S901 to S914 are briefly shown as generating single abnormal entity candidate data and single abnormal entity data, but abnormal entity candidate data and abnormal entity data are continuously generated as image data is continuously generated through image capture. and updated to operate to control pixel display.

Figure 10 is a flowchart of a method for detecting abnormal objects according to an embodiment of the present invention.

Referring to Figure 10, first, the photographing unit captures an image including a plurality of objects during the first period (S1001).

Next, the photographing unit generates first image data using an image including a plurality of objects. One image data may mean a single frame. The photographing unit generates a plurality of first image data using images sequentially photographed during the first period (S1002).

Image data according to an embodiment may be image data composed of N x M pixels.

Next, the first feature extraction unit generates first position distribution data representing the position distribution of a plurality of objects using the first image data (S1003).

Next, the second feature extraction unit uses the first image data to generate first motion data indicating the movement of a motion object among the plurality of objects (S1004).

Next, the control unit compares the first position distribution data and the first motion data to generate abnormal entity candidate data representing the abnormal entity candidate (S1005).

Next, the control unit controls pixel display for a plurality of blocks on the first image data according to the accumulated number of motion detections and the accumulated number of non-motion detections of the abnormal object candidate data (S1006).

Next, the photographing unit captures images including a plurality of objects during the second period (S1007).

Next, the photographing unit captures images including a plurality of objects during the second period and generates second image data (S1008).

Next, the first feature extraction unit generates second position distribution data indicating the position distribution of a plurality of objects using the second image data (S1009).

Next, the second feature extraction unit uses the second image data to generate second motion data indicating the movement of the motion object among the plurality of objects (S1010).

Next, the control unit generates abnormal entity data through a change in the width of at least one of the feeding area and the water supply area. The control unit determines the feeding area and water supply area near the abnormal object candidate through the positions of the feeder and water supply operating during the second period, and changes the width of the corresponding feeding area and water supply area to other feeding areas and water supply areas. Abnormal entity data is generated by comparing the width change (S1011).

Next, the control unit controls pixel display for a plurality of blocks on the second image data according to the accumulated number of motion detections and non-detection motions of the abnormal object data (S1012).

Next, the control unit outputs an alarm regarding abnormal entity data to the outside (S1013).

The steps of generating location distribution data and generating motion data according to the embodiment may be performed simultaneously, or either data generation process may be performed first.

Position distribution data generation and motion data according to the embodiment each generate position distribution data and motion data by performing separate operations in parallel, and motion data is generated using position distribution data or motion data is used. Therefore, it does not generate location distribution data.

Steps S1001 to S1013 are briefly shown as generating single abnormal entity candidate data and single abnormal entity data, but abnormal entity candidate data and abnormal entity data are continuously generated as image data is continuously generated through image capture. and updated to operate to control pixel display.

Figure 11 is a conceptual diagram of the configuration of a learning server according to an embodiment of the present invention. The abnormal object detection device of FIG. 11 may include the same configuration as the abnormal object detection device shown in FIG. 2 . The learning preprocessor 120 extracts learning data to be used by the learning server 400 for learning from the image data acquired by the photographing unit 111. The detailed method by which the learning preprocessor 120 extracts learning data will be described below.

Referring to FIGS. 11 and 12, the abnormal object detection device 100 uses the photographing unit 111 to obtain image data for an image in which a plurality of objects are photographed together (S1200), and the control unit 112 acquires image data. Abnormal entity information is extracted from image data (S1202). Here, one image data may mean a single frame, and the photographing unit 111 may generate a plurality of image data using sequentially captured images. At this time, the control unit 112 may extract abnormal entity information using a pre-stored algorithm and parameters applied thereto. Here, the pre-stored algorithm detects movement among objects using a first algorithm learned to display the area where the object is judged to be located within the image data and a second algorithm learned to detect motion using optical flow. It can refer to an algorithm that displays objects that do not exist. Here, the first algorithm may use a real-time object detection method using an algorithm that displays the area where the object exists, or the first algorithm may use a method of indicating distribution information of the object, that is, density information of the area where the object is located. You can also use . Figure 13 is a diagram for explaining an object density prediction network. Here, the object density prediction network is an example of a deep learning algorithm designed to display density information of the area where an object is located. An object density prediction network may be an algorithm that inputs an original image into a convolution network-based learning machine and then outputs a density image represented as a gray-scale probability map. According to this, abnormal object information can be easily extracted even in a breeding environment where objects with similar shapes are accommodated at a high density, such as a chicken coop.

The object density prediction network can be learned using the original image and the density image, and can be learned by the training server of FIG. 17, which will be described later.

As shown in FIG. 13, the object density prediction network may include at least one convolution network (layer).

A convolutional network can classify feature points of an image using at least one feature map (W).

A convolutional network can improve the performance of an object density prediction network through at least one pooler and/or activator.

The object density prediction network further includes a concatenator, which concatenates and rearranges the output results of at least one convolutional network to output object density (distribution) information using feature points of the original image. there is.

In a training server, which will be described later using FIG. 17, at least one feature map (W) may be trained (tuned) to output density information of an object.

In an embodiment of the present invention, an object density prediction network is configured so that the density image has the same or similar size (Width x Highet) as the original image, the positions of each pixel (or block) correspond to each other, and the density image The pixel value may represent the probability that an object (for example, a poultry) may exist in the corresponding pixel of the original image. For example, the higher the probability that an object exists in a pixel of the original image, the higher the value of the corresponding pixel in the density image may be. Or, it only indicates whether the object exists, and sets the pixel value differently when the object exists and when it does not.

The communication unit 113 of the abnormal object detection device 100 transmits the abnormal object information acquired by the control unit 112 to the administrator terminal 200 (S1204), and the administrator terminal 200 outputs the abnormal entity information (S1206) ). Accordingly, the manager can recognize abnormal entity information output to the manager terminal 200. Here, abnormal entity information may be mixed with abnormal entity data.

FIG. 14 is a diagram illustrating an example in which abnormal entity information is displayed on an administrator terminal. An image as shown in FIG. 14(c) may be displayed on the administrator terminal 200. That is, the administrator terminal 200 contains an image that matches the original image captured by the photographing unit 111 as shown in FIG. 1(a) and the abnormal entity data extracted by the control unit 112 as shown in FIG. 14(b). can be displayed. Here, the abnormal entity data may be a chicken density estimation image and may be displayed as a pseudo-density 8-bit gray image, probability map, or bitmap. At this time, the abnormal object data is not data indicating the probability of abnormality for each object, but data indicating the probability of abnormality for the object, body, or head in the corresponding area, block, or pixel for each divided area, block, or pixel in the image data. am.

If there is an error in the abnormal entity information exposed to the administrator terminal 200, the administrator may transmit feedback information to the abnormal entity detection device 100 through the administrator terminal 200 (S1208). For example, if the abnormal object detection device 100 determines that the object is an abnormal object even though it is not an abnormal object, or if the abnormal object detection device 100 determines that it is a normal object even though it is an abnormal object, the administrator terminal 200 ) may transmit feedback information indicating that there is an error in the abnormal object information to the abnormal object detection device 100. Here, the feedback information may include at least one of an error-prone time domain and a spatial domain among the abnormal entity information. The time area and space area in which the error occurs may be selected or designated by the administrator and may be input through the user interface unit of the administrator terminal 200. For example, as illustrated in FIG. 15(c), the administrator terminal 200 displays abnormal entity information (802, 804, 806, 808) for four areas within the screen 800, and the administrator displays 4 areas. If it is determined that one of the abnormal entity information (for example, 806) is an error, the administrator can select the erroneous abnormal entity information (806) and provide feedback through the user interface. Alternatively, if the abnormal object information is not displayed even though there is an abnormal object in the screen output by the administrator terminal 200, the administrator may designate a time region and a spatial region that appear to be the abnormal object and provide feedback.

The learning preprocessor 120 of the abnormal object detection device 100 extracts learning data using the feedback information received from the administrator terminal 200 (S1210), and transmits the extracted learning data to the learning server 400 (S1210). S1212).

Here, the training data may be part of the image data acquired in step S1200. The learning preprocessor 120 of the abnormal object detection device 100 may extract image data according to the time domain and spatial domain included in the feedback information received from the manager terminal 200 among the image data acquired in step S1200. . For example, if the manager terminal 200 determines that there is an error in some area of the nth frame (e.g., 806 in FIG. 15(c)), the training data is stored in some area of the nth frame (e.g., 806 in FIG. 15(c)). It may be 806) of (c). Alternatively, if the manager terminal 200 determines that there is an error in the abnormal entity data of the n+3th frame from the nth frame among all frames, the learning data may be the n+3th frame from the nth frame among all frames. . Accordingly, the abnormal object detection device 100 can extract only the image data with errors in abnormal object detection out of all the image data acquired by the photographing unit 111 and transmit it to the learning server 400. Accordingly, communication traffic between the abnormal object detection device 100 and the learning server 400 can be significantly reduced. Here, the training data may further include at least one of error information and correction information in addition to the extracted image data. Error information may be information indicating that the judgment by the control unit is incorrect, and correction information may be information indicating the direction to be corrected. Alternatively, the learning data may include extracted original image data and abnormal entity data after errors have been corrected. For example, if it is determined that there is an error in some area (e.g., 806 in FIG. 15(c)) within the nth frame to the n+3th frame, the training data is for the nth frame to the n+3th frame. It may include original image data and abnormal object data for the nth frame to n+3th frame, and the abnormal object data for the nth frame to n+3th frame may include some areas judged to have errors (e.g. , It may be a gray image after the error 806) in FIG. 15(c) has been corrected. The learning server 400 retrains using the learning data received from the abnormal object detection device 100 (S1214), and accordingly transmits update information for abnormal object detection to the abnormal object detection device 100. Do it (S1216). Here, the update information may be a parameter applied to an algorithm for detecting abnormal objects, and may be in the form of an adjustable matrix.

Thereafter, the abnormal object detection device 100 acquires image data for an image in which a plurality of objects are photographed together as in step S1200 (S1218), and the control unit 112 obtains image data using a pre-stored algorithm and update information. Abnormal entity information is extracted from image data (S1220).

Accordingly, the abnormal object detection device 100 can detect abnormal objects by reflecting the feedback information of the administrator terminal 200, and communication traffic between the abnormal object detection device 100 and the learning server 400 can be reduced. .

When the learning server 400 communicates with a plurality of abnormal object detection devices 100, the learning server 400 may transmit the same update information to the plurality of abnormal object detection devices 100. Alternatively, since each abnormal object detection device 100 may be in a different environment, the update information may be specific to each abnormal object detection device 100.

According to an embodiment of the present invention, the learning server 400 also stores an algorithm such as the object density prediction network of the abnormal object detection device 100, and the learning server 400 uses this to convert the original image corresponding to the error data. It is desirable to relearn.

Hereinafter, a method for detecting an abnormal object by the abnormal object detection device 100 according to an embodiment of the present invention will be described.

Figure 16 is a diagram explaining an abnormal object detection algorithm of the abnormal object detection device according to an embodiment of the present invention.

Referring to FIG. 16, the first feature extraction unit 600 extracts the location distribution of a plurality of objects in the image data for the image captured by the photographing unit 111, and the second feature extraction unit 602 extracts the position distribution of the plurality of objects in the image data. Extract the movements of multiple objects in the data. And, the control unit 604 generates abnormal object information, for example, abnormal object probability for each pixel, based on the position distribution extracted by the first feature extraction unit 600 and the movement extracted by the second feature extraction unit 602. Estimate .

Specifically, the first feature extraction unit 600 may generate position distribution data indicating the position distribution of a plurality of objects using image data. Here, the location distribution of a plurality of objects may mean the density distribution of objects for each location, and location distribution data may be used interchangeably with a density map. As described above, the first feature extraction unit 600 may use a real-time object detection method using a first algorithm learned to suggest a region in which an object is determined to be located within the image data, that is, a region proposal algorithm. For example, the first feature extraction unit 600 generates first position distribution data indicating the position distribution of the plurality of objects using first image data generated including the plurality of objects, and includes the plurality of objects. Second position distribution data representing the position distribution of a plurality of objects can be generated using the generated second image data. Here, the first position distribution data and the second position distribution data may be position distribution data for image data generated in time series.

In an embodiment of the present invention, position distribution data does not indicate the location of an individual object, but is data indicating the probability that an object, body, or head may exist in the corresponding area or block for each divided area or block of image data. Location distribution data may be a heatmap that expresses the probability that an object exists in each pixel in different colors.

Additionally, the first feature extraction unit 600 may detect animal entities from image data using an entity detection classifier. At this time, the object detection classifier is learned by building a training database from images of animal objects taken with different postures or external environments. This object detection classifier uses SVM (Support Vector Machine), neural network, and AdaBoost algorithm. A DB of animal entities is created through various learning algorithms, including: Specifically, the first feature extraction unit 600 detects the edge of an object corresponding to the foreground from the image data of the background in the previously photographed breeding farm, applies the edge of the foreground object detected in the image data, and extracts the edge of the foreground object. Animal objects can be detected by applying an object detection classifier to the area of the image data to which is applied.

The second feature extraction unit 602 may use image data to generate motion data representing the movement of a motion object among a plurality of objects. Here, motion data does not indicate the movement of an individual object, but is data indicating whether motion was present in the corresponding region or block for each divided area or block of image data, and motion data can be used interchangeably with a motion map. . Motion data may be data indicating whether movement exists in the corresponding pixel for each pixel. As described above, the second feature extraction unit 602 may use a second algorithm learned to detect motion using optical flow. The second feature extraction unit 602 may detect movement at a specific point, specific object, or specific pixel on the distribution map using single image data or a plurality of continuous image data.

For example, the second feature extraction unit 602 uses first image data to generate first motion data indicating the movement of a motion object among a plurality of objects, and uses the second image data to generate motion among a plurality of objects. Second motion data representing the movement of the object may be generated. Here, the first motion data and the second motion data may be motion data for a plurality of image data generated in time series.

The second feature extraction unit 602 can detect the movement of the moving object using the Dense Optical Flow method. The second feature extraction unit 602 may detect movement for each pixel by calculating a motion vector for all pixels in the image data. In the case of the Dense Optical Flow method, detection accuracy is improved because motion vectors are calculated for all pixels, but the amount of calculation is relatively increased. Therefore, the Dense Optical Flow method can be applied to specific areas that require very high detection accuracy, such as breeding farms where abnormal situations are suspected to have occurred or breeding farms with a very large number of individuals.

Alternatively, the second feature extraction unit 602 may detect the movement of the moving object using the Sparse Optical Flow method. The second feature extraction unit 602 may detect motion by calculating a motion vector only for some characteristic pixels that are easy to track motion, such as edges in the image. The Sparse Optical Flow method reduces the amount of computation, but results can only be obtained for a limited number of pixels. Therefore, the Sparse Optical Flow method can be applied to breeding grounds with a small number of individuals or to specific areas where individuals do not appear overlapping.

Alternatively, the second feature extraction unit 602 may detect the movement of the moving object using block matching. The second feature extraction unit 602 may divide the image equally or unevenly and detect motion by calculating a motion vector for the divided area. Block Matching reduces the amount of computation because it calculates motion vectors for each segmented region, but detection accuracy may be relatively low because it calculates results for motion vectors for each region. Therefore, the Block Matching method can be applied to breeding farms with a small number of individuals or to specific areas where individuals do not appear overlapping.

Alternatively, the second feature extraction unit 602 may detect the movement of the moving object using the Continuous Frame Difference method. The second feature extraction unit 602 can detect motion by comparing consecutive image frames for each pixel and calculating a value equal to the difference. Since the Continuous Frame Difference method detects motion using the difference value between frames, the overall computational amount is reduced, but the detection accuracy for bulky objects or overlapping objects may be relatively low. Additionally, the Continuous Frame Difference method cannot distinguish between background images and non-moving objects, so its accuracy may be relatively low. Therefore, the Continuous Frame Difference method can be applied to breeding grounds with a small number of individuals or to specific areas where individuals do not appear overlapping.

Alternatively, the second feature extraction unit 602 may detect the movement of the moving object using the Background Subtraction method. The second feature extraction unit 602 may detect motion by comparing successive image frames for each pixel while initially learning the background image, and calculating a value equal to the difference. Because the Background Subtraction method learns the background image in advance, it can distinguish between the background image and non-moving objects. Therefore, a separate process for filtering the background image is required, which increases the amount of computation but improves accuracy. Therefore, the Background Subtraction method can be applied to specific areas that require very high detection accuracy, such as breeding farms where abnormal situations are suspected to have occurred or breeding farms with a very large number of individuals. In the Background Subtraction method, the background image can be continuously updated.

The second feature extraction unit 602 detects movement on the distribution map using an appropriate method according to the environment within the breeding farm and external settings. The above motion detection method is only an example, and methods that can display the area (e.g., pixel/block) where movement occurred within the frame can be used.

The process of the first feature extraction unit 600 generating position distribution data and the process of the second feature extraction unit 602 generating motion data may be performed simultaneously, in parallel, or sequentially. That is, this may mean that the process of the first feature extraction unit 600 generating position distribution data and the process of the second feature extraction unit 602 generating motion data are processed independently of each other.

The control unit 604 compares the position distribution data and motion data of the image data by region, block, or pixel, and selects abnormal object candidate data representing an abnormal object candidate or abnormal object data representing an abnormal object by region, block, or pixel. Stars can be created.

The control unit 604 may control the display of pixels for a plurality of objects on the image data according to the accumulated number of motion detections and the accumulated number of non-motion detections of the abnormal object data. For example, the control unit 604 may control pixel display for a plurality of blocks on image data according to abnormal entity data. Here, pixel display may include all concepts for distinguishing and displaying a pixel corresponding to an arbitrary point from other pixels, such as pixel saturation, pixel brightness, pixel color, pixel outline, and mark display. In an embodiment of the present invention, the display of pixels can be controlled by adjusting pixel values. Pixel values can be adjusted in stages, and pixels with high pixel values can be displayed with greater visual emphasis than pixels with low pixel values. However, it is not limited to this, and pixels with low pixel values can be set to be displayed with greater emphasis than pixels with high pixel values. Here, the pixel value may mean the probability of an abnormal object for each pixel.

Hereinafter, in order to explain the same thing as the pixel in which movement is detected and the object, one pixel is assumed to represent one object. This is for convenience of explanation, and in reality, multiple pixels represent one object. In other words, in order to determine an abnormal situation by detecting only the movement of some body regions of poultry, a method of detecting movement for each pixel and controlling the display of pixels will be used.

The control unit 604 may classify a specific object as an abnormal object the less movement is detected, classify it as a normal object as movement is detected, and display pixel values differently.

Here, the first feature extraction unit 600, the second feature extraction unit 602, and the control unit 604 may each receive first update information, second update information, and third update information from the learning server 400. And each of the first update information, second update information, and third update information may be applied to each of the first feature extraction unit 600, the second feature extraction unit 602, and the control unit 604. The first update information, second update information, and third update information may be update information extracted by the learning server 400 as a result of learning the learning data transmitted by the abnormal object detection device 100. Here, the learning data may include a part of the image data acquired by the abnormal object detection device 100 and a part of the abnormal object data with errors corrected, which allows the abnormal object detection device 100 to use an algorithm for detecting an abnormal object. It can be obtained by using the feedback information of the manager terminal 200 about the abnormal entity information obtained by driving. Here, each of the first update information, second update information, and third update information may include an adjustable matrix.

Referring to Figure 16, first, the nth image data is acquired. Here, the image data may be, for example, RGB data of size W x H (S1601). Here, the nth image data can be used interchangeably with nth original data, nth original image, and nth original image data.

The control unit 112 of the abnormal object detection apparatus 100, for example, the first feature extraction unit 600, detects an object in the n-th image data and generates location distribution data of the object for the n-th image data (S1602) ). Here, location distribution data may be generated for each region, block, or pixel, and the control unit 112 may use a first algorithm learned to suggest an area in the image data where the object is determined to be located. Here, the first algorithm may use a real-time object detection method using an algorithm that displays the area where the object exists, as described above. Alternatively, the first algorithm may use the distribution information of the object, that is, the area where the object is located. A method of representing density information can also be used. Here, the location distribution data may be a first density map.

The following description explains a method of calculating or displaying abnormal object data using pixel-by-pixel calculation of image data.

The update parameter μ value is applied to the location distribution data. Alternatively, the update parameter μ value and the offset parameter ε value may be applied simultaneously. μ may be a very small value, for example 0.001. In other words, position distribution data is controlled to be gradually displayed in pixels only when it is accumulated over a long period of time. The offset parameter ε is a parameter that adjusts the accumulation of position distribution data and can have a value of 1/10 to 1/100 of μ (S1603).

The control unit 112 of the abnormal object detection device 100, for example, the second feature extraction unit 602, compares the n-1th image data and the nth image data to detect the movement of the object for the nth image data. do. At this time, the n-1th image data may be stored in a latch circuit or buffer circuit. The second feature extraction unit 602 of the abnormal object detection apparatus 100 generates motion data according to the detected movement (S1604). Here, the motion data may be a motion map. Here, motion data can be generated for each region, block, or pixel, and the control unit 112 can use a second algorithm (motion detector) learned to detect motion using optical flow. An update parameter κ may be applied to the motion data. κ is a parameter that adjusts the accumulation of motion data (S1605).

The control unit 112 of the abnormal object detection device 100, for example, the control unit 604, adds the n-1th abnormal object data to the position distribution data of the nth image data (S1606) and motion data of the nth image data. Subtracts to generate abnormal entity data for the nth image data (S1607). Here, the abnormal object data for the nth image data may be the nth abnormal object density map.

The abnormal object detection device 100 according to an embodiment of the present invention repeats processes S1601 to S1607 to display the color of the object for which movement has been detected lightly or close to the original color, and the color of the object for which movement has not been accumulated and detected. can be controlled to display in a darker or closer to red color.

Thereafter, the nth or higher object data may be matched to the nth image data, that is, the nth original data, and an image in which the nth or higher object data is matched to the nth original data may be displayed on the administrator terminal 200. there is. That is, the area where the abnormal object is located in the nth image can be masked using the nth abnormal object density map, and the masked image can be displayed on the administrator terminal 200.

Equation 1 below can be applied to the operation of the abnormal object detection device 100 according to an embodiment of the present invention.

In Equation 1, μ is an update parameter and can be changed depending on settings.

In Equation 1, Pixel _t and Pixel _{t -1} are abnormal object data, and may mean the density of the pixel as a value for indicating the presence or absence of the abnormal object in the pixel. Pixels with a higher probability of an abnormal object being present are displayed in a darker color. For example, if there is no probability that an abnormal entity exists, it is displayed in primary color (white); the higher the probability that an abnormal entity exists, the closer to red it is displayed; if it is determined that the probability of an abnormal entity existing is very high, it is displayed in the darkest red. It is to make it happen. Therefore, Pixel _t and Pixel _{t -1} can be set to have a value between 0 and 1. The closer it is to 0, the closer it is to the primary color (white), and the closer it is to 1, the more it is displayed in red.

In Equation 1, Pixel _{t - 1} is abnormal object data of the previous frame in which position distribution data and motion data were accumulated. In Equation 1, Pixelt is abnormal object data updated by applying the position distribution data and motion data of the current frame.

In Equation 1, W _t may be position distribution data of the current frame. Location distribution data is the probability that an object exists in the corresponding pixel and can have a value between 0 and 1. The update parameter μ can be applied to location distribution data. μ may be a very small value, for example 0.001. In other words, position distribution data is controlled to be gradually displayed in pixels only when it is accumulated over a long period of time.

In Equation 1, F _t may be motion data of the current frame. The motion data is calculated from the absolute value of the motion vector and may have a value of 0 or more. The size of the motion vector corresponds to the speed of the object, so it can have a value greater than 1. No separate parameters are reflected in the motion data, so when motion is detected in the pixel, the display of the pixel is controlled to be initialized.

That is, the abnormal object data of the current frame includes the abnormal object presence probability (Pixelt), and the abnormal object presence probability (Pixel _t ) is the abnormal object presence probability (Pixel _{t -1} ) of the abnormal object data generated from the video data of the previous frame. ), the entity presence probability (Wt) of the location distribution data generated from the image data of the current frame, and the motion value (Ft) of the motion data generated from the image data of the current frame.

Alternatively, Equation 2 below can be applied to the operation of the abnormal object detection device according to the embodiment.

※μ: update parameter, ε: offset parameter

Equation 2 is an offset parameter ε added to Equation 1, and the same explanation as Equation 1 is omitted. The offset parameter ε is a parameter that adjusts the accumulation of position distribution data and can have a value of 1/10 to 1/100 of μ.

Alternatively, Equation 3 or Equation 4 below can be applied to the operation of the abnormal object detection device according to the embodiment.

※μ: update parameter, ε: offset parameter, κ: update parameter

Equation 3 or Equation 4 is the operation data Ft multiplied by the update parameter κ, and the same contents as Equation 1 and Equation 2 are omitted. Constant κ is a parameter that adjusts the accumulation of motion data.

Alternatively, Equation 5 below can be applied to the operation of the abnormal object detection device according to the embodiment.

Equation 5 is a formula to prevent the value of Equation 1, 2, 3, or 4 from falling below 0. For example, if the size of the motion data (F _t ) becomes larger than the sum of other parameter values, and the value of Equation 1, 2, 3, or 4 becomes a negative number less than 0, correct it so that it is displayed as 0. This is a control method.

FIG. 17 is a diagram illustrating a method in which an abnormal object detection device according to an embodiment of the present invention detects an abnormal object using results re-trained by a learning server.

Referring to Figure 17, the first update information, second update information, and third update information received from the learning server 400 during processes S1701 to S1707 may be applied. In order to obtain the first update information, the second update information, and the third update information, the abnormal object detection device 100 according to an embodiment of the present invention may extract training data and transmit it to the learning server 400. At this time, the learning data may include part of the image data acquired by the abnormal object detection device 100, which provides feedback on abnormal object information extracted by the abnormal object detection device 100 by running an algorithm for detecting abnormal objects. It can be obtained using information. Here, the feedback information may include information modified by the manager terminal 200.

More specifically, referring to FIG. 17(a), the abnormal object detection device 100 according to an embodiment of the present invention can extract training data and transmit it to the learning server 400, and the learning server 400 can detect abnormalities. The object density prediction network can be retrained, that is, updated, using the training data received from the object detection device 100. Here, the training data may include image data indicated as having an error and abnormal entity data with the error corrected. For example, if an abnormal object is detected to exist in some area 806 within the nth frame even though the abnormal object does not exist, the training data is the image data of the nth frame and the abnormal object data of the nth frame. Some areas 806 may include abnormal entity data in which errors have been corrected. Here, the abnormal object data with the error corrected may be information whose density of the object has been corrected by the manager terminal 200, and the density distribution of the object by region, block, or pixel extracted by the abnormal object detection device 100. In the case of a first density map, the density distribution of objects for each region, block, or pixel modified by the administrator terminal 200 may be referred to as a second density map. The learning server 400 calculates the loss by comparing the output image of the object density prediction network within the learning server 400 with the error-corrected image included in the learning data, that is, the correct image, and uses a learner to minimize the loss. You can learn (calibrate) the variables (e.g. feature map, etc.) of the object density prediction network in (trainer).

Thereafter, referring to FIG. 17(b), the control unit 112 of the abnormal object detection device 100 generates location distribution data, that is, in S1702, the object density network retrained by the learning server 400 is generated. Available. That is, the control unit 112 of the abnormal object detection device 100 uses training data from the learning server 400, for example, the n-th image, and update information obtained by re-learning using the second density map to detect the n-th image. Afterwards, an abnormal object density map for a given image can be output.

The term '~unit' used in this embodiment refers to software or hardware components such as FPGA (field-programmable gate array) or ASIC, and the '~unit' performs certain roles. However, '~part' is not limited to software or hardware. The '~ part' may be configured to reside in an addressable storage medium and may be configured to reproduce on one or more processors. Therefore, as an example, '~ part' refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, and procedures. , subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and 'parts' may be combined into a smaller number of components and 'parts' or may be further separated into additional components and 'parts'. Additionally, components and 'parts' may be implemented to regenerate one or more CPUs within a device or a secure multimedia card.

Although the present invention has been described above with reference to preferred embodiments, those skilled in the art may make various modifications and changes to the present invention without departing from the spirit and scope of the present invention as set forth in the claims below. You will understand that you can do it.

Claims (15)

a first feature extraction unit that generates position distribution data indicating the position distribution of the plurality of objects using image data generated from an image including the plurality of objects;
a second feature extraction unit that generates motion data indicating movement of a motion object among the plurality of objects using the generated image data; and
Abnormal entity candidate data representing an abnormal entity candidate is generated by comparing the first location distribution data and the first motion data generated during the first period, and the second position distribution data and the second motion data generated during the second period are It includes a control unit that generates abnormal entity data representing an abnormal entity by comparing it with abnormal entity candidate data,
The first period is a period in which feeding and water supply are not in progress, and the second period is a period in which at least one of feeding and water supply is in progress.
In paragraph 1.
The abnormal entity candidate data includes a set of blocks in which there is a probability of existence of the entity in the first location distribution data and no movement is detected in the first motion data,
The abnormal object data includes a set of blocks among the abnormal object candidates in which there is a probability of existence of the object in the second position distribution data but no movement is detected in the second motion data.
According to paragraph 1,
The control unit is an abnormal object detection device that distinguishes the first period and the second period through a feeding and watering time table or the first operation data and the second operation data.
According to paragraph 1,
The control unit outputs a driving command to at least one of a feeder and a waterer between the first period and the second period when an abnormal entity candidate is detected using the abnormal entity candidate data.
According to clause 4,
An abnormal object detection device wherein the control unit uses the abnormal object candidate data to output a driving command to at least one of a feeder and a waterer assigned to an area where the abnormal object candidate is detected.
According to clause 4,
The control unit uses the abnormal object candidate data to output a driving command to at least one of a feeder and a waterer assigned to an area excluding the area where the abnormal object candidate was detected.
According to paragraph 1,
An abnormal object detection device wherein the first feature extraction unit detects at least one area of a feeding area and a water supply area from image data generated during a second period.
In clause 7,
The control unit generates the abnormal object data through a change in the width of at least one of the feeding area and the water supply area.
According to any one of claims 4 to 6,
An abnormal object detection device wherein the control unit outputs a driving command to adjust the height of at least one of the feeder and the waterer. delete delete delete delete delete delete

Images