JP2021048797A - Activity amount management program, activity amount management system, and activity amount management method - Google Patents

Abstract

To provide an activity amount management program, an activity amount management system, and an activity amount management method which output information for detecting abnormality in a plurality of animals at early stages.SOLUTION: On the basis of each of a plurality of individuals 30-1 to 30-5 in a captured first image and each of the plurality of individuals 30-1 to 30-5 in a second image captured before or after the first image in terms of time, a travel amount of each of the individuals 30-1 to 30-5 between positions thereof in the first image and the second image is calculated and the calculated travel amount of each of the individuals is used to output an activity amount of the individuals 30-1 to 30-5 as a whole.SELECTED DRAWING: Figure 3

Description

The present invention relates to an activity amount management program, an activity amount management system, and an activity amount management method.

A technique is known in which an image in a barn (which may be referred to as a "livestock barn") is imaged and the degree of abnormality of a plurality of animals is determined based on the distribution of a plurality of animals in the captured image. .. For example, in this technique, when the distribution of the captured animal and the distribution of the animal in the reference image are not similar, it is determined that the degree of abnormality is high.

JP-A-2017-192316 International Publication No. 2018/179219 Pamphlet

Early detection of anomalies in a group of animals, for example, early calculation of information used to detect anomalies in a plurality of animals, is important for safe and healthy breeding of animals.

In the above-mentioned technique for determining the degree of abnormality based on the distribution of a plurality of animals, the distribution of a plurality of animals changes due to the anomaly (in other words, after the occurrence of the anomaly), and the degree of anomaly is determined to be high. In some cases, it may not be sufficient for early detection of anomalies.

In one aspect, it is an object of the present invention to output information for early detection of abnormalities in a plurality of animals.

In one aspect, the activity management program may cause the computer to perform the following processes: The process is based on each of the plurality of individuals in the first image captured and each of the plurality of individuals in the second image captured before and after the first image. The amount of movement of each of the plurality of individuals between the image and the second image may be calculated. Further, in the process, the activity amount as a group of the plurality of individuals may be output by using the calculated movement amount of each of the plurality of individuals.

In one aspect, it is possible to output information for early detection of abnormalities in a plurality of animals.

It is a block diagram which shows the structural example of the system which concerns on one Embodiment. It is a figure which shows an example of the imaging data of t = N. It is a figure which shows an example of the locus of movement of an animal moving on a floor plane. It is a figure which shows an example of the identification result of an individual in the captured image of t = N. It is a figure which shows an example of the identification result of the individual in the captured image of t = N + 1. It is a figure which shows an example of the individual movement amount between the captured images of t = N, N + 1. It is a figure which shows an example of the individual movement amount between the captured images of t = N, N + 1. It is a figure which shows an example of the activity amount calculation information. It is a figure which shows an example of the graph which plotted the measurement result of the activity amount for each day. It is a figure which shows an example of the sum total of the activity amount of a plurality of individuals per second. It is a figure which shows an example of the activity amount of an individual per second and per head. It is a figure which shows an example of the change of the body length of an individual. It is a flowchart which shows the operation example of a server. It is a block diagram which shows the hardware configuration example of the computer which realizes a server function.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the embodiments described below are merely examples, and can be variously modified and implemented without departing from the spirit. In the drawings used in the following embodiments, the parts having the same reference numerals represent the same or similar parts unless otherwise specified.

[1] Embodiment [1-1] Configuration example of one embodiment FIG. 1 is a block diagram showing a configuration example of the system 1 according to the embodiment. System 1 is an example of an information processing system that calculates the amount of activity of a plurality of animals in the captured image based on the captured image captured in the barn. The amount of activity may be used to detect anomalies in multiple animals.

As shown in FIG. 1, the system 1 may optionally include one or more (plurality in the example of FIG. 1) surveillance cameras 2 and a server 4. The surveillance camera 2 and the server 4 may be connected to each other so as to be able to communicate with each other by, for example, the network 5.

The network 5 may be at least one of an Internet and an intranet, including, for example, a LAN (Local Area Network) or a WAN (Wide Area Network), or a combination thereof. Further, the network 5 may include a virtual network such as a VPN (Virtual Private Network). The network 5 may be formed by one or both of a wired network and a wireless network.

The surveillance camera 2 may capture (image) the space in the shooting direction and acquire a plurality of images (which may be referred to as "frames") that are continuous in time series, such as an image series, for example, a moving image. .. The surveillance camera 2 captures an image of the inside or outside of the barn 3 accommodating a plurality of domestic animals (hereinafter, may be simply referred to as “animals”) 30 at a predetermined time interval T, for example. You may get the series. In one embodiment, the animal 30 is taken as an example of a "pig".

The surveillance camera 2 may be installed in, for example, a barn 3 accommodating a plurality of animals 30, and as an example, a lens surface is provided in the breeding space so that the breeding space existing inside or outside the barn 3 is included in the imaging range. It may be arranged on the ceiling surface or the wall surface of the livestock barn 3 so as to face it. The image captured by the surveillance camera 2 may be a bird's-eye view image of the breeding space viewed from above.

In order to include the entire breeding space in the imaging range, for example, a plurality of surveillance cameras 2 may be installed at positions where they can complement each other's blind spots (or some imaging ranges overlap), or one or more movable cameras. The surveillance camera 2 of the above may be installed, or these may be combined.

The surveillance camera 2 may transmit the acquired video to the server 4 via the network 5. For example, the surveillance camera 2 may store the acquired video in a recorder or the like (not shown) and transmit the data in the recorder to the server 4 at a predetermined timing. As the predetermined timing, various conditions such as the arrival of a predetermined time, the passage of a predetermined time, the storage capacity in the recorder, the number of storage frames, and the like may be used. Alternatively, the surveillance camera 2 may transmit the captured video to the server 4 without going through the recorder. When not via a recorder, for example, the surveillance camera 2 may transmit an image (in substantially real time) every one to several frames.

Examples of the surveillance camera 2 include a box-type camera, a dome camera, a network camera, and the like. Examples of network cameras include IP (Internet Protocol) cameras and the like. Further, a night-vision camera such as an infrared camera may be used as the surveillance camera 2. As the surveillance camera 2, for example, cameras for various purposes such as a security camera, a surveillance camera, and a fixed point camera may be used.

The server 4 identifies the individual position of the breeding animal 30 to be observed in the breeding space of the barn 3 based on the image taken by the surveillance camera 2, and calculates the amount of change in the position of each breeding animal 30 between the captured images. , Each change amount is used to output the activity amount of the breeding animal 30 as a whole group. The details of the server 4 will be described later.

The "activity amount" output by the server 4 or the "information regarding the change in the domestic animal" detected by using the activity amount may be provided to the terminal device 6 as shown in FIG. 1, for example. .. As an example, the server 4 may have a function of a Web server (for example, an information presentation unit 18 described later), and the function of the Web server provides "activity amount" or "information about an incident" via the network 5. The Web page to be represented may be displayed on the terminal device 6.

The terminal device 6 may be a computer that receives information on the amount of activity or the change output by the server 4, and may be, for example, a computer owned by a dairy farmer who owns or operates the barn 3. Examples of the terminal device 6 include various information processing devices such as a personal computer (PC) such as a desktop, laptop or mobile, a tablet, a smartphone, and a mobile phone. The terminal device 6 may optionally include means for inputting information (operation request) from the user, means for outputting information to the user, means for communicating with the server 4, and the like.

As illustrated in FIG. 1, when the terminal device 6 is a PC, tablet, smart phone, mobile phone or the like that performs wireless communication, the connection with the network 5 is performed via the mobile network via the base station 7. It may be.

[1-2] Server Configuration Example Next, a server configuration example will be described. The server 4 is an example of one or more information processing devices or computers installed in a livestock barn 3, a dairy farmer, a data center, or the like. Examples of the server 4 include various physical server devices and / or virtual server devices. At least a part of the functions of the server 4 may be realized by using, for example, resources, frameworks, and / or applications provided by a cloud service. Further, at least a part of the functions of the server 4 may be distributed or redundantly arranged in a plurality of computers. The server 4 realized by one or more physical server devices and / or virtual server devices is an example of an activity management system.

As shown in FIG. 1, the server 4 is exemplified by a memory unit 11, an imaging data acquisition unit 12, an individual identification unit 13, a movement amount calculation unit 14, a normalization unit 15, an activity amount calculation unit 16, and a determination unit 17. , And the information presentation unit 18.

The memory unit 11 stores various information used for the processing of the server 4. The information stored in the memory unit 11 will be described later in the description of the function of the server 4. The memory unit 11 includes a memory, for example, a volatile memory such as DRAM, a non-volatile memory such as PM, and a storage unit, for example, a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive). One or both can be mentioned. DRAM is an abbreviation for Dynamic Random Access Memory, and PM is an abbreviation for Persistent Memory.

The imaging data acquisition unit 12 acquires imaging data. For example, the imaging data acquisition unit 12 may receive the imaging data transmitted from the surveillance camera 2 via the network 5 and store it in the memory unit 11 as the imaging data 111 in the memory unit 11.

FIG. 2 is a diagram showing an example of imaging data 111 at time t = N (N is an integer and represents a frame number of the captured image). As shown in FIG. 2, the imaging data 111 includes images of a plurality of animals 30 located in space in the photographing direction of the surveillance camera 2.

The imaging data 111 used by the server 4 according to the embodiment may be an image series of captured images (frames) captured at a predetermined time interval T. An example of the predetermined time interval T is a time interval of about 0.1 second. In other words, the imaging data 111 may be an image series of about 10 FPS (Frame Per Second). The predetermined time interval T is not limited to this, and may be appropriately changed depending on various conditions such as the type of the domestic animal 30, the purpose of observation, the time zone, and the like.

The imaging data acquisition unit 12 may process the received imaging data and store it in the memory unit 11. For example, when the imaging interval of the received imaging data is shorter than the predetermined time interval T described above, the imaging data acquisition unit 12 has a surplus from the imaging data so that the imaging interval becomes the predetermined time interval T. The data obtained by thinning out the frames may be saved as the imaging data 111.

The imaging data acquisition unit 12 may record the imaging time of each captured image (frame) captured by the surveillance camera 2 in the memory unit 11 or the like. For example, the imaging data acquisition unit 12 may record and manage the shooting time for each frame as follows.

Shooting time of the first image: 2019/07/01 10: 00: 00.000
Shooting time of the second image: 2019/07/01 10: 00: 00.100
・ ・ ・

The individual identification unit 13, the movement amount calculation unit 14, the normalization unit 15, and the activity amount calculation unit 16 use the imaging data 111 acquired by the imaging data acquisition unit 12 to form a group of a plurality of domestic animals 30. Calculate the "activity amount". The amount of activity is information that quantitatively represents the degree of activity of the group, and is an example of information used for detecting anomalies of a plurality of domestic animals 30.

FIG. 3 is a diagram showing an example of the locus of movement of the animal 30 moving on the floor plane. As shown in FIG. 3, each of the plurality of animals 30 moves on the floor plane in the region k corresponding to the imaging range of the imaging data 111. For example, the locus of movement of animals 30-1 to 30-4 within a certain period of time is shown by a solid line (and a broken line outside the region k) of reference numerals S1 to S4, respectively. In addition, the plurality of animals 30-5 remain in a certain region within a certain period of time, as shown by a solid line. The fixed time may be longer than, for example, the predetermined time interval T.

Here, the abnormalities of the animals 30 bred in a group may be detected due to a change in the distribution of the animals 30 or the like due to the spread of an infectious disease or the like. Therefore, it is important to output information for early detection of abnormalities in a plurality of animals in order to keep the animals safely.

For example, the server 4 according to one embodiment calculates the movement locus of each of the plurality of domestic animals 30, for example, the movement amount by comparing the frames of the imaging data 111 obtained by the fixed point observation of the livestock barn 3. The amount of movement of each domestic animal 30 is one of the parameters that can be used to grasp, for example, the degree of activity of the domestic animal 30, in other words, the amount of activity.

4 and 5 are diagrams showing an example of the identification result of the individual 31 in the captured image (frame) of t = N and N + 1, respectively. The server 4 may identify each of the plurality of animals 30 as an individual 31 in each image of the captured data 111. The individual 31 may be represented, for example, by a rectangular area. In FIG. 4, the individual 31 identified in the first image of t = N is shown by a broken line, and in FIG. 5, the individual 31 identified in the second image of t = N + 1 is shown by a solid line.

The first image and the second image are two images taken consecutively in time series, and the second image means an image taken before and after (for example, after) the first image. You can do it.

FIG. 6 is a diagram showing an example of the amount of individual movement between captured images with t = N and N + 1. The server 4 calculates the movement amount 32 between the images of t = N and N + 1 for each animal 30 for the identified individual 31. The movement amount 32 may be calculated by various methods. As an example, the movement amount 32 may be calculated as the amount of change (for example, the center of gravity, the center, etc.) of a certain individual 31 in each image of t = N and N + 1.

The activity amount Ak of the animal 30 located in the region k of FIG. 3 may be calculated by the following formula (1) as an example. In the following formula (1), Si (i is an integer of 1 or more) indicates the moving distance of the domestic animal 30-i on a plane within a certain period of time. Si may be calculated by summing the amount of movement of each animal 30 at a predetermined time interval T over a certain period of time. Since the portions indicated by the broken lines in S1, S3, and S4 are outside the area k, that is, outside the shooting range of the surveillance camera 2, the amount of movement of the portions indicated by the broken lines is not calculated.

Ak = Σi (Si) (1)

The server 4 according to one embodiment continuously measures the amount of activity of the breeding animal 30 of the group. For example, the server 4 continuously analyzes the movement (activity amount) of the domestic animals 30 as a group. As a result, the server 4 can detect unusual conditions such as a sign of the spread of the disease in the domestic animal 30 as a group, a malfunction of the feeder, and the invasion of vermin due to the change in the amount of activity, and the server 4 can detect an unusual state at an early stage. Measures are possible.

Hereinafter, the individual identification unit 13, the movement amount calculation unit 14, the normalization unit 15, the activity amount calculation unit 16, the determination unit 17, and the information presentation unit 18 will be described. A series of processes by the individual identification unit 13, the movement amount calculation unit 14, the normalization unit 15, and the activity amount calculation unit 16 may be executed at predetermined calculation intervals. Examples of the predetermined calculation interval include, but are not limited to, several minutes to several tens of minutes, one hour to several hours, one day to several days, and the like, and the types of domestic animals 30 and observations. It may be changed as appropriate depending on various conditions such as purpose.

The individual identification unit 13 identifies each of the plurality of individuals 31 from each of the plurality of images included in the captured data 111, and registers the identification result in the activity amount calculation information 112 of the memory unit 11.

The identification result may include, for example, the size of each individual 31 in the image. As described above, the region where the individual 31 exists may be surrounded by a rectangular region. Therefore, examples of the size include the lengths of the long side and the short side of the rectangular region, for example, the number of pixels or the number of pixels. Further, the identification result may include coordinates such as absolute coordinates or relative coordinates from the reference point of the image with respect to the position of each individual 31, for example, the center of gravity, the center, or any of the four corners of the rectangular region. Good.

FIG. 7 is a diagram showing an example of the amount of individual movement between captured images of t = N and N + 1. FIG. 7 shows a case where movement of three animals 30A to 30C is confirmed in two images taken at an interval of 0.1 seconds. As illustrated in FIG. 7, the individual identification unit 13 may identify the individual 31A of the animal 30A, the individual 31B of the animal 30B, and the individual 31C of the animal 30C. In FIG. 7, the individuals 31A to 31C in the captured image of t = N of the captured data 111 are shown by broken lines, and the individuals 31A to 31C after movement in the captured image of t = N + 1 are shown by solid lines.

For example, it is assumed that the individual identification unit 13 identifies the sizes of the animals 30A to 30C shown in FIG. 7 as follows.

-Animal 30A: Rectangle width 100 pixels, Rectangle height 30 pixels-Animal 30B: Rectangle width 50 pixels, Rectangle height 30 pixels-Animal 30C: Rectangle width 70 pixels, Rectangle height 30 pixels

FIG. 8 is a diagram showing an example of the activity amount calculation information 112. As shown in FIG. 8, the individual identification unit 13 sets “rectangular width”, “rectangular height”, and “body length” for the entry corresponding to the date and time of individual identification in the activity amount calculation information 112. You may register every 31.

In the example of FIG. 8, the individual identification unit 13 identifies the entry of “2019/07/01 10: 00: 00” based on the captured image of “2019/07/01 10: 00: 00.000”. Each size of the animals 30A to 30C shown in No. 7 is registered (see the region of reference numeral A in FIG. 8). The larger of the width and the height may be registered in the "body length".

Various image analysis techniques may be used for the identification of the individual 31 by the individual identification unit 13, and in one embodiment, as an example, deep learning (DL; Deep Learning) using a neural network (NN) is used. ) Shall be used.

The movement amount calculation unit 14 calculates the movement amount 32 at a predetermined time interval T for each individual 31, and registers the calculation result in the activity amount calculation information 112 of the memory unit 11.

In the example shown in FIG. 7, the movement amount calculation unit 14 calculates that the movement amounts 32A to 32C of the individuals 31A to 31C between the two images of t = N and N + 1 are 10 pixels, 5 pixels, and 8 pixels, respectively. .. The amount of movement 32A to 32C may be obtained by calculating the amount of change in the coordinates of the center of gravity, the center, or the four corners of the individuals 31A to 31C between the two images of t = N and N + 1. Alternatively, a DL calculation model using NN may be used for the calculation of the movement amounts 32A to 32C, and various other image analysis techniques may be used.

As illustrated in FIG. 8, the movement amount calculation unit 14 may register the “movement amount” for each individual 31 for the entry corresponding to the date and time when the movement amount calculation is performed in the activity amount calculation information 112. ..

In the example of FIG. 8, the movement amount calculation unit 14 responds to the entry of “2019/07/01 10:00:00” by “2019/07/01 10: 00: 00.000” and “2019/07/01”. Each movement amount 32 of the animals 30A to 30C shown in FIG. 7 calculated based on the captured image of "10: 00: 00.100" is registered. The movement amount calculation unit 14 uses the movement amount 32 during the predetermined time interval T (see the region of reference numeral B in FIG. 8), for example, the movement amount 32 for 0.1 seconds in the example of FIG. It may be registered in the calculation information 112.

The normalization unit 15 normalizes (in other words, corrects) the "movement amount" (in other words, the movement distance) of the individual 31 registered in the activity amount calculation information 112 according to the size of each individual 31. The converted value is registered in the activity amount calculation information 112.

For example, the normalization unit 15 may calculate the movement amount per second by dividing the “movement amount” calculated by the movement amount calculation unit 14 by a predetermined time interval T.

In addition, the normalization unit 15 divides the movement amount per second by the "body length" identified by the individual identification unit 13, so that the movement amount per second (normalization) by the animal 30 having the normalized body length. The amount of movement) may be calculated.

The normalization of the amount of movement by body length by the normalization unit 15 is such that the smaller individual 31 among the two or more individuals 31 having the same movement distance is relative to the other 31 among the two or more individuals 31. It can be said that the correction is made so that the moving distance becomes large.

In the example of FIG. 8, the normalization unit 15 sets the movement amounts of animals 30A to 30C of 10, 5, and 8 pixels for the entry of “2019/07/01 10:00:00” at a predetermined time interval T. By dividing by = 0.1, 100, 50, and 80 pixels are calculated as the amount of movement per second, respectively. Further, the normalization unit 15 divides the movement amount per second by the body lengths of animals 30A to 30C of 100, 50, and 70 pixels, respectively, so that the normalization movement amount is 1.00, 1.00, and 1, respectively. Calculate .14. Each movement amount calculated by the normalization unit 15 may be registered in the activity amount calculation information 112 as shown in the area of reference numeral C in FIG.

In this way, the movement amount calculation unit 14 and the normalization unit 15 calculate the movement amount of each of the plurality of individuals 31 based on each of the plurality of individuals 31 in the captured images of t = N and N + 1. This is an example.

The activity amount calculation unit 16 calculates the activity amount of the entire group of the plurality of animals 30 based on the normalized movement amount. The activity amount of the group of the plurality of animals 30 as a whole is a normalization of the movement amount of the entire group of the plurality of animals 30 photographed by the body length. The activity amount of the plurality of animals 30 as a whole is a normalization of the activity amount of the plurality of animals 30 to the activity amount per second and per animal.

For example, the activity amount calculation unit 16 calculates the total amount of the normalized movement amount for each entry corresponding to the date and time when the activity amount calculation information 112 is normalized, and the total amount is calculated as the number of the identified animals 30 ( The amount of activity may be calculated by dividing by the number of animals).

In the example of FIG. 8, the activity amount calculation unit 16 responds to the entry of “2019/07/01 10:00:00” with the normalized movement amounts of animals 30A to 30C 1.00, 1.00, 1. The sum of 3.14 of 14 is calculated, and the sum is divided by the number of animals 30 (= 3) to obtain an activity amount of 1.05. The sum of the normalized movement amounts and the activity amount calculated by the activity amount calculation unit 16 may be registered in the activity amount calculation information 112 as shown in the area of reference numeral D in FIG. The amount of activity indicates how much the animal 30 moved with respect to the body length in one second. For example, when the amount of activity is 1.00, the amount of activity indicates that the animal 30 has moved by its own body length. Shown.

In the example shown in FIG. 8, the amount of activity per hour is calculated, but the activity amount is not limited to this, and the amount of activity may be calculated every day, for example. Further, the activity amount calculation unit 16 averages or totals the activity amount calculated every hour on a daily basis, and registers the average value or the total value as the daily activity amount in the activity amount calculation information 112. May be good.

As described above, the activity amount calculation unit 16 is an example of an output unit that outputs the activity amount of the group of the plurality of individuals 31 as a whole by using the calculated movement amount of each of the plurality of individuals 31.

The determination unit 17 determines whether or not an abnormality has occurred in the plurality of animals 30 based on the activity amount of the entire group of the plurality of animals 30 calculated by the activity amount calculation unit 16.

FIG. 9 is a diagram showing an example of a graph plotting the measurement results of the daily activity amount. As shown in FIG. 9, with the passage of days, in other words, with the growth of the animal 30, the amount of activity tends to decrease. It is considered that this is related to the fact that the growth of the animal 30 reduces the movement and narrows the range of activity.

As illustrated in FIG. 9, since the activity amount tends to decrease as the animal 30 grows, the determination unit 17 calculates the activity amount that deviates from such a tendency by the activity amount calculation unit 16. In this case, it may be determined that the abnormality has occurred in the plurality of animals 30.

For the above-mentioned determination, the server 4 (for example, the activity amount calculation unit 16) stores (accumulates) the activity amount calculation information 112 calculated in the past breeding cycle in the memory unit 11 as a growth result, in other words, comparison data. ) May be done. By using the activity amount calculation information 112 calculated in one or more breeding cycles in the past as comparative data, it can be appropriately or easily determined whether or not an abnormality has occurred in a plurality of animals 30. can do.

In addition, the amount of activity may fluctuate depending on conditions such as the breeding time (season, etc.) of the animal 30, the temperature and humidity of the barn 3, and the time zone. By using the activity amount calculation information 112 calculated under various past conditions as the comparison data, the determination unit 17 appropriately determines whether or not the animal 30 has an abnormality based on the comparison data that matches the current conditions. Can be determined.

As an example, the memory unit 11 may store the activity amount calculation information 112 which is the growth result of a plurality of past breeding cycles (breeding seasons) and in which the animal 30 has not changed. The past activity amount calculation information 112 may be graphed as, for example, ideal transition information of the activity amount. This graph may have, for example, a "width" including an upper limit and a lower limit of the amount of activity, and may include information such as the shooting time zone of the captured image and the age or moon age of the animal 30.

In this way, the activity amount calculation unit 16 outputs the activity amount output using the movement amount calculated based on the first image and the second image captured in the past as the first image or the second image captured in the past. This is an example of a storage unit stored in the memory unit 11 for each imaging date and time.

When the determination unit 17 determines that an abnormality has occurred in a plurality of animals 30, the information presentation unit 18 presents information regarding the determined abnormality (outputs an alert). The information regarding the incident includes at least one of a notification that the incident may have occurred, at least a part of the activity amount calculation information 112, and a cause estimated from the tendency of the activity amount to increase or decrease. May be included.

For example, an abnormal increase in the amount of activity may be caused by vermin or invasion of a person into the barn 3 for the purpose of theft. Further, for example, an abnormal decrease in the amount of activity may be caused by the spread of infectious diseases, deterioration of the physical condition of the domestic animal 30 due to a failure of the feeder or water dispenser, or the like. Therefore, the information presenting unit 18 may present the invasion of vermin and humans into the barn 3, the deterioration of the physical condition of the domestic animal 30, and the like as the causes presumed from the tendency of the amount of activity to increase or decrease.

For example, the information presentation unit 18 may notify the dairy farmer, the manager of the system 1, and the like of information by e-mail, push notification, SMS (Short Message Service), or various other methods. The information presentation unit 18 may display or output the above-mentioned information to various output devices such as a monitor, a projector, or a printer included in the server 4.

In this way, the information presentation unit 18 has a plurality of individuals 31 based on the activity amount output by the activity amount calculation unit 16 and the plurality of activity amounts stored in the memory unit 11 (past activity amount calculation information 112). This is an example of an alert output unit that outputs an alert regarding an incident.

Based on the amount of activity output by the amount of activity calculation unit 16, the dairy farmer raises the number of animals 30 such as whether the spread of infectious diseases has occurred, whether there is a shortage of food or water, and the like. You can quickly notice unusual changes in the animal 30. In addition, based on such amount of activity, dairy farmers can quickly notice the invasion of vermin and the invasion of people for the purpose of theft. Therefore, the dairy farmer does not have to perform 24-hour monitoring, for example, monitoring by patrol, monitoring of the image of the surveillance camera 2, and the like, which is highly convenient.

As described above, according to the server 4 according to one embodiment, the individual 31 is recognized from the two captured images by, for example, DL, the movement amount of each individual 31 is measured, and a plurality of movement amounts are used. The amount of activity of the individual 31 group as a whole is output.

The server 4 can appropriately detect the abnormalities of the plurality of animals 30 by using the activity amount of the group of the plurality of individuals 31 as a whole. In addition, since the activity amount of the group of the plurality of individuals 31 can be easily calculated by the calculation using the movement amount of the plurality of individuals 31, information for early detection of the abnormality of the plurality of animals 30 is output. be able to. Further, the server 4 can detect the occurrence of an unusual situation such as an increase or decrease in the activity amount at an early stage by continuously measuring the activity amount in a predetermined cycle.

Further, since the activity amount can be easily output by periodically performing arithmetic processing using the imaging data 111, the activity amount of the group of the animals 30 within the range photographed by the surveillance camera 2 can be measured. It is possible to suppress an increase in the processing load on the server 4 while enabling the above.

Further, since the server 4 performs the normalization process in consideration of the size of the individual 31, for example, the body length when calculating the activity amount, the activity amount can be calculated based on the size of the individual 31, and the surveillance camera 2 is installed. The amount of activity that does not depend on the position or camera resolution can be output.

Here, for example, consider a case where the occurrence of anomalies is determined by using the sum of the activity amounts of a plurality of individuals 31 per second. FIG. 10 is a diagram showing an example of the total amount of activity of a plurality of individuals 31 per second. As illustrated in FIG. 10, the total amount of activity of the plurality of individuals 31 per second does not seem to change (flat) with the passage of days, but the number of animals 30 within the imaging range of the breeding space increases or decreases. Depending on the situation, the total amount of activity may increase or decrease. Therefore, the total amount of activity is insufficient as an index for quantitatively evaluating the entire animals 30 to be bred.

FIG. 11 is a diagram showing an example of the amount of activity of the individual 31 per second and per animal, and FIG. 12 is a diagram showing an example of a change in the body length of the individual 31. The vertical axis of FIG. 11 shows the amount of activity obtained by dividing the total amount of activity shown in FIG. 10 by the number of animals 30 (identified by the individual identification unit 13) within the imaging range.

As illustrated in FIG. 11, the range of the amount of activity is around 1.04 to 1.06, and a slight decreasing tendency is observed as the animal 30 grows. However, since the server 4 measures the movement amount of the animal 30 by the number of pixels, there is a possibility that the movement amount is greatly measured as the animal 30 grows (see FIG. 12). Therefore, the amount of activity of the individual 31 per second and per animal is insufficient as an index for quantitatively evaluating the entire animal 30 to be bred.

For the above reasons, the server 4 according to the embodiment outputs the activity amount of the entire group of the plurality of individuals 31 by normalizing the activity amount of the individual 31 per second and per animal according to the body length. (See FIG. 9).

[1-3] Operation example of one embodiment Next, an operation example of the server 4 according to the above-described embodiment will be described. FIG. 13 is a flowchart showing an operation example of the server 4.

As illustrated in FIG. 13, the server 4 waits for the arrival of the activity amount calculation timing (No in step S1 and step S1). The activity amount calculation timing may arrive at a predetermined calculation interval such as one hour.

When the activity amount calculation timing arrives (Yes in step S1), the imaging data acquisition unit 12 acquires a plurality of captured images and shooting dates from the captured data captured by the surveillance camera 2, and stores the captured data 111 in the memory unit 11. Store (step S2). The plurality of captured images may include a first image captured and a second image captured before and after the first image (for example, at a predetermined time interval T).

Note that step S2 may be executed each time the imaging data is received from the surveillance camera 2 before the determination in step S1.

The imaging data acquisition unit 12 identifies the positions and sizes of the plurality of individuals 31 from each of the plurality of captured images based on the imaging data 111 (step S3), and registers the identification results in the activity amount calculation information 112.

The movement amount calculation unit 14 calculates the movement amount 32 for each individual 31 based on the identification result (step S4), and registers the calculation result in the activity amount calculation information 112.

The normalization unit 15 normalizes the movement amount 32 to the movement amount per second (step S5), normalizes the movement amount per second by the body length (step S6), and calculates the normalized movement amount as the activity amount. Register in information 112. In addition, steps S5 and S6 may be executed in the reverse order.

The activity amount calculation unit 16 sums the movement amounts of the plurality of individuals 31 (step S7), and registers the total movement amount in the activity amount calculation information 112. Further, the activity amount calculation unit 16 calculates the activity amount by dividing the total movement amount by the number of individuals (step S8), and registers the calculated activity amount in the activity amount calculation information 112.

The determination unit 17 combines the activity amount of the activity amount calculation information 112 registered in steps S3 to S8 with the past activity amount calculation information 112 stored in the memory unit 11 by the activity amount calculation unit 16, that is, the past growth result (comparison). Data) is compared (step S9).

The determination unit 17 determines whether or not the occurrence of anomalies in a plurality of animals 30 has been detected by comparison (step S10). If it is determined that the occurrence of an abnormality has not been detected (No in step S10), the process proceeds to step S1.

On the other hand, when it is determined that the occurrence of the incident has been detected (Yes in step S10), the information presenting unit 18 presents the information regarding the detected incident (step S11), and the process proceeds to step S1.

[1-4] Hardware Configuration Example FIG. 14 is a block diagram showing a hardware (HW) configuration example of the computer 10 that realizes the function of the server 4. When a plurality of computers are used as HW resources that realize the functions of the server 4, each computer may have the HW configuration illustrated in FIG.

As shown in FIG. 14, the computer 10 has an HW configuration, for example, a processor 10a, a memory 10b, a storage unit 10c, an IF (Interface) unit 10d, an I / O (Input / Output) unit 10e, and a reading unit. It may be provided with 10f.

The processor 10a is an example of an arithmetic processing unit that performs various controls and operations. The processor 10a may be connected to each block in the computer 10 so as to be able to communicate with each other by the bus 10i. The processor 10a may be a multiprocessor including a plurality of processors, a multicore processor having a plurality of processor cores, or a configuration having a plurality of multicore processors.

Examples of the processor 10a include integrated circuits (ICs) such as CPUs, MPUs, GPUs, APUs, DSPs, ASICs, and FPGAs. A combination of two or more of these integrated circuits may be used as the processor 10a. CPU is an abbreviation for Central Processing Unit, and MPU is an abbreviation for Micro Processing Unit. GPU is an abbreviation for Graphics Processing Unit, and APU is an abbreviation for Accelerated Processing Unit. DSP is an abbreviation for Digital Signal Processor, ASIC is an abbreviation for Application Specific IC, and FPGA is an abbreviation for Field-Programmable Gate Array.

The memory 10b is an example of an HW that stores information such as various data and programs. Examples of the memory 10b include a volatile memory such as DRAM and a non-volatile memory such as PM.

The storage unit 10c is an example of an HW that stores information such as various data and programs. Examples of the storage unit 10c include a magnetic disk device such as an HDD, a semiconductor drive device such as an SSD, and various storage devices such as a non-volatile memory. Examples of the non-volatile memory include flash memory, SCM (Storage Class Memory), ROM (Read Only Memory) and the like.

Further, the storage unit 10c may store a program 10g (activity amount management program) that realizes all or a part of various functions of the computer 10. For example, the processor 10a of the server 4 can realize the function as the server 4 by expanding and executing the program 10g stored in the storage unit 10c in the memory 10b.

The IF unit 10d is an example of a communication IF that controls connection with a network and communication. For example, the IF unit 10d may include an adapter compliant with LAN, optical communication (for example, FC (Fibre Channel)), or the like. The adapter may support one or both wireless and wired communication methods. For example, the program 10g may be downloaded from the network to the computer 10 via the communication IF and stored in the storage unit 10c.

The I / O unit 10e may include one or both of an input device and an output device. Examples of the input device include a keyboard, a mouse, a touch panel, and the like. Examples of the output device include a monitor, a projector, a printer and the like.

The reading unit 10f is an example of a reader that reads data and program information recorded on the recording medium 10h. The reading unit 10f may include a connection terminal or device to which the recording medium 10h can be connected or inserted. Examples of the reading unit 10f include an adapter compliant with USB (Universal Serial Bus) and the like, a drive device for accessing a recording disk, a card reader for accessing a flash memory such as an SD card, and the like. The program 10g may be stored in the recording medium 10h, or the reading unit 10f may read the program 10g from the recording medium 10h and store it in the storage unit 10c.

Examples of the recording medium 10h include a non-temporary computer-readable recording medium such as a magnetic / optical disk or a flash memory. Examples of magnetic / optical disks include flexible discs, CDs (Compact Discs), DVDs (Digital Versatile Discs), Blu-ray discs, HVDs (Holographic Versatile Discs), and the like. Examples of the flash memory include semiconductor memories such as USB memory and SD card.

The HW configuration of the computer 10 described above is an example. Therefore, the increase / decrease of HW (for example, addition or deletion of arbitrary blocks), division, integration in any combination, addition or deletion of buses, etc. in the computer 10 may be performed as appropriate. For example, in the server 4, at least one of the I / O unit 10e and the reading unit 10f may be omitted.

[2] Others The technology according to the above-described embodiment can be modified or modified as follows.

For example, the functional blocks shown in FIG. 7 may be merged or divided in any combination.

Further, the server 4 may output (transmit) the calculated activity amount to another server. In this case, a configuration in which the server 4 does not include the information presentation unit 18 illustrated in FIG. 7 or the determination unit 17 and the information presentation unit 18 may be allowed.

Further, in one embodiment, the pig is taken as an example as the breeding animal 30, but the present invention is not limited to this, and the breeding animal 30 may be various animals such as cows and chickens.

Further, in one embodiment, a method of detecting anomalies of a plurality of animals 30 by using an amount of activity has been described, but the present invention is not limited to this. As mentioned above, the amount of activity of the animal 30 tends to decrease as it grows. Utilizing this tendency, the server 4 can determine the growth state of the animal 30 by comparing the amount of activity with the past comparison data.

For example, the determination unit 17 may determine an appropriate shipping time of the animal 30 by comparing the amount of activity with past comparative data corresponding to the shipping time of the animal 30. As an example, the determination unit 17 determines that the animal 30 can be shipped when the activity amount calculated by the activity amount calculation unit 16 is equal to or greater than the activity amount of the animal 30 that has reached the shipping time in the comparison data. You can do it. In this case, the information presentation unit 18 may notify the dairy farmer that the animal 30 can be shipped.

[3] Additional Notes The following additional notes will be further disclosed with respect to the above embodiments.

(Appendix 1)
The first image and the first image are based on each of the plurality of individuals in the first image captured and each of the plurality of individuals in the second image captured before and after the first image. The amount of movement of each of the plurality of individuals between the two images is calculated.
Using the calculated movement amount of each of the plurality of individuals, the activity amount of the plurality of individuals as a whole is output.
An activity management program that lets a computer perform processing.

(Appendix 2)
The calculation is the result of correcting the moving distance of each of the plurality of individuals between the first image and the second image according to the size of each of the plurality of individuals, and the result of each of the plurality of individuals. The amount of movement,
The activity management program described in Appendix 1.

(Appendix 3)
The correction is made so that the smaller the size of two or more individuals having the same movement distance, the larger the movement distance is relative to the other individuals of the two or more individuals.
The activity management program described in Appendix 2.

(Appendix 4)
The output outputs the result of dividing the sum of the movement amounts of each of the plurality of individuals by the number of the plurality of individuals as the activity amount.
The activity management program described in Appendix 2 or Appendix 3.

(Appendix 5)
By image analysis based on the first image and the second image, the size of each of the plurality of individuals is specified.
The activity amount management program according to any one of Supplementary notes 1 to 4, which causes the computer to execute the process.

(Appendix 6)
The amount of activity output using the movement amount calculated based on the first image and the second image captured in the past is stored in the storage area for each imaging date and time of the first image or the second image captured in the past. And
An alert regarding an abnormality of the plurality of individuals is output based on the output amount of activity and the plurality of the activity amounts stored in the storage area.
The activity amount management program according to any one of Supplementary notes 1 to 5, which causes the computer to execute the process.

(Appendix 7)
Each of the plurality of individuals is an image region showing a domestic animal to be observed identified based on the first image and the second image.
The first image and the second image are images obtained by fixed-point imaging of the inside of a barn accommodating a plurality of the domestic animals at predetermined time intervals.
The activity management program according to any one of Supplementary notes 1 to 6.

(Appendix 8)
The first image and the first image are based on each of the plurality of individuals in the first image captured and each of the plurality of individuals in the second image captured before and after the first image. A calculation unit that calculates the amount of movement of each of the plurality of individuals between the two images,
It is provided with an output unit that outputs the activity amount of the plurality of individuals as a whole by using the calculated movement amount of each of the plurality of individuals.
Activity management system.

(Appendix 9)
The calculation unit corrects the moving distance of each of the plurality of individuals between the first image and the second image according to the size of each of the plurality of individuals, and corrects the result of each of the plurality of individuals. The amount of movement of
The activity management system described in Appendix 8.

(Appendix 10)
The calculation unit corrects so that the smaller the size of two or more individuals having the same movement distance, the larger the movement distance is relative to the other individuals of the two or more individuals.
The activity management system described in Appendix 9.

(Appendix 11)
The output unit outputs the result of dividing the total movement amount of each of the plurality of individuals by the number of the plurality of individuals as the activity amount.
The activity management system according to Appendix 9 or Appendix 10.

(Appendix 12)
The amount of activity output using the movement amount calculated based on the first image and the second image captured in the past is stored in the storage area for each imaging date and time of the first image or the second image captured in the past. Storage unit and
It is provided with an alert output unit that outputs an alert regarding an abnormality of the plurality of individuals based on the output amount of activity and the plurality of said activity amounts stored in the storage area.
The activity management system according to any one of Appendix 8 to 11.

(Appendix 13)
The first image and the first image are based on each of the plurality of individuals in the first image captured and each of the plurality of individuals in the second image captured before and after the first image. The amount of movement of each of the plurality of individuals between the two images is calculated.
Using the calculated movement amount of each of the plurality of individuals, the activity amount of the plurality of individuals as a whole is output.
An activity management method in which a computer executes processing.

(Appendix 14)
The calculation is the result of correcting the moving distance of each of the plurality of individuals between the first image and the second image according to the size of each of the plurality of individuals, and the result of each of the plurality of individuals. The amount of movement,
The activity amount management method according to Appendix 13.

(Appendix 15)
The correction is made so that the smaller the size of two or more individuals having the same movement distance, the larger the movement distance is relative to the other individuals of the two or more individuals.
The activity amount management method according to Appendix 14.

(Appendix 16)
The output outputs the result of dividing the sum of the movement amounts of each of the plurality of individuals by the number of the plurality of individuals as the activity amount.
The activity amount management method according to Appendix 14 or Appendix 15.

(Appendix 17)
The amount of activity output using the movement amount calculated based on the first image and the second image captured in the past is stored in the storage area for each imaging date and time of the first image or the second image captured in the past. And
An alert regarding an abnormality of the plurality of individuals is output based on the output amount of activity and the plurality of the activity amounts stored in the storage area.
The activity amount management method according to any one of Supplementary note 13 to 16, wherein the processing is executed by the computer.

1 System 10 Computer 11 Memory unit 111 Imaging data 112 Activity calculation information 12 Imaging data acquisition unit 13 Individual identification unit 14 Movement amount calculation unit 15 Normalization unit 16 Activity calculation unit 17 Judgment unit 18 Information presentation unit 2 Surveillance camera 3 Livestock barn 30, 30-1 to 30-5, 30A to 30C Domestic animals 31, 31A to 31C Individuals 32, 32A to 32C Movement amount 4 Server 5 Network 6 Terminal equipment 7 Base station

Claims (9)

The first image and the first image are based on each of the plurality of individuals in the first image captured and each of the plurality of individuals in the second image captured before and after the first image. The amount of movement of each of the plurality of individuals between the two images is calculated.
Using the calculated movement amount of each of the plurality of individuals, the activity amount of the plurality of individuals as a whole is output.
An activity management program that lets a computer perform processing. The calculation is the result of correcting the moving distance of each of the plurality of individuals between the first image and the second image according to the size of each of the plurality of individuals, and the result of each of the plurality of individuals. The amount of movement,
The activity management program according to claim 1. The correction is made so that the smaller the size of two or more individuals having the same movement distance, the larger the movement distance is relative to the other individuals of the two or more individuals.
The activity management program according to claim 2. The output outputs the result of dividing the sum of the movement amounts of each of the plurality of individuals by the number of the plurality of individuals as the activity amount.
The activity management program according to claim 2 or 3. By image analysis based on the first image and the second image, the size of each of the plurality of individuals is specified.
The activity amount management program according to any one of claims 1 to 4, which causes the computer to execute the process. The amount of activity output using the movement amount calculated based on the first image and the second image captured in the past is stored in the storage area for each imaging date and time of the first image or the second image captured in the past. And
An alert regarding an abnormality of the plurality of individuals is output based on the output amount of activity and the plurality of the activity amounts stored in the storage area.
The activity amount management program according to any one of claims 1 to 5, which causes the computer to execute the process. Each of the plurality of individuals is an image region showing a domestic animal to be observed identified based on the first image and the second image.
The first image and the second image are images obtained by fixed-point imaging of the inside of a barn accommodating a plurality of the domestic animals at predetermined time intervals.
The activity management program according to any one of claims 1 to 6. The first image and the first image are based on each of the plurality of individuals in the first image captured and each of the plurality of individuals in the second image captured before and after the first image. A calculation unit that calculates the amount of movement of each of the plurality of individuals between the two images,
It is provided with an output unit that outputs the activity amount of the plurality of individuals as a whole by using the calculated movement amount of each of the plurality of individuals.
Activity management system. The first image and the first image are based on each of the plurality of individuals in the first image captured and each of the plurality of individuals in the second image captured before and after the first image. The amount of movement of each of the plurality of individuals between the two images is calculated.
Using the calculated movement amount of each of the plurality of individuals, the activity amount of the plurality of individuals as a whole is output.
An activity management method in which a computer executes processing.

Images