JP7406746B2 - Monitoring device, monitoring program, storage medium, and monitoring method - Google Patents

Description

The present invention relates to a monitoring device, a monitoring program, a storage medium, and a monitoring method, and particularly to a monitoring device, a monitoring program, a storage medium, and a monitoring method using a mobile monitoring device (such as a drone).

Surveillance cameras have been used for a long time, but with the spread of Internet connections, they are increasingly connected to networks and monitored by remote servers. Additionally, the number of pixels in cameras has increased, making it possible to analyze recorded images after the fact and recognize accidents and crimes. The current state of surveillance cameras is such that the most common way to use them is to install unmanned surveillance cameras in buildings, and have crime prevention personnel from security companies or other organizations monitor the footage from the cameras in remote locations to help prevent crime. It is something. Crime prevention personnel actually observe multiple images from surveillance cameras on multiple display devices, recognize suspicious objects (suspicious persons or objects), and, if necessary, dispatch security personnel or take action against suspicious persons. I'm giving you a warning.

However, crime prevention personnel have to view surveillance camera images of various locations and buildings.In recent years, the number of surveillance cameras installed has increased rapidly due to terrorist attacks and crimes. This situation increases the burden on crime prevention personnel who monitor the situation, leading to monitoring errors.

On the other hand, control technology for small flying objects called drones has rapidly improved, and it has become possible to fly them indoors with considerable precision and stability. Additionally, drones are equipped with small cameras, can be controlled by radio, and can fly autonomously along programmed routes. If such a surveillance system using drones were put into practical use, it would be possible to monitor safely and at low cost, but such a system has not yet been developed.

BACKGROUND ART As conventional technology for automating surveillance cameras, there are image processing devices, image processing systems, and programs (see Patent Document 1). This method involves setting a gaze area within a panoramic image of the monitoring area, using an image analysis means to analyze the image in the set gaze area to detect a moving object, and operating the zoom camera in the camera unit based on the detection. The moving object is photographed. At this time, spatial relationship information with the panoramic image is created in the focused partial image, and temporal relationship information indicating the transition of the focused partial image on the time axis in the panoramic image is created. Each of the above-mentioned relational information is added to the image and stored in the image storage unit. At the time of search, the above-mentioned each of the relational information is used to search for the panoramic image and the focused partial image, and the two images are superimposed and displayed on the display unit." This is the technology.

Japanese Patent Application Publication No. 2010-233185

In the above-mentioned conventional technology, after a monitoring area is set, a moving object within the monitoring area is zoomed in and photographed as a suspicious object. This is a technology that automatically tracks and zooms in on suspicious individuals, but it cannot "automatically set the monitoring area." Furthermore, if the monitoring area is large or if multiple moving objects enter the monitoring area, a high-speed computing device will not be able to handle the processing in time, resulting in disadvantages such as the inability to capture suspicious objects.

As described above, although the technology for automating surveillance cameras has been developed, the technology for identifying, specifying, and authenticating objects to be monitored has not yet reached a level of practical use.

Furthermore, although there have been attempts to mount surveillance cameras on mobile devices such as drones, no technology has been developed to set a flight route for a drone to conduct surveillance inside a building. In particular, there is a problem that programmed autonomous navigation using GPS signals cannot be carried out because GPS signals do not reach inside buildings. Furthermore, the use of drones can significantly reduce labor costs, and they are also suitable for use in dangerous locations and locations where it is difficult for people to enter and exit for long periods of time. In addition to drones, indoor mobile monitoring devices can significantly reduce labor costs and are suitable for use in dangerous locations and locations where it is difficult for people to enter and exit for long periods of time.

Therefore, an object of the present invention is to provide a monitoring device, a monitoring program, a storage medium, and a monitoring method using a drone, which sets a flight route based on CAD information.

Another object of the present invention is to provide a monitoring device, a monitoring program, a storage medium, and a monitoring method using a mobile monitoring device.

In order to solve the above-mentioned problems, a monitoring device according to a first invention has the following features:
A monitoring device including a drone and a server,
The drone,
an imaging unit that captures images during flight;
a communication unit that receives the flight route and position indicator placement information;
a control unit that controls the flight of the own device along the flight route while recognizing the position of the own device based on the position indicator arrangement information;
a monitoring unit that monitors whether a monitoring event has occurred in the image;
an output unit that, when a monitoring event occurs, outputs (sends) an alarm including the position where the monitoring event occurs and an image at the time of occurrence to the server via the communication unit or other communication means;
has
The server is
an acquisition unit that acquires CAD information of a building, position index arrangement information, and layout information of monitoring target installations (chicken house cages, desks, chairs, equipment, etc.) placed in the building;
a flight route setting unit configured to set a flight route of the drone within the building for monitoring an installation to be monitored, or to receive an input of the flight route;
a monitoring event setting unit for setting a monitoring event to be monitored (abnormal event, intruder, fire, death, specific action, etc.);
a communication unit that transmits the flight route, the position indicator arrangement information, and the monitoring event to the drone, and receives from the drone an alarm including the position where the monitoring event occurred and an image at the time of occurrence;
has,
It is characterized by

Further, the monitoring device according to the second invention includes:
A monitoring device including a drone and a server,
The drone,
an imaging unit that captures images during flight;
a communication unit that receives the flight route and position indicator placement information;
a control unit that controls the flight of the own device along the flight route while recognizing the position of the own device based on the position indicator arrangement information;
a monitoring unit that monitors whether a monitoring event has occurred in the image;
When a monitoring event occurs, the control unit is controlled to reduce the flight speed or stop in the air, and when the monitoring event continues for a predetermined time or more, the occurrence of the monitoring event is confirmed, and the monitoring an output unit that outputs (sends) an alarm including the location where the event occurred and an image at the time of occurrence to the server via the communication unit or other communication means;
has
The server is
an acquisition unit that acquires CAD information of a building, position index arrangement information, and layout information of monitoring target installations (chicken house cages, desks, chairs, equipment, etc.) placed in the building;
a flight route setting unit configured to set a flight route of the drone within the building for monitoring an installation to be monitored, or to receive an input of the flight route;
a monitoring event setting unit for setting a monitoring event to be monitored (abnormal event, intruder, fire, death, specific action, etc.);
a communication unit that transmits the flight route, the position indicator arrangement information, and the monitoring event to the drone, and receives from the drone an alarm including the position where the monitoring event occurred and an image at the time of occurrence;
has,
It is characterized by

Further, the monitoring device according to the third invention includes:
A monitoring device including a drone and a server,
The drone,
an imaging unit that captures images during flight;
a communication unit that receives the flight route and position indicator placement information;
a control unit that controls the flight of the own device along the flight route while recognizing the position of the own device based on the position indicator arrangement information;
a monitoring unit that monitors whether a monitoring event has occurred in the image;
When a monitoring event occurs, the temperature of the object of the monitoring event (for example, fire smoke, dead chicken) is measured (using an infrared camera function, or the imaging unit includes an infrared camera function), and the temperature is determined to be below a predetermined temperature. In this case, the occurrence of a monitoring event is determined, and an alarm including the location where the monitoring event occurred and an image at the time of occurrence is output (sent) to the server via the communication unit or other communication means. an output section;
has
The server is
an acquisition unit that acquires CAD information of a building, position index arrangement information, and layout information of monitoring target installations (chicken house cages, desks, chairs, equipment, etc.) placed in the building;
a flight route setting unit configured to set a flight route of the drone within the building for monitoring an installation to be monitored, or to receive an input of the flight route;
a monitoring event setting unit for setting a monitoring event to be monitored (abnormal event, intruder, fire, death, specific action, etc.);
a communication unit that transmits the flight route, the position indicator arrangement information, and the monitoring event to the drone, and receives from the drone an alarm including the position where the monitoring event occurred and an image at the time of occurrence;
has,
It is characterized by

Further, the monitoring device according to the fourth invention includes:
A monitoring device including a drone and a server,
The drone,
an imaging unit that captures images during flight;
a communication unit that receives the flight route, position indicator placement information, monitoring event conspicuousness information (feeding information (feeding location, time, etc.), intruder threat information (warning sound generation location, time, etc.)); ,
a control unit that controls the flight of the own device along the flight route while recognizing the position of the own device based on the position indicator arrangement information, and further based on the monitoring event salience information;
a monitoring unit that monitors whether a monitoring event has occurred in the image;
an output unit that, when a monitoring event occurs, outputs (sends) an alarm including the position where the monitoring event occurs and an image at the time of occurrence to the server via the communication unit or other communication means;
has
The server is
an acquisition unit that acquires CAD information of a building, position index arrangement information, and layout information of monitoring target installations (chicken house cages, desks, chairs, equipment, etc.) placed in the building;
a flight route setting unit configured to set a flight route of the drone within the building for monitoring an installation to be monitored, or to receive an input of the flight route;
a monitoring event setting unit for setting a monitoring event to be monitored (abnormal event, intruder, fire, death, specific action, etc.);
a communication unit that transmits the flight route, the position indicator arrangement information, and the monitoring event to the drone, and receives from the drone an alarm including the position where the monitoring event occurred and an image at the time of occurrence;
has,
It is characterized by

Further, the monitoring device according to the fifth invention (utilizing various sensors)
A monitoring device including a drone and a server,
The drone,
A gyro sensor that measures angular velocity,
An acceleration sensor that measures acceleration;
An altitude sensor that measures altitude,
an imaging unit that captures images during flight;
a communication unit that receives the flight route and position indicator placement information;
a control unit that controls the flight of the own device along the flight route while recognizing the position of the own device based on the position indicator arrangement information, the angular velocity, the acceleration, and the altitude;
a monitoring unit that monitors whether a monitoring event has occurred in the image;
an output unit that, when a monitoring event occurs, outputs (sends) an alarm including the position where the monitoring event occurs and an image at the time of occurrence to the server via the communication unit or other communication means;
has
The server is
an acquisition unit that acquires CAD information of a building, position index arrangement information, and layout information of monitoring target installations (chicken house cages, desks, chairs, equipment, etc.) placed in the building;
a flight route setting unit configured to set a flight route of the drone within the building for monitoring an installation to be monitored, or to receive an input of the flight route;
a monitoring event setting unit for setting a monitoring event to be monitored (abnormal event, intruder, fire, death, specific action, etc.);
a communication unit that transmits the flight route, the position indicator arrangement information, and the monitoring event to the drone, and receives from the drone an alarm including the position where the monitoring event occurred and an image at the time of occurrence;
has,
It is characterized by

Moreover, the monitoring device (chicken house monitoring) according to the sixth invention,
A monitoring device including a drone and a server for monitoring a poultry house,
The drone,
A gyro sensor that measures angular velocity,
An acceleration sensor that measures acceleration;
An altitude sensor that measures altitude,
an imaging unit that captures images during flight;
a communication unit that receives the flight route and position indicator placement information;
a control unit that controls the flight of the own device along the flight route while recognizing the position of the own device based on the position indicator arrangement information, the angular velocity, the acceleration, and the altitude;
a monitoring unit that uses image recognition technology to individually capture chickens in the image and monitors whether a monitoring event is occurring;
an output unit that, when a monitoring event occurs, outputs (sends) an alarm including the position where the monitoring event occurs and an image at the time of occurrence to the server via the communication unit or other communication means;
has
The server is
an acquisition unit that acquires CAD information of a building, position indicator arrangement information, and layout information of an installation to be monitored (a cage in a chicken house) placed in the building;
a flight route setting unit configured to set a flight route of the drone within the building for monitoring chickens in an installation to be monitored, or to receive an input of the flight route;
a monitoring event setting unit for setting a monitoring event to be monitored (chickens not moving, etc.);
a communication unit that transmits the flight route, the position indicator arrangement information, and the monitoring event to the drone, and receives from the drone an alarm including a position where the monitoring event occurs and an image at the time of occurrence;
has,
It is characterized by

Further, the monitoring device according to the seventh invention (monitoring type using a server, the drone is just an imaging device),
A monitoring device including a drone and a server,
The drone,
an imaging unit that captures images during flight;
a communication unit that receives the flight route and position indicator placement information;
a control unit that controls the flight of the own device along the flight route while recognizing the position of the own device based on the position indicator arrangement information;
has
The communication department,
transmitting the position of the own device and the image to the server;
The server is
an acquisition unit that acquires CAD information of a building, position index arrangement information, and layout information of monitoring target installations (chicken house cages, desks, chairs, equipment, etc.) placed in the building;
a flight route setting unit configured to set a flight route of the drone within the building for monitoring an installation to be monitored, or to receive an input of the flight route;
a monitoring event setting unit for setting a monitoring event to be monitored (abnormal event, intruder, fire, death, specific action, etc.);
a communication unit that transmits the flight route, the position indicator arrangement information, and the monitoring event to the drone, and receives the position of the own device and the image from the drone;
a monitoring unit that monitors whether a monitoring event has occurred in the image;
an output unit that outputs (sends) an alarm when a monitoring event occurs, including the position where the monitoring event occurred and an image at the time of occurrence;
has,
It is characterized by

As mentioned above, the solution of the present invention has been explained as a device, but the present invention can also be realized as a method, a program, or a storage medium on which a program is recorded, which is substantially equivalent to the above, and the scope of the present invention does not fall within the scope of the present invention. It should be understood that these are also included. In addition, each step of the method and program below uses arithmetic processing units such as a CPU and DSP as necessary for data processing, and input data, processed/generated data, etc. are transferred to magnetic tape. , HDD, memory, and other storage devices.

For example, a monitoring program according to an eighth invention that implements the present invention as a program is a monitoring program that causes one or more arithmetic processing units to function as the monitoring device according to any one of the first to seventh inventions.

Further, for example, a storage medium according to a ninth invention in which the present invention is implemented as a storage medium is a computer-readable storage medium storing the monitoring program according to the eighth invention.

Further, for example, a monitoring method according to a tenth invention in which the present invention is realized as a method is as follows:
A monitoring method using a drone and a server,
an imaging step in which the drone captures an image while in flight;
a communication step in which the drone receives a flight route and position indicator placement information;
a control step in which the drone controls the flight of the own device along the flight route while recognizing the position of the own device based on the position indicator arrangement information;
a monitoring step in which the drone monitors whether a monitoring event has occurred in the image;
When a monitoring event occurs, the drone outputs (sends) an alarm including the location where the monitoring event occurred and an image at the time of occurrence to the server via the communication unit or other communication means. step and
has
Acquisition by the server of acquiring CAD information of the building, position indicator arrangement information, and layout information of monitoring target installations (chicken house cages, desks, chairs, equipment, etc.) placed in the building. step and
a flight route setting step in which the server sets a flight route of the drone within the building for monitoring an installation to be monitored, or receives an input of the flight route;
a monitoring event setting step in which the server sets a monitoring event to be monitored (abnormal event, intruder, fire, death, specific action, etc.);
a communication step in which the server transmits the flight route, the position indicator arrangement information, and the monitoring event to the drone, and receives from the drone an alert including the position where the monitoring event occurred and an image at the time of occurrence; and,
has,
It is characterized by

Furthermore, the monitoring method (midair suspension) according to the eleventh invention is as follows:
A monitoring method using a drone and a server,
an imaging step in which the drone captures an image during flight;
a communication step in which the drone receives a flight route and position indicator placement information;
a control step of controlling the flight of the own device along the flight route while recognizing the position of the own device based on the position indicator arrangement information;
a monitoring step in which the drone monitors whether a monitoring event has occurred in the image;
When the monitoring event occurs, the drone controls the control unit to reduce the flight speed or stop the drone in the air, and determines the occurrence of the monitoring event when the monitoring event continues for a predetermined period of time or more. an output step of outputting (sending) an alarm including the location where the monitoring event occurred and the image at the time of occurrence to the server via the communication unit or other communication means;
Acquisition in which the server acquires CAD information of the building, position index arrangement information, and layout information of monitoring target installations (chicken house cages, desks, chairs, equipment, etc.) placed in the building. step and
a flight route setting step in which the server sets a flight route of the drone within the building for monitoring an installation to be monitored, or receives an input of the flight route;
a monitoring event setting step in which the server sets a monitoring event to be monitored (abnormal event, intruder, fire, death, specific action, etc.);
a communication step in which the server transmits the flight route, the position index arrangement information, and the monitoring event to the drone, and receives from the drone an alert including the position where the monitoring event occurred and an image at the time of occurrence; and,
has,
It is characterized by

Further, the monitoring method (temperature measurement) according to the twelfth invention is as follows:
A monitoring method using a drone and a server,
an imaging step in which the drone captures an image during flight;
a communication step in which the drone receives a flight route and position indicator placement information;
a control step in which the drone controls the flight of the own device along the flight route while recognizing the position of the own device based on the position indicator arrangement information;
a monitoring step in which the drone monitors whether a monitoring event has occurred in the image;
When a monitoring event occurs, the drone measures the temperature of the object of the monitoring event (for example, fire smoke, dead chicken) (or has an infrared camera function, or the imaging unit includes an infrared camera function); When the temperature is below (or above) a predetermined temperature, the occurrence of a monitoring event is determined, and an alarm including the location where the monitoring event occurred and an image at the time of occurrence is sent via the communication unit or other communication means, an output step of outputting (sending) to the server;
Acquisition in which the server acquires CAD information of the building, position index arrangement information, and layout information of monitoring target installations (chicken house cages, desks, chairs, equipment, etc.) placed in the building. step and
a flight route setting step in which the server sets a flight route of the drone within the building for monitoring an installation to be monitored, or receives an input of the flight route;
a monitoring event setting step in which the server sets a monitoring event to be monitored (abnormal event, intruder, fire, death, specific action, etc.);
a communication step in which the server transmits the flight route, the position index arrangement information, and the monitoring event to the drone, and receives from the drone an alert including the position where the monitoring event occurred and an image at the time of occurrence; and,
has,
It is characterized by

Furthermore, the monitoring method (utilization of salience information) according to the thirteenth invention is as follows:
A monitoring method using a drone and a server,
an imaging step in which the drone captures an image during flight;
The drone receives a flight route, position indicator arrangement information, and monitoring event salience information (feeding information (feeding location, time, etc.), intruder threat information (warning sound generation location, time, etc.)). a communication step to
a control step in which the drone recognizes the position of the own device based on the position index arrangement information and further controls the flight of the own device along the flight route based on the monitoring event saliency information; ,
a monitoring step in which the drone monitors whether a monitoring event has occurred in the image;
When a monitoring event occurs, the drone outputs (sends) an alarm including the location where the monitoring event occurred and an image at the time of occurrence to the server via the communication unit or other communication means. step and
has
Acquisition in which the server acquires CAD information of the building, position index arrangement information, and layout information of monitoring target installations (chicken house cages, desks, chairs, equipment, etc.) placed in the building. step and
a flight route setting step in which the server sets a flight route of the drone within the building for monitoring an installation to be monitored, or receives an input of the flight route;
a monitoring event setting step in which the server sets a monitoring event to be monitored (abnormal event, intruder, fire, death, specific action, etc.);
a communication step in which the server transmits the flight route, the position index arrangement information, and the monitoring event to the drone, and receives from the drone an alert including the position where the monitoring event occurred and an image at the time of occurrence; and,
has,
It is characterized by

Furthermore, the monitoring method (utilizing various sensors) according to the fourteenth invention is as follows:
A monitoring method using a drone and a server,
The drone,
A gyro sensor that measures angular velocity,
An acceleration sensor that measures acceleration;
An altitude sensor that measures altitude,
has
an imaging step in which the drone captures an image during flight;
a communication step in which the drone receives a flight route and position indicator placement information;
a control step in which the drone controls the flight of its own device along the flight route while recognizing its own position based on the position indicator arrangement information, the angular velocity, the acceleration, and the altitude;
a monitoring step in which the drone monitors whether a monitoring event has occurred in the image;
When a monitoring event occurs, the drone outputs (sends) an alarm including the location where the monitoring event occurred and an image at the time of occurrence to the server via the communication unit or other communication means. step and
has
Acquisition in which the server acquires CAD information of the building, position index arrangement information, and layout information of monitoring target installations (chicken house cages, desks, chairs, equipment, etc.) placed in the building. step and
a flight route setting step in which the server sets a flight route of the drone within the building for monitoring an installation to be monitored, or receives an input of the flight route;
a monitoring event setting step in which the server sets a monitoring event to be monitored (abnormal event, intruder, fire, death, specific action, etc.);
a communication step in which the server transmits the flight route, the position index arrangement information, and the monitoring event to the drone, and receives from the drone an alert including the position where the monitoring event occurred and an image at the time of occurrence; and,
has,
It is characterized by

Furthermore, the monitoring method (chicken house monitoring) according to the fifteenth invention is as follows:
A monitoring method using a drone and a server,
The drone,
A gyro sensor that measures angular velocity,
An acceleration sensor that measures acceleration;
An altitude sensor that measures altitude,
an imaging step in which the drone captures an image during flight;
a communication step in which the drone receives a flight route and position indicator placement information;
a control step in which the drone controls the flight of its own device along the flight route while recognizing its own position based on the position indicator arrangement information, the angular velocity, the acceleration, and the altitude;
a monitoring step in which the drone uses image recognition technology to individually capture chickens in the image and monitors whether a monitoring event is occurring;
When a monitoring event occurs, the drone outputs (sends) an alarm including the location where the monitoring event occurred and an image at the time of occurrence to the server via the communication unit or other communication means. step and
has
an acquisition step in which the server acquires CAD information of the building, position indicator arrangement information, and layout information of an installation to be monitored (a chicken house cage) placed in the building;
a flight route setting step in which the server sets a flight route for the drone in the building for monitoring chickens in an installation to be monitored, or receives an input of the flight route;
a monitoring event setting step in which the server sets a monitoring event to be monitored (such as the chicken not moving);
The server transmits the flight route, the position indicator arrangement information, and the monitoring event to the drone, and receives an alert from the drone that includes the location where the monitoring event occurred and an image at the time of occurrence. a communication step;
has,
It is characterized by

Further, the monitoring method according to the 16th invention (monitoring type using a server, drone is just an imaging device),
A monitoring method using a drone and a server,
an imaging step in which the drone captures an image during flight;
a communication step in which the drone receives a flight route and position indicator placement information;
a control step unit in which the drone controls the flight of the own device along the flight route while recognizing the position of the own device based on the position indicator arrangement information;
has
a step in which the drone transmits the position of its own device and the image to the server;
Acquisition in which the server acquires CAD information of the building, position index arrangement information, and layout information of monitoring target installations (chicken house cages, desks, chairs, equipment, etc.) placed in the building. step and
a flight route setting step in which the server sets a flight route of the drone within the building for monitoring an installation to be monitored, or receives an input of the flight route;
a monitoring event setting step in which the server sets a monitoring event to be monitored (abnormal event, intruder, fire, death, specific action, etc.);
a communication unit in which the server transmits the flight route, the position indicator arrangement information, and the monitoring event to the drone, and receives the position of the own device and the image from the drone;
a monitoring step in which the server monitors whether a monitoring event has occurred in the image; and when the monitoring event occurs, the server includes a position where the monitoring event occurs and an image at the time of occurrence. an output step for outputting (sending) an alarm;
has,
It is characterized by

As mentioned above, up to this point, the description has been made using a drone as a mobile monitoring device. The following description is not limited to drones, but is implemented as a more general mobile monitoring device.

Furthermore, the monitoring device according to the seventeenth invention includes:
A monitoring device including a mobile monitoring device and a server,
The mobile monitoring device includes:
an imaging unit that captures an image;
A drive unit that moves the own device;
an acquisition unit that acquires the position of the mobile monitoring device;
a communication unit that transmits monitoring information including a position of the mobile monitoring device and an image at the position to the server via a wireless or wired communication means;
has
The server is
a communication unit that receives the monitoring information, including the position of the mobile monitoring device and an image at the position, from the mobile monitoring device;
a monitoring event setting unit for setting a monitoring event to be monitored (abnormal event, intruder, fire, death, illness, specific movement, etc.);
a monitoring unit that monitors whether a set monitoring event is occurring in the image;
an output unit that outputs an alarm (preferably including the image and the position) when the monitoring event occurs;
has,
It is characterized by

Moreover, the monitoring device according to the 18th invention includes:
The mobile monitoring device includes:
further comprising a guide part movably supported, mounted, or suspended on the guide rail and guided along the guide rail,
The drive section is
moving the device along the guide rail while being supported, placed, or suspended by the guide rail;
It is characterized by

Further, the monitoring device according to the nineteenth invention includes:
The drive unit of the mobile monitoring device,
It is self-propelled,
It is characterized by

Further, the monitoring device according to the twentieth invention includes:
The server is
a storage unit that stores a large number of images and monitoring events associated with at least some of the images;
When the monitoring unit monitors whether the set monitoring event is occurring, using AI technique,
referring to the storage unit to monitor whether the set monitoring event is occurring;
It is characterized by

Moreover, the monitoring device according to the twenty-first invention is:
The communication unit of the mobile monitoring device,
Further receives monitoring event salience information (feeding information (feeding location, time, etc.), intruder threat information (alarm sound generation location, time, etc.)),
The drive unit of the mobile monitoring device,
controlling the movement of the own device based on the monitoring event salience information;
It is characterized by

Furthermore, the monitoring device according to the twenty-second invention includes:
The communication section of the server,
Further receives monitoring event salience information (feeding information (feeding location, time, etc.), intruder threat information (alarm sound generation location, time, etc.)),
The server is
controlling the drive unit of the mobile monitoring device to move the mobile monitoring device via the communication unit based on the monitoring event salience information;
It is characterized by

Furthermore, the monitoring device according to the twenty-third invention includes:
The monitoring event conspicuousness information is
Feeding information (feeding location, time, etc.), intruder threat information (warning light, warning vibration, warning sound location, time, etc.), odor information (odors disliked or aroused by monitored chickens, etc.), One or more items selected from the group consisting of light information (lighting, indoor illuminance, flashing, rotating lights, etc.), opening/closing information (opening/closing windows and ventilation fans, etc.), and air conditioning setting information.
It is characterized by

Further, the monitoring device according to the twenty-fourth invention includes:
The mobile monitoring device includes:
It further includes a monitoring event conspicuousness situation generating means for generating the condition of the monitoring event conspicuousness information (for example, a rotating light that emits light, a speaker that emits sound, a rod that comes into contact with the poultry house as it moves and makes vibrations and sounds). (a signal output unit that starts feeding, a control signal output unit that controls lighting and audio output, etc.),
It is characterized by

Furthermore, the monitoring device according to the twenty-fifth invention includes:
The mobile monitoring device includes:
further comprising an infrared camera that captures an infrared image;
The communication unit of the mobile monitoring device,
further transmitting the infrared image to the server;
The communication department of the server,
further receiving the infrared image;
The monitoring unit of the server,
When the monitoring event occurs, the temperature of the object of the monitoring event (for example, fire smoke, dead chicken) is measured based on the infrared image, and compared with a predetermined temperature (below the temperature, above the temperature, etc.). ), confirming the occurrence of the monitoring event;
It is characterized by

Furthermore, the monitoring program according to the twenty-sixth invention is
A monitoring program that causes one or more arithmetic processing devices to function as the monitoring device according to any one of the seventeenth to twenty-fifth inventions.

Furthermore, the storage medium according to the twenty-seventh invention is
A computer readable storage medium storing the monitoring program according to the twenty-sixth invention.

Furthermore, the monitoring method according to the twenty-eighth invention is as follows:
A monitoring method using a mobile monitoring device and a server,
an imaging step in which the mobile monitoring device captures an image;
a driving step for moving the own device;
an acquisition step in which the mobile monitoring device obtains a position of the mobile monitoring device;
a communication step in which the mobile monitoring device transmits monitoring information including a position of the mobile monitoring device and an image at the position to the server via a wireless or wired communication means;
a communicating step in which the server receives the monitoring information from the mobile monitoring device, including the location of the mobile monitoring device and an image at the location;
a monitoring event setting step in which the server sets a monitoring event to be monitored (abnormal event, intruder, fire, death, illness, specific operation, etc.);
a monitoring step in which the server monitors whether a set monitoring event is occurring in the image;
an output step in which the server outputs an alarm (preferably including the image and the location) when the monitoring event occurs;
has,
It is characterized by

Furthermore, the monitoring method according to the twenty-ninth invention is as follows:
a guide step in which the movement monitoring device is movably supported, mounted, or suspended on a guide rail and guided along the guide rail;
a driving step in which the movement monitoring device moves itself along the guide rail while being supported, placed, or suspended by the guide rail;
further having,
It is characterized by

Furthermore, the monitoring method according to the 30th invention includes:
the mobile monitoring device is self-propelled;
It is characterized by

Further, the monitoring method according to the 31st invention is as follows:
a storage step in which the server stores a number of images and monitoring events associated with at least some of the images;
In monitoring whether or not the set monitoring event is occurring, using an AI technique, referring to the storage unit and monitoring whether or not the set monitoring event is occurring. and,
It is characterized by further having.

Further, the monitoring method according to the 32nd invention is as follows:
a communication step in which the mobile monitoring device further receives monitoring event conspicuousness information (feeding information (feeding location, time, etc.), intruder threat information (warning sound generation location, time, etc.));
a control step in which the mobile monitoring device controls movement of its own device based on the monitoring event salience information;
It is characterized by further having.

Furthermore, the monitoring method according to the thirty-third invention is as follows:
a receiving step in which the server further receives monitoring event conspicuousness information (feeding information (feeding location, time, etc.), intruder threat information (warning sound generation location, time, etc.));
a control step in which the server controls the drive unit of the mobile monitoring device to move the mobile monitoring device via the communication unit based on the monitoring event salience information;
It is characterized by further having.

Furthermore, the monitoring method according to the thirty-fourth invention is as follows:
The monitoring event conspicuousness information is
Feeding information (feeding location, time, etc.), intruder threat information (warning light, warning vibration, warning sound location, time, etc.), odor information (odors disliked or aroused by monitored chickens, etc.), light one or more items selected from the group consisting of information (lighting, indoor illuminance, flashing, revolving lights, etc.), opening/closing information (opening/closing windows and ventilation fans, etc.), and air conditioning setting information.
It is characterized by

Further, the monitoring method according to the thirty-fifth invention is as follows:
The mobile monitoring device further includes a monitoring event conspicuous situation generation step of generating a situation of the monitoring event conspicuous information (for example, a revolving light that emits light, a speaker that emits sound, a rotating light that emits light, a speaker that emits sound, or a device that comes into contact with a poultry house while moving) rod-shaped part that produces vibrations and sounds, a signal output part that starts feeding, a control signal output part that controls lighting and sound output, etc.)
It is characterized by

Furthermore, the monitoring method according to the thirty-sixth invention includes:
The mobile monitoring device further includes an infrared imaging step of capturing an infrared image using an infrared camera,
a transmitting step in which the mobile monitoring device further transmits the infrared image to the server;
a receiving step in which the server further receives the infrared image;
When the monitoring event occurs, the server measures the temperature of the object of the monitoring event (for example, fire smoke, dead chicken) based on the infrared image, and compares it with a predetermined temperature (below the temperature). , temperature or higher, etc.), a determining step for determining the occurrence of the monitoring event;
It is characterized by further having.

In addition to the techniques described above, new inventions of monitoring techniques using distance sensors are listed below.

The monitoring device according to the 36th invention includes:
A monitoring device including a mobile monitoring device and a server,
The mobile monitoring device includes:
an imaging unit that captures an image;
A sensor that acquires distance information,
an acquisition unit that acquires identification information (such as a guide tape) that determines the movement direction of the mobile monitoring device;
a drive unit that moves the device based on the identification information;
a communication unit that transmits monitoring information including the distance information and the image to the server via wireless or wired communication means;
has
The server is
a storage unit that stores model change pattern information indicating a monitoring event;
a communication unit that receives the monitoring information including the distance information and the image from the mobile monitoring device;
a monitoring unit that compares the model change pattern information and the change pattern of the distance information to monitor whether a monitoring event (death, illness, sitting state, etc.) has occurred;
an output unit that outputs an alarm (preferably including an image of the position and the position) when the monitoring event occurs;
has,
It is characterized by

Further, the monitoring device according to the thirty-seventh invention includes:
A monitoring device including a mobile monitoring device and a server,
The mobile monitoring device includes:
an imaging unit that captures an image;
A sensor that acquires distance information,
an acquisition unit that acquires identification information (such as a guide tape) that determines the movement direction of the mobile monitoring device;
a drive unit that moves the device based on the identification information;
a communication unit that transmits monitoring information including the distance information and the image to the server via wireless or wired communication means;
has
The server is
a storage unit that stores a large number of distance change patterns;
an AI engine trained based on the plurality of distance change patterns and monitoring events associated with at least some of the change patterns;
a communication unit that receives the monitoring information including the distance information and the image from the mobile monitoring device;
The AI engine is
Based on the change pattern of the distance information, monitor whether a monitoring event (death, illness, sitting state, etc.) has occurred;
The server is
further comprising an output unit that outputs an alarm (preferably including an image of the position and the position) when the monitoring event occurs;
It is characterized by

Further, the monitoring device according to the thirty-eighth invention includes:
A monitoring device including a mobile monitoring device and a server,
The mobile monitoring device includes:
an imaging unit that captures an image;
a first sensor that acquires first distance information;
a second sensor that acquires second distance information;
an acquisition unit that acquires identification information (such as a guide tape) that determines the movement direction of the mobile monitoring device;
a drive unit that moves the device based on the identification information;
a communication unit that transmits monitoring information including the first distance information, the second distance information, and the image to the server via a wireless or wired communication means;
has
The server is
a storage unit that stores model change pattern information indicating a monitoring event;
a communication unit that receives the monitoring information including the first distance information, second distance information, and the image from the mobile monitoring device;
a pattern generation unit that compares the first distance information and the change pattern of the second distance information and generates an excluded change pattern that excludes a component caused by movement of a target object (such as a chicken) from each change pattern;
a monitoring unit that compares the model change pattern information and the excluded change pattern to monitor whether a monitoring event (death, illness, sedentary state, etc.) has occurred;
an output unit that outputs an alarm (preferably including an image of the position and the position) when the monitoring event occurs;
has,
It is characterized by

Further, the monitoring program according to the 39th invention is
A monitoring program that causes one or more arithmetic processing devices to function as a monitoring device according to any one of the thirty-seventh to thirty-eighth inventions.

Further, the storage medium according to the fortieth invention is:
A computer readable storage medium storing the monitoring program according to the thirty-ninth invention.

According to the present invention, it is possible to carry out monitoring work using a drone that sets a flight route based on CAD information. Further, according to the present invention, it is possible to perform monitoring work using a mobile monitoring device.

FIG. 1 is a block diagram showing an overview of a monitoring device according to an embodiment of the present invention. FIG. 2 is a flowchart showing an example of a process executed by the monitoring device shown in FIG. FIG. 3 is a schematic diagram illustrating the drone monitoring technique of this device. FIG. 4 is a schematic diagram illustrating the drone monitoring technique of this device. FIG. 5 is a schematic diagram illustrating the function of the acquisition unit in this device. FIG. 6 is a schematic diagram illustrating the flight route setting function of this device. FIG. 7 is a diagram showing a monitoring event (abnormal event) setting interface. FIG. 8 is a schematic diagram illustrating the height direction setting function among the flight route setting functions of this device. FIG. 9 is a block diagram showing an overview of a monitoring device according to an embodiment of the present invention. FIG. 10 is a flowchart showing an example of a process executed by the monitoring device shown in FIG. FIG. 11 is a schematic diagram illustrating a technique of monitoring by the monitoring unit. FIG. 12 is a schematic diagram illustrating map-shaped alarm information. FIG. 13 is a schematic diagram illustrating map-shaped alarm information. FIG. 14 is a block diagram showing an overview of a monitoring device according to an embodiment of the present invention. FIG. 15 is a flowchart showing an example of a process executed by the monitoring device shown in FIG. 14. FIG. 16 is a schematic diagram illustrating the monitoring technique of the mobile monitoring device of the present device. FIG. 17 is a schematic diagram illustrating another monitoring technique of the mobile monitoring device of the present device. FIG. 18 is a schematic diagram illustrating a monitoring technique of the mobile monitoring device of the present device when applied to a poultry house. FIG. 19 is a schematic diagram illustrating a monitoring technique of a mobile monitoring device of the present device provided with a monitoring event conspicuous situation generating means. FIG. 20 is a schematic diagram illustrating a monitoring technique of the mobile monitoring device of the present device when applied to a poultry house. FIG. 21 is a schematic diagram illustrating a monitoring technique of the mobile monitoring device of the present device when applied to a poultry house. FIG. 22 is a block diagram showing an overview of a monitoring device according to an embodiment of the present invention. FIG. 23 is a flowchart showing an example of processing executed by the monitoring device shown in FIG. 22. FIG. 24 is a schematic diagram illustrating the monitoring technique of the mobile monitoring device of the present device when applied to a poultry house. FIG. 25 is a diagram illustrating an example of sensor distance measurement using the present device. FIG. 26 is a diagram illustrating an example of sensor distance measurement using the present device. FIG. 27 is a diagram illustrating an example of waveform processing (an example of numerical processing) for distance measurement of the sensor by this device. FIG. 28 is a diagram illustrating an example of waveform processing (an example of numerical processing) for distance measurement of the sensor by this device. FIG. 29 is a diagram illustrating an example of waveform processing (an example of numerical processing) for distance measurement of the sensor by this device. FIG. 30 is a diagram showing an example of waveform processing (an example of numerical processing) for distance measurement of the sensor by this device.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Embodiment 1>
FIG. 1 is a block diagram showing an overview of a monitoring device according to an embodiment of the present invention. As shown in the figure, the monitoring device MOS1 includes drones DR1 and DR2 and a server SV1. The server SV1 includes a control section (CPU, arithmetic processing unit, processor) CON, an input section IN, an output section OUT, a communication section COM, a storage section MEM, and a display section DIS. The drone DR2 has the same configuration and functions as the drone DR1 unless otherwise specified, but some configurations are omitted for convenience of drawing and explanation.

The drone DR1 includes cameras CM1 to 3 as imaging units, sensors SNSR (barometer, gyro sensor, etc.), control unit CON (CPU, arithmetic processing unit, processor), communication unit COM, monitoring unit MON, output unit OUT, and storage unit. Although it has an STR, it may further include an input section (not shown) and a display section (not shown). It is preferable to have two cameras, one for photographing the monitoring target and one for photographing a position indicator installed on the ceiling, etc., but one or two cameras may serve the same function. . Furthermore, it is preferable to use a high-performance camera that can take pictures even at night with little ambient light. Furthermore, the camera may be provided with a separate device that detects infrared rays, or the above camera may be equipped with that function to detect the temperature of the monitored object and detect whether or not it is dead.

The drone DR1 uses, as a sensor SNSR, a position measuring unit (not shown, such as GPS) that measures the position of its own device, a gyro sensor that measures angular velocity, an acceleration sensor that measures acceleration, and an altitude sensor that measures altitude. and has.

Drone DR1 is
an imaging unit (cameras CM1 to CM3) that captures images during flight;
a communication unit COM that receives the flight route and position indicator placement information;
a control unit CON that controls the flight of the own device along the flight route while recognizing the position of the own device based on the position indicator arrangement information;
a monitoring unit MON that monitors whether a monitoring event has occurred in the image;
an output unit OUT that outputs (sends) an alarm including a position where the monitoring event occurs and an image at the time of occurrence to the server via the communication unit or other communication means when a monitoring event occurs;
has.

Server SV1 is
an acquisition unit ACQ that acquires CAD information of a building, position index arrangement information, and layout information of monitoring target installations (chicken house cages, desks, chairs, equipment, etc.) placed in the building;
a flight route setting unit FLS that sets a flight route of the drone in the building for monitoring an installation to be monitored, or receives an input of the flight route;
a monitoring event setting unit MES for setting a monitoring event to be monitored (abnormal event, intruder, fire, death, specific action, etc.);
a communication unit COM that transmits the flight route, the position indicator arrangement information, and the monitoring event to the drone, and receives from the drone an alarm including the position where the monitoring event occurred and an image at the time of occurrence;
has.
Note that suitable output warnings include sending an e-mail, outputting or transmitting a warning voice, outputting an image, warning a supervisor, warning a manager, and reporting to a security company, police, etc.

The CAD information of the building to be monitored may be acquired from a building information server or a storage unit MEM connected via the network NET. CAD information, position indicator arrangement information that corresponds to or corresponds to the spatial coordinate system of this CAD information, and installations to be monitored placed in the building (chicken house cages, desks, chairs, equipment, etc.) ), the flight route is set from the layout information. The display unit DIS can display information stored in the device and information generated.

It is preferable that each functional unit included in the control unit of the server SV1 is realized by a program module read into the memory space of the control unit. Further, the drone can realize each functional section by a program module read into the memory space of the control section, or can realize each functional section by a built-in GPS unit, camera unit, etc. Normally, the server downloads software from a storage unit or a website that causes the processor to function as each part of the device, installs it on the PC, and starts it, so that the PC operates as the device. Note that each functional section provided in the control section is nothing more than a collection of steps that have a certain set of functional functions, and it is possible to combine multiple functional sections into one functional section, or to combine some of them into other functional sections. It can be incorporated into a section or divided into other independent functional sections.

In this way, generated/extracted information, intermediate data, and acquired data can be sent to the outside, displayed on the display, and generated/extracted information, intermediate data, and acquired data can be stored in the storage. It should be noted that this is possible in other embodiments as well, as described below. This device may be a computer such as a general-purpose computer, a specific-purpose computer, a server, a PC, or a mobile terminal such as a smartphone, or a program module that implements (executes) the functions and processing procedures (methods) of this device on a computer. It is preferable to build this device on a computer by holding it in the computer's CPU or storage unit, or reading it from an external server or storage, and the same applies to each of the subsequent embodiments. Additionally, each functional unit may be distributed to separate computers or devices connected via a network. Alternatively, a configuration may be adopted in which a plurality of functional units are combined into one, or a part of the processing steps is performed by another functional unit. Further, in the present embodiment, the server and the drone are defined as separate entities, but some or all of the functions of the server may be included in the drone.

FIG. 2 is a flowchart showing an example of a process executed by the monitoring device shown in FIG. In the monitoring device, two devices, a server SV1 and a drone DR1, cooperate to perform processing. Once server SV1 stores information on other servers, cooperation with other servers may be unnecessary.

In step S11, CAD information of the building is acquired. Next, in step S12, position index arrangement information corresponding to the spatial coordinate system of the CAD information and information on the installation objects to be monitored (chicken house cages, desks, chairs, equipment, etc.) placed in the building are provided. Get layout information. Subsequently, in step S13, a flight route of a drone within a building for monitoring an installation to be monitored is set, or an input of the flight route is accepted. Then, in step S14, a monitoring event to be monitored (abnormal event, intruder, fire, death, specific action, etc.) is set (input may be accepted).

Then, in step S15, the drone is moved along the flight route while confirming the position using a position index and/or using a gyro sensor and an acceleration sensor (or GPS). This control may be performed on the server side, or may be autonomously controlled on the drone side. In step S16, a camera is used to photograph the monitoring target. In step S17, it is determined whether it is the end of the flight route or whether the remaining power of the battery mounted on the drone is insufficient. If the judgment conditions are met, the drone returns to the base. If the determination condition is not satisfied, the process advances to step S18.

In step S18, it is determined whether an abnormal event (monitoring event) has occurred in the image. If no abnormal event has occurred, the process returns to step S15 and the flight continues. If an abnormal event has occurred, a warning may be issued as is, but in order to reduce false recognition, the process proceeds to step S18-1 and the drone is stopped in the air (hovering) or decelerated, and the abnormality is detected. It is preferable to continue observing images of the event. If it is determined in step S18-2 that the abnormal event continues to occur, an alarm including the position where the abnormal event occurred and the image (including video) at that time is output in step S19. do. If it is determined in step S18-2 that the occurrence of the abnormal event is not continuing, it is determined that there is an erroneous recognition, and the process returns to step S15 without issuing an alarm and continues flight (monitoring). After issuing the warning in step S19, the process proceeds to step S20, and similarly to step S17, it is determined whether the flight route is at the end or whether the remaining power of the battery mounted on the drone is insufficient. do. If the judgment conditions are met, the drone returns to the base. If the determination conditions are not met, the process returns to step S15 and flight (monitoring) continues.

Incidentally, imaging may be performed continuously, but in order to reduce the amount of communication and processing, it is preferable to thin out frames or take still images at intervals of several seconds.

The storing step is performed once and does not need to be performed again until the information is changed. Alternatively, the storing step can be made unnecessary by installing a ROM or flash memory storing the information in the device or computer.

FIG. 3 is a schematic diagram illustrating the drone monitoring technique of this device. As shown in the figure, position indicators PI1 to PI4 installed on the ceiling of the building BLD are photographed by camera CM3, and the position of the own device is recognized based on the indicators. The position index is arranged at the coordinates of the position index arrangement information, and the drone DR1 sequentially recognizes the position indexes PI1 to PI4 along the flight route FR1 while recognizing the position of its own device using the position index. while controlling the flight of its own device. It is preferable that the positional index is decorated with one or more of fluorescent materials, letters, figures, patterns, colors, etc., and it is preferable that the positional index is recognized as an image by photographing it with the camera CM3 pointing toward the ceiling. It is. Each position index is unique, and by recognizing them individually and referring to the position index placement information LILI, it is possible to determine where the drone should be positioned. Alternatively, the position indicator may be a device that emits a specific wireless or optical signal, a WIFI base station, or a wireless tag (in this case, the signal is read by a wireless communication unit, optical receiver, etc. instead of a camera). It is also possible to provide an indoor GPS signal unit in place of the position indicator and use a device to read it with a drone.

Then, while flying, the drone uses cameras CM1 and CM2 to photograph the shelves SBJ1 and SBJ2 (installed in the building BLD) to be monitored on the left and right, and monitors whether a monitoring event is occurring there. do.

FIG. 4 is a schematic diagram illustrating the drone monitoring technique of this device. As shown in the figure, FIG. 4 is a schematic diagram of the building BLD of FIG. 3 viewed from above (ceiling). The drone DR1 controls the flight of its own device while recognizing its own position using a position index (not shown in FIG. 4) and sequentially recognizing position indexes PI1 to PI4 along the flight route FR1. do. Then, while flying, the drone DR1 photographs the left and right monitoring target shelves SBJ1 and SBJ2 (installed in the building BLD) with the cameras CM1 and CM2, and a monitoring event (abnormal event) occurs there. monitor whether there are any. In this example, the camera CM1 recognizes the abnormal event OABE on the shelf SHLF1 of the monitoring target SBJ1, and issues an alarm (report of abnormality occurrence).

FIG. 5 is a schematic diagram illustrating the function of the acquisition unit in this device. As shown in the figure, this device acquires CAD information CI of a building. This CAD information is three-dimensional CAD, and members, walls, ceilings, floors, etc. in the building have three-dimensional spatial coordinates. In addition, this device acquires layout information SLI of the monitoring target installations (chicken house cages, desks, chairs, equipment, etc.) placed in the building, which corresponds to the spatial coordinate system of the building's CAD information CI. . Incorporate (merge) the layout information SLI into the CAD information CI. Then, this device acquires the position index arrangement information PI1 to PI4 corresponding to the spatial coordinate system of the CAD information CI of the building, and also adds the position index arrangement information PI1 to the CAD information CI, as shown in the lower part of the figure. Incorporate ~4 (merge). As a result, CAD information CIALL is obtained that describes all of the building's internal structure, including the position index and the installed objects. This device sets a flight route in this CAD information CIALL, which will be explained with the following figure.

FIG. 6 is a schematic diagram illustrating the flight route setting function of this device. As shown in the figure, in the CAD information CIALL, which also describes the location of the drone base DB, there are monitoring targets SBJ1 and SBJ2, which are installations placed corresponding to the layout information SLI, and there are passages (suitable for flight) between them. space). This device automatically sets the flight route FR1 (manually ). Similarly, it is assumed that there are SBJ3 and SBJ4 to be monitored, and that there is a passage (a space suitable for flight) between them. This device has the function of recognizing a space large enough for a person to pass through as a passage. The flight route is based on route setting rules (memorized) such as setting a flight route in a space where the target can be monitored, a predetermined size, and a predetermined height (usually 160 cm, which is about eye level for an average height person). The flight route FR2 is also automatically set based on the information (stored in the section) (it can also be manually accepted and set). As shown in the figure, when the flight routes FR1 and FR2 are set, the order of the flight routes is set so that FR1 comes first and FR2 comes after. Alternatively, one flight route may be set by connecting all flight routes.

In this example, the monitoring target is assumed to be a one-tier chicken house, but in the case of two or three tiers, the flight route is the same coordinates in terms of plane, but a different flight route with sequentially changing altitudes is assumed. It is possible to set and monitor the second stage, third stage, etc. (details will be explained in FIG. 8).

FIG. 7 is a diagram showing an abnormal event (monitoring event) setting interface. The monitoring event may be a positive sign event (for example, entry of a prospective customer), but in this example, an abnormal event is set in a chicken house. As shown in the figure, the regular event (monitoring event) setting interface ABE-IN has the following options in advance:
Abnormal event ABE1: Immobility (illness, weakness, death)
Abnormal event ABE2: Abnormal behavior abnormal event ABE3: Attack abnormal event ABE5: Abnormal event of not eating food ABE6: Intruder (small animal)
Abnormal event ABE6: Intruder (human)
Abnormal event ABE7: Fire (smoke, temperature detection, etc.)
etc. are displayed as selectable windows (sections), which are displayed on the display section DIS.

From this selection, the user selects the window for the abnormal event (multiple selections are possible) to be monitored, and sets the abnormal event. This is written in the selected abnormal event ABE-SET window as shown in the figure. In this example,
Abnormal event ABE1: Immobility (illness, weakness, death)
Abnormal event ABE7: Fire (smoke, temperature detection, etc.)
is set.

FIG. 8 is a schematic diagram illustrating the height direction setting function among the flight route setting functions of this device. As shown in the figure, when monitoring a chicken house cage with a three-tier structure such as the first shelf SHLF-1F, the second shelf SHLF-2F, and the third shelf SHLF-3F, each shelf The drone's flight route is automatically set at an altitude where it can monitor the area. Flight route FR1-V goes from left to right in the diagram at an altitude of 1m that allows monitoring of the first shelf SHLF-1F, rises to an altitude of 2m from the right end, and flies from the right to the left end of the second shelf SHLF-2F. monitored by camera. Finally, in order to monitor the third shelf SHLF-3F, it ascends to an altitude of 3 m and moves from left to right in the figure and ends. This altitude setting is automatically set based on the dimension information (coordinate information) of the installed object and building passage in the CAD information (for example, in the case of an installed object such as a shelf with a height of 1.5 m, A rule has been established that sets the height to 1 m, which is 2/3 of that height, and this rule is applied.) The altitude is calculated from the atmospheric pressure measured by the atmospheric pressure sensor, and based on the altitude, the drone can maintain the set altitude and fly along the flight route.

FIG. 9 is a block diagram showing an overview of a monitoring device according to an embodiment of the present invention. This embodiment has the same functions and configurations as the monitoring device shown in FIG. 1 unless otherwise noted or mentioned. The major differences between FIG. 1 and FIG. 9 are that in this embodiment, monitoring event salience information is used, and that the drone DR5 has the function of autonomously flying and also has the function of issuing an alarm. be. This will be explained in detail below.

As shown in the figure, the monitoring device MOS2 includes drones DR5 and DR6 and a server SV2. The server SV2 includes a control section (CPU, arithmetic processing unit, processor) CON, an input section IN, an output section OUT, a communication section COM, a storage section MEM, and a display section DIS. The drone DR6 has the same configuration and functions as the drone DR5 unless otherwise specified, but some configurations are omitted for convenience of drawing and explanation.

The drone DR5 includes cameras CM1 to 3 as imaging units, sensors SNSR (barometer, gyro sensor, etc.), control unit CON (CPU, arithmetic processing unit, processor), communication unit COM, monitoring unit MON, output unit OUT, and storage unit. Although it has an STR, it may further include an input section (not shown) and a display section (not shown). It is preferable to have two cameras, one for photographing the monitoring target and one for photographing a position indicator installed on the ceiling, etc., but one or two cameras may serve the same function. . Furthermore, it is preferable to use a high-performance camera that can take pictures even at night with little ambient light. Furthermore, the camera may be provided with a separate device that detects infrared rays, or the above camera may be equipped with that function to detect the temperature of the monitored object and detect whether or not it is dead.

The drone DR5 uses, as a sensor SNSR, a position measuring unit (not shown, such as GPS) that measures the position of its own device, a gyro sensor that measures angular velocity, an acceleration sensor that measures acceleration, and an altitude sensor that measures altitude. (such as a barometer).

Drone DR5 is
As various sensor SNSR,
A gyro sensor that measures angular velocity,
An acceleration sensor that measures acceleration;
An altitude sensor that measures altitude,
has.

In addition, the drone DR5 is
an imaging unit (cameras CM1 to CM3) that captures images during flight;
A communication department COM that receives the flight route, position indicator placement information, monitoring event conspicuousness information (feeding information (feeding location, time, etc.), intruder threat information (warning sound generation location, time, etc.)) and,
a control unit CON that controls the flight of the own device along the flight route while recognizing the position of the own device based on the position indicator arrangement information, and further based on the monitoring event salience information;
a monitoring unit MON that monitors whether a monitoring event has occurred in the image;
an output unit OUT that outputs (sends) an alarm including a position where the monitoring event occurs and an image at the time of occurrence to the server via the communication unit or other communication means when a monitoring event occurs;
has.

The control unit can also control the flight of the own device along the flight route while recognizing the position of the own device based on the position indicator arrangement information, the angular velocity, the acceleration, and the altitude. .

For example, in the case of poultry house monitoring, the monitoring event conspicuousness information is acquired from the automatic feeding server AFS or the like via a network (LAN, Internet, etc.). This is prone to malfunctions as the chickens are usually asleep and in an immobile state which is a monitoring event. However, when fed, healthy chickens move en masse to eat, while sick or dead chickens remain motionless, making monitoring events more noticeable and causing false alarms. This reduces and prevents alarms. That is, by synchronizing the timing of the monitoring flight and the feeding time, it becomes possible to monitor efficiently and without false alarms.

FIG. 10 is a flowchart showing an example of a process executed by the monitoring device shown in FIG. In the monitoring device, two devices, a server SV2 and a drone DR5, cooperate to perform processing. Once the server SV2 stores information on other servers, cooperation with the other servers may be unnecessary.

In step S21, CAD information of the building is acquired. Next, in step S22, position indicator arrangement information corresponding to the spatial coordinate system of the CAD information and information on the installation objects to be monitored (chicken house cages, desks, chairs, equipment, etc.) placed in the building are provided. Get layout information. Subsequently, in step S22-1, monitoring event conspicuous factor information (for example, feeding information) is acquired. It is preferable that the information is acquired via a communication unit. Then, in step S23, a flight route of the drone within the building for monitoring the installation to be monitored is set, or an input of the flight route is accepted. The flight route is set using CAD information, position indicator arrangement information, and layout information, but after that, it is preferable to further utilize the monitoring event conspicuousness factor information to adjust the flight timing and route. Then, in step S24, a monitoring event to be monitored (abnormal event, intruder, fire, death, specific action, etc.) is set (input may be accepted).

Then, in step S25, the drone is moved along the flight route while confirming the position using the position index and/or using the gyro sensor and acceleration sensor (or GPS). This control may be performed on the server side, or may be autonomously controlled on the drone side. In step S26, the monitoring target is photographed with a camera. In step S27, it is determined whether it is the end of the flight route or whether the remaining power of the battery mounted on the drone is insufficient. If the judgment conditions are met, the drone returns to the base. If the determination condition is not satisfied, the process advances to step S18.

In step S28, it is determined whether an abnormal event (monitoring event) has occurred in the image. If no abnormal event has occurred, the process returns to step S25 and the flight continues. If an abnormal event occurs, it is possible to issue an alarm as is, but in order to reduce false recognition, the drone should be suspended in the air (hovering) or slowed down, and the drone should continue to observe images of the abnormal event. It is preferable that Then, in step S29, the drone side outputs an alarm including the position where the monitoring event (abnormal event) occurred and the image (including video) at that time (this is once sent to the server, and the server issues an alarm). ). After issuing the warning, the process advances to step S30, and similarly to step S27, it is determined whether the flight route is at the end or whether the remaining power of the battery mounted on the drone is insufficient. If the judgment conditions are met, the drone returns to the base. If the determination conditions are not met, the process returns to step S25 and flight (monitoring) continues.

FIG. 11 is a schematic diagram illustrating a monitoring technique. This will be explained using the example of monitoring a chicken house. As shown in the figure, chickens BD1 to BD7 exist on the screen interface SL1 captured by the camera. This device has logic that recognizes the crest, beak, etc. of a chicken's head based on shapes, colors, patterns, etc., and captures all individual chickens photographed within the screen. Then, the motion of each individual is observed, and those that do not move are extracted. Chickens BD4 and BD7 on the screen interface SL2 are chickens that do not move. A chicken that does not move for a predetermined period of time is then determined to be a monitoring event, ie, an abnormal event (death, disease, or deterioration). The chicken BD4 on the screen interface SL3 is the chicken for which the abnormal event has been confirmed, and a warning is issued at this point (or when the drone returns to the drone base). Additionally, because chickens have feathers covering their bodies, it is difficult to measure their body temperature from the outside using conventional methods. However, the crest part has no wings and blood vessels run directly below it, so body temperature can be easily measured by measuring the temperature of this part with an infrared camera or the infrared function of a regular camera. , it is possible to reliably determine whether the chicken is dead or not. This device uses a known image recognition technique to capture the crest as a region of interest, and furthermore, by measuring the temperature of the crest, it is possible to determine an abnormal event with higher accuracy.

FIG. 12 is a schematic diagram illustrating map-shaped alarm information. What is transmitted (output) from the drone are images and videos of the occurrence of the monitoring event and the coordinates of the point in question. In this case, it is difficult for the person in charge who received the alarm to understand the situation, so this device monitors abnormal event occurrence points OABE1 to OABE3 of buildings BLD1 and BLD2 where the monitoring event (abnormal event) has occurred, and furthermore, A map-shaped alarm information WRN1 is displayed that displays detailed information such as "An abnormal event has occurred (third stage), cage number A101BR031". This warning is obtained by flattening the CAD information and displaying the abnormal event occurrence point. Based on the warning information, the worker checks the cage and performs a predetermined task (in this example, removing a dead or weakened individual (chicken)).

FIG. 13 is a schematic diagram illustrating map-shaped alarm information. The map-shaped alarm information WRN1 is a plan view, but in a chicken house or the like, there are several cage shelves in the vertical direction, and it is unclear at what height the cage shelves are on the plane. The map-shaped warning information WRN2 shown in FIG. 13 corresponds to this. This is done by having the worker look at the map in Figure 12 and arrive at the point where the abnormal event occurred, then look at the map in Figure 13 and understand at what vertical height of the shelf the abnormality has occurred. used for a purpose.

<Embodiment 2>
FIG. 14 is a block diagram showing an overview of a monitoring device according to an embodiment of the present invention. As shown in the figure, the monitoring device MOS3 includes mobile monitoring devices MM1 and MM2 and a server SV3. The server SV3 includes a control section (CPU, arithmetic processing unit, processor) CON, an input section IN, an output section OUT, a communication section COM, a storage section MEM, and a display section DIS. Mobile monitoring device MM12 has the same configuration and functions as mobile monitoring device MM1 unless otherwise specified, but some of the configurations are omitted for convenience of drawing and explanation.

The mobile monitoring device MM1 includes cameras CM1 to CM3 as imaging units, a sensor SNSR (barometer, gyro sensor, etc.), a control unit CON (CPU, arithmetic processing unit, processor), a communication unit COM, and a monitoring unit MON (this function is (which may be provided on the server side), an output section OUT, and a storage section STR, but may further include an input section (not shown) and a display section (not shown). It is preferable to install two cameras: one to photograph the monitoring target such as the poultry house, and one to photograph the position indicator (point marker, line type, etc.) installed on the ceiling, floor, etc. However, one or two may serve the same function. Furthermore, it is preferable to use a high-performance camera that can take pictures even at night with little ambient light. Furthermore, the camera can be equipped with a separate device that detects infrared rays, or the camera can be equipped with that function to detect the temperature of the monitored object, and determine the condition of the object, for example, whether it is dead or sick. It is also possible to detect whether or not the information is present.

The mobile monitoring device MM1 includes, as a sensor SNSR, a position measuring unit (not shown, such as GPS) that measures the position of the own device, a gyro sensor that measures angular velocity, an acceleration sensor that measures acceleration, and an altitude. It has an altitude sensor and a distance meter that measures or estimates the distance traveled by the device itself.

The mobile monitoring device MM1 is
an imaging unit CM1-3 that captures an image;
a drive unit DR that moves the own device;
an acquisition unit RET that acquires the position of the mobile monitoring device;
a communication unit COM that transmits monitoring information including a position of the mobile monitoring device and an image at the position to the server via a wireless or wired communication means;
has.

The server SV3 is
a communication unit COM that receives the monitoring information including the position of the mobile monitoring device and an image at the position from the mobile monitoring device;
a monitoring event setting unit MES for setting a monitoring event to be monitored (abnormal event, intruder, fire, death, illness, specific action, etc.);
a monitoring unit MON that monitors whether or not a set monitoring event occurs in the image;
an output unit OUT that outputs an alarm (preferably including the image and the position) when the monitoring event occurs;
has.
Note that suitable output warnings include sending an e-mail, outputting or transmitting a warning voice, outputting an image, warning a supervisor, warning a manager, and reporting to a security company, police, etc. In addition, in monitoring whether or not the set monitoring event has occurred, the monitoring unit uses AI techniques to monitor the image information IM (or derived information derived from the image information (image characteristics) in the storage unit). It is preferable to monitor whether or not the set monitoring event is occurring by referring to a pattern, etc.).

The mobile monitoring device MMM1 is
It further includes a guide part GD that is movably supported, placed, or suspended on the guide rail and guided along the guide rail,
The drive unit DR is
The device is moved along the guide rail while being supported, placed, or suspended by the guide rail.

CAD information of buildings, shelves, chicken houses, etc. to be monitored may be acquired from a building information server or a storage unit MEM connected via the network NET. CAD information, position indicator arrangement information that corresponds to or corresponds to the spatial coordinate system of this CAD information, and installations to be monitored placed in the building (chicken house cages, desks, chairs, equipment, etc.) ), the travel route is set from the layout information. The display unit DIS can display information stored in the device and information generated.

It is preferable that each functional unit included in the control unit of the server SV3 is realized by a program module read into the memory space of the control unit. Further, the drone can realize each functional section by a program module read into the memory space of the control section, or can realize each functional section by a built-in GPS unit, camera unit, etc. Normally, the server downloads software from a storage unit or a website that causes the processor to function as each part of the device, installs it on the PC, and starts it, so that the PC operates as the device. Note that each functional section provided in the control section is nothing more than a collection of steps that have a certain set of functional functions, and it is possible to combine multiple functional sections into one functional section, or to combine some of them into other functional sections. It can be incorporated into a section or divided into other independent functional sections.

In this way, generated/extracted information, intermediate data, and acquired data can be sent to the outside, displayed on the display, and generated/extracted information, intermediate data, and acquired data can be stored in the storage. It should be noted that this is possible in other embodiments as well, as described below. This device may be a computer such as a general-purpose computer, a specific-purpose computer, a server, a PC, or a mobile terminal such as a smartphone, or a program module that implements (executes) the functions and processing procedures (methods) of this device on a computer. It is preferable to build this device on a computer by holding it in the computer's CPU or storage unit, or reading it from an external server or storage, and the same applies to each of the subsequent embodiments. Additionally, each functional unit may be distributed to separate computers or devices connected via a network. Alternatively, a configuration may be adopted in which a plurality of functional units are combined into one, or a part of the processing steps is performed by another functional unit. Furthermore, in this embodiment, the server and the mobile monitoring device are defined as separate entities, but the mobile monitoring device may include some or all of the functions of the server.

FIG. 15 is a flowchart showing an example of a process executed by the monitoring device shown in FIG. 14. In the monitoring device, two devices, server SV3 and mobile monitoring device MM1, cooperate to perform processing. Once server SV3 stores information on other servers, cooperation with other servers may be unnecessary.

In step S31, the camera of the mobile monitoring device captures an image. Next, in step S32, the acquisition unit acquires the position of the mobile monitoring device. Then, in step S33, monitoring information including the location of the mobile monitoring device and an image at the location is transmitted to the server via wireless or wired communication means. In step S34, the server sets an abnormal event to be monitored (death, illness, fire, intruder, abnormal behavior, etc.). In step S35, the drive unit of the mobile monitoring device moves the mobile monitoring device to the guide rail while checking the position using the position index and/or using the gyro sensor and acceleration sensor (or GPS, distance meter). move it along. For movement, simple methods such as supplying power to a motor that moves a predetermined distance (for example, the distance of one cage in a poultry house) or operating the motor for a predetermined period of time can be used without using sensors. But that's fine.

In step S36, the camera of the mobile monitoring device captures an image. Next, the position of the mobile monitoring device is acquired, and in step S36-1, monitoring information including the position of the mobile monitoring device and an image at the position is transmitted to the server via wireless or wired communication means. do. In step S37, it is determined whether it is the end of the rail or whether the remaining power of the battery mounted on the mobile monitoring device is insufficient. If the judgment conditions are met, return to base. If the determination condition is not satisfied, the process advances to step S38.

In step S38, it is determined whether an abnormal event (monitoring event) has occurred in the image. If no abnormal event has occurred, the process returns to step S35 and continues moving and imaging. If an abnormal event has occurred, an alarm may be issued as is, but in order to reduce false recognition, the process proceeds to step S38-1, where the mobile monitoring device is stopped or decelerated, and the mobile monitoring device is stopped or decelerated to prevent the abnormal event from occurring. It is preferable to continue observing the image. If it is determined in step S38-2 that the abnormal event continues to occur, an alarm including the position where the abnormal event occurred and the image (including video) at that time is output in step S39. do. If it is determined in step S38-2 that the occurrence of the abnormal event is not continuing, it is determined that there is an erroneous recognition, and the process returns to step S35 without issuing an alarm and continues moving (monitoring). After issuing the alarm in step S39, the process proceeds to step S40-1, and similarly to step S37, it is determined whether the rail (route) is at its end or whether the remaining power of the installed battery is insufficient. Determine. If the determination conditions are met, the mobile monitoring device returns to its base (charging spot). If the determination condition is not satisfied, the process returns to step S35 and movement (monitoring) is continued.

FIG. 16 is a schematic diagram illustrating the monitoring technique of the mobile monitoring device of the present device. As shown in the figure, position indicators PI1 to PI4 installed on the ceiling of the building BLD are photographed by camera CM3, and the position of the own device is recognized based on the indicators. The position indicators are arranged at the coordinates of the position indicator arrangement information, and the mobile monitoring device MM1 sequentially recognizes the position indicators PI1 to PI4 along the movement route RT1 while recognizing the position of its own device using the position indicators. While moving, it controls the movement (drive) of its own device. When moving, it moves along a guide rail GR placed on the floor (this may be placed or installed on the ceiling or wall). The position indicator may be integrated into the guide rail GR. Alternatively, the position may be measured and estimated using a distance meter or the like without using a position index. In addition, it is preferable that the positional index is decorated with one or more of fluorescent materials, letters, figures, patterns, colors, etc., and the positional index can be image-recognized by photographing it with the camera CM3 pointing toward the ceiling. is suitable. Each position index is unique, and by recognizing them individually and referring to the position index placement information LILI, it is possible to determine where the mobile monitoring device should be located. Alternatively, the position indicator may be a device that emits a specific wireless or optical signal, a WIFI base station, or a wireless tag (in this case, the signal is read by a wireless communication unit, optical receiver, etc. instead of a camera). Furthermore, it is also possible to provide an indoor GPS signal unit in place of the position indicator and read this signal with the GPS unit of the mobile monitoring device.

Then, while moving, the mobile monitoring device photographs the shelves SBJ1 and SBJ2 (installed in the building BLD) to be monitored on the left and right using cameras CM1 and CM2, and determines whether a monitoring event is occurring there. to monitor.

FIG. 17 is a schematic diagram illustrating another monitoring technique of the mobile monitoring device of the present device. As shown in the figure, the mobile monitoring device MM2 is self-propelled, driven by tires or endless tracks, and moves along a guideline GL placed on the floor or painted on it. Other functions are similar to those in FIG. 16. The guideline GL is preferably a magnetic tape, a coating containing magnetism, a film with a characteristic color scheme, a tape, a coating (for example, white, yellow, etc.). The guideline corresponds to the flight route described above and functions as a driving route.

FIG. 18 is a schematic diagram illustrating a monitoring technique of the mobile monitoring device of the present device when applied to a poultry house. As shown in the figure, a guide portion GD of the mobile monitoring device MM3 is supported by a guide rail GR installed on the ceiling SL of the building BLD, and the mobile monitoring device MM3 is suspended. The mobile monitoring device MM3 controls the movement (drive) of its own device along the movement route RT1 while recognizing its own position using some kind of position index, distance meter, or the like. When moving, it moves along a guide rail GR installed on the ceiling SL. The position indicator may be integrated into the guide rail GR. Alternatively, the position may be measured and estimated using a distance meter or the like without using a position index.

The mobile monitoring device MM3 has a camera CM2-3 for the upper chicken houses CG31 and CG41, and a camera CM4-5 for the lower chicken houses CG32 and CG42. In this example, the abnormal event is death or a weak chicken due to illness, and chickens BD11 and BD12 are normal and are not recognized as abnormal events. Since the chickens BD13 and BD14 are crouching or have fallen down, the server side determines that this is an abnormal event and issues an alarm. In the event of an alarm, the poultry house number CG32. Output the poultry house number CG42 or some position information and the image.

As is common between embodiment 1 and embodiment 2, the use of monitoring event salience information improves the precision of monitoring and can reduce false alarms to a considerable extent. Monitoring event salience information includes feeding information (feeding location, time, etc.), intruder threat information (warning light, warning vibration, warning sound location, time, etc.), odor information (e.g., information that the monitored chickens dislike, , exciting odors, etc.), light information (lighting, indoor illuminance, blinking, revolving lights, etc.), opening/closing information (opening/closing windows and ventilation fans, etc.), and air conditioning setting information. That's all.

Therefore, the mobile monitoring device is further provided with monitoring event conspicuousness generating means for generating the condition of the monitoring event conspicuousness information (for example, a revolving light that emits light, a speaker that emits sound, a device that comes into contact with the poultry house as it moves, etc.). A rod-shaped part that emits vibrations and sounds, a signal output part that starts feeding, a control signal output part that controls lighting and sound output, etc.) are suitable.

FIG. 19 is a schematic diagram illustrating a monitoring technique of a mobile monitoring device of the present device provided with a monitoring event conspicuous situation generating means. As shown in the figure, the mobile monitoring device MM4 uses light (preferably light or color of light that does not stress the chickens and activates them) only in the area to be monitored as means for generating a monitoring event conspicuous situation. The poultry house has rotating lights PL1 and PL2 (rotating lights) that generate vibrations and wavelengths to stimulate the chickens, and poultry house striking rods BR1 and BR2 that generate vibrations and sounds only in the area to be monitored and stimulate the chickens. It is preferable that the vibration and sound at this time be as close as possible to the sound and vibration of feeding. This is because when given the sound and vibrations of feeding, healthy chickens will always react, stand up, and become active. Further, it is suitable as a means for generating a monitoring event conspicuous situation in a speaker that reproduces and reproduces the sound during feeding. Alternatively, means for generating the odor of food, male odor (pheromone), etc. are also effective as means for generating a monitoring event conspicuous situation.

FIG. 20 is a schematic diagram illustrating a monitoring technique of the mobile monitoring device of the present device when applied to a poultry house. As shown in the figure, the line reading sensor LSN of the mobile monitoring device MM5 reads the guideline GL installed, laid, or applied on the floor FL of the building, and the mobile monitoring device MM5 automatically moves along the guideline GL. The mobile monitoring device MM5 controls the movement (drive) of its own device along the guideline GL while recognizing its own position based on the guideline (preferably using a distance meter, a gyro sensor, etc.). The guideline may include not only guidance but also an integrated position indicator. Alternatively, the position may be measured and estimated using a distance meter or the like without using a position index.

FIG. 21 is a schematic diagram illustrating a monitoring technique of the mobile monitoring device of the present device when applied to a poultry house. As shown in the figure, the line reading sensor LSN installed at the top of the mobile monitoring device MM6 reads the guideline GL installed, laid, or applied on the ceiling SL of the building, and the mobile monitoring device MM6 automatically moves along the guideline GL. . The mobile monitoring device MM5 moves (drives) its own device along the guideline GL while recognizing its own position based on the guideline (preferably using a distance meter, an accelerometer, and/or a gyro sensor, etc.). ). The guideline may include not only guidance but also an integrated position indicator. Alternatively, the position may be measured and estimated using a distance meter or the like without using a position index. The advantage of this embodiment is that it can prevent the guideline installed on the floor from being read incorrectly when the poultry house is contaminated with chicken droppings, feathers, etc. This is because there is almost no possibility that the ceiling will be contaminated with chicken droppings or feathers.

The above-described monitoring technique uses image information as a monitoring method, and a monitoring technique using a distance sensor will be described in the third embodiment below.

<Embodiment 3>
FIG. 22 is a block diagram showing an overview of a monitoring device according to an embodiment of the present invention. As shown in the figure, the monitoring device MOS4 includes mobile monitoring devices MM7 and MM8 and a server SV4. The server SV4 includes a control section (CPU, arithmetic processing unit, processor) CON, an input section IN, an output section OUT, a communication section COM, a storage section MEM, and a display section DIS. Mobile monitoring device MM7 has the same configuration and functions as mobile monitoring device MM1 unless otherwise specified, but some of the configurations are omitted for convenience of drawing and explanation.

The mobile monitoring device MM7 includes cameras CM1 to CM3 as imaging units, sensors SNSR1-4 (distance meter), control unit CON (CPU, arithmetic processing unit, processor), communication unit COM, output unit OUT, and storage unit STR. , and may further include an input section (not shown) and a display section (not shown). There are two cameras that take pictures of the monitoring target such as the chicken house, and one that takes pictures of the direction of movement and position indicators and identification information (point markers, line-shaped ones, etc.) installed on the ceiling, floor, etc. Although it is preferable to provide one, one or two may serve the same function. Furthermore, it is preferable to use a high-performance camera that can take pictures even at night with little ambient light. Furthermore, the camera can be equipped with a separate device that detects infrared rays, or the camera can be equipped with that function to detect the temperature of the monitored object, and determine the condition of the object, for example, whether it is dead or sick. It is also possible to detect whether or not the information is present.

Preferably, the mobile monitoring device MM7 includes a position measuring unit (not shown, such as a GPS) that measures the position of the mobile monitoring device, a gyro sensor that measures angular velocity, an acceleration sensor that measures acceleration, and a position measuring unit that measures the position of the mobile monitoring device MM7. It has a rangefinder to measure or estimate distance.

The mobile monitoring device MM7 is
A camera CM1-3, which is an imaging unit that captures an image,
a sensor SNSR1-4 that acquires distance information;
an acquisition unit RET that acquires identification information (such as a guide tape) that determines the movement direction of the mobile monitoring device;
a drive unit DR that moves the own device based on the identification information;
a communication unit COM that transmits monitoring information including the distance information and the image to the server via a wireless or wired communication means;
has.

Server SV4 is
a storage unit MEM that stores model change pattern information MCP indicating a monitoring event;
a communication unit COM that receives the monitoring information including the distance information and the image from the mobile monitoring device;
a monitoring unit MON that compares the model change pattern information and the change pattern of the distance information to monitor whether a monitoring event (death, illness, sitting state, etc.) has occurred;
an output unit OUT that outputs an alarm (preferably including an image of the position and the position) when the monitoring event occurs;
has.

Note that suitable output warnings include sending an e-mail, outputting or transmitting a warning voice, outputting an image, warning a supervisor, warning a manager, and reporting to a security company, police, etc. In addition, in monitoring whether or not the set monitoring event has occurred, the monitoring unit uses AI techniques to monitor the image information IM (or derived information derived from the image information (image characteristics) in the storage unit). It is preferable to monitor whether or not the set monitoring event is occurring by referring to a pattern, etc.). Examples using AI techniques are listed below.

The mobile monitoring device
an imaging unit that captures an image;
A sensor that acquires distance information,
an acquisition unit that acquires identification information (such as a guide tape) that determines the movement direction of the mobile monitoring device;
a drive unit that moves the device based on the identification information;
a communication unit that transmits monitoring information including the distance information and the image to the server via wireless or wired communication means;
has.

The server is
a memory unit MEM that stores a large number of distance change patterns;
an AI engine trained based on the plurality of distance change patterns and monitoring events associated with at least some of the change patterns;
a communication unit COM that receives the monitoring information including the distance information and the image from the mobile monitoring device;
The AI engine is
Based on the change pattern of the distance information, monitor whether a monitoring event (death, illness, sitting state, etc.) has occurred;
The server is
further comprising an output unit that outputs an alarm (preferably including an image of the position and the position) when the monitoring event occurs;
It is characterized by

FIG. 23 is a flowchart showing an example of processing executed by the monitoring device shown in FIG. 22. As shown in the figure, in step S41, model change pattern information MCP indicating a monitoring event is stored. Next, in step S42, the monitoring information including the distance information and the image is received from the mobile monitoring device. Further, in step S43, the model change pattern information is compared with the change pattern of the distance information to monitor whether a monitoring event (death, illness, sitting state, etc.) has occurred. Of course, as with other implementations, it is also possible to configure the monitoring events as desired. Further, in step S44, when a monitoring event occurs, an alarm (preferably including an image of the position and the position) is output.

FIG. 24 is a schematic diagram illustrating the monitoring technique of the mobile monitoring device of the present device when applied to a poultry house. As shown in the figure, the tape reading sensor TSN installed at the bottom of the mobile monitoring device MM7 reads the guide tape GT installed, laid, or applied on the floor FL of the building, and automatically moves the guide tape GT along the guide tape GT. Moving. The mobile monitoring device MM7 moves its own device along the guide tape GT while recognizing its own position using the guide tape (preferably using a distance meter, an accelerometer, and/or a gyro sensor, etc.). (drive) to control. The guide tape may have not only guidance but also a position indicator integrated therein. Alternatively, the position may be measured and estimated using a distance meter or the like without using a position index. An advantage of this embodiment is that even if the poultry house is dark, monitoring events can be identified by utilizing distance sensors without relying on image recognition.

Distance sensors SNSR1, 2, 3, and 4, which measure distance, emit measurement lines such as laser beams and infrared rays toward the upper and lower left and right cages CG31-34, respectively, and upon receiving the reflected lines, Measure distance. The measurement line is emitted to hit the chicken's legs, and a standing chicken will detect the distance along the shape of its thin legs, and a sitting chicken will detect the distance along the shape of its body.

FIG. 25 is a diagram illustrating an example of sensor distance measurement using the present device. As shown in the figure, the mobile monitoring device MM7 moves (progresses) from left to right. Then, while moving, the sensor SNSR1 measures the distance at a certain poultry house CG32. Here, it is assumed that there are chickens BD1-4 in the chicken house, but BD1, 3, and 4 are standing, and BD2 is crouching down (that is, weak or dead). When the measured distance is plotted in a graph G32, the vertical axis is the distance, and the horizontal axis is the distance in the traveling direction (travel distance or time). In graph G32, a standing chicken appears as a stick-shaped silhouette with a short distance (time), and a sitting chicken BD2 (dead chicken) appears as a rounded silhouette. In this device, the AI learns such a distance change pattern and stores it as model change pattern information, and compares it with it to determine normal chickens and abnormal chickens. Alternatively, rather than having a shape like a graph, the change pattern can be determined using a mathematical method that determines a carcass if it remains below a predetermined threshold for a predetermined period of time (distance traveled) after the distance has been displaced. may be recognized.

FIG. 26 is a diagram illustrating an example of sensor distance measurement using the present device. As shown in the figure, in graph G32, a standing chicken appears as a bar-shaped silhouette with a short distance (time), and a sitting chicken BD2 (dead chicken) appears as a rounded silhouette. However, in this graph, the actual position in the chicken house is upside down, making it difficult to grasp the actual position of the chickens. Therefore, this device is capable of generating an inverted graph G32-I that is inverted vertically through waveform processing. Then, it is possible to display a sitting chicken dBD (dead chicken) superimposed on the rounded silhouette position on the display unit.

FIG. 27 is a diagram illustrating an example of waveform processing (an example of numerical processing) for distance measurement of the sensor by this device. As shown in graph G33 in the figure, when distances are measured inside the poultry house, there are thin waveforms of only the legs (i.e., those with a short movement distance (time) from one large displacement to the next large displacement), and the thick waveforms of the sitting chickens. There is a waveform (that is, one in which the movement distance (time) from one large displacement to the next large displacement is long).The inventors focused on this and , has a filter function that excludes waveforms whose movement distance (time) is less than a predetermined threshold (that is, leg waveforms (normal chicken waveforms/values). Graph G33-1 shows that the filter function allows These are the waveforms and values of a sitting chicken, that is, the waveforms and values of only the carcass.

FIG. 28 is a diagram illustrating an example of waveform processing (an example of numerical processing) for distance measurement of the sensor by this device. As shown in the figure, when measuring the distance inside the poultry house, there are thin waveforms of only the legs (i.e., the movement distance (time) from one large displacement to the next large displacement is short), and thick waveforms of the chickens sitting (i.e. , the movement distance (time) from one large displacement to the next large displacement is long).Furthermore, in actual poultry houses, there are members that make up the fence, such as horizontal bars GR1-5, and these are measured. Graph G34-I is a waveform processed to invert the distance measurement results.

FIG. 29 is a diagram illustrating an example of waveform processing (an example of numerical processing) for distance measurement of the sensor by this device. As shown, graph G34-I is similar to that in FIG. This device can project a chicken's feet on a graph or a silhouette of a sitting chicken as a virtual chicken at the corresponding position in the actual gauge arrangement. For example, a standing normal chicken is projected as virtual chickens V-BD1, V-BD3, and 4. The sitting chicken is then projected as a virtual chicken V-BD2. Also, gratings GR1-5 are also projected. With this function, poultry house staff can easily recognize which poultry house and where the abnormal chicken is located, and remove it to improve hygiene management.

FIG. 30 is a diagram showing an example of waveform processing (an example of numerical processing) for distance measurement of the sensor by this device. This example shows waveforms from two sensors measuring distance in the same direction. That is, two sensors are provided separated by a predetermined distance (for example, 10 cm) in the traveling direction (horizontal direction). As shown in the figure, the graph G40 obtained by the sensor in front shows normal waveforms PK10, PK12, and PK13 of a standing chicken, and waveform PK11 of an abnormal chicken sitting. As shown in the figure, in the graph G41 obtained from the rear sensor, waveforms PK22 and PK23 of normal standing chickens and waveforms PK20 and PK21 of abnormal sitting chickens appear. The waveform PK20 here is an error caused by the movement of the chicken's legs, and is a detection error that is unavoidable when measuring a living chicken. Therefore, the present inventors came up with the idea of eliminating this detection error by using two sensors, overlapping the two graphs (shifted by a predetermined distance), and excluding misaligned and mismatched waveforms. Made it happen. As shown in graph G42 excluding errors due to movement of the monitoring target, it is possible to provide a waveform PK20 excluding components of unreliable detection errors. The device configuration is listed below.

Mobile monitoring equipment is
a first sensor that acquires first distance information;
a second sensor that acquires second distance information;
an acquisition unit that acquires identification information (such as a guide tape) that determines the movement direction of the mobile monitoring device;
a drive unit that moves the device based on the identification information;
a communication unit that transmits monitoring information including the first distance information, the second distance information, and the image to the server via a wireless or wired communication means;
has
The server is
a storage unit that stores model change pattern information indicating a monitoring event;
a communication unit that receives the monitoring information including the first distance information, second distance information, and the image from the mobile monitoring device;
a pattern generation unit that compares the first distance information and the change pattern of the second distance information and generates an excluded change pattern that excludes a component caused by movement of a target object (such as a chicken) from each change pattern;
a monitoring unit that compares the model change pattern information and the excluded change pattern to monitor whether a monitoring event (death, illness, sedentary state, etc.) has occurred;
an output unit that outputs an alarm (preferably including an image of the position and the position) when the monitoring event occurs;
has.

Although the present invention has been described based on the drawings and examples, it should be noted that those skilled in the art will be able to easily make various changes and modifications based on the present disclosure. Therefore, it should be noted that these variations and modifications are included within the scope of the present invention. For example, the processes and functions included in each part and each step can be rearranged to avoid logical contradictions, and multiple means/parts and steps can be combined into one or divided. It is possible. Alternatively, it should be noted that some components, functions, processes, steps, etc. of the apparatus, method, program, etc. according to the present invention may be located in a remote server or the like. Further, although the present invention has been described as a real-time monitoring device, it can also be used as an editing device or an editing system that takes captured images as input and cuts out the time period or moment when a suspicious object was photographed.

BD1-7 Chicken PI1-4 Position indicator arrangement information CM1-3 Camera OABE1-3 Abnormal event occurrence point ABE Setting interface ABE1 Abnormal event ABE2 Abnormal event ABE3 Abnormal event ABE5 Abnormal event ABE6 Abnormal event ABE7 Abnormal event ACQ Acquisition unit AFS Automatic feeding server BLD Building BLD1 Building CI CAD information CIALL CAD information CM1 Camera CM3 Camera COM Communication section CON Control section DB Drone base DIS Display section DR1 Drone DR2 Drone DR5 Drone DR5 Drone DR6 Drone FLS Flight route setting section FR1 Flight route FR1, FR2 Flight route FR2 Flight route IN Input section LILI Position indicator arrangement information MEM Storage section MES Monitoring event setting section MON Monitoring section MOS1 Monitoring device MOS2 Monitoring device NET Network OABE Abnormal event OUT Output section SBJ1, SBJ2 Monitoring target SBJ3, SBJ4 Monitoring target SHLF Shelf SHLF1 Shelf SL1 Screen interface SL2 Screen interface SL3 Screen interface SLI Layout information SNSR Sensor STR Storage section SV1 Server SV2 Server WRN1 Alarm information WRN2 Alarm information

Claims (5)

A monitoring device including a mobile monitoring device and a server,
The mobile monitoring device includes:
an imaging unit that captures an image;
A sensor that acquires distance information,
an acquisition unit that acquires identification information that determines the movement direction of the mobile monitoring device;
a drive unit that moves the device based on the identification information;
a communication unit that transmits monitoring information including the distance information and the image to the server via wireless or wired communication means;
has
The server is
a storage unit that stores model change pattern information indicating a monitoring event;
a communication unit that receives the monitoring information including the distance information and the image from the mobile monitoring device;
a monitoring unit that compares the model change pattern information and the change pattern of the distance information to monitor whether a monitoring event has occurred;
an output unit that outputs an alarm when the monitoring event occurs;
has,
A monitoring device characterized by: A monitoring device including a mobile monitoring device and a server,
The mobile monitoring device includes:
an imaging unit that captures an image;
A sensor that acquires distance information,
an acquisition unit that acquires identification information that determines the movement direction of the mobile monitoring device;
a drive unit that moves the device based on the identification information;
a communication unit that transmits monitoring information including the distance information and the image to the server via wireless or wired communication means;
has
The server is
a storage unit that stores a large number of distance change patterns;
an AI engine trained based on the plurality of distance change patterns and monitoring events associated with at least some of the change patterns;
a communication unit that receives the monitoring information including the distance information and the image from the mobile monitoring device;
The AI engine is
Monitoring whether a monitoring event is occurring based on the change pattern of the distance information,
The server is
further comprising an output unit that outputs an alarm when the monitoring event occurs;
A monitoring device characterized by: A monitoring device including a mobile monitoring device and a server,
The mobile monitoring device includes:
an imaging unit that captures an image;
a first sensor that acquires first distance information;
a second sensor that acquires second distance information;
an acquisition unit that acquires identification information that determines the movement direction of the mobile monitoring device;
a drive unit that moves the device based on the identification information;
a communication unit that transmits monitoring information including the first distance information, the second distance information, and the image to the server via a wireless or wired communication means;
has
The server is
a storage unit that stores model change pattern information indicating a monitoring event;
a communication unit that receives the monitoring information including the first distance information, second distance information, and the image from the mobile monitoring device;
a pattern generation unit that compares the first distance information and the change pattern of the second distance information and generates an excluded change pattern that excludes a component caused by movement of the object from each change pattern;
a monitoring unit that compares the model change pattern information and the excluded change pattern to monitor whether a monitoring event has occurred;
an output unit that outputs an alarm when the monitoring event occurs;
has,
A monitoring device characterized by: A monitoring program that causes one or more arithmetic processing devices to function as the monitoring device according to any one of claims 1 to 3. A computer-readable storage medium storing the monitoring program according to claim 4.

Images