JP6544501B1 - Monitoring system and control method of monitoring system - Google Patents

Abstract

There is provided a monitoring system capable of reliably alarming a position relationship between a position of a high temperature object in a three-dimensional space and another object. An infrared camera which scans a first area and outputs a distance image of a three-dimensional coordinate system, and an infrared camera which captures a second area overlapping the first area and outputs an infrared image of the two-dimensional coordinate system 104, acquiring a distance image and an infrared image, and specifying the position of the specific temperature object corresponding to the specific temperature portion in the three-dimensional coordinate system when the specific temperature portion is in the infrared image, in the distance image When another object is detected, the control unit 120 outputs an alarm signal when the distance between the specific temperature object and the other object is within a predetermined distance, and receives an alarm signal from the control unit 120 to issue an alarm. A monitoring system 100 having an alarm 140; [Selected figure] Figure 1

Description

  The present invention relates to a monitoring system and a control method of the monitoring system.

  Conventionally, as a technique for securing safety in a predetermined area, there is one using a radar image acquired from a rider (also referred to as a laser radar) and a camera image acquired from a surveillance camera. In this technique, a coordinate system of a radar image and a coordinate system of a camera image are superimposed, and auxiliary information for specifying a moving object is superimposed and displayed on the camera image. By this display, the situation in the monitoring area can be easily grasped (Japanese Patent Laid-Open No. 2004-212129).

  In addition, as a technology for detecting the movement state and position of a thermal object in space, there is one using an infrared sensor composed of a plurality of infrared detection elements. According to this technique, it is possible to easily grasp the movement condition and position of an object having a high temperature (Japanese Patent Application Laid-Open No. 9-297057).

  According to the technology described in Japanese Patent Laid-Open No. 2004-212129, it is possible to grasp the situation of a mobile object present in the area to be monitored. However, this conventional technology recognizes a high temperature object from many objects present in the area to be monitored, grasps the movement situation, and the distance between the high temperature object and another object. I can not understand Also, it is not possible to monitor the safety of other objects based on the temperature of the hot object.

  Further, according to the technique described in Japanese Patent Application Laid-Open No. 9-297057, it is possible to grasp the movement situation and position of an object having a high temperature in a two-dimensional direction. However, this conventional technique can not grasp the movement situation or position of a high temperature object moving in the three-dimensional direction.

  Therefore, an object of the present invention is to provide a monitoring system capable of issuing an alarm by reliably grasping the positional relationship between a high-temperature object in three-dimensional space and another object, and a control method of the monitoring system. It is.

  The above object is achieved by the following means.

(1) A lidar that outputs a distance image in which a distribution of distance values obtained by scanning laser light toward the first region is shown in a three-dimensional coordinate system;
An infrared camera which captures a second area in which at least a part of the first area overlaps with the first area and outputs an infrared image shown in a two-dimensional coordinate system;
Acquiring the distance image from the rider, detecting an object from the acquired distance image, and acquiring the infrared image from the infrared camera;
In the infrared image, when there is a specific temperature portion in a predetermined temperature range, an object corresponding to the specific temperature portion among the detected objects is specified as a specific temperature object in the three-dimensional coordinate system. And when the detected object includes another object different from the specific temperature object, it is determined whether or not the distance between the specific temperature object and the other object is within a predetermined distance. A control unit that outputs an alarm signal when the distance between the specific temperature object and the other object is within a predetermined distance;
Monitoring system with.

(2) The control unit
The range of the predetermined distance from the specific temperature object is set as an alarm area around the specific temperature object in the three-dimensional coordinate system, and the alarm signal is output when the other object is in the alarm area. The monitoring system according to the above (1).

  (3) The control unit adjusts the alarm area to the movement of the specific temperature object when it is determined that the specific temperature object is a moving object from the plurality of distance images acquired in time series from the rider. The monitoring system according to (2) above, which moves.

  (4) The monitoring system according to (2), wherein the control unit sets the alarm area fixed around the specific temperature object.

(5) The control unit
The relative distance and relative velocity between the specific temperature object and the other object are determined from the plurality of distance images acquired in time series from the rider, and the determined relative distance and relative distance between the specific temperature object and the other object The monitoring system according to any one of the above (1) to (4), wherein the length of the predetermined distance is changed according to the speed.

  (6) The controller according to any one of (1) to (5), wherein the control unit lengthens the predetermined distance as the temperature of the specific temperature portion in the infrared image captured by the infrared camera increases. Monitoring system.

  (7) The monitoring system according to any one of (1) to (6), including an alarm that receives the alarm signal and emits sound and / or light.

(8) A lidar for outputting a distance image in which a distribution of distance values obtained by scanning laser light toward the first region is shown in a three-dimensional coordinate system;
A control method of a monitoring system, comprising: an infrared camera for photographing a second area at least partially overlapping the first area and outputting an infrared image represented by a two-dimensional coordinate system,
Acquiring the distance image from the rider and detecting an object from the acquired distance image (a);
When the infrared image is acquired from the infrared camera, and the infrared image includes a specific temperature portion having a predetermined temperature range, an object corresponding to the specific temperature portion among the detected objects is a specific temperature object Identifying the position in the three-dimensional coordinate system as (b),
If the detected object includes another object different from the specific temperature object, it is determined whether the distance between the specific temperature object and the other object is within a predetermined distance, Issuing an alarm if the distance between the specified temperature object and the other object is within a predetermined distance (c);
A control method of a monitoring system having:

(9) In the step (c),
A range of the predetermined distance from the specific temperature object is set as an alarm area around the specific temperature object in the three-dimensional coordinate system, and an alarm is issued when the other object is in the alarm area. The control method of the monitoring system as described in 2.).

  (10) When it is determined that the specific temperature object is a moving object from the plurality of distance images acquired in time series from the rider, the alarm area is moved according to the movement of the specific temperature object. The control method of the monitoring system as described in (9).

  (11) The control method of the monitoring system according to (9), wherein the alarm area is fixedly set around the specific temperature object.

  (12) A relative distance and a relative velocity between the specific temperature object and the other object are determined from a plurality of distance images acquired in time series from the rider, and a relative between the determined specific temperature object and the other object The control method of the monitoring system according to any one of (8) to (11), wherein the length of the predetermined distance is changed according to the distance and the relative velocity.

  (13) The monitoring system according to any one of (8) to (12), wherein the size of the predetermined distance is changed according to the temperature of the specific temperature portion in the image captured by the infrared camera. Control method.

  (14) The control method of the monitoring system according to any one of (8) to (13), wherein the alarm is by sound and / or light.

FIG. 1 is a block diagram showing a configuration of a monitoring system of a first embodiment. It is an explanatory view for explaining a distance image obtained by the rider scanning the first area. It is explanatory drawing for demonstrating the infrared image obtained by the infrared camera imaging | photography of 2nd area | region. It is explanatory drawing for demonstrating the example which overlap | superposed the distance image and the infrared image simply. It is a flowchart which shows the process sequence of coordinate conversion factor calculation. It is explanatory drawing of coordinate transformation coefficient calculation. It is explanatory drawing of coordinate transformation coefficient calculation. It is a flowchart which shows the process sequence of the monitoring operation by a control part. It is a screen example figure showing a display example. It is an explanatory view for explaining a specific temperature object. It is an explanatory view for explaining an alarm field established around a specific temperature object. It is an explanatory view for explaining an alarm field established around a specific temperature object. FIG. 7 is a block diagram showing the configuration of a monitoring system of a second embodiment. It is a bird's-eye view which shows arrangement | positioning of a monitoring part. FIG. 10 is a subroutine flowchart showing a procedure of a display processing step in Embodiment 2. FIG.

  Hereinafter, embodiments of the present invention will be described with reference to the attached drawings. The present invention is not limited to the following embodiments. In the description of the drawings, the same elements will be denoted by the same reference symbols, without redundant description. In addition, since the drawings are created for the purpose of facilitating the understanding of the present invention, the drawings are described in an exaggerated manner, and dimensional ratios and the like of the drawings may be different from actual dimensional ratios.

(Embodiment 1)
(Configuration of monitoring system)
FIG. 1 is a block diagram showing the configuration of the monitoring system of the first embodiment.

  The monitoring system 100 includes a monitoring unit 110, a control unit 120, a display 130, and an alarm 140. The monitoring unit 110 is installed at a position where an object (for example, an object such as a high-temperature object, a person, a vehicle, or another object) can be captured.

  In the monitoring unit 110, a rider 102 (LiDAR: Light Detection And Ranging) and an infrared camera 104 are integrally mounted in the same housing. By doing this, for example, the direction of the optical axis of the rider 102 and that of the infrared camera 104 coincide with each other (the same direction as the Z direction (see FIGS. 2 and 3 described later)). The rider 102 and the infrared camera 104 are disposed adjacent to each other in the Y direction (vertical direction (see FIGS. 2 and 3 described later)), and the optical axes are aligned in the X direction (horizontal direction).

  The lidar 102 scans the laser light toward the space of the first region and measures the distance from the reflected light to an object present in the space to be scanned. The obtained distribution of distance values is also referred to as point cloud data, and the distance from the installation position of the rider 102 to the object, and the size and shape of the object can be known. When the lidar 102 scans the laser beam for one frame, an image is obtained which is a distance distribution to an object present in the space or a distribution of distance values which is an infinite distance in the portion where there is no reflected light. Such an image output from the rider 102 is also referred to as a distance image (sometimes referred to as a rider image) because it also includes information on the distance to the object. The rider 102 outputs this distance image to the control unit 120 as an image of a three-dimensional coordinate system.

  The infrared camera 104 captures an object in the space of the second region, and outputs the temperature distribution in the captured space as an infrared image of a two-dimensional coordinate system with monochrome shading. In the infrared camera 104, the output value of a pixel capturing a high temperature portion is high, and the output value of a pixel capturing a low temperature portion is low. In the case of the output as digital data, the output tone value becomes higher as the pixel captures the portion with high temperature.

  The first area scanned by the rider 102 and the second area captured by the infrared camera 104 at least partially overlap. This overlapping area is a monitoring range as the monitoring system 100. It is preferable to overlap the first area and the second area as much as possible in order to eliminate waste as much as possible.

  The distance image of the rider 102 and the infrared image of the infrared camera 104 are used to grasp the three-dimensional spatial position and temperature information of the object present in the overlapping area. The scanning interval of one frame of the rider 102 and the imaging interval of the infrared camera 104 need not be completely synchronized. For example, while the rider 102 has a scanning interval of about 10 frames per second, the infrared camera 104 can capture images at an interval of a few seconds to a few thousand seconds. However, preferably, by synchronizing these, the three-dimensional position of the object can be grasped from the distance image acquired at the same time, and the temperature information of the object can be matched.

  A plurality of distance images output from the rider 102 are arranged in time series to become a moving image. Similarly, a plurality of infrared images of the infrared camera 104 become moving images by arranging them in time series. One image in a movie is called a frame.

  The control unit 120 is a computer. The control unit 120 includes a central processing unit (CPU) 121, a read only memory (ROM) 122, a random access memory (RAM) 123, a hard disk drive (HDD) 124, and the like. The CPU 121 calls a program corresponding to the processing content from the HDD 124 to control the operation of the rider 102 and the infrared camera 104, and detects the three-dimensional position of the object, displays the temperature of the object, alarm operation, temperature information, etc. . The HDD 124, together with the RAM 123, serves as a storage unit, and stores programs and data required for each process. Although the HDD 124 is used in FIG. 1, a non-volatile semiconductor memory such as a flash memory may be used instead of the HDD 124.

  The control unit 120 includes an input device 125 such as a touch panel, a button, and a mouse, and a network interface 126 (NIF: Network Interface) for connecting an external device such as a server.

  The monitoring system 100 includes a display 130 and an alarm 140. In the present embodiment, the display 130 can be provided separately from the control unit 120 in order to install the display 130 in, for example, a monitoring room of a factory. Of course, it may be integrated with the control unit 120. Also, the display 130 and the alarm device 140 may be integrated depending on the monitored environment. The alarm device 140 issues an alarm, for example, by sound, light such as a flash light or a rotating light, or any other method that can be recognized by a person. The monitoring system 100 may perform another process instead of the alarm by the alarm device 140. The other process is, for example, a process of starting recording of an image obtained from the infrared camera 104 and the rider 102. By starting the recording, the movement of the object in the overlapping area and the display of the display 130 can be reliably recorded. Further, as another process, there is, for example, stop of an automatic machine such as a robot, a machine tool, or a transport vehicle.

  In the present embodiment, the case where the rider 102 and the infrared camera 104 are integrated is illustrated. However, the rider 102 and the infrared camera 104 may be separately installed, and each may be connected to the control unit 120 via a dedicated line or a network. However, when the rider 102 and the infrared camera 104 are separated, the range scanned by the rider 102 and the range captured by the infrared camera 104 overlap.

  Further, the control unit 120 may use a general-purpose computer instead of a dedicated computer. Also, conversely, the infrared camera 104, the rider 102, and the control unit 120 may be integrated. In addition, although the control unit 120 is illustrated here as a form mainly including CPU, RAM, and ROM, for example, the control unit 120 is configured by an integrated circuit such as an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). May be

  The operation of the monitoring system 100 will be described. The operation of the monitoring system 100 is roughly divided into an initial setting operation and a monitoring operation.

  (Initial Setting Operation) The initial setting operation will be described. FIG. 2 is an explanatory view for explaining a distance image obtained by the rider 102 scanning the first area. FIG. 3 is an explanatory view for explaining an infrared image obtained by the infrared camera 104 photographing the second area. FIG. 4 is an explanatory diagram for explaining an example in which the distance image of FIG. 2 and the infrared image of FIG. 3 are simply superimposed. In FIG. 2 and FIG. 3, the same area (space) is captured.

  As shown in FIG. 2, the rider 102 does not reflect light such as sky from reflected light from the objects ob1 to ob4 such as objects or people present in the first region and the ground (in the room, floor, and so on). The same applies to the following), and a distance image Im1 based on a three-dimensional coordinate system as shown in the figure is output. The distance image Im1 is an image constituted by three-dimensional point group data, and has a three-dimensional coordinate system (X1, Y1, Z1) as shown in the figure. Therefore, each point constituting the distance image Im1 is specified in the three-dimensional coordinate system including the X, Y, and Z axes. Here, the objects ob1 to ob3 are persons, and the object ob4 is an object to be a specific temperature object described later.

  As shown in FIG. 3, the infrared camera 104 captures infrared radiation emitted from objects ob1 to ob4 such as objects and people in the second region and the ground gr and outputs an infrared image Im2 as shown in the figure. Do. In the infrared image Im2, all of the objects ob1 to ob4 are objects whose temperature is higher than the ambient temperature (here, a temperature such as the ground gr or the background). The position (coordinate value) of each pixel constituting the infrared image Im2 is specified in a two-dimensional coordinate system including an X axis and a Y axis.

  Thus, the distance image Im1 has a three-dimensional coordinate system (X1, Y1, Z1), and the infrared image Im2 has a two-dimensional coordinate system (X2, Y2). Since the two coordinate systems do not match as they are, if these are simply superimposed, as shown in FIG. 4, the objects ob1 to ob4 originally at the same position will be displaced.

  Therefore, processing is performed to make the X and Y axes of the two-dimensional coordinate system (X2, Y2) correspond to the X and Y axes of the three-dimensional coordinate system (X1, Y1, Z1). Such processing is referred to herein as coordinate conversion, and the coefficients required to correspond are referred to as coordinate conversion coefficients. The initial setting operation is an operation for calculating this coordinate conversion coefficient.

  Calculation of coordinate conversion coefficients is performed as follows. FIG. 5 is a flowchart showing a processing procedure for calculating coordinate conversion coefficients. 6 and 7 are explanatory diagrams of coordinate conversion coefficient calculation. The processing of this coordinate conversion coefficient calculation (initial setting operation) is performed by the control unit 120 executing a program for coordinate conversion coefficient calculation.

  First, the control unit 120 acquires a distance image Im1 obtained by scanning and outputting a first area including a portion where the rider 102 will be an overlapping area (S1). In the first region where the rider 102 scans, objects serving as at least two reference points are set in advance.

  Here, a heating element such as an incandescent lamp or a heater, or an infrared emitting object such as an infrared LED is attached to the head (tip) of the rod as a reference point.

  The reference point is preferably a stationary object, but as described above, the reference point may be set on a moving object as long as the scanning time of the rider 102 and the photographing time of the infrared camera 104 coincide (synchronize).

  An example of the acquired distance image is shown in FIG. As shown in the figure, in this distance image Im1, it is shown as reference points P1 and P2 on the image.

  Subsequently, the control unit 120 acquires an infrared image Im2 obtained by photographing and outputting a second area including a portion where the infrared camera 104 is to be an overlapping area (S2). An example of the acquired infrared image is shown in FIG. As shown, in the infrared image Im2, reference points PP1 and PP2 are shown. This is because the infrared radiator is attached to the tip of the rod which is the reference point installed, and this portion is a portion with high luminance (tone value) in the infrared image Im2. The processing order of S1 and S2 may be reversed (it may be simultaneous).

  Subsequently, the control unit 120 calculates a conversion coefficient for aligning the obtained two-dimensional coordinate system of the infrared image Im2 with the three-dimensional coordinate system of the distance image Im1 (S3). The distance image Im1 and the infrared image Im2 are images obtained by scanning and photographing the same area (space). For this reason, the size and distance (distance between reference points) of the real objects of the objects present in both images are the same. However, in those images, the scale differs for each image because the equipment to be scanned or photographed is different. Therefore, if the two are simply superimposed (see FIG. 4), the position of the object may be shifted or the size may be different.

  In order to align the two-dimensional coordinate system of the infrared image Im2 with the three-dimensional coordinate system of the distance image Im1, conversion may be performed so that the scale of the two coordinate systems becomes the same. Here, the scale of the X axis and the Y axis of the infrared image Im2 is adjusted to the X axis and the Y axis of the distance image Im1. Two reference points were provided for this purpose.

  Specifically, first, distances in the X-axis and Y-axis directions in the image of the reference points P1 and P2 shown in the distance image Im1 are determined. Let the distance between P1 and P2 in the X-axis direction be Δ (P1-P2) x. Similarly, the distance between P1 and P2 in the Y-axis direction is Δ (P1-P2) y.

  Similarly, distances in the X-axis and Y-axis directions in the image of the reference points PP1 and PP2 shown in the infrared image Im2 are determined. For example, the distance between PP1 and PP2 in the X-axis direction is Δ (PP1-PP2) x, and the distance between PP1 and PP2 in the Y-axis direction is Δ (PP1-PP2) y.

  Then, conversion coefficients of the X axis and the Y axis are obtained. If the conversion coefficient of the X axis is αx, then αx = Δ (P1−P2) x / Δ (PP1−PP2) x. If the conversion coefficient of the Y axis is αy, then αy = Δ (P1−P2) y / Δ (PP1−PP2) y.

  As described above, in the present embodiment, in order to change the two-dimensional coordinate system of the infrared image Im2 to the three-dimensional coordinate system of the distance image Im1, the conversion coefficients of the X axis and the Y axis are obtained. The obtained conversion coefficient is stored in the RAM 123 or the HDD 124.

  The number of reference points is not particularly limited. Further, instead of setting the reference point in the space serving as the overlapping area for coordinate conversion, an object existing in the space may be used as the reference point. For example, a corner of an object or the like is designated as a reference point so as to be easily identified in an image. However, in order to cause the reference point to be reflected in the infrared image, the reference point needs to be an infrared emitting object.

  After the conversion coefficient is determined, the process of coordinate conversion coefficient calculation (initial setting operation) ends. Thereafter, using this conversion coefficient, the coordinate value (or the entire screen) of the object in the two-dimensional coordinate system (X-Y) in the infrared image is the same coordinate system as the X-Y plane of the three-dimensional coordinate system in the distance image Can be converted to

  In addition, in this initial setting operation, the distortion of the image of the infrared camera 104 is also corrected. The infrared camera 104 uses a lens as with a normal camera. For this reason, a slight difference in refractive index between the end of the lens and the optical center of the lens inevitably causes distortion in the image. Due to such distortion, even if the same object is at the same distance, in the infrared image obtained by photographing it, the size and the position are slightly different depending on whether it is in the periphery or in the center. It will

  Therefore, such distortion in the infrared image caused by the lens is also corrected in the initial setting operation. The correction operation itself may correct the image from the refractive index of the entire lens obtained from the lens design data. In addition, using the infrared image taken so that the reference point used for the above-mentioned coordinate conversion comes to the central part of the infrared image and the infrared image taken so as to come to the peripheral part, the two infrared images are compared It may be corrected from the difference in the position of the reference point and the distance between the reference points.

  Of course, correction may not be performed by using only the lens central portion without this distortion. In this case, for example, in the infrared camera 104, adjustment is made so that the range reflected at the lens central portion where distortion does not occur is the scanning range of the rider 102 (or a large aperture lens is used as the configuration of the infrared camera itself) The infrared imaging element (bolometer) receives light only from the lens center without distortion.

  The process of this coordinate conversion coefficient calculation (initial setting operation) is performed at a predetermined time, for example, when the monitoring system 100 is installed at a site, at a time of periodic maintenance, or at any time determined by the user To do).

  In addition, although the three-dimensional coordinate system and the two-dimensional coordinate system used the orthogonal coordinate system here, you may use a polar coordinate system.

  (Monitoring Operation) The monitoring operation will be described. FIG. 8 is a flowchart showing the processing procedure of the monitoring operation by the control unit 120. In the following description, the current frame refers to a frame acquired at the current point in time, and the previous frame refers to a frame one frame before the current frame in time series. In addition, since this procedure includes repeated processing, for convenience of explanation, processing using a result of processing at a later stage may be described first.

  Here, the scanning interval of the rider 102 and the imaging interval of the infrared camera 104 are synchronized.

  First, the control unit 120 acquires a distance image of one frame at the current time from the rider 102, and similarly acquires an infrared image of one frame at the current time from the infrared camera 104 (S11). Note that whichever of the distance image and the infrared image may be acquired first (or simultaneously).

  Subsequently, the control unit 120 uses the background subtraction method to cluster the objects detected in the distance image (S12). As well known, the background subtraction method compares the image registered in advance as a background image with the image of the acquired frame (here, the image of the frame acquired in S11), and if there is a portion different from the background image, The portion is detected as a newly appeared object. The background image may store a distance image acquired by scanning in the absence of an object in the range scanned by the rider 102 (the space of the first region). The background image is stored, for example, in the HDD 124 and read out to the RAM 123 for use.

  Clustering is to facilitate tracking detected objects in subsequent processing, and known methods can be used. For example, clustering refers to the number of pixels of the detected object, the size of the object obtained from the coordinate values in the three-dimensional coordinate system, and the like (the length, area, and volume in the X, Y, Z directions in the three-dimensional coordinate system of the object) Etc.) make each object into a cluster in the range image. For each cluster, for example, the coordinate value of the cluster center and the coordinate value of the cluster outline are stored in the RAM 123 as the position.

  Subsequently, the control unit 120 performs moving object tracking on the clustered object (S13). The motion tracking searches whether or not an object in the same cluster as an object clustered in the distance image of the current frame was in the previous frame. Then, if there is an object of the same cluster in the previous frame, the position of the previous frame of the object and the position of the current frame are compared, and the moving distance, moving direction, and velocity of the object Calculated by dividing by. As a result, since the movement distance, movement direction, and speed are known for each object, these are stored in the RAM 123 for each object. Also, if there is an object detected in the current frame but not present in the previous frame, the object stores the coordinate value (position) in the RAM 123 as an object appearing in the current frame.

  Although processing using the movement distance, movement direction, and speed of the object will be described later, if these are not used, the processing of S13 may not be performed, and the three-dimensional coordinate system of the object in the current frame may be performed together with clustering. The coordinate value (position) of may be determined.

  Subsequently, the control unit 120 associates a portion where the temperature is higher than the periphery (ground, background, etc.) in the infrared image with the object detected in the distance image (S14). As described above, the object in the infrared image and the object in the distance image can be correlated by coordinate conversion.

  Specifically, the control unit 120 extracts coordinate values of a two-dimensional coordinate system of pixels occupying a portion (for example, ob1 to ob4 illustrated in FIG. 3) whose temperature is higher than the periphery in the infrared image. For example, assuming that the coordinate value of one pixel in the two-dimensional coordinate system is (x1, y1), conversion using the conversion coefficients αx and αy already obtained results in (αx × x1, αy × y1). The same conversion is performed for each of the other pixels.

  It should be noted that such image conversion is performed even if only pixels showing a portion whose temperature is higher than the periphery in the infrared image, that is, pixels whose tone values are other than 0 or which are greater than a predetermined threshold are converted. You may convert all pixels of the infrared image.

  Then, the control unit 120 associates the pixel in the coordinate value after conversion with the object in the distance image overlapping. At this time, if the range of the coordinate values of the point cloud data shown as an object in the distance image and the pixels of the coordinate values of the high temperature part after conversion overlap with each other, the temperature in the infrared image It is assumed that the high part corresponds to the object in the distance image.

  This is because, in the case of a high-temperature object, radiation heat from the object may heat the floor surface or the periphery of the object and infrared radiation may be emitted from the periphery of the object. In such a case, in the infrared image taken by the infrared camera, the high temperature object and its surroundings appear as a high temperature part. Conversely, in an infrared image obtained by photographing an object such as a person with an infrared camera, the face appears as a high temperature part, and the body appears as a temperature lower than the face. In these cases, the size of the high temperature portion in the infrared image may not match the size of the point cloud (the size of the actual object) of the object in the distance image acquired from the rider. Therefore, in the present embodiment, if there is a portion where the high temperature portion in the infrared image and the object in the distance image partially overlap, they correspond to each other. Note that there is no limitation on the overlapping ratio. For example, in the case of a person, although it depends on the degree of skin exposure from clothes, the high temperature part (such as the face) is about 1 to 20% of the whole person, so if it overlaps by 1% or more, It shall correspond.

  For each object associated, the temperature associated with the object is stored in the RAM 123 as the temperature of the object. When there is a temperature distribution in one object, for example, the temperature of the human face is higher than that of the body, and the temperature distribution of the whole person is. In such a case, the highest temperature in the temperature distribution may be stored as the temperature of the object (this stored temperature is used when displaying an image to be described later).

  Subsequently, the control unit 120 causes the display 130 to display colors on the object in accordance with the temperature and the distance based on the infrared image (S15). The display at this time is based on an infrared image (two-dimensional coordinate system), in which the position of the object obtained from the distance image (three-dimensional coordinate system) can be known. Draw a border on the part corresponding to the position and display it. The position of the object in the infrared image can be obtained by converting the coordinates of the X-Y plane of the three-dimensional coordinate system of the object in the distance image into the coordinates of the infrared image by the coordinate conversion described above. The frame line extracts the outline of the cluster in the X-Y plane of the distance image of the three-dimensional coordinate system, and displays the frame line according to the coordinate value of the extracted outline.

  The border attached to the object is the first relevant information image, and such a border makes it easier to visually recognize the presence of the object. In particular, moving objects may lose sight of their eyes after they move away from the screen because the objects are moving, but it becomes easier to understand by adding frames in this way. The first related information image is not limited to the frame line, and may be, for example, an arrow or a triangle indicating an object. In addition to the frame lines, etc., numerical values such as the distance to the object and the temperature may be displayed.

  Furthermore, at the time of this display, the displayed object is colored based on the position information and the temperature information. The position information is information obtained from the distance image of the three-dimensional coordinate system obtained from the rider 102. Here, although the above-mentioned frame line (first related information image) is one of them, the color attached to the object is changed according to the distance from the installation position of the monitoring unit 110 (that is, the installation position of the rider 102) There is.

  On the other hand, temperature information is a temperature obtained from an infrared image of a two-dimensional coordinate system, and the color attached to the object is also changed by this temperature information.

  As a result, color-coded display is made on the object based on the position information and the temperature information. Specifically, for example, blue, yellow, red, and the like are used in order from the lower temperature to the higher temperature. Furthermore, the brightness of the color is increased as the distance from the monitoring unit 110 is closer (that is, the tone value of the pixel to be displayed is increased), and the brightness is decreased as the color is farther (the tone value of the pixel to be displayed is decreased). , Etc.

  FIG. 9 is a screen example showing a display example. The color to be displayed is assumed to be each color 0 to 255 as the gradation value of (R, G, B). In the figure, a high-temperature object (object to be a specific-temperature object described later) ob4 is displayed as medium-brightness red (150, 0, 0) because the temperature is high but the distance is long. Since the objects ob1 to ob3 other than the specified temperature object ob4 are humans and the temperature is lower than the specified temperature object ob4, they are displayed in a color close to yellow and different in lightness of those colors according to their respective distances. Do. For example, the nearest object ob1 is bright yellow (224,250,0), the medium distance object ob2 is medium brightness yellow (180,190,0), the farthest object ob3 is dark yellow (100 , 120, 0). At this time, if there is a temperature distribution in the infrared image corresponding to the position (pixel) clustered in one object, that is, the same color as the entire object at the temperature stored as the temperature of the entire object described above Apply. As a result, even an object such as a person whose temperature is partially high can be easily seen and recognized on the screen. In addition, regardless of the temperature of the objects ob1 to ob4, as described above, the frame line fb indicating the object is attached and displayed. For this reason, it becomes easy to understand that it is an object, and in particular, the visibility of a moving body is improved.

  Here, a distance line (0 m to 40 m) representing the distance from the installation position of the monitoring unit 110 is also shown.

  Subsequently, the control unit 120 determines whether there is a specific temperature portion in the infrared image (S16). The specific temperature portion is a portion having a predetermined temperature range (including the case where the predetermined temperature is set or higher). For example, in the case where an object to be monitored is monitored so that a person does not approach an object at a temperature that may harm a person and alarm is issued when approaching, a predetermined temperature range which is a specific temperature portion is 50 ° C. or more (In this case, the upper limit may be, for example, the temperature at which each pixel of the infrared camera saturates). Of course, the temperature range for the specific temperature portion is arbitrary, and is determined by the temperature of the object to be monitored (for example, a high temperature object that harms a person) or the temperature of the environment (indoor or outdoor, etc.) Just do it.

  Here, if there is no specific temperature portion, the control unit 120 returns to S11 to obtain the next frame (S16: NO). Although details are not shown, if no object is detected in S12, naturally the association in S14 is not performed, a screen in which no object is present is displayed in S15, and then NO in S16 and S11 It will return and continue processing. If NO in S16, if there is data indicating that an alarm area described later has been set, it is cleared to indicate that an alarm area is not set (this will be described in S17 described later). Required in the processing of

  If the control unit 120 detects a specific temperature portion (S16: YES), the control unit 120 then determines whether an alarm area already exists (S17). Here, the presence or absence of the alarm area is determined by confirming data indicating that the alarm area has been set, which is stored when the alarm area is set in S19 described later. If the alarm area has already been set here, the process proceeds to S20. The process of S20 will be described later.

  In S17, if the alarm area does not exist (S17: NO), the control unit 120 specifies an object corresponding to the specific temperature portion detected in S16 among the objects in the distance image (three-dimensional coordinate system) as a specific temperature object It identifies as (S18). At this stage, the objects in the distance image have already been clustered and are associated with a portion whose temperature is higher than the ambient temperature. In the case of a moving object, the moving object is tracked (S12 to S14 described above). Therefore, in S18, a specific temperature portion may be searched from the infrared image, and the coordinate value (position) of the object corresponding to it may be specified.

  However, at this time, there may be a case where it can not be associated with an object which is tracked in S13 in the specific temperature portion or an object newly appeared in the current frame. For example, when the background image is acquired (stored), it is a stationary object (an object which does not move) which is low temperature but then the temperature is increased. In such a case, it is assumed that an object (an object that can not be detected by the background subtraction method) present in the background image in S13 generates heat, and from the distance image of the three-dimensional coordinate system acquired in S11, S12 The specific temperature portion may be associated with a stationary object (hereinafter simply referred to as a stationary object) that is not detected as an object in the above. At this time, the coordinate value of the stationary object is stored in the RAM 123 as the coordinate value of the specific temperature object.

  In such a case, for example, if there is a boiler (a non-moving grounding type) in the first area, the boiler is naturally captured when the background image is acquired. Therefore, in the background subtraction method, the boiler can not be detected as an object at S12. On the other hand, when the boiler which has stopped at the time of background image acquisition starts operating from a certain point in monitoring operation and becomes hot, it will be reflected in the infrared image as a specific temperature portion. In such a case, as described above, if the specific temperature portion is associated with the point cloud data indicating the stationary boiler, the stationary boiler can be recognized as the specific temperature object thereafter.

  Subsequently, the control unit 120 sets an alarm area around the specific temperature object ob4 (S19). The size of this alarm area specifies the temperature of the specific temperature part (the part of particularly high temperature if there is a temperature distribution in the specific temperature object ob4) from the infrared image that detects the specific temperature part, and The size of the alarm area is made variable.

  FIG. 10 is an explanatory view for explaining a specific temperature object. FIG. 10 shows the distance image of the three-dimensional coordinate system and the infrared image of the two-dimensional coordinate system after coordinate conversion in an overlapping manner. As shown in FIG. 10, the specific temperature object is, for example, (x min, y min, z min), (x max, y min, z min), ((x, y, z)) when represented by coordinate values (x, y, z) of the end of the shape. xmax, ymax, zmin), (xmin, ymax, zmin), (xmin, ymin, zmax), (xmax, ymin, zmax), (xmax, ymax, zmax), (xmin, ymax, zmax). Here, since the specific temperature object is a grounded object, the lower end (ymin) in the Y-axis direction is 0 (zero) if the Y-axis origin (0) of the three-dimensional coordinate system is taken to the ground (floor surface). ).

  11 and 12 are explanatory diagrams for describing an alarm area provided around a specific temperature object. Here, only the periphery of the specific temperature object in the three-dimensional coordinate system is shown. FIG. 11 shows the case of the temperature T1 of the specific temperature object, and FIG. 12 shows the case of the temperature T2 of the specific temperature object. Here, each temperature is T1 <T2. Then, a predetermined distance for setting the alarm area is made to correspond to the temperature, and the distance D1 <D2.

  When the temperature of the specific temperature object is T1, as shown in FIG. 11, the alarm area m1 is set around the specific temperature object ob4 so as to be within a predetermined distance D1 from the periphery of the specific temperature object ob4. Specifically, the alarm area m1 sets the range of the distance D1 for each of the X, Y, Z axes from the coordinate value of the outer peripheral end of the specific temperature object ob4. Therefore, when the alarm area m1 is represented by coordinate values (x, y, z), (xmin-D1, ymin-D1, zmin-D1), (xmax + D1, ymin + D1, zmin + D1), (xmax + D1, ymax + D1, zmin + D1), (xmin + D1) , Ymax + D1, zmin + D1), (xmin-D1, ymin-D1, zmax + D1), (xmax + D1, ymin-D1, zmax + D1), (xmax + D1, ymax + D1, zmax + D1), (xmin-D1, ymax + D1, zmax + D1). However, since the specific temperature object is a grounded object as described above, it is not necessary to set the alarm area in the negative direction of the Y axis if the Y axis is the origin (0). For this reason, it becomes ymin-D1 = 0 in each coordinate value.

  When the temperature of the specific temperature object ob4 is T2, as shown in FIG. 12, the alarm area m2 is set around the specific temperature object ob4 to be within a predetermined distance D2 from the periphery of the specific temperature object ob4. . Specifically, the alarm area m2 is a range of the distance D2 for each of the X, Y, Z axes from the coordinate value of the outer peripheral end of the specific temperature object ob4. Therefore, when the alarm area m2 is represented by coordinate values (x, y, z), (xmin-D2, ymin-D2, zmin-D2), (xmax + D2, ymin + D2, zmin + D2), (xmax + D2, ymax + D2, zmin + D2), (xmin + D2) , Ymax + D2, zmin + D2, (xmin-D2, ymin-D2, zmax + D2), (xmax + D2, ymin-D2, zmax + D2), (xmax + D2, ymax + D2, zmax + D2), (xmin-D2, ymax + D2, zmax + D2). However, as described above, since the specific temperature object is a grounded object, it is not necessary to set the alarm area in the negative direction of the Y axis if the Y axis is the origin (0). Therefore, ymin-D2 = 0 in each coordinate value.

  As described above, in the present embodiment, the range of the alarm area is set wider as the temperature of the specific temperature object is higher. Such a relationship between the temperature and the predetermined distance may be stored in advance in the HDD 124 as table data of temperature vs. predetermined distance, for example, and may be read out to the RAM 123 for use. Then, at the time of processing, a predetermined distance is extracted with reference to table data from the temperature of the specific temperature portion detected from the infrared image in S19. Then, the extracted alarm area separated by a predetermined distance is set.

  In S19, after setting the alarm area, data indicating that the alarm area has been set is stored in the RAM 123.

  In the above description, the case where the specific temperature object is in contact with the ground (is on the ground (including the floor surface)) is described as an example, but for example, the specific temperature object is in the air (for example, the specific temperature The alarm area is set so as to extend also to the lower surface (the negative direction of the Y axis) of the specific temperature object when the object is suspended and transported or the like). However, also in this case, if ymin-D2 becomes smaller than 0, the origin may be set to 0 if the origin of the Y axis is the ground.

  In addition, although the case where the specific temperature object is a rectangular solid in the three-dimensional coordinate system has been described as an example, the shape of the specific temperature object is not limited to a rectangular solid, and may be another shape. In that case, the alarm area may be set as a range of a predetermined distance (D1, D2, etc.) from the outer periphery of the specific temperature object in accordance with the shape of the specific temperature object.

  Also, in this case, the alarm area is set as a range of a predetermined distance (D1, D2, etc.) from the outer peripheral edge of the specific temperature object, but instead, for example, a predetermined distance from the center of the specific temperature object May be longer than the distance from the center of the specific temperature object to the outline). As a result, in the case of a sphere or a shape close to the sphere, the range from the cluster center of the clustered specific temperature object to the sphere may be set as an alarm area, so that calculation becomes easy (speeding up of processing can be achieved).

  For example, when there is a temperature distribution as a specific temperature portion, it may be a range of a predetermined distance from a position corresponding to the highest temperature portion in the specific temperature object. As a result, even if there is a temperature distribution in the specific temperature object, the alarm area can be set around the high temperature portion.

  By setting the alarm area to be away from the specific temperature object by a predetermined distance or a distance according to the temperature as described above, when the specific temperature object is a moving object, the alarm area can be moved according to the movement ( See later S20).

  In addition to the above setting method, for example, when it is known that the specific temperature object does not move (in the case of the stationary object) or the moving range is known, the alarm area is In addition, the range of a predetermined distance fixed around the specific temperature object may be used as the alarm area. When setting such a fixed alarm area, for example, if the specific temperature object is a moving object, the predetermined distance may be shorter in the direction in which it does not move and longer in the direction in which it moves.

  Return to the flowchart and continue the explanation. After setting the alarm area, the control unit 120 is displayed on the display 130 so as to display the specific temperature object ob4 and the alarm area m1 (or m2) with a frame or line or a color-coded or line-typed marking. The screen is updated (S21). At this time, a frame line indicating the alarm area is taken as a second related information image. As a result, an object (specific temperature object) in a predetermined temperature range can be easily understood visually. Therefore, even when there is another object (person or object) that approaches the specific temperature object, the distance between the specific temperature object and the other object can be easily grasped on the screen. The second related information image is not limited to the illustrated frame lines, but may be, for example, an arrow or a triangle pointing to an object, or may lighten the entire alarm area (for example, a specific temperature object has its color) It is thick enough to see through.

  Subsequently, the control unit 120 determines whether or not another object (such as a person or another object) different from the specific temperature object is in the alarm area (S22). This comparison compares the range surrounded by the coordinate value of the outline of the cluster and the coordinate value indicating the alarm area, of the object clustered in S12 (ie, the object detected by the background subtraction method). Then, if the coordinate value of the outline of the cluster of another object is within the alarm area, it is determined that the other object is within the alarm area.

  Here, if it is not determined that another object is in the alarm area (S22: NO), the control unit 120 returns to S11, acquires each image of the next frame, and continues the subsequent processing.

  On the other hand, when it is determined that another object is in the alarm area (S22: YES), the control unit 120 outputs an alarm signal to the alarm device 140 (S23). Thereby, an alarm sound is emitted from the alarm device 140 that has received the alarm signal. In addition, the display 130 blinks an object (or a frame or mark surrounding the object, etc.) determined to be within the alarm area, changes the color of the entire screen, blinks, and displays a warning message. It is good to let it be understood visually that an alarm has been issued. In addition, various alarm operations such as turning on the rotary light, changing the color of the laminated indicator from blue to red, turning on the other lamps and blinking, etc. You may go.

  Thereafter, the process returns to S12 to continue the process. After returning from S23 to S12, when the specific temperature portion disappears (S16: NO) or when another object leaves the alarm area (S22: NO), the alarm signal You may stop the Further, after the output of the alarm signal, for example, it may be kept on as long as the alarm is not turned off manually.

  The process of S20 will be described. In S20, the alarm area is already set up to the previous frame. Moreover, moving object tracking (S13) is also performed in the current frame acquired at that time. For this reason, if the specified temperature object is a moving object, its moving distance, direction, and speed are known. Therefore, in S20, the coordinate value of the alarm area that has already been set is moved using the movement distance and direction of the specific temperature object. Thus, if the alarm area has already been set up to the previous frame, even if the specific temperature object is a moving object, it is sufficient to move the alarm area in accordance with the movement. For this reason, as in S18 to S19, the calculation becomes easier than setting the alarm area from the coordinate value of the specific temperature object in the current frame (the processing can be speeded up).

  After S20, the control unit 120 proceeds to S21, updates the screen so as to display the moved alarm area and the like, and continues the subsequent processing.

  In this way, the monitoring operation is performed as an iterative process.

  In the above description, the range of the predetermined distance predetermined around the specific temperature object is set as the alarm area. Instead of this, the length of a predetermined distance for setting the alarm area may be changed according to the relative distance and relative speed between the specific temperature object and the other object.

  The movement direction and velocity of the object are obtained at the stage of S13, as described above. The specific temperature object is also known as the value of moving object tracking in S13. Further, even when the specific temperature object is a stationary object, the position is known (when the stationary object is specified as the specific temperature object in S18).

  Therefore, when the specific temperature object and another object move in the direction of relatively approaching from the movement direction and speed, and the relative speed (approaching speed) is high, it is already set. Widen the alarm area. Thus, when at least one of the specific temperature object and the other object is a moving object, it is possible to quickly issue an alarm signal in consideration of not only the temperature of the specific temperature object but also the moving speed thereof. .

  According to the first embodiment described above, the following effects can be obtained.

  In the first embodiment, the coordinate system of the infrared image of the two-dimensional coordinate system that can detect the temperature of the object and the distance image of the rider 102 that captures the three-dimensional position in space as the three-dimensional coordinate system is matched. While detecting a high temperature specific temperature portion from an infrared image, the position of a specific temperature object corresponding to the specific temperature portion is specified from the distance image. Then, since the alarm is issued from the positional relationship between the specific temperature object and the other object, the alarm for ensuring safety by ensuring that the high temperature specific temperature object approaches the other object. It can be performed.

  The specific temperature part is, for example, a part of high temperature where there is a danger to a person, and another object may warn, for example, when it is a person, in order to prevent close objects with high temperature objects. it can.

  Further, in the first embodiment, since the alarm area is set around the specific temperature object, the object enters the alarm area without calculating the distance (distance) between the specific temperature object and the other object one by one. It can be determined whether or not. For this reason, the time (calculation processing time) concerning judgment of danger can be reduced.

  Also, since the alarm area is moved according to the movement of the specific temperature object, even if the specific temperature object is moving, the danger is judged by a simple calculation of whether or not the object is in the alarm area. can do.

  Further, in the first embodiment, the size of the alarm area is changed from the direction and speed at which the object approaches the alarm area. Therefore, when the speed at which the object approaches is high, the danger is caused earlier. Informing the user can more reliably prevent the object from approaching the specific temperature object.

  Further, in the first embodiment, since the size of the alarm area is changed according to the temperature of the specific temperature portion, the danger when the object approaches the alarm area can be more reliably avoided.

Second Embodiment
(Configuration of monitoring system)
FIG. 13 is a block diagram showing the configuration of the monitoring system of the second embodiment. FIG. 14 is a bird's-eye view showing the arrangement of the monitoring unit.

  The monitoring system 200 according to the second embodiment includes two monitoring units. The two monitoring units are a first monitoring unit 211 and a second monitoring unit 212. The first monitoring unit 211 and the second monitoring unit 212 are arranged to monitor the same monitoring target space from different directions. The internal configurations of the first monitoring unit 211 and the second monitoring unit 212 are the same as in the first embodiment, and each have an infrared camera 104 and a rider 102.

  The configuration of the control unit 220 is the same as that of the first embodiment except that the first monitoring unit 211 and the second monitoring unit 212 are controlled at one time. Therefore, a first monitoring unit 211 and a second monitoring unit 212 are connected to the control unit 220. The other configuration is the same as that of the first embodiment, so the description will be omitted.

  The coordinate conversion operation and the monitoring operation by the control unit 220 are performed for each of the first monitoring unit 211 and the second monitoring unit 212, but the processing procedure is the same as that of the first embodiment, so the description will be omitted.

  In the second embodiment, the processing of S15 and S21 (including the case where an alarm is displayed on the display in S23, the same applies hereinafter) in the display processing, that is, the processing procedure of the monitoring operation described in the first embodiment. It differs from the first embodiment.

  In the second embodiment, as shown in FIG. 14, the first monitoring unit 211 and the second monitoring unit 212 monitor the same area (space) from different directions. For this reason, even if it is the same object, the visible part (surface) differs. In particular, the temperature detected by the infrared camera 104 may differ depending on the surface of the object. That is, even if one object has a high temperature on one side and a low temperature on the other side, etc. (The low temperature side means that the side on which the object is covered by the infrared shielding material. Also included).

  For example, the temperature of one surface ho of the object ob5 shown in FIG. 14 is higher than that of the other surface co. In this case, the infrared camera 104 of the first monitoring unit 211 shoots the surface ho with high temperature, and the infrared camera 104 of the second monitoring unit 212 shoots the surface co with low temperature. On the other hand, each rider 102 scans the same object ob5 from each position and outputs a distance image.

  In such a case, when the same object is displayed as a low-temperature object, displaying as a high-temperature object makes it easier to understand the inherent temperature of the object on the screen. Therefore, in the second embodiment, the image of the monitoring unit (the first monitoring unit 211 in FIG. 14) of the person who is capturing the surface ho having a high temperature is displayed on the display 130 at the display processing stage.

  To this end, in the second embodiment, the control unit 220 performs processing different from that of the first embodiment in S15 and S21 (S23) which is the display processing stage. FIG. 15 is a subroutine flowchart showing the procedure of the display processing step (S15 and S21 (S23) in FIG. 8) in the second embodiment.

  First, as in the first embodiment, the control unit 220 uses the distance image and the infrared image acquired from the first monitoring unit 211, and the distance image and the infrared image acquired from the second monitoring unit 212 in FIG. The processing from S11 shown is performed in parallel.

  Then, when the process proceeds to S15 or S21 (or S23), the process proceeds to the subroutine shown in FIG. 15, and the control unit 220 extracts the maximum temperature in the infrared image captured by the infrared camera 104 of the first monitoring unit 211 This is set as the first screen temperature St1 (S31).

  Subsequently, the control unit 220 extracts the highest temperature in the infrared image captured by the infrared camera 104 of the second monitoring unit 212, and sets this as the second screen temperature St2 (S32). The order of the processes of S31 and S32 may be reversed (or may be simultaneous).

  Subsequently, the control unit 220 causes the screen with the highest maximum temperature to be displayed. In the processing here, it is determined whether or not St1StSt2 (S33).

  Here, if St1 ≧ St2 (S33: YES), the control unit 220 causes the display 130 to display the image on the first monitoring unit 211 side. The image to be displayed is displayed using the distance image and the infrared image acquired from the first monitoring unit 211, with colors and frames attached to the object, as described in the first embodiment.

  On the other hand, if St1 ≧ St2 (S33: NO), the control unit 220 causes the display 130 to display the image on the second monitoring unit 212 side. The image to be displayed is displayed by attaching a color or a frame to the object as described in the first embodiment, using the distance image and the infrared image acquired from the second monitoring unit 212. Thus, the subroutine in the second embodiment is completed, and the process returns to the main routine (flowcharts shown in FIGS. 8 and 9).

  Note that the distance image and the infrared image acquired from the first monitoring unit 211 also in the detection of the specific temperature object, the setting of the alarm area, the detection of the presence or absence of other objects in the alarm area, and each step of alarm (S18 to 23) , And each of the distance image and the infrared image acquired from the second monitoring unit 212 is performed. At this time, even if the alarm area is set only in one of the first monitoring unit 211 and the second monitoring unit 212, the monitoring operation is performed on the alarm area to perform an alarm or the like. Just do it.

  According to the second embodiment, in addition to the effects of the first embodiment, the following effects can be obtained.

  In the second embodiment, when the monitoring operation is performed using two or more monitoring units, it is possible to display an image of the monitoring unit that is capturing a portion with a higher temperature. For example, in the case of a person, it is possible to display on the display 130 an image (usually, the temperature of the face is higher than that of the back of the head) captured in which the face is facing.

  Although two monitoring units are provided in the second embodiment, more monitoring units may be provided. In the second embodiment, one control unit 220 controls two monitoring units, but a control unit 220 is provided for each of two monitoring units to switch the screen display. A control unit (computer) dedicated to screen switching may be further provided.

  In the second embodiment, the detection of the specific temperature object, the setting of the alarm area, the detection of the presence or absence of other objects in the alarm area, and the stages of alarm (S18 to 23) From each infrared image of the monitoring unit 212, it may be determined whether or not there is a specific temperature portion first, and thereafter, the process may be executed using the distance image and the infrared image of the one detecting the specific temperature portion. .

  Further, in the second embodiment, an image displayed by color-coding based on distance and temperature may be simply provided as an image in which a high temperature surface can be recognized. In this case, the detection of the specific temperature object, the setting of the alarm area, the detection of the presence or absence of another object in the alarm area, and the steps of the alarm (S18 to 23) may not be performed.

  Although the embodiments to which the present invention is applied have been described above, the present invention is not limited to these embodiments.

  For example, in the embodiment described above, the case where one specific temperature portion is detected is described as an example, but when a plurality of specific temperature portions are detected, identification of a specific temperature object, setting of an alarm area are made to correspond to them. It is better to monitor and alert.

  Further, for example, in the above-described embodiment, the coordinate conversion coefficient for converting the two-dimensional coordinate system of the infrared image to the three-dimensional coordinate system of the distance image is determined as the initial setting. The angle of view of the camera and the angle of view of the distance image obtained by the scan of the rider may be the same. For example, the angle of view of the infrared camera is adjusted to the angle of view of the distance image by changing the focal length (or magnification) of the lens of the infrared camera. Also in this case, the two-dimensional coordinate system of the infrared image and the coordinate system of the XY plane in the three-dimensional image of the distance image can be made the same.

  Further, for example, in the above-described embodiment, the alarm area is set around the specific temperature object, but after detection of the specific temperature object, the distance between the specific temperature object and the other object is calculated one by one, and The alarm signal may be output by determining whether the distance is equal to or less than a predetermined distance.

  Besides, the present invention can be variously modified based on the configuration described in the claims, and they are also within the scope of the present invention.

  This application is based on Japanese Patent Application No. 2018-029921 filed on February 22, 2018, the disclosure of which is incorporated in its entirety by reference.

100, 200 monitoring system,
102 riders,
104 infrared camera,
110, 211, 212 monitoring unit,
120, 220 control units,
130 display,
140 alarms,
211 first monitoring unit,
212 Second monitoring unit.

Claims (14)

A lidar for outputting a distance image in which a distribution of distance values obtained by scanning a laser beam toward the first region is represented in a three-dimensional coordinate system;
An infrared camera which captures a second area in which at least a part of the first area overlaps with the first area and outputs an infrared image shown in a two-dimensional coordinate system;
Acquiring the distance image from the rider, detecting an object from the acquired distance image, and acquiring the infrared image from the infrared camera;
In the infrared image, when there is a specific temperature portion in a predetermined temperature range, an object corresponding to the specific temperature portion among the detected objects is specified as a specific temperature object in the three-dimensional coordinate system. And when the detected object includes another object different from the specific temperature object, it is determined whether or not the distance between the specific temperature object and the other object is within a predetermined distance. A control unit that outputs an alarm signal when the distance between the specific temperature object and the other object is within a predetermined distance;
Monitoring system with. The control unit
The range of the predetermined distance from the specific temperature object is set as an alarm area around the specific temperature object in the three-dimensional coordinate system, and the alarm signal is output when the other object is in the alarm area. The monitoring system according to claim 1.   The control unit moves the alarm area in accordance with the movement of the specific temperature object when it is determined that the specific temperature object is a moving object from a plurality of distance images acquired in time series from the rider. The monitoring system according to claim 2.   The monitoring system according to claim 2, wherein the control unit sets the alarm area fixed around the specific temperature object. The control unit
The relative distance and relative velocity between the specific temperature object and the other object are determined from the plurality of distance images acquired in time series from the rider, and the determined relative distance and relative distance between the specific temperature object and the other object The monitoring system according to any one of claims 1 to 4, wherein the length of the predetermined distance is changed according to the speed.   The monitoring system according to any one of claims 1 to 5, wherein the control unit lengthens the predetermined distance as the temperature of the specific temperature portion in the infrared image captured by the infrared camera increases.   7. A surveillance system according to any one of the preceding claims, comprising an alarm which receives the alarm signal and emits sound and / or light. A lidar that outputs a distance image of a distribution of distance values obtained by scanning laser light toward the first region in a three-dimensional coordinate system;
A control method of a monitoring system, comprising: an infrared camera for photographing a second area at least partially overlapping the first area and outputting an infrared image represented by a two-dimensional coordinate system,
Acquiring the distance image from the rider and detecting an object from the acquired distance image (a);
When the infrared image is acquired from the infrared camera, and the infrared image includes a specific temperature portion having a predetermined temperature range, an object corresponding to the specific temperature portion among the detected objects is a specific temperature object Identifying the position in the three-dimensional coordinate system as (b),
If the detected object includes another object different from the specific temperature object, it is determined whether the distance between the specific temperature object and the other object is within a predetermined distance, Issuing an alarm if the distance between the specified temperature object and the other object is within a predetermined distance (c);
A control method of a monitoring system having: In the step (c),
A range of the predetermined distance from the specific temperature object is set as an alarm area around the specific temperature object in the three-dimensional coordinate system, and an alarm is issued when the other object is in the alarm area. The control method of the monitoring system as described in.   The alarm region is moved according to the movement of the specific temperature object when it is determined that the specific temperature object is a moving object from a plurality of distance images acquired in time series from the rider. Control method of the monitoring system described.   The control method of the monitoring system according to claim 9, wherein the alarm area is fixedly set around the specific temperature object.   The relative distance and relative velocity between the specific temperature object and the other object are determined from the plurality of distance images acquired in time series from the rider, and the determined relative distance and relative distance between the specific temperature object and the other object The control method of the monitoring system according to any one of claims 8 to 11, wherein the length of the predetermined distance is changed according to the speed.   The control method of the monitoring system according to any one of claims 8 to 12, wherein the size of the predetermined distance is changed according to the temperature of the specific temperature portion in the image captured by the infrared camera. The control method of the monitoring system according to any one of claims 8 to 13, wherein the alarm is sound and / or light.