JP2018124808A - Image processor device and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a low-cost space-saving system for monitoring a monitoring object by a clear camera image combining an infrared image and a visible image for use.SOLUTION: A processor 102 of an image processor captures not only a visible image photographed by a visible camera, but also an infrared image photographed by an infrared camera for detecting infrared emitted from an object within an imaging area by the visible camera. Regarding the infrared image, an area whose amount of infrared is a specified quantity or more is extracted. Then, the visible image is synthesized with the extracted infrared image and an obtained synthesized image is output on a display device.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an image processing apparatus and an image processing program capable of detecting an abnormality in an imaging region using a camera image.

  A system for capturing an imaging region including a monitoring target with an infrared camera called an infrared thermography and detecting a fire from an infrared image captured with the infrared camera is already known (for example, Patent Documents). 1).

  In addition, there is already known a system in which when a fire is detected from an infrared image, a visible camera is swung to a fire detection position so that a supervisor can grasp the situation of the fire from the visible image taken by the visible camera (for example, , See Patent Document 2).

JP 2009-100198 A JP 2013-196655 A

  Infrared cameras usually detect infrared rays emitted from objects in the imaging area, and visually display the distribution of the amount of infrared rays by the difference in color. Therefore, for example, when a system for monitoring an object to be monitored by using both an infrared image and a visible image is used, a display device for displaying an infrared image is displayed even if both images are superimposed and displayed. And a display device for displaying a visible image are generally prepared. For this reason, there are problems that the equipment cost is high and the installation space for the display device is also required.

  In addition, using an infrared camera, it is possible to visually detect abnormalities in heat like a fire, but it is not possible to visually detect abnormalities such as sound and light emitted from objects together with camera images. .

  The present invention has been made paying attention to the above circumstances, and a first object is to realize a low-cost and space-saving system for monitoring an object to be monitored with a clear camera image using both an infrared image and a visible image. An object of the present invention is to provide an image processing apparatus that can perform such processing.

  A second object of the present invention is to provide an image processing apparatus that can visually detect abnormalities in sensing data of a monitoring target in a system that monitors the monitoring target with a camera image.

  In order to solve the above problems and achieve the object, an image processing apparatus according to a first aspect of the present invention includes a visible image acquisition unit that captures a visible image captured by a visible camera, and an object in an imaging region by the visible camera. Infrared image acquisition means for capturing an infrared image taken by an infrared camera that detects infrared rays emitted from the camera. In addition, an extraction unit that extracts an area in which the amount of infrared rays is equal to or greater than a predetermined amount from the infrared image acquired by the infrared image acquisition unit is provided. The synthesizing unit synthesizes the infrared image extracted by the extracting unit with the visible image acquired by the visible image acquiring unit, and the output unit outputs the synthesized image obtained by the synthesizing unit to the display device. It is a thing.

  In order to solve the above problems and achieve the object, an image processing device according to a second aspect of the present invention is a data acquisition device that further captures sensing data in an imaging region into the image processing device according to the first aspect. Means, an abnormality detection means for detecting an abnormality in the sensing data acquired by the data acquisition means, and a warning image generation means for generating a warning image in response to detection of an abnormality in the sensing data by the abnormality detection means. The synthesizing means further synthesizes the warning image.

  In order to solve the above problems and achieve the object, an image processing apparatus according to a third aspect of the present invention includes the visible image acquisition means, the data acquisition means, the abnormality detection means, and the warning image generation means. Prepare. The synthesizing unit synthesizes the warning image generated by the warning image generating unit with the visible image acquired by the visible image acquiring unit, and the output unit outputs the synthesized image obtained by the synthesizing unit to the display device. It is a thing.

  In order to solve the above problems and achieve the object, an image processing apparatus according to a fourth aspect of the present invention includes the infrared image acquisition means, the data acquisition means, the abnormality detection means, and the warning image generation means. Prepare. The synthesizing unit synthesizes the warning image generated by the warning image generating unit with the infrared image acquired by the infrared image acquiring unit, and the output unit outputs the synthesized image obtained by the synthesizing unit to the display device. It is a thing.

  In the image processing device according to the fifth aspect of the present invention, in any one of the image processing devices according to the second to fourth aspects, the warning image generating means displays different warning images according to the degree of abnormality of the sensing data. It is made to generate.

  According to a sixth aspect of the present invention, a computer is caused to function as the visible image acquisition unit, the infrared image acquisition unit, and the extraction unit, and the visible image acquired by the visible image acquisition unit is extracted by the extraction unit. An image processing program for functioning as a combining unit that combines the extracted infrared images and an output unit that outputs the combined image obtained by the combining unit to a display device.

  According to a seventh aspect of the present invention, the computer functions as the visible image acquisition means, the data acquisition means, the abnormality detection means, and the warning image generation means, and the visible image acquired by the visible image acquisition means is displayed. And an image processing program for causing a warning image generated by the warning image generating means to be combined, and an output means for outputting the combined image obtained by the combining means to the display device.

  According to an eighth aspect of the present invention, a computer functions as the infrared image acquisition unit, the data acquisition unit, the abnormality detection unit, and the warning image generation unit, and the infrared image acquired by the infrared image acquisition unit is displayed. And an image processing program for causing a warning image generated by the warning image generating means to be combined, and an output means for outputting the combined image obtained by the combining means to the display device.

  According to the first aspect, even if the infrared image is superimposed on the visible image, the infrared image is limited to a region where the amount of infrared rays is equal to or greater than the predetermined amount, and thus does not become an unclear image. As a result, a system for monitoring a monitoring target with a clear image using both an infrared image and a visible image can be realized at low cost and in a small space.

  According to the second aspect, the sensing data abnormality to be monitored can be visually grasped from the image obtained by superimposing the visible image and the infrared image, and a more reliable monitoring system can be realized at low cost and in a small space. Can be made.

  According to the third aspect, since the warning image that warns the abnormal sensing data of the monitoring target displayed in the visible image is displayed on the visible image, the abnormal sensing data of the monitoring target is visually captured. And a highly reliable monitoring system can be realized at low cost and in a small space.

  According to the fourth aspect, since the warning image that warns the sensing data abnormality of the monitoring target displayed in the infrared image is displayed superimposed on the infrared image, the sensing data abnormality of the monitoring target is visually captured. And a highly reliable monitoring system can be realized at low cost and in a small space.

  According to the fifth aspect, it is possible to visually grasp the degree of abnormality in sensing data, and it is possible to realize a more reliable monitoring system with low cost and space saving.

  According to the sixth aspect, an infrared image is superimposed on a visible image and displayed, thereby realizing a low-cost and space-saving system for monitoring a monitoring target with a clear image using both an infrared image and a visible image. It is possible to provide an image processing program that can configure an image processing apparatus that can be configured by a computer.

  According to the seventh aspect, it is possible to reduce the cost of a highly reliable monitoring system by displaying a warning image on the visible image to warn of abnormalities in the sensing data of the monitoring target displayed on the visible image. An image processing program can be provided in which an image processing apparatus that can be realized in space can be configured by a computer.

  According to the eighth aspect, a more reliable monitoring system can be obtained at a low cost by displaying an infrared image superimposed on a warning image that warns of abnormal sensing data of a monitoring target displayed in the infrared image. An image processing program can be provided in which an image processing apparatus that can be realized in a space-saving manner can be configured by a computer.

1 is a schematic diagram of a monitoring system including an image processing apparatus according to an embodiment. FIG. 2 is a block diagram showing a functional configuration of a processor in FIG. 1 and a data configuration of a storage device necessary for supporting the realization of the function. The schematic diagram which shows an example of the arrangement | positioning relationship between a monitoring target, a visible camera, an infrared camera, and a microphone. The schematic diagram which shows the correspondence of the imaging area of a visible camera, and the infrared rays detection area of an infrared camera. The flowchart which shows an example of the control procedure which a processor performs. The figure which shows the example of a display of the image with which the infrared image of the area | region where the amount of infrared rays more than a threshold value was detected was combined with the visible image. The figure which shows the example of a display of the image with which the infrared image and the 1st warning image of the area | region where the amount of infrared rays more than a threshold value were detected to the visible image were synthesize | combined. The figure which shows the example of a display of the image with which the infrared image of the area | region where the infrared rays amount more than a threshold value was detected with the visible image, and the 2nd warning image were synthesize | combined. The flowchart which shows the other example of the control procedure which a processor performs.

  Hereinafter, an embodiment according to the present invention will be described with reference to the drawings. In the drawings and the description below, the same elements are denoted by the same reference numerals.

(Constitution)
FIG. 1 is a schematic diagram of a monitoring system 1 including an image processing apparatus 10 according to the present embodiment. The monitoring system 1 includes an image processing device 10, a visible camera 11, an infrared camera 12, a microphone 13, an image encoder 14 of the visible camera 11, an image encoder 15 of the infrared camera 12, an audio encoder 16, and a display device. 17.

  The visible camera 11 is disposed at a position where the monitoring target 2 that is an object can be photographed. The visible camera 11 repeatedly captures an area including the monitoring target 2 as the imaging area A1, and outputs a captured frame image, a so-called visible image, to the image encoder 14 each time. The image encoder 14 encodes and compresses the data of the visible image and then transmits the data to the image processing apparatus 10. The visible camera 11 may be, for example, a general video camera for video recording, a digital video camera, or the like.

  The infrared camera 12 is disposed at a position where infrared rays emitted from an object including the monitoring target 2 can be detected. The infrared camera 12 captures an area including the monitoring target 2 as an infrared detection area A <b> 2 and outputs the captured frame image, so-called infrared image, to the image encoder 15. The image encoder 15 encodes and compresses infrared image data and then transmits the data to the image processing apparatus 10. This type of infrared camera 12 is referred to as infrared thermography or the like.

  The microphone 13 is disposed at a position where sound generated in the monitoring target 2 can be detected. The microphone 13 has directivity toward the monitoring target 2, and outputs collected sound, so-called sensing data, to the sound encoder 16. The audio encoder 16 encodes and compresses audio data, and transmits the encoded data to the image processing apparatus 10.

  The image processing apparatus 10 includes a communication interface 101, a processor 102, a storage device 103, and the like. The image encoders 14 and 15 and the audio encoder 16 are connected to the communication interface 101. That is, the visible image data encoded and compressed by the image encoder 14 is taken into the image processing apparatus 10 via the communication interface 101. Similarly, infrared image data encoded and compressed by the image encoder 15 is also taken into the image processing apparatus 10 via the communication interface 101, and audio data encoded and compressed by the audio encoder 16 is also transmitted to the communication interface 101. Via the image processing apparatus 10.

  The processor 102 and the storage device 103 constitute a computer of the image processing apparatus 10. The processor 102 corresponds to the central part of the computer. The processor 102 controls each unit in order to realize the function as the image processing apparatus 10 according to the operating system and the application program.

  The storage device 103 includes a main memory corresponding to the main storage portion of the computer and an auxiliary storage device corresponding to the auxiliary storage portion. The main memory has a nonvolatile memory area and a volatile memory area. The nonvolatile memory area is an area for storing fixed data such as an operating system and application programs. The volatile memory area is used as a work area where data is appropriately rewritten by the processor 102. The auxiliary storage device is, for example, an EEPROM (Electric Erasable Programmable Read-Only Memory), an HDD (Hard Disc Drive), or an SSD (Solid State Drive). The auxiliary storage device stores data used when the processor 102 performs various processes and data generated by the process in the processor 102. The auxiliary storage device may store the above application program.

  The application program includes an image processing program. In accordance with this image processing program, the processor 102 captures each image data of the visible image, infrared image, and sound received by the communication interface 101 and performs image processing. Then, the processor 102 causes the display device 17 to display an image obtained by this image processing.

The display device 17 is typically a liquid crystal display, an organic EL display, or the like. However, the type of the display device 17 is not particularly limited. In short, an image generated by the image processing apparatus 10 may be displayed.
As the image processing apparatus 10 and the display device 17 described above, a general personal computer system can be used.

  FIG. 2 is a block diagram showing a functional configuration of the processor 102 and a data configuration of the storage device 103 necessary for supporting the realization of the function. As illustrated, the processor 102 includes a visible image reading unit 21 and a visible image trimming unit 22 as a visible image processing function. The processor 102 includes an infrared image reading unit 23, an infrared image trimming unit 24, and an abnormal temperature image extracting unit 25 as infrared image processing functions. The processor 102 also includes an audio reading unit 26, a comparison unit 27, and a warning image generation unit 28 as audio processing functions. Further, the processor 102 has functions as the image composition unit 29 and the display control unit 30.

  On the other hand, the storage device 103 forms a visible image memory 31 and an area setting memory 32 as memory areas that support the processing of visible images. In addition, the storage device 103 forms an infrared image memory 33, an area setting memory 34, and an abnormal temperature threshold memory 35 as memory areas that support infrared image processing. In addition, the storage device 103 forms an audio memory 36, a comparison value memory 37, and a warning image memory 38 as memory areas that support audio processing. Further, the storage device 103 forms a composite image memory 39 and an image storage memory 40.

  The visible image memory 31 stores the data of the visible image captured via the communication interface 101 in time series. The area setting memory 32 stores in advance trimming data for specifying a visible image trimming area.

  The visible image reading unit 21 sequentially reads the data of the visible image stored in the visible image memory 31 and gives it to the visible image trimming unit 22. The visible image trimming unit 22 trims the visible image based on the trimming data stored in the region setting memory 32 and then gives the image to the image composition unit 29.

  In the infrared image memory 33, the data of the infrared image captured via the communication interface 101 is stored in time series. In the area setting memory 34, trimming data for designating a trimming area of the infrared image is stored in advance. In the abnormal temperature threshold value memory 35, a lower limit value of the amount of infrared rays emitted from an object having a predetermined temperature or higher is stored in advance as a threshold value.

  The infrared image reading unit 23 sequentially reads infrared image data stored in the infrared image memory 33 and supplies the infrared image data to the infrared image trimming unit 24. The infrared image trimming unit 24 trims the infrared image based on the trimming data stored in the area setting memory 34 and then supplies the trimmed image to the abnormal temperature image extracting unit 25. The abnormal temperature image extraction unit 25 extracts an area in which an infrared amount equal to or greater than the threshold stored in the abnormal temperature threshold memory 35 is detected from the trimmed infrared image. Then, the abnormal temperature image extraction unit 25 gives the extracted infrared image to the image composition unit 29.

  The audio memory 36 stores audio data captured via the communication interface 101 in time series. In the comparison value memory 37, two types of values representing the sound volume are stored as a first comparison value L1 and a second comparison value L2, respectively. The second comparison value L2 represents a volume that is larger than the first comparison value L1. A warning image prepared in advance is stored in the warning image memory 38. The warning image includes a first warning image when the sound volume is equal to or higher than the first comparison value L1 and lower than the second comparison value L2, and a second warning image when the sound volume is equal to or higher than the second comparison value L2. The warning image memory 38 also stores coordinate data for specifying the position where the warning image is displayed.

  The voice reading unit 26 reads voice data from the voice memory 36 and supplies the data to the comparison unit 27. The comparison unit 27 compares the sound volume with two kinds of comparison values L1 and L2 set in the comparison value memory 37, and notifies the warning image generation unit 28 of comparison result data indicating the result. The comparison result data includes data less than L1 when the volume is less than the first comparison value L1, data between L1L2 that is greater than or equal to the first comparison value L1 and less than the second comparison value L2, and L2 that is greater than or equal to the second comparison value L2. There are three patterns with data.

  When the data from the comparison unit 27 is the data between L1L2, the warning image generation unit 28 reads the first warning image from the warning image memory 38 and outputs it to the image composition unit 29 together with the coordinate data. When the data from the comparison unit 27 is L2 or more data, the warning image generation unit 28 reads the second warning image from the warning image memory 38 and outputs it to the image composition unit 29 together with the coordinate data. Note that the warning image generation unit 28 does not output the warning image to the image composition unit 29 when the data from the comparison unit 27 is data less than L1.

  The image composition unit 29 synthesizes the infrared image extracted by the abnormal temperature image extraction unit 25 with the visible image trimmed by the visible image trimming unit 22. Furthermore, when a warning image is input from the warning image generation unit 28, the image synthesis unit 29 synthesizes the warning image at a position indicated by the coordinate data. Then, the image composition unit 29 writes the composite image data in the composite image memory 39.

  Each time composite image data is written in the composite image memory 39, the display control unit 30 reads the composite image data and sends it to the display device 17. With this process, the composite image is displayed on the display device 17. Further, the display control unit 30 transfers the composite image data to the image storage memory 40. By this processing, the composite image data is accumulated and stored in the image storage memory 40.

Here, in the processor 102, the visible image reading unit 21 constitutes a visible image acquisition unit, the infrared image reading unit 23 constitutes an infrared image acquisition unit, the abnormal temperature image extraction unit 25 constitutes an extraction unit, The image composition unit 29 constitutes composition means, and the display control unit 30 constitutes output means.
The voice reading unit 26 constitutes a data acquisition unit, the comparison unit 27 constitutes an abnormality detection unit, and the warning image generation unit 28 constitutes a warning image generation unit.

Here, the trimming data of the visible image and the infrared image and the coordinate data of the warning image will be described with reference to FIGS.
FIG. 3 is a schematic diagram illustrating an example of an arrangement relationship among the monitoring target 2, the visible camera 11, the infrared camera 12, and the microphone 13. FIG. 4 is a schematic diagram illustrating a correspondence relationship between the imaging region A1 of the visible camera 11 and the infrared detection region A2 of the infrared camera 12.

As shown in FIG. 3, the monitoring object 2 assumes a conveyor as one embodiment. Note that the monitoring target 2 is arbitrary and is not particularly limited.
The visible camera 11 is installed at a position where the conveyor that is the monitoring target 2 is included in the imaging region A1. Similarly, the infrared camera 12 is installed at a position where the conveyor that is the monitoring target 2 is included in the infrared detection area A2.

  However, the imaging area A1 of the visible camera 11 and the infrared detection area A2 of the infrared camera 12 do not necessarily match. In the first place, the frame size of the imaging area A1 and the frame size of the infrared detection area A2 are often different. In addition, it is inevitable that the imaging area A1 and the infrared detection area A2 are displaced due to the displacement of the arrangement positions of the visible camera 11 and the infrared camera 12.

  Therefore, in this embodiment, the visible image and the infrared image are each trimmed so as to leave a region A3 indicated by hatching in FIG. 4, that is, a region where the imaging region A1 and the infrared detection region A2 overlap. Therefore, trimming data is set in the region setting memory 32 on the visible image side so as to cut out the portion of the imaging region A1 that protrudes from the region A3. In the area setting memory 34 on the infrared image side, trimming data is set so as to cut off the portion of the infrared detection area A2 that protrudes from the area A3.

  On the other hand, in this embodiment, as shown in FIG. 3, the microphone 13 is attached to a box in which a motor that is a drive source of the conveyor is accommodated. That is, the microphone 13 is attached at a position where the sound of the motor can be collected. As shown in FIG. 4, in the area A3, the two-dimensional coordinate P (x, y) corresponding to the position of the microphone 13 is used as the coordinate data of the warning image.

(Operation)
FIG. 5 is a flowchart illustrating an example of a control procedure executed by the processor 102. Hereinafter, the operation of the image processing apparatus 10 will be described with reference to FIG. The control procedure shown in FIG. 5 is an example, and the control procedure and the content of the control are not particularly limited as long as similar results can be obtained.

  First, the processor 102 determines whether or not it is a data read timing in step S1. The processor 102 has a built-in clock oscillation circuit and generates an internal clock based on this clock signal to perform timing control. If it is determined that it is the data read timing based on the internal clock (YES in S1), the processor 102 operates the visible image reading unit 21, the infrared image reading unit 23, and the audio reading unit 26 in step S2. With this control, a visible image is read from the visible image memory 31, an infrared image is read from the infrared image memory 33, and audio data is read from the audio memory 36.

  The processor 102 waits for a visible image to be read in step S3, an infrared image to be read in step S4, or audio data to be read in step S5. If the visible image is read by visible image reading unit 21 (YES in S3), processor 102 operates visible image trimming unit 22 as step S7. By this control, a portion other than the imaging region A1 to the region A3 of the visible image is trimmed.

  If the infrared image is read by infrared image reading unit 23 (YES in S4), processor 102 operates infrared image trimming unit 24 as step S8. By this control, a portion other than the region A3 from the infrared detection region A2 of the infrared image is trimmed, and the processor 102 subsequently operates the abnormal temperature image extraction unit 25 as steps S9 and S10. With this control, it is determined whether or not the trimmed infrared image has a region where an infrared amount equal to or greater than the threshold is detected (S9). If there is a region where the amount of infrared rays equal to or greater than the threshold is detected (YES in S9), the region where the amount of infrared rays equal to or greater than the threshold is detected is extracted from the infrared image (S10). If there is no region where the amount of infrared rays equal to or greater than the threshold is detected (NO in S9), the infrared image is not extracted.

  When the audio data is read by the audio reading unit 26 (YES in S6), the processor 102 operates the comparison unit 27 and the warning image generation unit 28 as steps S11 to S15. By this control, the volume of the audio data is compared with the first comparison value L1 and the second comparison value L2 (S11). If the volume of the audio data is less than first comparison value L1 (YES in S12), no warning image is generated. On the other hand, when the volume is equal to or higher than first comparison value L1 and lower than second comparison value L2 (YES in S13), a first warning image is generated (S14). If the volume is equal to or higher than second comparison value L2 (NO in S13), a second warning image is generated (S15).

  Thereafter, the processor 102 operates the image composition unit 29 as step S16. By this control, the visible image trimmed by the control of step S7 is added to the infrared image extracted by the control of step S10 and the warning image (the first warning image or the second warning image generated by the control of step S14 or step S15). Warning image). However, if it is determined in step S9 that there is no region where the amount of infrared rays equal to or greater than the threshold is detected, the infrared image is not synthesized. If it is determined in step S12 that the volume of the audio data is less than the first comparison value L1, no warning image is synthesized.

  The composite image generated by the control in step S16 is stored in the composite image memory 39. Therefore, the processor 102 operates the display control unit 30 as steps S17 and S18. By this control, the composite image data stored in the composite image memory 39 is output to the display device (S17). The composite image data is transferred to the image storage memory 40 and stored and stored in the image storage memory 40 (S18).

  When the above control routine is completed, the processor 102 returns to the control in step S1. Therefore, at the next data read timing, the processor 102 executes the control of steps S2 to S18 again.

  FIG. 6 is a display example of an image obtained by combining the visible image G1 with the infrared image G2 in the region where the amount of infrared rays equal to or greater than the threshold is detected. As shown in the drawing, the region where the infrared image G2 is combined with the visible image G1 is limited to the region where the amount of infrared rays equal to or greater than the threshold is detected. Therefore, the monitor can easily identify the region where the infrared ray amount equal to or greater than the threshold value is detected, that is, the region where the temperature is higher than the abnormal temperature, without the visible image G1 becoming unclear.

  FIG. 7 is a display example of an image obtained by combining the visible image G1 with the infrared image G2 and the first warning image G3 in the region where the amount of infrared rays equal to or greater than the threshold is detected. As shown in the figure, the region where the first warning image G3 is combined is in the vicinity of the portion where the microphone 13 is attached. Therefore, the monitor can easily identify the part where the sound having the volume equal to or higher than the first comparison value L1 is generated without the visible image G1 becoming unclear. In this example, since the sound collected by the microphone 13 is considered to be the sound of the motor, the monitor is responsible for the reason that the region specified by the infrared image G2 is at a temperature higher than the abnormal temperature. It can be assumed that it is abnormal.

  FIG. 8 is a display example of an image obtained by combining the visible image G1 with the infrared image G2 and the second warning image G4 in a region where the amount of infrared rays equal to or greater than the threshold is detected. For example, by confirming that the composite image has changed from the state of FIG. 7 to the state of FIG. 8, the monitor increases the volume of the sound collected by the microphone 13 to the second comparison value L2 or more. Can be easily confirmed.

  As described above, according to the present embodiment, an infrared image and a visible image can be used together without preparing a display device for displaying a visible image and a display device for displaying an infrared image. Thus, it is possible to realize a system that can display on a single display device and monitor a monitoring target with a clear camera image. Accordingly, since a plurality of pieces of information other than the video can be visualized as a video, it becomes easier to grasp the abnormal situation, the equipment cost can be saved, and the installation space for the display device can also be saved. Specifically, it is possible to visually grasp temperature abnormalities by infrared images and sound abnormalities as sensing data to be monitored.

(Modification)
In the above embodiment, the threshold set in the abnormal temperature threshold memory 35 is the lower limit of the amount of infrared rays emitted from an object having a predetermined temperature or higher, but the threshold is not limited to this. For example, a lower limit value and an upper limit value of the amount of infrared rays emitted from an object in a predetermined temperature range may be set as threshold values. By doing so, it is possible to monitor whether or not the temperature of the monitoring target 2 is within a predetermined temperature range by using the infrared image combined with the visible image.

  In the embodiment, the case where all the synthesized images synthesized by the image synthesizing unit 29 are accumulated and saved in the image saving memory 40 has been exemplified. However, in that case, even the visible image in which the infrared image and the warning image are not combined is accumulated, so that the storage capacity of the image storage memory 40 becomes excessive. Therefore, only when the infrared image or the warning image is combined, the combined image may be stored and stored in the image storage memory 40.

  Such an embodiment can be realized by modifying the control procedure executed by the processor 102 as shown in FIG. That is, if it is determined in step S9 whether or not there is a region in which the amount of infrared rays equal to or greater than the threshold is detected in the infrared image (YES in S9), the processor In step S21, the storage flag is set. In step S11, as a result of comparing the volume of the audio data with the first comparison value L1 and the second comparison value L2, the volume of the audio data is greater than or equal to the first comparison value L1 (NO in S12). The processor 102 sets a save flag as step S22. Note that the save flag is stored in the storage device 103.

The processor 102 outputs the composite image data stored in the composite image memory 39 in step S17 to the display device, and then checks the save flag in step S23. If the save flag has been set (YES in S23), processor 102 transfers the composite image data to image save memory 40 in step S18 and resets the save flag in step S24. On the other hand, when the save flag is not set (NO in S23), the processes in steps S18 and S24 are not executed.
By modifying the control procedure executed by the processor 102 as shown in FIG. 9, the synthesized image can be accumulated and saved in the image saving memory 40 only when the infrared image or the warning image is synthesized.

  In the embodiment, the sound collected by the microphone is used as sensing data. Sensing data is not limited to voice. For example, when the monitoring target emits light when there is an abnormality, the light emission amount may be measured using an optical sensor, and the light emission amount data may be used as sensing data. Also in this case, it is possible to visually grasp the sensing data abnormality to be monitored.

  In the embodiment, the infrared camera 12 may be omitted. In this case, the visible image trimming unit 22 is omitted together with the infrared image processing function. That is, the image composition unit 29 composes the warning image with the visible image. Even in this case, since the warning image that warns of the abnormal sensing data of the monitoring target displayed on the visible image is displayed on the visible image, the abnormal sensing data of the monitoring target can be visually recognized. . As a result, a highly reliable monitoring system can be realized at low cost and in a small space.

  In the embodiment, the visible camera 11 may be omitted. In this case, the infrared image trimming unit 24 and the abnormal temperature image extracting unit 25 are omitted together with the visible image processing function. That is, the image composition unit 29 composes the warning image with the infrared image. Even in this case, a warning image that warns the sensing data abnormality of the monitoring object displayed in the infrared image is displayed superimposed on the infrared image, so that the abnormality of the sensing data can be visually confirmed together with the temperature abnormality of the monitoring object. Can be caught. As a result, a more reliable monitoring system can be realized at low cost and in a small space.

  In addition, this invention is not limited to the said embodiment, In the implementation stage, it can change variously in the range which does not deviate from the summary. Further, the embodiments may be implemented in combination as appropriate, and in that case, the combined effect can be obtained. Furthermore, the present invention includes various inventions, and various inventions can be extracted by combinations selected from a plurality of disclosed constituent elements. For example, even if several constituent requirements are deleted from all the constituent requirements shown in the embodiment, if the problem can be solved and an effect can be obtained, the configuration from which the constituent requirements are deleted can be extracted as an invention.

  DESCRIPTION OF SYMBOLS 1 ... Surveillance system, 10 ... Image processing apparatus, 11 ... Visible camera, 12 ... Infrared camera, 13 ... Microphone, 14, 15 ... Image encoder, 16 ... Audio encoder, 17 ... Display device, 21 ... Visible image reading part, 22 ... visible image trimming unit, 23 ... infrared image reading unit, 24 ... infrared image trimming unit, 25 ... abnormal temperature image extracting unit, 26 ... audio reading unit, 27 ... comparison unit, 28 ... warning image generating unit, 29 ... image Synthesizer, 30 ... display controller, 31 ... visible image memory, 32 ... area setting memory, 33 ... infrared image memory, 34 ... area setting memory, 35 ... abnormal temperature threshold memory, 36 ... audio memory, 37 ... comparison value memory 38 ... Warning image memory, 39 ... Composite image memory, 40 ... Image storage memory, 101 ... Communication interface, 102 ... Processor, 3 ... storage device.

Claims (8)

Visible image acquisition means for capturing a visible image captured by a visible camera;
An infrared image acquisition means for capturing an infrared image captured by an infrared camera that detects infrared rays emitted from an object in an imaging region of the visible camera;
Extraction means for extracting an area where the amount of infrared rays is a predetermined amount or more from the infrared image acquired by the infrared image acquisition means;
A combining unit that combines the infrared image extracted by the extracting unit with the visible image acquired by the visible image acquiring unit;
Output means for outputting a composite image obtained by the synthesis means to a display device;
An image processing apparatus comprising: Data acquisition means for capturing sensing data in the imaging region;
An abnormality detection means for detecting an abnormality of the sensing data acquired by the data acquisition means;
A warning image generating means for generating a warning image in response to an abnormality of sensing data detected by the abnormality detecting means;
Further comprising
The image processing apparatus according to claim 1, wherein the synthesizing unit further synthesizes the warning image. Visible image acquisition means for capturing a visible image captured by a visible camera;
Data acquisition means for capturing sensing data in an imaging region by the visible camera;
An abnormality detection means for detecting an abnormality of the sensing data acquired by the data acquisition means;
A warning image generating means for generating a warning image in response to an abnormality of sensing data detected by the abnormality detecting means;
Combining means for combining the warning image generated by the warning image generation means with the visible image acquired by the visible image acquisition means;
Output means for outputting a composite image obtained by the synthesis means to a display device;
An image processing apparatus comprising: An infrared image acquisition means for capturing an infrared image captured by an infrared camera that detects infrared rays emitted from an object in the imaging region;
Data acquisition means for capturing sensing data in the imaging region;
An abnormality detection means for detecting an abnormality of the sensing data acquired by the data acquisition means;
A warning image generating means for generating a warning image in response to an abnormality of sensing data detected by the abnormality detecting means;
Combining means for combining the warning image generated by the warning image generation means with the infrared image acquired by the infrared image acquisition means;
Output means for outputting a composite image obtained by the synthesis means to a display device;
An image processing apparatus comprising:   The image processing apparatus according to claim 2, wherein the warning image generation unit generates a different warning image according to a degree of abnormality of the sensing data. Computer
A visible image acquisition means for capturing a visible image captured by a visible camera;
An infrared image acquisition means for capturing an infrared image captured by an infrared camera that detects infrared radiation emitted from an object in an imaging region of the visible camera;
Extraction means for extracting an area in which the amount of infrared rays is a predetermined amount or more from the infrared image acquired by the infrared image acquisition means;
Synthesis means for synthesizing the infrared image extracted by the extraction means with the visible image acquired by the visible image acquisition means; and output means for outputting the synthesized image obtained by the synthesis means to a display device;
Image processing program to function as Computer
A visible image acquisition means for capturing a visible image captured by a visible camera;
Data acquisition means for capturing sensing data in an imaging region by the visible camera;
An abnormality detection means for detecting an abnormality of the sensing data acquired by the data acquisition means;
A warning image generating means for generating a warning image in response to an abnormality of the sensing data detected by the abnormality detecting means;
Combining means for combining the warning image generated by the warning image generating means with the visible image acquired by the visible image acquiring means; and output means for outputting the combined image obtained by the combining means to a display device;
Image processing program to function as Computer
Infrared image acquisition means for capturing an infrared image captured by an infrared camera that detects infrared radiation emitted from an object in the imaging region;
Data acquisition means for capturing sensing data in the imaging region;
An abnormality detection means for detecting an abnormality of the sensing data acquired by the data acquisition means;
A warning image generating means for generating a warning image in response to an abnormality of the sensing data detected by the abnormality detecting means;
Combining means for combining the warning image generated by the warning image generating means with the infrared image acquired by the infrared image acquiring means; and output means for outputting the combined image obtained by the combining means to a display device;
Image processing program to function as