JP5187275B2 - Photographing method and photographing apparatus - Google Patents

Description

  The present invention relates to a photographing method and a photographing apparatus, and more particularly, to a photographing method and a photographing apparatus that monitor a desired subject using an infrared camera and a visible light camera.

  Infrared cameras, for example, monitor wavelengths of about 8 μm to 14 μm, which corresponds to far infrared rays, and, unlike near infrared rays, directly detect radiant heat emitted by an object, and therefore have a feature that a light source is unnecessary. Therefore, the infrared camera is widely used for an obstacle recognition system for vehicles that frequently travel at night and for monitoring at night.

  However, the image obtained by the infrared camera is greatly different from the image obtained by visible light, and can recognize only the rough shape of the target. As a result, there is a problem in that it is not clear which part of the infrared camera is captured only by looking at the image obtained by the infrared camera. Therefore, a method of combining an infrared camera and an image obtained by a visible light camera has been developed.

  For example, Patent Document 1 discloses an invention in which an image of a visible light camera and an image of an infrared camera having a photographing area are used, and an image in a designated area is replaced with the other image. Patent Document 2 discloses an invention that uses a half mirror to align the optical axes of a visible light camera and an infrared camera.

  Furthermore, Patent Document 3 includes an infrared camera that captures an image of the same subject on different optical axes and a visible light camera with a zoom lens, and the imaging range of the visible light camera is set larger than that of the infrared camera in advance. Then, an invention is disclosed in which the size on the imaging surface of the visible light camera and the size of the subject on the imaging surface of the infrared camera are matched by adjusting the magnification of the zoom lens of the visible light camera.

Japanese Patent No. 3196122 JP-A-2-058480 JP-A-1-289390

  Conventionally, the resolution of an image sensor such as a charge coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) sensor used in a visible light camera is 640 pixels in the horizontal direction of the VGA (Video Graphic Array) standard, and the vertical direction. It was about 480 pixels and was not so large. However, in recent years, the number of pixels has increased significantly, and the resolution of the image sensor used in the visible light camera is 1024 pixels in the horizontal direction, 768 pixels in the vertical direction of the XGA (eXtended Graphics Array) standard, and HDTV (High Definition Television) standard. The number of pixels with 1920 pixels in the horizontal direction and 1125 pixels in the vertical direction has increased.

  On the other hand, the resolution of an image sensor of an infrared camera that handles far infrared rays has not increased much. For example, the resolution of a sensor of a micro bolometer array using a general MEMS (Micro Mechanical Systems) technology is a widespread 20,000 pixel class of 180 pixels in the horizontal direction and 120 pixels in the vertical direction. Then, the resolution of the image sensor of the infrared camera is 100 times different from the 2 million pixels of the HDTV image sensor of the visible light camera. Therefore, simply comparing the number of pixels, if the infrared camera and the visible light camera are photographing the same region, one pixel of the infrared camera corresponds to 100 pixels of the visible light camera. This causes a problem because the pixel number gap is too large.

  For example, even if there is a high-temperature object in the area for one pixel in the visible light camera, it is averaged to 1/100 in the infrared camera and is not noticeable. In addition, for example, when a human is imaged, even if it can be confirmed with a visible light camera, the number of pixels is too coarse for an infrared camera, and it may not be possible to confirm what the image looks like. This is inconvenient for nighttime monitoring with little visible light. Further, the invention described in Patent Document 1 in which the imaging areas of the visible light camera and the infrared camera are the same cannot be applied to an imaging apparatus having a large pixel number gap between the visible light camera and the infrared camera.

  In addition, the visible / far infrared switching system for avoiding obstacles in vehicles at night, such as the invention described in Patent Document 2, has a rough image in an infrared camera with a small number of pixels, and has low identification accuracy of a human body / artificial object. There is even a risk of causing an accident.

  The simplest solution to the above problem is to increase the resolution of the image sensor of the infrared camera to the resolution of the visible light camera. However, the price of the image pickup element of the infrared camera is expensive, and it is difficult to increase the resolution by simply increasing the number of pixels because it increases the cost. Furthermore, in the invention described in Patent Document 3, it is necessary to match the size on the imaging surface of the visible light camera with the size of the subject on the imaging surface of the infrared camera.

  The present invention has been made in view of the above points, and an object thereof is to provide a photographing method and a photographing device for low-cost and high-function monitoring without using an expensive high-resolution infrared camera. To do.

  In order to achieve the above-described object, the imaging method of the present invention is configured so that an arbitrary position within the first imaging range of the visible light camera is lower in resolution than the visible light camera and the imaging range is the first imaging range. The shooting direction can be freely moved to a specified step that is specified as the position of the second shooting range of the infrared camera that is smaller than the first shooting range of the visible light camera that is specified by the specifying step. The moving step of moving the second imaging range of the infrared camera, and the first imaging range obtained by imaging the infrared image of the second imaging range obtained by imaging with the infrared camera with the visible light camera. And an image composition step for performing composition to replace the visible light image portion in a region corresponding to the second imaging range at an arbitrary designated position in the visible light image, or to perform composition to superimpose on the visible light image portion. You .

  In order to achieve the above object, the photographing apparatus of the present invention has a lower resolution than that of the visible light camera and a first photographing range within the first photographing range of the visible light camera. The designation input means for designating the position of the second photographing range of the infrared camera that is smaller than the photographing range, and the photographing direction at an arbitrary position within the first photographing range of the visible light camera designated by the designation input means The moving means for moving the second imaging range of the movable infrared camera and the first image obtained by photographing the infrared image of the second imaging range obtained by photographing with the infrared camera with the visible light camera. And an image composition means for performing a composition to replace the visible light image portion in an area corresponding to the second photographing range at an arbitrary designated position in the visible light image of the photographing range, or to superimpose on the visible light image portion. It is characterized by .

  In order to achieve the above object, the photographing apparatus of the present invention has a resolution lower than that of the visible light camera within the first photographing range of the visible light camera and a photographing range that is smaller than the first photographing range. A moving unit that cyclically moves the second shooting range of the infrared camera that is small and whose shooting direction is movable, a threshold setting unit that sets a threshold value for abnormality determination, and an infrared camera An abnormal value indicating the degree of abnormality of the infrared image is calculated based on the infrared image of the second imaging range obtained by imaging, and the second imaging is performed depending on whether or not the abnormal value exceeds a threshold value. An abnormality determination means for determining whether or not there is an abnormality in the infrared image of the range, and a second imaging range obtained by photographing with an infrared camera at a position when the abnormality determination means determines that there is an abnormality Take infrared image with visible light camera Image composition means for performing composition to replace the visible light image portion of the second photographing range corresponding to the position in the visible light image of the first photographing range obtained in this way, or to superimpose on the visible light image portion It is characterized by having.

  In order to achieve the above object, in the photographing apparatus of the present invention, the moving means performs a cyclic movement of the second photographing range of the infrared camera within the first photographing range. It is also possible to perform the designated route or the route with the shortest traveling time for the imaging region.

  In order to achieve the above object, the imaging apparatus of the present invention has a first imaging range of the visible light camera being photographed, which has a lower resolution than the visible light camera and has a first imaging range. Moving means for moving the plurality of shooting areas cyclically with the infrared camera and shooting when divided into a plurality of shooting areas having substantially the same size as the second shooting range of the smaller infrared camera, The infrared image for each imaging area obtained by the infrared camera that cyclically moves a plurality of imaging areas is compared with the infrared image captured in the past in the corresponding imaging area to determine whether there is a change or not. An abnormality determination unit to perform, and a shooting time control unit that captures a shooting area captured by the infrared camera when the abnormality determination unit determines that an abnormality has occurred, for a longer time than other shooting areas that are not determined to be abnormal And an abnormality in the visible light image in the first photographing range obtained by photographing the infrared image obtained by photographing the photographing area determined to be abnormal by the abnormality determining means with the infrared camera. The image processing device is characterized by comprising image composition means for performing composition to replace the visible light image portion corresponding to the determined imaging region, or composition to superimpose on the visible light image portion.

  In order to achieve the above object, the imaging apparatus of the present invention uses an infrared camera to capture an imaging area captured by the infrared camera when the imaging time control unit determines that the abnormality is determined by the abnormality determination unit. It may be fixed and continuously photographed.

  Furthermore, in order to achieve the above object, the photographing apparatus of the present invention magnifies and captures a photographing region taken by the infrared camera when the infrared camera is determined to be abnormal by the abnormality determining means. A configuration having a zooming unit may also be used.

  According to the present invention, it is possible to obtain an imaging method and an imaging device for monitoring with high cost and low cost without using an expensive high-resolution infrared camera.

It is a block diagram of one embodiment of a photographing device of the present invention. It is a perspective view which shows an example of the positional relationship of the visible light camera and infrared camera in this invention. It is comparison explanatory drawing of an example of the imaging | photography range of a visible light camera and an infrared camera. It is explanatory drawing of 1st Embodiment of several area | region division | segmentation of the imaging | photography range of the visible light camera of the imaging device of this invention, and replacement | exchange / overlaying of the selection area | region to the infrared camera image. It is a flowchart for operation | movement description of 2nd Embodiment of the imaging device of this invention. It is a figure which shows the example of a display which the normal display of this invention apparatus switches to a display at the time of abnormality. It is a several explanatory view designation | designated explanatory drawing in the visible light camera imaging | photography range in 3rd Embodiment of the imaging device of this invention. It is a figure explaining the replacement | exchange / superimposition to the infrared image in the order which designated the several area | region in the visible light camera imaging | photography range in 3rd Embodiment of the imaging device of this invention. It is a figure explaining substitution / superposition to the infrared camera image in consideration of the shortest distance of the movement between fields in a plurality of fields in a visible light camera photography range in a 3rd embodiment of the photography device of the present invention. It is a figure explaining replacement / superposition to an infrared camera image continuously within a visible light camera photographing range in a 3rd embodiment of a photographing device of the present invention. It is explanatory drawing of an example of the imaging | photography area | region of the infrared camera in the imaging | photography range of the visible light camera in 4th Embodiment of the imaging device of this invention. It is a flowchart of an example of operation | movement of the infrared camera in 4th Embodiment of the imaging device of this invention. It is an example of the infrared image of an imaging | photography area | region. 14 is an infrared image when garbage is burning in the trash box in the infrared image of FIG. 13. It is a flowchart of the other example of operation | movement of the infrared camera in 4th Embodiment of the imaging device of this invention.

  Next, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows a block diagram of an embodiment of a photographing apparatus according to the present invention. In FIG. 1, the photographing apparatus 10 includes a visible light camera 11 and an infrared camera 12 including a zoom lens 13. As shown in FIG. 1, the photographing apparatus 10 includes A / D converters 14 and 16, frame memories 15 and 17, an image composition unit 18, a monitor 19, a recording / playback unit 20, and an image analysis unit. 21, an alarm issuing unit 22, a communication unit 23, an operation unit 24, and an infrared camera imaging region control unit 25.

  As shown in the perspective view of FIG. 2, the visible light camera 11 and the infrared camera 12 are arranged such that their optical axes are separated from each other by d. The visible light camera 11 is a camera that photoelectrically converts visible light using, for example, a high-resolution imaging device having about the number of pixels of the XGA standard or the HDTV standard. On the other hand, the infrared camera 12 is a camera that photoelectrically converts far-infrared rays having a wavelength of 8 μm to 14 μm using, for example, a low-resolution image sensor of the 20,000 pixel class. Therefore, as shown in FIG. 3, the shooting range of the infrared camera 12 is narrower than the shooting range 30 of the visible light camera 11, as shown by 41 and 42.

  In addition, the infrared camera 12 is rotated by the infrared camera shooting area control unit 25 in FIG. 1 in the vertical direction I and the horizontal direction II as shown in FIG. 2 with the predetermined position as a fulcrum. Possible configuration. As a result, the shooting range of the infrared camera 12 can be moved to any position within the shooting range of the visible light camera 11, and within the shooting range 30 of the visible light camera 11, for example, from the position indicated by 41 in FIG. It is possible to move to a position indicated by 42.

  Further, although the optical axes of the visible light camera 11 and the infrared camera 12 are not aligned, as shown in FIG. 2, the distance between the optical axes is d, and a range that is 10 times or more than d is captured. As a result, it is possible to obtain the same shooting situation as when the optical axes are substantially aligned. For example, if d is 10 cm, an image of 1 m or more is taken.

  Returning to FIG. The A / D converter 14 converts an analog video signal obtained by photoelectric conversion of visible light from the subject by the visible light camera 11 into first digital video data, and stores the first digital video data in the frame memory 15. The frame memory 15 is composed of, for example, a dynamic random access memory (DRAM). On the other hand, the A / D converter 16 converts an analog video signal obtained by the infrared camera 12 photoelectrically converting infrared rays from the subject into second digital video data, and stores the second digital video data in the frame memory 17. The frame memory 17 is composed of, for example, a DRAM.

  The infrared camera shooting area control unit 25 sets the shooting area (shooting range) of the infrared camera 12 within the shooting range 30 of the visible light camera 11 as shown in FIG. 3 according to the setting operated by the user with the operation unit 24. Control in As a control method, there are a method in which the direction of the body of the infrared camera 12 is directly controlled by a motor, and a method in which the direction of the incident light beam of the infrared camera 12 is controlled by shaking optical components such as a mirror and a lens. This includes a method of controlling the zoom magnification by the zoom lens 13 of the infrared camera 12. The infrared camera shooting area control unit 25 supplies shooting area information indicating the position of the shooting area of the infrared camera after the control and the like to the frame memory 17 for storage.

  The operation unit 24 is a device that can specify an area, such as a pen input device or a mouse, but the touch panel on the monitor 19 may be used as an input device. The operation unit 24 also sets analysis conditions for the image analysis unit 21.

  The image synthesis unit 18 synthesizes the first digital video data stored in the frame memory 15 and the second digital video data stored in the frame memory 17. As this image composition method, there is a method in which a photographed image of the infrared camera 12 is replaced or overlapped with an image of a part of the photographed region of the photographed image of the visible light camera 11. The monitor 19 displays a setting state at the time of setting operation of the operation unit 24, one of a visible light image and an infrared image, or a composite image thereof.

  The recording / reproducing unit 20 records the first digital video data stored in the frame memory 15 and the second digital video data stored in the frame memory 17 on the recording medium, and the above-described recording on the recording medium. The first and second digital video data are reproduced. The recording / reproducing unit 20 can also record the synthesized video data of the first and second digital video data synthesized by the image synthesizing unit 18 on a recording medium. On the other hand, the image synthesizing unit 18 can cause the recording / reproducing unit 20 to reproduce the synthesized video data from the recording medium and display the synthesized video data on the monitor 19.

  The image analysis unit 21 reads the digital video data from the recording medium on which the digital video data is recorded by the recording / playback unit 20, performs image analysis, extracts some features, and makes a determination. The characteristic is, for example, temperature information, temperature change from a certain time ago, or motion detection. The detection contents and determination criteria of the image analysis unit 21 are individually set using the operation unit 24 according to the monitoring target.

  The alarm issuing unit 22 generates an alarm based on the alarm information supplied from the image analysis unit 21 when the image analysis unit 21 analyzes the image and determines that the alarm is necessary based on a predetermined determination criterion. The alarm is performed by an alarm sound or by turning on or blinking a pilot lamp. Further, the alarm issuing unit 22 displays the alarm content on the monitor 19.

  The communication unit 23 transmits image information from the monitor 19 and alarm information from the alarm issuing unit 22 to the outside of the photographing apparatus 10 (for example, a security related person at a remote location or a security system).

  Hereinafter, each embodiment of the operation of the photographing apparatus 10 will be described with reference to the drawings.

(First embodiment)
First, the photographing apparatus 10 displays an image of a monitoring target that is a subject photographed by the visible light camera 11 on the entire screen of the monitor 19. Thereby, the supervisor can confirm the state of the entire monitoring target that cannot be obtained by the infrared camera 12 alone from the image of the monitor 19. Here, as shown in FIG. 4A, the image of the imaging range 30 of the visible light camera 11 displayed on the entire screen of the monitor 19 is, for example, 12 divided regions 31 of 4 in the horizontal direction and 3 in the vertical direction. It is divided into Each of the twelve divided areas 31 is equal to one photographing range 41 or 42 of the infrared camera 12 described with reference to FIG.

  The monitor operates the operation unit 24 to designate a region that is not illuminated or a region that is dark in the image from the visible light camera 11 displayed on the entire screen of the monitor 19. Assuming that the selected area designated by operating the operation unit 24 is, for example, one divided area indicated by 32 in FIG. 4A, the infrared camera shooting area control unit 25 performs the above operation from the operation unit 24. In response to the designation input of the selection area 32, the infrared camera 12 is rotated so as to photograph the selection area 32.

  Subsequently, the image composition means 18 replaces the image data of the selected area 32 in the digital image data of the visible light camera 11 from the frame memory 15 with the image composition of the infrared camera 12 from the frame memory 17. The video data after the image composition is output to the monitor 19. Thereby, as schematically shown in FIG. 4B, the monitor 19 displays an image of the region 43 corresponding to the selection region 32 shown in FIG. The image is replaced with the image from the infrared camera 12 and displayed.

  Thus, according to the present embodiment, the size on the imaging surface of the visible light camera is matched with the size of the subject on the imaging surface of the infrared camera without using an expensive high-resolution infrared camera. Therefore, it is possible to perform high-performance monitoring by combining the image of the visible light camera 11 excellent in visibility and the image of the infrared camera 12 excellent in monitoring of the dark part.

  Note that the visible light image from the visible light camera 11 in the selection region 32 and the infrared image from the infrared camera 12 may be displayed in an overlapping manner. In addition, instead of selecting one or more divided areas from a plurality of divided areas divided in advance, a selection area is selected that is the size of the shooting range of the infrared camera 12 centered on an arbitrarily input position. It can also be an area.

(Second Embodiment)
In the second embodiment of the present invention, the imaging device 10 replaces the infrared image of the area where the abnormality is detected, which is obtained by imaging with the infrared camera 12 only when an abnormality is detected, with the visible light image from the visible light camera 11. Or are displayed in duplicate.

  In the present embodiment, the photographing apparatus 10 displays a visible light image of a subject obtained by photographing with the visible light camera 11 on the entire screen of the monitor 19 when no abnormality is detected. FIG. 6A shows that the image of the photographing range 30 of the visible light camera 11 displayed on the entire screen of the monitor 19 is divided into, for example, twelve divided regions 31.

  On the other hand, the infrared camera 12 captures an image to be monitored in the capturing range 44 shown in FIG. 6B and stores it in the frame memory 17. The imaging range 44 is, for example, a position corresponding to one divided area 33 in the imaging range 30 shown in FIG. The infrared camera 12 senses infrared rays due to thermal radiation of the subject. Since the intensity of infrared rays due to thermal radiation of an object is determined by the temperature of the object, the temperature of the subject can be known by sensing infrared rays from the subject. Even if the visible light images obtained by photographing with the visible light camera 11 are the same, the infrared images obtained by photographing with the infrared camera 12 are different in the portion where the temperature change has occurred.

  In this state, the image analysis unit 21 sets a threshold temperature (for example, 28 ° C.) from the operation unit 24 according to the flowchart of FIG. 5 (step S1). Next, the image analysis unit 21 measures the temperature based on the infrared image of the imaging range (44 in FIG. 6B) specified by the operation unit 24 (step S2). Subsequently, the image analysis unit 21 monitors whether the temperature measured in step S2 is equal to or higher than the threshold temperature set in step S1 (step S3).

  When the image analysis unit 21 determines that the measured temperature is equal to or higher than the threshold temperature, the image analysis unit 21 controls the image composition unit 18 to display the visible light image of the imaging region 33 to be monitored illustrated in FIG. Instead, the image to be monitored in the imaging range 44 of the infrared camera 12 shown in FIG. 6B is synthesized (step S4). Thereby, the monitor 19 displays the synthesized image from the image synthesizing unit 18 (step S5). As shown in FIG. 6 (C), this synthesized image is replaced with a visible light image only in the imaging region 33 to be monitored in the visible light image of the photographing range 30 of the visible light camera 11 and has a temperature higher than a threshold value. This is an image in which the infrared image 45 to be monitored is replaced.

  In the above description, only the imaging region 33 to be monitored is described as being replaced with a visible light image and displayed as the infrared image 45 to be monitored at a temperature higher than the threshold, but is superimposed on the visible light image. Then, an infrared image may be displayed.

  As described above, according to the present embodiment, the monitoring target is imaged by the infrared camera 12, and when the monitoring target becomes equal to or higher than the predetermined threshold temperature, an abnormality is detected, and the monitoring target imaging area is detected. The infrared image is automatically switched and displayed in place of the visible light image or superimposed on the visible light image. Therefore, according to the present embodiment, an expensive high-resolution infrared camera is not used, and the size on the imaging surface of the visible light camera and the size of the subject on the imaging surface of the infrared camera are not matched. Only when an abnormality of the monitoring target is detected, an image of the infrared camera 12 excellent in monitoring of a dark part can be displayed, and efficient monitoring can be performed.

  In addition, abnormality detection can set other than temperature so that it may mention later.

(Third embodiment)
In the third embodiment of the present invention, the imaging apparatus 10 is configured to cyclically shoot a plurality of preset monitoring target areas in the imaging range of the visible light camera 11 with the infrared camera 12.

  In FIG. 7, the image of the imaging range 30 of the visible light camera 11 displayed on the entire screen of the monitor 19 is divided into, for example, 12 divided areas 31, and among these, four divided areas 34-1 and 34 are divided. -2, 34-3, and 34-4 indicate divided areas (monitoring target areas) where the monitoring target imaged by the infrared camera 12 is present. In the image analysis unit 21, the monitoring target area is set from the operation unit 24. As shown in FIG. 7, the monitoring target regions do not have to be separate from each other, and it is needless to say that the same region such as a region to be particularly noted can be designated a plurality of times.

  Next, the infrared camera imaging region control means 25 captures the imaging direction (optical axis) of the infrared camera 12 so as to sequentially capture the monitoring target regions 34-1, 34-2, 34-3, 34-4 shown in FIG. ) Is controlled cyclically. Thereby, the image composition unit 18 replaces the visible light images of the monitoring target areas 34-1, 34-2, 34-3, and 34-4 shown in FIG. 7 or converts them into visible light images of those monitoring target areas. The infrared images 46-1, 46-2, 46-3, 46-4 of the monitoring target areas 34-1, 34-2, 34-3, 34-4 from the infrared camera 12 are superimposed and shown in FIG. As shown in Fig. 4, the display is switched cyclically in the order of the arrows.

  Note that the cyclic movement control of the infrared camera 12 in the imaging direction by the infrared camera imaging area control means 25 is not limited to the above example. For example, the imaging apparatus 10 is configured so that the traveling time per cycle of the infrared camera 12 is the shortest for the monitoring target areas 34-1, 34-2, 34-3, and 34-4 illustrated in FIG. As shown in FIG. 9, the infrared images 46-1, 46-2, 46-3, 46-4 of the monitoring target areas 34-1, 34-2, 34-3, 34-4 from the camera 12 are shown by arrows. And is displayed on the monitor 19 in place of the visible light images of the monitoring target areas 34-1, 34-2, 34-3, 34-4 or overlapping the visible light images of the monitoring target areas. You may make it do.

  Further, the above is a method of discontinuously replacing the captured infrared image of the infrared camera 12 with a visible light image for each designated monitoring target area, or displaying the image by superimposing, but the present invention is not limited to this. As indicated by 47 in FIG. 10, a captured infrared image in the middle of moving the imaging direction (optical axis) of the infrared camera 12 from the designated monitoring target region to the adjacent designated target region is continuously replaced with a visible light image or superimposed. They can also be displayed together.

  As described above, according to the present embodiment, the imaging apparatus 10 cyclically captures a plurality of preset monitoring target areas in the imaging range of the visible light camera 11 with the infrared camera 12, and the infrared image is visible. Replace with an image or display it superimposed. Therefore, according to the present embodiment, an expensive high-resolution infrared camera is not used, and the size on the imaging surface of the visible light camera and the size of the subject on the imaging surface of the infrared camera are not matched. In addition, it is possible to perform efficient monitoring excellent in monitoring a plurality of monitoring target areas.

(Fourth embodiment)
In the fourth embodiment of the present invention, when the imaging device 10 detects an abnormality, the infrared image captured by the infrared camera 12 is replaced with the image from the visible light camera 11 in the region where the abnormality is detected longer than the other regions. Or are displayed in duplicate.

  In the present embodiment, the photographing apparatus 10 displays an image of a subject photographed by the visible light camera 11 on the entire screen of the monitor 19. FIG. 11 shows that the imaging range 30 of the visible light camera 11 displayed on the entire screen of the monitor 19 is divided into 12 imaging areas # 1 to # 12. Each of these twelve shooting areas # 1 to # 12 has substantially the same size as the shooting range of the infrared camera 12.

  Next, the operation of the photographing apparatus 10 of the present embodiment will be described with reference to the flowchart of FIG. First, the imaging device 10 captures the entire visible light imaging range 30 of the visible light camera 11 during normal times (when no abnormality is detected) with the infrared camera 12 (step S11). In step S11, the infrared camera 12 changes the shooting direction up, down, left, and right by the infrared camera shooting area control unit 25, and the shooting areas # 1, # 2, # 3,... # 11, # 12 in FIG. Shoot sequentially in the order of.

  Subsequently, the digital video data after digital conversion of the infrared image for each photographing region obtained by sequentially photographing with the infrared camera 12 is recorded on the recording medium by the recording / reproducing unit 20 (step S12). Next, the imaging device 10 captures the first imaging area # 1 out of the 12 imaging areas # 1 to # 12 with the infrared camera 12, and the image analysis unit 21 performs infrared imaging of the imaging area # 1. Is compared with the infrared image of the photographing area # 1 recorded in the normal state previously read from the recording medium, and it is determined whether or not there is a change greater than a preset threshold value (step S14). ). As described above, the infrared camera 12 can know the temperature of the subject by sensing infrared rays from the subject. Therefore, it can be determined that a change in the threshold value or more is a temperature change in a predetermined value or more, and the photographing apparatus 10 determines that it is abnormal.

  When there is no change, the photographing apparatus 10 moves the photographing direction of the infrared camera 12 to the next photographing region # 2 (step S15). Subsequently, the image analysis unit 21 compares the obtained infrared image of the imaging region # 2 with the infrared image of the imaging region # 2 in the normal state that has been previously read out from the recording medium, and the change is made. It is determined whether or not there is (step S14). Until the change is detected, the above steps S13 to S15 are repeated, and the infrared camera 12 sequentially images the imaging regions # 1 to # 12 of FIG.

  On the other hand, when it is determined in step S14 that there is a change, the infrared camera shooting area control unit 25 fixes the infrared camera 12 in the shooting area determined to have a change and performs shooting (step S16). Then, the image composition unit 18 superimposes the infrared image captured by the infrared camera 12 fixed in step S16 on the visible light image in the imaging range 30 captured by the visible light camera 11 (step S17). The monitor 19 displays the visible light image and the infrared image superimposed in step S17. Further, the alarm issuing unit 22 issues an alarm based on the abnormality detection result detected by the image analysis unit 21.

  13 and 14 show examples of infrared images of the infrared camera 12. As shown in FIG. 13, the infrared image of the imaging region 48 of the infrared camera 12 (any one of the imaging regions # 1 to # 12 in FIG. 11) is composed of a house 51 and a trash can 52. And If the trash in the trash box 52 is ignited by a cigarette that has been forgotten to be erased, the temperature of the trash box 52 rises.

  Then, as shown in FIG. 14, the infrared image of the imaging region 48 of the infrared camera 12 does not change the image of the house 51, but the image of the trash 53 with the temperature rising is different from the trash 52 of FIG. 13. Indicates. In the visible light image by the visible light camera 11, the trash can is the same image, but in the infrared image, the temperature of the trash can 53 increases, and the infrared rays increase, so that the trash can appears brighter than the trash can 52 in FIG. It can be judged that there was.

  When such a situation occurs, the imaging apparatus 10 according to the present embodiment determines that there is an abnormality, fixes the infrared camera 12 to the imaging region of the image in which the abnormality has occurred, continuously shoots, and captures detailed images. It is possible to take a picture and replace or superimpose it with a visible light image of a visible light camera to inform the details of the abnormal situation.

  FIG. 15 shows a flowchart of another operation example of the present embodiment. In the figure, the same steps as those in FIG. 12 are denoted by the same reference numerals, and the description thereof is omitted. In FIG. 15, when it is determined that there is a change in step S <b> 14, the infrared camera shooting area control unit 25 sets the shooting area determined to be changed for a longer time than the shooting area determined to be unchanged. The infrared camera 12 is fixed and photographed for a predetermined time (for example, 30 seconds) (step S21).

  Then, the image composition unit 18 superimposes the infrared image captured by the infrared camera 12 in step S21 for a predetermined time on the visible light image of the visible light camera 11 (step S22). That is, in step S22, the visible light image portion of the visible light image region corresponding to the photographing region determined to have a change in the infrared image in the visible light image of the photographing range 30 photographed by the visible light camera 11 is displayed. In step S21, the infrared image captured by the infrared camera 12 for a predetermined time is superimposed. The monitor 19 displays the visible light image and the infrared image superimposed in this way.

  Thereafter, the infrared camera imaging region control unit 25 confirms whether or not to end the cyclic imaging (step S23). When the process is to be ended (Yes in step S23), the process is ended as it is. If not finished (No in step S23), the process returns to step S15 to repeat the cyclic shooting.

  As described above, according to the present embodiment, the entire visible light photographing range 30 of the visible light camera 11 is cyclically photographed by the infrared camera 12, and the infrared image for each photographing range of the infrared camera 12 is dealt with in the normal state. Whether or not there is a change in the infrared image of the photographing range to be detected is detected, and when there is a change, it is determined that there is an abnormality. Then, according to the present embodiment, an infrared image obtained by fixing the shooting range of the infrared camera 12 determined to be abnormal or shooting for a longer time than the other shooting ranges is visible in the shooting range. It is replaced with a light image or superimposed and displayed on the monitor 19.

  Therefore, according to the present embodiment, an expensive high-resolution infrared camera is not used, and the size on the imaging surface of the visible light camera and the size of the subject on the imaging surface of the infrared camera are not matched. The abnormal state can be notified in detail by the infrared image obtained by photographing on the screen of the monitor 19 for a predetermined time, and a plurality of abnormalities can be found in the photographing range 30 of the visible light camera 11.

  In the description of the fourth embodiment, the traveling method in the photographing direction of the infrared camera 12 has been described so that the photographing area shown in FIG. 11 is visited in the left-right and up-down order, but the traveling method is limited to this. For example, it is possible to circulate in a designated order such as shooting area # 5, shooting area # 7, shooting area # 11, shooting area # 9, or abnormal in all shooting areas. It is also possible to preferentially circulate an imaging region where the image is likely to occur. In addition, the shooting time per shooting area of the infrared camera 12 during the cyclic shooting may not be constant. For example, each shooting time in a shooting area in which abnormalities frequently occur and in a shooting area in which abnormalities rarely occur can be detected. You can also change the patrol according to your ease.

  In addition, the zoom lens 13 of the infrared camera 12 can further magnify the shooting area where there was an abnormality, and shoot the area where there was an abnormality in more detail.

  In addition, the present invention is not limited to the above-described embodiment. For example, the abnormality detection unit only detects a temperature equal to or higher than a threshold value as in the second or fourth embodiment as an abnormality. In addition, depending on the application to be used, a case where a temperature below a threshold is detected as abnormal is included. Furthermore, the abnormality detection means in the present invention is not limited to means for detecting from the infrared image a temperature that cannot be in the normal state as described above. For example, the difference detection means, the leaving detection means, the animal detection means Also included is a disguise / abnormal disguise detection means.

  The difference detection unit is a unit that compares a plurality of images with a predetermined time difference and detects whether there is a change in the image, such as a subject moving or something appearing in the shooting range. Further, the above-mentioned leaving detection means is means for detecting whether or not a human has left something. In this leaving detection means, for example, assuming a bomb terrorism or the like, it is detected whether or not the person who had the trap did not put it somewhere. In principle, an alarm is issued when the area of a person in the image changes abruptly.

  The animal detection means is means for detecting the body temperature of the subject from the infrared image and detecting whether the subject is an animal from the body temperature. Furthermore, the animal detection means detects whether the animal is a human or other animal by using image recognition together. This animal detection means is used for the purpose of monitoring a farm from wild animals. In addition, the disguise / abnormal disguise detection means described above can detect infrared rays from a human face wearing a balaclava or mask, and so on. It is a means for detecting anomalies using the difference from the way of going out. Conversely, a naked human can also be detected from an infrared image.

  Note that the present invention is not limited to the above-described embodiment, and includes, for example, a shooting program for realizing the operation of each embodiment by a computer. The photographing program may be recorded on a recording medium and taken into a computer, or distributed via a network and loaded onto a computer.

10 Shooting device
DESCRIPTION OF SYMBOLS 11 Visible light camera 12 Infrared camera 13 Zoom lens 14, 16 A / D converter 15, 17 Frame memory 18 Image composition part 19 Monitor 20 Recording / reproducing part 21 Image analysis part 22 Alarm issuing part 23 Communication part 24 Operation part 25 Infrared camera Shooting area control unit 30 Visible light camera photographing range 31 Visible light camera divided photographing area 32 Selected divided photographing areas 34-1 to 34-4 Monitoring target areas 41 and 42 Infrared camera photographing ranges 43 and 45 Visible light image Infrared image 44 replaced with or superimposed on image capturing region 46-1 to 46-4 of infrared camera Infrared image 47 Continuous infrared image

Claims (7)

The second imaging range of the infrared camera whose arbitrary position within the first imaging range of the visible light camera is lower in resolution than the visible light camera and whose imaging range is smaller than the first imaging range. A designated step that is designated as the position of
A moving step of moving the second shooting range of the infrared camera whose shooting direction is freely movable to the arbitrary position within the first shooting range of the visible light camera specified by the specifying step;
The infrared image of the second imaging range obtained by photographing with the infrared camera is taken at the arbitrary designated position in the visible light image of the first photographing range obtained by photographing with the visible light camera. An image synthesizing method comprising: an image synthesizing step of performing composition for replacing with a visible light image portion in an area corresponding to the second image capturing range, or performing composition for superimposing on the visible light image portion. The second imaging range of the infrared camera whose arbitrary position within the first imaging range of the visible light camera is lower in resolution than the visible light camera and whose imaging range is smaller than the first imaging range. A designation input means to designate as the position of
Moving means for moving the second imaging range of the infrared camera whose imaging direction is movable to the arbitrary position within the first imaging range of the visible light camera designated by the designation input means;
The infrared image of the second imaging range obtained by photographing with the infrared camera is taken at the arbitrary designated position in the visible light image of the first photographing range obtained by photographing with the visible light camera. An image-capturing apparatus comprising: an image composing unit that performs compositing to replace a visible light image portion in an area corresponding to the second image capturing range, or to superimpose on the visible light image portion. An infrared camera having a resolution lower than that of the visible light camera within a first shooting range of the visible light camera, a shooting range smaller than the first shooting range, and a shooting direction that is movable. Moving means for cyclically moving the second imaging range;
Threshold setting means for setting a threshold for abnormality determination;
An abnormal value indicating the degree of abnormality of the infrared image is calculated based on the infrared image of the second imaging range obtained by photographing with the infrared camera, and whether or not the abnormal value exceeds the threshold value. And an abnormality determining means for determining whether or not there is an abnormality in the infrared image in the second imaging range;
The infrared image of the second photographing range obtained by photographing with the infrared camera located at the position when the abnormality judging means determines that there is an abnormality, the infrared image obtained by photographing with the visible light camera. Image synthesizing means for performing synthesis to replace the visible light image portion of the second photographing range corresponding to the position in the visible light image of the first photographing range, or to superimpose on the visible light image portion. An imaging apparatus characterized by that. The moving means is
A cyclic movement of the second imaging range of the infrared camera is performed with respect to a plurality of preset imaging areas in the first imaging range by a designated route or a route having the shortest traveling time. The photographing apparatus according to claim 3. The first photographing range of the visible light camera being photographed is substantially the same as the second photographing range of the infrared camera whose resolution is lower than that of the visible light camera and whose photographing possible range is smaller than the first photographing range. When dividing into a plurality of shooting areas of the same size, moving means for moving the plurality of shooting areas cyclically by the infrared camera and shooting,
Whether there is a change in the infrared image for each imaging region obtained by the infrared camera that cyclically moves the plurality of imaging regions by the moving means compared to the infrared image captured in the past of the corresponding imaging region An abnormality determination means for determining abnormality by
Shooting time control means for shooting the shooting area taken by the infrared camera when determined to be abnormal by the abnormality determination means for a longer time than other shooting areas that are not determined to be abnormal;
In a visible light image of the first photographing range obtained by photographing an infrared image obtained by photographing the photographing region determined to be abnormal by the abnormality determining unit with the infrared camera. An image synthesizing apparatus comprising: a composition for replacing the visible light image portion corresponding to the imaging region determined to be abnormal, or a composition for superimposing on the visible light image portion. The photographing time control means includes
6. The imaging apparatus according to claim 5, wherein the imaging region captured by the infrared camera when the abnormality determining unit determines that an abnormality has occurred is continuously captured with the infrared camera fixed. The infrared camera
The imaging apparatus according to claim 5 or 6, further comprising a zoom unit that enlarges and captures the imaging region captured by the infrared camera when the abnormality determination unit determines that an abnormality has occurred.