JP2009010669A - Image processing unit, imaging apparatus, image processing method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To detect a desired object reliably based on a captured image signal in an image processing unit. <P>SOLUTION: The image processing unit has: a white extraction section 21 for extracting a first image region in which a pixel value becomes not less than a prescribed level, based on an infrared image signal from a far infrared camera 11; a white extraction section 22 for extracting a second image region in which at least a pixel value becomes not less than a prescribed level, based on a visible light image signal from a visible light camera 12 capturing an image in the same direction as that of the far infrared camera 11; an AND processing section 23 for extracting the overlapped region between the first and second image regions; a black substitution section 24 for substituting each pixel value in the overlapped region in infrared image signals for a pixel value less than a prescribed upper-limit threshold; and a white extraction section 25 for extracting the image region in which the pixel value becomes not less than a prescribed threshold set to a band not less than the upper-limit threshold, based on the substituted image signal. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing apparatus for detecting a specific subject based on an input image signal, an imaging apparatus having this image processing function, an image processing method, and a program. In particular, the present invention relates to image signals from two types of cameras. The present invention relates to an image processing apparatus, an imaging apparatus, an image processing method, and a program that perform subject detection processing.

  Conventionally, various techniques for detecting a specific object within an imaging range by performing image processing based on a captured image signal have been considered, and such techniques have been used for surveillance cameras and the like. In addition to visible light cameras that detect and detect visible light, near-infrared cameras and far-infrared cameras are also known, and these objects can be identified by detecting the infrared radiation energy of the imaging target. It was.

  By the way, visible light cameras and infrared cameras (particularly far-infrared cameras) differ in the objects that can be identified relatively reliably due to the difference in their characteristics. For example, an object floating on a wave can be detected relatively easily from an image captured by a visible light camera, but it has not been easy to identify whether the object is a human or dust. On the other hand, from an image captured by a far-infrared camera, it is relatively easy to identify whether an object floating on a wave is a human or dust, but it is not easy to identify a white wave and a human.

In addition, as an image monitoring device that uses one type of camera to improve the detection accuracy of an object, a luminance difference from a reference image that captures background fluctuations, an image obtained by weighted averaging of images of a plurality of frames, and By referring to the brightness difference between the two, there is one that prevents an erroneous detection or detection omission of an intruding object even when the sunshine condition of the monitoring target area changes (see, for example, Patent Document 1).
JP 2002-352340 A (paragraph numbers [0034] to [0047], FIG. 1)

  As described above, in the visible light camera and the infrared camera, there are objects that are easy to detect and objects that are not, and therefore, each camera cannot always reliably detect a desired object. In the above example, when the visible light camera and the far-infrared camera are respectively used alone, it is impossible to reliably detect a person floating in a wave.

  The present invention has been made in view of the above points, and provides an image processing apparatus, an imaging apparatus, an image processing method, and a program that can more reliably detect a desired object based on a captured image signal. For the purpose.

  In the present invention, in order to solve the above problems, in an image processing apparatus for detecting a specific subject based on an input image signal, a pixel value is set to a first level band based on an infrared image signal from an infrared camera. A pixel value is included in the second level band based on a first region extraction unit that extracts a first image region included and a visible light image signal from a visible light camera that images the same direction as the infrared camera. A second region extracting unit that extracts the second image region, and the first image region and the second image region from either the first image region or the second image region, There is provided an image processing apparatus comprising: a third image area extraction unit that outputs a third image area excluding the overlapping area as an area including the specific subject.

  In such an image processing apparatus, subject detection processing is performed based on an infrared image signal from an infrared camera and a visible light image signal from a visible light camera that captures the same direction as the infrared camera. The first region extracting unit extracts a first image region whose pixel value is included in the first level band based on the infrared image signal, and the second region extracting unit is based on the visible light image signal. The second image area whose pixel value is included in the second level band is extracted. The third image region extraction unit removes an overlap region between the first image region and the second image region from either the extracted first image region or second image region. The image area is output as an area including a specific subject that has been a detection target. In such processing, when there is a subject that can be detected in common by the first image extraction unit and the second image extraction unit, the region that is considered to include another subject other than this subject is the third one. Extracted by the image extraction unit.

  According to the image processing apparatus of the present invention, when there is a subject that can be detected in common by the first image extraction unit and the second image extraction unit, an area that is considered to include another subject other than the subject. Therefore, the region extraction process by either the first image extraction unit or the second image extraction unit is indistinguishable from the object that can be detected in common by the third image extraction unit. Another subject can be accurately detected by the third image extraction unit. Therefore, it becomes possible to accurately detect a specific subject that is difficult to detect when the infrared camera and the visible light camera are used alone.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. An imaging apparatus described in the following embodiments includes two cameras that can detect light (electromagnetic waves) in different frequency bands that are reflected or emitted from a subject. Here, such an imaging apparatus is called a hybrid camera.

[First Embodiment]
FIG. 1 is a diagram schematically showing the configuration of the hybrid camera according to the first embodiment.

  The hybrid camera 1 shown in FIG. 1 mainly performs signal processing using a far-infrared camera 11 that can detect electromagnetic waves in the far-infrared region, a visible light camera 12 that can detect visible light, and images captured by the cameras. And an image processing circuit 13.

  The far-infrared camera 11 and the visible light camera 12 capture images in the same direction with the same angle of view. The far-infrared camera 11 includes, for example, a pyroelectric element, a bolometer, or the like that detects energy (heat) of the irradiated far-infrared rays as a sensor that detects far-infrared rays. The visible light camera 12 includes a solid-state imaging device such as a CCD (Charge Coupled Devices) or a CMOS (Complementary Metal Oxide Semiconductor) image sensor as a sensor for detecting visible light.

  The image processing circuit 13 performs various signal processing based on the image signals captured by the far-infrared camera 11 and the visible light camera 12. In particular, in the present embodiment, based on both of these image signals, a process is performed in which a specific object is detected from the imaging range and the detection of the object is clearly indicated in the output image.

  The far-infrared camera 11 and the visible light camera 12 have a function of converting an output signal from each sensor into a digital signal, and the image processing circuit 13 is a digital output from the far-infrared camera 11 and the visible light camera 12. Processing is performed based on the image signal. Further, the far-infrared camera 11 and the visible light camera 12 may each include sensors having different numbers of pixels. However, in this case, each camera (or one of them) is provided with a resolution conversion function, and each camera outputs a digital image signal having the same number of pixels per frame to the image processing circuit 13. To be.

  In this embodiment, the hybrid camera 1 can be connected to an external device through a communication line. In FIG. 1, a personal computer (PC) 2 is connected as an external device. For example, an output image from the image processing circuit 13 of the hybrid camera 1 is displayed on the monitor of the PC 2 and an output image signal is stored in the PC 2. Can be done. Thereby, the imaging area by the hybrid camera 1 can be monitored from the PC 2 side. Furthermore, operations for controlling the operation of the hybrid camera 1 and various settings may be performed from the PC 2 side.

FIG. 2 is a diagram showing a more detailed internal configuration of the hybrid camera according to the first embodiment.
As shown in FIG. 2, the image processing circuit 13 of the hybrid camera 1 includes white extraction units 21 and 22, a logical product (AND) processing unit 23, a black replacement unit 24, a white extraction unit 25, a display frame generation unit 26, and a composition. A processing unit 27 and a setting register 28 are provided. Further, the hybrid camera 1 is provided with a synchronization signal generator 29 for synchronizing the imaging operations of the far-infrared camera 11 and the visible light camera 12.

  An image based on a signal output from the far-infrared camera 11 can be represented by, for example, a black and white image. The white extraction unit 21 extracts a white region by detecting a region having a gradation equal to or higher than a predetermined threshold from the image signal from the far-infrared camera 11. The white extraction unit 21 operates by reading a gradation threshold value for extracting a white region from the setting register 28.

  On the other hand, the image signal output from the visible light camera 12 is composed of R / G / B (Red / Green / Blue) color components. The white extraction unit 22 extracts a white region from an image signal from the visible light camera 12 based on a comparison result between each color component of R / G / B and a threshold value for each component, and a relationship between the color components. Extract. Similar to the white extraction unit 21, the white extraction unit 22 operates by reading conditions such as threshold values of each color component for extracting a white region from the setting register 28.

  The AND processing unit 23 calculates the logical product of the white areas extracted by the white extraction units 21 and 22. That is, the AND processing unit 23 extracts only the area determined to be white by both the far-infrared camera 11 and the visible light camera 12, and removes the area determined to be white by only one of them.

The black replacement unit 24 replaces only the white region extracted by the AND processing unit 23 in the image signal from the far-infrared camera 11 with the lowest gradation value (0), and converts this region to black.
The white extraction unit 25 detects a region having a predetermined threshold value or more from the image signal from the black replacement unit 24 and extracts a white region. The white extraction unit 25 also operates by reading the gradation threshold value for extracting the white region from the setting register 28.

  The display frame generation unit 26 outputs an image region for visually showing the white region extracted by the white extraction unit 25 to the synthesis processing unit 27. Here, as an example, a display frame having a predetermined shape including a white area is generated, and the display frame area is output to the composition processing unit 27. Further, the display frame generation unit 26 reads the shape of the display frame, the width of the frame, and the like from the setting register 28, and generates a display frame accordingly.

  The synthesis processing unit 27 synthesizes the display frame generated by the display frame generation unit 26 with the image signal from the visible light camera 12. For example, the composition processing unit 27 replaces only the area of the display frame output from the display frame generation unit 26 in the image signal from the visible light camera 12 with a predetermined color. Thereby, an image in which a display frame of a predetermined color is displayed around the white area extracted by the white extraction unit 25 is generated. Further, the composition processing unit 27 reads a color or the like for displaying the display frame from the setting register 28 and synthesizes the display frame accordingly.

  The setting register 28 holds various setting information necessary for the operation in the image processing circuit 13. Further, the information in the setting register 28 can be rewritten according to an input operation from an input unit (not shown) provided in the hybrid camera 1 or an input operation in an external device such as the PC 2, whereby the image processing circuit The operation setting in 13 can be changed.

  Further, the far-infrared camera 11 and the visible light camera 12 perform the imaging operation at the same timing in synchronization with the timing signal generated by the synchronization signal generation unit 29. Note that these cameras do not necessarily have to perform imaging operations at exactly the same timing. For example, the imaging interval of each camera may be different. However, even in such a case, it goes without saying that it is desirable that the imaging timings of the cameras be as close as possible.

  In the hybrid camera 1 having the above configuration, by using the object detection results of both the far-infrared camera 11 and the visible light camera 12, it is possible to reliably detect an object that cannot be identified by only one of them. . Specifically, as will be described with reference to FIGS. 3 to 5 below, it is possible to reliably detect a human being existing near a waved water surface.

  3 to 5 are diagrams illustrating examples of images corresponding to image signals in each unit in the hybrid camera. Hereinafter, the flow of the object detection process in the hybrid camera 1 will be specifically described step by step using these drawings.

  First, in this embodiment, it is assumed that the far-infrared camera 11 outputs an 8-bit digital image signal. In this case, the pixel with the gradation value “0” is the strongest black, and the pixel with the gradation value “255” is the strongest white. On the other hand, the visible light camera 12 outputs a digital color image signal in which each R / G / B component is 8 bits. When the value of each component of R / G / B in the image signal is expressed as “(r / g / b)”, the pixel “(0/0/0)” is the strongest black, and “(255/255) / 255) "is the strongest white color.

  In FIG. 3, images p <b> 1 and p <b> 2 correspond to image signals P <b> 1 and P <b> 2 that are simultaneously captured and output by the far-infrared camera 11 and the visible light camera 12. In these images p1 and p2, white waves 101 and 102 on the sea surface, a sea surface 103 excluding these white waves 101 and 102, and a human 104 floating on the sea surface 103 exist as examples of subjects.

  Here, in the image p 1 captured by the far-infrared camera 11, the gradation value of the white wave 101 is “250”, the gradation value of the white wave 102 is “252”, the gradation value of the sea surface 103 is “10”, and the human 104 Is assumed to be “251”.

  On the other hand, in the image p2 captured by the visible light camera 12, the value of each color component in the white wave 101 is “(252/252/251)”, and the value of each color component in the white wave 102 is “(250/250/250). It is assumed that the value of each color component on the sea surface 103 is “(20/20/200)” and the value of each color component in the area of the human 104 is “(31/32/30)”.

  The white extraction unit 21 extracts a white region by detecting a region having a gradation equal to or higher than a predetermined threshold in the image signal P1 from the far-infrared camera 11. Here, it is assumed that “250” is set in the setting register 28 as an example of the gradation threshold value.

  The image p3 in FIG. 3 corresponds to the image signal P3 output from the white extraction unit 21, and the white waves 101 and 102 and the human 104 among the subjects of the image p1 are processed by the white extraction unit 21 based on the above setting. Only is extracted as a white region. Note that the image signal P3 is a binary image signal having only “0” or “1” as a pixel value, and the extracted white region (that is, each region of the white waves 101 and 102 and the human 104). The pixel value is set to “1”, and the pixel values in other areas are set to “0”. In the image p3 in FIG. 3, the region of the pixel value “1” is shown in a solid color.

On the other hand, in the white extraction unit 22, based on the comparison result between each color component of R / G / B and the threshold value for each component from the image signal from the visible light camera 12, and the relationship between the color components, Regions are extracted. Here, as an example, if the R / G / B component values are equal to or greater than the threshold values th_r, th_g, and th_b, respectively, and the difference between the color components is within a predetermined value, the pixel is Determined to be white. The latter condition is that the following three expressions are all satisfied.
r−rl ≦ g ≦ r + rh, g−gl ≦ b ≦ g + gh, b−bl ≦ r ≦ b + bh
In the above equation, r, g, and b represent the values of the R component, G component, and B component in the image signal P2, respectively. Further, rl, rh, gl, gh, bl, and bh are coefficients of 0 or more that indicate the difference ranges between the respective color components, and have a relationship of rl ≦ th_rh, th_gl ≦ th_gh, th_bl ≦ th_bh.

  In this embodiment, the threshold values th_r, th_g, and th_b are all “250”, the coefficients rl, rh, gl, gh, bl, and bh are all “2”. Shall be set to

  The image p4 in FIG. 3 corresponds to the image signal P4 output from the white extraction unit 22, and only white waves 101 and 102 are white among the subjects of the image p2 by the processing of the white extraction unit 22 based on the above setting. Extracted as a region. Similar to the image signal P3, the image signal P4 is a binary image signal having only “0” or “1” as a pixel value, and the extracted white region (that is, each of the white waves 101 and 102). The pixel value of “region” is set to “1”, and the pixel values of other regions are set to “0”. In the image p4 in FIG. 3, the region of the pixel value “1” is shown in a solid color.

  As another example of processing in the white extraction unit 22, a white region may be extracted on the condition that R / G / B component values are equal to or greater than threshold values th_r, th_g, and th_b, respectively. However, as described above, it is possible to accurately extract a region closer to white by using a condition relating to the relationship between the color components.

  Next, the AND processing unit 23 calculates the logical product of the corresponding pixel values in the image signals P3 and P4 from the white extraction units 21 and 22, respectively. As a result, an image signal P5 indicating an area determined to be white by both the far-infrared camera 11 and the visible light camera 12 is output.

  The image p5 in FIG. 4 corresponds to the image signal P5, and only white waves 101 and 102 are extracted as white regions from the white regions in the images p3 and p4 by the processing of the AND processing unit 23. The image signal P5 is a binary image signal that takes only “0” or “1” as the pixel value, and the pixel value of the extracted white area (that is, each area of the white waves 101 and 102) is “1” is set, and pixel values in other regions are set to “0”. In the image p5 in FIG. 4, the region of the pixel value “1” is shown in a solid color.

  Next, only the white region extracted by the AND processing unit 23 in the image signal P1 from the far-infrared camera 11 is replaced with the lowest gradation value (0) by the black replacement unit 24, and this region is converted to black. Is done. The image p6 in FIG. 4 corresponds to the image signal P6 output from the black replacement unit 24. By the processing of the black replacement unit 24, the pixel values of the areas of the white waves 101 and 102 in the image p1 are obtained in this image p6. Is set to “0” and converted to black.

  Next, the white extraction unit 25 detects an area having a predetermined threshold value or more from the image signal P6 from the black replacement unit 24, and the white area is extracted again. Here, it is assumed that “250” is set in the setting register 28 as an example of the gradation threshold value.

  The image p7 in FIG. 4 corresponds to the image signal P7 output from the white extraction unit 25. As described above, in the image signal P6 from the black replacement unit 24, the pixel values of the white waves 101 and 102 are “0”, the sea surface 103 is “10”, and the human 104 is “251”. Therefore, only the region of the human 104 is extracted as a white region from the subject of the image p6 by the processing of the white extraction unit 25 based on the above setting.

  Next, the display frame generation unit 26 generates a display frame having a predetermined shape including the white area extracted by the white extraction unit 25, and outputs an image signal P8 indicating the position of the display frame. The image p8 in FIG. 4 corresponds to the image signal P8. Here, as an example, a rectangular display frame 105 that includes the area of the human 104 extracted by the white extraction unit 25 is generated. In the image p8, only the pixel value of the area corresponding to the display frame 105 is “1”. ", And the pixel values in other areas are set to" 0 ".

  When a human is detected by the above processing and an image signal P8 indicating the detected position is output, the composition processing unit 27 synthesizes a signal indicating the display frame 105 with the image signal P2 from the visible light camera 12. Is executed. Here, as an example, among the pixels of the image signal P2 from the visible light camera 12, the pixel value of the pixel value of the image signal P8 from the display frame generation unit 26 is “1” is a predetermined color (for example, red). Converted to a value corresponding to. An image p9 in FIG. 5 corresponds to the image signal P9 output from the synthesis processing unit 27, and a display frame 105 of a predetermined color is provided on the image p9 so as to surround the detected region of the person 104. Composite display.

  According to the processing described above, even when a human is present on the water surface where white waves are standing, the human can be accurately detected. For example, the visible light camera 12 can detect an object floating on the water surface with relatively high accuracy. In particular, when the reflected light from the surroundings of the object is strong or a bright subject such as a white wave is reflected in the surroundings. Since an object on the water surface becomes relatively dark, it may be difficult to determine whether the object is a human being. On the other hand, since the far-infrared camera 11 can detect the temperature of an object, it can detect with high accuracy whether or not the object on the water surface is a human. However, when sunlight is strongly shining on the water surface, white waves on the water surface may take gradation values similar to humans, making it difficult to distinguish humans from white waves. There was a case.

  On the other hand, according to the above process, the white region is detected by detecting the white region from the image captured by the visible light camera 12, and the human detection region detected from the image captured by the far-infrared camera 11 is detected. By excluding the white wave detection region based on the visible light camera 12, it is possible to accurately detect a human being distinguished from the white wave. Such detection processing is simply performed mainly by comparison processing for comparing pixel values of each captured image signal of the far-infrared camera 11 and the visible light camera 12 with a threshold value, and logical product operation on the comparison result. It can be executed with simple processing. Therefore, for example, compared with a case where complicated processing such as pattern matching using a template corresponding to a human shape is performed, the circuit scale and processing load of the image processing circuit 13 are greatly suppressed, and the hybrid camera 1 Manufacturing cost and power consumption can be reduced, and the device itself can be reduced in size and weight.

  In the example of the above embodiment, a rectangular display frame is generated as an image for visually showing the white area extracted by the white extraction unit 25 (that is, the detection area of the human 104). For example, a display frame having another shape such as a circle may be generated. Further, the display frame may be configured by other types of lines such as dotted lines, and the display frame may blink on the composite image. Further, the display frame may be formed along the contour of the extracted region of the person 104. Further, the shape and thickness of such a display frame, the type of line, the display method, and the like may be arbitrarily set according to a user operation. Further, the display frame may be automatically set to a conspicuous color according to the color around the display frame. Further, an alarm sound may be generated together with the composite display of the display frame, or the sound signal may be output together with the image signal. Further, instead of the display frame, the color of the detected object (human 104) itself may be changed.

[Second Embodiment]
FIG. 6 is a diagram illustrating an internal configuration of the hybrid camera according to the second embodiment. In FIG. 6, blocks corresponding to those in FIG. 2 are denoted by the same reference numerals, and the description thereof is omitted.

  In the image processing circuit 13a shown in FIG. 6, the output image signals of the white extraction units 21 and 22 are arranged after the white extraction units 21 and 22 that extract white regions from the output image signals of the far-infrared camera 11 and the visible light camera 12. Is provided with a NAND (NAND) processing unit 31 that performs a NAND operation for each pixel.

  As described above, in each output image signal from the white extraction units 21 and 22, the pixel value of the white area extracted by each is set to “1”, and the pixel values of the other areas are set to “0”. Yes. Accordingly, by taking the negative logical product of these output image signals, the areas determined to be white by both the white extraction units 21 and 22 (areas of the white waves 101 and 102 in the images p3 and p4 in FIG. 3) are removed. On the other hand, only the region determined to be white (region of the human 104 in the image p3) is extracted.

  In the display frame generation unit 26, the position of the display frame is generated with respect to the white region extracted in this way, and in the synthesis processing unit 27, the detected white region (that is, the detected white region (that is, the image captured by the visible light camera 12). , A synthesized image signal in which a display frame is synthesized around a region where a human is detected is generated.

  Through the above processing, in the second embodiment as well, in the same way as in the first embodiment, it is possible to accurately detect a person by distinguishing them from white waves. Such detection processing is mainly performed by comparison processing for comparing the pixel value of each captured image signal of the far-infrared camera 11 and the visible light camera 12 with a threshold value, and a NAND operation on the comparison result. It can be executed with simple processing.

  In each of the above-described embodiments, humans are accurately detected mainly by comparison processing with a threshold value for captured image signals from two types of cameras and logical operation. On the other hand, more accurate human detection may be performed by combining known human detection processing using more complicated image processing. For example, based on the output image signal of the visible light camera, human detection processing using skin color detection or matching processing with a template image is performed, and the human detection area detected by this processing and the white extraction unit 25 in FIG. Alternatively, when the white extraction region output from the NAND processing unit 31 in FIG. 6 matches, the region may be determined as a region where a human is detected.

  In the above description, it has been described that a human can be detected separately from a white wave. However, when a far-infrared camera and a visible light camera are used, another object other than the white wave can be distinguished from a human. For example, even in an environment where light from a lamp (such as a fluorescent lamp) that is detected as almost white in both captured images of a far-infrared camera and a visible light camera is present, by excluding the detection area of this lamp by both, Only humans detected as white by an infrared camera can be accurately detected separately from electric lights.

  Moreover, in a snowy mountain etc., it is difficult to distinguish between snow and a human from a captured image signal by a visible light camera. Moreover, snow is generally detected as black in the far-infrared camera, and is detected as white in the visible light camera. Therefore, by taking the logical product of the black region extracted from the image signal captured by the far-infrared camera and the white region extracted from the image signal captured by the visible light camera, the snow region is excluded and the far-infrared region is extracted from the remaining region. Humans can also be detected by extracting a white region based on the camera.

  The camera used in combination with the visible light camera is not limited to the far-infrared camera, and a camera that detects an infrared region such as a near-infrared camera can also be used. Furthermore, not only a combination of an infrared camera and a visible light camera, but also having two cameras that can detect light (electromagnetic waves) in different frequency bands, a specific object can be distinguished from other objects. It becomes possible. That is, when the first object is detected by both of the two types of cameras, there is a second object that cannot be distinguished from the first object in the detection area of the first object by one camera. When the second object is not detected in the detection area of the first object by the other camera, the detection of the first object by the first camera is performed based on the detection area of the first object by both cameras. By accurately excluding the detection area of the first object from the area, it is possible to extract only the detection area of the second object.

  In each of the above embodiments, the case where the present invention is applied to an apparatus in which two types of cameras and an image processing circuit for object detection are integrated has been described. The present invention can also be applied to an image processing apparatus that can detect an object by connecting the camera to the outside. For example, in an information processing apparatus such as a PC, input of imaging signals from two types of cameras may be received through a communication cable or the like, and object detection processing may be performed based on these input signals.

  Such processing functions of the image processing apparatus can be realized by a computer. In that case, a program describing the contents of these processing functions is provided. These processing functions are realized on the computer by executing the program on the computer. The program describing the processing contents can be recorded on a computer-readable recording medium. Examples of the computer-readable recording medium include a magnetic recording device, an optical disk, a magneto-optical disk, and a semiconductor memory.

  In order to distribute the program, for example, portable recording media such as an optical disk and a semiconductor memory on which the program is recorded are sold. It is also possible to store the program in a storage device of a server computer and transfer the program from the server computer to another computer via a network.

  The computer that executes the program stores, for example, the program recorded on the portable recording medium or the program transferred from the server computer in its own storage device. Then, the computer reads the program from its own storage device and executes processing according to the program. The computer can also read the program directly from the portable recording medium and execute processing according to the program. Further, each time the program is transferred from the server computer, the computer can sequentially execute processing according to the received program.

It is a figure showing roughly about the composition of the hybrid camera concerning a 1st embodiment. It is a figure which shows the further detailed internal structure of the hybrid camera which concerns on 1st Embodiment. FIG. 5 is a first diagram illustrating an example of an image corresponding to an image signal in each unit in the hybrid camera. FIG. 6 is a second diagram illustrating an image example corresponding to an image signal in each unit in the hybrid camera. FIG. 10 is a third diagram illustrating an image example corresponding to an image signal in each unit in the hybrid camera. It is a figure which shows the internal structure of the hybrid camera which concerns on 2nd Embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Hybrid camera, 2 ... PC, 11 ... Far-infrared camera, 12 ... Visible light camera, 13 ... Image processing circuit, 21, 22, 25 ... White extraction part, 23 ... Logical product (AND ) Processing unit, 24... Black replacement unit, 26... Display frame generation unit, 27... Synthesis processing unit, 28... Setting register, 29.

Claims (15)

In an image processing apparatus for detecting a specific subject based on an input image signal,
A first region extraction unit that extracts a first image region whose pixel value is included in the first level band based on an infrared image signal from the infrared camera;
A second region extraction unit that extracts a second image region whose pixel value is included in a second level band based on a visible light image signal from a visible light camera that images the same direction as the infrared camera;
A third image area obtained by excluding an overlapping area between the first image area and the second image area from either the first image area or the second image area is defined as the specific subject. A third image region extraction unit that outputs as a region including
An image processing apparatus comprising:   The second area extracting unit extracts, as the second image area, an area composed of pixels included in a predetermined color area based on at least a comparison result between the second level band and a pixel value. The image processing apparatus according to claim 1, wherein: The first area extraction unit outputs a pixel area having a pixel value equal to or higher than a predetermined level as the first image area,
The second area extracting unit extracts a white area determined at least when each pixel value of each color component of the visible light image signal has a predetermined level or more as the second image area. The image processing apparatus according to claim 2.   The third area extraction unit outputs, as the third image area, an area obtained by excluding an overlapping area between the first image area and the second image area from the first image area. The image processing apparatus according to claim 3.   The third area extracting unit sets each pixel value of an overlapping area between the first image area and the second image area to a pixel value less than a predetermined upper limit threshold in the infrared image signal. The third image is extracted by extracting the image area included in the second level band in which the pixel value is set to a band equal to or higher than the upper threshold value based on the replaced image signal. The image processing apparatus according to claim 3, wherein the area is output.   The third area extracting unit determines an overlapping area between the first image area and the second image area as a logic for each image signal from the first area extracting unit and the second area extracting unit. 6. The image processing apparatus according to claim 5, wherein extraction is performed by performing product operation.   The third region extraction unit extracts the third image region by performing a NAND operation on each image signal from the first region extraction unit and the second region extraction unit. The image processing apparatus according to claim 1, wherein:   The image processing according to claim 1, further comprising an area display processing unit that performs image processing for visually notifying the user of the third image area with respect to the visible light image signal. apparatus.   The area display processing unit performs a process of synthesizing a display image for visually notifying a user of the third image area on an image based on the visible light image signal. Item 9. The image processing apparatus according to Item 8.   The image processing apparatus according to claim 9, wherein the display image is a frame disposed around the third image region.   The image processing apparatus according to claim 1, wherein the infrared image signal is an image signal in which a far infrared band is detected. In an imaging device having a function of detecting a specific subject based on a captured image signal,
An infrared camera,
A visible light camera that images the same direction as the infrared camera;
A first region extraction unit that extracts a first image region whose pixel value is included in a first level band based on an infrared image signal from the infrared camera;
A second region extraction unit that extracts a second image region whose pixel value is included in a second level band based on a visible light image signal from the visible light camera;
A third image area obtained by excluding an overlapping area between the first image area and the second image area from either the first image area or the second image area is defined as the specific subject. A third image region extraction unit that outputs as a region including
An imaging device comprising:   The imaging apparatus according to claim 12, further comprising a synchronization signal generation unit configured to generate a synchronization signal for synchronizing imaging timings of both the infrared camera and the visible light camera. In an image processing method for detecting a specific subject based on an input image signal,
The first region extraction unit extracts a first image region whose pixel value is included in the first level band based on the infrared image signal from the infrared camera,
A second region extraction unit extracts a second image region whose pixel value is included in a second level band based on a visible light image signal from a visible light camera that images the same direction as the infrared camera;
A third image area extraction unit that excludes an overlapping area between the first image area and the second image area from either the first image area or the second image area; Output the image area as an area including the specific subject.
An image processing method. In an image processing program for detecting a specific subject based on an input image signal,
Computer
A first region extraction unit for extracting a first image region whose pixel value is included in the first level band based on an infrared image signal from the infrared camera;
A second region extraction unit that extracts a second image region whose pixel value is included in a second level band based on a visible light image signal from a visible light camera that images the same direction as the infrared camera;
A third image area obtained by excluding an overlapping area between the first image area and the second image area from either the first image area or the second image area is defined as the specific subject. A third image region extraction unit that outputs as a region including
An image processing program characterized by functioning as

Images