JP2016170061A - Device, method, and program for detecting state of imaging optical system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique that can easily determine whether or not the state of an imaging optical system of a device capturing a distance image is appropriate.SOLUTION: A distance image on a detection area captured by an imaging device 10 is input to an image input unit 2. An image processing unit 3 extracts ones of pixels which are separated from each other by distances larger than a predetermined distance, in a distance image input to the image input unit 2 and a background distance image on the detection area stored in a background image storage unit 4 in the detection area, and determines whether or not the state of the imaging optical system of the imaging device 10 is appropriate based on a value associated with dispersion representing variations in distance of the extracted pixels.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a technique for determining whether or not the state of an imaging optical system of an imaging apparatus that captures a distance image of a detection area is appropriate by receiving reflected light of light irradiated to the detection area.

  Conventionally, there is a TOF (Time Of Flight) sensor as one of 3D sensors (Three Dimensional Sensors) that acquire a distance image of an imaging area (see, for example, Patent Document 1). The TOF sensor can capture a distance image and a received light intensity image of the imaging area at the same timing by projecting sinusoidal modulated light onto the imaging area and receiving the reflected light. The TOF sensor includes a light projecting unit having a light emitting element that projects sinusoidal modulated light (infrared light) to an imaging area, and a light receiving unit having an image sensor in which n × m light receiving elements are arranged in a matrix. I have.

  The TOF sensor measures the time (flight time) from when the infrared light is projected onto the imaging area until the reflected light is received by each light receiving element. The TOF sensor obtains the flight time by measuring the phase difference between the infrared light irradiated to the imaging area and the received reflected light. The TOF sensor obtains a distance image of the imaging area by calculating the distance to the reflecting surface that reflects the projected light from the flight time obtained by each light receiving element. Further, the TOF sensor acquires a received light intensity image of the imaging area from the intensity of the reflected light received by each light receiving element.

  There is also a 3D sensor (Three Dimensional Sensor) that scans an imaging area with laser light and receives reflected light thereof to acquire a distance image and a received light intensity image captured at the same timing.

  Recently, from the viewpoint of preventing the occurrence of blind spots where the presence or absence of an object cannot be detected (from the viewpoint of preventing the object from being overlooked), the above-described TOF sensor or the like is used instead of using a transmissive or reflective photoelectric sensor. It has been studied to detect a captured object (an object located within an imaging area) by processing the distance image captured in (see Patent Document 2).

Special table 2010-534000 gazette JP 2014-178186 A

  A 3D sensor such as a TOF sensor is in a state in which foreign matter adheres to the surface of a lens or lens cover that is an imaging optical system, or the lens or lens cover is damaged such as cracks (the imaging optical system is If the state is improper), the captured distance image is noisy due to the influence of scattered light. In particular, when the 3D sensor is used outdoors, dust, dust, raindrops, and the like are likely to adhere to the surface of the lens or lens cover that is the imaging optical system as foreign matter. As described above, the distance image captured in an inappropriate state of the imaging optical system has a lot of noise, and thus the object being captured cannot be detected properly.

  For this reason, there is a demand for development of a technique that can detect whether or not the imaging optical system of a 3D sensor such as a TOF sensor is in an inappropriate state.

  An object of the present invention is to provide a technique that can easily detect whether an imaging optical system of an imaging apparatus that captures a distance image is appropriate or inappropriate.

  The state detection device for an imaging optical system according to the present invention is configured as follows to achieve the above object.

  A distance image of the detection area imaged by the imaging device is input to the distance image input unit. As an image pickup apparatus that picks up a distance image of a detection area, for example, there is one using a TOF (Time Of Flight) sensor. The background distance image storage unit stores a background distance image of the detection area.

  The extraction unit, for the detection area, in the distance image input to the distance image input unit and the background distance image stored in the background distance image storage unit, a pixel whose distance difference between corresponding pixels exceeds a predetermined amount Extract. For example, for the detection area, the extraction unit generates a distance difference image between the distance image input to the distance image input unit and the background distance image stored in the background distance image storage unit, and the distance difference between corresponding pixels is Pixels exceeding the predetermined amount (ie, foreground pixels) are extracted.

  The state determination unit determines whether the state of the imaging optical system of the imaging device is appropriate based on a value relating to dispersion indicating variation in pixel distance extracted by the extraction unit in the distance image input to the distance image input unit. Determine if. The value relating to the variance may be a variance indicating variation in the distance of the pixels extracted by the extraction unit, or may be a deviation that is a positive square root of this variance.

  When the foreground pixel extracted by the extraction unit is an image of an object, the variation in the distance of the extracted foreground pixel is small. On the other hand, the foreground pixels extracted by the extraction unit are not images of the object, but are generated by foreign matter adhering to the surface of the lens or lens cover, which is the imaging optical system of the imaging device, or by damage such as cracks in the lens or lens cover. In the case of a noise pixel affected by the scattered light, the distance variation of the extracted foreground pixels becomes large.

  Therefore, it is possible to detect whether the state of the imaging optical system of the imaging apparatus is appropriate or inappropriate based on the dispersion value indicating the variation in pixel distance extracted by the extraction unit.

  For example, if the variance (or deviation) indicating the variation in pixel distance extracted by the extraction unit does not exceed a predetermined threshold (third threshold), the state of the imaging optical system of the imaging device that captures the distance image On the contrary, if the predetermined threshold value (third threshold value) is exceeded, it may be determined that the state of the imaging optical system of the imaging device that captures the distance image is not appropriate.

  When the variance (or deviation) indicating the variation in the pixel distance extracted by the extraction unit is a predetermined threshold (third threshold) (calculated variance (or deviation) = third threshold) May be determined that the state of the imaging optical system of the imaging device that captures the distance image is appropriate, or may be determined that the imaging optical state of the imaging device that captures the distance image is not appropriate. May be.

  In addition, the state determination unit is irrelevant to the dispersion value indicating the variation in the distance of the pixels extracted by the extraction unit when the number of foreground pixels extracted by the extraction unit exceeds a predetermined first threshold. It may be determined that the state of the imaging optical system of the imaging apparatus is not appropriate. The first threshold value may be a value that is 70 to 80 percent of the number of pixels of the target pixel (the pixel that is imaging the detection region).

  In the situation where most of the pixels are foreground pixels, there is a high possibility that a tape or the like is attached so as to cover the surface of the lens or lens cover that is the imaging optical system of the imaging apparatus. Therefore, even when a tape, gum, or the like is affixed so as to cover the surface of the lens or lens cover that is the imaging optical system, it can be determined that the state of the imaging optical system of the imaging apparatus is not appropriate. Further, when the number of foreground pixels extracted by the extraction unit exceeds a predetermined first threshold value, it is not necessary to calculate a value related to dispersion indicating variation in the distance of pixels extracted by the extraction unit. .

  When the number of foreground pixels extracted by the extraction unit is a predetermined first threshold (the number of foreground pixels = first threshold), the variation in the distance of the pixels extracted by the extraction unit is indicated. Regardless of the value related to dispersion, it may be determined that the state of the imaging optical system of the imaging device is not appropriate, or the imaging optics of the imaging device may be determined according to the value related to dispersion indicating the variation in pixel distance extracted by the extraction unit. It may be determined whether the state of the system is appropriate.

  In addition, the state determination unit determines the distance of the pixels extracted by the extraction unit when the number of foreground pixels extracted by the extraction unit does not exceed a predetermined second threshold (first threshold> second threshold). It may be determined that the state of the image pickup optical system of the image pickup apparatus is appropriate regardless of the value of the dispersion indicating the variation of This second threshold value may be a value that is 20 to 30 percent of the number of pixels of the target pixel (the pixel that is imaging the detection area).

  The relatively small number of foreground pixels means that the pixels (noise pixels) are affected by foreign matter adhering to the surface of the lens or lens cover that is the imaging optical system, or scattered light caused by damage such as cracks on the lens or lens cover. ) Is less. From this, if the number of foreground pixels extracted by the extraction unit does not exceed a predetermined second threshold, the imaging device regardless of the value relating to dispersion indicating the variation in the distance of the pixels extracted by the extraction unit It can be determined that the state of the imaging optical system is appropriate. In addition, when the number of foreground pixels extracted by the extraction unit does not exceed the predetermined second threshold, it is not necessary to calculate a value related to dispersion indicating variation in the distance of the pixels extracted by the extraction unit. .

  In the case where the number of foreground pixels extracted by the extraction unit is a predetermined second threshold value (the number of foreground pixels = second threshold value), the variation in the distance of the pixels extracted by the extraction unit is indicated. It may be determined that the state of the imaging optical system of the imaging device is appropriate regardless of the value related to dispersion, and the imaging device captures an image based on the value related to dispersion indicating the variation in pixel distance extracted by the extraction unit. It may be determined whether the state of the optical system is appropriate.

  Furthermore, the state detection device may be configured to include an output unit that outputs that when the state determination unit determines that the state of the imaging optical system of the imaging device is not appropriate. If comprised in this way, the administrator can recognize the imaging device whose imaging optical system state is inadequate at an early stage.

  According to the present invention, it is possible to easily detect whether the state of the imaging optical system of the imaging apparatus that captures a distance image is appropriate or inappropriate.

It is a block diagram which shows the structure of the principal part of a state detection apparatus. 2A and 2B are diagrams illustrating a known technique for obtaining a distance to an object. It is a figure explaining the imaging area of the distance image imaged with an imaging device. It is a flowchart which shows the state detection process of the imaging optical system in a state detection apparatus. It is a figure explaining the imaging area of the distance image imaged with an imaging device It is a figure which shows a difference image. It is a figure explaining the imaging area of the distance image imaged with an imaging device It is a figure which shows a difference image. It is a figure which shows a difference image. It is a figure which shows a difference image. It is a figure which shows the determination area | region whether the state of the imaging optical system of an imaging device is appropriate.

  Hereinafter, the state detection apparatus which is embodiment of this invention is demonstrated.

  FIG. 1 is a block diagram illustrating a configuration of a main part of the state detection device according to this example. An imaging device 10 is connected to the state detection device 1. The state detection device 1 detects (determines) whether the state of the imaging optical system of the connected imaging device 10 is appropriate or inappropriate, and outputs that fact if it is inappropriate. As shown in FIG. 1, the state detection device 1 includes an image input unit 2, an image processing unit 3, a background image storage unit 4, a threshold storage unit 5, and an output unit 6.

  The imaging device 10 connected to the state detection device 1 has a TOF (Time Of Flight) sensor and captures a distance image in an imaging area. Specifically, the imaging device 10 includes a light source that irradiates an imaging area with infrared light, and an imaging element in which n × m light receiving elements are arranged in a matrix. The imaging device 10 measures, for each pixel, the time (flight time) from irradiating the imaging area with infrared light to receiving the reflected light. The flight time is obtained from the phase difference between the light applied to the imaging area and the received reflected light. The imaging device 10 obtains the distance from the flight time measured for each pixel to the object (the reflection surface of the object reflecting the irradiation light). For example, the imaging apparatus 10 can capture a distance image of about 5 to 10 frames per second.

Although known, a method for obtaining the distance to the object will be briefly described. The light emitted from the light source is modulated in emission intensity. When receiving the reflected light from the imaging area, the modulation phase is shifted according to the propagation distance. The phase of the irradiated light is monitored by directly receiving a part of the light from the light source by a part of the light receiving element, and a deviation from the phase of the light received as reflected light is obtained. As shown in FIG. 2 (A), the phase shift includes received light signals (A0, A2) sampled every T / 2 period with respect to the modulation period T of the irradiation light, and further, as shown in FIG. 2 (B). Based on the received light signals (A1, A3) sampled at the timing shifted by T / 4, the phase shift amount φ is calculated based on the propagation distance. The amount of phase shift φ is
φ = arctan {(A3-A1) / (A0-A2)}
Can be calculated.

Further, the distance D to the object can be obtained from the phase shift amount φ obtained here. The distance D to the object is
D = Lmax × φ / 2π
Can be calculated. Here, Lmax is the reciprocating distance (measurement maximum distance) to the object when φ = 2π, and Lmax is 7.5 m if the modulation frequency is 20 MHz and 15 m if 10 MHz.

  Here, the imaging device will be described. As described above, the image pickup element has n × m light receiving elements arranged in a matrix. In this example, two adjacent vertical and horizontal light receiving elements (a total of four) are regarded as one set, and this is handled as one pixel. Each of the light receiving elements accumulates the photoelectrically converted charge at the sampling timing shifted every T / 4 period. As a result, the light receiving signals A0, A1, A2, and A3 described above can be obtained based on the accumulated charge for each T / 4 period. As another method, there is a method in which two light receiving elements are treated as one pixel and the above-described light receiving signals A0 to A3 are obtained.

  As used herein, the term “pixel” refers to a block of light receiving elements that is a unit for performing image processing to detect an object by obtaining a phase shift or distance, and is a single light receiving element. Alternatively, it may be a block composed of a plurality of adjacent light receiving elements (for example, two light receiving elements in the vertical and horizontal directions).

  Since the amount of light received for only one period of the above-described modulation of the irradiation light is too small, the imaging device 10 sets the exposure time as appropriate and calculates the phase shift using the amount of charge accumulated during that period. Get a distance image.

  Further, the imaging device 10 collects all accumulated charges for a predetermined period (for a plurality of periods) for each pixel, thereby associating the intensity of the reflected light (the amount of reflected light) received by the pixel with respect to each pixel. You can also get That is, the imaging device 10 can acquire a distance image of the imaging area and a received light intensity image captured at the same exposure timing (exposure period). However, since the state detection device 1 according to this example is configured to detect whether the state of the imaging optical system of the imaging device 10 is appropriate or not, without using a light reception intensity image, the light reception intensity. Description of the image is omitted.

  The imaging device 10 may have a configuration including a light source that emits laser light, a light receiving element that receives reflected light, and a scanning unit that scans laser light emitted from the light source in the imaging area. The imaging device 10 is not particularly limited as long as it is configured to capture a distance image of the imaging area.

  A distance image of the imaging area captured by the imaging device 10 is input to the image input unit 2. FIG. 3 is a diagram illustrating an imaging area of the imaging device. Here, it is an example in which the imaging area of the imaging device 10 is the end of the station platform. However, FIG. 3 does not show a distance image captured by the imaging device 10 (a visible light image is shown for convenience).

  In FIG. 3, a region surrounded by a broken line is a detection area for detecting the presence or absence of an object. The detection area is set by the administrator, and may be the entire imaging area of the imaging device 10 or may be a part of the imaging area of the imaging device 10 as shown in FIG. The image input unit 2 corresponds to the distance image input unit referred to in the present invention.

  The image processing unit 3 processes the distance image of the imaging area input to the image input unit 2 and determines whether the state of the imaging optical system of the imaging device 10 is appropriate or inappropriate. The image processing unit 3 has a configuration according to the present invention related to the extraction unit and the state determination unit. In addition, the image processing unit 3 executes the state detection method of the imaging optical system referred to in the present invention. Furthermore, the image processing unit 3 includes a computer for installing the state detection program for the imaging optical system according to the present invention.

  The background image storage unit 4 stores the background image of the detection area shown in FIG. The background image stored in the background image storage unit 4 is a distance image of the detection area. The background image stored in the background image storage unit 4 is updated at an appropriate timing on condition that the state of the imaging optical system of the imaging device 10 is detected to be appropriate. The background image storage unit 4 corresponds to the background distance image storage unit referred to in the present invention.

  The background image storage unit 4 may store a background image (distance image) of the imaging area shown in FIG. In this example, the background image of the detection area is stored in the background image storage unit 4 from the viewpoint of reducing the storage capacity of the background image storage unit 4.

  The threshold value storage unit 5 stores three threshold values (an upper limit pixel number threshold value, a lower limit pixel number threshold value, and a dispersion threshold value) used for determination for detecting the state of the imaging optical system of the imaging device 10. The upper limit pixel number threshold corresponds to the first threshold referred to in the present invention. The lower limit pixel count threshold corresponds to the second threshold referred to in the present invention. The dispersion threshold corresponds to the third threshold referred to in the present invention. The upper limit pixel number threshold is 70 to 80 percent of the total number of pixels in the detection area shown in FIG. The lower limit pixel count threshold is 20 to 30 percent of the total number of pixels in the detection area shown in FIG. The dispersion threshold is 20-30.

  When the image processing unit 3 detects that the state of the imaging optical system of the imaging device 10 is inappropriate, the output unit 6 outputs a message to that effect to a connected alarm device (not shown). When there is an input from the output unit 6 that the state of the imaging optical system of the imaging device 10 is inappropriate, the alarm device notifies the user that the state of the imaging optical system of the imaging device 10 is inappropriate by voice or the like. . The administrator recognizes that the state of the imaging optical system of the imaging device 10 is inappropriate by notification from the alarm device.

  Hereinafter, the operation of the state detection apparatus 1 according to this example will be described.

  FIG. 4 is a flowchart showing the state detection process of the imaging optical system in the state detection apparatus according to this example. When the distance image of the imaging area imaged by the imaging device 10 is input to the image input unit 2 (s1), the state detection device 1 cuts out the distance image of the detection area shown in FIG. 3 from this distance image (s2). ). When the detection area is the entire imaging area of the imaging device 10, the process relating to s2 is not necessary. The image processing unit 3 performs processing related to s2.

  The image processing unit 3 generates a difference image (distance difference image) between the background image (distance image) of the detection area stored in the background image storage unit 4 and the distance image of the detection area cut out in s2 (s3). ). In s3, the distance difference between corresponding pixels exceeds a predetermined amount in the background image (distance image) of the detection area stored in the background image storage unit 4 and the distance image of the detection area cut out in s2. A binary image is generated in which a pixel is a foreground pixel, and a pixel whose distance difference between corresponding pixels does not exceed a predetermined amount is a background pixel. When the distance difference between corresponding pixels is a predetermined amount, it may be extracted as a foreground pixel or a background pixel.

For example, when the distance image input in s1 is a distance image captured by the imaging device 10 when the umbrella is positioned in the detection area (when in the state illustrated in FIG. 5), the distance image illustrated in s3 6 is generated. In FIG. 6, the pixels indicated by hatching are foreground pixels, and the others are background pixels. The foreground pixels shown in FIG. 6 are pixels corresponding to the umbrella shown in FIG. Since the umbrella has a small outer shape and is located in a limited area in the real space, the variance δ ² indicating the variation in the distance of the foreground pixels shown in FIG. 6 is small.

When the distance image input in s1 is a distance image captured by the imaging device 10 when a person is located in the detection area (when in the state illustrated in FIG. 7), the distance image illustrated in s3 A difference image shown in FIG. 8 is generated. In FIG. 8, the pixels indicated by hatching are foreground pixels, and the others are background pixels. The foreground pixels shown in FIG. 8 are pixels corresponding to the person shown in FIG. Since the outer shape of a person is larger than that of the umbrella described above, the variance δ ² indicating the variation in foreground pixel distance shown in FIG.

  In the example illustrated in FIGS. 5 to 8, the state of the imaging optical system of the imaging device 10 is appropriate.

In addition, when dust, dirt, raindrops, or the like are attached to the surface of the lens or lens cover of the imaging optical system of the imaging apparatus 10, the object is located in the detection area when the state of the imaging optical system is inappropriate. Even if it is not, a difference image as shown in FIG. 9 is generated in s3 due to diffusion of light caused by dust, dirt, raindrops, or the like adhering to the surface of the lens or lens cover. In FIG. 9, pixels indicated by hatching are foreground pixels, and the others are background pixels. Since the foreground pixels shown in FIG. 9 are noise pixels, the variance δ ² indicating the variation in distance between these foreground pixels is relatively large.

Further, when a tape is applied so as to cover the entire surface of the lens or lens cover of the image pickup optical system of the image pickup apparatus 10 with mischief, in s3, as shown in FIG. A difference image is generated. As shown in FIG. 10, when the tape is applied so as to cover the entire surface of the lens and lens cover of the imaging optical system of the imaging device 10, even if the object is not located in the detection area, As shown in FIG. 10, almost all pixels in the detection area become foreground pixels. The foreground pixels shown in FIG. 10 are noise pixels, but the variance δ ² indicating the variation in distance between these foreground pixels is relatively small.

  In addition, even when a damage such as a crack occurs in the lens or lens cover of the imaging optical system of the imaging apparatus 10, the difference image shown in FIG. 9 or FIG. 10 is obtained depending on the degree of the damage.

  The image processing unit 3 counts the number of foreground pixels in the difference image generated in s3 (s4). The image processing unit 3 determines whether or not the number of foreground pixels counted in s4 exceeds an upper limit pixel number threshold stored in the threshold storage unit 5 (s5). If the image processing unit 3 determines that the number of foreground pixels counted in s4 exceeds the upper limit pixel number threshold stored in the threshold storage unit 5, the state of the imaging optical system of the imaging device 10 is inappropriate. It is determined that there is (s10), and the output unit 6 outputs that fact to the alarm device (s11).

  As described above, the image processing unit 3 performs the determination of s5, so that, for example, a mischief is attached with a tape or the like so as to cover the entire surface of the lens or the lens cover of the imaging optical system of the imaging device 10. Thus, when a differential image as shown in FIG. 10 is generated in s3, it can be detected that the state of the imaging optical system of the imaging device 10 is inappropriate.

  In s5, when the number of foreground pixels counted in s4 and the upper limit pixel number threshold stored in the threshold storage unit 5 are the same (when the number of foreground pixels = the upper limit pixel number threshold), s10 Therefore, it may be determined that the state of the imaging optical system of the imaging device 10 is inappropriate, or the processing after s6 may be performed.

  If the image processing unit 3 determines in s5 that the number of foreground pixels counted in s4 does not exceed the upper limit pixel number threshold stored in the threshold storage unit 5, the number of foreground pixels counted in s4 is It is determined whether or not the lower limit pixel count threshold stored in the threshold storage unit 5 is exceeded (s6). If the image processing unit 3 determines that the number of foreground pixels counted in s4 does not exceed the lower limit pixel number threshold stored in the threshold storage unit 5, the state of the imaging optical system of the imaging device 10 is appropriate. (S9).

The image processing unit 3 determines that the state of the imaging optical system of the imaging device 10 is appropriate when there are few foreground pixels, that is, when there are few noise pixels. Further, by appropriately setting the lower limit pixel number threshold, the image processing unit 3 captures an image in s9 even when an object having a relatively small outer shape (for example, an umbrella shown in FIG. 5) is located in the detection area. It can be detected that the state of the imaging optical system of the apparatus 10 is appropriate.

  In s6, when the number of foreground pixels counted in s4 is the same as the lower limit pixel number threshold stored in the threshold storage unit 5 (when the number of foreground pixels = the lower limit pixel number threshold), s9 Thus, it may be determined that the state of the imaging optical system of the imaging apparatus 10 is appropriate, or the processing after s7 may be performed.

If the image processing unit 3 determines in s6 that the number of foreground pixels counted in s4 exceeds the lower limit pixel number threshold stored in the threshold storage unit 5, the variance δ indicating variation in the distance of the foreground pixels ² is calculated (s7). This variance δ ² can be calculated by [Equation 1].

In [Expression 1], N is the number of foreground pixels (the number of foreground pixels counted in s4).
x _i (i = 1... N) is the distance of each foreground pixel;
m is the average of the foreground pixel distances.

The image processing unit 3 determines whether or not the variance δ ² calculated in s7 exceeds the variance threshold stored in the threshold storage unit 5 (s8). If the image processing unit 3 determines that the dispersion threshold is exceeded in s8, it determines that the state of the imaging optical system of the imaging device 10 is inappropriate in s10, and the output unit 6 informs the alarm device in s11. Output. The image processing unit 3 performs the determination in s8, for example, dust, pride, raindrops, and the like are attached to the surface of the lens or lens cover of the imaging optical system of the imaging device 10, and the process shown in FIG. When such a difference image is generated, it can be detected that the state of the imaging optical system of the imaging apparatus 10 is inappropriate.

  On the other hand, if the image processing unit 3 determines that the dispersion threshold is not exceeded in s8, it determines that the state of the imaging optical system of the imaging device 10 is appropriate in s9. When the image processing unit 3 performs the determination in s8, for example, as shown in FIG. 7, a person is located in the detection area, and a difference image as shown in FIG. 8 is generated in s3. Even so, it can be detected that the state of the imaging optical system of the imaging apparatus 10 is appropriate.

In s8, when the variance δ ² calculated in s7 is the same as the variance threshold stored in the threshold storage unit 5 (when variance δ ² = the variance threshold), the imaging of the imaging device 10 is performed in s9. It may be determined that the state of the optical system is appropriate, or it may be determined in s10 that the state of the imaging optical system of the imaging device 10 is inappropriate.

Summarizing the above description, the state detection apparatus 1 is configured such that the first inappropriate region (the region where the number of foreground pixels exceeds the upper limit pixel number threshold) or the second inappropriate region (foreground pixels) illustrated in FIG. Of the imaging optical system of the imaging apparatus 10 when the number of pixels is any one of the upper limit pixel count threshold and the lower limit pixel count threshold and the foreground pixel variance δ ² exceeds the variance threshold) Is detected as inappropriate. In other words, it is detected that the state of the imaging optical system of the imaging apparatus 10 is appropriate when it is neither the first inappropriate region nor the second inappropriate region shown in FIG.

  As described above, the state detection device 1 can detect whether the state of the imaging optical system of the imaging device 10 is appropriate or inappropriate by simple processing.

Note that the order of determination of s5, s6, and s8 shown in FIG. 4 may not be the order described above. For example, the state detection device 1 (image processing unit 3) may be configured to perform the determination in the order of s6, s5, and s8, the order of s8, s5, and s6. Further, when the determination relating to s8 is not performed, the processing relating to the calculation of the variance δ ² relating to s7 may not be performed.

In addition, the determination relating to s8 may use deviation δ (positive square root of variance δ ² ) instead of variance δ ² described above. When the deviation δ is used, the threshold storage unit 5 stores a deviation threshold instead of the dispersion threshold.

  If the image processing unit 3 determines that the imaging optical system of the imaging device 10 is appropriate in s9, the image processing unit 3 positions the foreground pixel on the difference image created in s3 and the distance of the foreground pixel. Grouping may be performed to determine the type of the object being picked up (object located in the detection area) by pattern matching or the like with respect to the grouped foreground pixel block.

DESCRIPTION OF SYMBOLS 1 ... State detection apparatus 2 ... Image input part 3 ... Image processing part 4 ... Background image storage part 5 ... Threshold storage part 6 ... Output part 10 ... Imaging device

Claims (11)

A distance image input unit for inputting a distance image of the detection area imaged by the imaging device;
A background distance image storage unit for storing a background distance image of the detection area;
For the detection area, a pixel in which a distance difference between corresponding pixels exceeds a predetermined amount in the distance image input to the distance image input unit and the background distance image stored in the background distance image storage unit An extraction unit for extracting
Whether or not the state of the imaging optical system of the imaging device is appropriate based on a value relating to dispersion indicating variation in pixel distance extracted by the extraction unit in the distance image input to the distance image input unit And a state determination unit for determining the state of the imaging optical system.   If the number of pixels extracted by the extraction unit exceeds a predetermined first threshold value, the state determination unit is irrelevant to the dispersion value indicating the variation in the distance of the pixels extracted by the extraction unit. The imaging optical system state detection device according to claim 1, wherein the imaging optical system state of the imaging device is determined to be inappropriate.   If the number of pixels extracted by the extraction unit exceeds the first threshold, the state determination unit does not calculate a value related to dispersion indicating variation in the distance of pixels extracted by the extraction unit. Item 3. The imaging optical system state detection device according to Item 2.   If the number of pixels extracted by the extraction unit does not exceed a predetermined second threshold, the state determination unit determines that the state of the imaging optical system of the imaging device is appropriate regardless of the value related to the dispersion The state detection device for an imaging optical system according to claim 1, wherein it is determined that   If the number of pixels extracted by the extraction unit is smaller than the predetermined first threshold value and does not exceed a second threshold value, the state determination unit may perform the imaging regardless of the value related to the dispersion. The imaging optical system state detection device according to claim 2, wherein the imaging optical system state of the apparatus is determined to be appropriate.   6. The imaging optical system according to claim 4, wherein the state determination unit does not calculate a value related to the dispersion unless the number of pixels extracted by the extraction unit exceeds the second threshold. Condition detection device.   The said state determination part determines with the state of the imaging optical system of the said imaging device being inadequate if the value concerning the said dispersion | distribution exceeds the predetermined 3rd threshold value. The state detection apparatus of the imaging optical system described.   The extraction unit generates a distance difference image between the distance image input to the distance image input unit and the background distance image stored in the background distance image storage unit with respect to the detection area, and between corresponding pixels The state detection device for an imaging optical system according to claim 1, wherein a pixel whose distance difference exceeds a predetermined amount is extracted as a foreground pixel.   The state detection device for an imaging optical system according to any one of claims 1 to 8, further comprising an output unit that outputs a message to the effect that the state determination unit determines that the state of the imaging optical system of the imaging device is not appropriate. . In the distance image of the detection area captured by the imaging device and input to the distance image input unit and the background distance image of the detection area stored in the background distance image storage unit, the distance difference between corresponding pixels has a predetermined amount. An extraction step of extracting pixels that exceed,
Whether or not the state of the imaging optical system of the imaging apparatus is appropriate based on a value relating to dispersion indicating variation in pixel distance extracted in the extraction step in the distance image input to the distance image input unit A state determination step for determining the state of the imaging optical system. In the distance image of the detection area captured by the imaging device and input to the distance image input unit and the background distance image of the detection area stored in the background distance image storage unit, the distance difference between corresponding pixels has a predetermined amount. An extraction step of extracting pixels that exceed,
Whether or not the state of the imaging optical system of the imaging apparatus is appropriate based on a value relating to dispersion indicating variation in pixel distance extracted in the extraction step in the distance image input to the distance image input unit A state determination step for determining the state of the imaging optical system, causing the computer to execute a state determination step.

Images