JP4466366B2 - Human body detection method and human body detection apparatus using range image - Google Patents

Description

  The present invention relates to a human body detection method and a human body detection device using a distance image.

  2. Description of the Related Art Conventionally, there has been provided a human body detecting device that is attached to, for example, a ceiling of a passage and detects the presence or the number of passers-by passing through a lower passage.

For example, in the human body detection device disclosed in Patent Document 1, a contour line is created by connecting pixels having substantially the same distance from the imaging unit based on a distance image obtained by imaging the detection target region by the imaging unit. The region surrounded by the contour lines is determined as a unimodal object (that is, a human body).
JP 2003-57007 A (paragraph numbers [0021] to [0023] and FIG. 3)

  However, even if there is a person in the imaging area, depending on the posture of the person, there may not be pixels that have substantially the same distance from the imaging means, and there are pixels that have substantially the same distance from the imaging means. However, since the contour lines formed by connecting the pixels may not close, the human body detection device described above may not be able to reliably detect a person in the detection target area.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a human body detection method and a human body detection device using a distance image that can reliably detect a person in a detection target region. Is to provide.

In order to achieve the above object, the invention of claim 1 is a human body detection method for detecting a human body in a target area using a distance image having a pixel value as a distance from an imaging means to an object in the target area. After obtaining the distance image by imaging the lower target area with the imaging means installed above the target area, the pixel area whose pixel value is below the threshold is extracted from the plurality of pixels constituting the distance image Each time the region extraction processing is performed a plurality of times by gradually increasing the threshold value from the pixel value of the pixel having the shortest pixel value, the newly extracted pixel region is at least partially compared to the already extracted pixel region. In the case of overlapping, the plurality of pixel areas are determined to be the same object, and an identifier assignment process is performed to assign the same identifier to the whole of the plurality of pixel areas, and is detected first in the plane of the distance image. Center of gravity of the pixel area Through, keep asking the two straight lines perpendicular to each other, among the plurality of pixel areas assigned the same identifier, if the newly extracted pixel region, is already a pixel area extracted encompassed, new A determination condition that an average value of four widths overlapping the two straight lines is obtained in a band-like portion excluding the pixel region that has already been extracted from the pixel region that has been extracted, and the average value is equal to or less than a predetermined threshold value When is established, the region extraction processing and the identifier assignment processing are terminated, and a pixel region to which the same identifier is assigned is extracted as a human body.

Here, when the human body of the target area is photographed from above, when the pixel area is extracted in order from the side closer to the imaging means, the same identifier is assigned as the height of the horizontal section of the head approaches the maximum position. The added pixel area is reduced. According to the first aspect of the present invention, when a newly extracted pixel area is included in a newly extracted pixel area among a plurality of pixel areas assigned the same identifier, the newly extracted pixel area If the determination condition that the average value of the width of the strip-shaped portion outside the already extracted pixel region is equal to or less than a predetermined threshold is satisfied, it can be determined that the shape of the human head can be detected, By detecting the shape of the human body, the human body can be reliably detected, and the human body can be reliably detected even when the contour lines are not closed as in the conventional human body detection method. In addition, since the region extraction process and the identifier assignment process have been completed for the pixel area to which the identifier is assigned when the determination condition is satisfied, compared with the case where the pixel area is extracted by extending the threshold value to the floor surface. Therefore, there is an advantage that the memory capacity for storing the pixel region to which the identifier is added can be reduced.

The invention according to claim 2 is the human body detection method using the distance image according to claim 1, wherein the number of pixels in a plurality of pixel areas to which the same identifier is assigned corresponds to the size of the human body. When both of the determination conditions that the number of pixels is exceeded are satisfied, the region extraction processing and the identifier assignment processing are terminated, and a pixel region to which the same identifier is assigned is extracted as a human body.

According to this invention, in the human body detection method according to claim 1, only when the determination condition that the number of pixels in a plurality of pixel areas to which the same identifier is assigned exceeds the number of pixels corresponding to the size of the human body is satisfied. Since the pixel area to which the same identifier is assigned is extracted as a human body, it is possible to prevent erroneous detection of an area smaller than the human body.

The invention of claim 3 is the invention according to any one of claims 1 and 2, wherein a pixel in the pixel area to which the same identifier is assigned among the plurality of pixels constituting the distance image, and the others These pixels are binarized.

  According to the present invention, since the pixels constituting the distance image are binarized into the pixels in the pixel area to which the same identifier is assigned and the other pixels, the process of extracting the pixel area corresponding to the human body In addition, the process of specifying the pixel area can be easily performed.

According to a fourth aspect of the present invention, there is provided a human body detection device, wherein the target region is obtained by executing the human body detection method according to any one of the first to third aspects using the distance image captured by the imaging unit. A human body detecting means for detecting the human body inside.

  According to the present invention, it is possible to realize a human body detection device that can reliably detect a human body and shorten the detection time.

  By the way, when the human body of the target area is photographed from above, if the pixel area is extracted in order from the side closer to the imaging means, the pixels assigned the same identifier as the head approaches the height position where the horizontal cross section becomes maximum. Less space is added.

According to the first aspect of the present invention, when a newly extracted pixel area is included in a newly extracted pixel area among a plurality of pixel areas assigned the same identifier, the newly extracted pixel area If the determination condition that the average value of the width of the strip-shaped portion outside the already extracted pixel region is equal to or less than a predetermined threshold is satisfied, it can be determined that the shape of the human head can be detected, By detecting the shape of the human body, the human body can be reliably detected, and the human body can be reliably detected even when the contour lines are not closed as in the conventional human body detection method. In addition, since the region extraction process and the identifier assignment process have been completed for the pixel area to which the identifier is assigned when the determination condition is satisfied, compared with the case where the pixel area is extracted by extending the threshold value to the floor surface. Therefore, there is an advantage that the memory capacity for storing the pixel region to which the identifier is added can be reduced.

According to a second aspect of the present invention, in the human body detection method according to the first aspect, the determination condition that the number of pixels of a plurality of pixel areas to which the same identifier is assigned exceeds the number of pixels corresponding to the size of the human body is established. Only in this case, since the pixel area to which the same identifier is assigned is extracted as a human body, it is possible to prevent erroneous detection of a pixel having a size smaller than that of the human body.

According to the invention of claim 3 , since the pixels constituting the distance image are binarized into the pixels in the pixel region to which the same identifier is assigned and the other pixels, the pixel region corresponding to the human body is The extraction process and the process of specifying the pixel area can be easily performed.

According to the invention of claim 4 , it is possible to realize a human body detection device that can reliably detect a human body and shorten the detection time.

  In the following, an embodiment of a human body detection apparatus that executes a human body detection method using a distance image according to the present invention will be described. First, a basic configuration of a distance image sensor used in the human body detection apparatus will be described with reference to FIG. explain.

  As shown in FIG. 1, the distance image sensor 10 includes a light emitting source 2 that irradiates light to a target space, and receives light from the target space and obtains an electrical output having an output value that reflects the amount of received light. Device 1 is provided. The distance to the object Ob existing in the object space is the time from when the light is emitted from the light source 2 to the object space until the reflected light from the object Ob enters the light detection element 1 (“flight time”). Call). However, since the flight time is very short, use the intensity-modulated light that is modulated so that the intensity of the light irradiating the target space changes periodically at a constant period, and use the phase when the intensity-modulated light is received. Ask for time. The technical idea of the present invention can be adopted not only in the configuration in which the distance image is generated by the time of flight, but also in the configuration in which the distance image is generated by the principle of triangulation method. However, the distance image sensor 10 having the configuration described below adopts the principle of triangulation because it can generate a distance image in a short time (almost real time) compared to the distance image sensor using the principle of triangulation. It is preferable to the distance image sensor.

  As shown in FIG. 9A, if the intensity of light radiated from the light source 2 into the space changes as shown by curve A, and the amount of received light received by the light detection element 1 changes as shown by curve B, Since the phase difference ψ corresponds to the flight time, the distance to the object Ob can be obtained by obtaining the phase difference ψ. Further, the phase difference ψ can be calculated using the received light quantity of the curve B obtained at a plurality of timings of the curve A. For example, it is assumed that the received light amounts of curve B obtained with the phases of curve A at 0, 90, 180, and 270 degrees are A0, A1, A2, and A3 (received light amounts A0, A1, A2,. A3 is indicated by hatching). However, the received light quantity A0, A1, A2, A3 in each phase is not an instantaneous value but a received light quantity integrated over a predetermined light receiving period Tw. Now, while obtaining the received light amounts A0, A1, A2, and A3, the phase difference ψ does not change (that is, the distance to the object Ob does not change), and the reflectance of the object Ob does not change. Shall. Further, it is assumed that the intensity of light emitted from the light emitting source 2 is modulated by a sine wave, and the intensity of light received by the light detection element 1 at time t is represented by A · sin (ωt + δ) + B. Here, A is the amplitude, B is the DC component (average value of the external light component and the reflected light component), ω is the angular frequency, and δ is the initial phase. The received light amounts A0, A1, A2, and A3 received by the light detection element 1 are instantaneous values, not integrated values of the light receiving period Tw, and time t = n / f (n = 0, 1, 2,. .., F is the modulation frequency), and the received light quantity A0 = A · sin (δ) + B, the received light quantity A0, A1, A2, A3 can be expressed as follows. The reflected light component means a component of light emitted from the light emitting source 2 and incident on the light detection element 1 after being reflected by the object Ob.

A0 = A · sin (δ) + B
A1 = A · sin (π / 2 + δ) + B
A2 = A · sin (π + δ) + B
A3 = A · sin (3π / 2 + δ) + B
In FIG. 9, since the phase difference is ψ, the initial phase δ (phase at time t = 0) of the waveform related to the amount of light received by the light detection element 1 is −ψ. That is, since δ = −ψ, A0 = −A · sin (ψ) + B, A1 = A · cos (ψ) + B, A2 = A · sin (ψ) + B, A3 = −A · cos (ψ) As a result, the relationship between each received light quantity A0, A1, A2, A3 and the phase difference ψ is expressed by the following equation.

ψ = tan ⁻¹ {(A2−A0) / (A1−A3)} (1)
In equation (1), the instantaneous values of the received light amounts A0, A1, A2, and A3 are used. However, even if the integrated values in the light receiving period Tw are used as the received light amounts A0, A1, A2, and A3, the phase difference in equation (1) ψ can be obtained.

  Further, when the intensity of light received by the light detection element 1 is A · cos (ωt + δ) + B, that is, at time t = n / f (n = 0, 1, 2,...) Synchronized with the modulation period. If the received light quantity is A0 = A · cos (δ) + B, the phase difference ψ can be obtained by the following equation.

ψ = tan ⁻¹ {(A1−A3) / (A0−A2)}
This relationship is a relationship in which the timing synchronized with the modulation period is shifted by 90 degrees. In addition, since the sign of the distance value is positive, if the sign is negative when the phase difference ψ is obtained, the order of the denominator in the parenthesis of tan ⁻¹ or each term of the numerator is changed, or An absolute value may be used.

  As described above, in order to modulate the intensity of the light irradiated to the target space, the light source 2 includes, for example, a structure in which a large number of light emitting diodes are arranged on one plane, a combination of a semiconductor laser and a diverging lens, or the like. Is used. The light source 2 is driven by a modulation signal having a predetermined modulation frequency output from the control circuit unit 3, and the intensity of the light emitted from the light source 2 is modulated by the modulation signal. The control circuit unit 3 modulates the intensity of light emitted from the light source 2 with, for example, a 20 MHz sine wave. The intensity of the light emitted from the light source 2 may be modulated by a triangular wave, a sawtooth wave or the like in addition to the modulation by a sine wave. In short, any configuration is acceptable as long as the intensity is modulated at a constant period. May be adopted.

  The light detection element 1 includes a plurality of photosensitive portions 11 regularly arranged. A light receiving optical system 19 is disposed on the light incident path to the photosensitive portion 11. The photosensitive part 11 is a part where light from the target space enters through the light receiving optical system 19 in the light detection element 1, and the photosensitive part 11 generates an amount of electric charge corresponding to the amount of received light. Further, the photosensitive portions 11 are arranged on the lattice points of the planar lattice, and are arranged in a matrix in which, for example, a plurality are arranged at equal intervals in the vertical direction (that is, the vertical direction) and the horizontal direction (that is, the horizontal direction). Is done.

  The light receiving optical system 19 associates the line-of-sight direction when viewing the target space from the light detection element 1 with each photosensitive portion 11. That is, the range in which light enters each photosensitive portion 11 through the light receiving optical system 19 can be regarded as a conical field of view having a small apex angle set for each photosensitive portion 11 with the center of the light receiving optical system 19 as the apex. . Therefore, if the reflected light emitted from the light emitting source 2 and reflected by the object Ob existing in the target space enters the photosensitive portion 11, the optical axis of the light receiving optical system 19 depends on the position of the photosensitive portion 11 that has received the reflected light. Can be known as the reference direction.

  Since the light receiving optical system 19 is generally arranged so that the optical axis is orthogonal to the plane on which the photosensitive portion 11 is arranged, the center of the light receiving optical system 19 is the origin, and the vertical and horizontal directions of the plane on which the photosensitive portion 11 is arranged If an orthogonal coordinate system is set in which the optical axis of the light receiving optical system 19 is in the direction of the three axes, the angle (so-called azimuth and elevation angle) when the position of the object Ob existing in the target space is expressed in spherical coordinates. It corresponds to each photosensitive portion 11. The light receiving optical system 19 can also be arranged so that the optical axis intersects at an angle other than 90 degrees with respect to the plane on which the photosensitive portions 11 are arranged.

  In the present embodiment, as described above, in order to obtain the distance to the object Ob, the received light amounts A0, A1, and A2 are synchronized at four timings synchronized with the intensity change of the light emitted from the light source 2 to the target space. , A3. Therefore, it is necessary to control the timing to obtain the desired received light amount A0, A1, A2, A3. In addition, since the amount of charge generated in the photosensitive portion 11 is small in one cycle of the intensity change of light irradiated from the light source 2 to the target space, it is desirable to accumulate the charges over a plurality of cycles. Therefore, as shown in FIG. 1, a plurality of charge accumulating units 13 for accumulating charges generated in the respective photosensitive units 11 are provided, and a plurality of sensitivity control units 12 for adjusting the sensitivity of the respective photosensitive units 11 are provided. Yes.

  In each sensitivity control unit 12, the sensitivity of the photosensitive unit 11 corresponding to the sensitivity control unit 12 is increased at any one of the four points described above, and in the photosensitive unit 11 with increased sensitivity, the received light amount A0, Since charges corresponding to A1, A2, and A3 are mainly generated, charges corresponding to the received light amounts A0, A1, A2, and A3 can be accumulated in the charge accumulating unit 13 corresponding to the photosensitive unit 11.

  Hereinafter, as a specific configuration of the sensitivity control unit 12, a technique for adjusting a ratio of charges given to the charge accumulating unit 13 among charges generated by the photosensitive unit 11, and a part that substantially functions as the photosensitive unit 11 will be described. The technology to change the area. The technique for adjusting the ratio of charges given to the charge accumulating unit 13 includes a technique for adjusting the passing rate from the photosensitive unit 11 to the charge accumulating unit 13, a technique for adjusting a discard rate for discarding charges from the photosensitive unit 11, There is a technique for adjusting both the passing rate and the discarding rate.

  In the technique of adjusting the pass rate and the discard rate in the sensitivity control unit 12, as shown in FIG. 10, a gate electrode 12a is provided between the photosensitive unit 11 and the charge accumulation unit 13, and the pass voltage applied to the gate electrode 12a. Is changed to control the movement of charges from the photosensitive portion 11 to the charge accumulating portion 13 (that is, the passing rate). Further, by providing the charge discarding part 12c and changing the discarding voltage applied to the disposal electrode 12b attached to the charge discarding part 12c, the movement of the charge from the photosensitive part 11 to the charge discarding part 12c (that is, the discard rate) is changed. Control. The charge accumulating units 13 are provided so as to correspond one-to-one for each photosensitive unit 11, and the charge discarding units 12c are provided so as to correspond to the plurality of photosensitive units 11 so as to correspond one-to-many. In the illustrated example, all of the photosensitive portions 11 of the photodetecting element 1 share a set of discarding electrode 12b and charge discarding portion 12c.

  In order to control the sensitivity, it is conceivable to control only the pass rate from the photosensitive unit 11 to the charge accumulating unit 13 without discarding the charge from the photosensitive unit 11. Since charges remain in the unit 11 for a while, unnecessary residual charges out of the charges generated in the photosensitive unit 11 are mixed as noise components in the used charges (hereinafter referred to as signal charges). Therefore, in order to prevent the residual charge from being mixed into the signal charge, not only the passing voltage applied to the gate electrode 12a but also the discard voltage applied to the discard electrode 12b is controlled.

  In order to control the sensitivity using the gate electrode 12a and the waste electrode 12b, the charge generated in the photosensitive portion 11 can pass through the charge accumulating portion 13 by keeping the passing voltage applied to the gate electrode 12a constant. In addition, a waste voltage is applied to the waste electrode 12b so that the charge moves from the photosensitive part 11 to the charge discarding part 12c except for a period in which the charge used for the signal charge among the charges generated by the photosensitive part 11 is generated. In short, the signal charge is not discarded to the charge discarding unit 12c only during the period in which the charge used as the signal charge is generated in the photosensitive unit 11, and the signal charge is discarded to the charge discarding unit 12c in the other period. Only the charges generated during the period to be used are accumulated in the charge accumulation unit 13.

  Now, it is assumed that the intensity of light emitted from the light source 2 to the space is modulated by the modulation signal as shown in FIG. The charge accumulation unit 13 accumulates charges corresponding to the received light amounts A0, A1, A2, and A3 in a specific section synchronized with the modulation signal in a plurality of periods (tens of thousands to hundreds of thousands) of the modulation signal. The accumulated signal charge is taken out for each charge accumulation, and the charge in the next section is accumulated. For example, when charges corresponding to the received light quantity A0 are accumulated for tens of thousands of cycles of the modulation signal, the signal charges corresponding to the received light quantity A0 are once taken out to the outside, and thereafter, the charges corresponding to the received light quantity A1 are converted to tens of thousands of modulation signals. Accumulate about the period.

  FIG. 11 shows a state in which charges corresponding to the received light quantity A0 are accumulated. As shown in FIG. 11B, the passing voltage applied to the gate electrode 12a is kept constant. Further, as the charge corresponding to the received light quantity A0, the charge generated by the photosensitive portion 11 in the interval where the phase of the modulation signal is 0 to 90 degrees is employed. In other words, a waste voltage is applied to the waste electrode 12b so that the charge generated by the photosensitive portion 11 is an unnecessary charge in a section where the phase of the modulation signal is 90 to 360 degrees as shown in FIG. This control makes it possible to accumulate signal charges corresponding to the received light quantity A0 in a desired section in the charge accumulating unit 13 as shown in FIG. The processing shown in FIG. 11 is performed for tens of thousands to hundreds of thousands of cycles of the modulation signal, and the signal charge obtained in the charge accumulating unit 13 during this period is taken out by the charge extracting unit 14 as a received light output corresponding to the received light amount A0. .

  The electric charge extracted from the electric charge extraction unit 14 is given to the image generation unit 4 as an image signal. In the image generation unit 4, the distance to the object Ob in the target space is determined by using the above-described equation (1). It is calculated from the received light output corresponding to A0, A1, A2, and A3. That is, the image generation unit 4 calculates the distance to the object Ob in each direction corresponding to each photosensitive unit 11, and calculates the three-dimensional information of the target space. By using this three-dimensional information, it is possible to generate a distance image in which the pixel values of the pixels matching each direction of the target space are distance values.

  In the above-described control, a passing voltage that is a constant voltage is applied to the gate electrode 12a during the period in which the discarding voltage is applied to the discarding electrode 12b, but the magnitude relationship between the discarding voltage and the passing voltage is appropriately determined. If set, it is possible to prevent signal charges from being almost integrated during a period in which unnecessary charges are discarded. The reason why charges are accumulated for tens of thousands to hundreds of thousands of cycles of the modulation signal is to increase the sensitivity by increasing the amount of charges to be accumulated, and if the modulation signal is set to 20 MHz, for example, 30 Even if signal charges are taken out at a frame / second, integration of several hundred thousand cycles or more is possible.

  As described above, the charge discarding unit 12c including the disposal electrode 12b is provided, and unnecessary charges that are not used as signal charges among the charges generated in the photosensitive unit 11 are actively discarded to the charge discarding unit 12c. In the unit 11, most of the charge generated in the photosensitive unit 11 during the period when no signal charge is given to the charge accumulating unit 13 is discarded as unnecessary charge, and mixing of noise components into the signal charge is greatly suppressed. The

  In the above-described example, a period in which the discard voltage is not applied by providing a period in which the discard voltage is applied to the discard electrode 12b and a period in which the discard voltage is not applied to the discard electrode 12b in the period in which the passing voltage that is a constant voltage is applied to the gate electrode 12a. In FIG. 12, the charge generated in the photosensitive portion 11 is used as a signal charge. As shown in FIG. 12, the period for applying the passing voltage to the gate electrode 12a and the period for applying the discard voltage to the discard electrode 12b do not overlap. You may control as follows.

  FIG. 12 shows the operation when signal charges corresponding to the received light quantity A0 are integrated. FIG. 12A shows a modulation signal for modulating the intensity of light emitted to the space from the light emitting source 2, and the gate electrode 12a has a timing corresponding to the received light amount A0 as shown in FIG. 12B. Apply the passing voltage with. The period during which the passing voltage is applied to the gate electrode 12a is set from 0 degrees in the phase of the modulation signal to a certain period (0 to 90 degrees in the illustrated example). During this period, the charge from the photosensitive portion 11 to the charge accumulation portion 13 is transferred. It becomes possible to move. On the other hand, as shown in FIG. 12C, a discard voltage is applied to the waste electrode 12b in a period other than the period in which the signal charge corresponding to the received light amount A0 is accumulated in the charge accumulation unit 13, and the signal charge is accumulated in other periods. The charges generated in the photosensitive unit 11 are discarded as unnecessary charges in the charge discarding unit 12c. Such control makes it possible to take out signal charges corresponding to the received light amount A0 as shown in FIG.

  In the control shown in FIG. 12, the period during which the passing voltage is applied to the gate electrode 12a is different from the period during which the discarding voltage is applied to the discard electrode 12b. Therefore, as shown in the control example in FIG. The magnitude of the passing voltage and the discarding voltage can be controlled independently without considering the magnitude relationship with the discarding voltage. As a result, the passing voltage and the discarding voltage can be easily controlled, and the photosensitive unit 11 receives light. Control of the sensitivity, which is the ratio of taking in the signal charge with respect to the light quantity, is facilitated, and control of the ratio of discarding unnecessary charges out of the charges generated in the photosensitive portion 11 is facilitated. Further, in the control example shown in FIG. 12, the period during which signal charges are accumulated in the charge accumulating unit 13 is defined by the passing voltage applied to the gate electrode 12a, so the period during which the discard voltage is applied to the discard electrode 12b is shortened. For example, it is possible to apply the waste voltage to the waste electrode 12b only during a predetermined period immediately before applying the pass voltage to the gate electrode 12a.

  When the control shown in FIG. 12 is performed, the charge generated in the photosensitive unit 11 is discarded as an unnecessary charge in a period in which the charge generated in the photosensitive unit 11 is not accumulated in the charge accumulating unit 13 as a signal charge. Mixing of noise components into is greatly suppressed.

  As an example of the control of the passing voltage and the discard voltage, as shown in FIG. 13, the discard voltage applied to the discard electrode 12b is maintained at a constant voltage so that a part of the charge generated in the photosensitive portion 11 is always discarded. May be. In the control example of FIG. 13, a period during which the passing voltage is applied to the gate electrode 12 a and a period during which the passing voltage is not applied are provided, and the period during which the passing voltage is applied is defined as a period during which signal charges are accumulated in the charge accumulation unit 13.

  FIG. 13 shows an operation when signal charges corresponding to the received light quantity A0 are integrated. FIG. 13A shows a modulation signal that modulates the intensity of light emitted from the light emitting source 2 to the space, and the gate electrode 12a provided in the charge accumulating unit 13 has a structure as shown in FIG. A passing voltage is applied during a period corresponding to the received light amount A0, and the charge generated in the photosensitive unit 11 is accumulated in the charge accumulating unit 13 as a signal charge corresponding to the received light amount A0. That is, the period during which the pass voltage is applied to the gate electrode 12a is set from 0 degree to a certain period (0 to 90 degrees in the illustrated example) in the phase of the modulation signal. Charge transfer is possible. On the other hand, as shown in FIG. 13C, a constant voltage discard voltage, which is a DC voltage, is always applied to the waste electrode 12b, and a part of the charge generated by the photosensitive portion 11 is always used as an unnecessary charge. Discard to 12c. In the above-described control, since the passing voltage is applied to the gate electrode 12a only during the period in which the signal charge is accumulated in the charge accumulation unit 13, the signal charge corresponding to the received light amount A0 is extracted as shown in FIG. Is possible.

  In the control shown in FIG. 13, a constant voltage discard voltage is applied to the waste electrode 12b regardless of whether or not a pass voltage is applied to the gate electrode 12a. Unnecessary charges that have not been accumulated as signal charges in the accumulation unit 13 are discarded as discard charges in the charge discard unit 12c. Here, the signal charge is continuously discarded from the photosensitive portion 11 to the charge discarding portion 12c even during a period in which a part of the charge generated in the photosensitive portion 11 is accumulated in the charge accumulating portion 13 as a signal charge. In order to properly integrate the voltage in the charge accumulation unit 13, it is necessary to consider the magnitude relationship between the passing voltage and the discard voltage. However, since the discard voltage is a constant voltage and is always applied to the discard electrode 12b, in practice, only the passing voltage needs to be controlled, and the control itself is easy.

  The photodetecting element 1 including the sensitivity control unit 12 illustrated in FIG. 10 can be realized by a CCD image sensor including an overflow drain. Any charge transfer method may be used in the CCD image sensor, and any of an interline transfer (IT) method, a frame transfer (FT) method, and a frame interline transfer (FIT) method may be used.

  FIG. 14 shows a configuration of an interline transfer type CCD image sensor having a vertical overflow drain. The illustrated example is a two-dimensional image sensor in which a plurality of photodiodes 41 serving as the photosensitive portions 11 are arranged in a horizontal direction and a vertical direction (3 × 4 in the figure), and the photodiodes 41 arranged in the vertical direction are arranged. A vertical transfer register 42 made of a CCD is provided on the right side of each column, and a horizontal transfer register 43 made of a CCD is provided below the area where the photodiodes 41 and the vertical transfer registers 42 are arranged. The vertical transfer register 42 includes two transfer electrodes 42 a and 42 b for each photodiode 41, and the horizontal transfer register 43 includes two transfer electrodes 43 a and 43 b for each vertical transfer register 42.

  The photodiode 41, the vertical transfer register 42, and the horizontal transfer register 43 are formed on one semiconductor substrate 40, and the photodiode 41, the vertical transfer register 42, and the horizontal transfer register 43 are formed on the main surface of the semiconductor substrate 40. An overflow electrode 44 which is an aluminum electrode is provided so as to directly contact the semiconductor substrate 40 without going through an insulating film over the entire circumference of the semiconductor substrate 40 so as to surround the whole. If an appropriate disposal voltage that is positive with respect to the semiconductor substrate 40 is applied to the overflow electrode 44, electrons (charges) generated by the photodiode 41 are discarded through the overflow electrode 44. The overflow electrode 44 functions as the discard electrode 12b because a discard voltage is applied when discarding unnecessary charges among the charges generated in the photodiode 41 which is the photosensitive portion 11, and the power supply for applying the discard voltage to the overflow electrode 44 Functions as a charge discarding unit 12c that discards electrons (charges) generated in the photosensitive unit 11. The surface of the semiconductor substrate 40 is covered with a light-shielding film 46 (see FIG. 15) except for the portion corresponding to the photodiode 41.

  For the CCD image sensor shown in FIG. 14, a portion related to one photodiode 41 is cut out and shown in FIG. 15. An n-type semiconductor is used for the semiconductor substrate 40, and a well region 31 made of a p-type semiconductor is formed in a region straddling the photodiode 41 and the vertical transfer register 42 on the main surface of the semiconductor substrate 40. The well region 31 is formed so that the thickness dimension of the region corresponding to the vertical transfer register 42 is larger than the region corresponding to the photodiode 41. An n + -type semiconductor layer 32 is provided in a region corresponding to the photodiode 41 in the well region 31, and the photodiode 41 is formed by a pn junction between the well region 31 and the n + -type semiconductor layer 32. A surface layer 33 made of a p + type semiconductor is stacked on the surface of the photodiode 41. The surface layer 33 is provided for the purpose of controlling the vicinity of the surface of the n + -type semiconductor layer 32 so as not to be a passage path for charges when the charge generated by the photodiode 41 is moved to the vertical transfer register 42. Such a structure is known as a buried photodiode.

An accumulation transfer layer 34 made of an n-type semiconductor is overlaid in a region corresponding to the vertical transfer register 42 in the well region 31. The surface of the accumulation / transfer layer 34 and the surface of the surface layer 33 are substantially flush with each other, and the thickness dimension of the accumulation / transfer layer 34 is larger than the thickness dimension of the surface layer 33. The accumulation transfer layer 34 is in contact with the surface layer 33, but a separation layer 35 made of a p + type semiconductor having the same impurity concentration as that of the surface layer 33 is interposed between the storage layer 34 and the n + type semiconductor layer 32. Transfer electrodes 42 a and 42 b are disposed on the surface of the accumulation transfer layer 34 via an insulating film 45. Two transfer electrodes 42a and 42b are provided for each photodiode 41, and one of the two transfer electrodes 42a and 42b is formed wider than the other in the vertical direction. Specifically, as shown in FIG. 16, of the two transfer electrodes 42a and 42b corresponding to one photodiode 41, the narrow transfer electrode 42b is formed in a flat plate shape, and the wide transfer electrode 42a. Are arranged on the same plane as the narrow transfer electrode 42b and disposed between the pair of transfer electrodes 42b, and from both ends in the vertical direction (left and right direction in FIG. 16) of the flat plate portion. And a curved portion that extends and overlaps the transfer electrode 42b. Here, the insulating film 45 is formed of SiO ₂ , the transfer electrodes 42 a and 42 b are formed of polysilicon, and the transfer electrodes 42 a and 42 b are insulated from each other through the insulating film 45. Further, the surface of the light detection element 1 is covered with a light shielding film 46 except for a portion where light is incident on the photodiode 41. In the well region 31, the region corresponding to the vertical transfer register 42 and the storage transfer layer 34 are formed over the entire length of the vertical transfer register 42. Therefore, the storage transfer layer 34 has a wide transfer electrode 42a and a narrow transfer electrode 42b. And are alternately arranged.

  In the light detection element 1 described above, the photodiode 41 corresponds to the photosensitive portion 11, the transfer electrode 42 a corresponds to the passing electrode 12 a, the overflow electrode 44 corresponds to the discard electrode 12 b, and the vertical transfer register 42 corresponds to the charge accumulation portion 13. And functions as a part of the charge extraction unit 14. Further, the horizontal transfer register 43 also becomes a part of the charge extraction unit 14. That is, if light enters the photodiode 41, a charge is generated, and the ratio of the charge generated as a signal charge to the vertical transfer register 42 among the charges generated by the photodiode 41 is equal to the passing voltage applied to the transfer electrode 42a and the overflow. It can be determined according to the relationship with the waste voltage applied to the electrode 44. When a pass voltage is applied to the transfer electrode 42a, a potential well is formed in the storage transfer layer 34, and the depth of the potential well can be controlled by controlling the pass voltage. Therefore, by controlling the depth of the potential well and the time during which the passing voltage is applied, the ratio of charges delivered from the photodiode 41 to the vertical transfer register 42 can be adjusted. Further, if the discard voltage applied to the overflow electrode 44 is controlled, the potential gradient between the photodiode 41 and the semiconductor substrate 40 can be controlled. Therefore, if the potential gradient and the time for applying the discard voltage are controlled. The rate of charge delivered to the vertical transfer register 42 can be adjusted. The control voltage and the discard voltage may be controlled as in the control examples shown in FIGS.

  The signal charges delivered from the photodiode 41 to the vertical transfer register 42 are signals corresponding to the received light amounts A0, A1, A2, and A3 of each one of the four received light amounts A0, A1, A2, and A3 described above. It is read each time charge is accumulated. For example, when a signal charge corresponding to the received light quantity A0 is accumulated in a potential well formed corresponding to each photodiode 41, the signal charge is read, and then a signal charge corresponding to the received light quantity A1 is accumulated in the potential well. Then, the operation of reading the signal charge again is repeated. Note that the period during which signal charges corresponding to the received light amounts A0, A1, A2, and A3 are accumulated is set to be equal.

  Of the control examples described above, in the control example shown in FIG. 11, the charge (electrons) generated by the photosensitive unit 11 (photodiode 41) is always delivered to the charge accumulation unit 13 (vertical transfer register 42). Therefore, the charges accumulated in the charge accumulating unit 13 include not only the charges generated during the period in which the target received light amounts A0, A1, A2, and A3 are obtained, but also the charges generated during periods other than the target. It will be. Now, in the sensitivity control unit 12, the sensitivity in the period for generating the charges corresponding to the received light amounts A0, A1, A2, A3 is α, the sensitivity in the other periods is β, and the photosensitive unit 11 is a charge proportional to the received light amount. Is generated. Under this condition, the charge accumulating unit 13 that accumulates charges corresponding to the received light amount A0 is proportional to αA0 + β (A1 + A2 + A3) + βAx (Ax is the received light amount other than the period during which the received light amounts A0, A1, A2, and A3 are obtained). Charges proportional to αA2 + β (A0 + A1 + A3) + βAx are accumulated in the charge accumulation unit 13 that accumulates charges and accumulates charges corresponding to the received light amount A2. As described above, when obtaining the phase difference ψ, (A2−A0) is obtained, and when a value corresponding to (A2−A0) is obtained from the charge accumulated in the charge accumulation unit 13, (α−β) ( Similarly, since the value corresponding to (A1-A3) is (α-β) (A1-A3), (A2-A0) / (A1-A3) Theoretically the same value is obtained regardless of the presence or absence, and the obtained phase difference ψ is the same value even if charges are mixed.

  In the configuration example described above, a CCD image sensor is used for the photodetecting element 1, and sensitivity control is performed by controlling at least one of the amount of charge passed through the charge accumulating unit 13 and the amount of charge discarded into the charge discarding unit 12c. Although the example which comprises the part 12 was shown, the sensitivity control part 12 shown below changes the area (substantially light reception area) of the area | region which produces | generates the electric charge which can be utilized in the photosensitive part 11. FIG.

  Hereinafter, a specific structural example of the light detection element 1 will be described. The photodetecting element 1 shown in FIG. 17 has a plurality of (for example, 100 × 100) photosensitive portions 11 arranged in a matrix, and is formed on, for example, a single semiconductor substrate. One photosensitive portion 11 has a configuration in which a plurality (five in the figure) of control electrodes 23 are arranged on a semiconductor layer 21 to which impurities are added via an insulating film 22 made of an oxide film. In the illustrated example, the direction in which the electrodes are arranged (left-right direction) is the vertical direction, and when taking out the charge generated by the photosensitive portion 11 (using electrons in this embodiment), the charge is transferred in the vertical direction by the vertical transfer register. After that, the data is transferred in the horizontal direction using a horizontal transfer register. That is, the charge extraction unit 14 is configured by the vertical transfer register and the horizontal transfer register. As the configuration of the vertical transfer register and the horizontal transfer register, the same configuration as the interline transfer (IT) method, the frame transfer (FT) method, and the frame interline transfer (FIT) method in the CCD image sensor can be adopted.

  That is, when the photosensitive portions 11 arranged in the vertical direction share the continuous semiconductor layer 21 and the semiconductor layer 21 is used as a vertical transfer register, the semiconductor layer 21 is used as both the photosensitive portion 11 and the charge transfer path. The structure allows the charge to be transferred in the vertical direction in the same manner as the FT type CCD image sensor, and if the charge is transferred from the photosensitive portion 11 to the vertical transfer register via the transfer gate, the IT type or FIT Charges can be transferred in the same manner as a CCD image sensor of the type.

  As described above, the semiconductor layer 21 is doped with impurities, the main surface of the semiconductor layer 21 is covered with the insulating film 22 made of an oxide film, and a plurality of control electrodes 23 are formed on the semiconductor layer 21 via the insulating film 22. Is arranged. This light detection element 1 has a structure known as a MIS element, but a normal MIS element is that a plurality of (five in the illustrated example) control electrodes 23 are provided in a region functioning as one light detection element 1. Is different. A material is selected for the insulating film 22 and the control electrode 23 so that light having the same wavelength as the light emitted from the light source 2 to the target space can be transmitted. When light enters the semiconductor layer 21 through the insulating film 22, the semiconductor layer 21. A charge is generated inside the. The conductivity type of the semiconductor layer 21 in the illustrated example is n-type, and electrons e are used as charges generated by light irradiation. FIG. 17 shows only a region corresponding to one photosensitive portion 11, and a plurality of regions having the structure shown in FIG. 17 are arranged on the semiconductor substrate (not shown) and the charge extraction is performed. A structure to be part 14 is provided. The vertical transfer register provided as the charge extraction unit 14 is assumed to transfer charges in the left-right direction in FIG. 17, but it is also possible to adopt a configuration in which charges are transferred in a direction orthogonal to the plane in FIG. is there. In addition, when transferring charges in the horizontal direction in the figure, it is desirable to set the width dimension of the control electrode 23 in the horizontal direction to about 1 μm.

  In the light detection element 1 having this structure, when a positive control voltage + V is applied to the control electrode 23, a potential well (depletion layer) 24 that accumulates electrons e in a portion corresponding to the control electrode 23 is formed in the semiconductor layer 21. The That is, when light is applied to the semiconductor layer 21 with a control voltage applied to the control electrode 23 so as to form the potential well 24 in the semiconductor layer 21, a part of the electrons e generated in the vicinity of the potential well 24. Are captured in the potential well 24 and accumulated in the potential well 24, and the remaining electrons e disappear due to recombination in the deep part of the semiconductor layer 21. Further, the electrons e generated at a location away from the potential well 24 are also extinguished by recombination in the deep part of the semiconductor layer 21.

  Since the potential well 24 is formed at a portion corresponding to the control electrode 23 to which the control voltage is applied, the potential well 24 along the main surface of the semiconductor layer 21 is changed by changing the number of the control electrodes 23 to which the control voltage is applied. (In other words, the area of a region that generates a charge that can be used on the light receiving surface) can be changed. That is, changing the number of control electrodes 23 to which the control voltage is applied means adjusting sensitivity in the sensitivity control unit 12. For example, when the control voltage + V is applied to three control electrodes 23 as shown in FIG. 17A and when the control voltage + V is applied to one control electrode 23 as shown in FIG. Since the area occupied by the potential well 24 on the light receiving surface changes, the area of the potential well 24 is larger in the state of FIG. 17A, so that the same amount of light is obtained as compared with the state of FIG. As a result, the ratio of the charge that can be used increases and the sensitivity of the photosensitive portion 11 is substantially increased. Thus, it can be said that the photosensitive portion 11 and the sensitivity control portion 12 are constituted by the semiconductor layer 21, the insulating film 22, and the control electrode 23. The potential well 24 functions as the charge accumulation unit 13 because it holds charges generated by light irradiation.

  As described above, in order to extract charges from the potential well 24, the same technique as that of the CCD image sensor is employed. For example, when the photosensitive portion 11 is used as a vertical transfer register, after the electrons e are accumulated in the potential well 24, a control voltage having a different application pattern from that at the time of charge accumulation is applied to the control electrode 23. The accumulated electrons e can be transferred in one direction (for example, in the right direction in the figure). Alternatively, it is possible to adopt a configuration in which charges are transferred from the photosensitive portion 11 via a transfer gate to a vertical transfer register provided separately from the photosensitive portion 11. Charge is transferred from the vertical transfer register to the horizontal transfer register, and the charge transferred to the horizontal transfer register is taken out of the photodetector 1 from an electrode (not shown) provided on the semiconductor substrate.

  The sensitivity control unit 12 in the configuration shown in FIG. 17 switches the sensitivity of the photosensitive unit 11 into two levels, high and low, by switching the area for generating available charges into two levels, the received light quantity A0, A1, A2, The sensitivity corresponding to any one of A3 is set to high sensitivity only during a period when the photosensitive portion 11 is to be generated (the area for generating charges is increased), and the sensitivity is set to be low during other periods. The period of high sensitivity and the period of low sensitivity are set in synchronization with the modulation signal that drives the light emitting source 2. Specifically, in a specific section (specific phase section) synchronized with the modulation signal, the charge generation area is increased to accumulate the charge generated by the photosensitive portion 11, and in other sections other than the specific section. The charge generated by the photosensitive portion 11 is accumulated by reducing the area for generating the charge. That is, in the photosensitive portion 11, the function of accumulating charges and the function of accumulating are realized alternately. Here, accumulation means collecting electric charges, and accumulation means holding electric charges. In other words, in the configuration shown in FIG. 17, by changing the size (area) of the charge accumulating unit 13 provided in the photosensitive unit 11, the integration rate of the charges generated in the photosensitive unit 11 during the period of accumulating charges. The charge accumulation rate generated in the photosensitive portion 11 is reduced during the period in which charges are accumulated.

  In addition, after the charges are accumulated in the potential well 24 over a plurality of periods of the modulation signal, the charges are extracted to the outside of the light detection element 1 through the charge extraction unit 14. Charges are accumulated over a plurality of periods of the modulation signal because the period during which the photosensitive unit 11 generates usable charges within one period of the modulation signal is short (for example, if the frequency of the modulation signal is 20 MHz). This is because less than a quarter of 50 ns is generated. That is, by integrating charges for a plurality of periods of the modulation signal, signal charges (charges corresponding to light emitted from the light emission source 2) and unnecessary charges (mainly generated in the external light component and the light detection element 1). The charge corresponding to the shot noise) can be made large, and a large SN ratio can be obtained.

  By the way, if the charges corresponding to the received light amounts A0, A1, A2, and A3 of the four sections necessary for obtaining the phase difference ψ are generated by one photosensitive portion 11, the resolution in the line-of-sight direction is increased. There arises a problem that the time difference for obtaining the charges corresponding to the received light amounts A0, A1, A2, A3 becomes large. On the other hand, if the charges corresponding to the received light amounts A0, A1, A2, A3 are generated by the four photosensitive portions 11, respectively, the time difference for obtaining the charges corresponding to the received light amounts A0, A1, A2, A3 becomes small. However, a shift occurs in the line-of-sight direction for obtaining the charges in the four sections, and the resolution in the line-of-sight direction decreases. Therefore, a configuration in which two types of charges corresponding to the received light amounts A0, A1, A2, and A3 are generated by using two photosensitive portions 11 within one cycle of the modulation signal may be employed. That is, two photosensitive portions 11 may be used as a set, and light from the same line-of-sight direction may be incident on the two photosensitive portions 11 in the set.

  By adopting this configuration, the resolution in the line-of-sight direction can be made relatively high, and the time difference for generating charges corresponding to the received light amounts A0, A1, A2, and A3 can be reduced. That is, by reducing the time difference for generating charges corresponding to the received light amounts A0, A1, A2, and A3, the distance detection accuracy is kept relatively high even for the object Ob moving in the target space. be able to. In this configuration, the resolution in the line-of-sight direction is lower than that in the case where the charge corresponding to the four types of received light amounts A0, A1, A2, and A3 is generated by one photosensitive unit 11, but the resolution in the line-of-sight direction is as follows. This can be improved by downsizing the photosensitive unit 11 or designing the light receiving optical system 19.

  The operation will be specifically described below. In the example shown in FIG. 17, an example in which five control electrodes 23 are provided for one photosensitive portion 11 is shown. However, two control electrodes 23 on both sides are charged (electrons e) by the photosensitive portion 11. In the case where two photosensitive portions 11 are used as a pair, the adjacent photosensitive portions are formed. Since any one of the photosensitive portions 11 forms a barrier between the 11 potential wells 24, it is sufficient to provide three photosensitive electrodes 11 for each of the photosensitive portions 11. With this configuration, the occupation area per one photosensitive portion 11 is reduced, and it is possible to suppress a decrease in resolution in the line-of-sight direction while using the two photosensitive portions 11 as a set.

  In the configuration example of the distance image sensor 10 described above, four sections corresponding to the received light amounts A0, A1, A2, and A3 are set so that the phase interval is 90 degrees within one period of the modulation signal. However, if the phase with respect to the modulation signal is known, the four sections can be set at appropriate intervals other than 90 degrees. However, the formula for obtaining the phase difference ψ differs if the interval is different. In addition, the period of taking out the electric charge corresponding to the received light quantity of the four sections is within one period of the modulation signal as long as the reflectance and the external light component of the object Ob do not change and the phase difference ψ does not change. Thus, it is not essential to take out signal charges in four sections. Furthermore, when there is an influence of disturbance light such as sunlight or illumination light, it is desirable to dispose an optical filter that transmits only the wavelength of light emitted from the light source 2 in front of the photosensitive portion 11.

  Next, the overall configuration of the human body detection apparatus using the above-described distance image sensor 10 will be described. As shown in FIG. 1, the human body detection device of this embodiment includes the above-described distance image sensor 10 and a human body detection unit 20.

  The distance image sensor 10 is installed above the target area 30 as shown in FIG. 2A, and images the target area 30 from above. In other words, the light source 2 irradiates the lower target region 30 with irradiation light, and the control circuit unit 3 modulates the intensity of the irradiation light from the light source 2 so as to periodically change. A distance value obtained by converting the phase difference of light from the generation unit 4 until the irradiation light from the light source 2 is reflected by the object in the target region 30 and received by the photosensitive unit 11 into the distance to the object is defined as a pixel value. A distance image is generated. In this embodiment, the distance image sensor 10 that obtains the distance image by the TOF method is used. However, a plurality of cameras that capture the target region from different directions are installed, and the plurality of images captured by the cameras are used. Using the principle of triangulation, a distance image having a pixel value as a distance value from a predetermined measurement point to an object in the target region may be created.

  The human body detection unit 20 includes a computer that detects a human body in the target region 30 by executing a human body detection method described later using a distance image captured by the distance image sensor 10, and the human body detection method is described below. This will be described with reference to the drawings. Here, a case where there are one adult 50 and one child 51 in the target area 30 as shown in FIG.

  When the image data of the distance image is input from the image generation unit 4, the human body detection unit 20 performs region extraction processing for extracting a pixel region having a pixel value equal to or less than a threshold from a plurality of pixels constituting the distance image. The threshold value is increased stepwise from the pixel value of the pixel with the shortest pixel value (hereinafter referred to as the shortest pixel), and is performed a plurality of times. For example, as shown in FIG. 3A, when the above threshold value is gradually increased from the pixel value L1 of the shortest pixel to L2, L3, L4, and L5, the distance from the distance image sensor 10 to the head of the adult 50 is increased. Therefore, the pixel areas A1 to A5 whose distance values are equal to or less than the threshold values L1 to L5 are substantially circular areas (see FIG. 3B). In addition, the area | region enclosed by the curve (Contour line formed by connecting the pixel whose pixel value becomes a threshold value) in FIG.3 (b) becomes pixel area A1-A5, respectively, and is the same also in another figure. Further, when the above threshold value is extended to L6 and L7, the distance from the distance image sensor 10 to the vicinity of the shoulder of the adult 50 is, so that the pixel areas A6 and A7 whose distance values are equal to or less than the threshold values L6 and L7 are substantially elliptical. It becomes an area. On the other hand, since the child 51 is shorter than the adult 50, as shown in FIG. 4A, when the above threshold value is gradually increased to L11 (> L7), L12, and L13, the child from the distance image sensor 10 The pixel areas A11, A12, and A13 whose distance values are equal to or smaller than the thresholds L11, L12, and L13 are substantially circular areas (see FIG. 4B). Further, when the above threshold value is extended stepwise to L14 and L15, the distance from the distance image sensor 10 to the vicinity of the shoulder of the child 51 is obtained, so that the pixel areas A14 and A15 having distance values equal to or less than the threshold values L14 and L15 are substantially elliptical. It becomes an area of shape.

  Here, FIG. 2B shows a step-by-step process for extracting the pixel region where the pixel value is equal to or smaller than the threshold value from the plurality of pixels constituting the distance image G1 to the threshold values L1 to L7 and L11 to L15. The pixel regions A1 to A7 and A11 to A15 corresponding to the adult 50 and the child 51, respectively, are extracted. Since the head of the child 51 is smaller than the adult 50 and the shoulder width is narrow, the pixel area A13 corresponding to the head of the child 51 is narrower than the pixel area A5 of the head of the adult 50, and the shoulder of the child 51 The pixel area A15 corresponding to the portion is also narrower than the pixel area A7 on the shoulder of the adult 50. Further, when the adult image 50 and the child 51 that are almost directly below are photographed from the distance image sensor 10 installed above, the head portion below the height position at which the horizontal cross section of the head portion becomes maximum appears in the distance image. Disappear. Similarly, in the body part of the adult 50 and the child 51, the body part below the height position where the horizontal cross section of the body is the maximum does not appear in the distance image.

  As described above, the human body detection unit 20 repeats the region extraction process by gradually increasing the threshold value, but each time the region extraction process is performed by increasing the threshold value, a newly extracted pixel region is already extracted. It is determined whether or not the pixel area partially overlaps, and if there are pixel areas that partially overlap, it is determined that these pixel areas indicate the same object, and a plurality of overlapping An identifier assignment process for assigning the same identifier to the entire pixel area is executed.

  For example, when a pixel area having a pixel value equal to or less than the threshold value L1 is extracted, there is only one pixel area, and therefore an individual identifier LB1 is assigned to the pixel area A1. FIG. 5B shows a pixel area to which the identifier LB1 is assigned, and the pixel area to which the identifier LB1 is assigned is hatched. Thereafter, when the pixel area A3 is extracted by extending the threshold value to L3, the newly extracted pixel area A3 includes the pixel areas A1 and A2 extracted in the previous extraction process. -A3 is assigned the same identifier LB1 (see FIG. 6A). That is, the pixel area A3 is newly added to the pixel area to which the identifier LB1 is assigned. Further, when the pixel area A4 is extracted by extending the threshold value to L4, the newly extracted pixel area A4 includes all the already extracted pixel areas A1 to A3. Are assigned the same identifier LB1 (see FIG. 6B).

  When a person in the target area 30 maintains an upright posture, when a plurality of pixel areas A1,... Are extracted in order from the side closer to the distance image sensor 10, the distance value from the distance image sensor 10 is slightly far. The pixel area includes a pixel area with a shorter distance value. However, as shown in FIGS. 6C and 6D, when the person in the target area 30 takes a forward leaning posture, the distance value is There is a case where a pixel region having a shorter distance value is not included in a pixel region that is a little far away and only partially overlaps. That is, the contour line formed by connecting pixels whose pixel values are threshold values may be interrupted.

  That is, when a plurality of pixel areas are extracted in order from the side closer to the distance image sensor 10 when a person in the target area 30 is in a forward leaning posture, the pixel areas A1 to A5 corresponding to the head of the human body Since the pixel region having a short distance value is included in the pixel region having a slightly far distance value, the human body detection unit is in a stage where the threshold value is extended to L5 and the pixel region A5 is extracted as shown in FIG. 20 determines that the pixel areas A1 to A4 included in the pixel area A5 are the same object, and assigns the identifier LB1 to the whole of these pixel areas A1 to A5, that is, the pixel area A5.

  On the other hand, in the pixel areas A6 and A7 corresponding to the vicinity of the shoulder of the human body, since this person maintains a forward leaning posture, the pixel area having a shorter distance value is not included in the pixel area having a slightly longer distance value. Since the pixel region A6 is extracted by extending the threshold value to L6 as shown in FIG. 6 (d), the human body detection unit 20 detects the partially overlapping pixel regions A5 and A6. The entirety is determined to be the same object, and the identifier LB1 is assigned to the whole of these pixel areas A1 to A6, that is, the union of the pixel areas A5 and A6.

  When the determination condition Z1 that the number of pixels in the pixel area to which the same identifier is assigned exceeds the number of pixels corresponding to the size of the human body is satisfied, the human body detection unit 20 performs the above-described area extraction processing and identifier assignment processing. When finished, the pixel area to which the same identifier is assigned is extracted as a human body. FIG. 8A shows a state in which the above-described determination condition Z1 is satisfied. As a result of the above-described region extraction processing and identifier assignment processing, the human body detection unit 20 generates pixel regions A1,. Assume that an identifier LB1 is assigned and a separate identifier LB2 is assigned to the pixel region A11... Corresponding to the child 51. When the number of pixels in the pixel area A1... To which the identifier LB1 is assigned exceeds a predetermined number of pixels, the human body detection unit 20 ends the area extraction process and the identifier assignment process for the pixel area of the identifier LB1. The assigned pixel areas A1 to A5 are extracted as a human body. In addition, when the number of pixels in the pixel area A11... To which the identifier LB2 is assigned exceeds a predetermined number of pixels, the human body detection unit 20 ends the area extraction process and the identifier assignment process for the pixel area of the identifier LB2, and assigns the identifier LB2. The obtained pixel areas A11 to A14 are extracted as a human body. Since the child 51 is smaller in size than the adult 50, the determination condition Z1 is established at the stage where the pixel areas A1 to A5 corresponding to the head are extracted in the case of the adult 50. In addition to the above, the determination condition Z1 is established when the pixel areas A11 to A14 corresponding to the shoulder are extracted.

As described above , when the pixel area is extracted in order from the side closer to the distance image sensor 10 and the newly extracted pixel area overlaps with the already extracted pixel area, the plurality of overlapping pixel areas are regarded as the same object. Therefore, the same identifier is assigned to all of these pixel areas, and the determination condition Z1 is established that the number of pixels in the pixel area to which the same identifier is assigned exceeds the number of pixels corresponding to the size of the human body. Then, since the pixel area | region which provided the same identifier is extracted as a human body, even when a contour line does not close like the conventional human body detection method, a human body can be detected reliably. In addition, since the region extraction process and the identifier assigning process have been completed for the pixel area to which the identifier is assigned when the determination condition Z1 is satisfied, compared with the case where the pixel area is extracted by extending the threshold value to the floor surface, There is an advantage that the presence of a person can be detected in a short time, and the capacity of the memory for storing the pixel region to which the identifier is added can be reduced.

Incidentally, the distance image sensor 10 is disposed above, but by capturing the lower target region 30, is attached to the side of the target area 30, after taking a person target region 30 from the side Alternatively, the distance image sensor 10 may be installed in the corner of the ceiling so that the person in the target area 30 can be photographed from obliquely above. Even if the distance image photographed in this way is used, It is possible to detect the human body in the target area 30 by the human body detection method.

Also, the human body detection unit 20, the number of pixels of the pixel region assigned the same identifier, but using a determination condition Z1 that exceeds the number of pixels corresponding to the body size, in place of the determination condition Z1 If the determination condition Z2 that the plurality of pixel areas to which the same identifier is assigned becomes a single peak object when the threshold value is extended from the pixel value of the shortest pixel to a certain distance, the area extraction process and End the identifier assignment process, and select the pixel area to which the same identifier is assigned.

  Here, it is assumed that the human body detection unit 20 assigns the identifier LB1 to the pixel region A1 corresponding to the adult 50, and assigns a separate identifier LB2 to the pixel region A11 corresponding to the child 51. When the human body detection unit 20 extends the above threshold from the pixel value of the shortest pixel to a certain distance (for example, 10 cm), the pixel areas to which the identifier LB1 is assigned are areas A1 to A3, and the identifier LB2 is assigned. The pixel areas are areas A11 to A13 (see FIG. 8B). When the fixed distance is set to a distance (for example, 10 cm) shorter than the length of the human head, the pixel areas A1 to A3 to which the identifier LB1 is assigned and the pixel areas A11 to A13 to which the identifier LB2 is assigned are In the case of a human head, these pixel areas A1 to A3 and A11 to A13 are all unimodal objects. Therefore, if the entire pixel area to which the same identifier is assigned when the above threshold is extended to a certain distance is a unimodal object, the human body detection unit 20 determines the pixel area to which the same identifier is assigned. It can be extracted by judging the human body. When this determination condition Z2 is used, it is possible to reliably detect the human body by detecting the shape of the human head even if the height of the person in the target region is different. Further, since the region extraction process and the identifier assigning process have been completed for the pixel region to which the identifier is assigned at the stage where the determination condition Z2 is satisfied, compared with the case where the pixel region is extracted by extending the threshold value to the floor surface, There is an advantage that the presence of a person can be detected in a short time, and the capacity of the memory for storing the pixel region to which the identifier is added can be reduced. If both of the above-described determination conditions Z1 are satisfied in addition to the determination condition Z2, the region extraction process and the identifier assignment process may be terminated, and the pixel area to which the same identifier is assigned may be extracted from the human body. Therefore, it is possible to extract the human body more reliably without erroneously detecting an object having a size smaller than that of the human body.

  In addition, the human body detection unit 20 overlaps the pixels in the pixel area that has already been extracted among the plurality of pixel areas to which the same identifier is assigned instead of the above-described determination condition Z1. If the determination condition Z3 that the number of pixels that have not been processed is equal to or less than a predetermined threshold is satisfied, the region extraction processing and the identifier assignment processing are terminated, and the pixel region to which the same identifier is assigned is extracted from the human body. You may do it.

  Here, it is assumed that the human body detection unit 20 assigns the identifier LB1 to the pixel region A1 corresponding to the adult 50, and assigns a separate identifier LB2 to the pixel region A11 corresponding to the child 51. In the case of an adult 50, from the top of the head to the vicinity of the threshold L4, each time a new pixel area is added to the pixel area to which the identifier LB1 is assigned, the number of pixels in this pixel area increases greatly, but the horizontal section of the head Since the increase in the horizontal cross section decreases as the height approaches the maximum height, the number of pixels in the pixel area to which the identifier LB1 is assigned hardly increases even when the threshold is changed between L4 and L6. . Similarly, in the case of the child 51, from the top of the head to the vicinity of the threshold A12, the number of pixels in the pixel area increases greatly each time a new pixel area is added to the pixel area to which the identifier LB2 is assigned. Since the increase in the horizontal section decreases as the horizontal section approaches the maximum height, the number of pixels in the pixel area to which the identifier LB2 is assigned is almost the same even if the threshold is changed between L12 and L14. Does not increase.

  Accordingly, as shown in FIG. 8C, in the case of an adult 50, the threshold values are increased to L1, L2,... L5, and the pixel regions A1 to A5 to which the identifier LB1 is assigned are newly extracted. Since the determination condition Z3 that the number of pixels in the pixel area A5 that do not overlap with the pixels of the pixel areas A1 to A4 that have already been extracted is equal to or less than a predetermined threshold value is satisfied, the human body detection unit 20 Therefore, it is determined that the shape of the head can be stably extracted by satisfying the determination condition Z3, and the pixel areas A1 to A5 to which the identifier LB1 is assigned are determined to be human bodies. Similarly, in the case of the child 51, when the pixel areas A11 to A13 to which the threshold value is increased to L11, L12, and L13 and the identifier LB2 is assigned are extracted, the pixels in the newly extracted pixel area A13 are already Since the determination condition Z3 that the number of pixels that do not overlap with the pixels of the extracted pixel areas A11 and A12 is equal to or less than a predetermined threshold is satisfied, the human body detection unit 20 recognizes that the determination condition Z3 satisfies the determination condition Z3. It can be determined that the shape of the part has been stably extracted, and the pixel areas A11 to A13 to which the identifier LB2 is assigned can be determined as the human body and extracted. Even when this determination condition Z3 is used, since the human body is determined by detecting the shape of the head of the human body, the human body can be reliably detected. In addition, since the region extraction process and the identifier assigning process have been completed for the pixel area to which the identifier is assigned at the stage where the determination condition Z3 is satisfied, compared with the case where the pixel area is extracted by extending the threshold value to the floor surface, There is an advantage that the presence of a person can be detected in a short time, and the capacity of the memory for storing the pixel region to which the identifier is added can be reduced.

  It should be noted that the human body detection unit 20 determines that the number of pixels in the pixel area to which the same identifier is assigned exceeds the number of pixels corresponding to the size of the human body, and a new one of the plurality of pixel areas to which the same identifier is assigned. If both of the determination conditions Z3 that the number of pixels that are not overlapped with the pixels in the already extracted pixel area are equal to or less than the predetermined threshold among the pixels in the extracted pixel area are satisfied, The identifier assigning process may be terminated, and the pixel region to which the same identifier is assigned may be extracted as a human body. In this case, as shown in FIG. At the stage of extraction, the pixel area to which the identifiers LB1 and LB2 are assigned is determined as a human body.

Meanwhile, the human body detection unit 20 of the present embodiment, instead of the above-described determination conditions Z1, among the plurality of pixel areas assigned the same identifier, the newly extracted pixel region, is already a pixel area extracted If included in the newly extracted pixel area, if the determination condition Z4 that the average value of the widths of the strip-like portions outside the already extracted pixel area is equal to or smaller than a predetermined threshold is satisfied, The extraction process and the identifier assignment process are finished, and the pixel region to which the same identifier is assigned is extracted as a human body .

  The human body detection unit 20 performs a region extraction process for extracting a pixel region whose pixel value is equal to or smaller than the threshold value by gradually increasing the threshold value, and determines a partially overlapping pixel region as the same object, In the case where the human body in the target area 30 is upright, the first of the plurality of pixel areas to which the same identifier is assigned is first executed. The extracted pixel area is included in the extracted pixel area with a little threshold. In particular, in the human head, a plurality of pixel regions to which the same identifier is assigned are concentric. In addition, since the increase in the horizontal cross section decreases from the top of the head toward the height position at which the horizontal cross section of the head reaches the maximum, a newly extracted pixel area to which the same identifier is assigned is newly extracted. If the determination condition Z4 that the average value of the width of the band-shaped portion outside the already extracted pixel area is equal to or less than a predetermined threshold is satisfied in the pixel area, the shape of the head can be detected reliably. The pixel area to which the same identifier can be determined is extracted as a human body.

  Here, in the newly extracted pixel area among the plurality of pixel areas to which the same identifier is assigned, the width of the strip-shaped portion outside the already extracted pixel area can be measured by the following method. As shown in FIG. 7, the human body detection unit 20 defines two straight lines PQ and RS that are orthogonal to each other through the barycentric position O of the pixel area A1 detected first. Next, the human body detection unit 20 obtains two points T1 and T2 at which the above-described band-like part (for example, a part obtained by removing the pixel area A4 from the pixel area A5) and the line segment OP, and obtains two points T1 and T2. In addition to obtaining the distance between them, six points T3 to T8 at which the above-mentioned band-like portion intersects with the other three line segments OQ, OR, and OS are obtained, and between two points T3 and T4, between T5 and T6, and T7. , T8, and the average of these four distances, the average value of the width of the band-like portion can be easily determined. Thus, even when this determination condition Z4 is used, since the human body is determined by detecting the shape of the head of the human body, the human body can be reliably detected. In addition, since the region extraction process and the identifier assigning process have been completed for the pixel region to which the identifier is assigned at the stage where the determination condition Z4 is satisfied, compared with the case where the pixel region is extracted by extending the threshold value to the floor surface, There is an advantage that the presence of a person can be detected in a short time, and the capacity of the memory for storing the pixel region to which the identifier is added can be reduced. If both of the above-described determination conditions Z1 are satisfied in addition to the determination condition Z4, the region extraction process and the identifier assignment process are terminated, and the pixel region to which the same identifier is assigned is extracted from the human body. The human body can be extracted more reliably without erroneously detecting a size smaller than the human body.

  In each of the detection methods described above, after the human body detection unit 20 performs the region extraction process and the identifier assignment process, the pixel value of the pixel in the pixel region to which the same identifier is assigned is set to “1”, A binarized image with a pixel value of “0” may be created. By creating an image in which the peak object (human body) and the background portion are separated, the subsequent peak object extraction process and easy When there are a plurality of ridged objects, it is possible to easily identify individual ridged objects.

It is a block diagram which shows embodiment of this invention. (A) (b) is explanatory drawing explaining a detection operation same as the above. (A) (b) is explanatory drawing in the case of detecting an adult. (A) (b) is explanatory drawing in the case of detecting a child. (A) (b) is explanatory drawing explaining a detection operation same as the above. (A)-(d) is explanatory drawing explaining the detection operation same as the above. It is explanatory drawing explaining the detection operation same as the above. (A)-(d) is explanatory drawing explaining the detection operation same as the above. (A)-(c) is operation | movement explanatory drawing same as the above. It is a block diagram which shows the structural example of the sensitivity control part in the same as the above. (A)-(d) is explanatory drawing which shows the operation example same as the above. (A)-(d) is explanatory drawing which shows the other operation example same as the above. (A)-(d) is explanatory drawing which shows the other operation example of the same as the above. It is a top view which shows the structural example of the photon detection element used for the same as the above. It is a principal part disassembled perspective view of the photon detection element shown in FIG. It is AA sectional view taken on the line of FIG. It is operation | movement explanatory drawing of the principal part of the photon detection element used for the same as the above.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Photodetection element 2 Light emission source 3 Control circuit part 4 Image generation part 10 Distance image sensor 20 Human body detection part

Claims (4)

A human body detection method for detecting a human body in a target area using a distance image having a pixel value as a distance from an imaging means to an object in the target area,
After obtaining the distance image by imaging the lower target area by the imaging means installed above the target area,
Every time a region extraction process for extracting a pixel region having a pixel value equal to or smaller than a threshold value from a plurality of pixels constituting the distance image is performed a plurality of times by gradually extending the threshold value from the pixel value of the pixel having the shortest pixel value. In addition,
When the newly extracted pixel area overlaps at least partly with the already extracted pixel area, the plurality of overlapping pixel areas are determined to be the same object, and the same as the whole of the plurality of pixel areas Perform identifier assignment processing to assign identifiers,
In the plane of the distance image, two straight lines orthogonal to each other are obtained through the barycentric position of the pixel area detected first.
When a newly extracted pixel area is included in a newly extracted pixel area among a plurality of pixel areas assigned with the same identifier , the already extracted pixel area is excluded from the newly extracted pixel area. In the belt-like portion, the average value of the widths of the four locations that overlap the two straight lines is obtained, and when the determination condition that this average value is equal to or less than a predetermined threshold is satisfied, the region extraction process and the identifier assignment process are terminated. A human body detection method using a distance image, wherein a pixel region to which the same identifier is assigned is extracted as a human body. When both the determination condition and the determination condition that the number of pixels of a plurality of pixel areas to which the same identifier is assigned exceed the number of pixels corresponding to the size of the human body are satisfied, the area extraction process and the identifier assignment process are terminated. 2. The human body detection method using a distance image according to claim 1, wherein a pixel region to which the same identifier is assigned is extracted as a human body. Among the plurality of pixels constituting the distance image, and pixels of the same identifier pixel region assigned to any one of claims 1 or 2, characterized in that binarizing the other pixels A human body detection method using the distance image described in 1 . A human body detection unit that detects a human body in a target region by executing the human body detection method according to any one of claims 1 to 3 using a distance image captured by the imaging unit. A human body detection device characterized by that.

Images