JP2014181949A - Information acquisition device and object detection device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information acquisition device and an object detection device provided with the information acquisition device that enable an influence of disturbance light to be prevented.SOLUTION: An information acquisition device comprises: a projection part 100 that projects infrared light to a target area; an imaging part 200 that images the target area by a CMOS image sensor 240; a light source control part 21a that controls the projection part 100; an imaging signal process circuit 23; and a distance acquisition part 21b that acquires distance information on the basis of a luminance value acquired by the imaging signal process circuit 23. The light source control part 21a emits the infrared light at a light-on timing p1, and stops emitting the infrared light at a light-off timing p2. The distance acquisition part 21b specifies a disturbance light area O on the basis of a difference between luminance values of a light-up image acquired in response to the light-up timing p1 and a light-off image acquired in response to the light-off timing p2, and acquires the distance information on the basis of a luminance value in an area other than the specified disturbance light area O.

Description

  The present invention relates to an information acquisition device that acquires information in a target area and an object detection device that includes the information acquisition device.

  Conventionally, object detection devices using light have been developed in various fields. An object detection apparatus using a so-called distance image sensor can detect not only a planar image on a two-dimensional plane but also the shape and movement of the detection target object in the depth direction.

  As a distance image sensor, a distance image sensor of a type that irradiates a target region with laser light having a predetermined dot pattern is known (for example, Non-Patent Document 1). In such a distance image sensor, a dot pattern when the reference surface is irradiated with laser light is picked up by the image pickup device, and the picked-up dot pattern is held as a reference dot pattern. Then, the reference dot pattern is compared with the actually measured dot pattern captured at the time of actual measurement, and distance information is acquired. Specifically, distance information with respect to the reference region is acquired by a triangulation method based on the position of the reference region set on the standard dot pattern on the measured dot pattern.

19th Annual Conference of the Robotics Society of Japan (September 18-20, 2001) Proceedings, P1279-1280

  However, since the distance image sensor needs to generate light of a dot pattern, there is a problem that the configuration of a projection unit that projects light onto a target area becomes complicated. In addition, since it is necessary to search for dots included in the reference area on the measured dot pattern, there is a problem that complicated calculation processing is required for obtaining distance information.

  Furthermore, when disturbance light such as a fluorescent lamp or sunlight enters the image sensor, a disturbance component may be included in the captured image. In this case, there is a possibility that an error is detected such that there is some object in the area where the disturbance light is incident.

  In view of the above problems, an object of the present invention is to provide an information acquisition device capable of suppressing the influence of disturbance light and an object detection device including the information acquisition device.

A first aspect of the present invention relates to an information acquisition device. An information acquisition apparatus according to this aspect includes a projection unit that projects light onto a target region, an imaging unit that captures the target region with an image sensor, a light source control unit that controls the projection unit, and a predetermined unit on the image sensor. A luminance acquisition unit that acquires a luminance value of the pixel, and an information acquisition unit that acquires information for detecting an object on the target area based on the luminance value acquired by the luminance acquisition unit. The light source control unit emits the light so that the amount of emitted light is different between the first timing and the second timing, and the information acquisition unit obtains the first luminance acquired corresponding to the first timing. Based on the value and the second luminance value acquired corresponding to the second timing, the pixel range excluded from the object detection target is specified, and the second pixel range corresponding to the pixel range other than the specified pixel range is specified. Based on the luminance value of 1, information for detecting the object is acquired.

  A 2nd aspect of this invention is related with an object detection apparatus. The object detection device according to this aspect detects an object present in the target region based on the information acquisition device according to the first aspect and information for detecting the object acquired by the information acquisition device. An object detection unit.

  ADVANTAGE OF THE INVENTION According to this invention, the object detection apparatus provided with the information acquisition apparatus which can suppress the influence of disturbance light, and the said information acquisition apparatus can be provided.

  The effects and significance of the present invention will become more apparent from the following description of embodiments. However, the embodiment described below is merely an example when the present invention is implemented, and the present invention is not limited to the following embodiment.

It is a figure which shows the external appearance structure of the personal computer which incorporated the object detection apparatus which concerns on embodiment. It is a figure which shows the structure of the object detection apparatus which concerns on embodiment. It is a figure which shows the structure of the projection part and imaging part which concern on embodiment. It is a figure which shows the sensitivity of the image sensor which concerns on embodiment, and the waveform of the distance conversion function which prescribes | regulates the relationship between a luminance value and distance. It is the schematic diagram which looked at the captured image and information acquisition apparatus which concern on embodiment from the side. It is the schematic diagram which looked at the lighting image and light extinction image, and information acquisition apparatus which concern on embodiment from the side. It is the schematic diagram which looked at the lighting image and light extinction image, and information acquisition apparatus which concern on embodiment from the side. It is the schematic diagram which looked at the lighting image and light extinction image, and information acquisition apparatus which concern on embodiment from the side. It is the schematic diagram which looked at the lighting image and light extinction image, and information acquisition apparatus which concern on embodiment from the side. It is a figure explaining the method to set a high-intensity area | region from the lighting image which concerns on embodiment. It is a figure explaining the method to set a disturbance light area | region from the lighting image which concerns on embodiment. It is a figure explaining the method to set the threshold value of the disturbance light area | region which concerns on embodiment. It is a flowchart which shows the light emission control processing which concerns on embodiment, and a timing chart which shows lighting, extinguishing timing, and imaging timing. It is a flowchart which shows the distance acquisition process which concerns on embodiment. It is a flowchart which shows the object detection process which concerns on embodiment. It is a flowchart which shows the light emission control process which concerns on the example of a change, and a timing chart which shows lighting, light extinction timing, and imaging timing. It is a flowchart which shows the distance acquisition process which concerns on the example of a change. It is a figure explaining the method to set the threshold value of the high-intensity area | region and disturbance light area | region which concerns on the example of a change. It is a flowchart which shows the distance acquisition process and object detection process which concern on the example of a change. It is a figure which shows the structure of the object detection apparatus which concerns on the example of a change. It is a flowchart which shows the object detection process which concerns on the example of a change.

  Embodiments of the present invention will be described below with reference to the drawings.

  In this embodiment, the object detection apparatus according to the present invention is applied to a notebook personal computer. In addition, the object detection apparatus according to the present invention can be appropriately applied to other devices such as a desktop personal computer and a television. Note that the object detection device according to the present invention does not necessarily have to be integrally mounted on other devices, and may constitute a single device alone.

  In the embodiment described below, the distance acquisition unit 21b corresponds to an “information acquisition unit” recited in the claims. The imaging signal processing circuit 23 corresponds to a “luminance acquisition unit” recited in the claims. The CMOS image sensor 240 corresponds to an “image sensor” recited in the claims. The distance information acquired by the distance acquisition unit 21b corresponds to “information for detecting an object” recited in the claims. The high luminance region H corresponds to a “pixel range that can be an object detection candidate” described in the claims. The disturbance light region O corresponds to “a pixel range excluded from the object detection target” recited in the claims. The lighting timing p1 corresponds to a “first timing” recited in the claims. The extinguishing timing p2 corresponds to a “second timing” recited in the claims. The difference ΔB corresponds to a “difference value” recited in the claims. The luminance difference threshold value ΔBsh corresponds to a “threshold value” recited in the claims. The distance acquisition range ΔL corresponds to the “distance range” recited in the claims. A configuration including the information acquisition device 2 and the information processing unit 3 corresponds to the object detection device according to claim 10. However, the description of the correspondence between the above claims and the present embodiment is merely an example, and the invention according to the claims is not limited to the present embodiment.

  FIG. 1 is a diagram showing a schematic configuration of a personal computer 1 according to the present embodiment. As shown in FIG. 1, the personal computer 1 includes an information acquisition device 2 and an information processing unit 3. In addition, the personal computer 1 includes a keyboard 4, an operation pad 5, and a monitor 6.

  The information acquisition device 2 projects light (infrared light) in an infrared wavelength band longer than the visible light wavelength band over the entire target region, and receives the reflected light with a CMOS image sensor, thereby achieving the target. A distance to an object existing in the region (hereinafter referred to as “distance information”) is acquired. Based on the distance information acquired by the information acquisition device 2, the information processing unit 3 detects a predetermined object existing in the target area, and further detects the movement of the object. Then, the information processing unit 3 controls the function of the personal computer 1 according to the movement of the object.

  For example, when the user performs a predetermined gesture using his / her hand, distance information corresponding to the gesture is transmitted from the information acquisition device 2 to the information processing unit 3. Based on this information, the information processing unit 3 detects the user's hand as a detection target object, and functions associated with the movement of the hand (screen enlargement / reduction, screen brightness adjustment, page turning, etc.) Execute.

  FIG. 2 is a diagram illustrating the configuration of the information acquisition device 2 and the information processing unit 3.

  The information acquisition device 2 includes a projection unit 100 and an imaging unit 200 as the configuration of the optical unit. The projection unit 100 and the imaging unit 200 are arranged so as to be aligned in the X-axis direction.

  The projection unit 100 includes a light source 110 that emits light in an infrared wavelength band.

  The imaging unit 200 includes an aperture 210, an imaging lens 220, a filter 230, and a CMOS image sensor 240. In addition, the information acquisition device 2 includes a CPU (Central Processing Unit) 21, an infrared light source driving circuit 22, an imaging signal processing circuit 23, an input / output circuit 24, and a memory 25 as a circuit unit. Yes.

  The configuration in which the projection unit 100 includes the light source 110 that emits light in the infrared wavelength band and the imaging unit 200 includes the imaging lens 220 and the filter 230 is an example of the configuration according to claim 9. It is.

  The light projected from the light source 110 onto the target area is reflected by an object existing in the target area and enters the imaging lens 220 via the aperture 210.

  The aperture 210 limits light from the outside so as to match the F number of the imaging lens 220. The imaging lens 220 collects the light incident through the aperture 210 on the CMOS image sensor 240. The filter 230 is a bandpass filter that transmits light in the wavelength band including the emission wavelength of the light source 110 and cuts light in the visible wavelength band.

  As will be described later, the CMOS image sensor 240 has sensitivity to the wavelength band of infrared light emitted from the light source 110. The CMOS image sensor 240 receives the light collected by the imaging lens 220 and outputs a signal corresponding to the amount of received light to the imaging signal processing circuit 23 for each pixel. Here, in the CMOS image sensor 240, the output speed of the signal is increased so that the signal of the pixel can be output to the imaging signal processing circuit 23 with high response from the light reception in each pixel.

  In the present embodiment, the effective imaging area of the CMOS image sensor 240 (area where signals are output as a sensor) is, for example, the size of VGA (640 horizontal pixels × 480 vertical pixels). The imaging effective area of the CMOS image sensor 240 may have other sizes such as an XGA (horizontal 1024 pixels × vertical 768 pixels) size or an SXGA (horizontal 1280 pixels × vertical 1024 pixels) size.

  The CPU 21 controls each unit according to a control program stored in the memory 25. With this control program, the functions of the light source control unit 21a and the distance acquisition unit 21b are given to the CPU 21.

  The light source control unit 21 a controls the infrared light source driving circuit 22. The distance acquisition unit 21b acquires distance information as described later based on a signal output from the CMOS image sensor 240.

  The infrared light source driving circuit 22 drives the light source 110 according to a control signal from the CPU 21.

  The imaging signal processing circuit 23 drives the CMOS image sensor 240 under the control of the CPU 21, acquires the luminance signal of each pixel from the signal output from the CMOS image sensor 240, and outputs the acquired luminance signal to the CPU 21. To do. As will be described later, the imaging signal processing circuit 23 applies the exposure time set by the CPU 21 to each pixel of the CMOS image sensor 240, and further sets the CPU 21 for the signal output from the CMOS image sensor 240. A gain signal is applied to obtain a luminance signal for each pixel.

  Based on the luminance signal supplied from the imaging signal processing circuit 23, the CPU 21 calculates the distance from the information acquisition device 2 to each part of the detection target object through processing by the distance acquisition unit 21b. The distance information is acquired for each pixel of the CMOS image sensor 240. The distance information acquisition process will be described later with reference to FIG.

  The input / output circuit 24 controls data communication with the information processing unit 3.

  In addition to the control program executed by the CPU 21, the memory 25 holds a distance conversion function used for acquiring distance information. In addition, the memory 25 is also used as a work area during processing in the CPU 21. The distance conversion function will be described later with reference to FIG.

  The information processing unit 3 includes a CPU 31, an input / output circuit 32, and a memory 33. In addition to the configuration shown in FIG. 2, the information processing unit 3 is provided with a configuration for driving and controlling each unit of the personal computer 1. For convenience, the configuration of these peripheral circuits is not shown.

  The CPU 31 controls each unit according to a control program stored in the memory 33. With this control program, the function of the function detection unit 31b for controlling the function of the personal computer 1 is given to the CPU 31 in accordance with the function of the object detection unit 31a and the signal from the object detection unit 31a.

  The object detection unit 31a extracts the shape of the object from the distance information acquired by the distance acquisition unit 21b, and further detects the movement of the extracted object shape. The function control unit 31b determines whether the movement of the object detected by the object detection unit 31a matches a predetermined movement pattern. If the movement of the object matches the predetermined movement pattern, the movement pattern The control corresponding to is executed.

  FIG. 3A and FIG. 3B are diagrams illustrating configurations of the projection unit 100 and the imaging unit 200. 3A is a plan view showing the projection unit 100 and the imaging unit 200 installed on the circuit board 300, and FIG. 3B is a cross-sectional view taken along line AA ′ of FIG. is there.

  Referring to FIGS. 3A and 3B, the light source 110 includes a light emitting diode (LED), a semiconductor laser, and the like. The light source 110 is adjusted by a lens or the like so that infrared light gathers in the vicinity of the imaging range in the target area in order to increase the use efficiency of infrared light. The light source 110 is mounted on the circuit board 300.

  The imaging unit 200 includes a lens barrel 250 and an imaging lens holder 260 in addition to the aperture 210, the imaging lens 220, the filter 230, and the CMOS image sensor 240 described above. The CMOS image sensor 240 is mounted on the circuit board 300. The imaging lens 220 is attached to the lens barrel 250, and the lens barrel 250 is attached to the imaging lens holder 260 while holding the imaging lens 220. The imaging lens holder 260 has a recess on the lower surface, and the filter 230 is attached to the recess. In this way, the imaging lens holder 260 is installed on the circuit board 300 so as to cover the CMOS image sensor 240 while holding the imaging lens 220 and the filter 230.

  In the present embodiment, the imaging lens 220 is composed of four lenses. However, the number of lenses constituting the imaging lens 220 is not limited to this, and the imaging lens 220 may be configured from other numbers of lenses.

  In addition to the projection unit 100 and the imaging unit 200, the circuit unit 400 constituting the information acquisition device 2 is mounted on the circuit board 300. The CPU 21, the infrared light source driving circuit 22, the imaging signal processing circuit 23, the input / output circuit 24 and the memory 25 shown in FIG. 2 are included in the circuit unit 400.

  FIG. 3C is a diagram schematically illustrating a projection state of infrared light on the target region and an imaging state of the target region by the imaging unit 200.

  In the lower part of FIG. 3C, an infrared light projection state by the projection unit 100 and a target region imaging state by the imaging unit 200 are shown. 3C schematically shows the infrared light projection range in the target area and the imaging range of the imaging unit 200 for the target area. Furthermore, an area corresponding to the effective imaging area of the CMOS image sensor 240 is shown in the upper part of FIG. In FIG. 3C, ΔL indicates a distance acquisition range by the information acquisition device 2, and Lmax and Lmin indicate a maximum distance and a minimum distance that can be acquired by the information acquisition device 2, respectively. In the upper part of FIG. 3C, a projection range, an imaging range, and an imaging effective area when the target area is at the position of the maximum distance Lmax are shown.

  As shown in FIG. 3C, the imaging range and the projection range overlap each other in the target area, and the imaging effective region is positioned in a range where the imaging range and the projection range overlap.

  In order to image the target area with the CMOS image sensor 240, an area corresponding to the effective imaging area of the CMOS image sensor 240 needs to be included in the projection range. On the other hand, as the distance becomes shorter, the projection range becomes narrower and eventually the projection range does not cover the area corresponding to the imaging effective area. Therefore, the minimum distance Lmin needs to be set to be longer than at least the minimum limit distance that the projection range can cover the entire imaging effective area.

  On the other hand, the maximum distance Lmax is set assuming a distance range in which a detection target object such as a hand can exist. If the maximum distance Lmax is too long, the background of the detection target object is reflected in the captured image, which may reduce the detection accuracy of the detection target object. Therefore, the maximum distance Lmax is set assuming a distance range in which a detection target object such as a hand can exist so that the background of the detection target object does not appear in the captured image.

  FIG. 4A is a diagram schematically illustrating the sensitivity of each pixel on the CMOS image sensor 240.

  In the present embodiment, a color sensor is used as the CMOS image sensor 240. Therefore, the CMOS image sensor 240 includes three types of pixels that detect red, green, and blue, respectively.

  In FIG. 4A, R, G, and B indicate the sensitivity of red, green, and blue pixels included in the CMOS image sensor 240, respectively. As shown in FIG. 4A, the sensitivity of the red, green, and blue pixels is substantially the same in the infrared wavelength band of 800 nm or more (the hatched portion in FIG. 4A). reference). Therefore, when the wavelength band of visible light is removed by the filter 230 shown in FIG. 3B, the sensitivity of the red, green, and blue pixels of the CMOS image sensor 240 becomes substantially equal to each other. For this reason, when the same amount of infrared light is incident on the red, green, and blue pixels, the values of the signals output from the pixels of the respective colors are substantially equal. Therefore, there is no need to adjust the signal from each pixel between the pixels, and the signal from each pixel can be used as it is for obtaining distance information.

  FIG. 4B is a diagram schematically illustrating the waveform of the distance conversion function held in the memory 25. For convenience, FIG. 4B shows the maximum distance Lmax and the minimum distance Lmin shown in FIG. 3C and the distance acquisition range ΔL.

As shown in FIG. 4B, the distance conversion function defines the relationship between the luminance value acquired via the CMOS image sensor 240 and the distance corresponding to the luminance value. In general, the amount of light traveling straight is attenuated in inverse proportion to the square of the distance. Accordingly, the infrared light emitted from the projection unit 100 is attenuated to one-quarter of the distance obtained by adding the distance from the projection unit 100 to the target area and the distance from the target area to the imaging unit 200. The light is received by the imaging unit 200.
For this reason, as shown in FIG. 4B, the luminance value acquired via the CMOS image sensor 240 decreases as the distance to the object increases, and increases as the distance to the object decreases. Therefore, the distance conversion function that defines the relationship between the distance and the luminance has a curved waveform as shown in FIG.

  FIG. 5A is a diagram illustrating a captured image obtained by capturing the hand and the window with the light source 110 turned on. FIG. 5B is a schematic view of the information acquisition device 2 in this case as viewed from the side. 5A shows that the darker the hatching, the lower the luminance, and the lower the hatching, the higher the luminance.

  As shown in FIG. 5B, in the target area, a hand is positioned as the detection target object in the distance acquisition range ΔL, and a wall surface is arranged farther than the maximum distance Lmax. The wall surface is provided with a window, and external disturbance light such as sunlight enters.

  As described above, when the detection target object is imaged in the state where the disturbance light is incident, the disturbance light is reflected in the captured image as shown in FIG. 5A, and the shape of the hand cannot be recognized properly.

  Therefore, in the present embodiment, in order to identify the region where disturbance light is incident in the captured image, first, the target region is imaged with the light source 110 turned on, and then the light source 110 is turned off. The target area is imaged. A captured image obtained by capturing the target area with the light source 110 turned on is referred to as a “lighted image”. A captured image obtained by capturing the target area with the light source 110 turned off is referred to as a “light-out image”. Then, the high luminance region is identified from the lit image, the high luminance region of the lit image is compared with the unlit image, the region with a small change in luminance value is determined as the disturbance region, and the disturbance region from the lit image is determined from the distance acquisition target. exclude.

  FIG. 6A is a diagram illustrating a lighting image obtained by capturing only a window in a state where the light source 110 is turned on. FIG. 6B is a schematic view of the information acquisition device 2 in this case as viewed from the side. FIG. 6C is a diagram illustrating a light-off image obtained by capturing only the window with the light source 110 turned off. FIG. 6D is a schematic view of the information acquisition apparatus 2 in this case as viewed from the side.

  As shown in FIG. 6B, since the window is positioned farther than the distance acquisition range ΔL, the amount of reflected light incident on the CMOS image sensor 240 is small even when the light source 110 is turned on. .

  On the other hand, as shown in FIG. 6B, even if the light source 110 is in the on state or as shown in FIG. 6D, the light source 110 enters through the window. The amount of disturbance light is very large.

  Therefore, the amount of light incident on the CMOS image sensor 240 when the light source 110 shown in FIGS. 6C and 6D is turned off, and the light source shown in FIGS. 6A and 6B. The difference in the amount of light incident on the CMOS image sensor 240 when 110 is lit is very small.

  FIG. 7A is a diagram illustrating a lighting image when the window is positioned in the distance acquisition range ΔL. FIG. 7B is a schematic view of the information acquisition device 2 in this case as viewed from the side. FIG. 7C is a diagram showing a light-off image when the window is positioned in the distance acquisition range ΔL. FIG. 7D is a schematic view of the information acquisition device 2 in this case as viewed from the side.

Usually, the amount of disturbance light such as fluorescent light and sunlight is very large compared to the amount of infrared light emitted from the light source 110, and therefore, as shown in FIG. 7B, the window has a distance acquisition range ΔL. When the information acquisition device 2 is moved closer to the position, the output of the CMOS image sensor 240 is saturated and the luminance value becomes the maximum value. Further, as shown in FIGS. 7C and 7D, even when the light source 110 is turned off, the output of the CMOS image sensor 240 is saturated and the luminance value becomes the maximum value.

  Thus, when the light source of disturbance light is approaching the information acquisition device 2, the output of the CMOS image sensor 240 is likely to be saturated.

  Accordingly, the amount of light incident on the CMOS image sensor 240 when the light source 110 shown in FIGS. 7C and 7D is turned off, and the light source shown in FIGS. 7A and 7B. The difference in the amount of light incident on the CMOS image sensor 240 when 110 is lit is substantially the same. Usually, the difference between the luminance in a region where disturbance light is incident with the light source 110 turned on and the luminance in a region where disturbance light is incident with the light source 110 turned off is substantially the same or very small.

  FIG. 8A is a diagram illustrating a lighting image obtained by capturing only the hand in a state where the light source 110 is turned on. FIG. 8B is a schematic view of the information acquisition device 2 in this case as viewed from the side. FIG. 8C is a diagram illustrating a light-off image obtained by capturing only the hand with the light source 110 turned off. FIG. 8D is a schematic view of the information acquisition apparatus 2 in this case as viewed from the side.

  As shown in FIG. 8B, since the hand is positioned in the distance acquisition range ΔL, the amount of reflected light incident on the CMOS image sensor 240 is large when the light source 110 is in a lighting state.

  On the other hand, as shown in FIG. 8D, when the light source 110 is in the extinguished state, no object is positioned in the distance acquisition range ΔL, and only the reflected light from the wall surface arranged in the distance is incident. Therefore, the amount of reflected light incident on the CMOS image sensor 240 is small.

  Therefore, the amount of incident light on the CMOS image sensor 240 when the light source 110 shown in FIGS. 8C and 8D is in the extinguished state, and the light source 110 shown in FIGS. 8A and 8B. The difference in the amount of light incident on the CMOS image sensor 240 when is turned on is very large.

  In this way, in the captured image, the region where the object is reflected has a large difference in luminance between the lighting state and the unlighting state of the light source 110, and the region where the disturbance light is incident is between the lighting state and the unlighting state of the light source 110. The difference in brightness is very small or substantially the same.

  Therefore, by comparing the luminance values of the lit image and the unlit image, it is possible to determine the area where the disturbance light is incident. Note that a region where disturbance light is incident is referred to as a disturbance light region.

  FIG. 9A is a diagram illustrating a lighting image obtained by imaging a hand and a window with the light source 110 turned on. FIG. 9B is a schematic view of the information acquisition apparatus 2 in this case as viewed from the side. FIG. 9C is a diagram illustrating a light-off image obtained by capturing the hand and the window with the light source 110 turned off. FIG. 9D is a schematic view of the information acquisition device 2 in this case as viewed from the side. Note that FIGS. 9A and 9B are similar to FIGS. 5A and 5B.

  In order to identify the disturbance light region from the lit image shown in FIG. 9A, as shown in FIG. 9C, the target region is imaged with the light source 110 turned off, and the extinguished image is acquired. In order to compare the pixels of the lit image and the unlit image, first, an area composed of high-luminance pixels is extracted from the lit image. Note that an area composed of high-luminance pixels is referred to as a high-luminance area.

  Fig.10 (a)-FIG.10 (c) are the figures explaining the method of setting a high-intensity area | region from a lighting image.

  In order to extract a high luminance region from the lighting image, the luminance value is referred to for each pixel of the lighting image, and pixels whose luminance value is equal to or higher than a predetermined luminance threshold Bsh are extracted. Then, a region where the luminance value is equal to or greater than a predetermined luminance threshold Bsh is extracted as the high luminance region H. Thereby, as shown in FIG. 10A, the position of the hand and the region where the disturbance light is incident are extracted as the high luminance region H.

  Note that the configuration in which the high luminance region H is set in the lighting image is an example of the configuration according to claim 4. A configuration in which a region having a luminance value equal to or higher than the luminance threshold Bsh is set as the high luminance region H is an example of a configuration according to claim 5.

  Normally, at the time of object detection, as shown in FIG. 10C, the detection target object is positioned in the distance acquisition range ΔL, so that the luminance value at the position of the detection target object in the lighting image is a maximum distance of the minimum distance Lmin. A luminance value corresponding to the minimum distance Lmax corresponding to the minimum luminance value is output. Therefore, if the luminance value of the pixel of the lighting image is equal to or higher than the luminance corresponding to the distance of the minimum distance Lmin of the distance acquisition range ΔL, it is assumed that at least the area where disturbance light is incident. Further, if the luminance value of the pixel of the lighting image is equal to or greater than the luminance corresponding to the distance of the maximum distance Lmax of the distance acquisition range ΔL, the region where the disturbance light is incident or the region where the detection target object is positioned in the distance acquisition range ΔL It is assumed that

  In the present embodiment, the brightness threshold Bsh for extracting the high brightness area H is set to a brightness value corresponding to the distance of the maximum distance Lmax of the distance acquisition range ΔL.

  For example, when the maximum distance Lmax is set to 100 cm and the luminance value corresponding to 100 cm is 30, the luminance threshold value Bsh is set to 30. Thereby, the area | region where the object was brought close to 100 cm or less, or the area | region where disturbance light entered can be specified.

  When the high-luminance region H is extracted in this manner, a background region in which neither an object nor disturbance light is reflected can be excluded from a comparison calculation target for extracting a disturbance light region to be described later. Thereby, the amount of calculation concerning extraction of a disturbance light area | region can be suppressed.

For example, since the pixel N0 of the lit image shown in FIG. 10A and the pixel F0 of the unlit image shown in FIG. 10B are not positioned in the high luminance region H, a comparison related to the extraction of the disturbance light region is performed. Excluded from calculation.

  In the present embodiment, the luminance threshold value Bsh is set to a luminance value corresponding to the maximum distance Lmax of the distance acquisition range ΔL, so that the position where the object is reflected is also extracted as the high luminance region H. Therefore, even if the amount of disturbance light is slightly small and the luminance corresponding to the distance acquisition range ΔL is output (for example, 110 when turned on, 100 when turned off), the disturbance light region should be determined. Can do.

  Fig.11 (a)-FIG.11 (e) are the figures explaining the method of setting a disturbance light area | region from a lighting image.

When the high luminance area H is set, the difference between the luminance value of each pixel of the lit image included in the high luminance area H and the luminance value of the pixel of the unlit image at a position corresponding to each pixel of the lit image is calculated. . Then, a pixel in which the difference between the luminance value of each pixel of the lit image and the luminance value of each pixel of the unlit image is equal to or smaller than a predetermined luminance difference threshold ΔBsh is extracted. Then, a region where the difference in luminance value is equal to or smaller than a predetermined luminance difference threshold ΔBsh is extracted as the disturbance light region O.

  For example, the difference between the luminance value of the pixel N1 included in the high luminance region H of the lit image and the luminance value of the pixel F1 of the unlit image at the position corresponding to the pixel N1 is obtained. Disturbance light is incident on each of the pixel N1 and the pixel F1, and the change in luminance value is substantially the same or very small. Therefore, the position of the pixel N1 in the high luminance region H is determined to be the ambient light region O. Further, for example, the difference between the luminance value of the pixel N2 included in the high luminance region H of the lit image and the luminance value of the pixel F2 of the extinguished image at a position corresponding to the pixel N2 is obtained. Since the reflected light of the infrared light emitted from the light source 110 is incident on the pixel N2 and no light is incident on the pixel F2, the change in the luminance value is large. Therefore, it is determined that the position of the pixel N2 in the high luminance region H is not the ambient light region O. Similarly, all pixels included in the high luminance area H are compared with the luminance value of the unlit image, and as shown in FIG. Extracted as light region O.

  In addition, the structure where the area | region where a change of a luminance value is small among the high-intensity area | regions H is set as the disturbance light area | region O is an example of the structure of Claim 1. The configuration in which the disturbance light region O is set based on the difference in luminance value between when the light source 110 is turned on and when the light source 110 is turned off is an example of a configuration according to claim 2.

  12A to 12C are diagrams illustrating a method for setting the luminance difference threshold ΔBsh of the disturbance light region O. FIG.

  Normally, at the time of object detection, as shown in FIG. 12C, the detection target object is positioned in the distance acquisition range ΔL. Therefore, when there is no light source other than the light source 110, the position of the detection target object (for example, pixel) As the brightness value of N3), a brightness value corresponding to at least the distance of the maximum distance Lmax of the distance acquisition range ΔL is output. In addition, when there is no light source other than the light source 110 at the time of object detection, 0 is output as the luminance value of each pixel (for example, the pixel F3) of the extinguished image. That is, in the high luminance region H, if the difference in luminance between when the light source 110 is turned on and when the light source 110 is turned off is equal to or less than the luminance corresponding to the distance of the maximum distance Lmax of the distance acquisition range ΔL, the ambient light is incident. It is assumed that there is.

  In the present embodiment, the luminance difference threshold ΔBsh for extracting the disturbance light region O is set to a luminance value corresponding to the distance of the maximum distance Lmax of the distance acquisition range ΔL.

  The configuration in which the region in which the difference in luminance between when the light source 110 is turned on and when the light source 110 is not more than the luminance difference threshold ΔBsh is set as the disturbance light region O is an example of the configuration according to claim 3. A configuration in which the difference in luminance between when the light source 110 is turned on and when the light source 110 is turned off is set to a luminance value corresponding to the distance of the maximum distance Lmax of the distance acquisition range ΔL is an example of a configuration according to claim 7.

  For example, when the maximum distance Lmax is set to 100 cm and the luminance value corresponding to 100 cm is 30, the luminance difference threshold ΔBsh is set to 30. As a result, a region where the difference between the luminance at the time of lighting and the luminance at the time of lighting is 30 or less is extracted as the disturbance light region O.

  Thus, when the disturbance light region O is extracted, the luminance value of each pixel included in the disturbance light region O is corrected to a value (for example, 0) corresponding to an error. Thereby, as shown in FIG. 11D, the disturbance light region O is excluded from the lighting image, and an image in which only the shape of the hand is reflected is generated.

  As described above, since the disturbance light region O is excluded from the lighting image, it is possible to appropriately detect the object.

  FIG. 13A is a flowchart showing the light emission control process. FIG. 13B is a timing chart showing the lighting timing and extinguishing timing of the light source 110 and the imaging timing of the CMOS image sensor 240. The process of FIG. 13A is executed by the function of the light source control unit 21a among the functions of the CPU 21 shown in FIG.

  The CPU 21 drives the infrared light source driving circuit 22 so that the light source 110 is turned on at a predetermined light emission interval T1 and the light source 110 is turned off at a predetermined light emission interval T2 (S11). As a result, as shown in FIG. 13B, the light source 110 emits pulses at a light emission interval T1 and a light extinction interval T2. The light emission interval T1 and the turn-off interval T2 are set in advance to be longer than the exposure time P of the CMOS image sensor 240 set as described above. The light emission interval T1 is set to the same time interval as the turn-off interval T2.

  Then, the CPU 21 determines whether or not the distance acquisition process by the distance acquisition unit 21b has ended (S12). When the distance acquisition process has not ended (S12: NO), the CPU 21 continues the pulse light emission of the light source 110. When the distance acquisition process ends (S12: YES), the CPU 21 ends the light source control process.

  FIG. 14A and FIG. 14B are flowcharts showing the distance acquisition process. 14A and 14B is executed by the function of the distance acquisition unit 21b among the functions of the CPU 21 shown in FIG.

  Referring to FIG. 14A, when the distance acquisition process is started, first, the CPU 21 reads the exposure time P and gain set as described above from the memory 25, and sends them to the imaging signal processing circuit 23. Set (S101). As shown in FIG. 13B, the exposure time P is a time interval that is half of the light emission interval T1 and the turn-off interval T2. The imaging start timing is set to coincide with the light emission start timing of the light source 110. Thus, the imaging process is repeated so that the CMOS image sensor 240 images the target area once at the timing when the light source 110 emits light, and the CMOS image sensor 240 images the target area once at the timing when the light source 110 turns off. It is. In the present embodiment, p1 is a lighting timing of the light source 110, and p2 is a lighting timing of the light source 110.

  When the lighting timing p1 of the light source 110 arrives (S102: YES), the CPU 21 acquires a lighting image from the CMOS image sensor 240 with the set exposure time P and gain (S103), and for each pixel from the acquired lighting image. The luminance value is acquired (S104). The acquired luminance value is transmitted to the CPU 21. Then, the CPU 21 holds the luminance value of each pixel received from the imaging signal processing circuit 23 in the memory 25, and executes disturbance light region correction processing (S105).

  Referring to FIG. 14B, when the lighting image is acquired, the CPU 21 determines whether or not the light extinguishing timing p2 has arrived (S111). When the turn-off timing p2 has not arrived (S111: NO), the CPU 21 waits for processing until the turn-off timing p2 has arrived.

  When the turn-off timing p2 arrives (S111: YES), the CPU 21 acquires a turn-off image from the CMOS image sensor 240 with the set exposure time P and gain, similarly to the turn-on image acquisition (S112). A luminance value is acquired for each pixel from the image (S113). The acquired luminance value is transmitted to the CPU 21.

The CPU 21 holds the luminance value of each pixel received from the imaging signal processing circuit 23 in the memory 25, and further sets the area of the lighting image whose luminance is equal to or higher than the luminance threshold Bsh as the high luminance area H (S114). The luminance threshold Bsh is set to a luminance value corresponding to the distance of the maximum distance Lmax of the distance acquisition range ΔL. For example, the luminance threshold value Bsh is set to 30, and a high luminance region H including only pixels having a luminance value of 30 or more is extracted.

  When the high luminance area H is extracted, the CPU 21 calculates the difference ΔB by subtracting the luminance value of each pixel of the extinguished image from the luminance value of each pixel of the high luminance area H of the lit image (S115). Then, the CPU 21 sets the area of the lighting image in which the luminance value difference ΔB is equal to or less than the luminance difference threshold ΔBsh as the disturbance light area O (S116). The luminance difference threshold value ΔBsh is set to a luminance value corresponding to the distance of the maximum distance Lmax of the distance acquisition range ΔL. For example, the luminance difference threshold value ΔBsh is set to 30, and the disturbance light region O including only pixels having the luminance difference ΔB of 30 or less is extracted.

  Then, the CPU 21 determines whether or not the disturbance light region O has been set in the lighting image (S117). When the disturbance light region O cannot be set in the lighting image (S117: NO), the CPU 21 determines that disturbance light is not included in the lighting image, and advances the processing to S106 in FIG. When the disturbance light region O can be set in the lighting image (S117: YES), the CPU 21 corrects the luminance value of each pixel included in the disturbance light region O of the lighting image to a value (for example, 0) corresponding to an error. (S118).

  Returning to FIG. 14A, the CPU 21 converts the luminance value of the lighting image into a distance by a calculation based on the distance conversion function held in the memory 25 (S106), and the distance acquired by the conversion is respectively calculated. A distance image is generated by setting the corresponding pixel (S107). At this time, the CPU 21 sets a value (for example, 0) indicating an error for the pixel in error in S118.

  After the distance image is generated in this way, the CPU 21 determines whether or not the distance information acquisition operation has ended (S108). If the distance information acquisition operation is not completed (S108: NO), the CPU 21 returns the process to S101 and waits for the next distance information acquisition timing.

  FIG. 15 is a flowchart showing the object detection process. The processing in FIG. 15 is executed by the function of the object detection unit 31a among the functions of the CPU 31 shown in FIG.

  When the distance image is acquired in S114 of FIG. 15 (S201: YES), the CPU 31 determines a predetermined value ΔD from the distance value of the highest gradation in the distance image (the distance value that represents the closest approach to the information acquisition device 2). A value obtained by subtracting is set as the distance threshold value Dsh (S202).

  Next, the CPU 31 classifies an area having a distance value (gradation value) higher than the distance threshold value Dsh as a target area on the distance image (S203). Then, the CPU 31 executes the contour extraction engine, compares the contour of the divided target area with the object shape extraction template held in the memory 25, and compares the contour corresponding to the contour held in the object shape extraction template. The target region is extracted as a region corresponding to the detection target object (S204). In addition, when a detection target object is not extracted in S204, extraction of the detection target object with respect to the distance image is an error.

  Thus, when the extraction process of the detection target object is finished, the CPU 31 determines whether or not the object detection operation is finished (S205). If the object detection operation has not ended (S205: NO), the CPU 31 returns to S201 and waits for the next distance image to be acquired. And if the next distance image is acquired (S201: YES), CPU21 will perform the process after S202 and will extract a detection target object from the said distance image (S202-S204).

<Effect of embodiment>
As described above, according to the present embodiment, since the disturbance light region O is excluded from the lighting image, it is possible to appropriately detect the object.

  Further, according to the present embodiment, since the high-intensity region H is extracted before the disturbance light region O is extracted, the amount of comparison calculation required for the extraction of the disturbance light region O can be suppressed.

  Further, according to the present embodiment, the luminance threshold value Bsh is set to a luminance value corresponding to the maximum distance Lmax of the distance acquisition range ΔL, so that even if the amount of disturbance light is slightly small, the disturbance light region O can be determined.

  Moreover, according to this Embodiment, since distance information is acquired from the calculation based on the distance conversion function hold | maintained at the memory 25, distance information can be acquired by simple calculation processing.

<Example of change>
While the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications can be made to the configuration example of the present invention.

  For example, in the above embodiment, the extinguishing image is acquired once in the extinguishing interval T2 and one extinguishing image is compared with the lighting image. However, the extinguishing interval T3 longer than the extinguishing interval T2 is set, The unlit image and the lit image may be compared.

  In the following modification example, the lighting timing p1 corresponds to a “first timing” recited in the claims. The first turn-off timing p21 corresponds to “second timing” described in the claims, and the second turn-off timing p22 corresponds to “another timing” described in the claims. However, the description of the correspondence between the claims and the modification is merely an example, and the claimed invention is not limited to the modification.

  FIG. 16A is a flowchart showing a light emission control process according to the modified example. FIG. 16B is a timing chart showing the lighting timing and extinguishing timing of the light source 110 and the imaging timing of the CMOS image sensor 240 in this case. The process of FIG. 16A is executed by the function of the light source control unit 21a among the functions of the CPU 21 shown in FIG.

  The CPU 21 drives the infrared light source driving circuit 22 so that the light source 110 is turned on at a predetermined light emission interval T1 and the light source 110 is turned off at a predetermined light emission interval T3 (S21). Accordingly, as shown in FIG. 16B, the light source 110 emits pulses at the light emission interval T1 and the extinction interval T3. The light emission interval T1 in this modification is set in the same manner as in the above embodiment, and the turn-off interval T3 is set to a time interval that is twice the light emission interval T1.

  Then, the CPU 21 determines whether or not the distance acquisition process by the distance acquisition unit 21b has ended (S22). If the distance acquisition process has not ended (S22: NO), the CPU 21 continues the pulsed light emission of the light source 110. When the distance acquisition process ends (S12: YES), the CPU 21 ends the light source control process.

  FIG. 17 is a flowchart illustrating disturbance light region correction processing according to the modified example. 17 is executed by the function of the distance acquisition unit 21b among the functions of the CPU 21 shown in FIG.

As shown in FIG. 14A, when a lighting image is acquired at lighting timing p1 (
S101 to S104), the CPU 21 determines whether or not the first turn-off timing p21 has arrived (S121). When the first turn-off timing p21 arrives (S121: YES), the lighted image and the first light-off image are compared in the same manner as in the above embodiment, and it is determined whether or not the disturbance light region O can be set (S122 to S122). S127).

  As a result of comparing the lit image and the first unlit image, if the disturbance light region O cannot be set (S127: NO), the CPU 21 determines that the lit image does not include disturbance light, and the process is illustrated in FIG. ) To S106.

  When the disturbance light region O can be set in the lit image (S127: YES), the CPU 21 determines whether or not any temporary disturbance light such as a camera flash is incident, so that the second turn-off timing p22 has arrived. It is determined whether or not (S131). When the second turn-off timing p22 arrives (S131: YES), the lit image and the second turned-off image are compared in the same manner as in the above embodiment, and it is determined whether the disturbance light region O can be set (S132- S137).

  When the disturbance light region O cannot be set as a result of comparing the lighting image and the second lighting image (S137: NO), the CPU 21 determines that the disturbance light region O extracted in the comparison between the first lighting image and the lighting image is Then, the process proceeds to S106 in FIG. 14A assuming that it is due to temporary disturbance light.

  As a result of comparing the lit image and the second unlit image, if the disturbance light region O can be set (S137: YES), the CPU 21 indicates that continuous disturbance light such as sunlight is incident on the CMOS image sensor 240. Judgment is made, and the luminance value of each pixel included in the disturbance light region O of the lit image is corrected to a value (for example, 0) corresponding to the error (S138).

  According to this modification, it can be determined whether or not any temporary disturbance light such as a camera flash is incident on the CMOS image sensor 240 when it is turned off. Accordingly, since the continuous disturbance light region O such as sunlight can be excluded from the lighting image, the object detection can be performed more appropriately.

  Thus, after comparing the lighting image acquired at the lighting timing p1 and the first lighting image acquired at the first lighting timing p21, the lighting image and the second lighting image acquired at the second lighting timing p22 are compared, The configuration for setting the disturbance light region O is an example of a configuration according to claim 8.

  In this modified example, the two unlit images and the lit image are compared, but the acquisition timing of the unlit image may be set three or more times, and the three or more unlit images and the lit image may be compared. In this way, although the extinguishing period becomes longer and the amount of calculation increases, it is possible to more reliably determine whether the disturbance light incident upon extinguishing is temporary disturbance light or continuous disturbance light.

  In the above embodiment, the luminance threshold value Bsh for extracting the high luminance region H is set to a luminance value corresponding to the maximum distance Lmax of the distance acquisition range ΔL, but is not limited to this. For example, as shown in FIG. 18A, a luminance value slightly higher than the luminance value corresponding to the maximum distance Lmax may be set.

  In this way, as shown in FIG. 18B, even if the background of the lit image is a little bright due to illumination or the like, the detection target object and the area where the disturbance light is incident are defined as the background. The high luminance region H can be set so that it can be properly distinguished.

In the above embodiment, the luminance difference threshold ΔBsh for extracting the disturbance light region O is set to a luminance value corresponding to the maximum distance Lmax of the distance acquisition range ΔL, but the maximum distance Lmax of the distance acquisition range ΔL. As long as it is less than or equal to the luminance value corresponding to, other values may be set.

  For example, as shown in FIG. 18C, 25 may be 10% of the maximum gradation 256 of the luminance value of the pixel of the CMOS image sensor 240.

  As described above, since the difference between the luminance value of the disturbance light at the time of lighting and the luminance value of the disturbance light at the time of extinction is substantially the same or very small, even in this case, the disturbance light region O is appropriately determined. be able to.

  The configuration in which the brightness difference threshold ΔBsh is set to be equal to or less than the brightness value corresponding to the maximum distance Lmax of the distance acquisition range ΔL is an example of a configuration according to claim 6.

  However, even in a region where disturbance light is incident, when the distance between the maximum distance Lmax of the distance acquisition range ΔL and the wall surface of the background is short, the infrared light emitted from the light source 110 reflects the wall surface, It can be assumed that the difference in luminance value between when the light source 110 is turned on and when the light source 110 is turned off is slightly larger. Therefore, it is desirable that the luminance difference threshold ΔBsh is set to a luminance value corresponding to the maximum distance Lmax of the distance acquisition range ΔL as in the above embodiment.

  Moreover, in the said embodiment, although the disturbance light area | region O was determined based on the difference of the luminance value at the time of lighting of the light source 110 and light extinction, the luminance value when infrared light is radiate | emitted with 1st output power. And the disturbance light area | region O may be determined based on the difference of the luminance value when infrared light is radiate | emitted with 2nd output power different from 1st output power.

  For example, first, a captured image when infrared light is emitted with 100% output power is acquired, and then a captured image when infrared light is output with 50% output power is acquired. A luminance value difference is calculated for each captured image, and an area where the luminance value difference is equal to or smaller than a predetermined luminance difference threshold ΔBsh is set as the disturbance light area O. Then, as in the above embodiment, the luminance value of each pixel included in the disturbance light region O is corrected to a value (for example, 0) corresponding to an error.

  Note that the luminance difference threshold value ΔBsh in the present modified example emits infrared light with the luminance corresponding to the maximum distance Lmax of the distance acquisition range ΔL when the infrared light is emitted with the first emission power and the second emission power. Is set to be equal to or less than the luminance difference corresponding to the maximum distance Lmax of the distance acquisition range ΔL.

  For example, when the luminance corresponding to the maximum distance Lmax of the distance acquisition range ΔL is set to 30 with the emission power of 100%, the luminance corresponding to the maximum distance Lmax of the distance acquisition range ΔL is 15 with the emission power of 50%. The brightness difference threshold ΔBsh is set to 15.

  As described above, when the brightness difference threshold ΔBsh emits infrared light with the brightness corresponding to the maximum distance Lmax of the distance acquisition range ΔL when the infrared light is emitted with the first emission power and the second emission power. A configuration that is set to be equal to or smaller than the difference in luminance corresponding to the maximum distance Lmax of the distance acquisition range ΔL is an example of a configuration according to claim 6.

  According to this modified example, the disturbance light region O is excluded from the captured image as in the above-described embodiment, so that object detection can be performed appropriately.

In the case of this modified example, the luminance difference threshold ΔBsh is smaller than that in the above embodiment. Therefore, in order to set the luminance difference threshold ΔBsh in the widest possible range, as in the above embodiment, 100 is set. It is desirable to compare a lighting image when infrared light is emitted with a% emission power and a light-off image when infrared light is not emitted.

  Further, in this modified example, the case where infrared light is emitted with 100% emission power and the case where infrared light is emitted with 50% emission power are illustrated, but other combinations of emission powers may be used. .

  In the above embodiment, the high brightness area H is set in order to reduce the comparison calculation process between the lit image and the unlit image. However, the high brightness area H is not set, and the brightness values of all the pixels of the lit image are set. The ambient light region O may be extracted by comparing the luminance values of all the pixels of the extinguished image.

  In this modified example, although the amount of calculation increases compared to the above-described embodiment, it is possible to prevent the area where disturbance light is incident by mistake by setting the high luminance area H.

  When the high luminance region H is not set, the disturbance light region O is set even in a region where the reflected light or disturbance light of the infrared light emitted from the light source 110 is not incident on the CMOS image sensor 240 such as a background. It is assumed that However, as shown in S202 of FIG. 15, since an area that is less than the predetermined distance (luminance value) is not extracted as an object detection target, even if the background or the like is excluded from the distance acquisition target, the influence on the object detection processing There is no.

  In the above embodiment, the distance value is acquired by converting the luminance value into the distance based on the distance conversion function. However, the luminance value may be acquired as it is as information about the distance.

  FIG. 19A is a flowchart showing a luminance image generation process in the case where the luminance value is directly acquired as information on the distance.

  In the flowchart of FIG. 19A, S106 of FIG. 14A is omitted, and S107 of FIG. 14A is changed to S109. That is, in the flowchart of FIG. 19A, the luminance value is set to the corresponding pixel without generating the luminance value of each pixel, and the luminance image is generated (S109). Similar to the above embodiment, a value indicating an error (for example, 0) is set to the pixel for which an error has been set in S118.

  FIG. 19B is a flowchart showing the object detection process. In the flowchart of FIG. 19B, a luminance image is referred to instead of the distance image.

  That is, when a luminance image is acquired in S109 of FIG. 19A (S211: YES), the CPU 31 determines the luminance value of the highest gradation in the luminance image (the luminance value indicating the closest approach to the information acquisition device 2). A value obtained by subtracting the predetermined value ΔB2 from the value is set as the luminance threshold Bsh2 (S212).

  Next, the CPU 31 classifies a region having a luminance value (gradation value) higher than the luminance threshold Bsh2 as a target region on the luminance image (S213). Then, the CPU 31 extracts a region having the highest average luminance value among the divided target regions as a region corresponding to the detection target object (S214). In addition, when a detection target object is not extracted in S214, extraction of the detection target object with respect to the said brightness | luminance image is made into an error.

  Thus, when the extraction process of the detection target object is completed, the CPU 31 determines whether or not the object detection operation is completed (S215). If the object detection operation has not ended (S215: NO), the CPU 31 returns to S211 and waits for the next luminance image to be acquired. Then, when the next luminance image is acquired (S211: YES), the CPU 31 executes the processing after S212 and extracts the detection target object from the distance image (S212 to S214).

  In the flowchart of FIG. 19A, since the luminance value is not converted to the distance based on the distance conversion function, the luminance value of each pixel on the luminance image does not represent an accurate distance. That is, as shown in FIG. 4B, since the luminance and the distance have a relationship represented by a curved graph, the luminance value is adjusted according to this curve in order to obtain the luminance value as an accurate distance. There is a need. In the flowchart of FIG. 19A, since the acquired luminance value is set as it is in the luminance image, the luminance value of each pixel on the luminance image does not represent an accurate distance and includes an error.

  However, in this case as well, the luminance value of each pixel represents a rough distance, so it is possible to detect the detection target object using the luminance image. Therefore, even in this modification, the detection target object can be detected by the flowchart of FIG.

  In the flowchart of FIG. 19A, the luminance value acquired in S103 is set as the luminance image as it is in S121. However, the value set in the luminance image does not have to be a luminance value. Any information that can be expressed may be used.

  Moreover, in the said embodiment, although the object detection apparatus was comprised by the information acquisition apparatus 2 and the object detection part 31a by the side of the information processing part 3, an object detection apparatus may be comprised by one apparatus.

  FIG. 20 is a diagram showing a configuration example in this case. In the configuration example of FIG. 20, the information acquisition device 2 of FIG. 2 is replaced with the object detection device 7. For convenience of explanation, the same reference numerals in FIG. 20 denote the same parts as in FIG.

  In the configuration example of FIG. 20, the function of the object detection unit 31 a is removed from the CPU 31, and the function of the object detection unit 21 d is added to the CPU 21. Further, in the configuration example of FIG. 20, the distance acquisition unit 21b of FIG. 2 is replaced with a luminance information acquisition unit 21c. The luminance information acquisition unit 21c generates a luminance image based on the luminance value acquired from the CMOS image sensor 240. The object shape extraction template is held in the memory 25.

  In the present modification, the luminance information acquisition unit 21c corresponds to an “information acquisition unit” recited in the claims. The object detection device 7 corresponds to the object detection device according to claim 10. However, the description of the correspondence between the above claims and this modification is merely an example, and the claimed invention is not limited to this embodiment.

  FIG. 21 is a flowchart showing object detection processing in the present modification. In the flowchart of FIG. 21, S101 to S109 are performed by the luminance information acquisition unit 21c of FIG. 20, and S212 to S214 are performed by the object detection unit 21d of FIG. The processing of S101 to S109 is the same as S101 to S109 of FIG. 19A, and the processing of S212 to S214 is the same as S212 to S214 of FIG. Is omitted.

  When the disturbance light region O is corrected to a value indicating an error by comparing the lit image and the unlit image (S101 to S105), and a luminance image is generated (S109), the CPU 21 executes the processing of S212 to S214 and detects it. The target object is detected. The comparison between the lit image and the unlit image, generation of the luminance image, and object detection processing (S101 to S214) are repeatedly executed until the object detection operation is completed (S216).

  According to this modified example, since the disturbance light region O is excluded from the lighting image as in the above embodiment, the object detection can be performed appropriately.

  In the above embodiment and the modification, the distance and the luminance value are acquired for all the pixels on the CMOS image sensor 240, but the distance and the luminance value are not necessarily acquired for all the pixels. A distance and a luminance value may be acquired every other pixel.

  In the above embodiment, the filter 230 of the imaging unit 200 is arranged between the imaging lens 220 and the CMOS image sensor 240. However, the arrangement position of the filter 230 is not limited to this, and the imaging lens 220 is not limited thereto. It may be closer to the target area.

  In the above embodiment, the distance information is acquired by software processing by the function of the CPU 21, but acquisition of the distance information may be realized by hardware processing by a circuit.

  Furthermore, in the above-described embodiment, the CMOS image sensor 240 is used as the light receiving element, but a CCD image sensor can be used instead. Also, light in a wavelength band other than infrared can be used for distance acquisition.

  In addition, the embodiment of the present invention can be variously modified as appropriate within the scope of the technical idea shown in the claims.

DESCRIPTION OF SYMBOLS 1 ... Personal computer 2 ... Object detection apparatus 3 ... Information processing part 21a ... Light source control part 21b ... Distance acquisition part (information acquisition part)
21c ... Luminance information acquisition unit (information acquisition unit)
21d: Object detection unit 23: Imaging signal processing circuit (luminance acquisition unit)
31a ... Object detection unit 100 ... Projection unit 110 ... Light source 200 ... Imaging unit 220 ... Imaging lens 230 ... Filter 240 ... CMOS image sensor (image sensor)
H ... High luminance region (pixel range that can be a candidate for object detection)
Lmax ... maximum distance Lmin ... minimum distance O ... disturbance light region (pixel range excluded from object detection target)
p1 ... lighting timing (first timing)
p2 ... extinguishing timing (second timing)
p21 ... 1st extinction timing (2nd timing)
p22 ... Second turn-off timing (another timing)
ΔB ... Difference (difference value)
ΔBsh ... Brightness difference threshold (threshold)
ΔL ... Distance acquisition range (distance range)

Claims (10)

A projection unit that projects light onto the target area;
An imaging unit for imaging the target area by an image sensor;
A light source control unit for controlling the projection unit;
A luminance acquisition unit for acquiring a luminance value of a predetermined pixel on the image sensor;
An information acquisition unit that acquires information for detecting an object on the target region based on the luminance value acquired by the luminance acquisition unit;
The light source control unit emits the light so that the amount of emitted light is different between the first timing and the second timing,
The information acquisition unit is configured to detect an object from an object detection target based on a first luminance value acquired corresponding to the first timing and a second luminance value acquired corresponding to the second timing. Specifying a pixel range to be excluded, and acquiring information for detecting the object based on the first luminance value corresponding to a pixel range other than the specified pixel range;
An information acquisition apparatus characterized by that. The information acquisition device according to claim 1,
The light source controller emits the light at the first timing and stops emitting the light at the second timing;
An information acquisition apparatus characterized by that. In the information acquisition device according to claim 1 or 2,
The information acquisition unit specifies a pixel range in which a difference value between the first luminance value and the second luminance value is a predetermined threshold or less as the pixel range excluded from the object detection target.
An information acquisition apparatus characterized by that. In the information acquisition device according to claim 3,
The information acquisition unit sets a pixel range that can be an object detection candidate based on the first luminance value, and identifies the pixel range that is excluded from the object detection target for the set pixel range. To
An information acquisition apparatus characterized by that. The information acquisition device according to claim 4,
The information acquisition unit sets a pixel range in which the first luminance value is a predetermined value or more as the pixel range that can be a detection candidate of the object.
An information acquisition apparatus characterized by that. In the information acquisition device according to any one of claims 3 to 5,
The threshold value is a difference value between the first luminance value and the second luminance value to be acquired by the luminance acquisition unit when the object is located at the maximum distance in a preset distance range. Set to the following values:
An information acquisition apparatus characterized by that. The information acquisition apparatus according to claim 6,
The threshold is set to a difference value between the first luminance value and the second luminance value to be acquired by the luminance acquisition unit when the object is at the position of the maximum distance.
An information acquisition apparatus characterized by that. In the information acquisition device according to any one of claims 1 to 7,
The light source controller further emits the light with an emitted light amount different from the emitted light amount at the first timing at at least one different timing different from the first timing and the second timing,
The information acquisition unit is configured to detect an object based on the first luminance value and the second luminance value, as well as the first luminance value and the luminance value acquired corresponding to the other timing. Identifying the pixel range to be excluded from,
An information acquisition apparatus characterized by that. In the information acquisition device according to any one of claims 1 to 8,
The projection unit includes a light source that emits light in an infrared wavelength band,
The imaging unit includes a filter that transmits light in the infrared wavelength band, and an imaging lens that focuses the light irradiated on the target region onto the image sensor.
An information acquisition apparatus characterized by that. An information acquisition device according to any one of claims 1 to 9,
An object detection unit that detects an object present in the target area based on information for detecting the object acquired by the information acquisition device;
An object detection apparatus characterized by that.

Images