WO2014108976A1 - Dispositif de détection d'objet - Google Patents

Dispositif de détection d'objet Download PDF

Info

Publication number
WO2014108976A1
WO2014108976A1 PCT/JP2013/007621 JP2013007621W WO2014108976A1 WO 2014108976 A1 WO2014108976 A1 WO 2014108976A1 JP 2013007621 W JP2013007621 W JP 2013007621W WO 2014108976 A1 WO2014108976 A1 WO 2014108976A1
Authority
WO
WIPO (PCT)
Prior art keywords
distance
region
pixel
value
object detection
Prior art date
Application number
PCT/JP2013/007621
Other languages
English (en)
Japanese (ja)
Inventor
岸田　孝範
夫馬　正人
山口　光隆
直史 生田
潤 江頭
眞梶　康彦
Original Assignee
三洋電機株式会社
Ｐｕｘ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 三洋電機株式会社, Ｐｕｘ株式会社 filed Critical 三洋電機株式会社
Publication of WO2014108976A1 publication Critical patent/WO2014108976A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
    • G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
    • G01B11/2513—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object with several lines being projected in more than one direction, e.g. grids, patterns
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/0304—Detection arrangements using opto-electronic means
    • G06F3/0317—Detection arrangements using opto-electronic means in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface
    • G06F3/0321—Detection arrangements using opto-electronic means in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface by optically sensing the absolute position with respect to a regularly patterned surface forming a passive digitiser, e.g. pen optically detecting position indicative tags printed on a paper sheet
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/521—Depth or shape recovery from laser ranging, e.g. using interferometry; from the projection of structured light

Definitions

  • the present invention relates to an object detection device that detects an object in a target area.
  • An object detection apparatus using a so-called distance image sensor can detect not only a planar image on a two-dimensional plane but also the shape and movement of the detection target object in the depth direction.
  • the distance image sensor light of a predetermined wavelength band is projected from a laser light source or LED (Light Emitting Diode) onto the target region, and the target region is imaged by an imaging element such as a CMOS image sensor.
  • a laser light source or LED Light Emitting Diode
  • Non-Patent Document 1 a type that irradiates a target region with laser light having a predetermined dot pattern is known (for example, Non-Patent Document 1).
  • a dot pattern when the reference surface is irradiated with laser light is picked up by the image pickup device, and the picked-up dot pattern is held as a reference dot pattern.
  • the reference dot pattern is compared with the actually measured dot pattern captured at the time of actual measurement, and distance information is acquired.
  • distance information with respect to the reference region is acquired by a triangulation method based on the position of the reference region set on the standard dot pattern on the measured dot pattern.
  • the distance image sensor generally can detect a distance in a predetermined range in the front-rear direction. If the detection target object is at a position outside this range, distance information for the detection target object cannot be acquired. Further, when the detection target object is at a position close to the distance image sensor, the dots of the dot pattern are considerably large on the detection target object, and the dot boundaries are unclear. In this case, it is impossible to acquire distance information with the above method.
  • an object of the present invention is to provide an object detection apparatus that can smoothly detect an object to be detected on a target area.
  • the first aspect of the present invention relates to an object detection apparatus.
  • the object detection apparatus includes a projection unit that projects a dot pattern of light onto a target region, an imaging unit that captures the target region with an image sensor, and a position of a dot on a captured image captured by the imaging unit. And a distance acquisition unit that acquires distance information for each position on the target area, and the object is determined to be inaccurate because the object is located at a position close to the projection unit and the imaging unit.
  • a distance correction unit that corrects the distance information to a predetermined value; and an object detection unit that detects a detection target object based on the distance information corrected by the distance correction unit.
  • the second aspect of the present invention relates to an object detection apparatus.
  • the object detection apparatus acquires a luminance value obtained by projecting light onto a target area, an imaging unit that images the target area with an image sensor, and a luminance value of each pixel in the image sensor.
  • an object extraction unit that specifies a high-luminance region on the captured image and extracts a region corresponding to the detection target object from the identified high-luminance region.
  • an object detection apparatus that can smoothly detect an object to be detected on a target area.
  • FIG. 1 is a diagram illustrating a configuration of an object detection apparatus according to a first embodiment.
  • 1 is a diagram illustrating a configuration of an object detection apparatus according to a first embodiment. It is a figure which shows the irradiation state of the laser beam with respect to the target area
  • FIG. 6 is a diagram illustrating a reference pattern generation method according to the first embodiment. It is a figure explaining the distance detection method which concerns on Example 1.
  • FIG. It is a figure which shows a distance image generation process and a brightness
  • FIG. FIG. 6 is a diagram illustrating an example of generating a distance image according to the first embodiment.
  • FIG. 6 is a diagram illustrating a configuration of an object detection device according to a second embodiment. It is a figure which shows the object detection process and object detection method which concern on Example 2.
  • FIG. 10 is a diagram illustrating a problem in the second embodiment. It is a figure which shows the object detection process which concerns on the example of a change of Example 2.
  • FIG. 10 is a diagram illustrating an example of object detection according to a modification of the second embodiment. It is a figure which shows the object detection process which concerns on the other modification of Example 2.
  • the laser drive circuit 22 and the projection optical system 100 correspond to a “projection unit” recited in the claims.
  • the imaging signal processing circuit 23 and the light receiving optical system 200 correspond to an “imaging unit” recited in the claims.
  • the distance acquisition unit 21b corresponds to a “distance acquisition unit” recited in the claims.
  • the object detection unit 21c corresponds to an “object detection unit” recited in the claims.
  • the distance information correction unit 21d corresponds to a “distance correction unit” recited in the claims.
  • the image processing unit 21e corresponds to an “object extraction unit” recited in the claims.
  • the CMOS image sensor 240 corresponds to an “image sensor” recited in the claims.
  • Example 1 The present embodiment shows an example of the configuration described in claim 1.
  • FIG. 1 shows a schematic configuration of an object detection apparatus 1 according to the present embodiment.
  • the object detection device 1 is built in a television 2.
  • the television 2 includes an information processing unit 3 inside, and is controlled by a signal from the information processing unit 3.
  • the object detection device 1 projects light having a wavelength longer than the visible light wavelength band over the entire target area, and receives the reflected light with a CMOS image sensor, thereby distances to various parts of the object existing in the target area. (Hereinafter referred to as “three-dimensional distance information”).
  • the object detection device 1 detects an object in the target area based on the acquired three-dimensional distance information, and detects the movement of the object in the target area from the change in the three-dimensional distance information. Then, the object detection device 1 transmits a signal corresponding to the movement of the object to the information processing unit 3 of the television 2 via the wiring in the television 2.
  • the information processing unit 3 controls the television 2 based on a signal corresponding to the movement of the object received from the object detection device 1.
  • a signal corresponding to the user's gesture is transmitted from the object detection device 1 to the information processing unit 3 of the television 2. Then, the information processing unit 3 executes functions of the television 2 such as channel switching and volume up / down associated with signals corresponding to gestures in advance.
  • the object detection device 1 may be built in a game machine, a personal computer, or the like.
  • the information processing unit of the game machine executes a game machine function associated with a signal corresponding to a gesture in advance.
  • a character on the television screen is operated in accordance with a user's gesture, and the game battle state changes.
  • FIG. 2 is a diagram illustrating a configuration of the object detection device 1 and the information processing unit 3 of the television 2.
  • the object detection apparatus 1 includes a projection optical system 100 and a light receiving optical system 200 as a configuration of an optical unit.
  • the projection optical system 100 and the light receiving optical system 200 are arranged in the object detection apparatus 1 so as to be aligned in the X-axis direction.
  • the projection optical system 100 includes a laser light source 110, a collimator lens 120, a mirror 130, and a diffractive optical element (DOE).
  • the light receiving optical system 200 includes an aperture 210, an imaging lens 220, a filter 230, and a CMOS image sensor 240.
  • the object detection apparatus 1 includes a CPU (Central Processing Unit) 21, a laser driving circuit 22, an imaging signal processing circuit 23, an input / output circuit 24, and a memory 25 as a circuit unit configuration.
  • CPU Central Processing Unit
  • the laser light source 110 outputs laser light having a wavelength longer than the wavelength band of visible light (for example, about 830 nm) in the direction away from the light receiving optical system 200 (X-axis negative direction).
  • the collimator lens 120 converts the laser light emitted from the laser light source 110 into substantially parallel light.
  • the mirror 130 reflects the laser beam incident from the collimator lens 120 side in the direction toward the DOE 140 (Z-axis positive direction).
  • the DOE 140 has a diffraction pattern on the incident surface. Due to the diffractive action of this diffraction pattern, the laser light incident on the DOE 140 is converted into laser light having a predetermined dot pattern and irradiated onto the target area.
  • the diffraction pattern of the DOE 140 has, for example, a structure in which a step type diffraction grating is formed in a predetermined pattern.
  • the pattern and pitch of the diffraction grating are adjusted so as to convert the laser light that has been made substantially parallel light by the collimator lens 120 into laser light of a dot pattern.
  • the DOE 140 irradiates the target region with the laser beam incident from the mirror 130 as a laser beam having a dot pattern that spreads radially.
  • the laser light reflected from the target area enters the imaging lens 220 through the aperture 210.
  • the aperture 210 restricts light from the outside so as to match the F number of the imaging lens 220.
  • the imaging lens 220 collects the light incident through the aperture 210 on the CMOS image sensor 240.
  • the filter 230 is a band-pass filter that transmits light in a wavelength band including the emission wavelength (for example, about 830 nm) of the laser light source 110 and cuts light in the visible wavelength band.
  • the CMOS image sensor 240 is configured to output a captured image with respect to light in the wavelength band of the laser light emitted from the laser light source 110.
  • the CMOS image sensor 240 receives the light collected by the imaging lens 220 and outputs a signal (charge) corresponding to the amount of received light to the imaging signal processing circuit 23 for each pixel.
  • the output speed of the signal is increased so that the signal (charge) of the pixel can be output to the imaging signal processing circuit 23 with high response from the light reception in each pixel.
  • the effective imaging area of the CMOS image sensor 240 (area where signals are output as a sensor) is, for example, the size of VGA (640 horizontal pixels ⁇ 480 vertical pixels).
  • the imaging effective area of the CMOS image sensor 240 may have other sizes such as an XGA (horizontal 1024 pixels ⁇ vertical 768 pixels) size or an SXGA (horizontal 1280 pixels ⁇ vertical 1024 pixels) size.
  • the CPU 21 controls each unit according to a control program stored in the memory 25.
  • the CPU 21 is provided with the functions of a laser control unit 21a, a distance acquisition unit 21b, an object detection unit 21c, and a distance information correction unit 21d.
  • the laser control unit 21a controls the laser driving circuit 22.
  • the distance acquisition unit 21b generates three-dimensional distance information as described later based on the captured image captured by the CMOS image sensor 240.
  • the object detection unit 21c extracts the shape of the object in the image from the three-dimensional distance information acquired by the distance acquisition unit 21b, and further detects the movement of the extracted object shape. Then, the object detection unit 21c determines whether the detected motion of the object matches a predetermined motion pattern, and transmits a signal corresponding to the predetermined motion pattern to the information processing unit 3 of the television 2.
  • the distance information correction unit 21d corrects the three-dimensional distance information based on the luminance value of each pixel of the CMOS image sensor 240.
  • the distance information correction unit 21d specifies a high luminance region on the captured image based on the luminance value of each pixel of the CMOS image sensor 240. Then, the distance information correction unit 21d generates a luminance image in which the luminance value of the pixel corresponding to the high luminance region is applied to the high luminance region, and the generated luminance image is the distance image generated by the distance acquisition unit 21b. To correct the distance image.
  • the function of the distance information correction unit 21d will be described later with reference to FIGS.
  • the laser drive circuit 22 drives the laser light source 110 according to a control signal from the CPU 21.
  • the imaging signal processing circuit 23 drives the CMOS image sensor 240 under the control of the CPU 21, and sequentially captures the signal (charge) of each pixel generated by the CMOS image sensor 240 for each line at a predetermined imaging interval. Then, the captured signals are sequentially output to the CPU 21. Based on the signal (imaging signal) supplied from the imaging signal processing circuit 23, the CPU 21 calculates the distance from the object detection device 1 to each part of the detection target object by processing by the distance acquisition unit 21b.
  • the input / output circuit 24 controls data communication with the information processing unit 3 of the television 2.
  • the memory 25 stores a control template executed by the CPU 21 and a reference template that is referred to when acquiring three-dimensional distance information and an object shape template that is referred to when detecting an object.
  • the memory 25 is also used as a work area during processing in the CPU 21.
  • the information processing unit 3 includes a CPU 31, an input / output circuit 32, and a memory 33.
  • the information processing device 3 is provided with a configuration for controlling the television 2, but for the sake of convenience, the configuration of these peripheral circuits is omitted.
  • CPU 31 controls each unit according to a control program stored in memory 33. With this control program, the function of the function control unit 31a for controlling the function of the television 2 is given to the CPU 31 in accordance with a signal from the object detection unit 21c. The function control unit 31a controls the function of the television 2 as described above based on the detection result by the object detection unit 21c.
  • the input / output circuit 32 controls data communication with the object detection device 1.
  • the projection optical system 100 and the light receiving optical system 200 are installed side by side with a predetermined distance in the X axis direction so that the projection center of the projection optical system 100 and the imaging center of the light receiving optical system 200 are aligned on a straight line parallel to the X axis. Is done.
  • the installation interval between the projection optical system 100 and the light receiving optical system 200 is set according to the distance between the object detection device 1 and the reference plane of the target area.
  • FIG. 3A is a diagram schematically showing the irradiation state of the laser light on the target region
  • FIG. 3B is a diagram schematically showing the light receiving state of the laser light in the CMOS image sensor 240.
  • FIG. 3B shows a flat surface (screen) in the target area and a light receiving state when a person is present in front of the screen.
  • the projection optical system 100 irradiates a target region with laser light having a dot pattern (hereinafter, the entire laser light having this pattern is referred to as “DP light”). .
  • the light flux region of DP light is indicated by a solid line frame.
  • dot regions hereinafter simply referred to as “dots” generated by the diffraction action by the DOE 140 are scattered according to the dot pattern by the diffraction action by the DOE 140.
  • the dots are generated when the laser light from the laser light source 110 is branched by the DOE 140.
  • DP light is distributed on the CMOS image sensor 240 as shown in FIG.
  • the light of Dt0 on the target area shown in FIG. 3A enters the position of Dt0 ′ shown in FIG. 3B on the CMOS image sensor 240.
  • An image of a person in front of the screen is taken upside down on the CMOS image sensor 240 in the vertical and horizontal directions.
  • 4A to 4C are diagrams for explaining a reference pattern setting method used in the distance detection method.
  • a flat reflection plane RS perpendicular to the Z-axis direction is disposed at a position at a predetermined distance Ls from the projection optical system 100.
  • the emitted DP light is reflected by the reflection plane RS and enters the CMOS image sensor 240 of the light receiving optical system 200.
  • an electrical signal is output from the CMOS image sensor 240 for each pixel in the effective imaging area.
  • the output electric signal value (pixel value) for each pixel is developed on the memory 25 of FIG.
  • reflection image an image composed of all pixel values obtained by reflection from the reflection plane RS
  • reflection plane RS is referred to as a “reference plane”.
  • FIG. 4B shows a state in which the light receiving surface is seen through in the positive direction of the Z axis from the back side of the CMOS image sensor 240. The same applies to FIGS. 5A and 5B.
  • a plurality of segment areas having a predetermined size are set for the reference pattern area thus set.
  • the segment areas are set so that adjacent segment areas are arranged at intervals of one pixel in the X-axis direction and the Y-axis direction with respect to the reference pattern area. That is, a certain segment area is set at a position shifted by one pixel in the X-axis direction and the Y-axis direction with respect to the segment area adjacent to the segment area in the X-axis direction and the Y-axis direction.
  • each segment area is dotted with dots in a unique pattern.
  • the pattern of dots in the segment area differs for each segment area in a search range L0 described later.
  • a template is stored in the memory 25 of FIG.
  • the CPU 21 calculates the distance to each part of the object based on the shift amount of the dot pattern in each segment area obtained from the reference template.
  • DP light corresponding to a predetermined segment area Sn on the reference pattern is reflected by the object, and the segment area Sn. It is incident on a different region Sn ′. Since the projection optical system 100 and the light receiving optical system 200 are adjacent to each other in the X-axis direction, the displacement direction of the region Sn ′ with respect to the segment region Sn is parallel to the X-axis. In the case of FIG. 4A, since the object is located at a position closer than the distance Ls, the region Sn 'is displaced in the X-axis positive direction with respect to the segment region Sn. If the object is at a position farther than the distance Ls, the region Sn ′ is displaced in the negative X-axis direction with respect to the segment region Sn.
  • the projection optical system 100 Based on the displacement direction and displacement amount of the region Sn ′ with respect to the segment region Sn (pixel displacement amount D shown in FIG. 4A), the projection optical system 100 reaches the portion of the object irradiated with DP light (DPn).
  • the distance Lr is calculated based on the triangulation method using the distance Ls.
  • the distance from the projection optical system 100 is calculated for the part of the object corresponding to another segment area.
  • Non-Patent Document 1 The 19th Annual Conference of the Robotics Society of Japan (September 18-20, 2001) Proceedings, P1279-1280).
  • the segment area Sn of the reference template is displaced at the time of actual measurement.
  • This detection is performed by collating the dot pattern obtained from the DP light irradiated onto the CMOS image sensor 240 at the time of actual measurement with the dot pattern included in the segment region Sn.
  • the effective imaging area of the CMOS image sensor 240 at the time of actual measurement is, for example, the size of VGA (horizontal 640 pixels ⁇ vertical 480 pixels), as in the case of acquiring the reference image.
  • FIGS. 5A to 5C are diagrams for explaining such a distance detection method.
  • FIG. 5A is a diagram showing a reference pattern region set in a standard image on the CMOS image sensor 240
  • FIG. 5B is a captured image (measured image) on the CMOS image sensor 240 at the time of actual measurement.
  • FIG. 5C is a diagram for explaining a method of matching the dot pattern of DP light included in the captured image with the dot pattern included in the segment area of the reference template.
  • FIGS. 5 (a) and 5 (b) show only a part of the segment areas
  • FIG. 5 (c) shows the size of each segment area as horizontal 9 pixels ⁇ vertical 9 Shown in pixels.
  • the search range L0 When searching for the displacement position at the time of actual measurement of the segment area Si in FIG. 5A, as shown in FIG. 5B, the area Si0 at the same position as the segment area Si is centered on the captured image at the time of actual measurement.
  • the range of + ⁇ pixels and ⁇ pixels in the X-axis direction is set as the search range L0.
  • the segment area Si is sent pixel by pixel in the search range L0, and the dot pattern of the segment area Si is compared with the dot pattern on the captured image at each feed position.
  • an area corresponding to each feed position on the captured image is referred to as a “comparison area”.
  • comparison areas having the same size as the segment area Si are set every other pixel.
  • the search range L0 is set according to the range of distances to be acquired. The wider the range of distances to be acquired, the wider the search range L0.
  • the matching degree between the dot pattern of the segment area Si and the dot pattern of the comparison area at each feed position is set at each feed position. Desired. As described above with reference to FIG. 4A, the segment region Si is sent only in the X-axis direction within the search range L0 as described above with reference to FIG. This is because it is displaced in the direction.
  • the difference between the pixel value of each pixel in the segment area Si and the pixel value of the corresponding pixel in the comparison area is obtained. Then, a value Rsad obtained by adding the obtained difference to all the pixels in the comparison region is acquired as a value indicating the similarity.
  • the value Rsad is obtained for all comparison regions within the search range L0. Then, the comparison region having the minimum obtained value Rsad is detected as the region after the movement of the segment region Si. At this time, for example, when the difference between the minimum value of the value Rsad and the second smallest value of the value Rsad is smaller than a predetermined threshold, the comparison region having the minimum value Rsad is the region after the movement of the segment region Si. Instead, the search for the segment region Si is regarded as an error. In this case, the distance value for the segment region Si is not acquired.
  • the dots included in the segment area Si have moved to the position of the comparison area Cj on the captured image (measured image). Therefore, in this case, the value Rsad for the comparison area Cj is minimized, and the comparison area Cj is detected as an area after the movement of the segment area Si. Then, a pixel shift amount in the X-axis direction between the comparison region Cj and the region Si0 located at the same position as the segment region Si is acquired. This pixel shift amount corresponds to the pixel shift amount D shown in FIG. Thereafter, based on this pixel shift amount D, distance information with respect to the segment region Si is acquired based on the triangulation method as described above.
  • the object detection device 1 stores in the memory 25 an image in which the distance to the detection target object corresponding to each segment area acquired as described above is assigned to pixel values expressed in white to black gradation. .
  • an image in which a distance corresponding to each segment area is expressed by a pixel value is referred to as a “distance image”.
  • the segment area is set in increments of one pixel with respect to the reference pattern area of the standard image. Therefore, the distance image has a resolution of approximately the same VGA size as the standard image (approximately 640). X approximately 480).
  • the distance of each pixel position of the distance image is expressed by 256 gradations, and gradations closer to white (pixel value 256) are assigned as the distance is shorter, and gradations closer to black (pixel value 0) are assigned as the distance is longer. . If the segment area search results in an error, a gray scale of black (pixel value 0) is assigned.
  • the object detection device 1 acquires the distance image every predetermined time. In the present embodiment, the object detection apparatus 1 generates 30 distance images per second.
  • the distance information correction unit 21d when the object is located closer to the object detection device 1 than the distance range corresponding to the search range L0, the distance information correction unit 21d superimposes a predetermined value for this object on the distance image. Then, the distance image is corrected.
  • FIG. 6A is a flowchart showing a distance image generation process. 6A is executed mainly by the function of the distance acquisition unit 21b among the functions of the CPU 21 shown in FIG.
  • the CPU 21 acquires a captured image from the CMOS image sensor 240 (S102), and obtains distance information based on the acquired captured image as described above. Obtain (S103). Further, the CPU 21 sets a distance value corresponding to the acquired distance information at each pixel position and generates a distance image (S104). And CPU21 synthesize
  • FIG. 6B is a flowchart showing a luminance image generation process. 6B is executed mainly by the function of the distance information correction unit 21d among the functions of the CPU 21 shown in FIG.
  • the CPU 21 performs a smoothing process on the captured image acquired in S102 of FIG. 6A (S201).
  • smoothing filter processing moving average filter, Gaussian filter, etc.
  • image filter processing By such smoothing processing, the luminance value of each pixel on the captured image is smoothed flat.
  • the CPU 21 specifies a pixel region (hereinafter referred to as “high luminance region”) having a luminance value equal to or greater than a predetermined threshold on the captured image in which the luminance value is smoothed (S202). Then, the CPU 21 adjusts the gradation of the luminance value of each pixel in the high luminance area so as to correspond to the gradation of the distance image created in S104, and sets the adjusted luminance value for each pixel of the high luminance area. Then, a luminance image is generated (S203).
  • high luminance region a pixel region having a luminance value equal to or greater than a predetermined threshold on the captured image in which the luminance value is smoothed
  • the CPU 21 corrects the gradation of the luminance value of each pixel smoothed in S202 so that the luminance value can be treated as a distance value, and generates a luminance image using the luminance value of the corrected gradation.
  • a correction table for correcting the gradation of the luminance value is held in the memory 25 in advance.
  • the CPU 21 refers to this correction table and adjusts the gradation of the luminance value of each pixel in the high luminance area so as to correspond to the gradation of the distance image, and sets the adjusted luminance value to each pixel of the high luminance area. Then, a luminance image is generated.
  • the configuration (S203) for specifying a pixel region (high luminance region) having a luminance value equal to or greater than a threshold value from the captured image is an example of the configuration according to claim 3.
  • the processing (S202, S203) for generating a luminance image by setting a value based on the luminance value to each pixel in the high luminance region is an example of a configuration according to claim 2.
  • the object When the object is at a position closer to the object detection device 1 than the distance range corresponding to the search range L0 in FIG. 5B, the object is irradiated with light with a high light amount. For this reason, the brightness
  • the CPU 21 synthesizes the luminance image of the high luminance area generated in S203 with the distance image generated in S104. Specifically, the distance value of each pixel in the region corresponding to the high luminance region on the distance image is replaced with the luminance value of each pixel generated in S203. In this way, a distance image (hereinafter referred to as “synthesized image”) in which the luminance image is synthesized is generated.
  • the process of replacing the distance value of each pixel in the region corresponding to the high luminance region on the distance image with the value corresponding to the luminance value of the luminance image (S105) is configured as described in claims 2 and 5. It is an example.
  • the distance value increases as the object approaches the object detection device 1.
  • the luminance value on the captured image also increases as the object approaches the object detection device 1.
  • the gradation of the luminance value is corrected so that the luminance value can be handled as a distance value.
  • the distance value and the luminance value are correlated with each other, and the luminance value is adjusted so as to correspond to the distance value in the luminance image. Even if the brightness value is set, the set brightness value can be used as a distance value for detecting the presence of an object in the area.
  • a value based on the luminance value is set in a region corresponding to the high luminance region on the distance image.
  • the threshold (S202 in FIG. 6B) for specifying the high-luminance region has an object closer to the object detection device 1 than the distance measurement possible range corresponding to the search range L0 in FIG. 5B.
  • the area corresponding to the object is set to be specified as a high-luminance area on the captured image.
  • the distance value of each pixel in the region corresponding to the high-luminance region on the distance image is not a value (zero) indicating an error
  • the distance value of each pixel in the region is generated in S203. It is replaced with the luminance value of the pixel.
  • the distance value is acquired by the matching process, it is possible that the distance value is erroneously set to a pixel for which acquisition of the distance value is an error. Therefore, even when a distance value has already been set in the region corresponding to the high-intensity region on the distance image, an incorrect distance value is obtained by executing the process of replacing the distance value with the luminance value. The effect of being updated to a more appropriate value can be obtained. Instead of this process, the luminance value may not be set for the pixels for which the distance value has already been set.
  • CPU21 performs the above process until distance acquisition operation is complete
  • 7 (a) to 7 (d) are diagrams showing examples of generating a composite image by the processes of FIGS. 6 (a) and 6 (b).
  • 7A to 7D for the sake of convenience, only the right hand HR and the left hand HL are shown, and the human torso and face are not shown.
  • the left hand HL exists in the distance measurement possible range W2 corresponding to the search range L0 in FIG. 5B, and is closer to the object detection device 1 than the distance measurement possible range W2.
  • the captured image captured by the CMOS image sensor 240 includes an image of the right hand HR and an image of the left hand HL as shown in FIG. 7B.
  • the image of the right hand HR is larger than the image of the left hand HL in the captured image, as shown in FIG.
  • the brightness of HR is higher than that of left hand HL.
  • the distance image generated in S104 of FIG. A distance value is set for the other area, and an error usually occurs in other areas.
  • the error area includes not only the area corresponding to the unmeasurable range W3 in FIG. 7A but also the area of the right hand HR positioned in the unmeasurable range W1.
  • the area of the right hand HR positioned in the unmeasurable range W1 has a high luminance value of each pixel, and thus is identified as a high luminance area by the process of FIG. 6B, and a luminance image is generated. Therefore, the synthesized image obtained by synthesizing the luminance image with the distance image in S105 of FIG. 6A includes the area of the right hand HR as shown in FIG. 7D. As described above, the luminance value whose gradation is corrected is set for each pixel in the region. In this way, the right hand HR existing before the left hand HL is captured in the composite image.
  • an incorrect distance value may be set in the right hand HR region. Also in this case, since the distance value of the right-hand HR region on the distance image is replaced with the luminance value, the incorrect distance value is updated to a more appropriate value (luminance value).
  • the object detection unit 21c shown in FIG. 2 extracts a detection target object based on the composite image generated in this way.
  • FIG. 8A is a diagram showing detection processing of a detection target object.
  • the process of FIG. 8A is executed mainly by the function of the object detection unit 21c among the functions of the CPU 21 shown in FIG.
  • FIG. 8B is a diagram illustrating a detection example of the detection target object.
  • a state in which the person H faces the object detection device 1 with the right hand HR protruding forward (in the negative direction of the Z axis) is shown.
  • the right hand HR is located closer to the object detection device 1 than the distance measurement possible range W2
  • the left hand HL of the person H, the trunk, and the like are in the distance measurement possible range W2.
  • the CPU 21 displays the distance image before the luminance image is combined (the distance image generated in S104 of FIG. 6A).
  • a value obtained by subtracting a predetermined value ⁇ D from the distance value of the highest gradation at (the distance value representing the closest approach to the object detection apparatus 1) is set as the distance threshold value Dsh (S302).
  • a value obtained by subtracting the predetermined value ⁇ D from the gradation of the distance value of the left hand HL is set as the distance threshold value Dsh.
  • the CPU 21 classifies an area having a distance value (tone value) higher than the distance threshold value Dsh from the composite image as a target area (S303). Then, the CPU 21 executes the contour extraction engine, compares the contour of the divided target area with the object shape extraction template held in the memory 25, and compares the contour corresponding to the contour held in the object shape extraction template. The target region is extracted as a region corresponding to the detection target object (S304). In the example of FIG. 8B, by the process of S303, the areas corresponding to the right hand HR and the left hand HL are each classified as a target area from the composite image. If the detection target object held in the object shape extraction template is the right hand, the target area for the right hand HR is extracted as the area corresponding to the detection target object from the two divided target areas by the process of S304.
  • the contour extraction engine compares the contour of the divided target area with the object shape extraction template held in the memory 25, and compares the contour corresponding to the contour held in the object shape extraction template.
  • the target region is extracted
  • the CPU 21 determines whether or not the object detection operation is completed (S305). If the object detection operation has not ended (S305: NO), the CPU 21 returns to S301 and waits for the next synthesized image to be acquired. When the next composite image is acquired (S301: YES), the CPU 21 executes the processing from S302 onward, and extracts the detection target object from the composite image (S302 to S304).
  • the region of the detection target object extracted in S304 is further used by the object detection unit 21c for motion detection of the detection target object.
  • control according to the gesture is executed on the television 2 side.
  • ⁇ Effect of Example 1> when the detection target object is present at a position closer to the object detection device 1 than the distance measurement possible range W2, the luminance image of the region corresponding to the detection target object is combined with the distance image. A composite image is generated. That is, a value based on the luminance value is set in a region corresponding to the detection target object among regions where the distance value is not properly acquired in the distance image, and a composite image is generated. Then, based on the generated composite image, extraction processing of the detection target object is performed.
  • the detection target object is present at a position closer to the object detection device 1 than the distance measurement possible range W2
  • the detection target object is smoothly smoothed by the composite image obtained by combining the luminance images. Can be extracted. Therefore, it is possible to appropriately control the television 2 based on the movement of the detection target object.
  • a pixel region having a luminance value equal to or greater than a threshold on the smoothed captured image is specified as a high luminance region (S202).
  • a predetermined determination area is set around the target pixel on the captured image, and is included in the set determination area.
  • the target pixel is set as a pixel included in the high luminance area (overexposed area) (S211).
  • FIG. 9B is a diagram for explaining the processing of S211.
  • FIG. 9B schematically shows a light receiving state of the CMOS image sensor 240 when DP light is irradiated to an object located close to the object detection apparatus 1. That is, since the object is in a position close to the object detection device 1, the DP light is irradiated to the object in a state where the dots are overlapped. For this reason, the dots are also overlapped on the CMOS image sensor 240. Yes.
  • one pixel on the captured image is set as the target pixel, and an area of a predetermined area (here, a vertical 5 pixel area and a horizontal 5 pixel area) is set as the determination area around the target pixel. Is done. Then, the luminance values of all the pixels included in the determination region are added, and when the added value is equal to or greater than a predetermined threshold value, it is determined that the target pixel is included in the high luminance region (overexposed region). . This process is performed using all pixels on the captured image as the target pixel.
  • a predetermined area here, a vertical 5 pixel area and a horizontal 5 pixel area
  • a value obtained by correcting the luminance value of the pixel to a gradation corresponding to the distance value is set for the pixel determined to be included in the high luminance region (out-of-white region), as in the first embodiment. (S203).
  • a high-luminance area (out-of-white area) is identified, and a luminance image in which a luminance value is set in the high-luminance area is generated.
  • the process of combining the luminance image with the distance image is the same as the process of FIG.
  • the luminance value of each pixel is added to the determination area having a predetermined area around the target pixel, and the high luminance area is specified by comparing the added value with a predetermined threshold value.
  • the configuration is an example of a configuration according to claim 4.
  • the luminance value of each pixel is compared with the threshold value to determine whether the pixel is included in the high luminance area.
  • the luminance of the pixel exceeds the threshold value, and it is determined that the pixel is included in the high luminance region. Can happen.
  • the pixel area (high luminance area) corresponding to the object at a short distance is specified based on the luminance.
  • the pixel area is at a short distance based on the distance value of the distance image.
  • a pixel area corresponding to the object is specified.
  • FIG. 10A is a flowchart showing a distance image correction process according to this modification. This process is executed in S105 of FIG.
  • FIG. 10B is a diagram schematically illustrating an example of the distance image generated in S105 of FIG. In FIG. 10B, the number written in each pixel is the distance value of that pixel.
  • the CPU 21 sets a target pixel in the distance image (S401), and further, a unit area (here, a predetermined area) centering on the target pixel. A region of 9 pixels vertically and 9 pixels horizontally) is set (S402). Next, the CPU 21 groups pixels having the same distance value in the unit area, and counts the number of groups having the same distance value (S403). In the example of FIG. 10B, since the unit area includes pixels with distance values of 0, 1, 2, 3, and 5, the number of groups to be counted is 5. Then, the CPU 21 determines whether or not the counted number of groups is equal to or greater than a predetermined threshold value Csh (S404).
  • a predetermined threshold value Csh Csh
  • the threshold Csh (S404: YES) If the number of groups is equal to or greater than the threshold Csh (S404: YES), it is determined that the target pixel is included in the region corresponding to the object at a short distance, and the distance value of the target pixel is the same as in the first embodiment. The luminance value of the pixel is replaced with a value corrected to a gradation corresponding to the distance value (S405). On the other hand, if the number of groups is less than the threshold Csh (S404: NO), it is determined that the target pixel is not included in the region corresponding to the object at a short distance, and the distance value of the target pixel is maintained as it is.
  • the above processing is performed with all pixels on the distance image as the target pixel (S405).
  • S401 to S405 is performed on all the pixels on the distance image (S406: YES)
  • S406 YES
  • a composite image in which a luminance value is set in an area corresponding to an object existing at a short distance on the distance image is generated. Is done.
  • the CPU 21 advances the processing to S106 in FIG.
  • the configuration in which the distance value is replaced with the luminance value based on the distance value of each pixel on the distance image is an example of the configuration according to claim 6.
  • a uniform distance value cannot be obtained for the region corresponding to this object, indicating an error.
  • the number of types of distance values for the pixels included in the unit area is obtained, and it is determined whether or not this number is equal to or greater than the predetermined threshold Csh, thereby detecting the object more than the distance measuring range W2.
  • a pixel region corresponding to an object located close to the device 1 can be specified appropriately.
  • the threshold value Csh is set so that an area where an object exists can be specified.
  • Example 2 In the first embodiment, the distance value of each pixel is acquired by the matching process shown in FIGS. 5A to 5C, and the correction process based on the luminance value is performed on the acquired distance value. However, it is also possible to perform object detection based only on the luminance value without acquiring the distance value.
  • the present embodiment shows a configuration example in the case where object detection is performed only by luminance values.
  • a present Example shows an example of the structure as described in Claim 7.
  • FIG. 11 is a diagram illustrating a configuration of the object detection device 1 according to the present embodiment.
  • the function of the CPU 21 is changed from the configuration of the first embodiment. That is, the functions of the distance acquisition unit 21b, the object detection unit 21c, and the distance information correction unit 21d are deleted from the CPU 21, and the function of the image processing unit 21e is added to the CPU 21 instead.
  • the memory 25 since the distance value acquisition process is not performed, the memory 25 does not hold the reference template.
  • Other configurations shown in FIG. 11 are the same as those in FIG.
  • the image processing unit 21e specifies a region corresponding to the object based on the luminance of each pixel of the captured image, and further extracts a region corresponding to the detection target object from the specified region. Further, the image processing unit 21e detects the movement of the detection target object based on the extracted region of the detection target object, determines whether the detected movement matches a predetermined movement pattern, and sets the predetermined movement pattern. The corresponding signal is transmitted to the CPU 31 (function control unit 31a) on the television 2 side.
  • FIG. 12A is a flowchart showing the object extraction process in the present embodiment. The process of FIG. 12A is mainly performed by the function of the image processing unit 21e of the CPU 21.
  • the CPU 21 acquires a captured image from the CMOS image sensor 240 (S502), and further performs a smoothing process on the acquired captured image (S503).
  • This smoothing process is a smoothing filter process (moving average filter, Gaussian filter, etc.) used as an image filter process, as in the first embodiment. By such smoothing processing, the luminance value of each pixel on the captured image is smoothed flat.
  • the CPU 21 sets a pixel area having a luminance value equal to or higher than a predetermined threshold Bsh as a region that can be an object extraction target (hereinafter referred to as “target area”) on the captured image with the smoothed luminance value. Sort (S504). Then, the CPU 21 executes the contour extraction engine, compares the contour of the divided target area with the object shape extraction template held in the memory 25, and compares the contour corresponding to the contour held in the object shape extraction template. The target region is extracted as a region corresponding to the detection target object (S505).
  • the configuration in which the pixel region having the pixel value equal to or larger than the threshold value is classified as the target region is an example of the configuration according to claim 8.
  • the target area in this embodiment corresponds to a “high luminance area” described in claim 7.
  • FIG. 12B is a diagram illustrating a detection example of the detection target object.
  • a state in which the person H faces the object detection device 1 with the right hand HR protruding forward (in the negative direction of the Z axis) is shown.
  • a scale of luminance that can be acquired for an object by the CMOS image sensor 240 is added.
  • the luminances Br and Bl acquired for the right hand HR and the left hand HL are respectively higher than the threshold value Bsh
  • the luminance acquired for the torso of the person H is the threshold value Bsh. Is lower than.
  • the areas corresponding to the right hand HR and the left hand HL are segmented as target areas from the captured image by the process of S504. If the detection target object held in the object shape extraction template is the right hand, the target area for the right hand HR is extracted as the area corresponding to the detection target object from the two divided target areas by the process of S505.
  • the detection target object held in the object shape extraction template is the right hand
  • the target area for the right hand HR is extracted as the area corresponding to the detection target object from the two divided target areas by the process of S505.
  • the CPU 21 determines whether or not the object detection operation is finished (S506). If the object detection operation has not ended (S506: NO), the CPU 21 returns to S501 and waits for the next captured image to be acquired. When the next captured image is acquired (S501: YES), the CPU 21 executes the processing from S502 onward, and extracts a detection target object from the captured image (S502 to S505).
  • the region of the detection target object extracted in S505 is further used for motion detection of the detection target object in the function of the image processing unit 21e.
  • control according to the gesture is performed on the television 2 side.
  • Example 2> it is possible to detect a detection target object only by processing for luminance without performing distance measurement. Therefore, compared to the first embodiment, the processing load for object extraction can be reduced, and the detection target object can be detected quickly by a relatively simple process.
  • a plurality of exposure times (T1 to Tk) are set at the time of imaging according to the exposure time table, and captured images corresponding to the respective exposure times are acquired.
  • the CMOS image sensor 240 and the imaging signal processing circuit 23 in FIG. 11 are configured so that a plurality of exposure times can be set.
  • the configuration in which the exposure time can be adjusted in the CMOS image sensor 240 and the imaging signal processing circuit 23 is an example of the configuration of claim 10.
  • the imaging signal processing circuit 23 drives the CMOS image sensor 240 so that the CMOS image sensor 240 charges each pixel during the exposure time designated by the CPU 21. As the exposure time is longer, each pixel is irradiated with light longer, so the luminance of each pixel is higher, and as the exposure time is shorter, the luminance of each pixel is lower.
  • the processing of S503 to S505 in FIG. 12A is performed on each captured image acquired in this way. Thereby, a detection target object is extracted from any captured image, and the television 2 is controlled using the extracted detection target object.
  • FIG. 14A is a flowchart showing object extraction processing in the present embodiment. The process of FIG. 14A is mainly performed by the function of the image processing unit 21e of the CPU 21.
  • the CPU 21 acquires the captured images I1 to Ik captured at the exposure times T1 to Tk from the CMOS image sensor 240 and stores them in the memory 25 (S602).
  • the exposure time T1 is the shortest and the exposure time Tk is the longest. Therefore, the captured image I1 is a captured image captured with the shortest exposure time T1, and the captured image Ik is a captured image captured with the longest exposure time Tk.
  • the CPU 21 sets 1 to the variable i (S603), and executes the detection target object extraction process for the captured image Ii (S604).
  • the variable i is 1, the detection target object extraction process is performed on the captured image I1 having the shortest exposure time.
  • the same processing as S503 to S505 in FIG. 12A is performed on the captured image Ii. That is, the CPU 21 divides a target area whose luminance is equal to or higher than the threshold Bsh from the captured image Ii, and extracts an area corresponding to the detection target object from the categorized target area. After executing the detection target object extraction processing in this way, the CPU 21 determines whether or not an area corresponding to the detection target object has been extracted from the captured image Ii (S605).
  • the configuration in which the target region is divided by changing the luminance value of each pixel of the captured image relative to the threshold value Bsh by changing the exposure time is an example of the configuration according to claim 9. It is.
  • the CPU 21 determines whether the variable I has reached K (S606). When the variable I has not reached K (S606: NO), the CPU 21 adds 1 to the variable i (S607), and returns to S604 to execute the detection target object extraction process for the next captured image Ii. To do.
  • the variable i is 2, the extraction process of the detection target object is performed on the captured image I2 having the second shortest exposure time.
  • the CPU 21 repeats the processing of S604 to 607, and tries to extract a region corresponding to the detection target object in order from a captured image with a short exposure time.
  • the CPU 21 stops the processing for the captured image with an exposure time longer than the exposure time, It is determined whether or not the object detection operation is finished (S608). If the object detection operation has not ended (S608: NO), the CPU 21 returns to S601 and waits for the next image acquisition timing.
  • the configuration in which extraction of the detection target object is tried in order from the captured image having the short exposure time, and the detection target object is extracted from the captured image (the captured image of the exposure time suitable for object extraction) that can be extracted is an example of a configuration according to claims 10 and 11.
  • the configuration for stopping the processing on the captured image having an exposure time longer than the exposure time is the configuration according to claim 12. It is an example.
  • the configuration for extracting the region corresponding to the detection target object in order from the captured image with the short exposure time is an example of the configuration according to claim 13.
  • the CPU 21 sets the detection target object extraction at the image acquisition timing as an error and waits for the next image acquisition timing ( S608 ⁇ S601). If it is determined in S608 that the object detection operation has ended (S608: YES), the CPU 21 ends the process.
  • FIG. 15A is a diagram schematically showing an object detection operation according to the process flowchart of FIG. Here, it is assumed that the person H is in the same position as in FIG.
  • the captured image I1 from which the object extraction is performed first has lower luminance than the other captured images I2 to Ik. Therefore, in the captured image I1, for example, the person H is in a state equivalent to being in the position of the upper broken line in FIG. In this case, the luminance acquired for the right hand HR and the left hand HL does not exceed the threshold Bsh. For this reason, the right hand HR that is the detection target object is not extracted from the captured image I1.
  • the luminance acquired for the person H gradually becomes higher, so that the person H gradually becomes equivalent to the object detection apparatus 1 being approached.
  • the luminance of the right hand HR becomes equal to or higher than the threshold Bsh, and the right hand that is the detection target object from the captured image Im. HR is extracted.
  • FIG. 15B schematically shows the luminance value acquired for each part of the person H in association with the luminance value scale in the captured image I1 where the object extraction is first performed. Is. Further, the lower broken line in FIG. 15B schematically shows the luminance value acquired for each part of the person H in association with the luminance value scale in the captured image In acquired by the exposure time Tn. It is a thing.
  • the processing for the captured image with the exposure time longer than the exposure time is stopped, so the CPU 21 Can reduce the processing load.
  • the luminance value of each pixel of the captured image is changed relative to the threshold value Bsh by changing the exposure time.
  • the luminance value of each pixel of the captured image may be changed relative to the threshold value Bsh by changing the threshold value Bsh while keeping the exposure time constant.
  • the threshold value Bsh is changed stepwise from the high luminance side to the low luminance side. Then, an area corresponding to the right hand HR is acquired as a target area at the timing when the threshold value Bsh is changed to a state slightly lower than the luminance Br of the right hand HR. Thereby, the area
  • the detection target object may not be detected properly when the detection target object exists at a position close to the object detection device 1 as described below. That is, when there is only one type of exposure time, the exposure time is usually suitable when the detection target object is present at a position farther from the object detection device 1 than the position of the right hand HR shown in FIG. Is set. For this reason, when the right hand HR is present at a position close to the object detection apparatus 1 as shown in FIG. 15B, the luminance with respect to the right hand HR is excessively increased at the exposure time set in this way, so-called halation. As a result, the outline of the right hand HR becomes unclear in the captured image.
  • the threshold value Bsh when the threshold value Bsh is changed to a state slightly lower than the luminance Br of the right hand HR, the region corresponding to the right hand HR is divided from the captured image as the target region, but the contour of the divided target region is normal The outline of the right hand HR is greatly collapsed. For this reason, it is difficult to properly detect the right hand HR.
  • the target area is divided by the exposure time suitable for the proximity of the object, so that the outline of the divided target area is caused by halation or the like. Large collapse is suppressed.
  • the contour of the segmented target area is not greatly collapsed due to halation or the like.
  • the exposure time is changed as in the above modification example, rather than changing the threshold value Bsh while fixing the exposure time. It is preferable to change the luminance value of each pixel of the captured image relative to the threshold value Bsh by changing the direction.
  • detection target object extraction processing is performed in order from a captured image with a short exposure time.
  • the detection target object extraction process may be performed in another order.
  • the detection target object extraction process may be performed in order from a captured image having a long exposure time.
  • the shorter the exposure time the easier it is to extract the detection target object approaching the object detection device 1 from the captured image. Therefore, the detection target object is usually the right-hand HR as described above.
  • the detection target object is extracted more quickly by performing the detection target object extraction process in order from the captured image with the short exposure time. can do.
  • the number of types of exposure time is k.
  • the number of types of exposure time, the interval between exposure times, and the width from the minimum value to the maximum value of the exposure time are shown in FIG.
  • the processing load and the distance range in which the detection target object can be detected can be set as appropriate.
  • the detection target object extraction process is performed on the captured image in order from the shortest exposure time.
  • the object extraction process is performed in parallel on a plurality or all of the captured images I1 to Ik. It may be done.
  • the captured images I1 to Ik are acquired for all the exposure times T1 to Tk, and then the object extraction process is performed in order from the captured images with the shortest exposure times.
  • the object extraction process is performed from the acquisition of the captured image I1 for the exposure time T1
  • the detection target object cannot be extracted by this, the object of the object is acquired from the acquisition of the captured image I2 for the exposure time T2.
  • processing from acquisition of a captured image to extraction of an object may be performed by a series of processing for each exposure time.
  • the k picked-up images I1 to Ik are acquired by changing the exposure time, and the object extraction process is performed.
  • the exposure time is changed to acquire the captured images I1 to Ik and the object extraction process is performed, and if the detection target object is extracted in this process, the detection target
  • the exposure time corresponding to the captured image in which the object is detected may be used in the subsequent processing. For example, if the exposure time of the captured image from which the detection target object is extracted is T3, the captured image may be acquired only by the exposure time T3 and the object extraction process may be performed at the subsequent image acquisition timing.
  • FIG. 16 is a flowchart showing an example of processing in this case.
  • the CPU 21 determines whether the exposure time T0 has already been set (S702). When the exposure time T0 is not set (S702: NO), the CPU 21 extracts the detection target object by changing the exposure time as in the above modification example (S707). In S707, for example, the same processing as S602 to S607 in FIG. 14 is performed. When the region corresponding to the detection target object is extracted by the process of S707 (S708: YES), the CPU 21 sets the exposure time corresponding to the captured image from which the detection target object is extracted as the exposure time T0 (S709). . Thereafter, the CPU 21 determines whether or not the object detection operation has ended (S706). If it has not ended (S706: NO), the CPU 21 returns to S701 and waits for the next imaging timing.
  • the CPU 21 acquires a captured image at the exposure time T0 (S703), and for the acquired captured image, extraction processing of a detection target object is performed. (S704). Thereby, when the area of the detection target object is extracted from the captured image (S705: YES), the CPU 21 determines whether the object detection operation is completed (S706), and if not completed (S706: NO). , The process returns to S701 to wait for the next imaging timing. On the other hand, if the region of the detection target object is not extracted from the captured image (S705: NO), the CPU 21 proceeds to S707 and changes the exposure time to extract the detection target object.
  • the CPU 21 updates the exposure time T0 to the exposure time corresponding to the captured image from which the detection target object is extracted ( S709). Thereby, subsequent captured image acquisition processing is performed with the updated exposure time T0.
  • the processing load on the CPU 21 can be reduced and the processing can be speeded up. You can plan. Also, when the region corresponding to the detection target object cannot be extracted by the exposure time T0, such as when the detection target object moves back and forth, the object extraction process is performed again while changing the exposure time, so that A region corresponding to the detection target object can be extracted. Furthermore, when the region corresponding to the detection target object is extracted from the captured image of the predetermined exposure time by this process, the exposure time is reset to the exposure time T0, and therefore, the updated exposure time T0 Subsequent processing can proceed smoothly.
  • the configuration in which the exposure time T0 at which the detection target object is detected is used as an exposure time suitable for object extraction in subsequent captured image acquisition, as in S702 to S704 of FIG. It is an example of a structure.
  • the target area is segmented from the imaging area by the processing in S503 and S504 of FIG.
  • the target area may be segmented from the captured image by the process according to the first modification of the first embodiment described with reference to FIG.
  • the dot pattern laser beam is projected from the projection optical system 100 onto the target area.
  • the laser light of the dot pattern does not necessarily have to be projected onto the target area.
  • the light is not uniformly converted into the dot pattern but spreads uniformly in the target area.
  • the light may be projected onto the target area.
  • the light since the distance detection by the dot pattern is not performed, the light may be projected onto the target area so that the object can be detected by the luminance.
  • the luminance value when a luminance image is combined with a distance image, the luminance value is corrected to a gradation corresponding to the distance value, and then the luminance value of each pixel in the high luminance area is converted to the distance. Applied to images. Instead, the luminance value of each pixel in the high luminance area may be applied to the distance image as it is.
  • the distance image is generated so that the gradation becomes lower as the distance to the object becomes longer.
  • the distance image is generated so that the gradation becomes higher as the distance to the object becomes longer. Also good.
  • the luminance value is converted so that the gradation becomes lower as the luminance is higher, and is synthesized with the distance image.
  • the distance information is obtained using the triangulation method.
  • the distance information acquisition method is not limited to this.
  • the deviation amount D may be acquired as it is as distance information.
  • the segment area set in the reference image is searched on the actual measurement image.
  • the segment area corresponding to the dot pattern of the area set on the actual measurement image is searched on the reference image. You may make it search with.
  • the search range L0 does not necessarily have to be set symmetrically in the positive and negative X-axis directions around the region Si0, and is asymmetric in the positive and negative X-axis directions around the region Si0.
  • the search range L0 may be set.
  • the CMOS image sensor 240 is used as the light receiving element, but a CCD image sensor can be used instead.
  • the laser light source 110 was used as a light source, it replaces with this and other light sources, such as LED, can also be used.
  • the configurations of the projection optical system 100 and the light receiving optical system 200 can be changed as appropriate.

Abstract

L'invention porte sur un dispositif de détection d'objet pouvant détecter sans à-coups un objet de détection dans une zone cible. Un dispositif de détection d'objet (1) comprend un système de projection optique (100) qui projette de la lumière laser à motif de points sur une zone cible, un système de réception de lumière optique (200) qui prend l'image de la région ciblée à l'aide d'un capteur d'image CMOS (240), une unité d'acquisition de distance (21b) qui acquiert des informations de distance tridimensionnelles, une unité de traitement d'image (21e) qui corrige les informations de distance tridimensionnelles sur la base de valeurs de luminance des pixels dans le capteur d'image CMOS (240), et une unité de détection d'objet (21c) qui détecte l'objet à détecter sur la base des informations de distance tridimensionnelles corrigées. L'unité de traitement d'image (21e) corrige les informations de distance tridimensionnelles, dont il a été déterminé qu'elles étaient imprécises étant donné que l'objet se trouve à un endroit qui se rapproche du dispositif de détection d'objet (1) à une valeur basée sur la valeur de luminance.
PCT/JP2013/007621 2013-01-10 2013-12-26 Dispositif de détection d'objet WO2014108976A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2013002979A JP2016040520A (ja) 2013-01-10 2013-01-10 物体検出装置
JP2013-002979 2013-01-10

Publications (1)

Publication Number Publication Date
WO2014108976A1 true WO2014108976A1 (fr) 2014-07-17

Family

ID=51166654

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2013/007621 WO2014108976A1 (fr) 2013-01-10 2013-12-26 Dispositif de détection d'objet

Country Status (2)

Country Link
JP (1) JP2016040520A (fr)
WO (1) WO2014108976A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114503543A (zh) * 2019-09-26 2022-05-13 株式会社小糸制作所 门控照相机、汽车、车辆用灯具、图像处理装置、图像处理方法

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017183114A1 (fr) * 2016-04-19 2017-10-26 株式会社日立エルジーデータストレージ Dispositif et procédé de génération d'images de distance
TWI636395B (zh) 2016-11-10 2018-09-21 財團法人金屬工業研究發展中心 基於深度值之手勢操作方法及其系統
KR102087081B1 (ko) * 2017-09-13 2020-03-10 네이버랩스 주식회사 영상센서 방식 라이다의 측정거리 향상을 위한 광 집속 시스템
US11440119B2 (en) 2018-10-12 2022-09-13 Teradyne, Inc. System and method for weld path generation
JP2020106711A (ja) 2018-12-28 2020-07-09 株式会社ジャパンディスプレイ 表示装置、及び、表示装置を組み込んだ電子機器
CN113597534B (zh) * 2019-03-26 2023-07-25 松下知识产权经营株式会社 测距成像系统、测距成像方法和程序

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH08315154A (ja) * 1995-02-21 1996-11-29 Mitsubishi Electric Res Lab Inc ジェスチャ認識システム
JPH11265241A (ja) * 1998-03-17 1999-09-28 Toshiba Corp 情報入力装置および情報入力方法および記録媒体
JP2000221007A (ja) * 1999-02-01 2000-08-11 Toshiba Corp 画像取得装置および方法
JP2000222585A (ja) * 1999-02-01 2000-08-11 Toshiba Corp 動き検出方法および動き検出装置および動き認識方法および動き認識装置および記録媒体
JP2004317126A (ja) * 2003-04-10 2004-11-11 Renesas Technology Corp はんだ印刷装置
JP2006061222A (ja) * 2004-08-24 2006-03-09 Sumitomo Osaka Cement Co Ltd 動き検出装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH08315154A (ja) * 1995-02-21 1996-11-29 Mitsubishi Electric Res Lab Inc ジェスチャ認識システム
JPH11265241A (ja) * 1998-03-17 1999-09-28 Toshiba Corp 情報入力装置および情報入力方法および記録媒体
JP2000221007A (ja) * 1999-02-01 2000-08-11 Toshiba Corp 画像取得装置および方法
JP2000222585A (ja) * 1999-02-01 2000-08-11 Toshiba Corp 動き検出方法および動き検出装置および動き認識方法および動き認識装置および記録媒体
JP2004317126A (ja) * 2003-04-10 2004-11-11 Renesas Technology Corp はんだ印刷装置
JP2006061222A (ja) * 2004-08-24 2006-03-09 Sumitomo Osaka Cement Co Ltd 動き検出装置

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114503543A (zh) * 2019-09-26 2022-05-13 株式会社小糸制作所 门控照相机、汽车、车辆用灯具、图像处理装置、图像处理方法

Also Published As

Publication number Publication date
JP2016040520A (ja) 2016-03-24

Similar Documents

Publication Publication Date Title
WO2014108976A1 (fr) Dispositif de détection d'objet
US10309770B2 (en) Three-dimensional sensor system and three-dimensional data acquisition method
JP5911934B2 (ja) 輪郭線計測装置およびロボットシステム
EP2568253B1 (fr) Procédé et système de mesure de lumière structurée
CN109751973B (zh) 三维测量装置、三维测量方法以及存储介质
TWI624170B (zh) 影像掃描系統及其方法
JP2016029564A (ja) 対象検出方法及び対象検出装置
JP6187640B2 (ja) 画像合成方法及び装置
JP5214062B1 (ja) 情報取得装置および物体検出装置
JP2009529824A (ja) ３次元映像獲得用ｃｍｏｓステレオカメラ
JP2018105682A (ja) ステレオカメラ
JP2010091426A (ja) 距離計測装置及びプログラム
US20160275359A1 (en) Information processing apparatus, information processing method, and computer readable medium storing a program
JP2019215811A (ja) 投影システム、画像処理装置および投影方法
JP2016024052A (ja) ３次元計測システム、３次元計測方法及びプログラム
JP2017142613A (ja) 情報処理装置、情報処理システム、情報処理方法及び情報処理プログラム
JP6009206B2 (ja) ３次元計測装置
JP2006058091A (ja) ３次元画像測定装置および方法
JP2014137762A (ja) 物体検出装置
CN106888344A (zh) 摄像模组及其像面倾斜的获取方法和调整方法
JP5138115B2 (ja) 情報取得装置及びその情報取得装置を有する物体検出装置
JP2015179077A (ja) 視差演算システム、情報処理装置、情報処理方法及びプログラム
JP2013126135A (ja) ステレオ画像生成装置、ステレオ画像生成方法及びステレオ画像生成用コンピュータプログラム
JPWO2012026180A1 (ja) 情報取得装置および物体検出装置
JP2014132219A (ja) 情報取得装置および物体検出装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13870523

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 13870523

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP