JP2019087022A - Solid object detection device, its control method, and program - Google Patents

Abstract

To provide a solid object detection device capable of accurately determining whether or not a solid object is placed on a flat surface.SOLUTION: A solid object detection device 100 generates a distance image containing distance measurement data from the solid object detection device 100 to a subject, and generates first reference distance measurement data on the basis of the distance measurement data, and detects a solid object on a flat surface 101 on the basis of a result obtained by comparing the distance measurement data with the first reference distance measurement data.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a three-dimensional object detection device, a control method therefor, and a program.

  There is known a three-dimensional object detection device provided with a distance image sensor that measures the distance to a subject. The three-dimensional object detection apparatus performs a determination process of capturing an image of a plane such as a desk using a distance image sensor and determining whether a three-dimensional object is placed on the plane based on an image obtained by imaging ( For example, refer to Patent Document 1). In the three-dimensional object detection device, a reference distance is set based on a result of ranging to the plane obtained from an image captured in advance when the three-dimensional object detection device is activated or the like. The three-dimensional object detection apparatus detects, for example, an object resulting in a distance measurement result closer than the reference distance in an image obtained by imaging as a three-dimensional object, and determines that a three-dimensional object is placed on a plane based on the detection result Do.

JP, 2016-128970, A

  However, in the conventional three-dimensional object detection apparatus, it may not be possible to accurately determine whether or not a three-dimensional object is placed on a plane. In the three-dimensional object detection device, the reference distance is set based on the distance measurement result at the time of activation of the three-dimensional object detection device. However, in view of the performance of the distance image sensor, the distance measurement result output from the distance image sensor is the distance measurement timing It may change depending on the situation. For this reason, the conventional three-dimensional object detection device can not set the reference distance suitable for the timing at which the determination process is performed, and can not accurately determine whether or not the three-dimensional object is placed on the plane. .

  An object of the present invention is to provide a three-dimensional object detection device capable of accurately determining whether a three-dimensional object is placed on a plane, a control method thereof, and a program.

  In order to achieve the above object, a three-dimensional object detection device according to the present invention is a three-dimensional object detection device which picks up an object including a plane and detects a three-dimensional object on the plane, the object from the three-dimensional object detection device Distance image acquisition means for acquiring a distance image including distance measurement data up to, reference distance measurement data generation means for generating reference distance measurement data of the distance image based on the distance measurement data, the distance measurement data, and And detection means for detecting a three-dimensional object on the plane based on a result of comparison of reference distance measurement data.

  According to the present invention, it can be accurately determined whether or not a three-dimensional object is placed on a plane.

It is an outline view of a solid thing detection device concerning an embodiment of the invention. It is a block diagram which shows roughly the hardware constitutions of the three-dimensional object detection apparatus of FIG. It is a block diagram which shows roughly the function structure of the three-dimensional object detection apparatus of FIG. It is a flowchart which shows the procedure of the user operation detection process performed by the solid-object detection apparatus of FIG. It is for demonstrating the distance image produced | generated by the solid-object detection apparatus of FIG. It is a figure for demonstrating determination of FIG.4 S105. It is a figure for demonstrating determination of FIG.4 S106. It is a figure for demonstrating the process of FIG.4 S108. It is a block diagram which shows the modification of the function structure of the three-dimensional object detection apparatus of FIG. It is a flowchart which shows the procedure of the modification of the user operation detection process of FIG. It is a figure for demonstrating the process of FIG.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is an external view of a three-dimensional object detection device 100 according to an embodiment of the present invention. In the present embodiment, a world coordinate system that defines position information in a three-dimensional space is used, and the x axis, the y axis, and the z axis in FIG. 1 indicate the coordinate axes of the world coordinate system.

  In FIG. 1, the three-dimensional object detection device 100 includes a distance image sensor 105 and a projector 106.

  The three-dimensional object detection apparatus 100 projects an image, for example, a UI image 102 from the projector 106 onto the plane 101. The user performs a user operation such as a tap operation on the UI image 102 projected on the plane 101 by his / her hand 103. In the tap operation, the user performs an operation (hereinafter, referred to as "touch") to bring a fingertip into contact with or approach an item in the UI image 102, for example, the item 104, and then an operation to release the fingertip from the touched item 104 It is. The user can select each item in the UI image 102 by performing a tap operation. In the present embodiment, a touch sensor or the like for detecting the position touched by the user is not provided on the plane 101, and the three-dimensional object detection device 100 is a three-dimensional object on the plane 101 based on the distance image acquired from the distance image sensor 105. The hand 103 is detected, and user operation is recognized. The distance image is composed of a plurality of pixels, and distance measurement data from the lens center of the distance image sensor 105 to the subject is set to each pixel. When the user taps on the item 104, the three-dimensional object detection device 100 changes the color of the item 104. In addition, when the user performs a tap operation on an item (not shown) for instructing image switching, the three-dimensional object detection device 100 switches the UI image 102 projected on the plane 101 to another image.

  The distance image sensor 105 is installed at a position at which imaging on the plane 101 can be performed, for example, above the plane 101. In the present embodiment, the distance image sensor 105 may be installed at a position where an image of the plane 101 viewed from above can be obtained. For example, a mirror (not shown) may be installed above the plane 101, and the distance image sensor 105 may be installed at a position at which the plane 101 can be imaged via the mirror. The distance image sensor 105 is a TOF sensor that measures the distance to the subject based on the time difference (phase difference) until the laser light output from the distance image sensor 105 is reflected by the subject and returns. The plane 101 is imaged at predetermined intervals set in advance. The distance image sensor 105 measures the distance from the distance image sensor 105 to the subject based on the captured image, generates a distance image including distance measurement data, and outputs the distance image to the three-dimensional object detection device 100. The distance image sensor 105 uses an ambient light or a method using infrared light with a small influence on the projection on the plane 101, a parallax method, or the like when performing imaging for generating a distance image.

  The projector 106 is installed at a position where the UI image 102 and the like can be projected on the plane 101, for example, above the plane 101. In the present embodiment, projector 106 may be installed at a position other than above plane 101. The projector 106 installed at a position other than above the plane 101 reflects projection light using a mirror (not shown) or the like to project the UI image 102 or the like on the plane 101.

  FIG. 2 is a block diagram schematically showing the hardware configuration of the three-dimensional object detection device 100 of FIG.

  In FIG. 2, the three-dimensional object detection device 100 includes a control unit 200 in addition to the distance image sensor 105 and the projector 106 in FIG. 1. The control unit 200 is connected to the distance image sensor 105 and the projector 106. The control unit 200 also includes a CPU 201, a ROM 202, a RAM 203, and a storage device 204. The CPU 201, the ROM 202, the RAM 203, and the storage device 204 are connected to one another via a system bus 205.

  The control unit 200 controls the entire three-dimensional object detection device 100. The CPU 201 executes an OS or a program stored in the ROM 202 or the storage device 204 to perform calculation of each processing, logical determination, and the like. The ROM 202 stores programs executed by the CPU 201 and respective data. The RAM 203 is used as a work area of the CPU 201 and as a temporary storage area of each data. The storage device 204 is an external storage device such as a hard disk drive or a USB memory, and stores programs and data.

  FIG. 3 is a block diagram schematically showing a functional configuration of the three-dimensional object detection device 100 of FIG.

  In FIG. 3, the three-dimensional object detection device 100 includes a distance image acquisition unit 301, a division unit 302, an approximate plane calculation unit 303, an interpolation unit 304, a reference image acquisition unit 305, a fingertip detection unit 306, a coordinate conversion unit 307, and a recognition unit 308. , And each functional unit of the display control unit 309. The processing of each functional unit described above is performed by the CPU 201 executing a program stored in the ROM 202 or the storage device 204.

  The distance image acquisition unit 301 acquires a distance image from the distance image sensor 105 at predetermined intervals set in advance, and stores the acquired distance image in the RAM 203. The dividing unit 302 divides the distance image into a plurality of divided areas of a predetermined size. The approximate plane calculation unit 303 generates first reference distance measurement data for each divided area based on distance measurement data included in the distance image. The first reference distance measurement data is data obtained by approximating the distance measurement data of each divided area, and is used as a reference distance in the detection process of a three-dimensional object. That is, the first reference distance measurement data is reference distance measurement data having the same distance measurement timing as the above-described distance measurement data. Further, the approximate plane calculation unit 303 calculates the difference between the distance measurement data and the first reference distance measurement data in each divided area. Furthermore, the approximate plane calculation unit 303 determines whether or not each divided area is a solid object outline area corresponding to the outline of the solid object based on the difference. The approximate plane calculation unit 303 specifies, as a three-dimensional object region, a region surrounded by a plurality of divided regions determined to be three-dimensional object contour regions.

  The reference image acquisition unit 305 acquires a reference image. The reference image is a distance image in a state where no object or the like is placed on the plane 101, and includes second reference distance measurement data which is a distance measurement value from the three-dimensional object detection device 100 to the plane 101. The reference image is generated, for example, based on an image obtained by imaging at the time of activation of the three-dimensional object detection apparatus 100 in which the object is not placed in the plane 101 and stored in advance in the storage device 204 at the time of factory shipment. That is, the second reference distance measurement data is reference distance measurement data of distance measurement timing different from the above-described distance measurement data.

  The interpolation unit 304 performs interpolation processing described later on the three-dimensional object region specified by the approximate plane calculation unit 303. For example, the interpolation unit 304 interpolates a region other than the three-dimensional object contour region among the specified three-dimensional object regions. The fingertip detection unit 306 identifies the position of the user's fingertip from the three-dimensional object region subjected to the interpolation processing (hereinafter, referred to as “interpolated three-dimensional object region”). In the present embodiment, for example, the interpolation unit 304 specifies, as the fingertip position, the coordinates of the pixel farthest from the outer periphery of the distance image in the three-dimensional object region after interpolation.

  The coordinate conversion unit 307 converts the fingertip position into the world coordinate system. In the present embodiment, the coordinates of the fingertip position in the distance image are converted into world coordinates by using the lens characteristics of the distance image sensor 105 and the translation / rotational movement parameters of the world coordinate system. The recognition unit 308 recognizes the user operation based on the locus of coordinates of the converted fingertip position. The display control unit 309 generates an image to be projected, for example, a UI image 102 using information stored in the ROM 202 or the storage device 204, and projects the UI image 102 on the plane 101.

  FIG. 4 is a flowchart showing a procedure of user operation detection processing executed by the three-dimensional object detection device of FIG.

  The process in FIG. 4 is performed by the CPU 201 executing a program stored in the ROM 202 or the storage device 204, and is executed when the CPU 201 acquires a distance image from the distance image sensor 105. The process of FIG. 4 is executed each time a distance image is acquired from the distance image sensor 105, and the cycle of acquiring the distance image from the distance image sensor 105 matches the frame rate of the captured image of the distance image sensor 105. In the process of FIG. 4, a process of detecting a tap operation will be described as an example of the user operation.

  In FIG. 4, first, the CPU 201 acquires the distance image 500 of FIG. 5A from the distance image sensor 105 (step S101) (distance image acquisition means). The distance image 500 includes a flat area 501 showing the plane 101 and a hand area 502 showing the user's hand. Next, as shown in FIG. 5B, the CPU 201 divides the acquired distance image 500 into a plurality of divided areas of a predetermined size. The size of the divided area is set by the user, or dynamically adjusted by the CPU 201 in accordance with the inclination of the distance image sensor 105 or the like. Next, the CPU 201 extracts one divided area from the plurality of divided areas constituting the distance image 500 (step S102) (extraction means). Next, the CPU 201 generates first reference distance measurement data by approximating distance measurement data of one divided area using the least squares method (step S103) (reference distance measurement data generation means). In step S103, for example, when the CPU 201 extracts the divided area 503 of FIG. 5B that is a part of the plane area, the CPU 201 approximates the distance measurement data 601 of FIG. First reference ranging data 602 of FIG. 6 (b) having characteristics is generated. Next, the CPU 201 compares the distance measurement data and the first reference distance measurement data in one divided area, and determines whether the comparison result is equal to or less than a predetermined threshold (step S104). Specifically, the CPU 201 calculates the difference between the distance measurement data and the first reference distance measurement data in one divided area in pixel units, and the total value of the calculation results of each pixel is less than or equal to the predetermined threshold value. Determine if

  If it is determined in step S104 that the compared result is equal to or less than the predetermined threshold value, the CPU 201 determines that one divided area is not a three-dimensional object outline area (step S105). For example, in the divided area 503, the yz characteristic of a certain x coordinate in the distance measurement data 601 is shown in FIG. 6C, and the yz characteristic of the same x coordinate in the first reference distance measurement data 602 as in FIG. It shows in FIG.6 (d). The divided area 503 is a part of the plane area 501, and the variation of the characteristic of the distance measurement data 601 is relatively small. For example, the yz characteristic of the same x coordinate of the distance measurement data 601 and the first reference distance measurement data 602 As shown in FIG. 6 (e), the difference is small, and the total value of the calculation results of each pixel is equal to or less than a predetermined threshold value. In the present embodiment, it is determined that the divided area in which the amount of change in the characteristic of the distance measurement data is relatively small is not the three-dimensional object outline area. Thereafter, the CPU 201 performs the process of step S107.

  As a result of the determination in step S104, when the compared result is larger than the predetermined threshold, the CPU 201 determines that one divided area is a three-dimensional object outline area (step S106). For example, in the divided area 504 of FIG. 5B including the boundary of the plane area 501 and the hand area 502, the yz characteristic of a certain x coordinate in the distance measurement data 701 of FIG. 7A is shown in FIG. FIG. 7D shows the yz characteristics of the same x coordinate as in FIG. 7C in the first reference distance measurement data 702 of FIG. 7B. In the divided area 504, the amount of change in characteristics corresponding to a part of the distance measurement data 701, specifically, the boundary between the plane area 501 and the hand area 502, is large. Therefore, for example, when the yz characteristics at the same x coordinate of the distance measurement data 701 and the first reference distance measurement data 702 are compared, as shown in FIG. 7E, the distance measurement data 701 and the first reference measurement The difference of the distance data 702 is large, and the total value of the calculation results of each pixel is larger than a predetermined threshold value. In the present embodiment, it is determined that the divided area in which the amount of change in part of the characteristics of the distance measurement data is large is the solid object outline area. Next, the CPU 201 determines whether the determination of all divided areas in the distance image 500 has been completed (step S107).

  As a result of the determination in step S107, when the determination of any divided area in the distance image 500 is not completed, the CPU 201 returns to the process of step S103. On the other hand, when the determination of all divided areas in the distance image 500 is completed as a result of the determination in step S107, the CPU 201 performs interpolation processing of an area surrounded by a plurality of divided areas determined as solid object outline areas in step S106. The operation is performed (step S108). In the present embodiment, the CPU 201 can specify a three-dimensional object contour region, for example, the divided region 801 in FIG. 8A by comparing the distance measurement data and the first reference distance measurement data. Even if it is a divided area corresponding to an object, a divided area having a relatively small amount of change in the characteristics of the distance measurement data 601, for example, the divided area 802 in FIG. 8A can not be identified. On the other hand, in the present embodiment, as shown in FIG. 8B, the CPU 201 interpolates so that the divided area 802 is also included in the divided area. The CPU 201 specifies a three-dimensional object area including a three-dimensional object outline area and an interpolated area. Thereby, the three-dimensional object detection device 100 detects the user's hand 103 which is a three-dimensional object on the plane 101. Next, the CPU 201 specifies the fingertip position in the three-dimensional object area 801 (step S109), and converts the coordinates indicating the fingertip position into the world coordinate system (step S110). Next, the CPU 201 determines whether a tap operation has been performed based on the locus of the fingertip position (step S111). In step S111, the time taken for the distance between the fingertip and the plane 101 to approach and leave the predetermined distance is equal to or less than the predetermined time, and the amount of movement of the fingertip in the horizontal direction of the plane 101 is equal to or less than the predetermined amount. Only, the CPU 201 determines that the tap operation has been performed. On the other hand, in the other cases, the CPU 201 determines that the tap operation is not performed. Note that the CPU 201 may perform the process of step S111 based on not only one image captured by the distance image sensor 105, but also the locus of the fingertip position identified from a plurality of images captured continuously.

  As a result of the determination in step S111, when a tap operation is performed, the CPU 201 determines whether the tap operation is a tap operation on the item 104 in the UI image 102 (step S112).

  When it is determined in step S112 that the tap operation is a tap operation on the item 104 in the UI image 102, the CPU 201 projects the UI image 102 whose content has been changed on the plane 101 (step S113). For example, the CPU 201 projects the UI image 102 in which the color of the item 104 is changed on the plane 101. Further, the CPU 201 projects the UI image 102 in which the item 104 is blinked on the plane 101. Thereafter, the CPU 201 ends the present process.

  As a result of the determination in step S111, when the tap operation is not performed or as a result of the determination in step S112, when the tap operation is not the tap operation for the item 104, the CPU 201 ends the present process.

  According to the process of FIG. 4 described above, the first reference distance measurement data is generated based on the distance measurement data, and the distance measurement data and the first reference distance measurement data included in the distance image 500 are compared. Thus, a three-dimensional object on the plane 101 is detected. That is, the first reference distance measurement data having the same distance measurement timing as the distance measurement data is set as the reference distance in the detection process of the three-dimensional object. Thereby, even if the distance measurement data outputted from the distance image sensor 105 fluctuates according to the distance measurement timing, the reference distance suitable for the distance measurement data can be set, and the three-dimensional object is placed on the plane 101 It is possible to accurately determine whether the

  Further, in the process of FIG. 4 described above, the three-dimensional object on the plane 101 is detected based on the result of comparing the distance measurement data of each divided region and the first reference distance measurement data of each divided region. As a result, it is possible to divide the distance measurement data, which is the processing target of the three-dimensional object detection process, and reduce the processing load in the three-dimensional object detection process.

  Furthermore, in the process of FIG. 4 described above, a divided area in which the total value of differences between pixels is larger than a predetermined threshold is detected as a solid object on the plane 101, and a divided area whose total value is equal to or smaller than the predetermined threshold is a plane. It is not detected as a three-dimensional object on 101. As a result, it is possible to detect an outline portion of a solid object having a large amount of change in characteristics of a part of the distance measurement data, such as the plane 501 and the hand area 502 in the distance image 500. It can be determined whether or not a three-dimensional object is placed.

  As mentioned above, although this invention was demonstrated using embodiment mentioned above, this invention is not limited to embodiment mentioned above. For example, a visible light sensor or an infrared sensor (not shown) may be installed above the plane 101. The detection accuracy of the three-dimensional object region can be improved by extracting the skin color region in the captured image using a visible light sensor.

  Further, in the embodiment described above, when extracting a skin color area, it is preferable to limit the color tone and the light amount of the UI image 102 projected by the projector 106.

  Furthermore, in the embodiment described above, the projection cycle of the projector 106 and the imaging cycle of the visible light sensor may be synchronized, and projection and imaging may be alternately performed at high speed at a level invisible to the user. Thereby, the skin color area can be extracted without being affected by the projection light.

  In the embodiment described above, the distance image sensor 105 may be a pattern light projection type sensor capable of measuring the distance to the subject, an infrared light sensor, or a stereo camera.

  In the embodiment described above, the three-dimensional object detected on the plane 101 is not limited to the user's hand, and may be a stylus, a robot arm, or the like for performing a user operation.

  Further, in the embodiment described above, the plane 101 is not limited to a desk that is horizontal to the ground, and may be on a screen or a white board that is perpendicular to the ground.

  In the embodiment described above, the processing corresponding to steps S103 to S106 may be performed only on a partial region of the distance image 500.

  FIG. 9 is a block diagram showing a modification of the functional configuration of the three-dimensional object detection device 100 of FIG.

  In FIG. 9, the three-dimensional object detection device 100 includes the distance image acquisition unit 301, the approximate plane calculation unit 303, the reference image acquisition unit 305, the fingertip detection unit 306, the coordinate conversion unit 307, the recognition unit 308, and the display control unit in FIG. In addition to 309, each function part of the virtual solid object area | region specification part 901 and the virtual solid object area expansion part 902 is provided. The processing of each functional unit described above is performed by the CPU 201 executing a program stored in the ROM 202 or the storage device 204.

  The virtual three-dimensional object area specifying unit 901 specifies a virtual three-dimensional object area in the distance image 500. In the present embodiment, the difference between the distance measurement data of distance image 500 and the second reference distance measurement data included in the reference image is calculated in pixel units. For example, the distance measurement data is smaller than the second reference distance measurement data A pixel is extracted, and a region adjacent to the extracted pixel is specified as a virtual three-dimensional object region. The virtual three-dimensional object region expanding unit 902 adds the pixels around the virtual three-dimensional object region to the predetermined number of virtual three-dimensional object regions that are pixels other than the pixels whose distance measurement data is smaller than the second reference distance measurement data. Expand virtual three-dimensional object area.

  FIG. 10 is a flowchart showing the procedure of a modification of the user operation detection process of FIG.

  The process in FIG. 10 is also performed by the CPU 201 executing a program stored in the ROM 202 or the storage device 204, and is executed when the CPU 201 acquires a distance image from the distance image sensor 105. Also in the process of FIG. 10, a process of detecting a tap operation will be described as an example of the user operation.

  In FIG. 10, the CPU 201 performs the process of step S101. Next, the CPU 201 compares the distance image 500 with the reference image to specify a virtual three-dimensional object region (step S201) (virtual three-dimensional object region specifying means), and expands the virtual three-dimensional object region (step S202). For example, when the virtual three-dimensional object regions 1101 and 1102 in FIG. 11A are specified in step S201, the CPU 201 expands the virtual three-dimensional object regions 1101 and 1102 in step S202. For example, the CPU 201 adds pixels other than pixels whose distance measurement data is smaller than the second reference distance measurement data and around the virtual three-dimensional object region 1101 to the predetermined number of virtual three-dimensional object regions 1101 (FIG. The virtual three-dimensional object region 1101 is expanded as in b). The CPU 201 performs the same process on the virtual three-dimensional object area 1102 as well. Next, the CPU 201 approximates distance measurement data of the extended virtual three-dimensional object region using the least squares method to generate first reference distance measurement data (step S203). Next, the CPU 201 compares the distance measurement data and the first reference distance measurement data in the extended virtual three-dimensional object region, and determines whether the comparison result is equal to or less than a predetermined threshold (step S204). Specifically, the CPU 201 calculates the difference between the distance measurement data and the first reference distance measurement data in the extended virtual three-dimensional object region in pixel units, and the total value of the calculation results of each pixel is equal to or less than a predetermined threshold value Determine if

  As a result of the determination in step S204, when the compared result is equal to or less than the predetermined threshold value, the CPU 201 determines that the extended virtual three-dimensional object region is not a three-dimensional object region (step S205), and performs processing in step S207 described later. On the other hand, as a result of the determination in step S204, when the compared result is larger than the predetermined threshold, the CPU 201 determines that the extended virtual three-dimensional object region is a three-dimensional object region (step S206). Here, for example, it is assumed that the virtual three-dimensional object area 1101 is a user's hand area pointing to the UI image 102 projected on the plane 101. Further, it is assumed that the virtual three-dimensional object region 1102 is a region detected as a virtual three-dimensional object region although no three-dimensional object is placed on the plane 101. At this time, the result of comparing the distance measurement data and the first reference distance measurement data in the virtual three-dimensional object region 1101 in which the user's hand actually exists on the plane 101 becomes larger than a predetermined threshold. Is determined to be a three-dimensional object region. On the other hand, the result of comparing the distance measurement data and the first reference distance measurement data in the virtual three-dimensional object region 1102 in which no three-dimensional object actually exists in the plane 101 is less than a predetermined threshold, and the virtual three-dimensional object region 1102 is a three-dimensional object It is determined not to be a region. Thus, in the present embodiment, only the virtual three-dimensional object region 1101 is specified as the three-dimensional object region among the virtual three-dimensional object regions 1101 and 1102 as shown in FIG. It is detected as a three-dimensional object on the plane 101. Next, the CPU 201 determines whether the determination of all virtual three-dimensional object regions has been completed (step S207).

  As a result of the determination in step S207, when the determination of any virtual three-dimensional object region is not completed, the CPU 201 returns to the process of step S203. On the other hand, when the determination of all the virtual three-dimensional object regions is completed as a result of the determination in step S207, the CPU 201 performs the process of step S109 and subsequent steps.

  In the process of FIG. 10 described above, the solid object on the plane 101 is detected based on the result of comparing the distance measurement data of each virtual solid object region in the distance image 500 and the first reference distance measurement data of each virtual solid object region. Be done. That is, the detection process of the three-dimensional object is performed only on a partial region of the distance image 500. Thereby, the processing load in the detection process of the three-dimensional object can be reduced compared to the case where the detection process of the three-dimensional object is performed on all the regions in the distance image 500.

  Further, in the process of FIG. 10 described above, in the virtual three-dimensional object region, the difference between the distance measurement data and the first reference distance measurement data is calculated in pixel units, and the total value of the differences of each pixel is larger than a predetermined threshold value A three-dimensional object region is detected as a three-dimensional object on the plane 101. In addition, a virtual three-dimensional object region whose total value is equal to or less than the threshold is not detected as a three-dimensional object on the plane 101. Thus, it can be determined whether or not the region specified as the virtual three-dimensional object region is actually a three-dimensional object region, and it can be accurately determined whether or not a three-dimensional object is placed on the plane 101.

  In the process of FIG. 10 described above, the virtual three-dimensional object area is an area in the distance image 500 where the distance measurement data is smaller than the second reference distance measurement data. That is, the virtual three-dimensional object region is a region assumed to be where a three-dimensional object is placed on the plane 101. Thus, the detection process of the three-dimensional object can be performed only in the region assumed to be where the three-dimensional object is placed on the plane 101, and the detection process of the three-dimensional object can be efficiently performed.

  Further, in the processing of FIG. 10 described above, the extended virtual three-dimensional object region includes a region other than the region where the distance measurement data is smaller than the second reference distance measurement data in the distance image 500. That is, the extended virtual three-dimensional object region includes both a pixel obtained by imaging the plane 101 and a pixel obtained by imaging the three-dimensional object. Thereby, the difference between the distance measurement data of each virtual three-dimensional object region and the first reference distance measurement data of each virtual three-dimensional object region when the three-dimensional object is placed on the plane 101 is made more noticeable to detect the three-dimensional object It can be made easy.

  In the process of FIG. 10 described above, when performing the process after step S109, a three-dimensional object area of a size obtained by restoring the portion expanded in the process of step S202 may be used instead of the three-dimensional object area of the same size. .

  Further, in the process of FIG. 10 described above, the difference between the distance measurement data of the distance image 500 and the second reference distance measurement data included in the reference image is calculated in pixel units, and the pixels whose difference is equal to or more than a predetermined threshold value are extracted. Alternatively, a region adjacent to the extracted pixels may be specified as a virtual three-dimensional object region.

  The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or storage medium, and one or more processors in a computer of the system or apparatus read the program. It can also be realized by the process to be executed. The present invention can also be implemented by a circuit (eg, an ASIC) that implements one or more functions.

Reference Signs List 100 three-dimensional object detection apparatus 101 plane 103 hand 500 distance image 503, 504 divided area 601, 701 distance measurement data 602, 702 first reference distance measurement data 1101, 1102 virtual three-dimensional object area

Claims (10)

A three-dimensional object detection apparatus for imaging a subject including a plane to detect a three-dimensional object on the plane, comprising:
Distance image acquisition means for acquiring a distance image including distance measurement data from the three-dimensional object detection device to the subject;
Reference distance measurement data generating means for generating reference distance measurement data of the distance image based on the distance measurement data;
What is claimed is: 1. A three-dimensional object detection device comprising: detection means for detecting a three-dimensional object on the plane based on a result of comparing the distance measurement data and the reference distance measurement data. Division means for dividing the distance image into a plurality of divided areas;
And extraction means for extracting each of the divided areas,
The reference distance measurement data generation unit generates reference distance measurement data of each of the divided areas based on the distance measurement data,
The three-dimensional object according to claim 1, wherein the detection means detects a three-dimensional object on the plane based on a result of comparison of distance measurement data of each of the divided regions and reference distance measurement data of each of the divided regions. Object detection device. The divided area is composed of a plurality of pixels,
And calculating means for calculating the difference between the distance measurement data and the reference distance measurement data in the pixel unit in the divided area.
The detection means detects, as a solid object on the plane, a divided area in which a total value of differences of the respective pixels is larger than a preset threshold value, and on the plane, a divided area having the total value equal to or less than the threshold The three-dimensional object detection device according to claim 2, wherein the three-dimensional object is not detected as a three-dimensional object. It further comprises virtual three-dimensional object region specifying means for specifying a virtual three-dimensional object region in the distance image,
The reference distance measurement data generation unit generates reference distance measurement data of each of the virtual three-dimensional object regions based on the distance measurement data,
The said detection means detects the solid object on the said plane based on the result of having compared the ranging data of each said virtual solid object area | region, and the reference ranging data of each said virtual solid object area | region. The three-dimensional object detection device according to 1. The virtual three-dimensional object region is composed of a plurality of pixels,
The virtual three-dimensional object region further includes calculation means for calculating the difference between the distance measurement data and the reference distance measurement data in the pixel unit,
The detection means detects a virtual three-dimensional object region in which the total value of differences of the respective pixels is larger than a preset threshold value as a three-dimensional object on the plane, and the virtual solid object region having the total value equal to or less than the threshold The three-dimensional object detection device according to claim 4, wherein the three-dimensional object detection device does not detect the three-dimensional object on the plane. A reference image including other ranging data of ranging timing different from the ranging data is acquired,
The other ranging data is a ranging value from the three-dimensional object detection device to the plane,
The three-dimensional object detection device according to claim 5, wherein the virtual three-dimensional object region is a region where the distance measurement data is smaller than the other reference distance measurement data in the distance image. It further comprises expansion means for expanding the virtual three-dimensional object region,
7. The three-dimensional object detection device according to claim 6, wherein the extended virtual three-dimensional object region includes a region other than a region where the distance measurement data is smaller than the other reference distance measurement data in the distance image.   The three-dimensional object detection device according to any one of claims 1 to 7, wherein the reference distance measurement data is data obtained by approximating the distance measurement data using a least squares method. A control method of a three-dimensional object detection device for imaging a subject including a plane and detecting a three-dimensional object on the plane,
A distance image acquisition step of acquiring a distance image including distance measurement data from the three-dimensional object detection device to the subject;
A reference ranging data generation step of generating reference ranging data of the distance image based on the ranging data;
And a detection step of detecting a solid object on the plane based on a result of comparison between the distance measurement data and the reference distance measurement data. A program that causes a computer to execute a control method of a three-dimensional object detection apparatus for capturing an object including a plane and detecting a three-dimensional object on the plane,
The control method of the three-dimensional object detection device is
A distance image acquisition step of acquiring a distance image including distance measurement data from the three-dimensional object detection device to the subject;
A reference ranging data generation step of generating reference ranging data of the distance image based on the ranging data;
A detection step of detecting a solid object on the plane based on a result of comparison between the distance measurement data and the reference distance measurement data.

Images