JP2016176816A - Image processor, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processor capable of detecting a designated object disposed in space without putting restrictions, such as an installation position of a camera and presence of an object, in question, and capable of setting a three-dimensional area by projecting coordinates of the designated object onto a reference surface.SOLUTION: An image processor 100 includes: distance information acquisition means 110 for acquiring distance information from an object in space and setting a reference surface of space and three-dimensional coordinates of the space based on the distance information; designated coordinates acquisition means 111 for detecting a designated object disposed in the space and acquiring coordinates of the designated object in the three-dimensional coordinates as designated coordinates; projection coordinates acquisition means 112 for acquiring projected coordinates obtained by projecting the designated coordinates onto the reference surface; and designated area setting means 113 for setting a designated area to the space, based on the designated coordinates and the projected coordinates.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program.

  Security cameras are being installed to ensure safety. In particular, in places where precious cultural properties such as art museums and museums are displayed, security cameras are installed to prevent crimes such as theft and deliberate damage, and to quickly resolve crime cases.

  As a monitoring system for protecting cultural properties, a monitoring system that automatically reports when a person or the like enters within a predetermined distance from the cultural property has been proposed. Such a monitoring system includes a distance measuring sensor that acquires distance information in addition to the camera, and changes in images and distance information in a three-dimensional region (for example, a region within a predetermined distance centered on a cultural property). And intrusion into a three-dimensional area such as a person is detected.

  In addition, as a method of setting a three-dimensional area, the floor surface (reference surface) is estimated based on distance information, and the floor surface coordinates are detected by detecting the floor surface coordinates from the two-dimensional image captured by the security camera. There is a method of setting the bottom surface of a three-dimensional region (for example, see Patent Document 1). In addition, there is a method of creating a three-dimensional columnar model of a columnar object from a three-dimensional point group obtained by measuring a columnar object (for example, see Patent Document 2).

JP 2007-271408 A Japanese Patent No. 5451457

  However, in the method of Patent Document 1, since the coordinates of the floor surface are detected from the two-dimensional image captured by the security camera, the two-dimensional image includes a range in which the floor surface corresponding to the bottom surface of the three-dimensional region is reliably recognized. It must be. Therefore, in order to image the floor surface, it is necessary to install a camera with an angle of view wider than necessary, or to tilt the camera optical axis downward. This may be a limitation when capturing an image of a desired area.

  Further, the method of Patent Document 2 creates a three-dimensional columnar model corresponding to an object from a three-dimensional point group obtained by measuring the object, so that a virtual three-dimensional columnar shape having no object exists. The model cannot be created.

  The present invention sets a three-dimensional region by detecting a designated object arranged in a space and projecting the coordinates of the designated object on a reference plane in order to solve restrictions such as the installation position of a camera and the presence of an object. An object of the present invention is to provide an image processing apparatus, an image processing method, and a program.

  An image processing apparatus according to the present invention acquires distance information from an object in a space, and sets distance information acquisition means for setting a reference plane of the space and a three-dimensional coordinate of the space based on the distance information; Designated coordinate acquisition means for detecting the designated object arranged and obtaining the coordinates of the designated object in the three-dimensional coordinates as designated coordinates; and projection coordinate obtaining means for obtaining projection coordinates obtained by projecting the designated coordinates onto the reference plane And designated area setting means for setting a designated area in the space based on the designated coordinates and the projected coordinates.

  According to the present invention, a three-dimensional region can be obtained by detecting a designated object arranged in a space without projecting restrictions such as the installation position of a camera or the presence of an object, and projecting the coordinates of the designated object on a reference plane. Can be set.

(A) It is a block diagram which shows an example of the image processing apparatus which concerns on this invention. (B) It is a block diagram which shows an example of the distance information acquisition part of the image processing apparatus which concerns on this invention. It is a block diagram which shows the specific structural example of the image processing apparatus which concerns on this invention. It is a figure which shows an example of the setting of a designation | designated area | region. 6 is a flowchart illustrating an example of an operation of the image processing apparatus according to the present invention. It is a figure which shows the example of a designated shape. It is a figure which shows an example of the method of calculating the polygon which includes all the projection coordinates. It is a figure which shows an example of the display screen of the display which is a notification part or a display part. It is a figure which shows an example in which the designated area setting part adjusts the magnitude | size of a designated area.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiments.

  FIG. 1A is a block diagram illustrating an example of an image processing apparatus according to the present invention. FIG. 1B is a block diagram illustrating an example of a distance information acquisition unit of the image processing apparatus according to the present invention.

  As illustrated in FIG. 1A, the image processing apparatus 100 includes an imaging unit 101, a control unit 102, an input / output unit 103, a notification unit 104, a display unit 105, and a storage unit 106. The control unit 102 includes a distance information acquisition unit 110, a designated coordinate acquisition unit 111, a projected coordinate acquisition unit 112, and a designated area setting unit 113. As shown in FIG. 1B, the distance information acquisition unit 110 includes a plane extraction unit 115, a reference plane setting unit 116, and a coordinate setting unit 117.

  The imaging unit 101 captures an object in space and acquires a two-dimensional image. The control unit 102 executes a computer program for causing the image processing apparatus 100 to function, and controls components of the image processing apparatus 100. The input / output unit 103 inputs / outputs various data to / from the image processing apparatus 100. The notification unit 104 notifies the user of various types of information. The display unit 105 displays a two-dimensional image and various data. The storage unit 106 stores various data.

  The distance information acquisition unit 110 acquires distance information from an object in a space, and sets a space reference plane and a space three-dimensional coordinate based on the distance information. The distance information is acquired by a distance sensor that measures the distance to an object using reflection of light or ultrasonic waves. Moreover, a predetermined pattern may be irradiated with an infrared laser or the like, and distance information may be acquired based on the distortion or shape of the pattern imaged by the infrared camera or the imaging unit 101.

  The plane extraction unit 115 acquires a three-dimensional point group indicating the surface of the object from the distance information, and extracts a plane based on the three-dimensional point group. The reference plane setting unit 116 sets a reference plane (for example, a bottom surface) from the plane extracted by the plane extraction unit 115. The coordinate setting unit 117 matches the coordinate system of the spatial coordinates in the two-dimensional image acquired by the imaging unit 101 with the coordinate system of the three-dimensional coordinates based on the distance information.

  The designated coordinate acquisition unit 111 detects the designated object arranged in the space, and obtains the coordinates of the designated object in the three-dimensional coordinates as the designated coordinates. The designated object is an object that designates arbitrary coordinates in space by at least one of color, light, shape, movement, signal, marker, and pattern. For example, the designated object is an LED attached to the tip of a pen-type device. In this case, the designated coordinate obtaining unit 111 obtains, as designated coordinates, a position where an LED having a light emission ON / OFF function emits light. The designated object may be a two-dimensional code such as a QR code (registered trademark) attached to the tip of the pen-type device.

  Further, the designated object may be a hand or a finger, and the designated coordinate acquisition unit 111 may obtain the designated coordinate by a gesture based on the movement of the hand or finger (such as an operation of drawing a circle with the finger).

  The projected coordinate acquisition unit 112 acquires the projected coordinates obtained by projecting the designated coordinates onto the reference plane. When the reference surface is the bottom surface, the coordinates obtained by projecting the designated coordinates onto the bottom surface along the perpendicular to the bottom surface are projection coordinates. The projected coordinate acquisition unit 112 may use, as a projected coordinate, a coordinate projected on a reference plane (for example, the bottom surface) at a predetermined angle in addition to a perpendicular line.

  The designated area setting unit 113 sets a designated area in the space based on the designated coordinates and the projected coordinates. For example, the designated area setting unit 113 sets, as the designated area, a quadrangular prism whose apexes are four designated coordinates and four projected coordinates. When a person or the like enters the designated area, at least one of the input / output unit 103, the notification unit 104, and the display unit 105 outputs a notification or an alarm.

  FIG. 2 is a block diagram showing a specific configuration example of the image processing apparatus according to the present invention. As illustrated in FIG. 2, the imaging device 121 corresponding to the imaging unit 101 is a digital camera or a video camera that images an object and converts external light into image data. A processing device 122 corresponding to the control unit 102 is a CPU, MPU, GPU, or the like that performs control processing on each element of the image processing device 100.

  An input / output device 123 corresponding to the input / output unit 103 is a UI (user interface) such as a button for inputting / outputting a signal to / from the image processing apparatus 100, a sensor (distance sensor, etc.), a touch panel, and a voice input / output device. Note that the input / output unit 103 only needs to be electrically connected to the image processing apparatus 100, and does not need to be connected by a bus as shown in FIG.

  The notification device 124 corresponding to the notification unit 104 is an LED, a speaker, a display, or the like that notifies the user of various information such as success / failure of input of designated coordinates, input contents of designated coordinates, image data, and distance information. A display device 125 corresponding to the display unit 105 is a display that displays a two-dimensional image of the imaging unit 101 and various types of information. A storage device 126 corresponding to the storage unit 106 is a memory, a hard disk, or the like that holds various data such as an image processing program, image data acquired by the imaging unit 101, and processing results of the control unit 102.

  Note that the configuration example of the image processing apparatus illustrated in FIG. 2 is an example, and may include components other than those illustrated here. Further, the image processing program may be executed using an external general-purpose computer or the like, or the image processing program may be executed on an electronic circuit.

  Next, setting of the designated area will be described. FIG. 3 is a diagram illustrating an example of setting a designated area. In FIG. 3A, a monitoring camera (imaging unit) 301 images a target object 303 to be monitored and acquires a two-dimensional image. Further, the distance sensor 302 measures the distance to the object, and outputs the distance information to the distance information acquisition unit 110.

  Here, the user designates designated coordinates in the space using a pointer 304 having a designated object (for example, LED) in order to set a three-dimensional region surrounding the object 303 as the designated region. The pointer 304 enables the LED to be turned on by a switch, and designates the position of the designated coordinate by turning on the LED at a position where the designated coordinate is desired to be designated.

  In FIG. 3B, the three-dimensional coordinates of the object in the space are output as a point group based on the distance information to the object surface. The plane extraction unit 115 estimates and extracts the planes 305 and 306 from a three-dimensional point group indicating the surface of the object. For example, the estimation of the planes 305 and 306 is realized by using the technique disclosed in Japanese Patent No. 4016180.

  The reference plane setting unit 116 sets the bottom surface from the planes 305 and 306 extracted by the plane extraction unit 115. For example, the lowermost plane 306 among the extracted planes may be set as the bottom surface. In addition, a marker displaying a predetermined pattern is attached to a part of the floor surface or a plane other than the floor surface (for example, a wall surface), and the position and direction of the marker are detected from the two-dimensional image captured by the imaging unit 101. Thus, the bottom surface may be set. Also, the coordinate system of the distance sensor may be calibrated in advance with respect to the coordinate system of the real space, and a virtually given plane may be set as the bottom surface.

  In this way, the bottom surface is extracted based on the distance information from the distance sensor, the marker attached to a part of the floor surface or a plane other than the floor surface, or the virtual plane by calibration. As a result, it is not necessary to detect the coordinates of the floor surface from the two-dimensional image, and it is not necessary to include the floor surface in a range where the floor surface can be reliably recognized. That is, it is not necessary to install the monitoring camera 301 with an angle of view wider than necessary, or to tilt the optical axis of the monitoring camera 301 downward, and an image of the area of the target that is truly desired can be taken.

  In FIG. 3C, the monitoring camera 301 captures a space and acquires a two-dimensional image. The designated coordinate acquisition unit 111 detects a pointer (designated object) 304 in the two-dimensional image and obtains designated coordinates from the position of the pointer 304 in the two-dimensional image. The designated coordinate obtaining unit 111 obtains a designated coordinate position 307 by detecting a locally high-brightness point caused by the light emission of the LED from the two-dimensional image. Also, the designated coordinate acquisition unit 111 acquires a designated coordinate position 307 by detecting a marker displaying a QR code (registered trademark) or a predetermined pattern from the two-dimensional image.

  As described above, the designated coordinate acquisition unit 111 acquires the designated coordinates corresponding to the designated area based on the position of the pointer (designated object) 304 in the two-dimensional image, and thus there is no target object 303 to be monitored. Even in this case, the designated area can be set.

  Note that the monitoring camera 301 includes a stereo camera or a plurality of cameras in order to acquire designated coordinates from a two-dimensional image. Further, if the monocular distance measuring technique is used, the monitoring camera 301 may be configured by a monocular camera. Thereby, the coordinate system of the coordinate of the space which the monitoring camera 301 images (spatial coordinates in the two-dimensional image) can be set.

  In FIG. 3C, the position 307 of the designated coordinate is displayed as a circular mark, but the mark may not be displayed in the actual two-dimensional image of the monitoring camera 301. Further, a mark representing the designated coordinate position 307 is displayed on the two-dimensional image displayed on the display of the notification unit 104, and the designated coordinate position 307 is displayed on the two-dimensional image displayed on the display of the display unit 105. The mark to represent may not be displayed. In this way, display / non-display of the mark may be distinguished by the apparatus.

  The designated coordinate acquisition unit 111 may execute at least one of erasing, moving, and changing the designated coordinates. In this case, the pointer 304 includes the input / output device 123, and these processes may be executed by pressing a button or a changeover switch of the pointer 304.

  In FIG. 3D, the designated coordinate acquisition unit 111 acquires the designated coordinate position 308 in the three-dimensional coordinates based on the designated coordinate position 307 obtained from the two-dimensional image. In this case, the coordinate setting unit 117 matches the coordinate system of the spatial coordinates in the two-dimensional image with the coordinate system of the three-dimensional coordinates based on the distance information. For example, if the relative position or reference position of the monitoring camera 301 and the distance measuring sensor 302 is known, the coordinate systems can be matched. By matching the coordinate system, the designated coordinate acquisition unit 111 can match the designated coordinate position 307 in the two-dimensional image with the designated coordinate position 308 in the three-dimensional coordinates based on the distance information.

  Further, the projection coordinate acquisition unit 112 acquires the position 309 of the projection coordinates obtained by projecting the designated coordinates on the bottom surface (reference plane). Note that the reference surface may be a curved surface or a surface other than the bottom surface. For example, when the object is a painting and is installed on a side surface (wall surface), a position 309 of projection coordinates obtained by projecting designated coordinates onto the side surface is acquired using the side surface as a reference surface. The display of the notification unit 104 or the display unit 105 may display the plane extracted by the plane extraction unit 115, and the input / output unit 103 may select the reference plane from the extracted plane. Further, the input / output unit 103 may arbitrarily set the reference plane in the three-dimensional coordinates based on the distance information.

  In FIG. 3E, four designated coordinates are designated, and four projected coordinates obtained by projecting these designated coordinates are shown. Then, the designated area setting unit 113 sets the designated area in the three-dimensional coordinates based on the designated coordinates and the projected coordinates. In FIG. 3E, the designated area setting unit 113 sets, as the designated area 310, a quadrangular prism having apexes at four designated coordinates and four projected coordinates.

  In FIG. 3F, the designated area setting unit 113 enlarges the designated area 310 with the center of the designated area 310 as a common center, and changes the designated area 310 to the designated area 311. The designated area 310 and the designated area 311 are similar.

  As described above, the designated area setting unit 113 may adjust at least one of the number, size, position, and shape of the designated area 310. As a result, by easily setting or changing the designated area 310, it is possible to quickly cope with the size and number of objects even when the object is replaced. Further, by setting a plurality of designated areas such as the designated area 310 and the designated area 311, it is possible to set a warning level according to the intrusion location of the person and the distance between the person and the target object. For example, a low level alert may be issued when a person enters the designated area 311, and a high level alert may be issued when a person enters the designated area 310.

  Next, the operation of the image processing apparatus 100 will be described. FIG. 4 is a flowchart showing an example of the operation of the image processing apparatus 100 according to the present invention.

  In step S401, the input / output unit 103 sets the shape (specified shape) of the specified region. A plurality of designated shapes (for example, a polygonal cylinder, a cylinder, and a plane) may be registered in the storage unit 106 in advance, and the input / output unit 103 may select the designated shape. In this case, the pointer 304 may include the input / output device 123, and the designated shape may be set by pressing a button or a changeover switch of the pointer 304. When the designated shape is set, a point (for example, a vertex) that defines the designated shape is held in the storage unit 106 together with the designated shape information as shape defining information, and is input to the control unit 102.

  In step S <b> 402, the imaging unit 101 captures a space including the object 303 using the monitoring camera 301 and acquires a captured image (two-dimensional image). In step S403, the designated coordinate acquisition unit 111 detects a pointer (designated object) 304 in the two-dimensional image.

  Separately from steps S402 and S403, in step S404, the distance sensor 302 acquires the distance information of the space including the object 303, and outputs the distance information to the distance information acquisition unit 110. In step S405, the plane extraction unit 115 estimates and extracts the planes 305 and 306 from the three-dimensional point group indicating the surface of the object, and the reference plane setting unit 116 extracts the planes 305 and 306 extracted by the plane extraction unit 115. To set the bottom as the reference plane.

  In step S406, the designated coordinate acquisition unit 111 matches the coordinate system of the imaging space of the imaging unit 101 and the distance information of the distance information acquisition unit 110, thereby obtaining the three-dimensional position of the designated coordinate acquired from the two-dimensional image. Get in the coordinate system of coordinates.

  In step S407, the notification unit 104 notifies that the designated coordinates have been acquired by the designated coordinate acquisition unit 111, and notifies the user of the success or failure of input of the designated coordinates. For example, if the designated object attached to the tip of the pointer 304 is outside the imaging range of the imaging unit 101 and does not appear in the captured image or is hidden by another object, the position of the designated coordinate is three-dimensional in step S406. It may not be obtained in the coordinate system of coordinates. Therefore, when the position of the designated coordinate can be acquired in the coordinate system of the three-dimensional coordinate, the input of the designated coordinate intended by the user can be supported by notifying the user that the designated coordinate is input.

  For example, the notification of the notification unit 104 is realized by turning on an LED provided in the monitoring camera 301. The notification of the notification unit 104 may be realized by voice, vibration of the pointer 304, a mark in a captured image displayed on the display, and the like.

  In step S408, the designated area setting unit 113 determines whether all the designated coordinates necessary for defining the designated shape set in step S401 have been designated. If all the designated coordinates are not designated, the process returns to step S402 and step S404, and designated coordinates are designated. If all of the designated coordinates are not designated, the process proceeds to step S409.

  For example, when the designated shape set in step S401 is a quadrangular prism and the number of designated coordinates required to define the quadrangular prism is four, the designated area setting unit 113 designates four designated coordinates. It is determined whether or not it has been done. If four designated coordinates are not designated, the process returns to step S402 and step S404, and if four designated coordinates are designated, the process proceeds to step S409.

  In step S409, the projection coordinate acquisition unit 112 acquires the position of the projection coordinates obtained by projecting the designated coordinates on the bottom surface (reference plane) in the three-dimensional coordinate system. In step S410, the designated area setting unit 113 sets the designated area as a three-dimensional coordinate from the position of the designated coordinate acquired in step S406 and the position of the projection coordinate acquired in step S409. For example, when four designated coordinates are designated according to the shape definition information of the quadrangular prism, a quadrangular prism having apexes of the four designated coordinates and the four projected coordinates is set as the designated area.

  In step S <b> 411, the designated area setting unit 113 determines whether the designated area needs to be adjusted according to the input from the input / output unit 103. If adjustment of the designated area is necessary, the process proceeds to step S412 and the designated area setting unit 113 adjusts at least one of the number, size, position, and shape of the designated area. If it is not necessary to adjust the designated area, the process ends.

  As described above, the image processing method according to the present embodiment includes a step of acquiring distance information from an object in a space and setting a reference plane of the space and a three-dimensional coordinate of the space based on the distance information. In addition, the image processing method according to the present embodiment includes a step of detecting a designated object arranged in a space, obtaining the coordinates of the designated object in three-dimensional coordinates as designated coordinates, and projected coordinates on which the designated coordinates are projected on a reference plane And obtaining a process. Further, the image processing method according to the present embodiment includes a step of setting a designated area in the space based on the designated coordinates and the projected coordinates.

  As mentioned above, although embodiment concerning this invention was described, this invention is not limited to these, It can change and change within the range described in the claim.

  For example, in step S401, the input / output unit 103 may select a specified shape from a plurality of specified shapes, or may input a predetermined specified shape. FIG. 5 is a diagram illustrating an example of the designated shape. The black circular marks in FIG. 5 indicate the designated coordinates necessary for defining the designated shape among the points defining the designated shape. White circular marks in FIG. 5 indicate projection coordinates necessary for defining the designated shape among the points defining the designated shape.

  FIG. 5A shows the designated shape of the N prism. N is an arbitrary integer of 3 or more, and the required number S1 of designated coordinates is N (shape defining information: N prism, S1 = N). In this case, N designated coordinates are acquired according to the shape defining information, and N projected coordinates obtained by projecting them on the reference plane are acquired. Then, an N prism having the N designated coordinates and the N projected coordinates as vertices is set as the designated area. The height of the N prism may be the average of the heights of the N designated coordinates, or may be a preset height H. In addition, a columnar body that forms an upper surface with an arbitrary plane or curved surface including N designated coordinates, instead of an N prism, may be set as the designated area.

  FIG. 5B shows a specified shape of a regular N prism having shape defining information different from that in FIG. N is an arbitrary integer of 3 or more, and the required number S2 of designated coordinates is 2 (shape defining information: regular N prism, S2 = 2). In this case, two specified coordinates are acquired according to the shape defining information, and two projection coordinates obtained by projecting them on the reference plane are acquired. Then, a regular N-gon is formed with one of the two designated coordinates as the center of the regular N-gon and the other as a vertex, and the vertex coordinates of each vertex of the regular N-gon are acquired. Also, projection coordinates obtained by projecting each vertex of the regular N-gon on the reference plane are acquired. A normal N prism having N vertex coordinates and N projected coordinates as vertices is set as the designated area.

  (C) of FIG. 5 is the designated shape of a cylinder. The required number S3 of designated coordinates is two (shape defining information: cylinder, S3 = 2). In this case, two specified coordinates are acquired according to the shape defining information, and two projection coordinates obtained by projecting them on the reference plane are acquired. A circle having one of the two designated coordinates as the center of the circle and the other as the radius to the center is formed, and the circle coordinates are acquired. Also, the projection coordinates obtained by projecting the circle onto the reference plane are acquired. A cylinder obtained by translating circular coordinates in the projection direction is set as the designated area.

  FIG. 5D shows a designated shape of a plane. For example, when the number of persons who have passed through the plane designated area is counted, the plane designated shape is set. The required number S4 of designated coordinates is two (shape defining information: plane, S4 = 2). In this case, two specified coordinates are acquired according to the shape defining information, and two projection coordinates obtained by projecting them on the reference plane are acquired. A plane having two designated coordinates and two projected coordinates as vertices is set as the designated area.

  FIG. 5E shows an arbitrary designated shape. The required number S5 of designated coordinates is indefinite (shape defining information: arbitrary shape, S5 = undefined). In this case, arbitrary M designated coordinates are acquired according to the shape defining information, and M projected coordinates obtained by projecting them onto the reference plane are acquired. A columnar body having apexes at M designated coordinates and M projected coordinates is set as the designated area. The designation end of the designated coordinates may be output to the designated area setting unit 113 via the captured image of the imaging unit 101 by pressing the button of the pointer 304 or the changeover switch to blink the LED in a predetermined pattern.

  In the case of an arbitrary designated shape, a polygon that includes all M projection coordinates may be calculated, and a polygonal column obtained by translating the polygon in a direction opposite to the projection direction may be set as the designated region. . FIG. 6 is a diagram illustrating an example of a technique for calculating a polygon that includes all M projected coordinates. As shown in FIG. 6A, the designated area setting unit 113 acquires a projected coordinate 601 that minimizes the y coordinate among the M projected coordinates. The designated area setting unit 113 obtains each angle formed by a straight line connecting the projection coordinates 601 and other projection coordinates with the x axis, and obtains the projection coordinates 602 that minimizes the angle.

  Then, as shown in FIG. 6B, the designated area setting unit 113 sets a straight line connecting the projected coordinates 601 and 602 as one side of the polygon. Next, the designated region setting unit 113 obtains each angle formed by the straight line connecting the projection coordinate 601 and other projection coordinates with the x axis, and obtains the projection coordinate 603 that minimizes the angle. The designated area setting unit 113 repeats this process until the projection coordinates 601 acquired first are acquired again. And the polygon which makes the projection coordinate finally acquired the vertex becomes a polygon which includes all M projection coordinates.

  As described above, the designated area setting unit 113 sets the designated area as a point that defines the shape of the designated area based on at least one of the designated coordinates and the projected coordinates based on the shape of the designated area set in advance. It may be set.

  FIG. 7 is a diagram illustrating an example of a display screen of a display that is the notification unit 104 or the display unit 105. As illustrated in FIG. 7, the display 701 displays a captured image 702 captured by the imaging unit 101. The captured image 702 includes a reference plane (bottom surface) 703 set by the reference plane setting unit 116, a specified coordinate position 704 acquired by the specified coordinate acquisition unit 111, and a projection coordinate acquired by the projection coordinate acquisition unit 112. The position 705 is superimposed by highlighting.

  The captured image 702 includes a connection 706 between the position 704 and the position 705, a position 707 of the designated object, a position 708 of the projected coordinates obtained by projecting the position 707 on the reference plane, a connection 709 connecting the position 704 and the position 707, and A connection line 710 connecting the position 705 and the position 708 is superimposed.

  As described above, the display of the notification unit 104 or the display unit 105 includes a reference plane (bottom surface) 703, a designated object position 707, a designated coordinate position 704, a projected coordinate position 705, a line 706 connecting the designated coordinate and the projected coordinate, At least one of the line 709 connecting the specified object and the specified coordinates, the line 710 connecting the plurality of projection coordinates, and the position of the specified area may be superimposed on the captured image (two-dimensional image) 702. Further, the display of the notification unit 104 or the display unit 105 may superimpose and display at least one of a line connecting a plurality of designated coordinates and a position of the designated area on the captured image (two-dimensional image) 702.

  Note that the designated object may be changed in order to distinguish the designated object position 707 from the designated coordinate position 704. For example, the pointer 304 includes a marker and an LED. When the specified coordinate acquisition unit 111 detects a marker, the position 707 of the specified object is superimposed on the captured image 702, and when the specified coordinate acquisition unit 111 detects light emission of the LED, the position 704 of the specified coordinate is captured. It may be superimposed on the image 702. That is, the designated coordinates are acquired by the light emission of the LED.

  Moreover, the light emission pattern of LED may change. In this case, when the designated coordinate acquisition unit 111 detects the first light emission pattern, the position 707 of the designated object is superimposed on the captured image 702, and when the designated coordinate acquisition unit 111 detects the second light emission pattern, A designated coordinate position 704 is superimposed on the captured image 702.

  That is, various commands can be input by changing the designated object. The designated object may be changed by at least one of color, light, shape, movement, signal, marker, and pattern. For example, the designated object may be a hand or a finger, and various commands may be input by gestures (movement of opening or closing the hand) by movement of the hand or finger.

  FIG. 8 is a diagram illustrating an example in which the designated area setting unit 113 adjusts the size of the designated area. FIG. 8 shows the relationship between the first designated area 801 set using the pointer 304 and the second designated area 802 in which the size of the first designated area 801 is adjusted. 8A is a bird's-eye view, FIG. 8B is a top view, FIG. 8C is a side view, and FIG. 8D is a front view.

  The designated area setting unit 113 enlarges the width and depth of the first designated area 801 by a predetermined distance with the center of the first designated area 801 as a common center, and changes to the second designated area 802. Note that the designated area setting unit 113 may enlarge the height of the first designated area 801 by a predetermined distance.

  In addition, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and the computer (including a CPU, MPU, GPU, and the like) of the system or apparatus. It may be read and executed. This program and a computer-readable storage medium storing this program are included in the present invention.

  As described above, according to the present embodiment, a designated object placed in a space is detected and the coordinates of the designated object are projected onto a reference plane without causing problems such as camera installation position and the presence of an object. By doing so, a three-dimensional area can be set.

DESCRIPTION OF SYMBOLS 101 Image pick-up part 102 Control part 103 Input / output part 104 Notification part 105 Display part 106 Storage part 110 Distance information acquisition part 111 Designated coordinate acquisition part 112 Projection coordinate acquisition part 113 Designated area setting part 115 Plane extraction part 116 Reference plane setting part 117 Coordinate Setting unit 121 Imaging device 122 Processing device 123 Input / output device 124 Notification device 125 Display device 126 Storage device

Claims (11)

Distance information acquisition means for acquiring distance information from an object in space, and setting a reference plane of the space and a three-dimensional coordinate of the space based on the distance information;
Designated coordinate acquisition means for detecting a designated object arranged in the space and obtaining the coordinates of the designated object in the three-dimensional coordinates as designated coordinates;
Projected coordinate acquisition means for acquiring projected coordinates obtained by projecting the designated coordinates onto the reference plane;
An image processing apparatus comprising: a designated area setting unit configured to set a designated area in the space based on the designated coordinates and the projected coordinates. The distance information acquisition means includes
Plane extraction means for acquiring a three-dimensional point group indicating the surface of the object from the distance information, and extracting a plane based on the three-dimensional point group;
The image processing apparatus according to claim 1, further comprising: a reference plane setting unit that sets the reference plane from the plane. Imaging means for capturing the space and acquiring a two-dimensional image;
The distance information acquisition means includes coordinate setting means for matching a coordinate system of spatial coordinates in the two-dimensional image with a coordinate system of the three-dimensional coordinates,
The said designated coordinate acquisition means detects the said designated object in the said two-dimensional image, and acquires the said designated coordinate from the position of the said designated object in the said two-dimensional image. Image processing device.   4. The specified coordinate acquisition unit detects the specified object by at least one of a color, light, shape, movement, signal, marker, and pattern of the specified object. The image processing apparatus according to item 1.   The image processing apparatus according to claim 1, wherein the designated coordinate acquisition unit executes at least one of erasing, moving, and changing the designated coordinate.   The image processing apparatus according to claim 1, further comprising a notification unit that notifies that the specified coordinate is acquired by the specified coordinate acquisition unit. Imaging means for capturing the space and acquiring a two-dimensional image;
A position of the reference plane; a position of the designated object; a position of the designated coordinate; a position of the projected coordinate; a line connecting the designated coordinate and the projected coordinate; a line connecting the designated object and the designated coordinate; The display unit for displaying at least one of a line connecting the designated coordinates, a line connecting a plurality of the projected coordinates, and a position of the designated area on the two-dimensional image. The image processing apparatus according to any one of the above.   The designated area setting means sets at least one of the designated coordinates and the projection coordinates as a point that defines the shape of the designated area based on a preset shape of the designated area. The image processing device according to claim 1, wherein the image processing device is set.   The image processing apparatus according to claim 1, wherein the designated area setting unit adjusts at least one of the number, size, position, and shape of the designated area. Obtaining distance information from an object in space, and setting a reference plane of the space and a three-dimensional coordinate of the space based on the distance information;
Detecting a designated object arranged in the space, and obtaining coordinates of the designated object in the three-dimensional coordinates as designated coordinates;
Obtaining projected coordinates obtained by projecting the designated coordinates onto the reference plane;
A step of setting a designated area in the space based on the designated coordinates and the projected coordinates. The computer program for functioning a computer as each means of the image processing apparatus of any one of Claims 1 thru | or 9.

Images